Duplication of characters during kanji conversion in a Japansese IME(ATOK) still remains in some places

We strongly recommend you to search the forum with possible keywords before submitting a new bug report. Please also try your repro steps with third-party plugins and custom CSS disabled and see if it’s still reproducible. If it’s an issue with third-party plugins or themes, try contacting the author for help. Once you’ve done the above, delete this line.

Steps to reproduce

I reported in the linked thread about duplicate strings in the Japanese IME called ATOK. And I am not sure if it was due to changes in Obsidian itself or CodeMirror6, but most of the problem was fixed in v0.14.8 or v0.14.9.

https://forum.obsidian.md/t/duplication-of-character-strings-during-kanji-conversion-in-japanese-ime-atok/29279

Since then, I have generally been able to type Japanese with ATOK comfortably, but the problem with duplicate strings still remains in some places even in v0.15.8.

  • If you select characters that includes the first character in a list item and then enter a new string, the string will be duplicated when converted. In other words, for some reason, the bug remains only at the beginning of each item in the list.
  • In the editor, when I try to drag the word while conversion, the word is duplicated(I accidentally found it).

The following is not about the editor part, but about other UI parts.

  • If you click on a suggested command while converting a string in the command palette (before confirming it), the character being converted is duplicated twice and the selected command fails to execute. The same behavior is observed in the Quick Switcher and Move File commands.

  • In the search box of file explorer, if I click on a file name in the search results during conversion, the string being converted is duplicated in the search box. The clicked file opens correctly, but the search results are not be displayed correctly because the string being searched for changes(duplicated). The same behavior is observed in the filtering input fields in the backlinks pane.

  • A different issue, when using ATOK, the link and tag suggestions were not working for the word being converted (Microsoft IME and google Japanese input do not have this problem). However, this time I noticed that for the command palette and quick switcher mentioned above, the suggestions are working correctly for strings that are being converted, even when using ATOK. This may also be related to CodeMirror, but I would appreciate it if you could fix the editor part so that suggestions work for strings that are being converted.

Like the linked thread, this problem does not occur in all Japanese IMEs, but only in ATOK. It is an inconvenience, but not a critical one, so if it is a difficult problem to solve, I do not strongly wish to see it resolved. However, since the problem of duplicate strings in the editor section has been solved before, if there is a way to solve this problem, I would like to see it improved.

Does it happen in source mode?

Does is happen here: https://codemirror.net/

Yes, LP mode and source mode, in sandbox.
It doesn’t happen in CodeMirror field in the website.

are you using chrome?

About https://codemirror.net/ ?
I used Firefox, but it doesn’t happen in Chrome either.

As of the thread linked above, the CodeMirror website did not have this duplicate problem.
Up until v0.14.7, it occurred everywhere in the new editor, but since v0.14.8 or v0.14.9, it no longer occurs in most of normal text entry.
I have recently discovered under what situations the duplication problem still occurs, so I am reporting it again.

does is happen now on chrome?

I am not good at English, so I am sorry if I have misunderstood what you meant.
This duplication problem has never happened on the CodeMirror website with Chrome or Firefox, now or ever.

I have found other situations where strings are duplicated during Kanji conversion.
If you range-select a string that includes the character immediately following symbols such as quotation (>), link ([[,]]), highlight(==), heading or tag (#), etc., and then type a new string and try to convert it to kanji, the string is duplicated.
I suspect that all markdown symbols will cause the above symptoms. The symptoms reproduce in both LP and source mode.

  1. Suggest does not work well in the editor section (however, suggestions appear to work correctly while typing in the text box such as command pallete).
  2. However, when clicking on a suggested choice while converting in a text box, the string being typed is duplicated.
  3. If you select the string immediately following the markdown symbol and then type in a new string, the string is duplicated when converted.

Unfortunately, above problems that occur when using ATOK continue in the latest Obsidian.
However, now I have learned the rudiments of programming, so I can investigate a bit more what is going on.
I have examined events such as compositionstart and compositionupdate that fire during kanji conversion, as well as the behavior of the community plugin which do link suggest(Various Complements and Boost Link Suggestion). I found that at least the reason why suggest fails is that ATOK is designed to move the cursor to the left edge of the string being converted while converting a string.(See attached video for reference. The blinking cursor is located at the left end of the block being converted in ATOK, and always at the right end in MS-IME and Google Japanese Input).

I think link suggest uses the string between the wakeword ([[ for example) and the cursor position as a query, but since the cursor jumps (to the position of the wakeword when converting the first phrase, for example), the query becomes empty and the suggestion is failing.
I think the reason why the suggestion works for text box input is because the entire text box string is used for the query, regardless of the cursor position.

Following the above observation and hypothesis, I modified the parameters of above plugins and confirmed that the suggestions generally work in ATOK (although if I select the suggested choice before finishing the conversion, the string is duplicated, as in problem 2 above).
Since the data property of the compositionupdate event stores the string being converted, we can know that the cursor is jumping by checking that the left side of the cursor does not contain that string. And the length of the data string is the distance the cursor is jumping, so we can calculate the parameters that should be passed as EditorSuggestTriggerInfo.

The detailed cause of the other two behaviors is still unclear (I guess that a wrong replacement or something has occurred), but it seems to me that it could well be the same cause (special cursor position during conversion).

So, I would like to ask you to help.

  • Could you please support IMEs with this kind of behavior? …I think this might be an unrealistic request, since ATOK is the only major Japanese IME that has this behavior, but if by chance another language IME has the same behavior, it should cause the same problem, so I made a request.)

  • As for the suggest in the editor part issue, it seems like it could be solved by modifying the EditorSuggestTriggerInfo to take cursor movement into account. However, from what I have searched, it appears that Obsidian’s native link suggest cannot be replaced. If there is a way to do this, could you please let me know? If it is not possible at the moment, could you please implement a setting to turn native link suggestions on or off, or a setting to make the wake word something other than [[ ?

2023-02-14_Tuesday-obsidian-Obsidian-v1.1.12-2023-02-14-09-56-35

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.