Tags with non UTF-8 characters cannot be deleted

Tags appear in Tag pane with odd characters, displaying the number of notes affected, but when selected in the tag pane the ensuing result is “no match found”, so I cannot delete the tags in the notes or remove them from the tag pane.

Probably related to Better Handle md files not stored in UTF8 format

20210223_114722

Steps to reproduce

Create an non-UTF-8 file with accents - I don’t know because none of the files will open, so I’m not sure if there needs to be a # in front of the weird characters

Expected result

If the tag tree shows that there is 1 note affected, I expect clicking on the tag to display that note.

Actual result

Clicking on the tag shows “no results” instead of displaying the affected notes.

20210223_120554

Environment

  • Operating system: Win10
  • Obsidian version: v0.11.0!

Additional information

This absolutely related to that. I suggest you run a program that converts all your notes to UTF8 before importing to obsidian.

I’ve been looking all day and can’t find anything suitable. Apparently there’s an issue such that if you do convert all files, then those that are already UTF8 can lose the accented characters with such programs. So, one has to first distinguish between existing UTF8 files and those that are not, and then perform the conversion on only those that are not - and there is no way to do that, that I have found.

I confess that my migration to text only files is turning out to be much more complicated than keeping my files in Evernote after all. My text files were fine until I brought them into Obsidian. If I’m being honest, I think this is a huge deal, and Obsidian ought to make it clear to all users that they only support UTF-8 files.

I am surprised that Evernote saved your files in ANSI Latin 1. It’s a very old format.

Nevertheless, I agree with you that we should make that clear.

Is it possible that you have settings on the mac on how to save all new files?

Do you have backups of your files pre migration to obsidian?

Sorry - I wasn’t saying that Evernote did. I was bringing up Evernote because at the end of the day it was simpler - stick stuff in, done.

Most of these text files were Word or Wordperfect files that I converted to text files (or copy/pasted directly into Obsidian). And of course after doing all that work, I’m pretty sure I deleted most of the original files and now I’m snookered.

It honestly never occurred to me that anything could go wrong now that they were text files (WITH the accents)

And I should add…I’m talking about 13 years of University (in French) starting in 1990 - so basically my entire Academic life is now mostly text files I can’t read - because I added a tag in Obsidian. Hopefully I didn’t get around to tagging all of them, but of course I have no way of knowing which were touched and which were not.

If you are on discord, can you zip your vault and send it to me privately? I will see if there is something that can be done quickly?

@WhiteNoise - sorry, there was a Criterion Flash Sale I had to take care of :wink:
Let me see how bad it is first. I’ve already moved most of the files out of Obsidian. I’ll take a look at them and see just how bad it is. I’ll let you know on Discord if I need your help. And thank you very much for the offer. Much appreciated.

@WhiteNoise I’ve since removed all my academic files from the vault thinking that would get rid of the odd tags, but it did not. I then deleted the entire ObsidianCache folder (made a copy first, of course) in

C:\Users<username>\AppData\Roaming\obsidian\ObsidianCache\

as per your instructions here but still nothing. #tags that do, and don't exist but remain in tags list - #2 by WhiteNoise While I was there I also removed the main “Cache” file. Nothing.

Any other trick about regenerating the list of tags?

That method does not work anymore because we made changes. However, I am surprised by what you are writing.

Is you vault empty but it still shows the tags in the tag pane?

No, my vault is not empty. I just removed all the offending files.

are you sure you removed all the offending files? Does closing and reopining obsidian help?

@WhiteNoise I was just about to post… I forgot that tag pane was (core) plugin, so I disabled, reopened my vault, enabled, reopened, and the tags were still there. However, this time I actually got a search result when clicking on one of the tags (which I never had before, so progress) and the offending file with about 14 weird tags were associated with a picture (png)! I deleted it, put it back to make sure, and yup, all the bad tags came back. Still trying to figure out how a png can cause so many bad tags but at least I found it.

that’s very odd! If you add the image the tags appear? Are you sure it’s .png perhaps is it .png.md??

it gets weirder. It is definitely a png file (with no tags in it, basically no image properties - I checked with Faststone). I did the test again, took it out, put it back in, only this time nothing happened - the bad tags never came back. Unless opening the png with Faststone did something to it to remove something? I don’t know. It works fine now and the png doesn’t cause any issues. What a weird one.

If it happens again, let me know or even better send the file too.

1 Like