Fix wrong YAML in multiple files

Hi,

i tried multiple versions of regex, but i simply dont get it right. also searched the forum and google but cant find the correct regex.

i have hundreds of files with wrongly formatted YAML:
i had a template like this:


Date::
Project::
Created::

Activity::
Contact::

so there are inline and YAML key:value pairs … but all of them are with double colon. And this is apparently wrong, so dataview will not work. i am now trying to fix this double colon in YAML only …
so basically “Change all :: to : between the first — and the second — of any file”

any way to do this?

thanks.

Which tools do you have available? Visual Studio Code? Only Obsidian?

i have VS Code, but if it works within Obsidian with the regex replace it would be good too

but VS Code is a valid option of course and probably more capable :slight_smile:

If I’m not mistaken the regex search is global, whilst the search-and-replace is local to the note.

I’ll see if I can conjure up some magic for VS code later today. Kind of tricky to keep the search within the frontmatter…

1 Like

thanks! and i am relieved, that its not an easy task at least :slight_smile:

@holroy, I stumbled over a very good idea over at the discord: putting the YAML in a Callout instead to have full inline data fields (with links and all) available … this does actually work even better for me.

now i need to find a way to get the yaml, transform it to a callout and place it at the bottom of each file … :slight_smile:

Before we start

Please take a backup

You’re about to search-and-replace to an entire vault, so I can’t stress enough to make a backup of your folder before you start, in case something really should go wrong. I’ve tested the regex on my own vault, so I’m pretty sure there aren’t any large errors, but please do take a backup.

During the testing of these regex’s I found that VS Code hung up on me in some occasions, so I had to restart it a few times. This was when I was typing the regex one by one character, and it kept trying to execute the search whilst I was typing.

Toggle the regex option, .*, to avoid hang ups

Therefore, I can’t stress enough that before you do a change to the search (and/or replace) strings, turn off the regex option, .* on the far right, so that it doesn’t do the search with regex before you’ve completed entering the search. I also recommend to copy-paste the strings, instead of typing them in. These beasts, are just that, beasts!

Finally, you need to repeat some of the operations, and the easiest way I found to repeat them is to toggle the regex option, .*, off and back on again. It’ll then repeat the search (and/or replace). Keep doing them, until you see “No results found.” before proceeding to the next step.

Open up VS Code

I’ve not mentioned it before, but open up VS Code, and open up your vault folder, and get ready to do this.

So, take the backup, and let’s continue…

Multiple frontmatter fences?

Search: (---\n(?:.*\n)*?---\n(?:.*\n)*?)---\n
Replace: $1-#-#-\n

This will check if any of your files has multiple occurences of the frontmatter fence, ---, and if so, it’ll replace the extra ones with the -#-#-, which I’m hoping doesn’t exist in your files. If you suspect that, come up with another string, not likely to be present.

Look through the search result, and verify that it seems correct, and either go through and replace each and every one of the matches, or hit the Replace all icon next to the replace field.

After the first run, toggle the regex-option, .*, off and back on again, so that VS Code repeats the search. Repeat the replacement, and repeat this until it says “No results found”.

Frontmatter not a start of file?

Furthermore we need to verify that we’re actually only checking frontmatter, which actually are at the front of the file. So let’s do a search for this. Remember to turn off regex-option, .*, if you are to type this in manually. It’ll most likely hang VS Code if you don’t…

Search: (?:.*\n)+?---\n(?:.*\n)*?---

When it’s entered, re-enable regex-option, .*, and see if you’ve got any matches. If you do have any matches, then go in manually, and change the triplet, like we did automatically before. In other words, change --- into -#-#-#.

Repeat the search, by toggling the regex-option, .*, off and back on again, and verify there are “No results found” before proceeding.

The actual replacement of the double colon

As before, turn off the regex-option, .*, before doing changes to the search and replace strings.

Search: (---\n(?:.*\n)*?)([^:]+?)::(.*\n)((?:.*\n)*?---\n)
Replace: $1$2: $3$4

This search-and-replace needs to be repeated as many times as the longest frontmatter in your vault has fields. So get used to toggle the regex option, .*, and hitting the Replace all icon, after a quick glance through the search results.

Repeat until you get “No results found”, yet another time. When done, you’re nearly complete with the entire operation.

Home stretch

To finalise this operation, we need to fix those sanity checks we did in the start.

Search: -#-#-
Replace: ---

And when you’ve search and replaced these, and repeated until you don’t find any more, your vault should now not have any double colons in the frontmatter anymore.

3 Likes

This is a brilliant solution. Spent an hour or two trying to hack at this myself(learning more than I ever wanted to about regex in the process), and couldn’t figure it out for the life of me. This works like a charm, and I hope OP is able to execute it without issue.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.