Add a property to 1.500 files

What I’m trying to do

I want to add “tag: #m/diary” to all of my files in the folder “Diary”

Things I have tried

  • googled it
  • searched for plugin

For y’all who code this probably doesnt seem like a big problem, but for me its tough :smiley: And I can’t do the same repetitive task with 1500 items.

Would be sweet if someone had a clue how I could do that :))

2 Likes

I tried Linter just now. Not working (reliably with the hack, either, involving adding tagz instead of tags, then doing a regex replace).
Metadata Menu? Mapping classes to tags? Possible? Learning curve issues, anyway, especially if it involves canvases.

There should be a feature request made for such things.
There might even be a help thread with such things done, possibly with regex replacements?

Can you paste here the metadata structure of 3-4 files that these diary files usually contain? Any tags already featured, etc?
I don’t need the actual prose, just the top of the files (YAML, frontmatter, whatyoumaycallit).
There may not even be any YAML, just prose?

There might even be a plugin that does this? Mapping folder to tags?

Anyway, if there are irregularities among YAML keys – if any – Linter can still help.
Go to YAML section (second tab).
Type in your Properties or keys like so, in the order you want them. In my case:
Screenshot from 2023-09-21 00-35-29

Also, a necessary step. Make at least your tags multi-line. Might as well make other properties multi-line as well, as that’s what Obsidian and Properties support. In my vault:
Screenshot from 2023-09-21 00-41-22

If these precautions are taken, the job is far easier. And as I said, it’s good practice to standardise the YAML or Properties structure.

Then go out of the Linter settings and into the Files pane and run “Lint Folder” on folder “Diary” (right-click).

Then when you are done we can more easily insert the tag, as either the first or the last among the values (if any).
Then the easy step comes last, using VSCode, Notepad++ or Sublime Text.

If there is no existing YAML or Properties in files in Diary, no need to do anything with Linter (not for me anyway).

2 Likes

I had the same requirement and found a python script that did the job. You have to edit the code and know the python language to use it. It’s a python library at py-obsidianmd · PyPI. Maybe that wil help as i had no yaml in my notes and needed it for using publishing tools and then to categorise my notes for a dataview page listing the notes in those categories.

1 Like

It was way past my bedtime so I did not finish. I wanted more information anyway.
But for other people interested as well, I’ll put up the solutions one can pick from.

Scenario 1

No existing YAML. We’ll make one.
See:

  • Instructions are there.


Scenario 2 and 3 need your files Linted in the way shown above.

  • Well, that’s how I started out, but because I did not get feedback and made my answer relate to people with different needs, the linting is not really necessary for me but I did employ the multi-line method that Properties likes.

Scenario 2

There was at least one existing tag.
See:

Scenario 3

No existing tags. We’ll anchor to some other thing existing in your YAML keys.
See:

  • Rename pickakeytoanchor to your featured key. We’re putting the tag above it.

Scenario 4

If out of the 1500 files you have files with mixed YAML, meaning you have e.g. 850 flies without tags and 650 files with tags, you can run searches and separate the types into Diary1 and Diary2 temporarily.
I’m not saying this is the only way, just writing it here because these file manipulation skills will come handy in the future when dealing with text/markdown files.

In all cases in the URLs above the regex parts will be used to match or search and the substitution parts will be used to replace. As I said you’ll need to install VSCode, Notepad++, Sublime Text – any of these available on your system. I cannot go into tutorials on how to use them. Sublime will make changes and wait for you to save all files manually.
What you need to do is:

  • Make backups before regex changes.
  • Pick the folder you want to make changes on (/vaultPATH/Diary).
  • Copy and paste the match and replace parts in the URLs above to the corresponding boxes in your text editor and press replace all.

It might be worthwhile to play around in Regex101, to see if you make changes to the substitution line, what other results you’d get.
Actually, it is a necessary thing as the keys I provided are not for everyone’s taste or requirements.
So finally I provide a simple case as well, with no other YAML keys:

Typo:
# project1 is not an inline tag but heading one. I accidentally put a space in there. Doesn’t make a difference.

1 Like

If you are on windows, notepad++ is nice for this kind of thing, makes it easy to undo as opposed to a script.

You can select containing folder to open all of the notes in question. Then replace with:
Find: \z
Replace: \r\ntag: #m/diary
Options: Regular expression

Replace All in Opened Documents, then Save All if everything looks ok.

The requirement was for Property, not for tag just anywhere.
At any rate, all Help answers must preferably target a wider audience with related issues or use cases.

Gino’s solution is good but i found the python method worked better. The library opens up the files and parses them then lets you add a new property and if there is no yaml front matter it creates it with the new property added. It then saves the file and goes on to the next one. It’s only really useful for programers who know python though as you have to edit the code to specify the filenames, directory and what you want it to do: add, delete etc. As I am a programmer that was ok for me. But i’m thinking there might be another way by using the macro ability of vs code (via an addon) to do the same thing. Still needs you to learn the macros though.

To give you an example, i was using Jekyl to publish my content and that needs to have a layout : page property which i didn’t have. So i used the python libray to add one into all my files. I subsequently added a lot more properties using templates so all new documents had the new properties.

And then there is Templater and Obsidian API if you are handy with that sort of thing and want to do it within Obsidian, using soft coding for modified dates, etc.
It may not be very difficult but I simply tried to 1) shy away from that 2) cater to needs of the everyday Property/YAML user.
I agree that programatically speaking my solution is rudimentary.
I installed Python on all my PCs and my JB-en iPad (have Pythonista there as well) but never really touch it. A few years ago I thought Python was a possible candidate for a language to learn, now I see JavaScript more useful (but a complete novice at it), especially because of the integration of Obsidian and Templater, Quickadd and a myriad other plugins. I seem to remember you can do Python inside Obsidian as well (on PC).

NB: There is another Python method to do with YAML if you search the forums. The person has a YouTube guide as well.

  • This person uses the DataView inline field method, though and most people want their Properties core functions working.
2 Likes

One final thing about the plugin Linter.
There is only one global setting, which means if you want different metadata (YAML keys and field values) set for different folders, you need to disable the plugin, go into .obsidian/plugins/obsidian-linter, make a copy of the data.json file and juggle those from different settings scenarios.

  • Meaning that you need to rename one of your copies with a particular set of settings to data.json and then re-enable the plugin to lint a different folder, for example. All this after making the changes in Linter again.

This was just a general observation, mind you, not related to the drill required as the YAML Key Sort setting relate to any files with more or less keys (any keys not among the list will be placed last).

In this way, the MetaData Menu plugin can be more helpful, but one needs to learn one’s way around yet another plugin.

I’m guessing you’re using Windows? I suggest a third party tool that’s specifically built for searching and replacing in bulk. There are a bunch out there, such as

That one lets you preview the changes. Of course you should make a copy of all of your files before making changes like this, and I’d suggest keeping a backup anyway just in case you decide you don’t want all the changes you made.

Are you looking to put the tag in a specific location in each file?

If using YAML, a search and replace tool should allow you to add the properties you want below the opening YAML tag, or above/below any other line that appears in all of the files you want to modify. Actually you would have to check for the first opening YAML tag only, or limit the change to once per file. I don’t know which search and replace tools allow that.

Good luck!

I personally find Visual Code Studio to be invaluable for find and replace across many files. Out of laziness, check out this link (Visual Studio Code: How to actually search and replace a word in all files? - Stack Overflow) but let me know if you have any questions.

oh damn…

all of these solutions seem to do the trick, more or less—
but I’m like a dyslexic when I see code.

I can’t even get the vscode search-and-replace-in-all-files thing going.

@gino_m I do appreciate your input, I tried to understand. hopefully it’ll be useful for someone out there!
@WhiteNoise I’m unable to manage this, mark a solution if you want

literally, I click on this, and don’t even know which button to press to make changes, or apply this.
or if I should insert this into a local programming tool, such as vscode, or… idk………

If you want to create YAML, you will need to copy the lines from both the Regular Expression and Substitution boxes on the Regex101 site to a text editor.

If you haven’t done this before and have no preference for VSCode or Sublime Text, dowload and install Notepad++, if you are on Windows. If you are on Mac, probably you’ll need to download either of the other two programs and install it.

Then when you copy the lines, you can try on a single file, after ticking on Regex.

You can also modify any key or key value, like I do here with topic changed to project, false to true:
notepad++_YntsfEndHP

If you are happy with how the changes are made on the test file, you can replace all your files in 'Diary". Make a backup before you do.

If you don’t need a YAML, you’ll need to look at the other scenarios. Again, try with individual files first to see what is changed to what and modify the English words before the n parts, and also, if you want alternative key values, you need to put your own ones based on this format: tags: \n - m/diary.

Finally, you can do your whole Diary folder. You can make changes to 1500 files in less than a minute.

You can play around with it for an hour. Nobody’ll know.

1 Like

Hi M, I heard once a great conversation around RegEx:

Dev 1: “I’ve got a problem and I think RegEx can solve it”
Dev 2: “Now, you’ve got two problems”

And it is 100% true.

I have your problem in my head and as a mad-keen developer I’ve writing some PowerShell that will help you, and hopefully the wider community once it’s done. It’s gotten bigger than Ben Hur, but would look like the following, if you’re on Windows and using PowerShell:

Update-MarkdownFrontmatter -Path . -Property “status” -Value “new” -Force

[this would update any “status” yaml in your files to have a value of “new” and would create it if it doesn’t exist].

I’m also working on Remove-MarkdownFrontmatter - to bulk remove properties.

And, of course, Add-MarkdownFrontmatter.

The Update and Add CmdLets will also support appending tags.

Will publish back here when it’s done. :slight_smile:

2 Likes

MarkdownFrontmatter.zip (5.9 KB)
Ok, here it is. A cool 695 lines of code. But the result should be worthwhile. It’s only minimally tested. But at least I did stress test it with >2,000 (small) markdown files (note that Obsidian really isn’t very happy when 2,000 files are modified while it’s open).

Once I figure out how, I’ll publish this to PowerShellGet to make it easier to access and update.

I can provide guidance of downloading, placing and importing the module into your PowerShell environment if you need. But I wanted to get to the fun bit - how to use it to solve your specific issue:

(excuse my formatting, too, I’m new into the Obsidian forum)

Import-Module .\MarkdownFrontmatter.psm1
$VaultPath = "C:\MyVault\Diary"
Add-MarkdownFrontmatter -Path $VaultPath -List "tags" -Item "m/diary"

The above will ADD the item to existing tags, leaving any other tags you have intact. If, instead, you ONLY want the tag to be m/diary, you could use this:

# overwrite the tags property with this array of (a single item) m/diary
Add-MarkdownFrontmatter -Path $VaultPath -Property "tags" -Value @("m/diary") -Force

If you’re worried about what it’ll do, add -WhatIf to the list of parameters. If you really love pressing enter, add -Confirm to the parameters and it will confirm each file update.

By default this will also backup any .md file to .bak. If that’s too much, add -NoBackup to the parameters to avoid that.

Only want to do the one folder and not all subfolders? -NoRecurse

If you’re new into PowerShell you can find the commands I’ve created with:

Get-Command -Module MarkdownFrontmatter

And each command has full help and examples for use:

Get-Help Remove-MarkdownFrontmatter -ShowWindow

I’d love to know:

  • does it work
  • does it work on a Mac

Oh, sick. I just worked out that the Obsidian forum uses markdown. Well, derrrr.

PS> Get-Help Add-MarkdownFrontmatter

NAME
    Add-MarkdownFrontmatter

SYNOPSIS
    Adds properties to the frontmatter of markdown files


SYNTAX
    Add-MarkdownFrontmatter -Path <String[]> -Property <String> -Value <Object> [-NoRecurse] [-NoBackup] [-Force]
    [-WhatIf] [-Confirm] [<CommonParameters>]

    Add-MarkdownFrontmatter -Path <String[]> -List <String> -Item <String[]> [-NoRecurse] [-NoBackup] [-Force]
    [-WhatIf] [-Confirm] [<CommonParameters>]


DESCRIPTION
    Adds properties to frontmatter if it doesn't exist, and assigns a value to it. If the property already exists, it
    will not be overwritten unless the Force switch is used. Can also be used to add items to a list in the
    frontmatter.

3 Likes

Never tried this so I don’t know how successful the outcome will be:

1 Like

fez-github now has a more robust plugin that will add any kind of property. The plugin is in Community Properties and Works Great! It’s called: “Multi Properties”

Thank You for posting about this plugin and about this developer!

3 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.