New Properties and templater prompts and commands?

[Feralflora]

Hi Gino,
Thanks a lot for testing and feedback! I’m glad you’re putting it to use in your workflow

Regarding 1, all the functionality you coded is already built into the script (unless I misunderstood something).

By default, missing keys are always added. It’s only in the case of existing keys that the treatments come into play. You can see how that’s handled here:

// Check if property exists, if not, add it

if (!fileCache || !fileCache.hasOwnProperty(prop.key)) {
    console.log(`${target.basename} doesn't contain ${prop.key}`);
    console.log(`Adding property: ${prop.key} to ${target.basename} with value: ${prop.value}`);
    frontmatter[prop.key] = prop.value;

// If property exists, handle it according to treatment type   

} else {
    console.log(`${target.basename} contains property: ${prop.key}`);
    if (treatments.hasOwnProperty(prop.treatment)) {
        treatments[prop.treatment](frontmatter, prop, value);
    } else {
    // Default treatment
    treatments['add'](frontmatter, prop, value);
    }
}

So the following should do the same as what were trying to prep before letting the script handle it:

<%*
tp.hooks.on_all_templates_executed(async () => {
	const formattedDateTime = tp.date.now("YYYY-MM-DDTHH:mm:ss");
	const targetFile = tp.config.target_file;
	const properties = [
    	{key:'dg-updated', value: formattedDateTime, treatment: 'update'},
    	{key:'dg-created', value: formattedDateTime},
    	{key:'dg_upload', value: 'VNC done', treatment: 'update'},
    	{key: 'status', value: 'dg_uploaded', treatment: 'update'},
		{key: 'share', value: true, treatment: 'update'},
		{key: 'dg-publish', value: true, treatment: 'update'},
    	{key: 'bool', value: true, treatment: 'update'}
    ];
	await tp.user.merge_frontmatter_function2(targetFile, properties);
});
-%>

See demo:

Here, I run the script several times with the same parameters and treatments to showcase how the behavior changes, depending on the treatment, whether the property exists, and whether the values are different.

So if you want something added if it doesn’t exist, but not updated if it does, use add or omit treatment to get the default (same, same). update will add if it doesn’t exist, update if it does (unless the values are the same, in which case, it won’t do anything).

Please let me know if I misunderstood what you meant.

Btw, I noticed that tp is now redundant as a parameter since I changed the script to accept the TFile as a parameter. I’m updating the scripts above to reflect that. I also removed some minor redundancy from the update case.

Regarding point 2:
I couldn’t test this due to the insider bug in processFrontmatter. But thanks for sharing your solution!

[holroy]

@Feralflora

Thanks a lot for testing and feedback!

Been a busy day, so I’ve not been able to get around to getting you some feedback, but I’ve noted your efforts, and will give some feedback in the not so distant future.

@Feralflora

Btw, I noticed that tp is now redundant as a parameter since I changed the script to accept the TFile as a parameter. I’m updating the scripts above to reflect that. I also removed some minor redundancy from the update case.

Don’t be too eager to remove the tp variable. One of the reason it’s often present is that it allows for the user functions to be used outside of Templater. For example it’s possible to access Templater functions from Dataview, but that do require for the tp variable to be set by Dataview. So if you remove it, you also remove the possibility to use your functions from Dataview, and that would be a little sad. I can imagine using this kind of function from within dataviewjs in various contexts…

[gino_m]

@Feralflora

unless I misunderstood something

I was talking about any status properties that had and accepts one value. My conditional (which is kind of pointless as I already had removed one condition and over time I extended the scope that this particular template was originally intended to cover) also shows what I had in mind.

• EDIT: well, there used to be two if's there but one had to go. Anyway. I had to update two kinds of statuses, not make a new property.

I’m not sure (or: up until now I wasn’t entirely sure) how accepting one or more values (which ultimately would – in my mind – necessitate using the add or update treatment as per your settings) works under the hood.
Possibly like this (anyone is welcome to counter these if I’m mistaken):

• If you have key: value format, then Obsidian treats it as text in a single-line array (“array” in my mind means a series of more than one, but that’s the term Linter uses in its settings, in which one can define which property one wants multi-line). The icon in the Properties UI looks like this:
  
  The icon seems a little similar to that of list. The important thing is that it is a ‘text string’. In my case, it is a status (either one of the two): status can be either…or, not usually and logic-wise.
• Among the multi-line properties there is list – again a word denoting or at least hinting at a series of more than one – speaks for itself and the icon is probably more graphically spot on.
  The multi-line array is clearly seen in the source view where each value is headed with the required syntax -:
  
  
  
  When Obsidian knows or “made up its mind” about these types as lists, it will (or at least should) allow ‘updating (the list) by adding’ – I am using the function names used for treatments on purpose here.
  
  
  Similarly to list items, Obsidian also has on board default properties tags and aliases which – already shown by the use of the much-debated plural (“more than one”) in the property names – will be used as multi-line properties, which is shown by the syntax used; even if there is (currently) only one tag(s) and alias(es) in the YAML, it will not be populated in key: value format (with the key and value on the same line), but in multi-line format:
  

So to get back to the difference and similarity of add and update, it depends on the property type. Where the status was set (by whom and where? does Obsidian look for ‘status’ word?) as a single-line property value, Obsidian will not budge on (default) add, as it is not a multi-line (list). It expects your update treatment. When it’s a list, as with tags, your default add will update the list.
I don’t want to go into dates and Booleans here as these cases are more straightforward, although my date modified field was only updated on update treatment (which is fine, nothing wrong with that).

From the above, I reckon that it could be programatically achieved that single-line properties and dates could be updated on default add as well, but as I said, it’s not a big deal if the script stays as it is. What I have in mind again is the people who will in 2-3 months time will start using this and you could save (ourselves) some headaches going forward. I may be going over the top on this, granted.

What I haven’t tried yet is your override parameters. So all in all, I feel a bit overwhelmed by the fantastic effort yet – so bear with me.

@Feralflora

I’m glad you’re putting it to use in your workflow

• People out there will be following suit for the reasons mentioned above. The js script needs to be done and filed away once and the template md files are quickly done by anyone not too savvy about coding. This will hugely make it easier with non-tech users.
• Some concern is 1) reliance on Templater which can break or be abandoned 2) processFrontmatter – which is used in the js script – issues cropping up here and there; then we have to go back to search and replace scripts which I recommend everyone to not delete but keep with telling filenames.

@Feralflora

{key:'dg-updated', value: formattedDateTime, treatment: 'update'},

• I used value last and it worked like that as well. Great stuff.

@Feralflora

Regarding point 2:
I couldn’t test this due to the insider bug in processFrontmatter. But thanks for sharing your solution!

• Probably there is a better solution. It was a quick ad hoc treatment on my part.

So, again, chapeau from me and I’ll be advertising the method once it passes (scrutiny from) under some Norwegian clouds first to be blown back down south again…

[Feralflora]

@holroy

I’ve noted your efforts, and will give some feedback in the not so distant future

Thanks, I appreciate it!

@holroy

One of the reason it’s often present is that it allows for the user functions to be used outside of Templater.

In this case, it was only present because I needed it for tp.config.target_file inside the script. Now, no templater functions are used inside the script.

You can still use the user script together with dataview if you just access tp in the dataviewjs script using the methods described here: Templater snippets

Perhaps you can expand on it, but I’m not seeing how that is contigent on tp being passed as a parameter to the script. On the other hand, without the tp dependency, this script can be run by anything that runs JS. An then again, adding tp back is a minor change.

[Feralflora]

@gino_m

I was talking about any status properties that had and accepts one value.

Text properties will accept multiple values and change into list properties, if you pass multiple values using processFrontmatter.

@gino_m

If you have key: value format, then Obsidian treats it as text in a single-line array

If you really meant a single-line array, like Linter supports, then this is not correct. Obsidian doesn’t support single line arrays. It will reformat to multi-line lists / arrays. A text property is not an array (doesn’t accept multiple values), but I think you were actually getting at that. As my test above shows, text will become list if you force multiple values onto a text property.

@gino_m

When Obsidian knows or “made up its mind” about these types as lists, it will (or at least should) allow ‘updating (the list) by adding’ – I am using the function names used for treatments on purpose here.

Using add on a property whose existing value is an array will push the new values to the existing array:

This only takes into account the existing values in the target note, not how the property is configured in the property UI. A value can only be considered an array if it’s formatted like this:

This is not nessarily the case even though the property UI says that the property type is list.

Using update on a property whose value is an array will overwrite the existing values with the new values:

Perhaps update should be called overwrite? In my mind, update updates existing values (but overwrite is just as descriptive, if not more), while add adds to existing values, but doesn’t update anything.

@gino_m

Where the status was set (by whom and where? does Obsidian look for ‘status’ word?) as a single-line property value, Obsidian will not budge on (default) add, as it is not a multi-line (list). It expects your update treatment.

This seems to be a misconception, because add can handle both single values and arrays:

'add': (frontmatter, prop, value) => {
    if (!value || (value && override === 'destructive' && value !== prop.value)) {
        console.log(`Assigning ${prop.value} to ${prop.key} in ${target.basename}`);
        frontmatter[prop.key] = prop.value;
    } else if (value && Array.isArray(value) && prop.value != null) {
        arrayPush(target, frontmatter, value, prop);
    }
},

However, since add is meant to be the safe option that doesn’t overwrite existing values, it will only add values, if the key doesn’t have an existing value. The destructive override changes this, so that add will overwrite single values. Perhaps that’s the behavior you are looking for?

@gino_m

although my date modified field was only updated on update treatment (which is fine, nothing wrong with that).

I mean, that is exactly the intended behavior and usecase for update.

@gino_m

From the above, I reckon that it could be programatically achieved that single-line properties and dates could be updated on default add as well, but as I said, it’s not a big deal if the script stays as it is.

Well yes, with the destructive override, but not by default, because this is meant to be an option that only adds, and does not remove / overwrite data.

@gino_m

What I have in mind again is the people who will in 2-3 months time will start using this and you could save (ourselves) some headaches going forward. I may be going over the top on this, granted.

I appreciate you having and discussing these concerns with me.

@gino_m

What I haven’t tried yet is your override parameters. So all in all, I feel a bit overwhelmed by the fantastic effort yet – so bear with me.

Sounds like you would benefit from the destructive one. And of course, no worries!

@gino_m

Some concern is 1) reliance on Templater which can break or be abandoned

There’s really no reliance on Templater, because the script is all javascript and Obsidian API. But of course, it’s usefulness is amplified by the Templater templating environment. However, you can run it using QuickAdd or whatever runs JS. Of course, that leaves point 2, but I hope the devs appreciate how people are using processFrontmatter.

@gino_m

I used value last and it worked like that as well. Great stuff.

Yeah, this is because these keys are named, and not index based, so it provides this flexibility.

@gino_m

So, again, chapeau from me and I’ll be advertising the method once it passes (scrutiny from) under some Norwegian clouds first to be blown back down south again…

Thanks! And, hehe, yeah, it will be interesting to hear what @holroy has to say, and perhaps hear more about those dataviewjs ideas as well.

[gino_m]

Yeah, I didn’t take into consideration the [field1, field2] array, which I hardly ever used, especially since the Properties launch, where the list values superseded those; in coding, apparently you can still continue to make use of these.
It’s really useful, what you put up here in animations as well.

@Feralflora

There’s really no reliance on Templater, because the script is all javascript and Obsidian API.

I saw that, I meant the hook call, the way we invoke it.
Also, can we use const currentFile = app.workspace.getActiveFile(); instead of const targetFile = tp.config.target_file;?

@Feralflora

The destructive override changes this, so that add will overwrite single values. Perhaps that’s the behavior you are looking for.

Yeah, as soon as I mentioned above, I knew I should not haste with the post. But God had his plans with me…

In any case, all of the above goes to show that… (“Oh-oh, yes, I’m the great pretender”)…I am a great—interviewer who brings the most out of my subject because if I get everything right, all the general public would see here on your part is “Yeah, Gino, you rock.”
:)))

So, with my job duly done here…
…I’ll be “Up on the roof” – to quote another oldie…

Thanks again for the great explanations – I’m sure even older pieces of furniture round here were enlightened by all this.

[gino_m]

@Feralflora

it’s much more powerful if you can use it in a loop and pass the target file

Although this doesn’t sound like something I’d do in my workflow, examples would be nice when you get the chance, because this is important for automation.

• Coupled with DataView, which example or boilerplate based on this will probably done by @holroy, I presume.
  Bulk modifying potentially hundreds of files based on search criteria which cannot be done outside of Obsidian is an important next step.

[holroy]

@Feralflora

Thanks! And, hehe, yeah, it will be interesting to hear what @holroy has to say, and perhaps hear more about those dataviewjs ideas as well.

It seems like Gino has covered some of the bases, but I’ve not read all of this very thoroughly, so I might repeat some things, and miss out on somethings already covered, but here goes my comments at the first level, and suggestion for improvements at lower levels:

• Regarding switch or function, I’m a little undecided, but intuitively I find both approaches hard to read as both the switch and the function definitions hides away what’s happening when. It gets convoluted to follow the logic of the script.
  • Define handleProperty as its own function outside of mergeFrontmatter(), and use a switch within it
  • Use the forEach() variant without await’ing in the main callback function used with processFrontMatter().
• The handling of the safe option also makes your code a lot harder to read, so I would preferably bail out sooner if its present.
  • Within handleProperty() do a simple test at the start and bail out if the safe option is set, possibly with a message stating that you bailed out of doing treatment…
• The add handling and the default handling is not identical, although I think I saw somewhere that you said they should be. That’s a little dangerous, as you then can experience different behaviour depending on how you call it.
  • Make the logic for both of these operations the same
• You’ve tested some of the cases of the property being an array or a simple list, but not thoroughly, so you might be changing an array directly into a simple value when doing update, or you’ll not add anything if the property has a single value but rather change it?. Similar cases exists if you do custom on array values…
  • Not sure how to tackle this, as the exceptions are many to the most basic rule, but be sure someone will trigger that strange combination
  • I’m thinking that an add on a single value should change it into an array
• How does your script handle changes to array values? You can add values _if its already an array, but not when its a single value. And can you update, and/or delete elements from arrays? Or do custom stuff to them? Or increment?
• For debugging purposes it’s nice to have loads of console.log() messages, but in a “production” environment less so.
  • Provide an option to disable all log messages, but the error related ones
• Dependency on variables into async method is inherently unsafe. You set the fileCache early on, and then after loads of awaiting here and there, you suddenly refer to it again. By now it could be changed or gone out of scope (not entirely sure on this one, as it depends on garbage collection/asynchronous call handling within javascript). In general it’s kind of unsafe to do stuff like that. Same logic applies to target and value at least. They’re suddenly appearing in a context where I’m not sure you can rely on their value, and where you don’t see where they came from easily.
  • Clean up the code, so that each function either defines locally, or through parameters all of its variables. No global stuff (except for function definitions in general)
• This might be related to the previous item, but why do you refer to fileCache within the Promise.all() when it already has the frontmatter passed along to that callback?

I’m sorry if this was hard reading, as I do believe there could be a good reason for doing something like you’re doing. Especially when we talk about mass updating across multiple files. However, that is also the case why one needs the script to be extremely precise and not leave certain property value/type/treatment combinations to cause different behaviours.

And I’m a strong believe that readable code is usable code, and being able to follow the logic (and separations of concern) is vital aspects to making it readable. Sadly, I’m finding either version somewhat hard to read, especially due to the mix of defining functions and callbacks and whatnots, before they’re actually called.

---

Hmm… Right before posting I now see something about dataviewjs ideas? What was that? Maybe it was me mentioning that a script like this could be useful to call from within dataviewjs as part of a larger file update thingy? Like if one wanted to change all the daily notes from an earlier stage with a mismatch in templates, it could be useful using a tool like this to update stuff.

[msfz751]

I love app.fileManager.processFrontMatter but can it be used to create new notes from a template? I unsuccessfully tried. Maybe an example may help.

I have used app.fileManager.processFrontMatter to recalculate formulas and write the results back in properties such as numFigures, numCallouts, etc., based on changes in the notes themselves.

[Feralflora]

Yes, if you use a timeout, or even better, the tp.hooks.on_all_templates_executed module from Templater (docs).
See these two examples here and here.

[holroy]

That last link of yours, to zachyoung, seems to indicate that app.fileManager.processFrontMatter can’t create the file by itself. But after the templater hooks have completed, which indirectly depends on Templater to create the file (or Obsidian if created through a link), then processFrontmatter can be called to do further work.

[Feralflora]

You’re right, I read @msfz751’s question too quickly and thought he was asking whether processFrontmatter could process newly created notes.

The best way to do that is using tp.hooks, if you are working with Templater. So since you already gave the right answer that processFrontmatter can’t create notes, my alternative suggestion would be to use tp.file.create_new to create new files using a specific template.

[holroy]

The file is already created, so no need to create another file…

[Feralflora]

I don’t think that’s the case:

@msfz751

can it be used to create new notes from a template? I unsuccessfully tried. Maybe an example may help.

[holroy]

I’m going to leave this thread now. I feel we’ve diverged to far from the original request, which also has been marked with solved. There could be made a multitude of new requests based upon this thread, but I’m out for now.

[gino_m]

Quits Obsidian on iPad alas, so had to revert to old versions for now.

[Feralflora]

Oh really? I haven’t gotten around to testing the new vesion on mobile. I am going to check it out on the mobile console when I have time sometime this week.

[Feralflora]

@stef.rausch

<%_* tR += "" _%>

Shouldn’t this be <%_* tR = "" _%>? It didn’t work for me until I removed the +. Otherwise, it’s not removing anything.

[Badger]

I’m going to have to read this a few more times to get the full jest of this.

…but what I’m understanding is: if you use Templater ‘code’ in your YAML, you have to create a full block of code, including all properties you currently have in the Template.

For example, I use these along with some ‘standard’ ones:

modification date: <% tp.file.last_modified_date(“dddd Do MMMM YYYY HH:mm:ss”) %>

UID: <% Math.round(((Math.random()10**15)(tp.date.now(“YYYYMMwwDDHHmmss”))/(Math.random()*10**15))) %>

File Creation Date: <% tp.file.creation_date() %>

The ‘modification date’ will never again update though…

[Hendrik_P]

I have read this a few times and obviously am missing things. I get the basic functionality ( replacing and adding frontmatter in existing file) working, but the rest of my content ( other headings etc…) are somehow replaced when the script starts running. Any idea why? My basic operation is cmd n - for a new note, followed by a shortcut key for the Templater script. Any direction would be highly appreciated.