New Properties and templater prompts and commands?

Thanks for the explanation.
I was asking especially because users may find it cumbersome having to deal with an additional file that they may forget about later.

I think so, but it was late and I’m not sure. More testing is needed.
I will play around with it some more during the day – if I can incorporate it in my default template and see if I can seamlessly call it in my snippets.

You’re welcome, and thanks for the exchange. I’m looking forward to your findings!

1 Like

If you escape out of a Templater prompt…

parent_topic:
  - <% tp.system.prompt("Want to define parent topic?") %>
  - <% tp.system.prompt("Care to add more parent topic?") %>

…empty multi-line values are added:

---
title: Testing
foreignTitle: true
aliases:
  - testing
status: formatted
share: true
dg-publish: true
dg_upload_status: down
parent_topic:
  - 
  - 
date created: 2024-02-01
date modified: 2024-02-01
---

This is my own default template, in which we can ask for user input. How can you ask for user input with your method?

As for the merging changes pop-ups, they do occur.
I used a hybrid of my own default template and added…

<%*
tp.hooks.on_all_templates_executed(async () => {
	const properties = [
        {key:'origin', value: parentLink},
    ];
	await tp.user.merge_frontmatter(properties, tp);
});
-%>

…then it gave me the pop-up 3 times in succession, when creating a new file.

Am I correct that your merge script actually calls await app.fileManager.processFrontMatter() once for every parameter you pass it? So when having 6 parameters, like in your example, you actually end up with with 6 independent write operations on the file?

Oh wow, I hadn’t even considered that for some reason. I suppose I should accumulate the changes according to the processing logic, and then do everything in one go. I hadn’t noticed an issue, because it would still complete fast, but I suppose this could be why @gino_m is seeing the merging changes popups. Do you have any suggestions for this refactor, @holroy?

@gino_m Using my alternative frontmatter approach, you would put the prompts into the same tp.hooks section that defines other property values to add to the properties.

<%*
tp.hooks.on_all_templates_executed(async () => {
    const parent_topic = await tp.system.prompt("Want to define parent topic?", null, false, false);
    const properties = [
        {key:'parent_topic', value: parent_topic},
    ];
	await tp.user.merge_frontmatter(properties, tp);
});
-%>

Ah, okay. Condensing them into one line didn’t work.

Well, when I do something similar (but simpler) and for many files, I made the loop on my multiple files detect the correct tFile handle, and then pass on everything to app.fileManager.processFrontMatter() using local scope to transfer variables. It seemed to do the trick just nicely.

In your case, your function would then just find the tFile, and call the function and have all the logic within the callback. This also allows for somewhat simpler logic, as you already have access to the frontmatter object.

Does that make sense?

1 Like

@Feralflora The very cryptical holroy may have alluded to this method of his:

1 Like

HeHe… It was not the meaning to be cryptical, but in this context I just wanted to focus on keeping the other part simpler, so doing a loop on multiple files didn’t seem to be that much of a point in elaborating on. :smiley:

1 Like

I just wanted to illustrate the principle in that example. To have multiple prompts for multiple themes / topics, I would adapt this Infinite prompt until no value snippet from @Zachatoo like so:

<%*
tp.hooks.on_all_templates_executed(async () => {
	let allTopics = [];
	let isAddingTopic = true;
	while (isAddingTopic) {
		  const topic = await tp.system.prompt("Want to define parent topic?");
		  if (topic) {
			  allTopics.push(topic);
		  } else {
			  isAddingTopic = false;
		  }
	}
    const properties = [
        {key:'parent_topic', value: allTopics},
    ];
	await tp.user.merge_frontmatter(properties, tp);
});
-%>

With this, you can enter as many themes as you want. When you are done, either use escape or press enter without writing an input. This ends the loop, and your input values are then added to the properties. I’ve tested this, and it seems to work well :+1: In action:

Obsidian_LHxUoVMcFJ

Thanks for the advice, I’m making progress on this and will share the refactored userscript when I’m done. I think it will include the ability to specify the update behavior as mentioned before. This could either be done on a per-property basis in the properties array, or as a global setting that affects the treatment of all properties in the array.

Thanks for the link! Even without it, Holroy’s advice did come in handy, though :slight_smile:

3 Likes

By the way, the loop picking topics illustrates another key point related to my thinking on what to do and not to do inside of the frontmatter processing. The logic inside should be simple and fast, and it should definitely not include user interaction. Just like you illustrate here with building the topic list outside of the frontmatter processing before instead inside. Kudos to you for that choice!

I believe this is the better option since being within app.fileManager.processFrontMatter internally in Obsidian should flag that others need to keep their hands off from editing this file, so it’s kind of crucial we limit the time spent inside. So I wouldn’t await other stuff or do user interaction on the inside.

1 Like

@holroy & @gino_m Interested?

Wassup, app.fileManager.processFrontMatter ain’t workin’ for Insidas?
Shoot, Sheriff.

Yeah, I saw reports of that, but I think it only occurs when there are no properties to begin with. Seems to work as expected in this test, although I’m on version 1.5.5.

Why shouldn’t it work for Insiders? And the not working for empty frontmatters was fixed some time ago. At least the cases I knew about and had tested.

The format of your new version looks nice, @Feralflora , and indicates a much nicer segregation of duties, as you seem to have prepared what to do ahead of the time spent inside app.fileManager.processFrontMatter().

When will the be a Share & showcase ?

Oops… And now I read the latest bug reports. I’ve not seen that before just now and haven’t tested it or used the functions that recently…

Yeah, it’s a recent bug. No --- surrounding the properties, when processFrontmatter creates the properties from scratch.

Yes, please.

Anything to take me away from having to do thousands of file updates to do with translations (machine-translation revisions).

I’m glad you like it. As for the showcase, I was actually hoping to get some feedback before then. I will share the code below here.

In trying to optimize the performance, I actually made two versions, one using a switch, and one using an object of functions. Both versions seem fairly fast most of the time, but it varies. It’s a bit hard to judge these things from my end, because my PC is not very powerful.

I’m still a bit unsure about the need for async in this context. In the object-based script, I didn’t make the frontmatterCallback function async, and it still works, but I’m not sure if it affects the performance.

Code

merge_frontmatter_switch
/**
 * Merge frontmatter properties into a target file's frontmatter.
 * @param {Tfile} target - Target's TFile
 * @param {Array<Object>} properties - Array of property objects to merge.
 * @param {string|null} [override=null] - Optional parameter to specify override behavior.
 */
async function mergeFrontmatter(target, properties, override = null) {

    try {
        // Get the target file's frontmatter cache
        const fileCache = app.metadataCache.getFileCache(target)?.frontmatter;

        // Handle existing properties according to treatment
        const handleProperty = async (frontmatter, prop) => {
            const value = fileCache?.[prop.key];

            // Switch based on the treatment type of the property
            switch (prop.treatment) {
                case 'delete':
                    // Delete frontmatter property if override is not set to safe
                    if (override !== "safe") {
                        console.log(`Deleting ${prop.key} from ${target.basename}`);
                        delete frontmatter[prop.key];
                    }
                    break;

                case 'custom':
                    // Apply custom code, if override is not set to safe
                    if (override !== "safe") {
                        try {
                            // Execute custom function to get new value
                            const customFunction = new Function(`return this.${prop.code}`);
                            const newValue = customFunction.call(value);
                            console.log(`Changing ${prop.key} from "${value}" to "${newValue}" using custom code: ${prop.code}`);
                            frontmatter[prop.key] = newValue;
                        } catch (error) {
                            console.error(`Error applying custom treatment ${prop.code} for ${prop.key}: ${error}`);
                        }
                    }
                    break;

                case 'increment':
                    // Increment property value with new value, if override is not set to safe
                    if (override !== "safe") {
                        console.log(`Incrementing ${prop.key} by ${prop.value} in ${target.basename}`);
                        frontmatter[prop.key] += prop.value;
                    }
                    break;

                case 'add':
                    // Check if property value is not already set
                    if (!value || (value && override === "destructive" && value !== prop.value)) {
                        console.log(`Assigning ${prop.value} to ${prop.key} in ${target.basename}`);
                        // If value is null or override is destructive, set property value to new value
                        frontmatter[prop.key] = prop.value;
                    } else if (value && Array.isArray(value) && prop.value != null) {
                        // Push new values into existing array
                        arrayPush(target, frontmatter, value, prop);
                    }
                    break;

                case 'update':
                    // Replace property value with new value, if override is not set to safe
                    if ((override !== "safe") && (value !== prop.value)) {
                        console.log(`Updating ${prop.key} to ${prop.value} in ${target.basename}`);
                        frontmatter[prop.key] = prop.value;
                    }
                    break;

                default:
                    if (Array.isArray(value) && prop.value != null) {
                        arrayPush(target, frontmatter, value, prop);
                    }
                    break;
            }
        };

        // Callback function to handle frontmatter
        const frontmatterCallback = async (frontmatter) => {
            await Promise.all(properties.map(async (prop) => {
                if (!fileCache || !fileCache.hasOwnProperty(prop.key)) {
                    console.log(`${target.basename} doesn't contain ${prop.key}`);
                    console.log(`Adding property: ${prop.key} to ${target.basename} with value: ${prop.value}`);
                    frontmatter[prop.key] = prop.value;
                } else {
                    console.log(`${target.basename} contains property: ${prop.key}`);
                    // Handle existing property
                    await handleProperty(frontmatter, prop);
                }
            }));
        };

        // Process frontmatter of the target file
        await app.fileManager.processFrontMatter(target, frontmatterCallback);
    } catch (error) {
        console.error("Error in processing frontmatter: ", error);
    }
}

// Function to push new elements into existing array property
function arrayPush(target, frontmatter, value, prop) {
    console.log(`${prop.key} is an array`);
    const valueSet = new Set(value);
    for (const item of prop.value) {
        if (!valueSet.has(item)) {
            console.log(`Adding ${item} to ${prop.key} in ${target.basename}`);
            frontmatter[prop.key].push(item);
        }
    }
}

module.exports = mergeFrontmatter;
merge_frontmatter_function
/**
 * Merge frontmatter properties into a target file's frontmatter.
 * @param {Tfile} target - Target's TFile
 * @param {Array<Object>} properties - Array of property objects to merge.
 * @param {string|null} [override=null] - Optional parameter to specify override behavior.
 */
async function mergeFrontmatter(target, properties, override = null) {
    
    try {
        // Get the target file's frontmatter cache
        const fileCache = app.metadataCache.getFileCache(target)?.frontmatter;

        // Object containing treatment functions for different property treatments
        const treatments = {
            'delete': (frontmatter, prop) => {
                console.log(`Deleting ${prop.key} from ${target.basename}`);
                if (override !== 'safe') delete frontmatter[prop.key];
            },
            'custom': (frontmatter, prop, value) => {
                if (override !== 'safe') {
                    try {
                        const customFunction = new Function(`return this.${prop.code}`);
                        const newValue = customFunction.call(value);
                        console.log(`Changing ${prop.key} from "${value}" to "${newValue}" using custom code: ${prop.code}`);
                        frontmatter[prop.key] = newValue;
                    } catch (error) {
                        console.error(`Error applying custom treatment ${prop.code} for ${prop.key}: ${error}`);
                    }
                }
            },
            'increment': (frontmatter, prop) => {
                if (override !== 'safe') {
                    console.log(`Incrementing ${prop.key} by ${prop.value} in ${target.basename}`);
                    frontmatter[prop.key] += prop.value;
                }
            },
            'add': (frontmatter, prop, value) => {
                if (!value || (value && override === 'destructive' && value !== prop.value)) {
                    console.log(`Assigning ${prop.value} to ${prop.key} in ${target.basename}`);
                    frontmatter[prop.key] = prop.value;
                } else if (value && Array.isArray(value) && prop.value != null) {
                    arrayPush(target, frontmatter, value, prop);
                }
            },
            'update': (frontmatter, prop, value) => {
                if ((override !== 'safe') && (value !== prop.value)) {
                    console.log(`Updating ${prop.key} to ${prop.value} in ${target.basename}`);
                    frontmatter[prop.key] = prop.value;
                }
            }
        };

        // Callback function to handle frontmatter
        const frontmatterCallback = (frontmatter) => {
            properties.forEach((prop) => {
                const value = fileCache?.[prop.key];

                // Check if property exists, if not, add it

                if (!fileCache || !fileCache.hasOwnProperty(prop.key)) {
                    console.log(`${target.basename} doesn't contain ${prop.key}`);
                    console.log(`Adding property: ${prop.key} to ${target.basename} with value: ${prop.value}`);
                    frontmatter[prop.key] = prop.value;

                // If property exists, handle it according to treatment type   

                } else {
                    console.log(`${target.basename} contains property: ${prop.key}`);
                    if (treatments.hasOwnProperty(prop.treatment)) {
                        treatments[prop.treatment](frontmatter, prop, value);
                    } else {
                        // Default treatment
                        treatments['add'](frontmatter, prop, value);
                    }
                }
            });
        };
        // Process frontmatter of the target file
        await app.fileManager.processFrontMatter(target, frontmatterCallback);
    } catch (error) {
        console.error("Error in processing frontmatter: ", error);
    }
}

function arrayPush(target, frontmatter, value, prop) {
    if (Array.isArray(value) && prop.value != null) {
        console.log(`${prop.key} is an array`);
        const valueSet = new Set(value);
        for (const item of prop.value) {
            if (!valueSet.has(item)) {
                console.log(`Adding ${item} to ${prop.key} in ${target.basename}`);
                frontmatter[prop.key].push(item);
            }
        }
    }
}

module.exports = mergeFrontmatter;

Usage

In both scripts, properties can have the following treatments:

  • add: Only adds values to keys without values, and pushes additional values to arrays.
  • update: Replaces the existing value with a new one.
  • increment: Increases the existing value by the new value. (should probably be renamed)
  • delete: Deletes the key / property.
  • custom: Applies any custom code for string manipulation like toUpperCase(), slice(0,4), replace().

If no treatment is specified, the default condition equals add.

There are two override parameters: safe and destructive.

  • safe: Prevents all changes that overwrite existing properties, allowing only for add.
  • destructive: Makes add work like update.

Usage example

<%*
tp.hooks.on_all_templates_executed(async () => {
	const date = tp.date.now("YYYY-MM-DD HH:mm");
	const parentFile = tp.config.active_file;
	// Use tp.config.target_file to target the current file
	const targetFile = tp.config.target_file;
    const parentLink = app.fileManager.generateMarkdownLink(parentFile, tp.file.folder(true));
	const properties = [
        // Adding keys without values
    	{key:'connections', value: null},
        {key:'aliases', value: null},
        // Using add to add to existing values
        {key:'tags', value: ['tag3', 'tag4'], treatment: 'add'},
        // Increase existing value by new value
        {key:'number', value: 2, treatment: 'increment'},
        // Applying custom code
        {key:'status', treatment: 'custom', code: 'toUpperCase()'},
        {key:'too-long', treatment: 'custom', code: 'slice(0,7)'},
        // Add does nothing, if the keys exists with a single value
        // But update always replaces old value with new value
        {key:'date-created', value: date, treatment: 'add'},
        {key:'date-modified', value: date, treatment: 'update'},
        // Passing a variable as a value
        {key:'origin', value: parentLink},
        // Deleting a key
        {key:'toDelete', treatment: 'delete'}
    ];
	await tp.user.merge_frontmatter_function2(targetFile, properties, tp);
});
-%>

Edit: I refactored the scripts so that the target TFile is a parameter you pass to the user script. Initially, I was only using the script to target the current file, so the target was hard-coded as such. But obviously, it’s much more powerful if you can use it in a loop and pass the target file.

3 Likes

First of all. Disclaimer: I’m not a coder, so from where I’m sitting (on the fence), great work! :slight_smile:

I tried out the second, merge_frontmatter_function.

My testing is not comprehensive.

I really like the option of being able to select functions. It’s like plugin settings.


1

If I may make a suggestion (not sure how easy it would be to implement)…

From what I understand about the workings and YAML treatment of Obsidian (largely through the workings/settings of the fantastic community plugin Linter and the chats I’ve had with the developer/maintainer of the plugin – I sent him on more wild goose chases than he had bargained for), some values are Boolean, some are text strings, some are multi-line, some are not…etc. The Properties UI panel sometimes used to alert me to invalid properties such as topic and parent_topic fields being not properly addressed – not anymore, it seems.

So my question is, is it possible to merge the add and update functions – based on whether the keys in question are sporting single-line status fields?
I have a feeling that Obsidian still has some quirks handling these cases and you need to wait with the final solution but we should keep our options open – otherwise the user must be aware that the status properties will NOT be updated unless he adds update. As I managed it using some conditionals in my use case:

const fileContent = await app.vault.read(currentFile);
	const formattedDateTime = await tp.date.now("YYYY-MM-DDTHH:mm");
	const properties = [];

	// Check if dg_upload or status exists in frontmatter
	if (!fileContent.includes('dg-updated:')) {
		// If "dg-updated" doesn't exist, add it
		properties.push(
			{ key: 'dg-created', value: formattedDateTime },
			{ key: 'dg-updated', value: formattedDateTime },
        	{ key: 'dg_upload', treatment: 'update', value: 'VNC done' }
		);
	} else {
		// If "dg-updated" already exists, just update the timestamp
		properties.push(
			{ key: 'dg-updated', value: formattedDateTime },
        	{ key: 'dg_upload', treatment: 'update', value: 'VNC done' }
		);
	}

	// Add or update other properties even if they existed before
	properties.push(
        { key: 'status', treatment: 'update', value: 'dg_uploaded' },
		{ key: 'share', value: true },
		{ key: 'dg-publish', value: true }
	);

	// Call merge_frontmatter with the properties array
	await tp.user.merge_frontmatter(tp.config.target_file, properties, tp);
});

I’ve sent myself on a wild goose chase on this too, actually.
My original YAML keys to do with two types of status were dg_upload_status and status. No matter what I did (a week ago as well, when I wanted to make a processFrontmatter version of my search-and-replace version of one of my key templates), they keys’ values were not updated.
My first thought was it is some kind of Obsidian properties issue again, along the lines of this, meaning that I cannot have properties that has a conflict in names – I even went ahead and changed dg_upload_status to dg_upload everywhere in my vault, when I realized that I need to use update treatment…so finally at least I got it working. But as I said, these two should somehow be merged on keys that can take only one value (if it’s not easy to implement, leave it, just make sure you alert people to the quirks of this).
On date modified, I had to add update treatment as well to make it work. So it works as advertised, is all I’m saying.

There is no ambiguity on Booleans (default add without having to specify anything else works). Didn’t really test tags, as they were solved by you before and I use tags less and less…

2

A minor thing. If the file had no YAML but some content, there was no empty line added after the second instance of ---. In my case I had to add the empty line before my Heading 1 title:

I added this into my Templater js md file:

setTimeout(async () => {
    const currentFile = app.workspace.getActiveFile();
    let fileContent = await app.vault.read(currentFile);

    // Replace if the pattern matches
    fileContent = fileContent.replace(/(---)\n(^#{1,6}\s.*)/gm, (match, p1, p2) => {
        return `${p1}\n\n${p2}`;
    });

    // Modify the file with the updated content
    await app.vault.modify(currentFile, fileContent);
}, 250);
  • This is very rare of course, but I thought I’d mention it.

I hope some of my ramblings made sense. And as I said, I have a hunch some smoothing out still needs to be done under the hood as well before you can finalize the script(s).

Conclusion

The method offered is an upgrade on the formerly introduced methods. I was going to add a GIF here showing my earlier failings using a script from a week ago, but it seemed like too much work.
This definitely seems like an upgrade in functionality on this simpler method, which will not always work as expected, as I mentioned.

P.S.:
I’m going to start updating my templates using this and if in the meantime I see anything, I’ll let you know.

Cheers

1 Like