Thanks for taking the time to look into this. I’m afraid I could only skim the code as I’m nearing bed time and otherwise not too js-savvy. Although yesterday I made the DVJs version of the query too (wasn’t hard as I had 5-6 different versions of much the same thing) and was looking to cook something up with my Dumb and Dumber partner, the bad-hair-day C-Three.5Pio but I bailed out, thinking there must be some other way.
I mean my write-up sounds like a niche case, but it isn’t. Anybody who uses Obsidian for a database will time and again find the need to modify the files in the results of a query – whether to scrap, archive or whatever.
There is not an innate Obsidian way of doing this and it’s up to the users to carve out some use cases for themselves from various third-party plugins.
I actually find the capabitity of the Projects plugin quite handy. You can batch modify the files of your query (based on various criteria) by adding a field (I used Dummy for key and 123 for field value) and with this way you can temporarily mark your files (as in Notepad++ you can mark files, for instance) – I say temporarily, because I don’t need a YAML key after my date created and modified keys.
But I thought I’d write this up so people with zero coding and regex skills might benefit from some answers.
I expect Obsidian to be able to work a bit like the Projects plugin (but better) drawing on its own Databases core function within 6-10 months time, because modifying properties in batch is a necessity. (Not sure what Notion or other software can do.)
Anyway, I’ll look into what you were up to with your code.
Cheers
P.S. The (flat(list(file.tags)) I also nicked from one of your posts, BTW. I couldn’t make it work any other way.
That should work just fine, as I also had those kind of properties in my test setup. So your error is related to something else. Could you show us your version?
I’ve not tested with lists, but that should work just fine I reckon. I’m more curious on how links would be treated, and might need to check that out.
I added some other frontmatter keys and values and was working after I wrote these lines… I mean your script with the test files and frontmatter values.
Then on my own script, the error changed to
Evaluation Error: TypeError: Cannot read properties of undefined (reading 'file')
at eval (eval at <anonymous> (plugin:dataview), <anonymous>:15:28)
at async DataviewJSRenderer.render (plugin:dataview:18670:13)
Script used:
```dataviewjs
const tp = app.plugins.plugins['templater-obsidian'].templater.current_functions_object;
const result = await dv.query(`
TABLE file.inlinks as Backlinks, length(file.inlinks) as Total
FROM "999"
WHERE !econtains(flat(list(file.tags)), "#dg_uploaded")
AND !econtains(flat(list(file.tags)), "#nopublish-formatted🟢")
AND length(file.inlinks) = 1
SORT length(file.inlinks) DESC
`);
if (result.successful) {
const values = result.value.values;
for (const f of values) {
const tFile = await tp.file.find_tfile(f.path);
await app.fileManager.processFrontMatter(tFile, (fm) => {
console.log(fm, Object.keys(fm).length);
if (Object.keys(fm).length === 0) {
return;
} else {
if (fm['tags']) {
fm['tags'] = 'nopublish-formatted🟢';
}
}
});
}
} else {
dv.paragraph("~~~~\n" + result.error + "\n~~~~");
}
```
I don’t want to create but update the tag anyway; do we need to delete first? Or use search and replace? I usually always use search and replace for frontmatter fields as well because Templater frontmatter changes were causing those annoying Merging changes… pop-ups.
Frontmatter of result in DV query done separately (one test file in temp folder 999 used in js script):
---
title: Ad
aliases:
- ad
tags:
- unformatted⚪
share: true
dg-publish: true
dg_upload_status: down
date created: 2022-12-14
date modified: 2023-10-22
---
Was doing some back and forth with chat robot earlier this afternoon, but I think it got sidetracked on the first error message, then I had other things to do.
I tried some 6-8 variations on the script, changing the frontmatter as well, so I don’t really know anymore.
I can hazard a guess that Templater cannot access the tp.file from the DV query?
I think the main problem is that your query returns a list of links, and a second column, instead of a single list of file links. This means that the f.path isn’t valid and it v seems to create some havoc elsewhere.
So you either need to change your query so that it is using LIST on a single level, or you need to change and add another level of looping around the file&frontmatter stuff.
Yes, I was thinking that too. But when hours before I was using other types of queries (which may not have qualified either, granted) and it was those which spewed the other error.
As for the other thing, create vs. update (update is like rename: delete + create, right, says the non-programmer in me), minutes after my post I got thinking I shouldn’t use the tags I’m using; rather I should use status key with one field. Tags have their own advantages and before Properties came along, we could only list tags. I could convert those tags to status fields to more easily manipulate them (without having to resign myself to search&replace jobs), but that would entail rewriting some 10-12 Templater scripts (wouldn’t take long, of course).
I will do that tomorrow (today; in Europe), and will get back to you.
After rereading your script I do see that you’re setting the value of fm["tags"], not changing the list which would be needed to preserve other tags.
With that in mind I think it’s safer to use a dedicated status property to avoud interference with other tags. Then you can change it to your hearts desire and not worry about side effects related to other potential tags.
If you want to keep using the tags property you’ll also need to do the remove and insert operations in order to preserve other tags.
And I don’t really need to rewrite my Templater scripts as they are all search and replace operations.
Only need to do regex replace for whole vault, but I’ve done that like 15 times over the last year…
I converted the tags in question to status fields, then I tried the script, to no avail.
Even the example script using test files in yesterday’s test vault didn’t work.
I tried making a Templater Js script with the DV API, but couldn’t make it work. For that I needed to move the file into my Templater folder. When I reverted to the DVJs version, and moved back the file into the folder where the test files resided, it worked again.
It is some kind of bug with DataView. I tried on Linux, then on Windows, and then I found this thread:
So in any case, the logic sticks and it works, but there are many ifs and inconsistencies.
Your query, your properties, even the file names were the same.
It is some kind of indexing issue (both Linux and Windows environments were freshly indexed after making full-vault changes). Strange thing is, the testing was done in a separate test vault.
I tried a non-tp way (Obsidian API) as well, with similar reading error.
I like to end all things on a good note. In spite of trying and failing to iron out the kinks of the programmatic solution, and still saying it is a neat but not too user-friendly way of handling it, and bearing it mind I had the general user base in mind when I started out making this thread, I’d like to refer back to the GUI solution with the Projects plugin (which seems far more integrated to core Obsidian than the other plugin mentioned by me above).
Unfortunately, it doesn’t have a docs or how-to page, and I don’t mean to go into details, either.
But the usage is simple: you can use it without DataView installed or enabled but the real power is with DataView and even if a red rectangle appears about it being a read-only status, you can create and even update fields in batch safely, with the same Add field button.
Status fields are updated without creating duplicate YAML keys (or properties). A slight beef is that you cannot create new tags if there is one already in your list of tags. It will be updated with the new one given.
I’m wondering, but it’s hard to test and debug when it’s not happening in my vault, that your error is caused not by a bug in dataview but rather by trying to access file details of part of a non-existing file. At least that is what the error message eludes at…
I am using the Obsidian program as interface. In the folder structure, the files are there. Plugins are installed and enabled. DataviewJs example script kindly shared by you (and working twice before) is there, example files with properties used by you are there. I even went out of my way to put in all properties used in if clauses in the script, although even I know how if clauses work: if there is no match, it goes to the next step.
So the file read issue comes from somewhere and after 2-3 hours of trying to find a cause so I can get past this and maybe file away a nice snippet I could use sometime later, I had to leave this behind, being out of ideas now.
Even the trick of moving the file out of the folder and moving it back (as a way to update the index?) didn’t work a second or third time. There is something off.
If you’ve localised the issue to be within a given folder of files and queries, would it be an option to zip that stuff together and post it here, so someone could look at it?
(And I have to ask, you are on the last version of Obsidian and Dataview and so on? )
As for the second question, after doing my bit with the chat robot trying to find the problem, the latter part of the day was spent on GH issues pages of the Metadata Menu, Projects and DataView plugins, and I actually copied out some URL’s where there is talk of issues or stuff not working, etc., but I am simply not cut out for investigating – simply because my main vault rarely needs needs update jobs and I’m not too personally interested in the quirks of databases, etc.
So yes, I even thought about rolling back versions of DataView and Templater as I have latest versions.
I rolled back DataView version for a try – nothing.
I used the script in a Sandbox vault, on four files having the exact same properties in files – same error.
I forgot to mention with Chat-GPT I made various scripts with console logs after each line and checked the Obsidian console for progress. It found the files but cannot update frontmatter.
I am doing this again in Sandbox vault:
Ooppsie… This script worked so well in my test vault, that I’ve now got 923 files having that one property in their frontmatter. So I’m not sure what’s happening at your end, the script works just fine. You’ve tried restarting Obsidian and/or your computer?
Oh, I’ve tried most anything short of converting to a new faith.
What I did not try, though is not running the main vault in the background.
I just came back to the main vault and the dataview query I used yesterday for reciprocal backlinks gave me zero results. So I restarted Obsidian, still gave zero results, then I copied the working script again into the editor and it gave me the correct 28 results.
I consulted GitHub Desktop for a git diff on this query file: nothing, actually it was not among the files being changed.
So there are other anomalies here.
I’ve known about them.
Automatic refresh was turned off in my Dataview settings, well, yeah…
I was just about to launch into a tirade of what my drill is and what I have in mind to remedy the situation when I realized in the Sandbox vault it also didn’t work.
So I have to live with this.
And to remind you: twice your script worked; not anymore.
