Cache Dataview results for less intensive queries?

I’m using Obsidian to plan a long fiction series (which, me being me, has quite a lot of detail in several parts of the background), and have written a number of rather complex DataviewJS queries to populate overview/index pages with what’s going on in any given time period – for example, one goes through every character and figures out their birthday/year (by first checking if they have a date specified on their own page, and if they don’t then checking all of the lists of “it’s some time this year, I just don’t know the month+day”), determines whether they were born after/died before the period in question, calculates what age they’re turning, and finally prints all that in a table interspersed with every one of my scenes happening around the same time so I can tell at a glance if I need to note in the chapter that someone’s birthday is/has/will be happening.

Everything works from a final-data perspective, but when I open those index pages which include five or six different but similarly complex queries, it takes a noticeable second for all the data to be processed and loaded, while in the mean time the reading view is unresponsive. I’m sure there’s a good amount of logic optimization I can do from this point and I’m going to try to learn Javascript asynchronous handling as two ways to speed it up on my side, but I was also wondering if there was any built-in request caching I could leverage? I really don’t need to have things be up-to-date every time I load the overview. If not, I’m already planning what I can do myself with dv.io.csv or similar, but I figured I’d make sure there wasn’t something I’m overlooking before embarking on another weekend of programming.

2 Likes

A couple of simple things right off the bat:

  1. You should paste the code so people can tell if you there are some immediate places to optimise it.
  2. If your code has figured out the birthdate of a character, it should be storing that in the character note (or in a general “Birthdays” note) on the first runthrough. Then on subsequent runthroughs it should just fetch the saved birthday. The way you’ve written in your post sounds like you’re figuring them all out from scratch each time you open your notes.

I can find a lot of the optimizations well enough on my own, I’ve just not gotten around to it yet. Thanks though!

For the second, I had been using the yearly notes since they allow me to both put it into a task list (which I can then pull to another file tracking what I still need to do) and give me a machine-readable earliest/latest date, but I suppose with some of the later tweaks I’ve made to the templates/scripts I now have the ability to input the data as *-within-range pairs on the file itself and still have it show up everywhere I need it to. I’ll be doing that move manually, but is there actually a syntax for editing the frontmatter like you suggest? I know it’s possible with Templater, but it’s a bit late for basing an entire solution around that plugin.

Still, that’s an answer about optimal code, rather than about caches/reusing data.

If you don’t want to be reliant on any plugins, here’s my code to update frontmatter fields:

function updateFrontmatter(contents, field, value) {
    const f = contents.match(/^---\r?\n(.*?)\n---\r?\n(.*)$/s),
        v = `${field}: ${value}`,
        x = new RegExp(`^${field}:.*$`, 'm'),
        [s, e] = f ? [`${f[1]}\n`, f[2]] : ['', contents]
    return f && f[1].match(x) ? contents.replace(x, v) : `---\n${s}${v}\n---\n${e}`
}

If the frontmatter field doesn’t exist it will add it, otherwise it will update the existing field.

Just read the contents of your file, update the field or fields that you want, and save the file again:

let contents = await app.vault.read(file) // read file
contents = updateFrontmatter(contents, 'field1', 'value1')
contents = updateFrontmatter(contents, 'field2', 'value2')
app.vault.modify(file, contents) // save file

Ah, I didn’t realize that contents would still be linked to the physical file. That will do nicely, thanks!
Wait, now I see the app.vault.modify call. Still, big thanks even if I misread the solution originally!

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.