Query list items that match a condition (with grouping) plus their child elements (that works with Digital Garden)

trezor · May 25, 2025, 9:56pm

Based on this (I can’t post links, sorry):

Dataview query to extract a list item and all its sublist items
Include children (sub-list items) in query for list times with a tag
Retrieve list of bullet points and their sub-sub-items

And this:

icewind.quest/characters/arcane-brotherhood/vellynne-harpell/

What I’m trying to do

I’m interested in:

quering all notes and their lists in a specific folder
pulling the lines that contain this.file.link
pulling their children
grouping results by item.link (/folder/file_name/#slosest_heading)

The goal is basically the same as icewind.quest’s character pages - aggregating and displaying all encounters with said character on a single page.

However, icewind.quest’s solution requires marking every line with these kinds of contraptions (x:: ), which is a nightmare for me to compose. Also, it only pulls in single paragraphs, and it’s not really readable - you don’t always want to keep everything in a single slab of text.

So, for myself I’ve decided that it would be best to try to format it as a list.

Things I have tried

I want to apologize in advance for the horrors that you may encounter below in the code. I don’t really know what I’m doing - blindly poking around, trying to make something that works. It’s too much, and I’m asking for help.

Dataview LIST with grouping

First, I’ve cobbled together this:

LIST
	regexreplace(
		replace(
			rows.item.text,
			"[[" + this.file.name + "]]", 
			this.file.name
		),
		"\[\[" + this.file.name + "\|(.+?)\]\]", 
		"$1"
	)
	FROM "Journal"
	FLATTEN file.lists AS item
	WHERE contains(item.text, this.file.name)
	SORT file.name desc
	GROUP BY item.link

The regex part cleans up the redundant links that loop back to this.file.name while trying to keep link aliases where they’re present - pay no attention to that part.

It produces a readable result that I sort of like, which (most importantly) works with Digital Garden.
It looks something like this (in the browser it renders as a clean bullet list with tags and links):

- [[03. After the Siege > Evening at the manor]]: (this is the item.link, which links to the session_file#closest_heading)
-- [[NPC1]] tried to steal our cakes and hotdogs
- [[03. After the Siege > Morning]]:
-- #Quest01 [[NPC1]] met us at the Tavern
-- [[NPC1]] sought us out again and spoke of [[NPC3]] and an [[Important artifact]] once more.
- [[04. City at night > At the tavern]]:
-- He said that [[NPC1]] stole his cakes as well
- [[04. City at night > Shaded streets]]:
-- #Quest02 [[NPC1|The same guy]] tried to follow us in the dark
-- #Quest02 [[NPC1|The cake and hotdog bandit]] suddenly appeared before us, holding a new batch of cakes
-- [[NPC1]] got scared and ran off, dropping the cakes along the way

The good

It works in Digital Garden flawlessly - nothing is broken
There is grouping by file/heading - which is very convenient - you can easily jump to sections that contain these matches

The bad

It returns only direct matches - there are no child elements which makes this approach mostly useless for me (while you can hover over the heading link on desktop and preview it - to see if there’s something more, on mobile you cannot)
The files are in the right order, but the headings are reversed (Evening before Moning, for example) - tinkering with SORT did nothing

Modified holroy’s function

This is the closest I’ve got to the result I want.
I’ve tried to modify the holroy’s code to work for my case (see link above).

const dvF = dv.evaluationContext.functions

// Create a new DQL function
dvF.listChildren =  (ctx, children, offset = "" ) => {
  let result = ""
  for (const child of children) {
    result += offset + "- " +
       child.text + "\n"
    
    if (child.children.length)
      result += dvF.listChildren(ctx, child.children, offset + "  ") 
  }
  return result
}


const result = await dv.query(`
	LIST WITHOUT ID 
	item.link + "\n" + item.text + "\n" + listChildren(item.children)
	FROM "Journal"
	FLATTEN file.lists as item
	WHERE !item.parent AND contains(item.text, this.file.name)
`)

if ( !result.successful ) {
    dv.paragraph("~~~~\n" + result.error + "\n~~~~")
    return
}

dv.list(result.value.values)

// Cleanup after our new function
delete dvF.listChildren

The good

It returns child elements!

The bad

There is no grouping

Things that should look like this:

- [[file_heading1]]
-- match1
-- match2

Look like this:

- [[file_heading1]]\n match1
- [[file_heading1]]\n match2

It doesn’t handle cases such as this:

- [[file_heading]]
-- NOT a match
--- NOT a match
---- match

Removing the WHERE !item.parent handles these, but produces unwanted results.

And the worst thing

When publishing to Digital Garden, all links contained in lines produced by listChildren(item.children) are broken (data-href can say “Tavern” while href will say “404”). Links that are produced by LIST item.link are fine. Inside Obsidian itself, all links are working.

I’ve also tried AI, but it’s a total mess - I won’t be posting that here…

Is it too complex? Should I stick to basic list?

trezor · May 28, 2025, 3:58pm

Disclaimer

The code below was NOT looked at or endorsed by any person that knows what he’s doing! Use at your own risk.

What it does

aggregates all list items (single lines) that have a match (it can be current file name or a tag, for example)
gets child elements (sub-lists)
gets parent elements (if a match was not on the top level, but somwhere inside)
groups them by item.link

Code

let scope = '"Journal/Sessions"';
let term = dv.current().file.name;
// let term = "#Quest01";

const dvF = dv.evaluationContext.functions;

// Pass 1: Check if this item or any child contains the term
dvF.branchMatches = (item, term) => {
  if (item.text.includes(term)) return true;
  return (item.children || []).some(child => dvF.branchMatches(child, term));
};

// Pass 2: Render everything under this item (regardless of term match)
dvF.renderEntireBranch = (item, offset = "") => {
  let result = `${offset}- ${item.text}\n`;
  for (const child of item.children || []) {
    result += dvF.renderEntireBranch(child, offset + "  ");
  }
  return result;
};

// Gather list items from target scope
const pages = dv.pages(scope)
  .flatMap(p => (p.file?.lists || []).map(item => ({ item, file: p.file })));

// Start only from top-level items
const grouped = {};
for (const { item } of pages) {
  if (item.parent) continue;

  // If any match exists in this top-level item’s tree
  if (dvF.branchMatches(item, term)) {
    const link = item.link;
    const rendered = dvF.renderEntireBranch(item);

    if (!grouped[link]) grouped[link] = [];
    grouped[link].push(rendered);
  }
}

// Build and display output
let output = "";
for (const [link, blocks] of Object.entries(grouped)) {
  output += `\n${link}\n`;
  for (const block of blocks) {
    output += block;
  }
}

dv.paragraph(output);

// Cleanup
delete dvF.renderEntireBranch;
delete dvF.branchMatches;

Output

It produces something like this:

01. File Name > Heading1
- MATCH
-- nested line
-- nested line
- NOT a match
-- nested line
--- MATCH
---- nested line
01. File Name > Heading5
- NOT a match
-- MATCH
--- nested line

The File Name > Heading part is clickable - it jumps you to the file in question and the closest heading / section in it - which is very handy.
Also all links and tags that were in the source notes are preserved and work in Obsidian by default (they do not if you want to publish to Digital Garden - for that see below).

Digital Garden problems

For links to work

For links to resolve properly you have to set ‘New link format’ setting in Obsidian to ‘Absolute path in vault’. This, sadly, does not update all existing links automatically - it only applies to newly created ones, so you’ll have to update them manually. So what looked before as [[Tavern|that place]] should now look like [[Places/Tavern|that place]].

Tag duplication

When publishing, tags inside text will render, and will be usable, but have some sort of a rogue <a> element before them, that has the same id as the tag after it but that leads to nowhere - it looks like a small clickable tag with no text in it. I don’t know why that happens. Regular dataview queries produce no such thing - tags there look as they should. Other than that, it kinda works.

trezor · May 28, 2025, 8:14pm

Redundant link and tag clean-up

This is an alternate version of the code above.
Difference is - it cleans up links (or tags) that match the search term for increased readability (also, you don’t really want loopback links that lead to the same page).
For links - it turns them into plain text (or leaves only alias - alt. text - if present), for tags - it removes them completely.

Disclaimer (again)

The code below was NOT looked at or endorsed by any person that knows what they’re doing! Use at your own risk.

let scope = '"Journal/Sessions"';
let term = dv.current().file.name;
// let term = "#Quest01";

const dvF = dv.evaluationContext.functions;

// Clean up an item's text by removing or simplifying links and tags
const isTag = term.startsWith("#");
function cleanText(text) {
	// Remove tag if term is a tag
	if (isTag) {
		return text.replaceAll(term, "").trim();
	}
	// Otherwise, process potential links that match the term
	return text.replace(/\[\[([^\|\]]+)(\|([^\]]+))?\]\]/g, (match, fullLink, _, alias) => {
		const baseName = fullLink.split("/").pop().split("#")[0];
		if (baseName === term) {
			  return "**" + alias + "**" || "**" + baseName + "**";  // remove brackets or show alias
		}
		return match; // leave other links untouched
	});
}

// Check if this item or any child contains the term
dvF.branchMatches = (item, term) => {
  if (item.text.includes(term)) return true;
  return (item.children || []).some(child => dvF.branchMatches(child, term));
};

// Render everything under this item
dvF.renderEntireBranch = (item, offset = "") => {
  let line = cleanText(item.text, term);
  let result = `${offset}- ${cleanText(item.text)}\n`;
  for (const child of item.children || []) {
    result += dvF.renderEntireBranch(child, offset + "  ");
  }
  return result;
};

// Gather and filter data
const pages = dv.pages(scope)
  .flatMap(p => (p.file?.lists || []).map(item => ({ item, file: p.file })));

const grouped = {};
for (const { item } of pages) {
  if (item.parent) continue;
  if (dvF.branchMatches(item, term)) {
    const link = item.link;
    const rendered = dvF.renderEntireBranch(item);
    if (!grouped[link]) grouped[link] = [];
    grouped[link].push(rendered);
  }
}

// --- Render output ---
let output = "";
for (const [link, blocks] of Object.entries(grouped)) {
  output += `\n${link}\n`;
  for (const block of blocks) {
    output += block;
  }
}

dv.paragraph(output);

// Cleanup
delete dvF.cleanText;
delete dvF.branchMatches;
delete dvF.renderEntireBranch;