Dataview JS Getting The Tree Root

[BorisZ]

I want to implement a recursive tree with a Dataview JS. My files have the top-down approach, so the file itself declare its children (without parent declaration):

# File: A
type:: [[Directive]]
directive:: [[B]]
directive:: [[C]]

# File: B
type:: [[Directive]]

# File: C
type:: [[Directive]]

In a tree representation:

- A
  - B
  - C

What I want is to retrieve the root (here, [[A]]). Why [[A]] is a root? Because no directives have [[A]] as a child. For example, [[B]] is not a root since [[A]] define it as a child.

The first idea was to check file’s ingoing links to determine whether it’s a root since no ingoing links means no child definitions.

However, this does not consider whether the ingoing link is a directive field:

# File: A
directive:: [[B]]

^^^ I **do** want it to be considered as a B"s parent since it is in a `directive` field

# File: B
blah blah [[C]] blah

^^^ I **don't** want it to be considered as a C"s parent.

To sum up:

• I want to retrieve directive’s parents
• Parents are those that have an explicit children definition (directive:: [[B]])

[I-d-as]

Great idea! I have recently been thinking about how useful it would be to have this type of outline produced automatically. There are some benefits to managing it manually like the ability to fold, but I would certainly use the automated version regularly if it were available. It would be most useful if the items in the tree diagram were links.

I was wondering how the tree would show all the top level notes that are not children of other notes. I would assume they would just be shown as equally top level on the tree diagram, or perhaps a default top level node could be automatically added for clarity. Also, I was wondering whether it would be okay, based on the logic of your system, for a note to be the child of multiple notes. I have some other questions, but mainly want to express interest and excitement for a solution if someone is able to create one.

As a side note, if you aren’t already using the Excalibrain plugin, I would recommend checking it out. Of course, there is also the Breadcrumbs plugin.

Thanks!

[BorisZ]

There are some benefits to managing it manually like the ability to fold

Agree, that’s why I still think about manually choosing roots (or tagging them to select later). Unfoldable lists would become too cumbersome.

It would be most useful if the items in the tree diagram were links

Also agree. I think that Obsidian is missing types of relations. Just relations are often not enough.

Here, I like RemNote’s approach. If you add a field to a note, it just creates an under the hood note with a key-value pair, so this approach:

# File A
type:: [[Directive]]

Becomes this in RemNote:

# File A
[[[[type]]]-[[Directive]]]     <<<<<< Link to a note which title is a key-value pair

That, in terms of graphs, is the approach. However, if RemNote does it under the hood, with Obsidian it would take too much effort.

I was wondering how the tree would show all the top level notes that are not children of other notes. I would assume they would just be shown as equally top level on the tree diagram, or perhaps a default top level node could be automatically added for clarity.

A matter of taste, top-level list items or second-level under the single top-level entry point.

I was wondering whether it would be okay, based on the logic of your system, for a note to be the child of multiple notes.

Yes, that’s why I use top-down approach here, so for this structure:

# File A
directive:: [[C]]

# File B
directive:: [[C]]

I expect the following output:

- A
  - C
- B
  - C

As a side note, if you aren’t already using the Excalibrain plugin, I would recommend checking it out. Of course, there is also the Breadcrumbs plugin.

I’m still considering the Breadcrumbs plugin, but the main idea is that I need strict children order. I don’t know whether these plugins support this. Gotta revisit their features.

[BorisZ]

So far I’ve come up with my custom drop-in solution:

const rootEl = dv.el("ul", "")

dv
  .array(
    input.rootNames.map(it => dv.page(it))
  )
  .forEach(it => renderNode(it, rootEl))

function renderNode(node, container) {
  const liText = node?.path ?? node.file?.link
  const liEl = dv.el("li", liText, { container })
  const ulEl = dv.el("ul", "", { container: liEl })
  dv.array(node[input.childrenKey])
    ?.map(it => dv.page(it))
    .filter(it => !!it)
    .forEach(it => renderNode(it, ulEl))
}

This is a custom view meaning that you have to create a separate .js file and then connect it where needed:

await dv.view("dataviews/treeview", {
  childrenKey: 'child',
  rootNames: [
    "File A",
    "File B",
    "File C"
  ]
});

• Here I decided to handpick roots (see the comment above about “cumbersomeness”)
• It is agnostic meaning that it expects those root names AND children key

Below are inputs and outputs of this view:

• Input (File Structure)

# File A
child:: [[C]]

# File B
child:: [[C]]

# File C

• Output (Generated List)
• A
  • C
• B
  • C

---

What to improve:

• It assumes that you don’t have circulars. It is sufficient for me, but be careful since it can end up with an infinite loop (and crash Obsidian). Add a depth variable if needed.
• It works with file names (A), not with file links ([[A]]). I’m too lazy to figure out how to process links.

[I-d-as]

Wow! That’s impressive. Thanks for sharing.

And maybe someone else will be able to adjust it so that it shows links. I look forward to trying this out.

Nice work.

[BorisZ]

Optional addition:

You can format the output whatever you like. Since I prefer visual recognition, I use icons. Moreover, some optional description can be provided:

// Before:
const liEl = dv.el("li", liText, { container })

// After:
const icon = node?.icon ?? ''
const link = node?.path ?? node?.file?.link
const description = node?.description ?? ''
const liEl = dv.el("li", icon + link + description, { container })

You can even create pass an input function to make it more agnostic:

await dv.view("dataviews/treeview", {
  childrenKey: 'child',
  rootNames: [
    "File A",
    "File B",
    "File C"
  ],
  format: (node) => {
    const icon = node?.icon ?? ''
    const link = node?.path ?? node?.file?.link
    const description = node?.description ?? ''
    return icon + link + description
  }
});

And then:

// Before:
const liEl = dv.el("li", format(node), { container })

[holroy]

@BorisZ

To sum up:

• I want to retrieve directive’s parents
• Parents are those that have an explicit children definition (directive:: [[B]])

So, I started out thinking about how to do this, and then I wanted to a visualise the tree I wanted to use for testing, and I started thinking about mermaid, and how it would show stuff, so I made this model:

```mermaid
flowchart LR

A[A] --> B[B]
A[A] --> C[C]
B[B] --> F[F]
C[C] --> D[D]
D[D] --> E[E]
D[D] --> F[F]

I[I] --> J[J] --> K[K] --> I[I]

class A,B,C,D,E,F,I,J,K,InternalLinks internal-link;
```

Which displays as:

Does this look interesting, and would it be a better solution to actually build the mermaid chart for connecting your nodes?

If so, then the question becomes how would we build this chart within some query in Obsidian. I could possibly see a way using Templater in combination with dataviewjs to insert a static variant of the graph from the point you inserted the template. It would need to be refreshed/re-inserted every now and then, to stay up to date.

A true dynamic variant, I’m not so sure on how to build, as that would require access to the mermaid API. I think it’s doable, but I don’t now (just now) how to access the mermaid API from within a dataviewjs query.

But first of all, would it be interesting to build graphs like that above instead of just a list? Or do you want the list for some other reason?

[I-d-as]

Very cool! This would be incredible!

Another cool solution would be to generate the hierarchy in an actual ordered list of links, thus allowing folding. Using mermaid is a really cool solution in that it can simply point to the same node from multiple branches as opposed to using duplicates. However, if I had to choose, I would opt for an ordered list that gets generated (and regularly manually refreshed) via the magic of Templater and Dataviewjs, like you mentioned. But, then again, the mermaid solution would be pretty amazing in certain situations. I really can’t decide.

Thank you so much for the time you put into solving these problems, helping people, and furthering the possibilities of these amazing tools.

[holroy]

@I-d-as

Very cool! This would be incredible! …

Using mermaid is a really cool solution in that it can simply point to the same node from multiple branches as opposed to using duplicates. …

But, then again, the mermaid solution would be pretty amazing in certain situations. I really can’t decide. …

Thank you so much for the time you put into solving these problems…

I think I’ve mentioned it once (or twice) before, but one of the things which drives me when solving stuff, is personal interest. And whilst doing it the “normal” way is interesting enough, doing it using a mermaid graph, just peaked my interest. A lot.

So no matter what you feel , the first solution to this request is the mermaid variant. It works for me, and I’m hoping it’ll work for you, as well. I built the data set matching my example in the previous reply, and lo and behold, here is the script actually doing this!

Script to build the mermaid graph

function toLetters(num) {
 let mod = num % 26,
     pow = num / 26 | 0,
     out = mod ? String.fromCharCode(64 + mod) : (--pow, 'Z')
  return pow ? toLetters(pow) + out : out;
}


const result = await dv.query(`
  LIST directive
  WHERE type AND type = [[Directive]]
  SORT file.name
`)
console.log(result)

if (result.successful) {
  let linkCounter = 0
  let linkSet = {}
  const nodes = result.value.values
  const mFront = "```mermaid\ngraph LR\n"
  let mNodes = [] 
  for (let node of nodes) {
    let leftIdx, leftLink, rightIdx, rightLink

    if ( !node.value?.length)
      node.value = [node.value]
      
    for (let connection of node.value) {
      if ( !connection ) 
        continue; // Bail out if it doesn't link anywhere
        
      leftLink = node.key.path.match(/\/([^\/]+)\.md/)[1]
      rightLink = connection.path.match(/\/([^\/]+)\.md/)[1]
      
      if ( leftLink in linkSet )
        leftIdx = linkSet[leftLink]
      else {
        linkCounter += 1
        leftIdx = toLetters( linkCounter )
        linkSet[leftLink] = leftIdx
      }
    
	  if ( rightLink && rightLink in linkSet )
        rightIdx = linkSet[rightLink]
      else {
        linkCounter += 1
        rightIdx = toLetters( linkCounter )
        linkSet[rightLink] = rightIdx
      }
      
      mNodes.push(` ${ leftIdx }[${ leftLink }] --> ${ rightIdx }[${ rightLink }]`)
    } 
  }
  let classList = []
  for (let i = 1; i<= linkCounter; i++) {
    classList.push( toLetters(i) )
  }
  const mEnd = "\nclass " + classList.join(",") + " internal-link\n```\n"
  
  dv.span(mFront + mNodes.join("\n") + mEnd)
} else 
  dv.paragraph("~~~~\n" + result.error + "\n~~~~")

Explanation of script

This is going to be a cursory explanation. It’s getting late, and it’s kind of complex script, so if you don’t know javascript at all, it’s hard to explain it…

• The first function toLetters is a helper function, which converts number to a letter combination, in the sequence: A, B, C, … Y, Z, AA, AB, … These are later on used to name the nodes internally in the graph, and to build a classList ... internal-link at the end
• The query is pretty basic, but it finds all files having the type of [[Directive]], and lists the corresponding directive field of that file. If this field is not a link, things will go astray…
• A random console.log(result) just to check the result of the query. This line could be removed, but I forgot
• Given a successful result, lets do the magic

  • Reset linkCounter and linkSet. The linkSet will hold all links found in the query, and return the corresponding name when using the link as key into the set

  • Fetch the actual nodes from the query into nodes

  • Define mFront which is the front of the mermaid block

  • Define mNode as an empty array to hold all of the nodes to come

  • Loop through each of the files (or nodes) of the result, and do:

    • Define some helper variables, and if the node.value (aka the directive being singular) is not a list, then make it into a list of one element, just to ease the logic later on
    • Now loop through each of the connections (aka the directive fields) of a given file to create the mermaid link:
      • First of all, if there is no connection (aka directive field), then bail out, as there is nothing to link to
      • Pull out the unique(!) name from the file name and the connection into leftLink and rightLink
      • For each of these, check if it exists in the linkSet. If it does exist use the letter from the linkSet, and if it doesn’t exists get a new letter, and insert the new link into the linkSet. Store the letter into either leftIdx or rightIdx
      • Push the new link connection into mNodes, using the A[note] syntax for both sides. The note is used for display, whilst the A is used by mermaid to keep track of the nodes internally

  • Back out after the for loop, we need to retrack how many nodes we’ve created and add them into a classList, and we also need to end the mermaid script. All this is stored into mEnd. Part of what makes this tick is the concept of doing the class A,B,C,...,K internal-link line. This enables the mermaid engine to actually turn our nodes into the links.

  • Final action is simply to output the mermaid script using dv.span() and let the magic unfold

• If however the result of the query failed, print a suitable error message

Below is an attached zip file to a folder containing all the needed files to showcase this script:
f54538.zip (3.4 KB)

There is one caveat with the script as of now, and that is if your files don’t have a unique name, it’s kind of random which file you’re getting. I haven’t found a way to add aliases to mermaid links.

Update: Credit to Hassle free linking in mermaid graphs - #9 by WhiteNoise for giving me the idea to use mermaid for this

[I-d-as]

Congratulations on this success! I am about to try it out now. Thanks for sharing!

In terms of aliases for mermaid links, I actually made this request a while back: Mermaid Links with Use of Aliases or Custom Link Previews

My apologies if it came off as if you needed any suggestion of which route to go. I only shared my interest in the list option, in terms of the folding capabilities, in case it might trigger some inspiration as to how such functionality might be accomplished.

I admire your curiosity, and surely appreciate the fruits of your dedication.

---

Edit: Wow! Just tested it and, of course, it works!

[holroy]

@I-d-as

My apologies if it came off as if you needed any suggestion of which route to go.

HeHe… I’m not offended, I just tried to explain that in some cases my curiosity takes overhand, and no matter what is asked for I got to give in to my urges, so to speak! It’s part of my fuel to keep answering.

[holroy]

I found a bug in my script, related to files residing in the root folder, so the corresponding section needs to be fixed:

      leftLink = node.key.path.match(/\/?([^\/]+)\.md/)[1]
      rightLink = connection.path.match(/\/?([^\/]+)\.md/)[1]

In addition, I forgot to mention, that as long as the start query generates a list of link list, the script is happy to try to make it into a graph. So I totally didn’t do the following:

const result = await dv.query(`
  LIST file.inlinks
  WHERE file.inlinks
  SORT file.name
  LIMIT 100
`)

And I didn’t get this output, which I had to zoom out a tiny bit, to only get the upper third visible on my monitor… No siree, I didn’t do it. Nope.

[I-d-as]

@BorisZ Hello again. Thanks for creating this thread. It has been so cool to learn all of this. I will continue trying to troubleshoot, but I figured I might as well ask, in case it is something simple that you could help me see.

I have been trying to get the setup from your original solution to work: Dataview JS Getting The Tree Root - #4 by BorisZ

I would receive the following results:

I figured it was because at one point the note is called File C and at another point is shown as simple C. So, I adjusted the setup to instead be:

# File A
child:: [[File C]]

# File B
child:: [[File C]]

# File C

After making this adjustment, I received these results:

Out of curiosity, I changed the setup to:

# File A
child:: [[File C]]

# File B
child:: [[File C]]

# File C
child:: [[File D]], [[File E]]

With these adjustments, I received the following results.

Do you see anything that might be causing the issue? Thanks in advance!

[I-d-as]

Great! Thanks!

Just to be clear. The only lines that need to be replaced are the two where leftlink and rightlink are defined?

I am not 100% clear on what occurred in order to produce the gargantuan graph you attached, but I am guessing that you temporarily altered the code so as to essentially graph your entire vault. My mistake, that was actually what you didn’t do. Thanks for the insight. Very useful. Of course, I didn’t take the bait. And, I absolutely didn’t have my computer asking for a time out.

[holroy]

@I-d-as

Do you see anything that might be causing the issue? Thanks in advance!

I’m not Boris, but I saw some issues:

• In the latter dv.array, the .filter(it => it) should come before the .map, so as to eliminate the leafs from trying to render an undefined element
• And it does indeed cause an endless loop if you run it within a folder like my setup. Ooppsie…

Here is a non dv.view() version which does work for my evil data setup with loops:

```dataviewjs
const childrenKey= 'directive',
  rootNames= [
    "A", "I"
  ]

const rootEl = dv.el("ul", "")

let renderedNodes = {}

dv.array( rootNames.map(it => dv.page(it)) )
  .forEach(it => renderNode(it, rootEl))

function renderNode(node, container) {
  const liText = node?.path ?? node.file?.link
  const alreadyRendered = liText in renderedNodes
     
  renderedNodes[liText] = 1
  
  let liEl = dv.el("li", liText + (alreadyRendered ? " ➰" : "") , { container })

  if ( alreadyRendered ) 
    return
    
  const ulEl = dv.el("ul", "", { container: liEl })

  dv.array(node[childrenKey])
    .filter(it => it)
    .map(it => dv.page(it))
    .forEach(it => renderNode(it, ulEl))
}
```

It produces this output:

It’s a simplistic handling of the loop condition, as it outputs the node again with an added thingy, a unicode loop character: .

The code still doesn’t detect the parent nodes by itself, so it needs a little start help. And it also showcases one feature with dataview, and that while this code is dynamic and will update if you change relationships, it doesn’t allow for collapsing the lists. Not sure, besides using the templater trickery, how to get it to be collapsable.

(Oops: Time is running when having fun… Local time: 03:19 am)

[holroy]

@I-d-as

Just to be clear. The only lines that need to be replaced are the two where leftlink and rightlink are defined?

Yes, that is correct.

And No, I didn’t do what you (not?) suggested. Not at all…

Disclaimer: Any picking up of bait, is done on your own risk.

[I-d-as]

Unstoppable! I’ll always remember the afternoon/night that @holroy casually created a new graph view, among many other things.

[BorisZ]

I already see @holroy’s great solution, but here are some more additional thoughts:

• more optional chaining should be used since:
  • child may not be present at all
  • child’s link (value) is just a link thus the file may not be created yet (aka ghost link)
• about child:: [[File D]], [[File E]]: I don’t quite know how Dataview process comma separated values, I personally prefer defining values in separate fields:

child:: [[File D]]
child:: [[File E]]

Also I figured out few more pitfalls:

• Something besides links doesn’t work now (i.e. child:: just a text). Can be easily fixed, however, it’s sufficient for my needs.
• Links with some additions will also not work (child:: [[File D]] qwerty). Better solution for me is to have additional [[File D]] field (description:: qwerty). Then just display it as shown here. It is more “structured” approach in my opinion.

[BorisZ]

@holroy

Does this look interesting, and would it be a better solution to actually build the mermaid chart for connecting your nodes?

What I want to implement is a step-by-step checklist for my life/work/etc. That’s why I need strict children order (and that’s why I prefer top-down over down-top linking).

However, your solution definitely gives me insights and ideas how to improve my current solution. Especially your approach to avoid infinite loops. Many thanks!

[holroy]

@BorisZ

What I want to implement is a step-by-step checklist for my life/work/etc. That’s why I need strict children order (and that’s why I prefer top-down over down-top linking).

What do you mean with “strict children order”? Just the simple list variation? The order does show in the mermaid graph as arrows, and I do believe it can be further enhanced.

However, the mermaid solution can’t add that much additional text of any sorts if that’s what you’re looking for.

And what do you mean when saying “top-down” is preferred over “down-top”? That doesn’t really make sense me just now.