#1

Hi,
A behavioral question for graphs:
I am working on my research article vault (84000+) articles for which I refined the article (Markdown file) extraction.
This extraction start form an HTML-file which is queried from the internet.

The article has a title (#-notation), a date and time (##-notation) followed by the article body and keywords (in a ###-heading) with double brackets ([[) and (]] of course).

The global graph takes some time to generate (which is more than normal) - not complaining about that at all.

Suppose I am working on the [[Core banking systems]] theme for which I am selecting ‘dot’ in the graph which automatically creates a Core banking systems.md which is fine. Generating the local graph for this term caught my attention.

It’s now ‘regenerating’ every x-seconds (4-5) for some minutes now…
Seems in the background backlinks are searched and ‘added’ to the graph which causes the ‘regeneration’.

I have the impression that the global graph has this behaviour too.
It starts showing thing rather quick (which is fine for the visual aspect). When zooming out a lot of research articles are ‘around’ the core of the graph (visuals not connected yet). Also there you see ‘regeneration’ by set intervals (4-5 seconds approx.).

I have the impression that after a while this stops but my impression is that the graphing module ‘hangs’.

While writing the articles Obsidian found another 20-ish more backlinks for ‘Core banking systems’, the local graph is still not finished.

Anyone a clue for what is going on?

Edit
Would Obsidian work ‘faster’ if I generate the md.file (Core banking systems) in this case with all the forward links to the files?
P.S. Obsidian still generating the local graph. 1046 backlinks now and growing…

New Edit
I did write a little Python-script to generate the md.file… with between [[]] the filename of the files that contain the search term (in this case “Core banking systems”)…
When opening the file Core banking systems.md the Linked Mentions ‘Quickly’ grow to 1500-ish mentions and than slow down to add backlinks at the rate the Graph refreshes.
When I close the file (now at 2000-ish backlinks) and reopen it again, the ‘Quick’ behavior goes until 2000-ish files and then slows down again.
So it’s clear this is a caching thing.
Perhaps this ‘triggers’ something for the coders?
Important Hopefully the caching can still be used when closing Obsidian and reopening it again?
Otherwise this would be a big blow for people using Obsidian to do research on big vaults (as I am testing right now).

1 Like
#2

Generating the local graph for this term caught my attention.

It’s now ‘regenerating’ every x-seconds (4-5) for some minutes now…
Seems in the background backlinks are searched and ‘added’ to the graph which causes the ‘regeneration’.

I might suggest filing this under #bug-reports

#3

Obsidian is looking for links clearly in other documents. Which exists because the backlinks were created in the research articles.
I kind of “found” an interesting side-track during these investigations:
I created a MOC-Core banking systems.md with links to every document containing the search term “Core banking systems” which mimics the behaviour of clicking the Core banking systems node in the graph because that create the file Core banking systems.md but without the links. Apparently Obsidian than frantically goes and searches for all the mentions of “Core banking systems” in the vault, which in my case is rather ‘huge’.
With the MOC page you get exactly the same graphs without the ‘searching’ anymore - more correctly the search only returns the MOC-page as backlink when you are in a research article.
This opens some “new” approaches for me in my search to try and find good ways to tackle SuperVaults with very large amount of research articles.
I’m still more than convinced Obsidian has it’s place in this area too.

1 Like
#4

The graph seems to use the same ‘methods’ as the one generating the links in the mentions. After a certain ‘refresh’ period has passed, the graph detects that some other nodes need to be added to the graph and regenerates it.
You can clearly see this happening when making the global graph for the 80K+ articles. The ‘cloud’ grows until all the article nodes are added to the graph. During that generation you can clearly see the forward and backward links forming ‘subclouds’.

#5

What exactly is the issue here? That the local graph is taking too long to render?

#6

@sam.baron
Sorry for the late reply. Again some busy ‘non-obsidian’ days behind me.
The graph indeed seems to need some times when links are used in there original form.
With that I mean that graphs does not show this behavior when generating MOC’s (read: new files containing explicit links to other files). This way you ‘circumvent’ the normal backlinking Obsidian normally uses.
Sorry for the ‘fuzzy’ way of describing things. This concerns a personal research project which I cannot disclose. Always willing to share some more info via other more secure and private channels.

#7

No problem, unfortunately, I’m not following the problem. If you are open to sharing more information, you may want to post on Discord in the supporter-vip-lounge channel where one of the devs should respond.

1 Like
#8

Can’t seem to find that lounge :frowning:

#9

It’s in the Discord community, not this forum. https://discord.gg/veuWUTm

#10

Indeed, I was looking in Discord… Not here :wink: