Hi,
A behavioral question for graphs:
I am working on my research article vault (84000+) articles for which I refined the article (Markdown file) extraction.
This extraction start form an HTML-file which is queried from the internet.
The article has a title (#-notation), a date and time (##-notation) followed by the article body and keywords (in a ###-heading) with double brackets ([[) and (]] of course).
The global graph takes some time to generate (which is more than normal) - not complaining about that at all.
Suppose I am working on the [[Core banking systems]] theme for which I am selecting âdotâ in the graph which automatically creates a Core banking systems.md which is fine. Generating the local graph for this term caught my attention.
Itâs now âregeneratingâ every x-seconds (4-5) for some minutes nowâŚ
Seems in the background backlinks are searched and âaddedâ to the graph which causes the âregenerationâ.
I have the impression that the global graph has this behaviour too.
It starts showing thing rather quick (which is fine for the visual aspect). When zooming out a lot of research articles are âaroundâ the core of the graph (visuals not connected yet). Also there you see âregenerationâ by set intervals (4-5 seconds approx.).
I have the impression that after a while this stops but my impression is that the graphing module âhangsâ.
While writing the articles Obsidian found another 20-ish more backlinks for âCore banking systemsâ, the local graph is still not finished.
Anyone a clue for what is going on?
Edit
Would Obsidian work âfasterâ if I generate the md.file (Core banking systems) in this case with all the forward links to the files?
P.S. Obsidian still generating the local graph. 1046 backlinks now and growingâŚ
New Edit
I did write a little Python-script to generate the md.file⌠with between [[]] the filename of the files that contain the search term (in this case âCore banking systemsâ)âŚ
When opening the file Core banking systems.md the Linked Mentions âQuicklyâ grow to 1500-ish mentions and than slow down to add backlinks at the rate the Graph refreshes.
When I close the file (now at 2000-ish backlinks) and reopen it again, the âQuickâ behavior goes until 2000-ish files and then slows down again.
So itâs clear this is a caching thing.
Perhaps this âtriggersâ something for the coders? Important Hopefully the caching can still be used when closing Obsidian and reopening it again?
Otherwise this would be a big blow for people using Obsidian to do research on big vaults (as I am testing right now).
Generating the local graph for this term caught my attention.
Itâs now âregeneratingâ every x-seconds (4-5) for some minutes nowâŚ
Seems in the background backlinks are searched and âaddedâ to the graph which causes the âregenerationâ.
Obsidian is looking for links clearly in other documents. Which exists because the backlinks were created in the research articles.
I kind of âfoundâ an interesting side-track during these investigations:
I created a MOC-Core banking systems.md with links to every document containing the search term âCore banking systemsâ which mimics the behaviour of clicking the Core banking systems node in the graph because that create the file Core banking systems.md but without the links. Apparently Obsidian than frantically goes and searches for all the mentions of âCore banking systemsâ in the vault, which in my case is rather âhugeâ.
With the MOC page you get exactly the same graphs without the âsearchingâ anymore - more correctly the search only returns the MOC-page as backlink when you are in a research article.
This opens some ânewâ approaches for me in my search to try and find good ways to tackle SuperVaults with very large amount of research articles.
Iâm still more than convinced Obsidian has itâs place in this area too.
The graph seems to use the same âmethodsâ as the one generating the links in the mentions. After a certain ârefreshâ period has passed, the graph detects that some other nodes need to be added to the graph and regenerates it.
You can clearly see this happening when making the global graph for the 80K+ articles. The âcloudâ grows until all the article nodes are added to the graph. During that generation you can clearly see the forward and backward links forming âsubcloudsâ.
@sam.baron
Sorry for the late reply. Again some busy ânon-obsidianâ days behind me.
The graph indeed seems to need some times when links are used in there original form.
With that I mean that graphs does not show this behavior when generating MOCâs (read: new files containing explicit links to other files). This way you âcircumventâ the normal backlinking Obsidian normally uses.
Sorry for the âfuzzyâ way of describing things. This concerns a personal research project which I cannot disclose. Always willing to share some more info via other more secure and private channels.
No problem, unfortunately, Iâm not following the problem. If you are open to sharing more information, you may want to post on Discord in the supporter-vip-lounge channel where one of the devs should respond.
