The issue is memory consumption with Javascript objects that makes most browsers page and slow down things, including eating up batteries. Hence many browser implementations are now trying to optimize this by wasm. By using rust as wasm frontend. But might use rust in native mode anyway.

So I imported the entire pathfinder 1e wiki into obsidian using a custom script to convert the HTML files I had into markdown files.
The entire vault contains around 35k files.
The indexing took about 10 minutes and the navigation is surprisingly fluid. The search takes quite a bit of time to find all the results, but that is expected.
The graph view on the other side is not usable with so many files. After loading for about 5 minutes obsidian black screened and I had to force close it.
So I guess I found the limit xD

This is the last image I have of it before the black screen.
(looks kind of cool, though I would love a full render of the graph)

System specs:
RTX 3070
Ryzen 3800 XT
32 GB RAM

5 Likes

Thanks much for that data point. Bad news that it can’t handle that load well. And you’ve got an ample system. Even worse, there’s not much chatter about development activity related to performance improvements at volume and scale, at least nothing I can see. Gorgeous graph!

how’d you actually do this? I wondered if something like this can be done for other systems/editions. Would certainly be interesting as a quick lookup :stuck_out_tongue:

Also, dang that image looks sweet already.

only the global graph view doesn’t work at scale, the rest seems to work even at 100k notes Interlude: Obsidian vs. 100,000 - by Alexander Rink. Also apparently Obsidian actually performs better at scale than competitors TfT Performance: Interim Results - by Alexander Rink

4 Likes

This sounds like good news. I’m skeptical, but Rink looks quite thorough in his methodology, and even that he wants to – or sees a need to properly wade into performance waters. Great link. Thank you.