The issue is memory consumption with Javascript objects that makes most browsers page and slow down things, including eating up batteries. Hence many browser implementations are now trying to optimize this by wasm. By using rust as wasm frontend. But might use rust in native mode anyway.
So I imported the entire pathfinder 1e wiki into obsidian using a custom script to convert the HTML files I had into markdown files.
The entire vault contains around 35k files.
The indexing took about 10 minutes and the navigation is surprisingly fluid. The search takes quite a bit of time to find all the results, but that is expected.
The graph view on the other side is not usable with so many files. After loading for about 5 minutes obsidian black screened and I had to force close it.
So I guess I found the limit xD
This is the last image I have of it before the black screen.
(looks kind of cool, though I would love a full render of the graph)
System specs:
RTX 3070
Ryzen 3800 XT
32 GB RAM
10 Likes
Thanks much for that data point. Bad news that it canât handle that load well. And youâve got an ample system. Even worse, thereâs not much chatter about development activity related to performance improvements at volume and scale, at least nothing I can see. Gorgeous graph!
howâd you actually do this? I wondered if something like this can be done for other systems/editions. Would certainly be interesting as a quick lookup 
Also, dang that image looks sweet already.
only the global graph view doesnât work at scale, the rest seems to work even at 100k notes Interlude: Obsidian vs. 100,000 - by Alexander Rink. Also apparently Obsidian actually performs better at scale than competitors TfT Performance: Interim Results - by Alexander Rink
7 Likes
This sounds like good news. Iâm skeptical, but Rink looks quite thorough in his methodology, and even that he wants to â or sees a need to properly wade into performance waters. Great link. Thank you.
Sorry to revive an old thread; itâs still relevant, though.
This isnât true. There are standard tricks for all of these problems that would make it work just fine on even pentium-class hardware.
Also, seeing your messages in another thread, it should be noted that âadding an indexâ doesnât mean rigid categorization, it means building (without human intervention) an inverted-index and maintaining it every time the OS notifies you of files changes. âFull-textâ search doesnât mean duplicating the full text of all your documents; it means building a mapping that lets you quickly dispatch short queries as theyâre entered.
(Noteâyou donât poll the filesystem constantly for changes; you let the OS tell you about them. That way, there should be 0 added idle load.)
You, @Dor might yourself know all of this already, but I thought the thread in general was lacking it.
4 Likes
Although I donât have many notes yet I would love to see performance improvements like this! 
Evernote searches local files, not web files. You have to download the whole database to your local machine before it will search. And generally, local searches can always be faster than server searches because the local machine has nearly as much power as the server and yet is running only one account at a time. Thereâs no use for slow local searching with todayâs tech.