I’m not worried that much about how complex the searches are-- they are plenty complex from what I see-- I care about indexing to allow for scalability and performance to large note numbers. We simply don’t have it. Some apps that track notes and second brain type items have been stress tested to 1 million, 2 million notes, at good performance at 250 thousand notes and above. We have documented forum posts about real-world tests at 12 thousand, 15 thousand notes Obsidian notes, with things slowing way down.
For this index plugin, nothing out-out-of-this-world under the hood, just a full text indexing system, where query results can be near-instantaneous. Like my Thunderbird email client, with 15 years of email, the SQLite index itself is over 1.5 gigs. But when I search for a search term, it somes up in 1 second.
@aidenlx, I appreciate you helping. I will be looking into external tools in the interim, because i have no other choice. Obsidian is really a great product, which is busting at the seams with great UX design and features.
I suggest that it’s a great time to turn more attention now to a robust highly-scalable back end. The adoption at enterprises would skyrocket, because Obsidian would go from being seen as a beautiful and remarkably versatile tool, to a must-have, mission-capable, scalable and reliable home for much more knowledge than previously possible.
And again, there are many “low tech” ways (not needing large license fees) to accomplish this – SQLite, firebirdsql.org, so on. Far be it for me to say any more than this, because the implementation and considerations are far above me, but I see what is used elsewhere to smartly wrangle and present – quickly-- large data sets. The ease of use and quality of experience at large note counts GREATLY increases.
Obsidian can do it. I know they can!
