What is the problem
Current search in Obsidian is barely helpful when the number of notes exceeds a thousand.
Grep by keywords with sorting by date is 90th technology.
What is needed: search
Search based on embeddings. Which is: generate embeddings for each note, and when searching, order by distance to embeddings.
It would help a lot when searching for:
- texts written in different spellings (like British/American)
- texts with typos
- texts written in two languages (like I don’t remember did I write a note in my mother language or in English) — language models are doing that great
- different weight for different words. Like if some word is unique, it is probably more important than a word which exists in every second note
Not being an expert in ML, I guess this can be done locally with small model, and not having to pay for anything or upload data anywhere.
What we need: chatbots
When we have embeddings, chatbots is logical conclusion. When asking a question, find relevant notes, attach them to the chat, and keep chatting.
Would it work?
I did a little experiment. I uploaded all my notes to “OpenAI assistant playground” (which does more or less what I described above), and to my surprise, it actually started finding relevant notes, and was able to chat about the notes.
Plugins
There are some plugins for Obsidian, “Smart connections” seems to be most relevant/popular.
“Smart connections” just does not work. Whatever I tried, it tells me there’s an “error” without any diagnostics. Interface is too complicated, too many options. It is not ready to use.
And on top of that it tries to sell premium without explaining what premium is supposed to do.
And on top of top of that, having to pay three times: Obsidian, Plugin, and OpenAI API — is too much trouble.
Bottom line
Proper search is must have. Chat about notes would be very nice optional additional with extra pay.
Please add it.