Add an HTTP search API to Obsidian Sync Service

(existing FR Establish a unified search API as a platform-level service asks for a client-side search API, which would also be great, but just assists with making Obsidian plugins. I’m asking for an API into my text vault from any application with the API key)

Use case or problem

I’d like to be able to build applications that provide rapid views of subsets of my Obsidian Text vault.

Sometimes these views would be interactive, sometimes not.

This is, as far as I can tell, an untapped opportunity in the “own your life’s information” space.

  1. Dropbox has a search API, but does not parse the .md extension today (gasp, I know).
  2. iCloud does not provide a way (afaik) for users to search the text of their own files.

I like the way obsidian understands tags, links, filenames, etc. I would like it to provide a searchable view of my own my digital life and would pay for this service.

This enables me and many other people to build fantastic applications against an Obsidian indexed “back end”

use cases:

  1. Rapid search, shows matching lines only, tap link to open file in obsidian. GET obsidian.md?API_KEY=123&q=cartography → return files and matching lines
  2. My shopping list, displayed as an interactive checklist GET obsidian.md?API_KEY=123&q=f:grocery_list.md%20ORtag:groceries → return files and matching lines with content and line numbers
  3. My grocery list, displayed as an interactive checklist GET obsidian.md?API_KEY=123&q=f:shopping_list.md%20ORtag:shopping → return files and matching lines with content and line numbers

Proposed solution

I’d love to see a secure API provided for every Obsidian sync user. I’d want access to the full power of Obsidian search (text inside, filename, tags).

Current workaround (optional)

  1. advanced URL plugin with obsidian native can both deep link to files and deep link to search. Problems: the ios app is so slow to load that this often fails the first time. deep linking to search results is not useful because th left nav and search bar cover up the search results.
  2. tried some very fancy stuff with icloud bookmarks, the shortcuts app, and the scriptable app. it works but only for specific bookmarked files because of icloud’s security practices.
  3. Today I started looking at dropbox, but I’d need to run and maintain a full sync, then run my own search against that text :confused:

Related feature requests (optional)

1 Like

This is not possible. Obsidian Sync is End-to-End encrypted. We don’t have (plaintext) access to your notes so we can’t build a search service in the cloud over your notes.

3 Likes

@WhiteNoise, while I recognize you have authority here, I want to call out that I have, in the couple times I’ve interacted with you, found your responses borderline rude and definitely dismissive.

In my view, this shows a drive to close issues at any cost and do not reflect the care and effort people are putting in to the feedback they provide the Obsidian team.

I like this product. a lot. I want to build on it and I want to have discussion with the other members of the community to see what they think. I don’t need rapid issue closure.

In this case, you mention e2e as a reason to archive my FR, but that doesn’t reflect an understanding of what End to end encryption actually means. E2e is about wire transfer. It does not imply at rest encryption on Obsidian owned servers, although they may also do that.

Even if the FR was infeasible, I’d like it to be visible and vote-able. I put a lot of care into the writing of it.

2 Likes

I am sorry for the previous interactions. I closed it because it is unfeasible. The only places where Obsidian doesn’t encrypt data is in your local devices.
In transit and at rest in Obsidian’s server the data is encrypted and we don’t have the key.
Dropbox is not end-to-end encrypted, they can read your data.

3 Likes

E2E does imply encryption on Obsidian’s servers — the ends are your devices.

And if you are concerned about security you should be asking for HTTPS not HTTP :):):slight_smile:

Your care is obvious and appreciated.
Unfortunately for you, the FR contains the implied premise that it is technically feasible. whitenoise is 100% right, it’s not.

Regarding E2E encryption, your devices are both “ends” in Obsidian Sync E2E encryption. Again, whitenoise is using the term correctly.

Cheers.

1 Like

A few months back, MongoDB announced a product based on some new research in encryption that claims to allow server-side searching of encrypted DBs without the server holding keys. It’s quite novel.

At the time of the announcement, a MongoDB employee promised the algorithm would be published along with source, but I’m not aware they’ve fulfilled that promise or if it will withstand scrutiny from the cryptography community.

Either way, Obsidian Sync uses AES-256 encryption and if you choose to retain the key, nothing they can do to search it.

For any FR that involves providing server side functionality - there is an FR for a full web interface too - I think it is assumed that E2E would become optional.

I care very much about security and have implemented a lot of security protocols/code in my time, but I think E2E is overrated. In choosing to use Obsidian I am trusting them with my data whether they implement E2E or not.

So, E2E makes sense as long as they are not providing any server side functionality, but I would not have a problem opting to turn it off in exchange for new functionality.