VaultChat: chat with your Obsidian vault, cited answers, and published eval numbers (free & open source)

Disclaimer

Is this project open source? Yes

Is this project completely free? Yes. The app itself is free. You bring your own OpenAI and Anthropic API keys and pay those providers directly per use (about $0.02 per question on my vault).

Is this project made with AI beyond basic tools? Yes. I built it with Claude (Anthropic) as a coding assistant. I designed it, wrote the evals, and tested it myself.

Like a lot of you, my vault outgrew my memory. I wanted to ask it things like “what did I conclude about X?” without pasting notes into a chatbot or sending my whole vault to someone’s cloud. So I built VaultChat, and I just relaunched it after an honest detour (more below).

It’s a small open-source Python tool, not a plugin. It reads your vault folder and lets you chat with your notes in the browser. Two things I cared about:

  1. It cites the source. Every answer links back to the exact note and section. If the answer isn’t in your notes, it says “I don’t know” instead of inventing one.
  2. I measured that. I wrote an eval harness (15 hand-written questions, LLM-as-judge) and ran it on my own vault: recall@8 100%, fact coverage 97%, 100% abstention on out-of-scope questions, about $0.019 per query. The evals caught a real bug: the right note was always retrieved, but the specific fact sometimes sat just outside the chunks the model saw. Widening retrieval fixed it (86% to 97%).

The honest part: at first launch I sent people to a “live demo” that had gone to sleep, and I had checked a status code instead of opening the page. I rebuilt it, verified it by asking it a real question, and added an uptime monitor and keep-alive so it wakes fast and stays up.

How it works and its limits:

It is not fully offline. Retrieved chunks go to the OpenAI (embeddings) and Anthropic (answers) APIs at query time, under your keys. There is no server of mine in the middle, and I store nothing. Local embeddings are on the roadmap.

Setup is about 5 minutes on the command line. v1 is single-turn.

The numbers come from my vault, not a universal benchmark.

Try it without installing: https://chat-vault-demo.streamlit.app/

Code and setup: https://github.com/CristianG-AI/Chat-Vault-Rag

Landing page and hosted-version waitlist: https://cristiang-ai.github.io/Chat-Vault-Rag/

It’s free and open source, and I’m sharing it for genuine feedback. What would make this useful in your workflow, and what’s the first question you’d ask your vault?

Cristian Galvez