# Khoj: An AI powered Search Assistant for you Second Brain **URL:** \ **Category:** Plugins\ **Tags:** plugin-release\ **Created:** [February 4, 2023, 2:57pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756 "2023-02-04T14:57:42Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [February 4, 2023, 2:57pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/1 "2023-02-04T14:57:42Z")
- **Overview** : Khoj is a fast, private, AI powered search assistant for Obsidian - **Background** : - **Features** : Use natural language to privately explore your markdown notes in Obsidian. Includes _Incremental Search_ and _Find Similar Notes_ capability - **Quickstart** - **Resources** ---
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [February 4, 2023, 3:34pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/2 "2023-02-04T15:34:55Z")
**Screenshot of Khoj Search Modal** ![Khoj Search Modal](https://forum.obsidian.md/uploads/default/original/3X/5/6/562524f97078524d50cef01f13ca1cca074bc3a5.jpeg) ---
**Author:** ![scwunch](https://forum.obsidian.md/user_avatar/forum.obsidian.md/scwunch/32/20287_2.png) [@scwunch](https://forum.obsidian.md/u/scwunch)\ **Post date:** [February 22, 2023, 5:55am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/3 "2023-02-22T05:55:24Z")
After installing and re-installing, I keep getting this error, and I don’t know how to troubleshoot. ```auto Traceback (most recent call last): File "/home/ryan/.local/bin/khoj", line 8, in sys.exit(run()) File "/home/ryan/.local/lib/python3.10/site-packages/khoj/main.py", line 80, in run configure_server(args, required=False) File "/home/ryan/.local/lib/python3.10/site-packages/khoj/configure.py", line 44, in configure_server state.model = configure_search(state.model, state.config, args.regenerate) File "/home/ryan/.local/lib/python3.10/site-packages/khoj/configure.py", line 82, in configure_search model.markdown_search = text_search.setup( File "/home/ryan/.local/lib/python3.10/site-packages/khoj/search_type/text_search.py", line 168, in setup extract_entries(config.compressed_jsonl) if config.compressed_jsonl.exists() and not regenerate else None File "/home/ryan/.local/lib/python3.10/site-packages/khoj/search_type/text_search.py", line 60, in extract_entries return list(map(Entry.from_dict, load_jsonl(jsonl_file))) File "/home/ryan/.local/lib/python3.10/site-packages/khoj/utils/jsonl.py", line 27, in load_jsonl for line in jsonl_file: File "/usr/lib/python3.10/gzip.py", line 314, in read1 return self._buffer.read1(size) File "/usr/lib/python3.10/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/usr/lib/python3.10/gzip.py", line 496, in read uncompress = self._decompressor.decompress(buf, size) zlib.error: Error -3 while decompressing data: invalid block type ``` > **System Details** > > Operating System: Kubuntu 22.10 > KDE Plasma Version: 5.25.5 > KDE Frameworks Version: 5.98.0 > Qt Version: 5.15.6 > Kernel Version: 5.19.0-31-generic (64-bit) > Graphics Platform: X11 > Processors: 16 × 12th Gen Intel® Core™ i5-1240P > Memory: 7.5 GiB of RAM > Graphics Processor: Mesa Intel® Graphics > Manufacturer: Framework > Product Name: Laptop (12th Gen Intel Core) > System Version: A4 ---
**Author:** ![AlanG](https://forum.obsidian.md/user_avatar/forum.obsidian.md/alang/32/55225_2.png) [@AlanG](https://forum.obsidian.md/u/AlanG)\ **Post date:** [February 22, 2023, 6:28am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/4 "2023-02-22T06:28:14Z")
@hoperyto it would be good to add some specific details both here and on Github as to how this actually functions. 1. What is Khoj backend? Is this your code, or is Khoj an existing AI/ML platform which you’ve integrated into Obsidian? 2. Does this all run locally? Is your vault data sent to any third-party or external (non-localhost) service? ---
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [February 23, 2023, 12:08am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/5 "2023-02-23T00:08:07Z")
> it would be good to add some specific details both here and on Github as to how Khoj actually functions. Does [the architecture diagram](https://github.com/debanjum/khoj#architecture) help clarify how Khoj actually functions? > What is Khoj backend? Is this your code, or is Khoj an existing AI/ML platform which you’ve integrated into Obsidian? - The Khoj backend is where most of the indexing, search logic is implemented. The Obsidian plugin provides a frontend to configure and use the Khoj backend - Yes, the Khoj project has been created by me - The backend/platform allows indexing multiple different content types (currently org-mode, markdown notes, beancount transactions and images) and exposes an API for the different frontends/interfaces (currently Obsidian, Emacs and Web) to interact with the application > Does this all run locally? Is your vault data sent to any third-party or external (non-localhost) service? Yes, Khoj runs locally on your machine. Your vault data does not leave your machine, it is not sent to any third-party or external services. Only the AI models are downloaded from [HuggingFace](https://huggingface.co/) on first run. You do have the option to use OpenAI models for [search](https://github.com/debanjum/khoj#use-openai-models-for-search) and the (currently beta) [chat](https://github.com/debanjum/khoj#chat-with-notes) features. But this is something you’ll have to manually enable/configure. In such cases your note(s) will be sent to OpenAI for processing. ---
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [February 23, 2023, 12:25am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/6 "2023-02-23T00:25:16Z")
Hey @scwunch, I haven’t seen any folks hit this zlib decompression issue yet. Seems like your khoj notes index (which are stored in gzipped jsonl format) is corrupted somehow. One thing to try is to delete your `.khoj` directory at `~/.khoj`, restart khoj and reinstall/re-enable the khoj obsidian plugin. This will index your notes again and hopefully fix your issue. Also it’d be great if you can [open a new discussion](https://github.com/debanjum/khoj/discussions/new?category=q-a) on the Khoj Github to investigate this issue further. ---
**Author:** ![scwunch](https://forum.obsidian.md/user_avatar/forum.obsidian.md/scwunch/32/20287_2.png) [@scwunch](https://forum.obsidian.md/u/scwunch)\ **Post date:** [February 23, 2023, 4:49am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/7 "2023-02-23T04:49:19Z")
That did the trick, thanks 🙂 I might have been using different/older versions for the backend and the frontend, if that’s possible that corrupted the index. Do you still want me to open a discussion? ---
**Author:** ![SteveLambert](https://forum.obsidian.md/user_avatar/forum.obsidian.md/stevelambert/32/25881_2.png) [@SteveLambert](https://forum.obsidian.md/u/SteveLambert)\ **Post date:** [February 28, 2023, 2:53am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/8 "2023-02-28T02:53:14Z")
Hi, I got Khoj installed today and have been playing around. Are there any examples you have where you said “wow, khoj is perfect for this!” Or things you feel like it enables that wouldn’t otherwise be possible? Like what are the itches for you that this scratches? ---
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [March 2, 2023, 6:12pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/9 "2023-03-02T18:12:53Z")
Sorry, missed your message. No need to open a discussion if its fixed for you. (Such corruptions can randomly occur for e.g if app closed in middle of updating index) ---
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [March 2, 2023, 7:07pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/10 "2023-03-02T19:07:28Z")
Hey @SteveLambert, great that you got to play with Khoj! > Or things you feel like it enables that wouldn’t otherwise be possible? Like what are the itches for you that this scratches? The specific itches that Khoj alleviates for me are: 1. I know I’ve written about topic X somewhere in my notes but I can’t find it with my keyword based search tools. So I’ve to fallback to do a global search with Google instead 2. After having researched on Y, finding out that I’d already collected notes on that topic 3. Reducing the general cognitive load of searching as I can be lax with the words I use in my query, unlike in traditional keyword based search where I need to use the exact word used in a note or it wouldn’t show up in the results. E.g if I search for “Buy a car”, it’ll find notes that mention a Corolla (or Ferrari) 4. Reducing the general cognitive load of note taking as I don’t need to worry about using the right tags, words in the note to be able to retrieve it later 5. The speed to get relevant results opens up more uses for my notes to actually function as a (good) second brain As a user I’ve observed myself default to using Khoj instead of keyword based search tools. I think because of the reduced cognitive load of searching it enables. For context: I have a decade worth of notes in Emacs/Org-mode and my memory/recall capabilities are not that great. ---
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [March 2, 2023, 7:11pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/11 "2023-03-02T19:11:17Z")
> Are there any examples you have where you said “wow, khoj is perfect for this!” I haven’t (unfortunately) collected good examples (but I should). One concrete recent example I can recall: I was collecting advice for friends about to have their first child. I know I’ve notes on this stuff. With Khoj, I could just search for _“advice for kids”_, get notes on infant care, educational toys for children etc and use those notes as a jumping point to build from. It makes these kind of processes easier, more enjoyable than having to start from scratch ---
**Author:** ![SteveLambert](https://forum.obsidian.md/user_avatar/forum.obsidian.md/stevelambert/32/25881_2.png) [@SteveLambert](https://forum.obsidian.md/u/SteveLambert)\ **Post date:** [March 15, 2023, 5:38pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/12 "2023-03-15T17:38:39Z")
I see. I will mess around with it again. I think because of the way I was approaching the search, I found the results “noisy” and there seemed to be a lot of them. Is there a way to see accordion folded results that you can expand to see more? I think being able to skim through titles would be helpful. Thanks for your answer, it’s helpful! ---
**Author:** ![hoperyto](https://forum.obsidian.md/letter_avatar_proxy/v4/letter/h/58f4c7/32.png) [@hoperyto](https://forum.obsidian.md/u/hoperyto)\ **Post date:** [March 16, 2023, 5:26pm UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/13 "2023-03-16T17:26:33Z")
> I think because of the way I was approaching the search, I found the results “noisy” Yeah, this doesn’t work like the traditional keyword search, works better with more verbose natural language kind of searches. E.g “_What is the meaning of life?_” vs “_life meaning_” Also sometimes the index can get corrupted (haven’t found out the exact path to fix it). So regenerate the index if the results seem too noisy. You can do that from the khoj obsidian settings page > and there seemed to be a lot of them. You can set the number of results you want from the settings too. It defaults to 5/6 > Is there a way to see accordion folded results that you can expand to see more? I think being able to skim through titles would be helpful. Ah yeah, makes sense. That’s available in the khoj plugin for emacs. I’ll add that to the khoj obsidian plugin soon. ---
**Author:** ![K1kianian](https://forum.obsidian.md/user_avatar/forum.obsidian.md/k1kianian/32/51049_2.png) [@K1kianian](https://forum.obsidian.md/u/K1kianian)\ **Post date:** [July 31, 2023, 10:30am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/14 "2023-07-31T10:30:01Z")
after I try to do the first step: 1. Install & Start Khoj Backend: `pip install khoj-assistant && khoj --no-gui` I got this error: ![Screenshot 2023-07-31 at 1.55.58 PM](https://forum.obsidian.md/uploads/default/original/3X/c/3/c3ab200070d6512d3c2f83c8e13da5a667cb0aa8.jpeg) can anyone help me? ---
**Author:** ![K1kianian](https://forum.obsidian.md/user_avatar/forum.obsidian.md/k1kianian/32/51049_2.png) [@K1kianian](https://forum.obsidian.md/u/K1kianian)\ **Post date:** [July 31, 2023, 10:33am UTC](https://forum.obsidian.md/t/khoj-an-ai-powered-search-assistant-for-you-second-brain/53756/15 "2023-07-31T10:33:33Z")
the error is continued: ![Screenshot 2023-07-31 at 2.01.56 PM](https://forum.obsidian.md/uploads/default/original/3X/c/6/c6b839d930885dd57fcae1ba8e76f46399ba8f16.jpeg) ![Screenshot 2023-07-31 at 2.00.28 PM](https://forum.obsidian.md/uploads/default/original/3X/3/d/3de3e2d4e1cc1d179283ade4b61c136d1b22d1f0.jpeg)