Introducing Obsidian AI Assistant: Your Gateway to AI Models in Obsidian Notes

Hello Obsidian users!

I am excited to announce the release of a new plugin called “AI Assistant”. It enables seamless interactions with cutting-edge AI models, such as OpenAI ChatGPT, DALL·E2, and Whisper, directly from your Obsidian notes.

With Obsidian AI Assistant, you can now enjoy the following features:

  1. Text assistant with GPT-3.5 and GPT-4: Get access to two commands to interact with the text assistant, “Chat Mode” and “Prompt Mode”. Chat Mode allows you to discuss with the AI Chatbot from your Vault to generate content for your notes, while Prompt Mode lets you use selected text from your note as input for the assistant (for instance to summarize, format them, ask questions etc).
  2. Image generation with DALL·E2: Generate beautiful images for your notes with ease.
  3. Speech-to-text with Whisper: Dictate your notes with the Speech-to-Text command, and the transcript will be immediately added to your note at your cursor location.

The AI Assistant is also compatible with your mobile! I hope you will enjoy using the plugin and look forward to hearing your feedback!

Happy note-taking!

5 Likes

Would you be able to use a local LLM/Agent, like lama.cpp, in place of GPT?

The cost associated with using cloud-based LLMs is the thing keeping me from working with it.

1 Like

Currently, there is no support for local models. I also would like to have that possibility in the future, but I am afraid that the response from the model will take too much time to arrive if you do not have powerful enough hardware, making it unusable in practice.

Good quality and fast (dependant on hardware of course) local and private LLMs are coming soon, weeks to months. They’re already available with technical skills. Plugins will be able to simply point to local or cloud models.

1 Like

I am super excited to see AI integration. One step closer to the AR AI assisted productivity suite that I always dreamed of :smiling_face_with_three_hearts:

Whilst the text capabilities are awesome, does the AI prompt have the capability to handle linking and backlinking?

For example:
“Could you show me all the nodes that mention the term ‘XYZ’”
“Could you link nodes that mention ‘XYZ’”

Being able to add / remove links and have deeper interface interaction with AI like this would be super helpful, especially for those with larger vaults that might need reorganising, but would otherwise be a pain to do manually.

1 Like

Can you give us a rundown of privacy of the plugin? what is the chance of man in the middle capturing data that the plugin scans from our obsidian? Of course i have concerns about Openai also collecting, but indy chatbots create a big opportunity to vacuum up personal data.
Thanks for sharing plugin also. Im excited to give it a go.

4 Likes

Very interesting.

Continue my guy.

When I ask question about remaining task it doesn’t seems to understand that - is a task that been done.

This sounds great, thanks.

@sqlc you asked about local model due to cost of cloud based models - just want to make sure you are aware that GPT-3.5 is free in their UI and near free via the API. I know this is highly debated, but even if you have very powerful hardware the local solutions are close to GPT-3.5, not GPT-4.

@subrosafarian I don’t think MitM attacks are relevant at the application layer. Stuff like Obsidian and plug-ins just use encrypted communication links established at lower layers that must be held accountable for any vulnerabilities.

(I have no association with Obsidian or this plug-in so feel free to ignore my responses :laughing:)

I love the idea and the showcase; however, I keep getting an error about a missing “}” in the JSON file. There is a ticket regarding that on Github, but still no update.

Isn’t anybody else concerned about this? Subrosafarian asked a valid question.

3 Likes

Anyone a clue what I could do about the following error message that pops up when I post anything in the AI Assistent chat:

"## OpenAl API ## SyntaxError: Expected property name or “’ in JSON at position 1”

Happy to read your suggestions!

2 Likes

Can it do what the AI in Evernote does which is…

  1. Summarise your notes
  2. Search your notes e.g “Find me the article I wrote about AI. I think it was about 3 months back and mentioned both Chat GPT and Bing”

on phone we can’t use whisper only chat is available.

Hi @anshulpatyal
It seems you are in a new tab without having a note opened. Can you retry the command with the cursor in an existing/new note?

Hi @RammyLeeds

  • Use case 2 is not possible currently since the content of your vault is not processed by GPT for privacy reasons,
  • Summarizing a note is possible using the Assistant Prompt command: Select the text you want to summarize → Open the Assistant Prompt command → Ask for the summary

obsidian_summary

@quentin_42fr it seems Reor offers a local ai tool. I haven’t tried yet Is it planned to do it with Obsidian anytime soon ? https://www.reorproject.org/

Ok, I tried it, it has less functionnalities than Obsidian, but performs a local IA. Good point is that it works with exactly the same kind of files and that it is sufficient to set the vault directory in Reor as the one used by Obsidian to use local IA on Obsidian notes …