Core Database Plugin

I saw one similar request, but it’s already archived and OP’s use-case is not really the same as mine

Use case or problem

Dataview, DB Folder, Obsidian Query Language, and all those metadata manager plugins… Databases and metadata management are so widely used in the community, and Obsidian not having native support for them makes the app less future-proof.

Although unlikely, what if I one day wake up to find that all my Dataview queries are no longer working because of a pesky bug and the dev’s nowhere to be found?

Proposed solution

I think it would be better that Obsidian just release its own core plugin, sherlocking some features from all those other plugins that are already doing the job.

Current workaround (optional)

Notion databases. They are excellent, and it’s one of the reasons I still keep going back to it and why I can never fully switch to Obsidian. I also use dataview for some stuff.

Related feature requests (optional)

4 Likes

I would love the ability to recreate my old databases on Obsidian. Using tables for it would be difficult. Being able to sort and organize files and more in a database would be great.

1 Like

Obsidian works with portable Markdown documents. Documents you’ll be able to read 60 years from now, long after Obsidian has ceased to exist, in any Markdown viewer or simple editor. All you need to do is keep them around.

Plugins like Dataview destroy this concept. Sure they’re powerful, but by using them you are giving up on what I call the number one reason for using Obsidian in the first place. If this is what you want to do, then why use Obsidian? I see demo vaults of “movie databases” that contain hardly any Markdown. Almost all content is embedded in YAML front-matter. Have fun reading those in your favorite text editor or any random Markdown viewer, or converting them to HTML or some other format (e.g. with pandoc).

There are basically two things you can do:

  1. Forget about portable Markdown aka “take the blue pill”. If this is your avenue, then my suggestion is to forget about Markdown too. Pick some tool that has nice visuals, a rich text editor, native links and a query language. Like Notion. Your content will probably be locked up in some kind of database you can’t access by yourself, but as long as you can export your content out of the tool, you’re good to go.
  2. Embrace portable Markdown aka “take the red pill”. Remember that a document written in portable Markdown means that the document should be fully readable in any Markdown viewer, and editable in any text editor.

Me, I’m the option 2 kind of guy. Which is why I built a little tool (in Java) that can run queries over Markdown documents and that embeds their output back into these same documents. All content is plain Markdown, editable in any editor. At any time, the content of my vaults is internally consistent. I can pick any file and convert it to (e.g.) HTML with any Markdown processor. Basically I have dynamic content in static pages.

I call this “taking the red pill” for a reason. If you use Markdown, you do that because it’s portable. Markdown is great because it’s just plain text. There are hundreds of tools out there that manipulate text and thus also Markdown. And the cool thing: they’re probably already present on your machine. My advice: embrace that! Don’t try turn Markdown into something it’s not supposed to be. Don’t stick too many fancy plugins in Obsidian that kill the portability of your content. Instead, spend some time in learning generic text processing tools. You’ll discover a world that’s much more powerful!

Yes, this takes an effort, maybe even including learning programming and writing your own tools. But it’s well worth it!

6 Likes

Actually, it’s Obsidian that manages/caches the metadata (YAML, links, tags, etc) and the plugin are built on top of this info.

The idea being that obsidian does the “foundation work” and plugins access/use this data via API for whatever they want to do.
What more can be done?

  1. Improve the obsidian search language so it can query all metadata attributes
    Search operator for YAML similar to Line / Section
  2. Do the foundation work and support inline attributes (
    Native support for Inline (Intext) attributes [YAML Fields] (`key:: value` a la dataview)

The same thing can be said about any software, including Notion.

A side note, if your note taking systems is heavy on the structured data/databases. I am not sure a “many files with metadata” solution is good fit. It’s not performant, it’s brittle to changes that are not made according to the schema, it going to be hard to make sense of your data without a query/view component.
Small and simple databases, I can see an use for those in Obsidian. Large and complex databases, I would use a specialized tool for those.

6 Likes

I liked your red pill idea. Do you have any recomendation in material to start the journey?? I’m a newbie myself and to be honest I do use some data view querys, and i do wonder if this doesnt break the time-proof concept.

I have soo many question. But i supose that i’ll start with what do u think should be the main focus to start ?

I’d suggest:

  • Learn a bit of programming. Try Python. It’s one of the easiest to learn, runs on all OSes, and has lots of support for reading and writing files and for processing text. Also see Learning Python.
  • I’m sure you have a couple of documents in your Obsidian vaults that are basically reports, like lists of links, or overview tables. You probably use Dataview to generate those reports. Try replacing some of those with scripts that process your documents (remember: they’re just plain text files on disk), extract the information you need, and write the extracted information to disk again (remember again: Markdown is just plain text).

This may feel a bit convoluted and backwards, but:

  • The only dependency you’ll have is on your own skills.
  • Scripts you’ll write in Python you’ll easily be able to run 15 years rom now.
  • You’ll no longer be “locked” into Obsidian and a bunch of plugins. From now on any Markdown editor will do.
  • You’ll get a lot smarter in the process. Learning Python (or some other general purpose programming language) is much more rewarding and useful than learning some SQL-esque proprietary query language with extremely limited applicability (which is what DataView is).
2 Likes