Super FR: Enhance Obsidian with a type system for notes and database-like views (metadata)

Even though Obsidian doesn’t natively provide database-like features (ala Notion/Airtable), the introduction of YAML attributes and a constellation of plugins have emerged to provide some support for this functionality (dataview, dbfolder, metaedit, metadata menu, etc) [Recently Obsidian Project].

It would be nice if Obsidian could provide direct support.
Specifically, Obsidian’s notes could be enhanced with type system and queries/views be generated over notes of a certain type.

Broadly speaking, the following workflow should be possible:

  1. Define a “note type” or “note class” using a special file where the user specifies a schema (a set of attributes and maybe their base types (Boolean, number, string, date))
  2. Use a tag, directory, or YAML entry to associate a note with its type (or types)
  3. Populate said note with YAML or inline attributes according to its type(s)
  4. Build graphical view(s) (tables, boards) using the note type definition and pulling data from all notes that are associated with that type.

This can be implemented either completely by Obsidian or with a (tighter) coordination between Obsidian and the third party plugins.

Some related Feature Requests:

34 Likes

I’ve been trying to solve getting block level filtering and querying with dataview for basically a year now, and I finally stumbled on this which beautify solves the problem. Sharing here for inspiration.

By using markdown comments (and hiding them) I can create a block level metadata system that reproduces most of the functionality of other outliners.

Normal View:

With highlighted line:

There are multiple metadata fields that can be embedded in a markdown comment for each outlier block and then hidden with some CSS link

Together with dataviewjs enables just about all the functionality of Tana and most outliners. The only prevailing issue for me is the ability to write back through a dataview query and fancy UI like dynalist, tables, etc.

The really clean thing about this is that the markdown file is still valid when rendered in other viewers. The comments would just be hidden anyways in preview modes.

Just sharing here for inspiration

3 Likes

That sounds like a great solution. i wonder if there are any downsides. We’ll have to play with that for a while.

We are not going to go down the path of bullet-level types. Feel free to open a “plugin idea” around your workflow.

This is a good write-up. to be honest, this is also what I was thinking. You mention having a separate file that basically contains the type information, and then other notes either use a #tag or frontmatter value to “connect” to the type.

I would add the type folder could be hidden (toggled on and off) not to get in the way of the user unless they want to work with it.

Each plugin then can read from the type note, or better yet Obsidian would have a basic APi around the type system for discovery.

The second problem though is how users define what a field is in the actual note. Obviously strucutred data can be captured in frontmatter (probably solves much of the problem), but also having native inline field support would solve the remainder.

Finally, @blacksmithgu hits on a core issue compared to other tools like Tana. The editing of YAML, etc is painful. Tana has a simple UI that makes defining and thne using schema in your notes very fast process. Yesterday I created a data models in Tana, then replicated with Metadta and db folder. Tana could make a table structure in a few minutes, and in obsidian lots of clicking and trial and error.

Obviously, if I want to do what Tana does I can just use Tana. but Obsidian offers a lot of advantages: notes in clear text, local first, and the electron wrapper is a great environment for building solutions. It’s an OS for note-taking.

6 Likes

I think the path for Obsidian would not be to implement it themselves but to improve the existing API so the quality of those plugins. A more complete file editing functionality would give those plugins wings. In my case dbfolder

  • file.editField(key)
  • file.editInlineField(key)
  • file.editTask(position?)
3 Likes

Hi, this was very inspiring thanks!

I think that with the last version (0.3.6) Metadata Menu now covers most of it, even if the UI is still not very easy to use from my point of view (i’m not a UX design expert and I’ve built the plugin the way I use it on a daily basis):

  1. this is the purpose of fileClasses. Even if I’ve put some features to facilitate the creation of new fields, the rendering of the fileClass itself is very raw (JSON). Though it’s not meant to be used directly, but I would be happy to hear suggestion about how to better manage them

  2. It’s the purpose of supercharged tags and fileClass queries: you can associate fileClasses (1) with tags or with files matching a specific query (a folder path for example)

  3. since 0.3 there are several way to insert a field in a note: from command palette, from a dataview table, from a dedicated api method, from the note’s fields’ modal, and now from the fileClass view

  4. This is what I’ve added in 0.3.6 : the fileClass view

I do agree with @RafaelGB. I think that these database features shouldn’t be the main focus of obsidian, and that fields editing shouldn’t be the focus of dataview. Those team do great job at building tools and API that are robust and fast, increasing complexity would put their core product at risk imho.

In addition to what Rafael has said about improving the API, i would like Obsidian and Dataview to work together on the indexing capabilities. If we want to build powerful but efficient plugins, we shouldn’t re-create indexing in each plugin, this will lead to slowness and reduce the user experience with vaults growing in size and plugins adding more and more features.

That being said, this all markdown-files-based-database topic is still green field and it’s very fun to work on it and see all those great ideas and features popping every day

4 Likes

I agree that the first and most important step is to have Obsidian Core provide the backend indexing and API to read/write metadata, beginning with what we already support: YAML.

4 Likes

What you all say about API access to the metadata is on spot. However, this is not what this original post is about. It’s about a type system, a way of defining those metadata elements. The two plugins in context here, Metadata and DB folder, user two different systems for accomplishing this.

While it’s cool what is accomplished, long term having a common type system for all plugins would be valuable for all plugin devs and for Obsidian devs as well.

10 Likes