Super FR: Enhance Obsidian with a type system for notes and database-like views (metadata, object oriented model)

[WhiteNoise]

Even though Obsidian doesn’t natively provide database-like features (ala Notion/Airtable), the introduction of YAML attributes and a constellation of plugins have emerged to provide some support for this functionality (dataview, dbfolder, metaedit, metadata menu, etc) [Recently Obsidian Project].

It would be nice if Obsidian could provide direct support.
Specifically, Obsidian’s notes could be enhanced with type system and queries/views be generated over notes of a certain type.

Broadly speaking, the following workflow should be possible:

1. Define a “note type” or “note class” using a special file where the user specifies a schema (a set of attributes and maybe their base types (Boolean, number, string, date))
2. Use a tag, directory, or YAML entry to associate a note with its type (or types)
3. Populate said note with YAML or inline attributes according to its type(s)
4. Build graphical view(s) (tables, boards) using the note type definition and pulling data from all notes that are associated with that type.

This can be implemented either completely by Obsidian or with a (tighter) coordination between Obsidian and the third party plugins.

Some related Feature Requests:

Native support for Inline (Intext) properties (`key:: value` a la dataview)

Use case or problem YAML can be unwieldy, and sometimes just doesn’t support certain values (e.g. links). It would also be great to give yaml fields in the content of a note. Proposed solution The Dataview plugin recently gave support for inline yaml fields in the content of a note. One can type key:: value in the body (not the frontmatter), and dataview recognises it as if it were key: value in the yaml. I propose that Obsidian supports inline fields so that other plugins can also use this f…

[rapatel0]

I’ve been trying to solve getting block level filtering and querying with dataview for basically a year now, and I finally stumbled on this which beautify solves the problem. Sharing here for inspiration.

By using markdown comments (and hiding them) I can create a block level metadata system that reproduces most of the functionality of other outliners.

Normal View:

With highlighted line:

There are multiple metadata fields that can be embedded in a markdown comment for each outlier block and then hidden with some CSS link

Together with dataviewjs enables just about all the functionality of Tana and most outliners. The only prevailing issue for me is the ability to write back through a dataview query and fancy UI like dynalist, tables, etc.

The really clean thing about this is that the markdown file is still valid when rendered in other viewers. The comments would just be hidden anyways in preview modes.

Just sharing here for inspiration

[obsequious]

That sounds like a great solution. i wonder if there are any downsides. We’ll have to play with that for a while.

[WhiteNoise]

We are not going to go down the path of bullet-level types. Feel free to open a “plugin idea” around your workflow.

[TfTHacker]

This is a good write-up. to be honest, this is also what I was thinking. You mention having a separate file that basically contains the type information, and then other notes either use a #tag or frontmatter value to “connect” to the type.

I would add the type folder could be hidden (toggled on and off) not to get in the way of the user unless they want to work with it.

Each plugin then can read from the type note, or better yet Obsidian would have a basic APi around the type system for discovery.

The second problem though is how users define what a field is in the actual note. Obviously strucutred data can be captured in frontmatter (probably solves much of the problem), but also having native inline field support would solve the remainder.

Finally, @blacksmithgu hits on a core issue compared to other tools like Tana. The editing of YAML, etc is painful. Tana has a simple UI that makes defining and thne using schema in your notes very fast process. Yesterday I created a data models in Tana, then replicated with Metadta and db folder. Tana could make a table structure in a few minutes, and in obsidian lots of clicking and trial and error.

Obviously, if I want to do what Tana does I can just use Tana. but Obsidian offers a lot of advantages: notes in clear text, local first, and the electron wrapper is a great environment for building solutions. It’s an OS for note-taking.

[RafaelGB]

I think the path for Obsidian would not be to implement it themselves but to improve the existing API so the quality of those plugins. A more complete file editing functionality would give those plugins wings. In my case dbfolder

• file.editField(key)
• file.editInlineField(key)
• file.editTask(position?)
• …

[Mat]

Hi, this was very inspiring thanks!

I think that with the last version (0.3.6) Metadata Menu now covers most of it, even if the UI is still not very easy to use from my point of view (i’m not a UX design expert and I’ve built the plugin the way I use it on a daily basis):

1. this is the purpose of fileClasses. Even if I’ve put some features to facilitate the creation of new fields, the rendering of the fileClass itself is very raw (JSON). Though it’s not meant to be used directly, but I would be happy to hear suggestion about how to better manage them

2. It’s the purpose of supercharged tags and fileClass queries: you can associate fileClasses (1) with tags or with files matching a specific query (a folder path for example)

3. since 0.3 there are several way to insert a field in a note: from command palette, from a dataview table, from a dedicated api method, from the note’s fields’ modal, and now from the fileClass view

4. This is what I’ve added in 0.3.6 : the fileClass view

I do agree with @RafaelGB. I think that these database features shouldn’t be the main focus of obsidian, and that fields editing shouldn’t be the focus of dataview. Those team do great job at building tools and API that are robust and fast, increasing complexity would put their core product at risk imho.

In addition to what Rafael has said about improving the API, i would like Obsidian and Dataview to work together on the indexing capabilities. If we want to build powerful but efficient plugins, we shouldn’t re-create indexing in each plugin, this will lead to slowness and reduce the user experience with vaults growing in size and plugins adding more and more features.

That being said, this all markdown-files-based-database topic is still green field and it’s very fun to work on it and see all those great ideas and features popping every day

[WhiteNoise]

@Mat

In addition to what Rafael has said about improving the API, i would like Obsidian and Dataview to work together on the indexing capabilities. If we want to build powerful but efficient plugins, we shouldn’t re-create indexing in each plugin, this will lead to slowness and reduce the user experience with vaults growing in size and plugins adding more and more features.

I agree that the first and most important step is to have Obsidian Core provide the backend indexing and API to read/write metadata, beginning with what we already support: YAML.

[TfTHacker]

What you all say about API access to the metadata is on spot. However, this is not what this original post is about. It’s about a type system, a way of defining those metadata elements. The two plugins in context here, Metadata and DB folder, user two different systems for accomplishing this.

While it’s cool what is accomplished, long term having a common type system for all plugins would be valuable for all plugin devs and for Obsidian devs as well.

[WhiteNoise]

An API to access YAML metadata is available in v1.1

[gmich]

First and foremost, thanks for this amazing app!
I would like to suggest some features that could help make Obsidian better and more well-organized for users and devs too.

My suggestion comes from Dendron which has some amazing features but its capabilities are limited compared to Obsidian.

1. Vault schema using just YAML

Users can create a vault schema (for folders and notes) by using just a plain YAML schema

pos:
a. easy for users to create/refactor/delete a vault’s schema using some functions (actions in the command menu)
b. users can create their own vault schema by creating a YAML schema on their own [very useful for non-tech guys, which is a major problem we saw with the dataview (lot of people prefer Notion for that reason)]
c. easier for devs to have a common framework to dev more plugins

2. in addition to (1), there could be a second YAML schema just for metadata**

Anybody, even the most basic computer users, could edit their metadata schema in which they can edit their preferred metadata field (and predefined values, if they will) easily without any special tech knowledge

pos:
a. ready/user-created metadata fields based on the type/name of the metadata schema which then users could apply in their notes. This can be done by a “marriage” of the two YAML schemas (vault + metadata schemas)
b. when a user creates a specific type of note (ie. course), he/she can select(or automatically be applied in all course children) which metadata schema/as wants to be applied in all courses notes and furthermore, based on the metadata schema which the user selected, a relevant template could be applied too.

---

Another cool future (may some Obsidian users don’t need it but I leave them here as for further thinking)

3. dot naming convention

  ie. dev.prog-lang.python.loops
       dev.prog-lang.python.functions

pos:
a. you can get rid of folders and just work fast with files only
b. easy way to search your notes along with a fuzzy search feature in the search engine

With the all above, it would be very easy for anybody to create, maintain and manage their notes. Also easy to apply metadata fields and templates to the notes/child notes etc based on the schemas that they will have created and choose to be applied in individual notes or in a particular “family”/type of notes

---

Imagine just choosing “course” metadata schema to be applied to “course” (vault) schema and then every metadata field be populated with the relevant metadata without having to use third-party plugins (buggy sometimes/errors after updates etc) or writing them manually and most important easily done from everybody without losing yourself between plugins, docs etc.

[gmich]

Example of a bigdata schema:

very easy to combine two schemas for a specific topic!

[AP2023]

Use case or problem

With the character — I can add properties (metadata) to my notes.
It’s not really structured data, though.
I would like to be able to add real “classes” to the notes: with a list of properties of a certain type.

Proposed solution

I would like to define a class and assign one or more to my notes. With the ability to automatically assign certain classes to all notes I create in a certain folder.

If in the future I decide to add new fields to the class I would like the new field to be assigned to all the notes I had assigned that class to.
I would also like some of the types to be classes themselves. And that one class can extend another (inheritance).

For example if I am defining a person in the future I can decide to add his address divided into city, street and house number, then to create an Employee subclass that extends Person by inheriting all its fields and adding more.

If a class is used as a field it will also need to be defined how to show its data in a row or as a table (toString).
Let me try to explain better: when I say show as a table I always mean in the view within the note but the class field is divided into its component properties.

Current workaround (optional)

Currently I can create a metadata template and manually assign it to the new notes, but if I edit the template I have to add the new fields to all the notes by hand.

I can link a file among the metadata, but it is shown as a link and not as content, and this would help for the Address class.

Related feature requests (optional)

I would also like to be able to define a folder so that all notes I create there have a certain template.