Wikilinks in YAML front matter

Feature archive
, ,
1

Mod-Edit: I renamed this thread to focus on wikilinks in YAML frontmatter. The other part of the title “Notion-like databases from metadata” is already possible with Dataview, Database Folder, and all related plugins.

First, I’d like to thank all contributors to Obsidian for making the absolute best knowledge management application!

Over the years many applications never quite delivered what I was looking for and ideas of a better system started accumulating in my mind. The most notable idea being able to link anything to anything and use those links as tags themselves. Notion almost made this possible, but not completely. Then Obsidian appeared, turning this dream into a possibility!

As I’ve become acquainted with Obsidian I’ve realized that many of my notes are of a particular type or kind. Let’s use Movie for example. For any note that’s a movie I can just add a [[Movie]] link to the top of my file and BAM, that link itself serves as a tag for all movies.

However, all notes that have an “is a” relationship to movie also happen to share a standard set a metadata such as director, run time, IMDB rating, etc. The obvious solution would be to create a template for Movie that I can apply to all “is a Movie” notes. This does begin to solve the problem of keeping all of the metadata standardized, but as you will see some problems still remain.

My next thought was for this metadata to be parsable in a way that I or someone else could build a plugin to view all notes that are a movie in a table just like a Notion database with a standard set of properties. Then I found YAML support for Obsidian which seemed perfect! That is until I realized links do not yet work in YAML front matter. Therefore links could not serve as the metadata values to be used by plugins.

Having the metadata with links for a movie note would look something like the following:

---
is a: [[Movie]]
director: [[Ridley Scott]]
run time: 2hr
IMDB rating: 7.8
---

Such metadata formatting would allow plugins to be built that create Notion-like database views (table, cards, kanban) and at the same time maintain a rich backlink ecosystem. The last thing I want to have to do is add duplicate metadata in YAML and my own block below it.

I’m certainly not the only one that uses links as tags. It seems to be quite a popular trend. Another popular trend for many users seems to be adding links to the top of their notes as metadata that’s nearly identical to YAML front matter. This is a very common pattern. Also notice that this pattern is already very similar to the pattern of Notion Page properties. Perhaps we see some standard design patterns emerging for knowledge management systems.

TLDR

I deeply desire the option for YAML metadata to be universally link compatible. I want to build, or see built, plugins that can take advantage of YAML metadata that are also links, while at the same time Obsidian also observes them as links in the graph.

I propose that the syntax could be something like the following and just use the standard link formatting already used:

---
customProperty01: [notalink, randomstring]
customProperty02: [[[isalink]], [[another link]]]
customProperty03:
- notalink
- [[isalink]]
links: [NoteB, NoteC] <-- Similar to "tags", but creates links.
---

Related feature requests (optional)

130 Likes
2

I am copy/edit here where we stand currently from the archived thread. Then, I’ll add some remarks.

You can add your custom attributes in the YAML frontmatter but:

  • They MUST be in proper yaml format.

  • If an attribute is not handled by obsidian or plugin, it will be ignored. But still has to be YAML format.

  • The YAML frontmatter is not scanned when searching for unlinked mentions (and probably never will be). That is, if NoteA has this frontmatter:

---
connections: [NoteB, NoteC]
---

NoteA will not appear as an unlinked mention in NoteB and NoteC.

  • There is no support (yet) for adding a linked mention to another note, that is, if NoteA has this frontmatter:
---
links: [[[NoteB]], [[NoteC]]]
---

It will not work. NoteA will not appear as a linked mention (backlink) in NoteB and NoteC.

If at some point we add support for linking other notes in the YAML block, it will be specific and limited to a YAML attribute, like related: . But it still won’t be possible to add linked mentions in other attributes.

Remarks

Thare is an interesting dichotomy between the YAML block and the rest of the text.
The YAML block is meant for database-like approach: highly structured data that is handled by specialized software.
Obsidian, and tools like it, were born under the idea of using unstructured plain text data (or not so structured) and rely exclusively on connections and search/filters.

You can solve many productivity problems using either approach. Some workflows look cleaner with one or the other.

Now back to your proposal, you can either 1) write a plugin to explicitly handle your movie data or 2) put it in the main text, perhaps in block right after the YAML block, and rely on Obsidian linking engine.

---
# YAML BLOCK
---
# Your Block
___

(Edited Example)

13 Likes
3

IMHO the proposal of having another block after the YAML front matter block is just a bandage on top of the problem and not a real first principles solution. Problems that are addressed by applying bandages just create compounded problems in the future.

If YAML front matter is intended to serve as the metadata for a note, then we shouldn’t have to resort to other methods to add metadata because of some incompatibility. Resorting to a second block below YAML will cause that metadata not to work with other plugins because it is no longer part of a standard. I absolutely want any and all metadata that I add to every note to work with any plugin regardless if the metadata values are links or not.

Additionally, if backlinks are a core feature and value proposition of Obsidian, then they should be first-class citizens in order to maintain flexibility and give more power to the user. That’s what I love about Obsidian, it’s extremely flexible with just the right amount of structure that allows for that flexibility.

It seems that a very simple solution would be a setting to let the user choose if Obsidian recognizes backlinks within YAML. Also, I personally don’t care to have Unlinked mentions recognized from YAML, but perhaps this should also be configurable and up to the user.

I do understand that the backlink syntax might cause some unforeseen issues with YAML syntax, but this seems easily resolved by wrapping all YAML backlink values in quotes. Maybe I am ignorant of other issues and reasons for not wanting to support backlinks in YAML and would love to hear more thoughts on the matter.

16 Likes
4

I hope Folder View like a table of front matter contents.

for example:

| note name | aliases | tags | updated time |
| --- | --- | --- | --- |
| [[noteA]] | alias-a1, alias-a2 | tag-a1 | 2020-12-17 13:27 |
| [[noteB]] | alias-b1 | tag-b1, tag-b2 | 2020-12-17 13:40 |
| [[noteC]] | alias-c1, alias-c2, alias-c3 | tag-c1 | 2020-12-17 13:50 |
| [[noteD]] | alias-d1, alias-ad | tag-d1 | 2020-12-15 13:10 |
2 Likes
5

YAML stands literally for YAML Ain’t Markup Language, and just that says a lot about this topic.

As you can read in its specification, YAML is based on three basic primitives:

  • Mappings (hashes/dictionaries).
  • Sequences (arrays/lists).
  • Scalars (strings/numbers).

Everything that doesn’t respect this premise will potentially break most of YAML parsers, and even though this might be implemented, it will definitely break compatibility with other programs.

Apart from that, a related tag can be useful in some specific use cases, +1

2 Likes
6

(1.) @pyrotecx What do you mean by

backlink formatting already used

and/or

backlink syntax

?

( In Obsidian pane which shows backlinks for current note, I see just quotations/transclusions of pieces of text that contains filename of current note. Either inside link markup [[]] or []() for linked mentions or outside / anywhere else for unlinked mentions. )

(2). Links in YAML were also discussed in thread about support for link types:

7

Are we able to add our own YAML block after the first YAML front matter block like you mention here…

@WhiteNoise said:

Now back to your proposal, you can either 1) write a plugin to explicitly handle your movie data or put it in the main text, perhaps in block right after the YAML block, and rely on Obsidian linking engine.

---
# YAML BLOCK
---
# Your Block
---

When I try this, Obsidian renders my block as an H2 header in both edit and preview mode. Not sure if this is a bug or intentional.

8

add a blank line or use another marker _ or +

1 Like
9

@malecjan, when I said

backlink formatting already used

and

backlink syntax

all I meant was the wiki style bracket syntax [[link]] that Obsidian already uses for links.

Also, thanks for the reference to Add support for link types - #31 by malecjan. It inspired me to refine my original idea of including the [[link]] syntax within YAML. I think that including the bracketed [[link]] syntax within YAML is messy and unclean. Imagine a YAML sequence of links like so:

---
is a:        [[Movie]]
director:    [[Ridley Scott]]
run time:    2hr
IMDB rating: 7.8
genres:      [[[crime]], [[drama]], [[mystery]]] <-- fairly messy
---

This array of links within YAML could also be parsed as nested array’s containing a single item.

I think I much better approach that aligns with how @WhiteNoise said YAML link support will probably be added to Obsidian is to use a YAML anchor and alias like so:

---
is a:        &l1 Movie
director:    &l2 Ridley Scott
run time:        2hr
IMDB rating:     7.8
genres:      &l3 [crime, drama, mystery]
related:         [a link, another link, *l1, *l2, *l3] <-- Obsidian recognized links
---

The former YAML would be equivalent to the following YAML:

---
is a:        Movie
director:    Ridley Scott
run time:    2hr
IMDB rating: 7.8
genres:      [crime, drama, mystery]
related:     [a link, another link, Movie, Ridley Scott, crime, drama, mystery] <-- Obsidian recognized links
---

Additionally Obsidian should support a custom YAML tag such as !link. In the following example links would be parsed equivalently to the previous two examples.

---
is a:        !link Movie                   <-- Obsidian recognized link
director:    !link Ridley Scott            <-- Obsidian recognized link
run time:          2hr
IMDB rating:       7.8
genres:      !link [crime, drama, mystery] <-- Obsidian recognized link
related:           [a link, another link]  <-- Obsidian recognized links
---
6 Likes
10

Vote+1

2 Likes
11

Sorry @pyrotecx, I just want to clarify:

What is &l1? Is that a tagging feature already built into YAML?

If I understand correctly, under your proposed syntax Obsidian will only recognize a link if either:

  1. it’s included under related: or
  2. it has the YAML tag !link
    If neither or those conditions are met then it will just be metadata, but not an Obsidian recognized link?
1 Like
12

Perhaps this might need to be a separate feature request, I’m not sure but:

What if each custom property like director: or genre: could optionally have it’s own Obsidian note automatically created. This could have the following benefits.

  1. When editing YAML front matter, Obsidian could suggest code completion for us.
    • For example: While typing gen, Obsidian could overlay a list of possible completions like genre: and general: etc.
    • Advantages of this approach:
      • Code completion helps us not make accidental duplicates when we misspell.
      • faster typing
      • we’re already familiar with code completion when we type [[ and Obsidian suggests notes for us
  2. When we add an attribute to director:, Obsidian could auto add backlinks.
    • For example:
      1. Obsidian could auto-add the Ridley Scott note as a “child” of the director note.
      2. Obsidian could auto add is a: director to the YAML front matter of the Ridley Scott note.

This way we can make simple changes in one place and have related changes propagate out to other places automatically.

6 Likes
13

This is great idea/workaround/solution for extending links into whole yaml front-matter when recognition of links in the field “related” is implemented. I would consider incorporating it into main post or at least point to post #9 from the main post.
Additionally, for readability in basic text-editors, we can use yaml comments to materialize text locally near YAML references.

---
is a: &Movie Movie
director: &l2 Ridley Scott
run time: 2hr 
IMDB rating: 7.8 
genres: &l3 [crime, drama, mystery]
related: 
  - a link
  - another link
  - *Movie
  - *l2 # Ridley Scott
  - *l3 # [crime, drama, mystery]
---
3 Likes
14

@Fanshu , @luke85
To vote for the feature request, do not forget to press :heart: under original post, officially prefered method of voting according to FAQ:

Rather than posting
“+1” or “Agreed”,
use the Like button.

@DandyLyons Yes, these are yaml anchors and yaml custom types.

2 Likes
15

@pyrotecx
Thank you for clarification of what you mean when you say “backlink”.

  • When a page “P” contains [[A|Link to page A]], then From viewpoint of page “P” it is called just a “link”. It is called “backlink” only from viewpoint/perspective of the page “A”.
  • Obsidian is said to have “backlinks” functionality because it is able to show which pages do link to currently active page “A” (via “backlinks pane”).
  • I suggest to edit the original post to change relevant instances from “backlink” to “link”, to avoid confusion of readers (especially the novices) due to misuse of terms. I have encountered this confusion several times in recent times since introduction of Roam Research. I imagine how easily the self-strenghtening misuse of (now cool) term “backlink” can lead to messy discussions and consequently to the need for new word (e.g. “backbacklink”) in order to express/distinguish the original meaning from potentially new common interpretation of word “backlink”.
  • For example, part of the title will change to Allow links in YAML front matter; which explains the intention immediately without need to decrypt anything and so potentially attracts more votes from people who do not have time to read the whole post.

Edit: thank you for making the corrections.

6 Likes
16

Hello all,

Thank you for Obsidian.

TL;DR;

Here’s a simple suggestion for the problem of links in YAML:

  • Treat all -almost- YAML scalars as potential labeled links (implicit, without any special link syntax)
  • This would include those scalars that are deeply buried within nested YAML structures

This may sound awkward or even stupid at first, but please bare with me…

In a freeform mostly unstructured context (like markdown), we obviously do need a special syntax for links and the like.

In a structured context (like YAML), however, information is already entered and than parsed in small chunks.

Quite often, those chunks refer to other entities (whether they aready exist as an note entry or not).

Most of the time, even things that seem as pure attributes (like dates, e.g.: “2021-03-16”) have the potential to become first class notes (entries) some day: if the user so desires.

This kind of approach (or something equivalent) would allow Obsidian to bring together the best of both worlds (structured and unstructured approaches mentioned by @Whitenoise) for managing a knowledge base on any topic.

Any entry may start out as an unstructured freeform markdown note…

With time, as the need arises, structure could emerge… naturally and gradually… all within the same “entry” (file).

EXAMPLE

Below is an example that illustrates both a use case and the workings of above solution.

The demonstrated example happens to be for a hobby knowledge-base for plants, but the suggested solution is entirely generic so it could be applied to any use case involving semi-structured granular knowledge-base on any topic (movies, books, articles, …, anything).

YAML Front matter (and then some)

---
title: Spinach
author: John Smith
tags: [“gardening”, “plant/vegetable/leafy”]

source: 
  - {name: Wikipedia, link: "https://en.wikipedia.org/wiki/Spinach", primary: true }
  - {name: Wikimedia, link: "https://upload.wikimedia.org/wikipedia/commons/c/cd/Spinach.jpg" }
---
type: species
name: Spinach
species: Spinacia oleracea
lifecycle: annual
taxonomy:
    genus: Spinacia
    family: Amaranthaceae
    clade: Tracheophytes/Angiosperms/Eudicots/Caryophyllales
    kingdom: Plantae

…

Followed by the BODY (markdown)

Spinach (Spinacia oleracea) is a leafy green flowering plant native to central and western Asia. It is of the order Caryophyllales, family Amaranthaceae, subfamily Chenopodioideae. Its leaves are a common edible vegetable consumed either fresh, or after storage using preservation techniques by canning, freezing, or dehydration. It may be eaten cooked or raw, and the taste differs considerably; the high oxalate content may be reduced by steaming.

bla bla bla...

Optionally followed by REFERENCE LINKS

[spinach:primary]: https://en.wikipedia.org/wiki/File:Spinacia_oleracea_Spinazie_bloeiend.jpg
[spinach in the fields]: https://upload.wikimedia.org/wikipedia/commons/c/cd/Spinach.jpg

What are the links in the above example ?

All the scalars found within the YAML frontmatter(s) above can be treated as potential -labeled- links:

  • Spinach
  • John Smith
  • gardening
  • plant/vegetable/leafy
  • Wikipedia
  • https://en.wikipedia.org/wiki/Spinach
  • …
  • Amaranthaceae
  • …

It gets better… Those are already labeled links, ready to be represented on a directed labeled graph (or go into a graph database :-), or just generate semantic triples…

For example :

link label(s)
Spinach doc.title, name
John Smith doc.author
Wikipedia doc.source[0].name, doc.source.name
Spinacia oleracea species
annual lifespan
Spinacia taxonomy.genus
… …

The above mapping convention probably needs some discussion and improvements. But it should give the general idea.

How does this approach compare to other solutions ?

PROS:

  1. Brings together the best of both worlds of managing data (structured and unstructured), allowing structure to emerge naturally and gradually.

  2. Generic solution for all kinds of data and metadata, without the need to reinvent the wheel for each and every problem domain (authors, articles, plants, movies, recipes, …)

  3. Totally compatible with existing -and future- YAML syntax, without the need of any custom convention.

  4. YAML data stays clean, void of extra syntactic noise (which might have otherwise been problematic for automated processing of YAML by other scripts that could choke on any special link syntax).

  5. Gives us an easy and natural way of entering labeled “semantic” links (using dot notation or similar)

  6. Probably quite easy to implement without breaking the existing Obsidian code (this is just a guess; I don’t have sufficient knowledge of the code base)

CAVEATS:

  1. Could lead to “link noise”, listing too many things as links or backlinks.

    This risk could easily be mitigated by some furher implementation choices:

    • Ideally, such links could internally be marked as “implicit” and the user could be given the choice of displaying those (or not)

    Also, auto-creation of unwanted entries should be avoided.

  2. Could lead to structrure frenzy… prompting some users to over-structure their data, unnecessarily or just too early.

    Well, when something is so easy (to do)… it takes some discipline to learn not to over-do…

OPEN QUESTIONS :

  • What impact on performance ?

FUTURE POSSIBILITIES (for later on)

As the careful readers have probably noticed, this concept could be extended in may ways differnt later on… enabling many other goodies…

Here are just a few :

  • Basic text templating within the markdown body (something like jinja2 syntax or similar)
    This would help avoid repeating oneself and also minimize errors.

  • Detect URLs and possibly treat them differently

  • More knowledge-base/graph capabilities (or exporting triples into graph databases) thanks to implicit labels that come from YAML

  • Ability to specify labeled links within the markdown body itself, in a way that is interoprable with the described approach (in YAML).

The last two items resemble the #juggl add-in being developed by @Emile (the current version is called Neo4j Graph View)

  • …

Needless to say, care would be needed to avoid feature-creep and software bloat.

Obsidian was intended as an easy to use note taking app, in which it excels. So, it should not evolve into some complicated bloatware.

IMHO, with a few carefully chosen features, it has the potential to fill a huge gap… and still remain simple.

As Larry Wall once coined it: “Simple things should be easy, and complicated things should be possible [to accomplish] :-)”

Thank you for reading this far :slight_smile:
Tabulo[n]

24 Likes
17

I like the idea of using scalars in yaml rather than a second front-matter block, which seems kind of odd.

2 Likes
18

Hm. I do somehow like the concept of YAML frontmatter, but please let me object to this idea.

Why? Obsidian is a real great PKM application, and I enjoy it a lot, but I think we have seen too many “unplanned-for” uses for YAML and code blocks lately. Just because it can be done it mustn’t always be done.

I simply fear too much non-standard uses of code blocks and YAML frontmatter creeping in, with the eventual possibility of Obsidian becoming more and more of a “locked-in” system again.

Let’s take myself: I write a lot of nonfiction and reference material and have traditionally been using YAML frontmatter mainly as kind of “preprocessor directives” for LaTeX output. I fear that if YAML scalars would be made “auto link targets”, it might simply generate to much “digital noise”.

Even a small text here already has something like the following front matter:

---
title: 'Linux für Dich: Electronic Publishing mit Markdown'  
documentclass: 'scrbook'  # book (no abstract), memoir, …; default: article
author: 'Matthias C. Hormann'  
email: '[email protected]'  
editor: 'Matthias C. Hormann'  
publisher: 'Eigenverlag'  
copyright: '© 2015 Matthias C. Hormann. All rights reserved.'  
rights: '© 2015 Matthias C. Hormann. All rights reserved.'  
cover-image: 'images/cover-image.jpg'  
stylesheet: 'supportfiles/epub-style.css'  
css: 'supportfiles/style.css'  
highlighting-css: 'supportfiles/highlighting.css'  
date: '2015-08-07'  
lang: 'de'  
keywords: 'Markdown, Electronic Publishing, epub, PDF, HTML, how-to, Linux, Ubuntu, Mint'  
subject: 'Markdown, Electronic Publishing, epub, PDF, HTML, how-to, Linux, Ubuntu, Mint'  
header: ''  
footer: 'Linux für Dich: Electronic Publishing mit Markdown'  
numbersections: 'yes'  
#geometry: 'a4paper, portrait, twoside, includeall, nomarginpar, inner=20mm, outer=20mm, top=20mm, bottom=20mm, bindingoffset=10mm'  
#geometry: 'nomarginpar, twoside, bindingoffset=10mm'  
date-meta: '2015-07-28'  
pagetitle: 'Linux für Dich: Electronic Publishing mit Markdown'  
title-prefix: ''  
quotes: 'yes'  
crossrefYaml: 'supportfiles/pandoc-crossref-de.yaml'  
bibliography: 'literatur.bib'  
biblio-files: 'literatur.bib'  
#biblio-title: 'Literatur'  
csl: 'supportfiles/iso690-numeric-brackets-de.csl'  
description: 'Linux für Dich: Electronic Publishing mit Markdown'  
#toc-title: Inhalt  
lof: 'yes' # List of Figures  
lot: 'yes' # List of Tables
lol: 'yes' # List of Listings
#abstract-title: Kurzübersicht  
#abstract: |  
# In der Reihe "Linux für Dich!" findest du praxisnahe Tipps und Tricks für den alltäglichen Umgang mit deinem Linux-System.
# 
# In der Ausgabe "Electronic Publishing mit Markdown" wird eine hocheffiziente Arbeitsumgebung für Autoren beschrieben – von der Installation benötigter und sinnvoller Programme bis hin zur #Erstellung fertiger Dokumente als E-Book, PDF oder HTML-Dokument.
# 
# Das Buch wendet sich an Autoren, die unter Linux (Ubuntu oder Mint) arbeiten und sich zielgerichtet möglichst ohne Ablenkung aufs Wesentliche konzentrieren wollen: Das Schreiben – und die effiziente Weitergabe ihrer Texte über verschiedenste Medien. Angesprochen werden Studenten, Lehrende, Ersteller technischer Dokumentationen und professionelle Autoren.
# 
# Ausgehend von einer einfachen und vor allem *lesbaren* Textdatei wird der Weg bis zum fertigen Buch (oder einer Anleitung wie dieser) erklärt – zielgerichtet, ohne Ablenkung, reproduzierbar, möglichst automatisiert bis zum fertigen Dokument in den verschiedensten Ausgabeformaten.
# 
# Es wird erklärt, welche Werkzeuge und Programme man *wirklich* braucht und wie sie genutzt werden:
# 
# * *Geany*, ein Texteditor (es kann aber nahezu jeder andere verwendet werden)
# * *Markdown*, eine einfache Syntax für Textdokumente
# * *JabRef*, Software zur Verwaltung von Literaturangaben und Zitaten
# * *LaTeX*, eine Satz-Software, die »im Hintergrund« für schöne Ausgabe sorgt
# * *Pandoc*, das »schweizer Armeemesser« unter den Dokumentenkonvertern
# * *Calibre*, Software zum Verwalten, Anzeigen und Bearbeiten von E-Books
# * *Freeplane*, eine Mindmapping-Software zum visuellen Gedankensammeln

---

and it also has many code blocks, with the intention of having syntax-highlighted code examples and instructions.

Whatever gets decided upon, please don’t break compatibility for people who use Obsidian to write academic papers, nonfiction and text books intended for LaTeX output.

I’d be interested what others think about this.

7 Likes
19

“non-standard” bells and whistles feature requests keep growing.

Let’s hope most of them do not get implemented.

Simplicity is key, or database-based-apps-with-an-export-to-text-linked-files-with-backlinks will become the better alternative.

Drafts was great. Then, it was flooded with bells and whistles.

1 Like
20

I think this important. After all, what has evolved? ed → WordStar → Word. A monster. Probably that’s why we found back to Markdown. Compatible. Simple text. Easy to use.

Combined with an easy-to-use, blindingly fast tool like Obsidian: Allow to focus back on your own, original ideas. No distraction but thinking and output again.

I’m not against “bells & whistles”. Just against the distraction they can provide. Which often hinders great new ideas to evolve. Even WYSIWYG too often makes people think of cosmetics instead of content.

And that was what Obsidian was meant for. I think.

1 Like