Add support for link types

Having read through these interesting posts, I decided to throw in my thoughts for consideration.

A few days ago I came up with the same link-type method described by @usergenic here:
https://forum.obsidian.md/t/add-support-for-link-types/6994/20

Then I came upon this thread of discussion. As I was reading through it, I became more and more concerned about, simply put, complexity. I live and breathe by a foundational Unix philosophy:

The only universal interface is text

This is a deeply profound statement, all the more impressive given that it was proffered decades ago during the birth of information theory and well before we had to deal with deprecated file formats and the like; they saw the future.

Personally I like to store my most important information in the most foundational format possible (text) using the most foundational tools possible (Unix utilities, which I believe hits the 50th birthday next year). I’ve lived long enough to see software come and go, file formats come and go, and indeed entire languages come and go. What, then, is the best method for information representation over the long haul? I’ll get to that. But first, my concerns with the conversation:

  • Layers upon layers of software complexity
    Reading through the list of products mentioned in this discussion, I became concerned with the additional layers of software complexity and software dependencies. Which one of these products will be around in 10 years? Because my notes will still be there (23 years of notes and counting…) but I would never want to have to worry about maintaining an entire software stack to make them useful. My notes have to work for me, not the other way around. Which is a nice segue to my next concern
  • Littering notes with information expressed in outdated languages.
    There’s a reason why the legal profession likes Latin terms, and it’s not to confuse people. It’s because dead languages don’t change. In the law, it’s important that a contract written 50 years ago is still unambiguous 50 years hence. Therefore the law uses certain Latin terms to protect against ambiguities created by changes in word usage over time. I am seeing in this thread more and more complexity suggested, via language specifications that could go by the wayside in the future. Do you really want to have to go through a language translation on thousands of notes later? It’s more likely you’ll be hoping someone else wrote a utility to move those specifications into the Next Great Language, and you’ll be on that treadmill to the grave.

To each his own. Returning to my original rhetorical question:

What, then, is the best method for information representation over the long haul?

My notes are designed for the following model:
Inforep

My notes are foundational. I represent information using my own methods, so they will never have to change. The important thing is that my methods represent an unambiguous grammar. Since they’re my methods, I understand them well and can extract information when necessary. I use external (3rd-party) tools for visualization and analysis. It only requires that I create an interface between the notes and the tool. This is typically just a script. If I want to change tools, or add another tool for some new, cool visualization, nothing about my notes will have to change. I just need to extract the relevant data via a new interface script. We can call the scripting the “Middle Tier.” By separating the information from the tools by a Middle Tier, the information remains unchanged.

Addressing “my” methods

It’s ok to adopt some existing specification; you don’t have to roll your own. The key questions to ask yourself are:

  • "Can I live with this forever? (Think in terms of information representation, extensibility, and parse-ability.)
  • Will I ever have to go back and change hundreds or thousands of notes?
  • (Addressing parse-ability) Using these representations, do I have the skills to extract my own information?
  • If this information has to survive me, while my family may know nothing about my toolset, can they at least read and understand the notes?

Don’t fall for the siren’s call of the concept of One Tool to Rule Them All. Searching for the Grail might be more successful. I prefer a modular architecture with an everlasting foundation.

6 Likes

This is an insightful message, @datum . Thanks!. It sparkled a lot of thoughts in my head.

I really believe that to write personal notes, we should use familiar language, simply because we are the future consumers of our data.

But if we want to use a cool application that helps us exploit our notes better, we will need to build or enable this middle tier.

Also if we’re going to share our messages with others or work together writing notes. Look at Wikipedia, for instance. They agreed about minimal stuff to communicate.

I agree with what you say; probably, there is no ‘ultimate’ representation method.

On the other hand, some get adopted better! Look, for example, Markdown itself; nobody would prefer to author in HTML anymore! It provides a better experience, less friction, is less ugly, etc.

I believe that the most vital thing is to embrace diversity, accepting a variety of representation methods.

We can experiment :slight_smile:

Absolutely @cristian we can experiment. If I wasn’t willing to do that, I wouldn’t be looking at Obsidian! I would agree that a standard is useful for the purpose of data interoperability.

The funny thing is, I never intended to develop my own methods. Years ago I was looking for the best platform for my important notes. In the meantime, I decided that I would just store them as text because it should be easy to move them into some future tool. At the time I didn’t realize how foundational that decision would be. I continued to search for tools, and along the way I kept adding refinements to my system of note-taking to capture necessary information. A few years ago, I started questioning why I was looking for something better when I realized that my text was starting to look like a rich representational system. I was also developing my own notational system for basic formatting before Markdown came along, because I wanted to write a CGI-based web application for searching / rendering my notes however I saw fit, without having to write HTML into my notes. I had planned to integrate GraphViz to render my graph queries. I have a lot of experience with that at work, so it was a natural choice for me.

There are certain things that Obsidian will never do because they’re just outside the scope of what Obsidian is trying to be. For example, if I had a semantic network for characters in a book, I could easily use Obsidian to answer the question, “Who are James’ friends,” because his friends are a direct link to James, therefore all I’m really asking is, “What are all the outbound links of type “friend” for James?” I can do this in Obsidian. However, if I wish to ask, “Is there any relationship between James and Howard,” the best I could do in Obsidian is eyeball the graph, which isn’t very productive past a certain number of nodes. I’ll have to go outside of Obsidian. I’ll need to do a reachability analysis against the edges, to see if a route exists. Luckily it’s easy enough to extract the edges from the vault. Could a plugin provide this feature in the future? I can’t plan for what might be, because it also might not be. I’m at a point where I need my notes to really work for me. I’m willing to pay for good software, but it has to be around for the long haul, or at least allow me to “pull that parachute cord” and take my notes elsewhere easily.

If we want Obsidian to be successful, the solution is modularity and ease of use. I don’t think it’s entirely there yet, but it’s coming along. When I first looked at Obsidian, I wanted to change the font in edit mode to be monospace because it drives me crazy when I’m formatting something like a Markdown table, and things don’t line up for no good reason. Since I’m not conversant in CSS, the whole experience was as if I walked into a coat store, asked to buy a fur coat, and was handed a bow & arrow. If Obsidian is to truly be successful in a big-market way, it needs to be more than an esoteric tool for the technorati. In my opinion, Microsoft didn’t get it’s huge success by writing the best software, but rather by writing software that everyone could use. This is what makes the difference between a small beautiful pebble that in the end gets washed down a stream into oblivian, and a large immovable boulder that could be Obsidian.

4 Likes

This would be a very useful and simple way of labeling connections. I’ve always wanted to add labels to some connections because it would add so much clarity to some graphs. I’m trying to create and refine concept maps for studying, like I can with Cmaptools, but using my linked notes database.

I would not label all my connections because the labels could also restrict thinking, but they could help to highlight the core concepts of a topic.

5 Likes

@ringelblum @soma Now I’ve gotten myself confused. I had to remind myself that this topic is about what we want link types to look like as a supported feature in Obsidian, not how to do it in Obsidian as-is. For now I’m thinking of using the [[TargetNote|:linktype]] notation. The colon distinguishes it as a link type to differentiate it from a regular link rename.

What’s interesting about @soma’s suggestion is that he’s defining a triple, and therefore the full relationship, in a “3rd-party” note. This is distinctive from a notation to simply link the current note to a target with a link type. These are two very different things. Currently Obsidian graphs reflect relationships between notes implied by reference. Whereas this notation expects Obsidian graphs to reflect relationships explicitly defined by notes. And the grammar could be tricky:

NamedEdge -> Link Words – Predicate -> Link
Link -> [[ Words ]]
Words -> Word Words
Words -> Word
Words -> Lambda?
Predicate -> Words

(This grammar isn’t completely correct because it allows for blank note names and blank predicates, but was shortened for clarity, if you can overlook that.) This will parse, but there’s a lot of room for error. for example:

[[Berlin] is the – capital of -> [[Germany]]

If “[[Berlin]” is not seen as a link, then the “capital of” link will never be defined. Obsidian isn’t trying to be a compiler; if it doesn’t see the first link then the rest is just free-form text, and there is no error, and you may never know the link wasn’t defined. Worse, it means seeing [[this notation]] in two contexts. So if a syntactical error leaves the named-edge undefined, it also means that a link from this defining note to [[Germany]] is created, which wasn’t wanted. I’m not suggesting that other methods cannot similarly fall victim to such a mistake, but it speaks to the need to keep the notation as concise as possible. And also there would have to be a decision regarding Obsidian continuing to only recognize relationships between notes, or include relationships between notes defined by other notes. Personally I don’t see this happening due to the ambiguity created.

1 Like

I take notes for 12 years now. I also decided to use plain text after moving from one tool to another more than once.

Sometimes I moved from one tool because it became too slow, sometimes because it was aesthetically unpleasant, etc. I moved into Obsidian because it offered to autocomplete the ‘[[ ]]’ and the other didn’t.

If you use text, you are not ‘married’ to any tool.

Along the way, I developed my super primitive symbolic system with little cues that I could remember and type in a search box or a grep command. To look at all the notes of my friends, I search the phrase ‘is a friend’, and If I want all the addresses of my friends, I search for ‘is a friend’ and ‘has address’. I’ve seen that many people in this community do this with #tags .

I only write graphs because they force me to think better about things; to go for coherence and completeness.

I need to look at the graph and to know what the links are about

1 Like

Since we control our notes, programs like Semantic Markdown (v0.3.4): Visualize and Query your Obsidian vault in Neo4j! of @emile can be used to extract information from our notes and write a graph.

Information could be extracted directly from the Markdown, such as tables, mermaid diagrams, other graphs’ ‘dialects’ etc. This ‘middleware’ can have several outputs, different graph dialects, csv, tables, etc. Or the omnipresent ‘Excel’.

Or perhaps there is something already there that could be reused or built upon? I discovered https://unifiedjs.com/ the other day.

When I write notes, I try to think about how they will be processed years from now. I believe that something that generates structure data is valuable to be used by the current tools or that AI program of the future that will surprise me in the next years, who knows!

This “single” Unix command will extract links from a vault, and reformat them in a form that can go directly into GraphViz:

egrep -roI ‘\[\[[^]]+\]\]’ VaultName|sed ‘s/^/"/;s/.md:\[\[/" -> “/;s/\]\]/”/’

Should also work on Mac, but I didn’t test. I only spent about 60 seconds coming up with this so there might be edge cases I didn’t consider. This is what I love about text, and what I love about Obsidian. It doesn’t tie up my data in a format I can’t use elsewhere easily. Easy extraction of links and their types (I didn’t process for types in the above command) means I can externally answer questions like, “How is A related to B?” Then just apply a route-finding algorithm that will output each possible route and the association types that get us there. It would be great if I could tie that functionality into Obsidian, but I’m not holding my breath. Besides, Obsidian doesn’t render self-referencing links (notes with a link to itself) so I can’t completely rely on the visualization. Still, as a tool, Obsidian is compatible with my way of working with notes.

Edit: Escape sequences disappeared, fixed.

Hi @cristian it looks like we have some philosophical alignment on how we handle notes. I avoided moving my notes to different tools only because I never committed to a tool while I continued to research options. There are many great tools out there by which we can build workflows of knowledge management. What I love about Obsidian is that it is aligned closely enough with how I already structure my notes, that moving to it is easy, and moving away from it if necessary is also easy. There’s an elegant power in that freedom. I look forward to what comes next for Obsidian. Hopefully we’ll get named-links. This will provide the context to walk graphs and ask deeper questions via a plugin.

This is a fascinating discussion. I am a neuroscience researcher, and I’ve dreamed of being able to have a system that I can input research findings and use graphs to make novel insights and inferences.

I love the flexibility and futureproofing of plain-text notes. But sometimes it would be nice to have the functionality of a more formal database to integrate pieces of data.

There is a tool that does this (based on Neo4J)

But I’m reluctant to build on someone else’s tool.

The ability to specify link types in plain text with Obsidian would allow one to build such maps of interactions between notes in powerful ways. I hope this thread and accompanying tools continue to develop.

This discussion might also be of interest:

1 Like

Hi all. For the Neo4j Graph View plugin, I want to support a better way to represent link types. I’ve been writing a bit about this here https://publish.obsidian.md/semantic-obsidian/Link+Types

I’d love to get some input from the people in this thread, and to discuss together with the goal of finding a good Markdown specification for link types and link properties. I’ve opened a Discord for the plugin, where we’re having a discussion about exactly this topic: https://discord.gg/sAmSGpaPgM . Feel free to join :slight_smile:

I’m reticent to put my data into yet another tool. I like where it is, in a folder of my hard drive :stuck_out_tongue:

Nowadays I miss something that helps you to draw small maps of interactions between notes. A complementary tool for thought, not meant to discover new things but to synthesize, simplify, organize, and merge content.

Perhaps we can imagine a simple way to draw these small maps?

Some use mindmaps, that can be embedded in Maps of Content.

I personally would like a dialect to embed this info, specifying the nodes positions, colors, everything aesthetical that matters or help us to think clearly

```think-map

In 1957 Walt Disney drew the media empire he was trying to build.

He didn’t have a computer to draw connections through…

Proposal to extend the existing link syntax to support user-defined labels on edges in graph view.

Issue

Links in and of themselves provide no meaningful context therefore context must be supplied through the written word. This works when reading the note directly but when viewing the graph view these transitionary contexts are lost – it is not clear from the graph view why a link exists, only that the link exists.

This is a fundamental weakness in all current note taking systems and is a weakness of the web itself.

Benefits

Allowing the user to define labels for edges will make it easy to see at a glance in the graph view the nature of all relationships between this note and the linked notes. Users can quickly “surf” from note to note via the graph view if desired and then “dive in” to a specific note.

Unlike the web there is no need for a centralized canonical registry of labels with agreed upon meanings – users are free to use arbitrary labels with arbitrary meanings.

Labeled arcs also approximate a mind map of the user’s notes. All in the filterable graph view.

Syntax

Given a link [[notename]] the syntax can be extended as follows: [[[email protected]]].

@ is chosen as it represents labeling the arc between the nodes.

Ex: [[Teach cats how to build giant [email protected] idea]]

Users could even use emoji labels for shorthand:

Ex: [[Teach cats how to build giant [email protected]🙀]]

Possible labels

Possible labels may include: (but are limited only by imagination)

  • is a
  • has a / has N
  • supports
  • opposes / contradicts
  • works well with
  • do not use with
  • elaboration/amplification (as in thought A evolved into concept B branched into two more lines of thought…)
  • owns / owned by
  • works in / works on / works with
  • good idea
  • bad idea / dead end (for those of us who want to keep the bad ideas and warn ourselves not to go that way)
  • :heart:
  • :bulb:

Display

Display the label as small text alongside the arrow, with a selectable CSS class so it can be customized by themes. Default to the same color as the arrow.

Consider also creating a second custom class for each label, containing the name of the label itself, for additional CSS customization by the user/theme.

Ex:

Given a link [[Teach cats how to build [email protected] idea]] generate something like this:

<div class="graph-arc-label graph-arc-labeltext--great-idea">great idea</div>

allowing users to write custom CSS like this:

/* all hail the cat overlords */
.graph-arc-labeltext--great-idea { 
    color: red; 
    background-color: yellow; 
}

so the brilliant idea is more prominent when viewing the graph.

This also helps encourage more focused note taking, since transitions can be clearly and easily established in the link itself.

2 Likes

Hi Dave,

I agree with some things you say, and also the way we can customize items with CSS. (I would love to use something like this to customize nodes in the graph, with things like images).

Regarding the notation, I think it’s too aggressive because it can break compatibility with wikilinks.

So for example:

[[[email protected] idea]]

It would be interpreted as ‘[email protected] idea’ by some apps and ‘tchtbf’ by others.

I wish markdown had something to embed ‘invisible’ metadata.

I differ with this part:

The ‘Semantic Web’ that is already part of the Web! :wink:

Hi @cristian, as someone who was heavily into the Semantic Web work ca 2000-2003 I would agree, if it had actually developed into something useful and widely used. The fact that it did not indicates it was an ineffective concept, thus the web as it exists today has a fundamental lack of this capability.

@davecan , I differ.

Just one example:

When you search for ‘shoes’ in google, you get an info-box with structured data.
This data was gathered processing RDF embedded in the HTML pages.

This is widely used.

However, I agree that it failed as a concept. I believe we cannot represent 100% reality. But that’s another topic, I guess.

Adding my vote here that the externalized triple format proposed by @datum while interesting is absolutely not something I would use. My entire workflow relies on fluidity and I’ve gotten it (in an extremely short time – couple weeks) to a point where when I’m creating and linking notes it feels more like I’m interacting directly with the notes physically rather than working through Obsidian. Obsidian in that sense just melts away a little bit.

Expressing link types directly at the point of link definition is the way to go IMO. Requiring an external note to create the relationships through the triple format is friction that will almost never be used by me, and I would suspect others as well, because it interrupts the flow and forces us to stop thinking about our notes and start thinking about Obsidian again.

The colon-based format seems useful to me, but perhaps it should become a separate delimiter rather than using it after the pipe, since the pipe is already used to separate the file name from the display name which can be autocompleted from the alias front matter. I used this a lot since I prefix my notes with ZK-style UIDs.

Possible syntax alternatives:

  • [[the note:label|link text]]
  • [[the note|link text]]:label
  • [[the note->label|link text]]
  • [[the note|link text]:label]
  • [[the note|link text]->label]
  • [[the note|link text]]->label (risks collision with existing notes, but maybe not a huge deal)

I quite like the -> syntax in the third bullet…

@davecan, if you’re referring to my 12/19 post, it wasn’t my idea and I wasn’t supporting the notion. Actually I was explaining why it isn’t a good idea. I finished with:

Personally I don’t see this happening due to the ambiguity created.

I wish markdown had something to embed ‘invisible’ metadata.

At least for reference-style links/images in MultiMarkdown, one can add attributes to links and images in the link reference information, e.g.:

[link]:  http://path.to/link.html "link title" class="attribute1 attribute2"

These attributes won’t show up in a typical Markdown preview but can influence the rendered output, and could be used for semantic purposes.