Add support for link types

Feature requests
,
38

Hello Malecjan,

I think one can embed a graph in our notes using snippets


## This is my typical Note

It contains contents in standard Markdown and also snippets with graphs:

```Aspen
default Person, name
default_attribute Employer, company_name
reciprocal knows, is friends with

# Write out the narrative data
(Matt) [is friends with] (Brianna).
(Eliza) [knows] (Brianna).
(Matt) [is friends with] (Eliza).

(Matt) [works for] (Employer, UMass Boston).
\```

More content..

Then we can implement a processor of ‘graph snippets’. Implementation wise, for Aspen, I think this is to invoke their library to generate the corresponding Cypher…

… If we need the cipher we can process the markdown to extract it, or possibly other format. One could even have a universal graph model that helps to go from one format to another.

What uses do you foresee of Aspen? or Turtle embedded in markdown? (I’m curious :slight_smile:

39

Oh, I see, Aspen it’s Ruby.

40

I’m kind of wondering too. Is this to use semantic links between notes? Or just to construct graphs in general?

The turtle syntax sounds a bit cumbersome, maybe? But at least it’s easy to implement :joy:

41

I’m pretty sure that Aspen or turtle would allow us to build a graph, embedded within the text of the markdown

In the same way, we put code in a Jupyter Notebook!. The concept is powerful…but I don’t have a use case right now :smiley:

42

I imagine that if one embeds structured data, one can write factual data, that can be extracted and procesed afterwards. Or you could query your vault, ask complex questions, or do faceted search.

Depends on the person, and what are they using their vault for I guess :slight_smile:

What things can be done with the Neo4J Bloom?

43

@cristian If (Subject)[predicate](object) is standard Aspen’s syntax then this is definitely an option.
I initially dismissed Aspen when looking on it’s page, as just another syntax to clutter the world. But the demo shows the better part of it.
I understand it as a way to integrate Obsidian / simple markdown files, editable by any custom tool, with powerful graph-database querying and analysis. It can interpret texts already formulated in any comprehensible syntax.

Using fenced code blocks to include another syntax in Obsidian files is very good idea IMHO, however If I enclosed everything relevant to block …

```Aspen
... ...
```

… Then I would likely enclose every file in a vault, including other code-blocks and YAML headers which Obsidian would not understand. I might also want to include relationships between files and directories.

If you mean to store code for Aspen in code block and then process data of whole vault based on that, then I do not have any caveat.

The things missing now:

  • Make cooperation with graph-database tools work out of the box via Obsidian plugin.
  • Apply it to whole vault or any selection of notes produced via filtering. The demo shows only application to content of single file.
  • Two-sided interaction, e.g. open note in Obsidian by clicking on corresponding node in Neo4j Bloom, automatically edit metadata (YAML header) of markdown file based on results from Neo4J analysis, or manually edit note e.g. in Obsidian and (instantly) update graph-database in neo4j.

Use cases: many shown in presentations of neo4j, with many built packages for analysis, but applied to bunch of existing personal files.
I do not know if Aspen and neo4j is the best option, I would just like to have such options/tools for knowledge/… management tool and Aspen and Emile’s toolbox go in that direction.

1 Like
44

Having read through these interesting posts, I decided to throw in my thoughts for consideration.

A few days ago I came up with the same link-type method described by @usergenic here:
https://forum.obsidian.md/t/add-support-for-link-types/6994/20

Then I came upon this thread of discussion. As I was reading through it, I became more and more concerned about, simply put, complexity. I live and breathe by a foundational Unix philosophy:

The only universal interface is text

This is a deeply profound statement, all the more impressive given that it was proffered decades ago during the birth of information theory and well before we had to deal with deprecated file formats and the like; they saw the future.

Personally I like to store my most important information in the most foundational format possible (text) using the most foundational tools possible (Unix utilities, which I believe hits the 50th birthday next year). I’ve lived long enough to see software come and go, file formats come and go, and indeed entire languages come and go. What, then, is the best method for information representation over the long haul? I’ll get to that. But first, my concerns with the conversation:

  • Layers upon layers of software complexity
    Reading through the list of products mentioned in this discussion, I became concerned with the additional layers of software complexity and software dependencies. Which one of these products will be around in 10 years? Because my notes will still be there (23 years of notes and counting…) but I would never want to have to worry about maintaining an entire software stack to make them useful. My notes have to work for me, not the other way around. Which is a nice segue to my next concern
  • Littering notes with information expressed in outdated languages.
    There’s a reason why the legal profession likes Latin terms, and it’s not to confuse people. It’s because dead languages don’t change. In the law, it’s important that a contract written 50 years ago is still unambiguous 50 years hence. Therefore the law uses certain Latin terms to protect against ambiguities created by changes in word usage over time. I am seeing in this thread more and more complexity suggested, via language specifications that could go by the wayside in the future. Do you really want to have to go through a language translation on thousands of notes later? It’s more likely you’ll be hoping someone else wrote a utility to move those specifications into the Next Great Language, and you’ll be on that treadmill to the grave.

To each his own. Returning to my original rhetorical question:

What, then, is the best method for information representation over the long haul?

My notes are designed for the following model:
Inforep

My notes are foundational. I represent information using my own methods, so they will never have to change. The important thing is that my methods represent an unambiguous grammar. Since they’re my methods, I understand them well and can extract information when necessary. I use external (3rd-party) tools for visualization and analysis. It only requires that I create an interface between the notes and the tool. This is typically just a script. If I want to change tools, or add another tool for some new, cool visualization, nothing about my notes will have to change. I just need to extract the relevant data via a new interface script. We can call the scripting the “Middle Tier.” By separating the information from the tools by a Middle Tier, the information remains unchanged.

Addressing “my” methods

It’s ok to adopt some existing specification; you don’t have to roll your own. The key questions to ask yourself are:

  • "Can I live with this forever? (Think in terms of information representation, extensibility, and parse-ability.)
  • Will I ever have to go back and change hundreds or thousands of notes?
  • (Addressing parse-ability) Using these representations, do I have the skills to extract my own information?
  • If this information has to survive me, while my family may know nothing about my toolset, can they at least read and understand the notes?

Don’t fall for the siren’s call of the concept of One Tool to Rule Them All. Searching for the Grail might be more successful. I prefer a modular architecture with an everlasting foundation.

14 Likes
45

This is an insightful message, @datum . Thanks!. It sparkled a lot of thoughts in my head.

I really believe that to write personal notes, we should use familiar language, simply because we are the future consumers of our data.

But if we want to use a cool application that helps us exploit our notes better, we will need to build or enable this middle tier.

Also if we’re going to share our messages with others or work together writing notes. Look at Wikipedia, for instance. They agreed about minimal stuff to communicate.

I agree with what you say; probably, there is no ‘ultimate’ representation method.

On the other hand, some get adopted better! Look, for example, Markdown itself; nobody would prefer to author in HTML anymore! It provides a better experience, less friction, is less ugly, etc.

I believe that the most vital thing is to embrace diversity, accepting a variety of representation methods.

We can experiment :slight_smile:

2 Likes
46

Absolutely @cristian we can experiment. If I wasn’t willing to do that, I wouldn’t be looking at Obsidian! I would agree that a standard is useful for the purpose of data interoperability.

The funny thing is, I never intended to develop my own methods. Years ago I was looking for the best platform for my important notes. In the meantime, I decided that I would just store them as text because it should be easy to move them into some future tool. At the time I didn’t realize how foundational that decision would be. I continued to search for tools, and along the way I kept adding refinements to my system of note-taking to capture necessary information. A few years ago, I started questioning why I was looking for something better when I realized that my text was starting to look like a rich representational system. I was also developing my own notational system for basic formatting before Markdown came along, because I wanted to write a CGI-based web application for searching / rendering my notes however I saw fit, without having to write HTML into my notes. I had planned to integrate GraphViz to render my graph queries. I have a lot of experience with that at work, so it was a natural choice for me.

There are certain things that Obsidian will never do because they’re just outside the scope of what Obsidian is trying to be. For example, if I had a semantic network for characters in a book, I could easily use Obsidian to answer the question, “Who are James’ friends,” because his friends are a direct link to James, therefore all I’m really asking is, “What are all the outbound links of type “friend” for James?” I can do this in Obsidian. However, if I wish to ask, “Is there any relationship between James and Howard,” the best I could do in Obsidian is eyeball the graph, which isn’t very productive past a certain number of nodes. I’ll have to go outside of Obsidian. I’ll need to do a reachability analysis against the edges, to see if a route exists. Luckily it’s easy enough to extract the edges from the vault. Could a plugin provide this feature in the future? I can’t plan for what might be, because it also might not be. I’m at a point where I need my notes to really work for me. I’m willing to pay for good software, but it has to be around for the long haul, or at least allow me to “pull that parachute cord” and take my notes elsewhere easily.

If we want Obsidian to be successful, the solution is modularity and ease of use. I don’t think it’s entirely there yet, but it’s coming along. When I first looked at Obsidian, I wanted to change the font in edit mode to be monospace because it drives me crazy when I’m formatting something like a Markdown table, and things don’t line up for no good reason. Since I’m not conversant in CSS, the whole experience was as if I walked into a coat store, asked to buy a fur coat, and was handed a bow & arrow. If Obsidian is to truly be successful in a big-market way, it needs to be more than an esoteric tool for the technorati. In my opinion, Microsoft didn’t get it’s huge success by writing the best software, but rather by writing software that everyone could use. This is what makes the difference between a small beautiful pebble that in the end gets washed down a stream into oblivian, and a large immovable boulder that could be Obsidian.

6 Likes
47

This would be a very useful and simple way of labeling connections. I’ve always wanted to add labels to some connections because it would add so much clarity to some graphs. I’m trying to create and refine concept maps for studying, like I can with Cmaptools, but using my linked notes database.

I would not label all my connections because the labels could also restrict thinking, but they could help to highlight the core concepts of a topic.

6 Likes
48

@ringelblum @soma Now I’ve gotten myself confused. I had to remind myself that this topic is about what we want link types to look like as a supported feature in Obsidian, not how to do it in Obsidian as-is. For now I’m thinking of using the [[TargetNote|:linktype]] notation. The colon distinguishes it as a link type to differentiate it from a regular link rename.

What’s interesting about @soma’s suggestion is that he’s defining a triple, and therefore the full relationship, in a “3rd-party” note. This is distinctive from a notation to simply link the current note to a target with a link type. These are two very different things. Currently Obsidian graphs reflect relationships between notes implied by reference. Whereas this notation expects Obsidian graphs to reflect relationships explicitly defined by notes. And the grammar could be tricky:

NamedEdge → Link Words – Predicate → Link
Link → [[ Words ]]
Words → Word Words
Words → Word
Words → Lambda?
Predicate → Words

(This grammar isn’t completely correct because it allows for blank note names and blank predicates, but was shortened for clarity, if you can overlook that.) This will parse, but there’s a lot of room for error. for example:

[[Berlin] is the – capital of → [[Germany]]

If “[[Berlin]” is not seen as a link, then the “capital of” link will never be defined. Obsidian isn’t trying to be a compiler; if it doesn’t see the first link then the rest is just free-form text, and there is no error, and you may never know the link wasn’t defined. Worse, it means seeing [[this notation]] in two contexts. So if a syntactical error leaves the named-edge undefined, it also means that a link from this defining note to [[Germany]] is created, which wasn’t wanted. I’m not suggesting that other methods cannot similarly fall victim to such a mistake, but it speaks to the need to keep the notation as concise as possible. And also there would have to be a decision regarding Obsidian continuing to only recognize relationships between notes, or include relationships between notes defined by other notes. Personally I don’t see this happening due to the ambiguity created.

1 Like
49

I take notes for 12 years now. I also decided to use plain text after moving from one tool to another more than once.

Sometimes I moved from one tool because it became too slow, sometimes because it was aesthetically unpleasant, etc. I moved into Obsidian because it offered to autocomplete the ‘[]’ and the other didn’t.

If you use text, you are not ‘married’ to any tool.

Along the way, I developed my super primitive symbolic system with little cues that I could remember and type in a search box or a grep command. To look at all the notes of my friends, I search the phrase ‘is a friend’, and If I want all the addresses of my friends, I search for ‘is a friend’ and ‘has address’. I’ve seen that many people in this community do this with tags .

I only write graphs because they force me to think better about things; to go for coherence and completeness.

I need to look at the graph and to know what the links are about

2 Likes
50

Since we control our notes, programs like Juggl (Out now! 1.0.1): A completely interactive, stylable and expandable graph view plugin! of @emile can be used to extract information from our notes and write a graph.

Information could be extracted directly from the Markdown, such as tables, mermaid diagrams, other graphs’ ‘dialects’ etc. This ‘middleware’ can have several outputs, different graph dialects, csv, tables, etc. Or the omnipresent ‘Excel’.

Or perhaps there is something already there that could be reused or built upon? I discovered https://unifiedjs.com/ the other day.

When I write notes, I try to think about how they will be processed years from now. I believe that something that generates structure data is valuable to be used by the current tools or that AI program of the future that will surprise me in the next years, who knows!

51

This “single” Unix command will extract links from a vault, and reformat them in a form that can go directly into GraphViz:

egrep -roI ‘\[\[[^]]+\]\]’ VaultName|sed ‘s/^/“/;s/.md:\[\[/” → “/;s/\]\]/”/’

Should also work on Mac, but I didn’t test. I only spent about 60 seconds coming up with this so there might be edge cases I didn’t consider. This is what I love about text, and what I love about Obsidian. It doesn’t tie up my data in a format I can’t use elsewhere easily. Easy extraction of links and their types (I didn’t process for types in the above command) means I can externally answer questions like, “How is A related to B?” Then just apply a route-finding algorithm that will output each possible route and the association types that get us there. It would be great if I could tie that functionality into Obsidian, but I’m not holding my breath. Besides, Obsidian doesn’t render self-referencing links (notes with a link to itself) so I can’t completely rely on the visualization. Still, as a tool, Obsidian is compatible with my way of working with notes.

Edit: Escape sequences disappeared, fixed.

52

Hi @cristian it looks like we have some philosophical alignment on how we handle notes. I avoided moving my notes to different tools only because I never committed to a tool while I continued to research options. There are many great tools out there by which we can build workflows of knowledge management. What I love about Obsidian is that it is aligned closely enough with how I already structure my notes, that moving to it is easy, and moving away from it if necessary is also easy. There’s an elegant power in that freedom. I look forward to what comes next for Obsidian. Hopefully we’ll get named-links. This will provide the context to walk graphs and ask deeper questions via a plugin.

1 Like
53

This is a fascinating discussion. I am a neuroscience researcher, and I’ve dreamed of being able to have a system that I can input research findings and use graphs to make novel insights and inferences.

I love the flexibility and futureproofing of plain-text notes. But sometimes it would be nice to have the functionality of a more formal database to integrate pieces of data.

There is a tool that does this (based on Neo4J)

But I’m reluctant to build on someone else’s tool.

The ability to specify link types in plain text with Obsidian would allow one to build such maps of interactions between notes in powerful ways. I hope this thread and accompanying tools continue to develop.

This discussion might also be of interest:

2 Likes
54

Hi all. For the Neo4j Graph View plugin, I want to support a better way to represent link types. I’ve been writing a bit about this here https://publish.obsidian.md/semantic-obsidian/Link+Types

I’d love to get some input from the people in this thread, and to discuss together with the goal of finding a good Markdown specification for link types and link properties. I’ve opened a Discord for the plugin, where we’re having a discussion about exactly this topic: https://discord.gg/sAmSGpaPgM . Feel free to join :slight_smile:

1 Like
55

I’m reticent to put my data into yet another tool. I like where it is, in a folder of my hard drive :stuck_out_tongue:

Nowadays I miss something that helps you to draw small maps of interactions between notes. A complementary tool for thought, not meant to discover new things but to synthesize, simplify, organize, and merge content.

Perhaps we can imagine a simple way to draw these small maps?

Some use mindmaps, that can be embedded in Maps of Content.

I personally would like a dialect to embed this info, specifying the nodes positions, colors, everything aesthetical that matters or help us to think clearly

```think-map
56

disney
disney1480×1295 277 KB

In 1957 Walt Disney drew the media empire he was trying to build.

He didn’t have a computer to draw connections through…

4 Likes
57

Proposal to extend the existing link syntax to support user-defined labels on edges in graph view.

Issue

Links in and of themselves provide no meaningful context therefore context must be supplied through the written word. This works when reading the note directly but when viewing the graph view these transitionary contexts are lost – it is not clear from the graph view why a link exists, only that the link exists.

This is a fundamental weakness in all current note taking systems and is a weakness of the web itself.

Benefits

Allowing the user to define labels for edges will make it easy to see at a glance in the graph view the nature of all relationships between this note and the linked notes. Users can quickly “surf” from note to note via the graph view if desired and then “dive in” to a specific note.

Unlike the web there is no need for a centralized canonical registry of labels with agreed upon meanings – users are free to use arbitrary labels with arbitrary meanings.

Labeled arcs also approximate a mind map of the user’s notes. All in the filterable graph view.

Syntax

Given a link [[notename]] the syntax can be extended as follows: [[notename@label]].

@ is chosen as it represents labeling the arc between the nodes.

Ex: [[Teach cats how to build giant robots@bad idea]]

Users could even use emoji labels for shorthand:

Ex: [[Teach cats how to build giant robots@🙀]]

Possible labels

Possible labels may include: (but are limited only by imagination)

  • is a
  • has a / has N
  • supports
  • opposes / contradicts
  • works well with
  • do not use with
  • elaboration/amplification (as in thought A evolved into concept B branched into two more lines of thought…)
  • owns / owned by
  • works in / works on / works with
  • good idea
  • bad idea / dead end (for those of us who want to keep the bad ideas and warn ourselves not to go that way)
  • :heart:
  • :bulb:

Display

Display the label as small text alongside the arrow, with a selectable CSS class so it can be customized by themes. Default to the same color as the arrow.

Consider also creating a second custom class for each label, containing the name of the label itself, for additional CSS customization by the user/theme.

Ex:

Given a link [[Teach cats how to build flamethrowers@great idea]] generate something like this:

<div class="graph-arc-label graph-arc-labeltext--great-idea">great idea</div>

allowing users to write custom CSS like this:

/* all hail the cat overlords */
.graph-arc-labeltext--great-idea { 
    color: red; 
    background-color: yellow; 
}

so the brilliant idea is more prominent when viewing the graph.

This also helps encourage more focused note taking, since transitions can be clearly and easily established in the link itself.

7 Likes