In what ways can we form useful relationships between notes [LONG READ]

[nickmilo]

UPDATE: I updated this and turned this into a Medium post: In what ways can we form useful relationships between notes? | by Nick Milo | Medium

Are you into personal knowledge management (PKM)? Are you confused about when to use a folder versus a tag versus a link versus a Map of Content? Let’s explore each “relationship-builder” and how we can use them expertly.

Main Ideas

• Top-down categorizing has gotten a bad rap. It has a place.
• Bottom-up creating is great, but it needs a bit of top-down structure as your library grows. But using just folders is the WRONG way.
• Maps of Contents (MOCs) are game-changers.

A flat structure of notes — i.e. one big folder — is “frictionless”. There is no thinking “now where exactly does this go?” However, over time it becomes messy and overwhelming. To overcome the chaos, we need to build healthy, flexible frameworks that can grow with our knowledge over time.

Let’s examine the known— and lesser known — ways we form useful relationships between notes.

Direct Links

Direct links are the strongest type of relationship. They are the purest and most explicit way of connecting two notes. This is the core method of growing a healthy, dynamic zettelkasten.

Despite the proclamations, direct links alone are not enough. They do not allow for easy, dependable high-level navigation. They are a system of streets connecting one block to another. But sometimes you want to zoom out and view the map.

Folders

Folders are rigid and exclusionary by their nature. Whatever is in a folder lives separated from the main collection. It’s a rigid hierarchy that imposes order. Without special workarounds this limits note accessibility, discourages interdisciplinary thinking, and stunts the long-term growth of a complex and interesting zettelkasten.

There are some strategic uses for folders, but their overuse may be a sign that the user is uncomfortable with other, arguably better methods of building relationships between notes while maintaining the long-term health of their digital library. If you have some quixotic notion about creating a digital library that acts as an interesting conversation partner — folders don’t make it as easy.

What about for projects? Depending on the use cases, a person might like using folders for projects.

• If your goal is to manage projects, folders are great — maybe even ideal. You might decide that you want your projects to be actively walled-off units.
• But once they are final, you take the extra steps of reformulating any good stuff into your main digital library.

A possible use for a folder is as a temporary inbox, or “incubation” folder. As long as you view these research streams as temporary staging grounds, to be deleted or assimilated with the Borg- er, I mean, your main-brain-frame; you should be fine. Otherwise, that inbox folder can get awfully crufty.

Another possible use for folders would be for “very clearly defined” notes. For example, things like: Images, People, Quotes, and Source Materials (things written by other people). You don’t have to use these folders; you could easily add their contents into the main vault; but you might find it keeps things slightly more tidy.

But know this: retreating too much into project-based folder management will cripple your long-term thinking partner (i.e., your zettelkasten) from growing complex, dynamic, and interesting cross-genre connections. You won’t have a zettelkasten, you’ll have a collection of silo-ed folders. Those are massive costs to consider.

That said, limiting accessibility may be ideal for private information like finances, health, and private journaling. Then a folder is the perfect instrumental to intentionally cordon off those notes from the rest.

Tags

Tags are relatively weak associations, but don’t discount them. They are easy ways to quickly filter large swathes of notes. Tags are more fluid: one note can have multiple tags. The problem is that tags alone don’t scale.

Say you have a #PermanentNotes tag or tag page. Whenever you take notes on something you read or watch, you apply that tag. It’s great when only 50 notes have that tag. It’s far less useful when 500 notes have that tag.

Multiple Tags

A partial solution is to use multiple tags, so you can search for: “#PermanentNotes #habits” to filter in only those results.

But the problem here is that you have to remember each tag!

What happens when you can’t remember that when you’re tagging a new note? You end up using #improvement or something else. Sure, maybe you can remember this single #habits example. But over time you’ll lose track of how to tag a new note in a way you can reliably retrieve (meaning that it will scale well and you’ll remember it). By the way, software that allows for “starred” notes is just exercising a form of tagging.

Tags with Saved Boolean Search

Saved Searches are another partial and robust solution. You can save a tag search like “#PermanentNotes AND #habits”.

Proximity

Everyone tends to forget how we use the proximity between notes to build relationships. Organizing by proximity can be weak and arbitrary or strong and meaningful. It just depends on what level you apply it. There are 3 basic levels of proximity.

1. Organizing by Proximity in the main folder:

• Alphabetical: Sure “Apple” and “Banana” start next to each other and hold a strong relationship. But over time that proximity will change; because years later the list looks like “Apple”, “aqua”, “aqualung”, “arabesque”, “arachne”, “arbiter”, “arc”, “arena”, “aries”, “arise”, “Arkansas”, “Banana”. So relationships that start out strong can weaken over time.

• Chronological: If you sort by creation date, the proximities between notes won’t change but they are inherently more arbitrary (random), except for their temporal context (which holds varying value depending on the context).

2. Organizing by Proximity in the same subfolder:

• Notes in the same subfolder will have a closer relationship since they are grouped in the same vicinity. But this comes at the cost of being silo-ed from the rest of the note library.

3. Organizing by Proximity in an MOC (Map of Content):

• Notes in the same MOC are very likely to be closely related, especially after they are manually sorted in some form of relational positioning. This is the best way to apply the power of proximity.

Using just a single note, you can identify and build relationships between notes.

Map of Contents (MOC)

An MOC is just a note — a very special type of note. It uses proximity effectively and acts like a tag and a non-exclusive folder at the same time.

• Tag-like: It groups the links of associated notes in a non-exclusive way. (The notes themselves live freely elsewhere.)
• Folder-like: It assembles notes in a tightly-packed grouping.
• Proximity: It grants you the ability to deliberately position each note in relation to the other notes.

In this way, MOCs don’t limit access, they curate it — while keeping your notes free.

Using MOCs is like being in your own warehouse full of workbenches, where each workbench contains a selection of highly curated index cards for you to engage with.

Another way to consider MOCs is through emergence.

MOCs are Evergreen notes, just at the next level of emergence.

In an MOC, the party is always happening. It’s the “room where it happens.” Individual notes can “shadow-clone” themselves and essentially be in multiple parties simultaneous — interacting and developing complexity from each party at the same time!

Q: How do MOCs compare to TOCs (Table of Contents)?

A: Whereas MOCs are fluid, TOCs are rigid. This is by design. A table of contents is for assembling a specific and linear order. MOCs serve much broader purposes. Oftentimes an MOC can morph into a TOC as a project starts to finalize.

A Home Note

Think of your Home note as the highest-level of your zettelkasten / digital library. It has links to the main MOCs in your library along with your most relevant tags.

It is an excellent access point because it encourages you to focus on the areas in your life that you’ve deemed as important.

A Home note creates stronger, longer-lasting connections, especially as one’s library grows far beyond 1000 notes.

Don’t fall for the dogma

Until recently, folders were all we had. They weren’t good enough. Now we have links. Now many people have taken a hard stance that all they need are links. Hard stances become fragile stances. Don’t fall for the dogma. The right tools for a healthy digital library include a multitude of relationship-builders.

These relationship-builders include: a Home note. Maps of Content. Direct Links. Proximity. Tags.

And yes, even a few folders.

[Alf]

Very insightful, I think I’ll play around with the indexing bit. Thanks!

[nickmilo]

There was a great discussion on discord about this. Here are my highlights (if I quoted anyone who would not like to be quoted on discourse, just let me know and I’ll remove):

---

Regarding the use of “folders”

I was a little harsh above and it lend to a great dialogue. I clarified my thoughts a bit:

• If your goal is to manage projects, folders are great—maybe even ideal.
• If you have some quixotic notion about creating a Frankenstein (zettelkasten) digital library—that acts as a conversation partner—folders don’t make it as easy

@cotemaxime employs a balance of para with folders and a zk. So projects are actively walled-off units. But once they are final, the good stuff gets reformulated into the ZK library. (NOTE: I can see that working quite well.)

@ryanjamurphy counters that: “Cleaning up the dust or moving tools around a workspace is usually annoying and non-generative. “Processing” or reorganizing/resummarizing/rethinking existing notes is generative … I try to anchor my thinking on these ideas in cognitive overload. If I’m in the middle of juggling four different concepts but then I have to move folders or tags around (or whatever), I (might) lose some of those connections/ideas/etc” (NOTE: Which I agree with.)

@nickmilo (Regarding using temporary “inbox” folders) Maybe you don’t need to immediately integrate generative thinking (the zettelkasten) from foldered research streams. (I mean, you could and use tags and maps of content to structure your research streams…). But as long as you view those research streams as temporary staging grounds, to be deleted or assimilated with the Borg- er, I mean, the zettelkasten; you should be fine.

@sam.baron I’ve had mixed experiences with using an inbox. It’s good to have a place to put anything, but I always have a lot of things that go nowhere. For me, doing some initial organization through links goes a long way.

@j.split [inboxes] can get kind [of] crufty and backed up if you don’t make a habit of processing and reviewing.

@sam.baron Yeah, it’s like the email inbox problem all over again

@cotemaxime I use an inbox just to reduce the time I have to put into taking notes, I put the text but then later comes back to see where it would fit and work and tags and stuff, but yeah I have an habit of doing that once a week and sometimes it doesn’t happen but then the note are still there and surfaceable still by search and stuff (NOTE: A reasonable approach.)

---

Regarding how organizing digital information always requires thought, even when it seems like it doesn’t:

@ryanjamurphy I am suspicious of Roam’s lack of files. I think it’s actually a little handwavy. … I think, in Roam, files have only disappeared through sleight of hand. You’re still juggling some of the questions we’re discussing here, except instead of using the word “files” you’re deciding where to put “pages” and how to “index” pages, etc.

The terms are switched and so it looks like they’ve magically resolved some of these core tensions of organizing (digital) information. But in reality it’s just a magic trick , because you still have to invent your own forms of organizing.

This all works well enough in the first few weeks or months, but once you’ve got years of notes, you’re depending on your poor little brain to remember your own organizing structure and what files pages matter and so on! (NOTE: I 100% agree)

[ryanjamurphy]

Thanks @nickmilo for practicing what we preach here on the forum!

Note, the chat started with Nick sharing the original post here.

I wanted to pull Ambient Complexity’s comments too, lightly edited:

I’m just gonna throw in here: I would be careful with going too far thinking of your knowledge work in computer metaphors.

Memory isn’t exactly storage / recall. It’s reconstruction relative to a context, so a lot of the “payoff” of knowledge work is not “accumulating information that you can reproduce 1:1,” but influencing your thinking and giving you “material” that gets “reformed” when you enter specific types of conversations.

With that [comment], I tried to formulate that I strongly believe the goal of your system shouldn’t necessarily be storage-retrieval of thoughts measured “amount of notes” […] but rather what you understand and the connections from it. The way your consciousness works is tacit on many levels, so I wouldn’t try to reinforce a literal survival of information.

[AmbientComplexity]

Hi,

this discussion and some others on the Discord prompted me to look at a the constitution of Luhmanns Zettelkasten again. A short and systematic overview (in German), can be found here: https://niklas-luhmann-archiv.de/nachlass/zettelkasten.

This in turn prompted me to change my stance (or lets say reconsider) on folders. Luhmanns Zettelkasten basically works on the structure of folders. In his own reflections he states that hierarchy is the enemy of an expansive note taking system because it produces a preset structure, which will inevitably have to be changed later. He himself uses a hierarchical structure though, but not seen in the strong sense of a Taxonomy (non-overlapping classification) but rather loosely along the lines of connected and differentiated thought.

Connection and disconnection are the two elements that structure his Zettelkasten (funnily, these also the two elements that structure his (thousands of pages of) theory). I made a little note of how his system works. Sorry for my handwriting:

The way he organized his system was based on (a) his practice as a researcher (b) experience with note taking (his ZK1 had actually 108 base level notes and a bibliography and register) and © the media he had available. So he used space, time and writing on paper. Our media are a bit different, we still have time, we dont have space in the same sense, we mainly have a 2D representation of paper or a filesystem through the interface but most importantly we have data (and its handling). For the current discussion, i don’t think it will be too worthwhile to think about media and correlated possiblity spaces, so let me jubilantly jump over the analysis and just connect my thoughts loosely to it.

A more worthwhile line of investigating would maybe be to ask about the purpose and requirements and then look at the structure and practice of using Luhmann’s system (we can generalize to “system for taking research notes”):

• What is the purpose of Luhmann’s (our) note taking system? To store notes in a way that facilitates later retrieval and retrieve them based on future problem contexts. When engaging with our notes the arrangement of our notes should help us think.

• What are the requirements? Note taking should be easy (else we’re not gonna do it) and we need to solve the problem of note retrieval (, while having many notes). Ideally being able to retrieve problem contexts more so than singular notes.

What is the structure and practice of Luhmanns solution?

1. He has a folder like differentiation on the main level. 108 base categories in ZK 1 (23k notes), only 11(!) in ZK 2 (67k notes). (Here is a scrollable list of the contents of the first zettelkasten click.)
2. He progresses his note taking by organizing them according to their connection to prior notes. Either its something new, then it gets connected by his ‘data serialization scheme’ and placed appropriately in space (the pink parts in my picture) or its something different, then it gets ‘disconncted’ by his ‘data serialization scheme’ and placed appropriately in space (the blue parts in my picture).
3. To allow him to think with the Zettelkasten, he uses links between physical ‘lines’ of notes often implemented as backlinks (the red line in my picture).
4. To retrieve notes he has index cards (a) on a global level (b) on a (sub)topical level. (@nickmilo talked of these as MOC).

What are the consequences of practice with this strucutre?

1. It has a hierarchy of topics for general orientation pruposes, which helps when (a) thinking of where to put a note and (b) helps of thinking how to retrieve your note. This creates as @nickmilo called it “friction” but some friction is not all bad. It helps you organize your own thought. If you work on big picture projects, I think this friction is very valuable because it helps you contruct your own ‘mental model’ of your topics.
2. It is non hierarchical in the sense of a strong taxonomy, notes are connected by lines of thought, which reduce friction when doing storage/retrieval and allows the topics to grow organically without having to restructure things or agonize over where to put them.
3. The index cards are indespensible but they are also incomplete,which forces you to interact with your notetaking system by continously sifting through it. This creates a type of accidental spaced repetition just based on side effects of the approach and medium. I suppose its super fun to do as well, when you have a big-ass Zettelkasten.
4. The links and backlinks he uses help to get from topic to topic and allow him to “converse” with his Zettelkasten. At a certain level of complexity, you lose the overview and memory of what you stored and when you approch your notetaking system with a specific problem in mind, you can be surprised by the solutions it provides. This is why he jokingly reflected on his Zettelkasten as “conversation partner” or as “writing his books”.

So how does this relate to digital notetaking?

I’d say this system is pretty good for building a database of research notes and i haven’t seen a more convincing sytem yet. But there is a problem of lifting the system wholesale because you can’t relly reproduce space, in the sense of stacking notes (or what is discussed as “Folgezettel” i think) well. Sure if you want to add something to a note, you just add it to the original text document, but if you want to connect thought, where do you go?

• If you add the second dimension by utilizing the file name or some type of uid and data serialization scheme as luhmann did, you lose the practice of ‘sifting’ through your notes (at least at higher complexity), unless there is a specific interface implemented for it.

• Folders within Folders are a bad replacement because it becomes awkward to add notes which have little diffrentiation on the second dimension (meaning its folder -> 1 file, folder -> 1 file and so on).

This leaves us with the question: How can we add this second dimension to Obisidian or if we don’t, how do we replace a KB that is structed along two dimensions (connectedness of thought and differentation or disconnectedness of thought), while keeping the benefits elaborated on above?

Purely on a note level it seems, you can only reproduce one dimension properly, either differentiation or connectedness and instead of being able to ‘sift’ or ‘rummage’ through your notes, you can use tag based or lexical search to find the starting point of what youre looking for (connections are still provided by wiki-style links). It seems to me, the solution to this problem might be maps, as a new interface type. This solution would skip the second dimension of ordering but visualize interlinks in a way that Luhmann couldn’t in his medium.

So my preliminary conclusions:

Reproducing connectedness and disconnectedness in a digital medium, requires you to:

• Realize differentation by creating new notes (each different note is a different thought)
• Realize intraconnectedness by appending to a note (each note can be continously added to)
• Realize interconnectedness on a link and backlink basis
• On an interface level, we replace the linear ordering of stacked notes, with the 2 dimensional ordering of the map and allow exploration of the note taking system through the map

Finally, my open questions boil down to this:

1. Is it important/usefull to differentiate between a hierarchical type of structure and connectedness (connectedness dimension in my picture) and links between notes (the red line) or is this just an artifact of a physical notetaking system?
2. Is the map an adequate solution to allow explorative search and would focusing on the map as an interface yield great benefits on notetaking?

edit: some small fixes

[ryanjamurphy]

Thanks for this line of inquiry. The best phrase in it might be:

big-ass Zettelkasten

I think you’re right. Nick’s Map of Content and Andy Matuschak’s Associative Ontologies are probably pointing to the same conclusion of “maps,” too. The question is what are the best ways of realizing these maps? This question needs to be answered both in practice (“what habits or algorithms do we use to create and maintain them?”) and in instantiation (“is a markdown list of links and descriptions ideal, or should other UX elements be implemented to support rich content mapping?”)

[ja_rule]

This is a fascinating discussion, as I’ve just been diving into developing a PKM system in the last week. Was pointed to Zettelkasten by someone in a completely unrelated context — an online music forum — and have since been reading all I can find to help me solidify my personal conception of the ideal system. I’ve been using Evernote for almost 9 years, and more recently a combination of GoodNotes for day-to-day research notes — which bounce back and forth between typed and [Apple Pencil] hand-written, depending on my mood — and LiquidText for PDF careful reading and markup. Have been frustrated that each of these 3 tools does only part of the job I want done, and would love to see Obsidian become a unifying app…

But to the topic at hand of forming useful relationships between notes, I wanted to comment specifically on the use of Folders. For my first vault, I’ve employed a combination of project specific folders, which map to my work as an R&D scientist, and a simple folder structure I’ve used successfully in the past: yyyy/mm/. I find this to be the most generic way to solve the problem of one giant, flat folder, because at the start of each month, I create a new bucket to dump notes into. Obvious advantages:

1. No bucket is ever too full to quickly browse through
2. The structure naturally captures one of the axes — creation time — by which I tend to access data. My daily work and thinking most likely lies in the current bucket; I can remember the last several buckets without too much effort; and due to the forgetting curve, past projects are remembered more by [approximate] year than anything else.
3. Having now seen the Zettelkasten UID, the full file path in this system seems to be not entirely unlike the dreaded [to me at least] 14 digit time stamp, without the offense of a time stamp as a file name. I should note that I’ve been adding my daily notes with the simple name <dd.md>, so today’s note would be 2020/05/30.md. Easy!
4. By dropping new notes relating to any topic into the current bucket, I’m implicitly not relying on the bucket to provide any categorical information about the meaning of the note, other than as a marker of when I was thinking about it.

This leaves me with a system where I have to explicitly provide the relationships between notes or concepts via links, tags, MOCs, or whatever else I may use — and I believe the goal of any PKM system is to make you think about the linkages in your knowledge — while providing a rough temporal framework in the physical storage structure.

I’m sure I’ll evolve on this, but that’s what I’ve created as a starting point.

[masonlr]

Is anyone here interested in collaborating on a set of reference starter examples? Or, are these systems best uncovered through personal experiments and spontaneous forum discussions?

@ja_rule I was experimenting this week and think I converged on a similar structure. (Date-stamped log, project folders and an emphasis on linked evergreen concept notes.) Do you think a set of reference starter templates would be useful for other researchers? (I also work in R&D.)

This is a sketch of what a reference set could look like: Obsidian Starter Templates GitHub repository. Additional descriptive summary information could list opinions, comparisons, benefits, drawbacks and an example graph screenshot for each approach.

@nickmilo I am very new to associative notes and was taken aback by the MOC ideas that you have been explaining. Would you be supportive if a minimal example of your detailed IMF_AdvancedStarterKit_v2 was added to a common starter repository? Or, do you think we should avoid collections of reference kits, e.g. “The hope though, is not to be caught up in the details and then make dogma out of it”.

Strategies might be easier to discover from a common templates set, rather than being spread across multiple discord discussions and forum links. The starter set could be integrated into Awesome Obsidian. Maybe this is all a distraction if strategy building blocks are later baked into the UI through a templating system, but a shared set could help in the meantime.

[lizardmenfromspace]

Multiple people are developing “starter sets”, @mediapathic is collecting them I believe.

[nickmilo]

@ja_rule Thanks for sharing your method. It seems like a solid way of working, and it’s working for you which matters most of all. To me, I’d just let all those timestamped notes run together in one folder so I could just scroll through, but it’s just a preference because I also think there’s a need to chunk information in PKM that isn’t getting talked about enough…

There’s something I can’t quite pinpoint, but there are certain limits or chunks the human brain seems to prefer, and if we go over those limits, overwhelm can set in. “12 months in a year” “30 days in a month” “10 categories in Dewey’s system” We’re only able to hold 4-7 pieces of info in our heads at any given time… just food for thought.

@masonlr Thanks for you thoughts. I think you and @ja_rule already have a strong grasp on a way to make your digital libraries function well for researching (and more) with a temporal log, project folders, and emphasizing evergreen notes. I would be curious to see usage examples of R&D in action, if there’s anything that isn’t too private of course.

[nickmilo]

@AmbientComplexity Well, you’ve really gone down the rabbit hole!

Your first question:

Is it important/usefull to differentiate between a hierarchical type of structure and connectedness (connectedness dimension in my picture) and links between notes (the red line) or is this just an artifact of a physical notetaking system?

It is useful, but not in the way shown above where hierarchy brands a note’s position forever. Those notes are forced to live in one specific “note sequence”. That’s not fair to the note. It should be able to live freely and connect by direct links.

But very often, we have the need to sequence notes together: that’s where a map of contents becomes a curator of content, allowing us the ability to apply fluid hierarchies that don’t affect the notes themselves. The notes stay autonomous. It’s like a somebody curating a list of 10 music albums. That curated list has hierarchy, but those albums live separately, with all the other contexts and nuances attached to them there.

I’ll put it another way, NOTES ARE LIKE PEOPLE. Just because I was born in the 80’s, it would be crazy if you had to go specifically to the 80’s folder to find me. I have interests. I like football. But just because I like football doesn’t mean I should only be found in a folder on football. Try not to pigeonhole people, and try not to pigeonhole your notes!

Your second question:

Is the map an adequate solution to allow explorative search and would focusing on the map as an interface yield great benefits on notetaking?

Generally, yes, especially because it’s fluid and non-destructive. It’s non-hierarchical on the outside because it’s non-exclusive, but it can be hierarchical on the inside because you can manually structure things. Best of both worlds.

This is hard to talk about so hopefully the rash of metaphors help, and don’t hinder anyone’s understanding.

[ja_rule]

@nickmilo I think you’ve hit on a key issue here:

I’ll put it another way, NOTES ARE LIKE PEOPLE. Just because I was born in the 80’s, it would be crazy if you had to go specifically to the 80’s folder to find me. I have interests. I like football. But just because I like football doesn’t mean I should only be found in a folder on football. Try not to pigeonhole people, and try not to pigeonhole your notes!

I’ve got a fair bit of experience with designing relational databases to store scientific or engineering data, and when I first started using this as a research tool, I had learn to start thinking in terms of data objects. What is the underlying thing I’m trying to capture information about? What is it’s relationship to other things in the system? Does this thing deserve it’s own table, or is it just a property of another thing?

If I think of you as an object — without objectifying, of course! — you are fundamentally a Person. And you have lots of properties: likes, dislikes, hobbies, skills, connections to other Persons, physical characteristics, and a birthdate in the 80s. I agree that filing you in the 80s folder would be the wrong place to start, but filing you in the People folder would be perfectly rational. Linking you to your hobbies and likes/dislikes would be a reasonable choice.

My struggle with all this is that some concepts or notes have an obvious base object — People, Locations, Projects, Events — while many of the notes I want to capture are thoughts about these base objects. I use the base objects to provide some underlying structure, the calendar based buckets to provide my unbiased, but chunked “slip box”, and the linkages to provide the neurons or breadcrumbs that tie it all together.

[ryanjamurphy]

How weird. This thread is suddenly all about conceptual modeling:

Conceptual model

Here’s food for thought. The conventional approach to conceptual modeling is the main paradigm being discussed here: classification. When we set up any information system, we tend to begin by answering “What kinds of data are we putting in this system?” Implicit in this question is the idea of “kinds”: People, Locations, Projects, Events, as @ja_rule mentions, are an example of a “kind.” But so are “Notes” or “Blocks” as has been discussed elsewhere here. The idea of assuming that everything has a kind is so embedded in our thinking that we rarely question it.

Before I go on, let me clarify something real quick. In information modelling, we typically refer to these types of kinds as classes. An entry in a database is an instance. In other words, the real-world stuff we put into information systems are instances of stuff, and we conventionally instantiate our stuff by saying it is one of a given class of thing.

In the conventional paradigm, classes (and the relationships between them) make up the system’s conceptual model. (The model is a digital representation of the ontology of the domain, from our perspective.)

However, another option exists. After all, why do things need to be a kind before they are a thing?

As the “You are a person” example illustrates, every instance is actually unique. By classifying everything, all the time, we may actually degrade the quality of information in that instance. As @nickmilo’s example shows, when we classify a person as “born in the 1980s,” we tend to lose the data that they were “born in Canada.” (That is a strikingly bad conceptual model, but hopefully it illustrates the point.)

As it happens, by developing these ideas in this thread, y’all have essentially reproduced my PhD supervisor’s early papers (e.g., “Emancipating instances from the tyranny of classes in information modeling”).

This—all of this!—is why flatter structures tend to be better for information quality. This is especially true when the purpose of the information isn’t known from the start, or when the information may be used in ways originally unintended.

In order to resist the paradigm of classify-first information-later, we can embed as much richness in our instances as possible. In other words, when capturing information, go for richness, and include rich semantics and metadata so that you’re representing the thing you’re capturing as completely and usefully as possible.

In other other words: when you’re note-taking, capture rich notes, use useful tags, and keep good metadata (e.g., always list a book’s title in the same form along with a quote from the book).

Obviously, though, just because flat structures are good for information quality doesn’t mean they’re good for information use. Classification has many cognitive benefits. Economy is one: it is easier to think of a person by representing only their basic demographics than by empathizing with their person-hood—especially at scale. Inference is another: when we see a piece of data and know that it’s a Person, we can infer that it has a birth date and some kind of citizenship.

So what? Well, if we’ve captured rich data in a flat structure, we can then layer purpose-built conceptual models on top of that data.

In other words, with a flat data structure, we can use tools like smart searches, tag panels, and ontology notes (what Nick calls “maps of content” ) to view, filter, query, and organize rich data according to whatever need we might have in a given moment.

None of will be new or surprising to most, I suspect, but I hope it’s interesting to note that there’s science behind the conclusions that have been discussed already!

In terms of this thread, I think the key practices people should adopt are:

• Capture notes with richness and good metadata.
• Flatten your data structure as much as possible.
• Create purposeful relationships between notes. I.e., don’t create conceptual models—that is, build organizing structure, or relate notes to one another—based on some predicted, anticipated need. Only organize your notes according to an actual, current need.
  • Such a need, of course, could simply be to explore an interest of yours. The “need” to put your newest note somewhere doesn’t count.
• When you do have a purpose, choose how to organize (and what tools you should use) based on that purpose.
  • If exploring an interest is the purpose, a Map of Content/Ontology note is probably be best approach to organizing your notes.
  • If managing inline tasks while you research is the purpose, creating a backlink collection with a pseudo-tag (as explained here by @deftdeg) is probably the best organizing approach. Hashtags could also suit.
  • If you’re putting together a project or a publication, a speculative outline is probably the best organizing approach.

I hope this landed! If it did, a question for anyone interested: what other purpose → organizing approach pairs are out there?

…apparently I’ve been paying attention in my PhD

[Eugleo]

@nickmilo Thanks, as always, for bringing up an interesting problem with an interesting set of views. And @ryanjamurphy — bravo, the comment is beautifully written and contains a lot of insights.

I wholeheartedly agree that flat structures with metadata are better for organisation, at least in principle.

Why is flat better than structured?

This point is even not that hard to argue: Given the choice of sticking to one organisational principle (folders, categories, or whatever else) or being able to summon any of the aforementioned organisational mental tools (or different views, as I prefer to call them) and even combine them to suit the problem at hand — who in their right mind would choose the first option?

You might ask, why do people keep choosing the first option then? In my opinion, the problem is that the flatter (and the nonexistent-er) the structure, the better the metadata need to be in order for you to be able to find your way through the notes and make some sense of them. Saved searches, MOCs and all the other views work if and only if the metadata are good.

What does it mean for metadata to be “good”?

You need to have as much information in them as possible (i.e. including the book author, name, topics, year, publisher etc), but crucially you also need the metadata to be structured in a predictable way so that the app can work with them (i.e. not having #habit and #custom).

I also slightly disagree with the following quote:

Only organize [sic] your notes according to an actual, current need.

I believe that “organising” is a bad mindset, even if you’re thinking in the context of metadata. When adding metadata, don’t think about your “actual, current need” — think about the content of the note, about the raw information that’s included in there. Add the metadata based on that and you can be sure you aren’t hiding your primal categorising instincts behind a veil of metadata.

How to write good metadata

Anyway, we’ve settled that the most important part is having quality metadata. Fortunately, an easy solution seems to be at hand (to take another friendly stab at @ryanjamurphy):

In other other words: when you’re note-taking, capture rich notes, use useful tags, and keep good metadata (e.g., always list a book’s title in the same form along with a quote from the book).

I.e. “just write good metadata, people!”. Of course, that doesn’t work that well — think about how hard it is to assign a note to a single folder. If you think that providing more choices (tags and other kind of metadata) simplifies things, think twice.

The problems

The original topic of this thread, as I see it, is: what views can you employ to gain insight from your notes? And it turns out we already have different tools at our disposal for viewing a properly metadata-ed note in a variety of contexts: see the bottom of the comment above for three examples.

The real problem (or rather, the one left unsolved) is filling in the relevant metadata in such a way that so that the views are useable. We need a system for adding metadata, but we need to do it carefully, in order not to end up with “folders 2.0”.

Fools and tools

If you remember, I split the problem of metadata in two categories (I love me some good categories): content and form.

While the burden of doing the content-part right lies mostly on the writer, I think that the form-part could be solved by our tools for us.

If you tag a note #habit and another one #common, the app can list them both if you search for #habit. This alone would solve a big problem that a lot of people are having with zk: it requires a mighty discipline to keep them useable.

Also, did I say that the content burden lies on the writer? Why not have the app generate the metadata for you, based on the contents of the note, on the context it was written in and on the links that lead from it?

Conclusion

Flat structures are great for information store, but not for information retrieval. In order to view your notes using different tools, you need to provide good metadata for them: that is metadata with enough information and in the right form.

We need to devise some system that we’ll use for metadata in the short run (deciding both on what the content should be and what the form should look like), and I’m happy to discuss about it.

However, there’s big untapped potential for some tool to step in and help us solve both of those issues — I, for one, am looking forward to that day.

[Klaas]

@nickmilo Just a remark about Dewey: someone (I forget who) in the knowledge-management section mentioned Johnny Decimal, which is a simplified version mostly for personal use. JD might be a good alternative for Dewey’s because the latter is quite complex and more suited for libraries, from what I understand.

[nickmilo]

@ryanjamurphy This post landed! Very rich and well laid out ideas. I would like to see you continue building out these use cases. Definitely deserves a standalone thread, if you’d like to make one (or I can later). It’s closely intertwined with this topic on the ‘relationships between notes’.

Now that we have this excellent discourse on the tools, I do think next logical step is indeed a deeper dive into all the various purposes people have to use the tools in specialized ways.

[ja_rule]

Indeed this is a great discussion, and I really appreciate the perspectives of @ryanjamurphy and @Eugleo with respect to creating the flattest structure possible, backed up by really awesome metadata and tools to create the right view at the right time. As @Eugleo said:

“just write good metadata, people!”

Easier said than done…

In response to @nickmilo’s request to take a deeper dive into the purpose(s) of these tools, I’d suggest that I have two broad categories – that word again! – of need, which are quite different and drive a fair bit of the conceptualization disconnect. The first is simple and practical: I just want to remember how to do stuff that I already “know” how to do, or recall details of some stored but not particularly important piece of information. Some examples:

1. I need to look up an address or other contact information
2. If I only do some task every 6 months, I forget the details, and it frustrates me
3. There’s some specialized syntax to get my LaTeX figure or jupyter plot to look just right

I’d put all these under the general heading of Augmented Memory. I forget facts and details over time, and this is stuff I don’t need to burden my brain with remembering, so long as I have a way to recall it quickly and without much effort. This may be boring, but it’s both necessary and the more common use of a note taking system.

The second category is far more interesting and is generally the one I think folks in this thread like contemplating, which I’d put under the heading of Aspirational Thinking: I want to have amazing new thoughts that no one has ever had before! And I want this system to help me do that. Asimov’s notion of creativity, which heavily shapes my thinking here, held that new ideas come from people taking two [or more] things they already know and combining them in a new way. From what I’ve read about Zettelkasten, this seems entirely consistent with taking all the things I know, breaking them into index-card-digestible pieces, and noodling on their possible linkages, at which point new ideas will reveal themselves.

Given these two different uses – the mundane and the revelatory – I’d like a system that can handle both, and I think there’s a commonality that can be leveraged. The problem with human memory is that we lose the handle – the pointer if you’re a computer nerd – to stuff that our brain knows. The challenge is to find a way to refresh the handle or at least give ourselves more places to grasp on to the things we already know, if only we had the pointer to it. As Tiago Forte said in his note on Progressive Summarization:

Once we capture something, how do we structure the note so that it’s easily discoverable and usable in the future? How do we make sure what we’re saving today adds value to future projects, even when we can’t predict or even imagine what those projects might be?

There’s agreement here to add lots of metadata, as @ryanjamurphy said:

In other other words: when you’re note-taking, capture rich notes, use useful tags, and keep good metadata.

But there’s a part of me that see this as just adding many classes, instead of just one, when I add a note to the system.

In an attempt to tie these thoughts together, I see base classes or categories as a set of default handles or entry points that I can grab onto without much effort. I may remember that an idea was sparked by a particular author or a conversation with a colleague, so my entry point in either case is Person. So I can think of “Asimov” or “Luhmann” or “Forte” or “ryanjamurphy” as short-hand for a body of thought, a handle or pointer to a complex set of concepts. A tool that can help me follow this entry point to a web of other ideas is what I’m looking for.

[masonlr]

@ja_rule

The problem with human memory is that we lose the handle – the pointer if you’re a computer nerd – to stuff that our brain knows.

I’m taking Fun Fun Function’s article on Composition over Inheritance and replacing “types” with “notes”:

Inheritance is when you design your types notes after what they are, while composition is when you design your types notes after what they can do.

@ryanjamurphy’s brilliant comments on purpose are echoed:

But the really big problem with inheritance is that you’re encouraged to predict the future.

@nickmilo’s pigeonholing comments are echoed:

We simply cannot fit the MurderRobotDog nicely into this inheritance hierarchy.

The process of forging MOCs/ontology notes is echoed:

What it writing a MOC does is that it takes an object takes a note, in this case a new, empty object empty note, and assigns the properties from other objects to it references other notes from it.
…
So in this case, it creates a barker, a driver, a killer, and then merges them all into the new object, and returns it.

I’m not trained in information modelling or computer science, so would really value a discussion on the insights from these fields. For example, the rich metadata discussion above feels like structural subtyping. Obsidian itself feels like an integrated development thinking creativity environment – the common core of vscode has spawned powerful user extensions, and I’m looking forward to the equivalents in Obsidian.

[ryanjamurphy]

Thanks, all, for expanding and advancing the discussion in this thread! I am learning much. My thinking is definitely shifting, though to what, I’m not sure.

I will try to add to the major themes I’ve picked up.

Solutions to metadata note capture and their problems

Both @Eugleo and @ja_rule point out the tension—paradox, really—in maintaining “good metadata” without falling into the trap of predictive organizing.

I think solutions to this tension fall into four categories:

1. Go for volume. When writing notes, add as many kinds of metadata as you can think of in the form of tags, key phrases, and so on.
2. Automation: your fAIry godmother. Use scripting and other automation tools to add contextual information based on any available cues. E.g., you may use a script to add a project-specific tag to any note created while a Toggl timer is running, or you could detect any note that mentions “note-taking” and “knowledge management” in close proximity and add a “PKM” tag.
3. Don’t sweat the small stuff. Assume that your notes will contain relevant metadata based on the fact that they’re actually about something, and that when you need it you’ll be able to search for it.
4. Moments Notes snap together like magnets. The metadata isn’t in or attached to the notes themselves—it’s in the relationships between the notes. Use linking and structural notes to develop themes and clusters. This is essentially grounded theory for your notes.

(1)’s challenges have been discussed at length above. The practice of adding a bunch of speculative tags or key phrases to a note as you write it is really just a sneaky kind of predictive organizing.

(2) has both a shallow and a deep manifestation.

Shallow: construct Rube-Goldberg machines of triggers and actions to enrich your notes with automatically-added metadata. While this seems great, arguably you are just doing predictive organizing behind the veil of automation. You still have to make predictions about what kinds of conditions will add useful context later on.

The deep implementation is essentially waiting for a fairy to solve your organizing problems. There are some neat innovations taking root in this space (e.g., use graph analytics to algorithmically identify important relationships in data, use machine learning to identify key features of data). However, in my opinion, these are way off from applicability (especially in the general purpose sense). They’re also way over my head without a big cup of coffee and a few interrupted hours spent understanding them.

(3) is effectively Tiago Forte’s position. Your notes already contain the metadata in the content. Organize according to non-content rules and use your computer’s search capabilities to find what you need, when you need it. A key problem with this approach, though, is that it struggles with scale. Even though I have great search tools, it still takes me far too long to find key ideas in 10,000+ PDFs.

(4) involves processing, and is the most hands-on of these solutions—though, as smarter people than I have argued, “processing is the work.” At a basic level, this is the approach Luhmann took, and it is the root idea of the compositions @masonlr mentions. It is also evident in @nickmilo’s structural Map of Contents notes and other tools. However, if you don’t have time at the front of the workflow, you won’t do this—thus it requires a building of habit.

Further thinking on purpose-based note organizing strategies

Earlier I advocated for pushing as much organizing as possible to the use-case—that is, try to embed information in your notes, and then come up with effective ways of finding and using that information based on what you need it for.

In turn, I suggested that there were probably “design patterns” in note use, such as using a pseudo-tag with tasks you want to review later.

A paradox is that in order to implement these kinds of patterns, there is an implication that you need some anchoring metadata in the note data. In other words, you do need to kinda predict the use when you’re writing it—else you wouldn’t add the pseudo-tag.

Here, I think it may be useful to delineate the different kinds of purposes. @ja_rule mentions “augmented memory” and “aspirational thinking” (I have been calling the latter “augmented creativity,” as it happens). These might be useful categories in figuring out design patterns and recognizing when to use “anchor” metadata.

I’ll first focus on aspirational thinking. I think this kind of use-case is best suited for purpose-based organizing patterns like Maps of Content. In other words, you should never try to predict aspirational thinking organizing needs.

The former is, as @ja_rule put it, mundane—and, I think, the most insidious. It’s the mundane that most depends on our at-time-of-capture good metadata. This is a nice insight, because it means you can probably relax about metadata unless it’s something obvious, like a task or a person.

So, if this rings true, it means that good metadata depends on whether the information we’re looking to organize is of a mundane or aspirational purpose. If it’s the former, use organizing design patterns that make sure you have a standard anchor to that data over your database (e.g., a task pseudo-tag). If it’s the latter, use organizing design patterns like note linking and Maps of Content to make sure you can trace back to the idea based on some creative need.

[masonlr]

I’m trying to work out whether there is an analogy between MOCs and the “mapping” process in object–relational mapping. One similarity that we’re possibly facing is the impedance mismatch problem.