In what ways can we form useful relationships between notes [LONG READ]

@nickmilo Thanks, as always, for bringing up an interesting problem with an interesting set of views. And @ryanjamurphy — bravo, the comment is beautifully written and contains a lot of insights.

I wholeheartedly agree that flat structures with metadata are better for organisation, at least in principle.

Why is flat better than structured?

This point is even not that hard to argue: Given the choice of sticking to one organisational principle (folders, categories, or whatever else) or being able to summon any of the aforementioned organisational mental tools (or different views, as I prefer to call them) and even combine them to suit the problem at hand — who in their right mind would choose the first option?

You might ask, why do people keep choosing the first option then? In my opinion, the problem is that the flatter (and the nonexistent-er) the structure, the better the metadata need to be in order for you to be able to find your way through the notes and make some sense of them. Saved searches, MOCs and all the other views work if and only if the metadata are good.

What does it mean for metadata to be “good”?

You need to have as much information in them as possible (i.e. including the book author, name, topics, year, publisher etc), but crucially you also need the metadata to be structured in a predictable way so that the app can work with them (i.e. not having #habit and #custom).

I also slightly disagree with the following quote:

Only organize [sic] your notes according to an actual, current need.

I believe that “organising” is a bad mindset, even if you’re thinking in the context of metadata. When adding metadata, don’t think about your “actual, current need” — think about the content of the note, about the raw information that’s included in there. Add the metadata based on that and you can be sure you aren’t hiding your primal categorising instincts behind a veil of metadata.

How to write good metadata

Anyway, we’ve settled that the most important part is having quality metadata. Fortunately, an easy solution seems to be at hand (to take another friendly stab at @ryanjamurphy):

In other other words: when you’re note-taking, capture rich notes, use useful tags, and keep good metadata (e.g., always list a book’s title in the same form along with a quote from the book).

I.e. “just write good metadata, people!”. Of course, that doesn’t work that well — think about how hard it is to assign a note to a single folder. If you think that providing more choices (tags and other kind of metadata) simplifies things, think twice.

The problems

The original topic of this thread, as I see it, is: what views can you employ to gain insight from your notes? And it turns out we already have different tools at our disposal for viewing a properly metadata-ed note in a variety of contexts: see the bottom of the comment above for three examples.

The real problem (or rather, the one left unsolved) is filling in the relevant metadata in such a way that so that the views are useable. We need a system for adding metadata, but we need to do it carefully, in order not to end up with “folders 2.0”.

Fools and tools

If you remember, I split the problem of metadata in two categories (I love me some good categories): content and form.

While the burden of doing the content-part right lies mostly on the writer, I think that the form-part could be solved by our tools for us.

If you tag a note #habit and another one #common, the app can list them both if you search for #habit. This alone would solve a big problem that a lot of people are having with zk: it requires a mighty discipline to keep them useable.

Also, did I say that the content burden lies on the writer? Why not have the app generate the metadata for you, based on the contents of the note, on the context it was written in and on the links that lead from it?

Conclusion

Flat structures are great for information store, but not for information retrieval. In order to view your notes using different tools, you need to provide good metadata for them: that is metadata with enough information and in the right form.

We need to devise some system that we’ll use for metadata in the short run (deciding both on what the content should be and what the form should look like), and I’m happy to discuss about it.

However, there’s big untapped potential for some tool to step in and help us solve both of those issues — I, for one, am looking forward to that day.

5 Likes

@nickmilo Just a remark about Dewey: someone (I forget who) in the knowledge-management section mentioned Johnny Decimal, which is a simplified version mostly for personal use. JD might be a good alternative for Dewey’s because the latter is quite complex and more suited for libraries, from what I understand.

3 Likes

@ryanjamurphy This post landed! Very rich and well laid out ideas. I would like to see you continue building out these use cases. Definitely deserves a standalone thread, if you’d like to make one (or I can later). It’s closely intertwined with this topic on the ‘relationships between notes’.

Now that we have this excellent discourse on the tools, I do think next logical step is indeed a deeper dive into all the various purposes people have to use the tools in specialized ways.

3 Likes

Indeed this is a great discussion, and I really appreciate the perspectives of @ryanjamurphy and @Eugleo with respect to creating the flattest structure possible, backed up by really awesome metadata and tools to create the right view at the right time. As @Eugleo said:

“just write good metadata, people!”

Easier said than done…

In response to @nickmilo’s request to take a deeper dive into the purpose(s) of these tools, I’d suggest that I have two broad categories – that word again! – of need, which are quite different and drive a fair bit of the conceptualization disconnect. The first is simple and practical: I just want to remember how to do stuff that I already “know” how to do, or recall details of some stored but not particularly important piece of information. Some examples:

  1. I need to look up an address or other contact information
  2. If I only do some task every 6 months, I forget the details, and it frustrates me
  3. There’s some specialized syntax to get my LaTeX figure or jupyter plot to look just right

I’d put all these under the general heading of Augmented Memory. I forget facts and details over time, and this is stuff I don’t need to burden my brain with remembering, so long as I have a way to recall it quickly and without much effort. This may be boring, but it’s both necessary and the more common use of a note taking system.

The second category is far more interesting and is generally the one I think folks in this thread like contemplating, which I’d put under the heading of Aspirational Thinking: I want to have amazing new thoughts that no one has ever had before! And I want this system to help me do that. Asimov’s notion of creativity, which heavily shapes my thinking here, held that new ideas come from people taking two [or more] things they already know and combining them in a new way. From what I’ve read about Zettelkasten, this seems entirely consistent with taking all the things I know, breaking them into index-card-digestible pieces, and noodling on their possible linkages, at which point new ideas will reveal themselves.

Given these two different uses – the mundane and the revelatory – I’d like a system that can handle both, and I think there’s a commonality that can be leveraged. The problem with human memory is that we lose the handle – the pointer if you’re a computer nerd – to stuff that our brain knows. The challenge is to find a way to refresh the handle or at least give ourselves more places to grasp on to the things we already know, if only we had the pointer to it. As Tiago Forte said in his note on Progressive Summarization:

Once we capture something, how do we structure the note so that it’s easily discoverable and usable in the future? How do we make sure what we’re saving today adds value to future projects, even when we can’t predict or even imagine what those projects might be?

There’s agreement here to add lots of metadata, as @ryanjamurphy said:

In other other words: when you’re note-taking, capture rich notes, use useful tags, and keep good metadata.

But there’s a part of me that see this as just adding many classes, instead of just one, when I add a note to the system.

In an attempt to tie these thoughts together, I see base classes or categories as a set of default handles or entry points that I can grab onto without much effort. I may remember that an idea was sparked by a particular author or a conversation with a colleague, so my entry point in either case is Person. So I can think of “Asimov” or “Luhmann” or “Forte” or “ryanjamurphy” as short-hand for a body of thought, a handle or pointer to a complex set of concepts. A tool that can help me follow this entry point to a web of other ideas is what I’m looking for.

5 Likes

I’m taking Fun Fun Function’s article on Composition over Inheritance and replacing “types” with “notes”:

Inheritance is when you design your types notes after what they are, while composition is when you design your types notes after what they can do.

@ryanjamurphy’s brilliant comments on purpose are echoed:

But the really big problem with inheritance is that you’re encouraged to predict the future.

@nickmilo’s pigeonholing comments are echoed:

We simply cannot fit the MurderRobotDog nicely into this inheritance hierarchy.

The process of forging MOCs/ontology notes is echoed:

What it writing a MOC does is that it takes an object takes a note, in this case a new, empty object empty note, and assigns the properties from other objects to it references other notes from it.

So in this case, it creates a barker, a driver, a killer, and then merges them all into the new object, and returns it.

I’m not trained in information modelling or computer science, so would really value a discussion on the insights from these fields. For example, the rich metadata discussion above feels like structural subtyping. Obsidian itself feels like an integrated development thinking creativity environment – the common core of vscode has spawned powerful user extensions, and I’m looking forward to the equivalents in Obsidian.

3 Likes

Thanks, all, for expanding and advancing the discussion in this thread! I am learning much. My thinking is definitely shifting, though to what, I’m not sure.

I will try to add to the major themes I’ve picked up.

Solutions to metadata note capture and their problems

Both @Eugleo and @ja_rule point out the tension—paradox, really—in maintaining “good metadata” without falling into the trap of predictive organizing.

I think solutions to this tension fall into four categories:

  1. Go for volume. When writing notes, add as many kinds of metadata as you can think of in the form of tags, key phrases, and so on.
  2. Automation: your fAIry godmother. Use scripting and other automation tools to add contextual information based on any available cues. E.g., you may use a script to add a project-specific tag to any note created while a Toggl timer is running, or you could detect any note that mentions “note-taking” and “knowledge management” in close proximity and add a “PKM” tag.
  3. Don’t sweat the small stuff. Assume that your notes will contain relevant metadata based on the fact that they’re actually about something, and that when you need it you’ll be able to search for it.
  4. Moments Notes snap together like magnets. The metadata isn’t in or attached to the notes themselves—it’s in the relationships between the notes. Use linking and structural notes to develop themes and clusters. This is essentially grounded theory for your notes.

(1)’s challenges have been discussed at length above. The practice of adding a bunch of speculative tags or key phrases to a note as you write it is really just a sneaky kind of predictive organizing.

(2) has both a shallow and a deep manifestation.

Shallow: construct Rube-Goldberg machines of triggers and actions to enrich your notes with automatically-added metadata. While this seems great, arguably you are just doing predictive organizing behind the veil of automation. You still have to make predictions about what kinds of conditions will add useful context later on.

The deep implementation is essentially waiting for a fairy to solve your organizing problems. There are some neat innovations taking root in this space (e.g., use graph analytics to algorithmically identify important relationships in data, use machine learning to identify key features of data). However, in my opinion, these are way off from applicability (especially in the general purpose sense). They’re also way over my head without a big cup of coffee and a few interrupted hours spent understanding them.

(3) is effectively Tiago Forte’s position. Your notes already contain the metadata in the content. Organize according to non-content rules and use your computer’s search capabilities to find what you need, when you need it. A key problem with this approach, though, is that it struggles with scale. Even though I have great search tools, it still takes me far too long to find key ideas in 10,000+ PDFs.

(4) involves processing, and is the most hands-on of these solutions—though, as smarter people than I have argued, “processing is the work.” At a basic level, this is the approach Luhmann took, and it is the root idea of the compositions @masonlr mentions. It is also evident in @nickmilo’s structural Map of Contents notes and other tools. However, if you don’t have time at the front of the workflow, you won’t do this—thus it requires a building of habit.

Further thinking on purpose-based note organizing strategies

Earlier I advocated for pushing as much organizing as possible to the use-case—that is, try to embed information in your notes, and then come up with effective ways of finding and using that information based on what you need it for.

In turn, I suggested that there were probably “design patterns” in note use, such as using a pseudo-tag with tasks you want to review later.

A paradox is that in order to implement these kinds of patterns, there is an implication that you need some anchoring metadata in the note data. In other words, you do need to kinda predict the use when you’re writing it—else you wouldn’t add the pseudo-tag.

Here, I think it may be useful to delineate the different kinds of purposes. @ja_rule mentions “augmented memory” and “aspirational thinking” (I have been calling the latter “augmented creativity,” as it happens). These might be useful categories in figuring out design patterns and recognizing when to use “anchor” metadata.

I’ll first focus on aspirational thinking. I think this kind of use-case is best suited for purpose-based organizing patterns like Maps of Content. In other words, you should never try to predict aspirational thinking organizing needs.

The former is, as @ja_rule put it, mundane—and, I think, the most insidious. It’s the mundane that most depends on our at-time-of-capture good metadata. This is a nice insight, because it means you can probably relax about metadata unless it’s something obvious, like a task or a person.

So, if this rings true, it means that good metadata depends on whether the information we’re looking to organize is of a mundane or aspirational purpose. If it’s the former, use organizing design patterns that make sure you have a standard anchor to that data over your database (e.g., a task pseudo-tag). If it’s the latter, use organizing design patterns like note linking and Maps of Content to make sure you can trace back to the idea based on some creative need.

9 Likes

I’m trying to work out whether there is an analogy between MOCs and the “mapping” process in object–relational mapping. One similarity that we’re possibly facing is the impedance mismatch problem.

1 Like

Another analogy can be made to the command query responsibility segregation (CQRS) pattern: one of the reasons that we’re adding and maintaining MOCs is to support future exploration. The way we get ideas in (e.g. through evergreen notes) can be different from the way that we get ideas out (e.g. through MOCs).

2 Likes

Was thinking late last night about these Map of Contents that I have been reading about as a way to organize my notes. The analogy of Congress came to my mind. Specifically, each Congress Person is a “note.” And each Congress Person belongs to a whole slew of caucases (i.e. MOCs). They belong to their:
- State
- Political Party
- Various interest groups
- Regional interests
- Ethnic affiliations
- Gender associations
- Congressional Committees
- Religious/Non Religious Affiliations
- And how many more ways to cut and dice them?

  • But the idea is, that these “identies” or “affiliations” of the individual Congress Persons overlap and give multiple perspectives to each of the individual Congress Persons or “notes” in the same way that MOCs can be structured in the Obsidian database. In other words, each Congress Person belongs to multiple “caucauses” or “MOCs”.
  • So my next question is, just how do I structure the syntax of both the individual Congress Person’s “note” and the various MOC notes to intuitively move back and forth between and among them all? Is a MOC in essence a pre-defined filter? Or does my analogy just confuse me with my emerging understanding?
4 Likes

No, that could work.

Say you have a Ryan J. A. Murphy person. In the Ryan J. A. Murphy note, you’d add the caucuses they’re a member of:

# Ryan J. A. Murphy

Caucuses: [[Technology and Communications]], [[Research and Higher Education]]

You could qualify/describe those caucuses in the note as much or as little as you want.

Then, in a [[Technology and Communications]] note, you might mention who the members have been at given points in time.

# Technology and Communications
## 2019-2020
The caucus has prioritized rural Internet access this year while maintaining net neutrality.
### Members:
[[Ryan J. A. Murphy]], [[kdjamesrd]]

The person note includes a linked directory of the caucuses they’re involved in, and the caucus note becomes a linked directory of the members that have been involved in it.

This example is a little contrived, though, because without note “data” (e.g., what you’d actually be writing about in the note!) it looks like a spreadsheet or database software’s a better fit for this data. The real value of the MoC concept is that you can map and direct yourself towards linked notes in-line with descriptive, qualitative information about the concept at hand. Structure and content are provided simultaneously.

2 Likes

Copying and Pasting from my thoughts initially posted in #knowledge-management on Discord.

In response to @nickmilo 's question:

I keep asking for the community to think of good metaphors that cover the three parts: (1) curating the relevant notes into one place, (2) working on them: rearranging/merging/deleting/adding/etc (my favorite part), and finally (3) having that map act as a summation of thought (which should continue to evolve into the future)

A Metaphor: Notes as a Train…

They’re like a train. People all get on the train at some point. At stop one the first passengers (ideas) get on board. More get added at each stop. Some get off. All the passengers are on the train at some point. Sometimes passengers have a reserved seat. Sometimes there is only two people on the entire train. Sometimes it’s packed and people have to sit on the floor. Even if you have a reserved seat you aren’t obligated to sit in that seat. You can sit somewhere else. You can give up your seat to someone else. You can move between carriages. You can get off three stops earlier than you intended, or if you’re sneaky, you can hide in the bathroom and stay on three stops longer. The train has a destination, but it also has many destinations. The train never has a final destination because when it reaches the end, it will likely wait for more passengers and then head back the way it came. With new passengers. Or perhaps old passengers, returning from their day trip. Or maybe the train is designated an entirely new route. Maybe it’s picked up off the tracks and dropped and plonked onto a new route. Maybe it’s getting old and is taken out of commission so it goes into the train museum. When the train is on the route, the tracks can always change. It may be stalled and have to sit stagnant and waiting for go ahead.

The passengers have to sit and busy themselves with the idle time. Maybe someone hits the wrong lever and the tracks change and instead of ending up at Brighton, then end up in Grand Central Station. There are all sorts of trains. There are steam trains, and their are high-speed trains, there are double-Decker trains, there is the orient express and there is the train that runs only once a week between two small villages. There is the train that goes underground and crosses an ocean. Every passenger whilst on the train is heading somewhere, but wherever they end up also isn’t their final destination. They may get off at stop three, but they don’t sit at stop three’s train station all day. They go elsewhere. You can speak to your neighbour or you can put in your headphones. You can observe the family sitting a few seats down from you. You can stare out of the window. A train has a map that it follows, but there are many many maps. But all train tracks will in some way or other be connected with all other tracks. Just as all roads lead to Rome, and all rivers lead to the ocean. A train can be a single carriage, or it can be one hundred long. A train can be de-couple or coupled up or quadrupled up. People can jump on board without a ticket. There are conductors that check the tickets and make sure everyone is actually supposed to be there. But they can also make exceptions if they’re feeling particularly kindly.

There was also something about it being a playground, or a painters easel or a whiteboard, but I’m not sure. (You have an easel or a white board, and you can have lists, but you can also wipe them out at will). All your tools are sitting in the little tray attached to the board or easel, but you can always swap out the tools that you use.

I think a map can be a little too definitive. Sometimes a map lasts a thousand years, and yes, maps have to be updated to stay relevant, but I feel the initial idea of a map tends to be a little more fixed. Unless you mean a map as in say, Google Maps, where the map is everywhere, the terrain, the atlas, if you will, and the route that you punch in is the your notes and ideas, and sometimes you have regular routes, like to work, or the supermarket. But sometimes you go to a new shop, or you change job, and you have to put a new route into the map. Map for me conjures up a pirate’s map, a map to x marks the spot, and whilst it could be said that you don’t have to follow a map to a T, you usually are quite passive when following a map. You’re looking at it expecting it to take you to the X, but you usually know what the X is, because that’s why you picked up the map in the first place (or that’s why you started to draw your own map to get there).

Also I’d add that, to bolster the thinking factor, people get inspired when they’re on a train. They’re planning what they’re going to do when they get to their destination. Or they get inspired to start planning another trip entirely. Or they’re just busy thinking about that stupid conference call they have to get on later, or that they forgot to pick up this ingredients for lasagna. But all these people have their own webs, their own lives, even whilst they all sit in the same carriage. There is the world beneath, and then there is the carriage itself, the train itself. The MOC is the train. The notes are the passengers. Or something like that.

9 Likes

@masonlr Good thinking here. If this is helpful for your synthesizing, perhaps a helpful way of considering MOCs is through emergence.

MOCs are Evergreen Notes, just at the next level of emergence.

2 Likes

I agree that filing you in the 80s folder would be the wrong place to start, but filing you in the People folder would be perfectly rational. Linking you to your hobbies and likes/dislikes would be a reasonable choice.

@ja_rule Returning to this notion of “base objects”, I’m currently using a few base object / resource folders and haven’t ran into any issues yet: Images, People, Quotes, and Source Materials (things written by other people). I don’t have to use these folders; I could easily add their contents into the main vault; but I slightly prefer keeping them slightly more tidy. I like your description of base objects, thanks.

2 Likes

Yes to this! The analogy of notes as people is a strong one. As is the caucus. I’ll add that the purpose of an MOC is to have a sort of evergreen, ongoing, caucus.

In an MOC, the party is always happening. Individual notes can “shadow-clone” themselves and essentially be in multiple parties simultaneous—interacting and developing complexity from each party at the same time!

1 Like

I may give this folder structure a try. I’ve been using tags and links more for finding things and currently use folders for Types of notes, but I’m not happy with it in some ways. However, the advantage to using Folders in Obsidian is that we can start typing for a link and it will include the folder, giving you an idea of it is a Project, Idea, Meeting note, etc. What I don’t like is deciding which folder to put things in. I tend to add more folders than I need and always have to cut back.

I’m now almost 2 months into Obsidian use, and I’ll say that I’m reasonably happy with where I landed, with slight modifications. As I noted somewhere up-thread, I’ve taken a hybrid approach, with relatively few base folders, and most of my notes going into date-based buckets. So I have a top-level 2020 folder, and in this an 06 folder for June, an 07 folder for July, etc. The main change I’ve made is that each new note starts with a DD designation for the day followed by a short, descriptive name. So notes created today would look like “15 My awesome thoughts on PKM”, where previously I hadn’t prepended the DD. I found a bucket full of alphabetically sorted notes less helpful than date sorted. My daily notes get just the DD, with no label. This naturally sorts everything in the File Explorer by date, which is great, and is perhaps my compromise with the hardcore Zettlekasten folks who prefer the 20200715104223 type identifiers. I average 3-4 notes a day including the daily note, so at the end of the month I’ve got 90-120 files in my monthly folder, which is a manageable chunk size for me. The daily notes, with their very short DD name, make it visually easy to scroll through the list, since the extra blank space breaks things up by day, and it’s oddly satisfying to open a new month with a fresh folder. For me, remembering roughly when I was thinking about something is a useful axis for recall.

I try to make everything a regular note, meaning it just goes into the date based hierarchy with the above naming scheme. I’ve also had a [small] bit of stress over the decision of “should this be a base note or go in the date bucket”, and try to just put stuff in the bucket, but be sure to link it somewhere. Topics will naturally bubble up that want to have a MOC to link them together, so I add them when I feel it’s necessary.

2 Likes

I’m starting from 0 with Obsidian and spent a decent amount of time last week reading interesting lines in this forum. This post really clarified many assumptions I’ve made and the approach I’d like to implement and leverage within my personal PKM.

I’m definitely leaning towards a radical flat approach based on “as atomistic notes as possible”. There is something which however is still now working in my mental model and I’m still pretty confused about… notes titles.

In principle, I’m leaning towards not giving notes titles and just navigating notes through backlinks and graphs. Reasons:

  1. Note titles seem an unnecessary layer and a first, altough inherently preclusive, attempt to classify a body of information/knowledge which I want to develop differently. I’d like the system to supports my thinking at higher levels of abstractions and spur unexpected connections rather than labeling and siloing things.
  2. Considering that I want to take super atomistic notes, in many cases, the title would just be a repetition of the note body.

The question is, would this approach be viable at all or is just supported by my current limited understanding of the topic? Would really appreciate if you can share your thinking.
To let you better understand what I’m trying to achieve, you should know that my use case revolves around:

  1. Consolidating professional knowledge scattered around years of folder based reading and note taking
  2. Develop knowledge in a variety of personal interests

Also, haven’t checked yet evergreen notes and the maps of content which seems pretty crucial concept to develop the PKM along the lines I’m foreseeing.

Finally, could you suggest some intro reading (but not too basic) about
ingormation modelling?

1 Like

I can see the argument for not titling. For me, titles are kinds of cognitive landmarks—when I read a title I can generally remember-imagine what’s in the note. I think that’s useful, but I’ve never tested the alternative. It might be worth trying it out—give yourself a week, leave titles off, and see how it feels.

I will have to get back to you on some readings on information systems/conceptual modeling. One of the gaps in the field is knowledge translation—that is, providing practical insights based on the theory. A lot of what has been done is focused on conventional databases and design for major organizational use, not individualized PKM as we’re discussing. So there might not be much on it as yet! I’ll have a look, though.

@paoloap Andy mentioned that Evergreen note titles are like APIs. The titles become abstractions for the notes itself, and an entire note can be referenced using the title.

APIs is a crucial component to effective high-level programming (higher abstraction level of thinking), since we are reusing fine-grained (atomic) abstractions (note titles). Using this analogy, there are several principles of API design that we can apply to the note title design (separation of concerns, etc). Over time, as our ideas mature, we go back and refactor the APIs (refactor note title and its content) to achieve a higher level of atomicity.

I think that’s one of the arguments for note titles. Personally I employ this approach. My note titles often start out as in-the-moment-conceptualization, and as I revisit them within a day or two I often reflect better and refactor the note titles to something more representative of the notes’ ideas

8 Likes

On a more practical level, how do you link to a note that has no title?

1 Like