Increasingly Atomic Folders: A Workflow

Thank you for clarifying! I’m envious of your ability to find an audience for all of your writing—the hardest part for me is trying to figure out who would want to read the content I’m producing and how to get it in front of them—but your description very much answers my question.

…and did you just say Obsidian mobile?

@ianlhayes

Start with family and friends? Tell them they may share it with others if they find the content worth it.

2 Likes

@EleanorKonik This is a fantastic writeup thank you for taking the time to post it!

While my own approach is structurally different we both appear to follow many of the same fundamental principles:

  • processing from source to individual notes representing ideas from the source, to evergreen notes
  • we both provide meaningful titles for the concepts from the source (as your title _ writing emerged around the same time as state-level warfare_) but I create individual literature notes for each one (tied back to the original source note, which contains biblio info and also serves as the structured index of all lit notes for that source)
  • atomic notes meaning a single idea rather than specific length
    • I have an “atomic note” that is over 10 printed pages but still encapsulates a single process and concept

I don’t use the daily notes feature much but your resonance calendar idea is very intriguing and I’m extremely interested in the plugins you are working on as they sound like they could be significant enhancements to my own workflow.

There are two key differences in our systems, and I’m curious to learn more about your thought process that led to your decisions on these points:

  1. Why do you keep all of the concepts in a single literature note and then transclude those as sections into other notes? Wouldn’t you gain more flexibility by making each one its own standalone note (e.g. a separate note titled writing emerged around the same time as state-level warfare with the contents you show) that is linked from the source and can be linked to / transcluded in to any other note? I ask because I found transcluding sections to be relatively brittle because you are creating a dependency that prevents you from rewriting or reformatting the note containing the content without risking breaking transclusions. In software engineering this is called tight coupling and is something to be avoided, and I find a lot of similarities between software engineering principles and sound note taking. Additionally I’ve found it easier to scan titles in Obsidian search or in the file system than to first think “oh idea X was in source Y” – instead I can just search for X and see the title (again, writing emerged around the same time as state-level warfare) in the search, or see it directly in the file browser.

  2. Why do you write contents into an article note without also capturing them in an evergreen note? It seems like you can end up with some concepts in your evergreen notes and some concepts in your article notes, and I’m struggling to understand the reasoning. If something is interesting enough to capture then isn’t it interesting enough to capture as an evergreen note and then write an article that builds on 1…N evergreen notes including that one? That way the evergreen note is always there for future relationships with other notes (and subsequent articles) as well. To me it seems the content is “trapped” inside an article instead of free-floating as an evergreen note where it can engage in conversations and arguments with other notes freely.

(This may be more a matter of style – I tend to write my evergreen notes almost as mini essays similar to Andy Matuschak, so to me that satisfies the itch you mention about writing articles, which to me would become written output based on those mini-essay evergreen notes – with you it sounds like the articles are almost atomically-focused evergreen notes in and of themselves)

None of this is a criticism of your approach BTW, it clearly works for you. I’m just genuinely curious how we are following very similar core principles but ended up with divergence in those two respects.

Thanks again for your thoughts, its very insightful!

3 Likes

Re: Obsidian mobile, it’s currently in private beta; VIPs can access. Someone took a video of the reactions in the discord and posted it on twitter. There is a new section on the forum to collect the bugs, but overall it’s pretty amazing.

Re: Finding your audience… I just mention things I wrote when they become relevant in conversation, and periodically I stumble across outlets that cater to people like me (as a consumer) and offer to write for them. It’s a slow process, but as I’ve built my body of work, I’ve gained some regular readers. The general idea is “be interesting, talk to the people you enjoy talking to, and when you have a longform version of the thing you’re talking about — share that with people you’re already talking to about that topic.”

I’m a nerd, I know a lot of nerds who like the things I like, I write about stuff that interests me, and a lot of the time, it turns out that stuff interests other people too.

2 Likes

The answer to your question 1 is essentially laziness and the inability of the refactor plugin to “update links to headings.”

I understand the value of having each quote + annotation broken out into individual notes, but then I’d have to actually name all of them (I did in the example but I don’t always, it’s extra work that is often unnecessary), and contrary to most principals of “what should go into a pkm vault” I’ve read, I actually think it’s an enormous waste of my time to re-word everything I read before letting it have a home in my notes. I understand the value of rewording things in order to ensure one’s own comprehension — I am a teacher, after all — and I understand the dangers of “raw capture” instead of “recording your own knowledge” but most of my litnote+annotation workflow is with sources I have only temporary access to, are comprised of information I’m not so much “trying to understand” as “trying to make sure I don’t forget the details of,” so in that case, having the actual quotation (plus my annotation about why it’s important and what it’s for and what it means) is the important part. From a workflow perspective, that means that most of my litnote is exported from something like Readwise or an annotated pdf’s highlights, and going through and refactoring it it often more trouble than it’s worth (but having it searchable in my “database of knowledge” is invaluable).

If I were a student using my vault to record learning I would absolutely have a different approach, but since I’m not, forcing everything to be artificially atomic makes it harder to do things like write a book review or review the “flow” of the important parts of a particular book. And usually I remember where I read something (“Didn’t The Horse, The Wheel, and Language say something about cattle, too?” might wind up relating to a section that would not reasonably have had a title about cattle, horses, wheels, or language :stuck_out_tongue: and while I could use a source note, that adds an unnecessary step that feels like added friction) but not what the rest of the surrounding context of the data point was.

the tl;dr on that is that I totally agree with you about the dangers of the whole “tight coupling” thing but in the end I decided not to let the perfect be the enemy of the good and that saving time by minimizing extra steps was more important than having the optimally atomic layout, given my particular goals.

The answer to your question 2 is also efficiency. Why do the same work twice? Once I’ve had a particular thought, I don’t need a note that says the same thing. I’m not going to transclude my evergreen atomic note into another source, I’m going to say “This relates to [[article]] where I said thing.” Being able to do "This relates to [[article]] where I said [[thing]]" doesn’t really gain me much, and means that if I search for [[thing]] I’m going to get two results, which makes my results harder to trawl for useful information, not easier.

I almost always never write evergreen notes for the sake of writing notes. I don’t take notes “just to have them.” I take notes because I plan to use them, and thus far I haven’t hit a point where I’ve been like "gee, I should develop my thinking on topic" that hasn’t resulted in something I want to share with the world. If I notice a connection like “Suttee, Sacral Kingship, and Human Sacrifice are related in ways that haven’t been explored by any academics I’ve seen,” I’m not going to make an evergreen note about suttee, sacral kingship, and human sacrifice and then go write an article about it. I’m just going to write an article about it. Maybe I’ll create MOCs about suttee, sacral kingship, and/or human sacrifice that point to various notes I’ve taken on the subject, but the way my brain works, creating an evergreen note for each and then synthesizing them later is just unnecessary work, and I’m pathologically opposed to unnecessary work.

Basically, I only do as much work as necessary to facilitate my goals — and I think our goals are different. But maybe I’ll shift over to a more atomic-evergreen-note centric focus as I grow my vault. I’ve made other shifts and adaptations over the months!

9 Likes

I try to make an overview table when there are more than 3-5 similar concepts; that helps me to keep them consistent and also helps me find the exact thing to look for.
I remove the specific links in the leaf nodes and just link them back to the index; so that if a new concept is added it is immediately available (with 1 hop) from all the previous leaves and it all stays DRY.
It’s difficult to decide what fields to add in the overview table and keeping DRY. Sometimes I keep a file like this taxonomy index to keep things consistent and manageable. That allows for things to show up in an index, in a hierarchy, and in the files themselves in a consistent manner with little extra work.

Thanks for the thorough response! Your motivations make sense and if I were writing articles like that I would probably adopt the same approach for the same reasons.

I’d also posit that, given the way you write your articles, your articles in a sense can be considered atomic notes themselves – atomic meaning “focused on a single concept” not “short.”

Agreed on the quotes vs own words. I’m finding similar forces in my own notes – some things warrant rewriting because they are significant, others are more about being able to jog my memory by seeing the snippet of context around the quote. (I’m also finding that I need a good workflow for extracting annotations from digital documents, and extracting highlights & notes seems the reasonable approach.)

Interestingly though this is where I find the separate literature notes with phrase-based titles works well – the title alone may completely encapsulate the concept and the contents may be nothing more than a paragraph or three of quoted content from which the title was derived, with perhaps a sentence or two annotation exactly as you describe. (which is arguably what is meant by writing it in our own words…)

But again that’s a matter of style and preference I suppose, since it seems you and I are doing essentially the exact same thing except you make the titles headings instead of separate notes. So its fascinating to see us doing essentially the same thing in two different ways with the same fundamental needs and motivations. Tomato / tomahto. :slight_smile:

1 Like

Honestly I agree — it’s just that it’s so much easier to import the .md file from Readwise and then tweak it so that it has the headings and all as one big file doing it all at once… and then by the time I’m done, I can’t refactor it because then I’d have to update all of the times I’d liked to the headings :frowning:

If note refactor plugin updated the backlinks to the headings so that they pointed to the new note I would 100% do it the same way as you do for exactly the reasons you’ve stated.

(and yeah being honest a lot of the articles and newsletters do wind up functionally being atomic notes, again, just with very particular formatting (and none of the interlinks that would be valuable of it were intended for internal use — instead the interlinks point to my other “atomic notes” that are already published))

If note refactor plugin updated the backlinks to the headings so that they pointed to the new note I would 100% do it the same way as you do for exactly the reasons you’ve stated.

Hmm, this is interesting, I’m having trouble picturing what you mean here. Can you give an example?

Imagine I’ve been reading a 400 page academic text on, say, Iron Age Syracuase. As I read, I take notes in my annotation app, not Obsidian, because it’s a physical book and I prefer to use Readwise’s OCR instead of literally typing up all of my quotations. But because it’s a long book, it takes me several days, and during that time, I use my resonance calendar to record big-picture impressions that link directly to the source note “Tyrants of Syracuse by Jeff Champion.”

So far, no big deal if I atomize the actual highlights and annotations and stuff later, because I can use the source note as an index and the connections will be maintained.

But then I go ahead and imported all of theq quotes+annotations (from a pdf or kindle or Readwise or whatever; in this case it’s Readwise). I’m left with a document that looks something like this:

quote

Note: Annotation

quote

quote

Note: Annotation

quote

quote

it’s annoying because it’s not really formatted how I like it (and Readwise doesn’t give me the darn page numbers on export, or even separate out which quotes are part of the same section and which paragraphs are on their own, unless there’s an annotation to split them up), which is much more:

ch03p93 descriptive heading

quote

annotation

ch04p103 descriptive heading

quote

annotation

but the process of taking the raw data dump from my pdf or Readwise or whatever and formatting it correctly is significantly easier for me if I just go straight through and fix it without making a bunch of files. I know that there are probably plugins like Workbench and probably Refactor that can split off each section as I go using some sort of hotkey magic, but I’m a teacher, not a programmer, and hotkeys aren’t totally intuitive to me yet. So I go down the list and do it all at once.

But as I go, I realize that my sixth quote and my fifteenth quote address the same topic, so I do create a new note called “unusual uses for greek temples” and embed quote 6 (“ch3p23 greek temples were used as banks”) and quote 15 (“ch4p30 greek temples were used as a base for raiding”) into it.

I do that as I go, synthesizing thoughts and connecting things and then I get to the end and…

… if I use the Note Refactor plugin to turn all of those subheadings into individual notes with descriptive names, none of my synthesis notes will point to the right place anymore and my embeds will be broken.

Now, yeah, I could do it “right” the first time and pull each individual quote and annotation out into their own file, but that’s a lot of extra point-and-click that is the whole reason the Note Refactor plugin exists — because that kind of thing is a bigger timesink than just doing it all in one file. Plus, I kind of like having everything right there in one place for me to look at it as a block while I’m still working with it.

I could also wait until everything is formatted and then refactor it into atomized notes and then create the synthesis notes… but what if I forget one of my ideas? And even if I don’t, now I’m doing the same cognitive work twice — because it’s not like I can “turn off” the part of my brain that notices those connections originally.

So one way I do twice as much work but wind up with more “visible” atomic notes in my file bar, and the other I do half as much work but have to utilize search more heavily to find my notes again.

I decided I’d prefer to rely on search… but in a perfect world, when Note Refactor splits my long litnote, the links to what used to be headers and are now their own notes would get updated. But my understanding is that from a programming perspective that’s a pretty heavy lift.

2 Likes

Aha ok yeah I see exactly what you mean now about the note refactor plugin. As a former dev myself I would think the solution from a programmatic standpoint would be for the plugin to (1) check if the selected content contains a ^block-id and if so (2) find all instances of that block ID in the vault and update the links that contain them. (1) is pretty trivial, but changing the links in part (2) could possibly be more tricky.

For highlighting, I’ve experimented with an app that may or may not help you, called Highlighted. It lets me select passages in physical books and OCRs them, lets me tag them, and lets me group them together and attach the page number as well, then export to text or Markdown or PDF. The end result is a large block of text similar to what you describe but I think it is broken down into sections a bit more, because I can essentially pre-chunk a bit in the app. It also lets me type in notes along with the highlights and keeps them in the output.

Example from Ahrens book:

p. 65

It is not only impossible to a focus on more than one thing at time, but also to have a different kind of attention on more than one thing at a time.

When it comes to focused attention, we focus on one only, something we can sustain for only a few seconds.

Focused attention is different from "sustained attention, which we need to stay focused on one task for a longer period and is necessary to learn, understand or get something done.

#term

I make no claim the page numbers are correct – it tries to automatically assign page numbers and seems to use the kindle or a pdf version which is usually off a bit, and this is where its my turn to be lazy and not bother updating it! :smiley:

The ... (which I manually add as free-text notes in the highlight group) signifies that those are multiple highlights I captured from different places in the book (usually quite close together) and assigned to a single “note” in the app, all attached to the same page (again, maybe the wrong page ¯\_( ツ )_/¯).

The Focused attention is different ... at the end is my own free text note added in the Highlighted app, included in the exported text. #term is a tag I created to denote defined terms in the extract. The app supports tagging as a first class construct.

The ability to add my own notes directly in the same group allows me to also add notes like these, which are attached to another single group of highlights:

N) New ideas must overcone existing rotines to tale hold

N) Silo busting optimizes flow

N) innovation created hidden second order benefits

These were conceptual evergreen note or lit note titles I thought of while extracting the highlighted content so I just typed them directly in the app into the highlighted note as free text. Just to jog my memory when I process it after extraction. They may or may not make the cut when processing.

One weird bug seems to occur though where sometimes the passages in a section with multiple highlights (the ... stuff, which I manually add between captured highlights in a chunk in the app) are sometimes jumbled out of order which is annoying. Not sure what the issue is there.

Also as it is OCR it isn’t perfect and sometimes makes some annoying mistakes that need to be corrected.

But it does seem to be a useful tool, though I haven’t used it much just yet. I’ll take a look at Readwise as well, but you may find Highlighted useful as well.

Thanks for the explanation!

I am an android user :sob:

1 Like

Incidentally if anyone is curious, I did an interview / live notetaking session with Nick Milo over on the Linking Your Thinking channel where I showcase this process “with the garage door open,” as it were: How to turn your notes into published articles and books using the Obsidian app with Eleanor Konik - YouTube

12 Likes

Yes I watched that video Sunday evening. And (speaking as someone who HATES the tendency toward hour-long youtube videos, and has a general rule against ever clicking anything over 20 minutes) it was very good. I learned several quite useful things about Obsidian, and found myself wanting to participate in your conversation. I’ll bet you were a heck of a teacher - you have the natural talent.

7 Likes

Some people have asked to see my full folder layout, so here you go, here’s my folder system as of 2021-03-31. Note that my “docs-vault” sits directly in my root for my NAS, and contains functionally all of my files (the only exceptions are my collection of a jillion vacation and baby photos, and the unorganized stuff in my writing folder that I haven’t had a chance to sort through).

Hope this helps someone see one possible implementation of Johnny Decimal:

  • 00 Meta
    • 01 Attachments (sometimes attachments wind up in other places, this isn’t a hard-and-fast “attachments only go here” thing)
    • 02 Pending (I try to use this folder sparingly, mostly for my Tasking file and scratchpads that link to other apps that depend on a ‘default file’ opening, aka Writemonkey. I guess it’s like an Inbox?)
    • 03 Structure (these are my meta-notes about how my vault is structured, and where I put my “example notes” for things like the Palatinate snippets)
    • 04 Templates
    • 05 Tools (I keep “index notes” about external tools for different workflows here, for example my “research tools” has Zotero and Unpaywall and such on it. I also have one for useful Obsidian plugins that get asked about a lot)
  • 10 Dated
    • 11 Daily Roundups (this is where my daily notes — see link for template — go)
    • 12 Feedback (this is where my edit letters, fiction critique obligations, etc go).
    • 13 Monthly Roundups (this is where the raw data dumps for monthly spaced repetition / reflection stuff go)
  • 20 Personal
    • 21 Parenting
    • 22 School (mostly papers I wrote in college)
    • 23 Teaching (I know JD is all about the “no more than 2 levels deep” but let’s be real, that’s a guideline, not a rule, and I could have a whole JD-style vault for JUST teaching materials, so be gentle)
      • global-studies (this is the class I’ve taught most often. I used to have subfolders for every year, but I wound up re-using stuff often enough that I just use the numerical numbering system my curriculum gives to pre-pend filenames and it works fine)
      • hiring-materials (resumes, cover letters, particularly showcase-y things, etc)
      • psych
      • us-gov
      • us-history
    • 24 Taxes
      • 2017
      • 2018
      • 2019
      • 2020
    • 25 Legal (wills, copies of my ID cards, contracts, etc)
    • 26 Medical
    • 27 House
    • 28 Events (Mostly stuff like this Thanksgiving reflection so I can keep track of how much food was useful to have for next year, lol)
    • 29 Activism (records of letters I’ve written to congressmen, reference notes for charities I’m involved with, etc)
  • 30 Interests
    • 31 Programming
    • 32 Games
    • 33 Gardening
    • 34 Writing
  • 40 Slipbox (as in Zettelkasten)
    • 41 Indexes
    • 42 Encyclopaedic (this is stuff that isn’t “properly atomic” because it’s more “overview” style but still heavily paraphrased and mashed up from various sources)
    • 43 References (yeah yeah yeah it’s three layers deep, but you can surely see why)
      • 43.01 Books (literature notes, long but processed, with lots of direct quotations and annotations)
      • 43.02 Journals (literature notes but on journal articles)
      • 43.03 Discussions (honestly these are mostly raw dumps of conversations from Discord that I want to refer back to later)
      • 43.04 Audiovisual (notes on youtube videos and podcasts)
      • 43.05 PDFs (this is where Zotfile puts all of my academic PDFs it downloads from the internet — see this guide for details)
      • 43.06 Zotdumps (this is where Zotfile puts all of my raw annotations, as described in the guide linked above)
    • 44 Insights
    • 45 Questions
    • 46 Synthesis
  • 50 Worldbuilding
    • 51 Verraine (notes for my epic fantasy universe — could easily be a wiki by itself)
    • 52 GeneE (my science fiction universe)
    • 53 Neith (my urban fantasy universe)
  • 60 Characters
    • 61 Character Ideas (this is where I keep useful references like “a list of verbal tics I brainstormed” and "different ways to describe a person’s nose)
    • 62 Character References
    • b. ATA 100-199 (ATA indicates that these are characters from Verraine who were born during that timeframe. Since I write a lot of flash fiction and characters might appear in multiple stories, it’s easier to keep track of this way).
    • b. ATA 200-299 (“ATA” stands for “After the Archivist” aka after the invention of written records)
    • b. CE 1900-2099 (I can tell at a glance that this is my urban fantasy universe)
    • b. AGA 2250 (AG stands for “After Globalization Achieved”)
  • 70 Newsletters (I write my weekly research roundup newsletter in markdown in Obsidian, they’re organized by date. I separate them from my daily&weekly notes by using a 2021.03.21 format instead of dashes or W03 for example, so my navigation searches stay clean)
  • 80 Stories
    • 81 Universes (these are the “master” files for Verraine, which has an overview summary and relies on dataview to pull all stories set in this universe)
    • 82 Indexes (these are basically the “table of contents” pages for individual novels)
    • 83 Pitch materials (queries, blurbs, elevator pitches, etc)
    • 84 Prose
    • 85 Snippets (deleted scenes, scribbled ideas, snatches of a scene, etc)
    • 86 Publication (information about different markets, resources for self-pub, etc)
    • 87 Marketing
  • 90 Articles
    • 91 Images
    • 92 Seeds
    • 93 Recurring (notes for recurring features like my reading roundups or Thesis Thursday)
    • 94 Published
    • 95 Nonfic Markets (mostly indexes that leverage dataview and formatting guides. My “seeds” folder has like eleven billion things in it so I really rely on dataview and these index pages to see what I’ve got going on)
32 Likes

This is off-topic but I tried to look it up unsuccessful. Why is a two level limit a guideline in JD (law I’m assuming?)?

Heh, as a former lawyer I should have known better than to give that acronym there. “JD” in a personal knowledge management context usually refers to the Johnny Decimal system. It was really designed for multi-user corporate file structures, but for my purposes it’s a flexible guideline and philosophy that lets me have folders without having too many folders, which I like.

2 Likes

Ah, I see. I can understand that.

Thanks for the clarity and your contributions to the community!

1 Like

Thanks for sharing your process! I’m currently trying to re-design my slipbox as a workflow for producing publications instead of just a personal wikipedia of my ideas. I’ve read and re-read this thread and watched your video w/ Nick Milo and found it super helpful.

I have a few follow-up questions about your workflow, if you don’t mind…

Ok, let me try to play this back.

  1. You have a pre-existing interest in Egypt so there’s an Egypt note in your 41 Indexes.
  2. You read a book and export your highlights and annotations to Women in Egypt by Whoever in 43.01 Books and mark it up with headers and more annotations.
  3. While processing that lit note you make a mental connection (matching) to something you read about Sumeria and create a new note Egyptian Princesses had less work but more prestige than their Sumerian counterparts in 46 Synthesis that links to the relevant lit note header(s) about Egypt and Sumeria. The title states the insight.
  4. As you learn more about Egypt and Sumeria, this synthesis note might get recruited into a growing index note Comparing Egyptian and Sumerian royalty which links to all relevant synthesis notes (and maybe lit note sections directly as well?).
  5. This interest might spark an article idea Egypt vs. Sumeria: The Role of Princesses in the Fertile Crescent in 92 Seeds into which you pull the relevant sections from your index.

Do I have that right?

Your descriptions of creating and adding to Indexes are generally bottom-up, i.e. driven by interest and emergence. Do you have any top-down ‘rules’? Like some folks require all Zk notes, e.g. contents of your 46 Synthesis, to be linked from an index somewhere and will create one if it doesn’t exist.

What gets added to your broad Egypt index?

Why do you add numerical location identifiers to the header here?

What’s the difference between 44 Insights and 46 Synthesis?

You’ve got the right of it, although the odds of me every making an actual index note for “Comparing Egyptian and Sumerian royalty” are pretty low — it would probably stay a synthesis note unless I somehow do a lot with Egyptian and Sumerian royalty.

But I suppose if it came to that, then yea I would make an index note for it. I might broaden it out to just an index note for Royalty, depending on what else is in my vault. Generally speaking my index notes currently are very single-word in terms of naming structures, it helps me identify them faster.

As far as bottom-up vs top-down, the idea of making extra work for myself just to make sure that everything is linked to an index note feels very artificial to me; I imagine there are some use cases where that makes sense, but for what I do it would be a waste of time, which I am pathologically allergic to.

I use my index notes mostly as a way to keep track of concepts I do a lot with, but I do have a template I use for them.

Somewhat amusingly, because I do have a lot of notes related to Egypt, I haven’t actually bothered to make an Index page for Egypt because I don’t generally sit down and do much noddling about Egypt per se.

Here’s a screenshot that might help demo a little better.

The numerical prepending is because I use the Johnny Decimal system to help ensure folder order. I actually recently aligned it to my physical filing cabinet, which has been surprisingly helpful.

As for insights vs. synthesis, for me, insights are usually a little more … disconnected from a source than synthesis notes. Synthesis notes are usually where I summarize multiple pieces of information on a particular atomic topic, whereas insights are things I realized independently of the text.

It will probably seem like a somewhat artificial distinction just looking at the examples:

but for me it’s the difference between “ah yes, I have noticed that I now have several small notes in my literature notes about a very narrow topic, I should combine them into one top-level paragraph that pulls the information together” (sort of a very mini narrowly focused index I guess?) and “ah! I have realized a thing that was not told to me.” I generally use the insights as fodder for articles like this: 3 Theses, 3rd Thursday: Managing Things » Eleanor Konik

But there’s a fair amount of overlap and depending on what kind of person you are and what kind of writing you’re doing, you could probably just combine them into a folder called “claims” or something.

Hope this helps!

11 Likes