Increasingly Atomic Folders: A Workflow

The answer to your question 1 is essentially laziness and the inability of the refactor plugin to “update links to headings.”

I understand the value of having each quote + annotation broken out into individual notes, but then I’d have to actually name all of them (I did in the example but I don’t always, it’s extra work that is often unnecessary), and contrary to most principals of “what should go into a pkm vault” I’ve read, I actually think it’s an enormous waste of my time to re-word everything I read before letting it have a home in my notes. I understand the value of rewording things in order to ensure one’s own comprehension — I am a teacher, after all — and I understand the dangers of “raw capture” instead of “recording your own knowledge” but most of my litnote+annotation workflow is with sources I have only temporary access to, are comprised of information I’m not so much “trying to understand” as “trying to make sure I don’t forget the details of,” so in that case, having the actual quotation (plus my annotation about why it’s important and what it’s for and what it means) is the important part. From a workflow perspective, that means that most of my litnote is exported from something like Readwise or an annotated pdf’s highlights, and going through and refactoring it it often more trouble than it’s worth (but having it searchable in my “database of knowledge” is invaluable).

If I were a student using my vault to record learning I would absolutely have a different approach, but since I’m not, forcing everything to be artificially atomic makes it harder to do things like write a book review or review the “flow” of the important parts of a particular book. And usually I remember where I read something (“Didn’t The Horse, The Wheel, and Language say something about cattle, too?” might wind up relating to a section that would not reasonably have had a title about cattle, horses, wheels, or language :stuck_out_tongue: and while I could use a source note, that adds an unnecessary step that feels like added friction) but not what the rest of the surrounding context of the data point was.

the tl;dr on that is that I totally agree with you about the dangers of the whole “tight coupling” thing but in the end I decided not to let the perfect be the enemy of the good and that saving time by minimizing extra steps was more important than having the optimally atomic layout, given my particular goals.

The answer to your question 2 is also efficiency. Why do the same work twice? Once I’ve had a particular thought, I don’t need a note that says the same thing. I’m not going to transclude my evergreen atomic note into another source, I’m going to say “This relates to [[article]] where I said thing.” Being able to do "This relates to [[article]] where I said [[thing]]" doesn’t really gain me much, and means that if I search for [[thing]] I’m going to get two results, which makes my results harder to trawl for useful information, not easier.

I almost always never write evergreen notes for the sake of writing notes. I don’t take notes “just to have them.” I take notes because I plan to use them, and thus far I haven’t hit a point where I’ve been like "gee, I should develop my thinking on topic" that hasn’t resulted in something I want to share with the world. If I notice a connection like “Suttee, Sacral Kingship, and Human Sacrifice are related in ways that haven’t been explored by any academics I’ve seen,” I’m not going to make an evergreen note about suttee, sacral kingship, and human sacrifice and then go write an article about it. I’m just going to write an article about it. Maybe I’ll create MOCs about suttee, sacral kingship, and/or human sacrifice that point to various notes I’ve taken on the subject, but the way my brain works, creating an evergreen note for each and then synthesizing them later is just unnecessary work, and I’m pathologically opposed to unnecessary work.

Basically, I only do as much work as necessary to facilitate my goals — and I think our goals are different. But maybe I’ll shift over to a more atomic-evergreen-note centric focus as I grow my vault. I’ve made other shifts and adaptations over the months!

5 Likes

I try to make an overview table when there are more than 3-5 similar concepts; that helps me to keep them consistent and also helps me find the exact thing to look for.
I remove the specific links in the leaf nodes and just link them back to the index; so that if a new concept is added it is immediately available (with 1 hop) from all the previous leaves and it all stays DRY.
It’s difficult to decide what fields to add in the overview table and keeping DRY. Sometimes I keep a file like this taxonomy index to keep things consistent and manageable. That allows for things to show up in an index, in a hierarchy, and in the files themselves in a consistent manner with little extra work.

Thanks for the thorough response! Your motivations make sense and if I were writing articles like that I would probably adopt the same approach for the same reasons.

I’d also posit that, given the way you write your articles, your articles in a sense can be considered atomic notes themselves – atomic meaning “focused on a single concept” not “short.”

Agreed on the quotes vs own words. I’m finding similar forces in my own notes – some things warrant rewriting because they are significant, others are more about being able to jog my memory by seeing the snippet of context around the quote. (I’m also finding that I need a good workflow for extracting annotations from digital documents, and extracting highlights & notes seems the reasonable approach.)

Interestingly though this is where I find the separate literature notes with phrase-based titles works well – the title alone may completely encapsulate the concept and the contents may be nothing more than a paragraph or three of quoted content from which the title was derived, with perhaps a sentence or two annotation exactly as you describe. (which is arguably what is meant by writing it in our own words…)

But again that’s a matter of style and preference I suppose, since it seems you and I are doing essentially the exact same thing except you make the titles headings instead of separate notes. So its fascinating to see us doing essentially the same thing in two different ways with the same fundamental needs and motivations. Tomato / tomahto. :slight_smile:

1 Like

Honestly I agree — it’s just that it’s so much easier to import the .md file from Readwise and then tweak it so that it has the headings and all as one big file doing it all at once… and then by the time I’m done, I can’t refactor it because then I’d have to update all of the times I’d liked to the headings :frowning:

If note refactor plugin updated the backlinks to the headings so that they pointed to the new note I would 100% do it the same way as you do for exactly the reasons you’ve stated.

(and yeah being honest a lot of the articles and newsletters do wind up functionally being atomic notes, again, just with very particular formatting (and none of the interlinks that would be valuable of it were intended for internal use — instead the interlinks point to my other “atomic notes” that are already published))

If note refactor plugin updated the backlinks to the headings so that they pointed to the new note I would 100% do it the same way as you do for exactly the reasons you’ve stated.

Hmm, this is interesting, I’m having trouble picturing what you mean here. Can you give an example?

Imagine I’ve been reading a 400 page academic text on, say, Iron Age Syracuase. As I read, I take notes in my annotation app, not Obsidian, because it’s a physical book and I prefer to use Readwise’s OCR instead of literally typing up all of my quotations. But because it’s a long book, it takes me several days, and during that time, I use my resonance calendar to record big-picture impressions that link directly to the source note “Tyrants of Syracuse by Jeff Champion.”

So far, no big deal if I atomize the actual highlights and annotations and stuff later, because I can use the source note as an index and the connections will be maintained.

But then I go ahead and imported all of theq quotes+annotations (from a pdf or kindle or Readwise or whatever; in this case it’s Readwise). I’m left with a document that looks something like this:

quote

Note: Annotation

quote

quote

Note: Annotation

quote

quote

it’s annoying because it’s not really formatted how I like it (and Readwise doesn’t give me the darn page numbers on export, or even separate out which quotes are part of the same section and which paragraphs are on their own, unless there’s an annotation to split them up), which is much more:

ch03p93 descriptive heading

quote

annotation

ch04p103 descriptive heading

quote

annotation

but the process of taking the raw data dump from my pdf or Readwise or whatever and formatting it correctly is significantly easier for me if I just go straight through and fix it without making a bunch of files. I know that there are probably plugins like Workbench and probably Refactor that can split off each section as I go using some sort of hotkey magic, but I’m a teacher, not a programmer, and hotkeys aren’t totally intuitive to me yet. So I go down the list and do it all at once.

But as I go, I realize that my sixth quote and my fifteenth quote address the same topic, so I do create a new note called “unusual uses for greek temples” and embed quote 6 (“ch3p23 greek temples were used as banks”) and quote 15 (“ch4p30 greek temples were used as a base for raiding”) into it.

I do that as I go, synthesizing thoughts and connecting things and then I get to the end and…

… if I use the Note Refactor plugin to turn all of those subheadings into individual notes with descriptive names, none of my synthesis notes will point to the right place anymore and my embeds will be broken.

Now, yeah, I could do it “right” the first time and pull each individual quote and annotation out into their own file, but that’s a lot of extra point-and-click that is the whole reason the Note Refactor plugin exists — because that kind of thing is a bigger timesink than just doing it all in one file. Plus, I kind of like having everything right there in one place for me to look at it as a block while I’m still working with it.

I could also wait until everything is formatted and then refactor it into atomized notes and then create the synthesis notes… but what if I forget one of my ideas? And even if I don’t, now I’m doing the same cognitive work twice — because it’s not like I can “turn off” the part of my brain that notices those connections originally.

So one way I do twice as much work but wind up with more “visible” atomic notes in my file bar, and the other I do half as much work but have to utilize search more heavily to find my notes again.

I decided I’d prefer to rely on search… but in a perfect world, when Note Refactor splits my long litnote, the links to what used to be headers and are now their own notes would get updated. But my understanding is that from a programming perspective that’s a pretty heavy lift.

1 Like

Aha ok yeah I see exactly what you mean now about the note refactor plugin. As a former dev myself I would think the solution from a programmatic standpoint would be for the plugin to (1) check if the selected content contains a ^block-id and if so (2) find all instances of that block ID in the vault and update the links that contain them. (1) is pretty trivial, but changing the links in part (2) could possibly be more tricky.

For highlighting, I’ve experimented with an app that may or may not help you, called Highlighted. It lets me select passages in physical books and OCRs them, lets me tag them, and lets me group them together and attach the page number as well, then export to text or Markdown or PDF. The end result is a large block of text similar to what you describe but I think it is broken down into sections a bit more, because I can essentially pre-chunk a bit in the app. It also lets me type in notes along with the highlights and keeps them in the output.

Example from Ahrens book:

p. 65

It is not only impossible to a focus on more than one thing at time, but also to have a different kind of attention on more than one thing at a time.

When it comes to focused attention, we focus on one only, something we can sustain for only a few seconds.

Focused attention is different from "sustained attention, which we need to stay focused on one task for a longer period and is necessary to learn, understand or get something done.

#term

I make no claim the page numbers are correct – it tries to automatically assign page numbers and seems to use the kindle or a pdf version which is usually off a bit, and this is where its my turn to be lazy and not bother updating it! :smiley:

The ... (which I manually add as free-text notes in the highlight group) signifies that those are multiple highlights I captured from different places in the book (usually quite close together) and assigned to a single “note” in the app, all attached to the same page (again, maybe the wrong page ¯\_( ツ )_/¯).

The Focused attention is different ... at the end is my own free text note added in the Highlighted app, included in the exported text. #term is a tag I created to denote defined terms in the extract. The app supports tagging as a first class construct.

The ability to add my own notes directly in the same group allows me to also add notes like these, which are attached to another single group of highlights:

N) New ideas must overcone existing rotines to tale hold

N) Silo busting optimizes flow

N) innovation created hidden second order benefits

These were conceptual evergreen note or lit note titles I thought of while extracting the highlighted content so I just typed them directly in the app into the highlighted note as free text. Just to jog my memory when I process it after extraction. They may or may not make the cut when processing.

One weird bug seems to occur though where sometimes the passages in a section with multiple highlights (the ... stuff, which I manually add between captured highlights in a chunk in the app) are sometimes jumbled out of order which is annoying. Not sure what the issue is there.

Also as it is OCR it isn’t perfect and sometimes makes some annoying mistakes that need to be corrected.

But it does seem to be a useful tool, though I haven’t used it much just yet. I’ll take a look at Readwise as well, but you may find Highlighted useful as well.

Thanks for the explanation!

I am an android user :sob:

Incidentally if anyone is curious, I did an interview / live notetaking session with Nick Milo over on the Linking Your Thinking channel where I showcase this process “with the garage door open,” as it were: How to turn your notes into published articles and books using the Obsidian app with Eleanor Konik - YouTube

9 Likes

Yes I watched that video Sunday evening. And (speaking as someone who HATES the tendency toward hour-long youtube videos, and has a general rule against ever clicking anything over 20 minutes) it was very good. I learned several quite useful things about Obsidian, and found myself wanting to participate in your conversation. I’ll bet you were a heck of a teacher - you have the natural talent.

5 Likes

Some people have asked to see my full folder layout, so here you go, here’s my folder system as of 2021-03-31. Note that my “docs-vault” sits directly in my root for my NAS, and contains functionally all of my files (the only exceptions are my collection of a jillion vacation and baby photos, and the unorganized stuff in my writing folder that I haven’t had a chance to sort through).

Hope this helps someone see one possible implementation of Johnny Decimal:

  • 00 Meta
    • 01 Attachments (sometimes attachments wind up in other places, this isn’t a hard-and-fast “attachments only go here” thing)
    • 02 Pending (I try to use this folder sparingly, mostly for my Tasking file and scratchpads that link to other apps that depend on a ‘default file’ opening, aka Writemonkey. I guess it’s like an Inbox?)
    • 03 Structure (these are my meta-notes about how my vault is structured, and where I put my “example notes” for things like the Palatinate snippets)
    • 04 Templates
    • 05 Tools (I keep “index notes” about external tools for different workflows here, for example my “research tools” has Zotero and Unpaywall and such on it. I also have one for useful Obsidian plugins that get asked about a lot)
  • 10 Dated
    • 11 Daily Roundups (this is where my daily notes — see link for template — go)
    • 12 Feedback (this is where my edit letters, fiction critique obligations, etc go).
    • 13 Monthly Roundups (this is where the raw data dumps for monthly spaced repetition / reflection stuff go)
  • 20 Personal
    • 21 Parenting
    • 22 School (mostly papers I wrote in college)
    • 23 Teaching (I know JD is all about the “no more than 2 levels deep” but let’s be real, that’s a guideline, not a rule, and I could have a whole JD-style vault for JUST teaching materials, so be gentle)
      • global-studies (this is the class I’ve taught most often. I used to have subfolders for every year, but I wound up re-using stuff often enough that I just use the numerical numbering system my curriculum gives to pre-pend filenames and it works fine)
      • hiring-materials (resumes, cover letters, particularly showcase-y things, etc)
      • psych
      • us-gov
      • us-history
    • 24 Taxes
      • 2017
      • 2018
      • 2019
      • 2020
    • 25 Legal (wills, copies of my ID cards, contracts, etc)
    • 26 Medical
    • 27 House
    • 28 Events (Mostly stuff like this Thanksgiving reflection so I can keep track of how much food was useful to have for next year, lol)
    • 29 Activism (records of letters I’ve written to congressmen, reference notes for charities I’m involved with, etc)
  • 30 Interests
    • 31 Programming
    • 32 Games
    • 33 Gardening
    • 34 Writing
  • 40 Slipbox (as in Zettelkasten)
    • 41 Indexes
    • 42 Encyclopaedic (this is stuff that isn’t “properly atomic” because it’s more “overview” style but still heavily paraphrased and mashed up from various sources)
    • 43 References (yeah yeah yeah it’s three layers deep, but you can surely see why)
      • 43.01 Books (literature notes, long but processed, with lots of direct quotations and annotations)
      • 43.02 Journals (literature notes but on journal articles)
      • 43.03 Discussions (honestly these are mostly raw dumps of conversations from Discord that I want to refer back to later)
      • 43.04 Audiovisual (notes on youtube videos and podcasts)
      • 43.05 PDFs (this is where Zotfile puts all of my academic PDFs it downloads from the internet — see this guide for details)
      • 43.06 Zotdumps (this is where Zotfile puts all of my raw annotations, as described in the guide linked above)
    • 44 Insights
    • 45 Questions
    • 46 Synthesis
  • 50 Worldbuilding
    • 51 Verraine (notes for my epic fantasy universe — could easily be a wiki by itself)
    • 52 GeneE (my science fiction universe)
    • 53 Neith (my urban fantasy universe)
  • 60 Characters
    • 61 Character Ideas (this is where I keep useful references like “a list of verbal tics I brainstormed” and "different ways to describe a person’s nose)
    • 62 Character References
    • b. ATA 100-199 (ATA indicates that these are characters from Verraine who were born during that timeframe. Since I write a lot of flash fiction and characters might appear in multiple stories, it’s easier to keep track of this way).
    • b. ATA 200-299 (“ATA” stands for “After the Archivist” aka after the invention of written records)
    • b. CE 1900-2099 (I can tell at a glance that this is my urban fantasy universe)
    • b. AGA 2250 (AG stands for “After Globalization Achieved”)
  • 70 Newsletters (I write my weekly research roundup newsletter in markdown in Obsidian, they’re organized by date. I separate them from my daily&weekly notes by using a 2021.03.21 format instead of dashes or W03 for example, so my navigation searches stay clean)
  • 80 Stories
    • 81 Universes (these are the “master” files for Verraine, which has an overview summary and relies on dataview to pull all stories set in this universe)
    • 82 Indexes (these are basically the “table of contents” pages for individual novels)
    • 83 Pitch materials (queries, blurbs, elevator pitches, etc)
    • 84 Prose
    • 85 Snippets (deleted scenes, scribbled ideas, snatches of a scene, etc)
    • 86 Publication (information about different markets, resources for self-pub, etc)
    • 87 Marketing
  • 90 Articles
    • 91 Images
    • 92 Seeds
    • 93 Recurring (notes for recurring features like my reading roundups or Thesis Thursday)
    • 94 Published
    • 95 Nonfic Markets (mostly indexes that leverage dataview and formatting guides. My “seeds” folder has like eleven billion things in it so I really rely on dataview and these index pages to see what I’ve got going on)
28 Likes

This is off-topic but I tried to look it up unsuccessful. Why is a two level limit a guideline in JD (law I’m assuming?)?

Heh, as a former lawyer I should have known better than to give that acronym there. “JD” in a personal knowledge management context usually refers to the Johnny Decimal system. It was really designed for multi-user corporate file structures, but for my purposes it’s a flexible guideline and philosophy that lets me have folders without having too many folders, which I like.

2 Likes

Ah, I see. I can understand that.

Thanks for the clarity and your contributions to the community!

1 Like

Thanks for sharing your process! I’m currently trying to re-design my slipbox as a workflow for producing publications instead of just a personal wikipedia of my ideas. I’ve read and re-read this thread and watched your video w/ Nick Milo and found it super helpful.

I have a few follow-up questions about your workflow, if you don’t mind…

Ok, let me try to play this back.

  1. You have a pre-existing interest in Egypt so there’s an Egypt note in your 41 Indexes.
  2. You read a book and export your highlights and annotations to Women in Egypt by Whoever in 43.01 Books and mark it up with headers and more annotations.
  3. While processing that lit note you make a mental connection (matching) to something you read about Sumeria and create a new note Egyptian Princesses had less work but more prestige than their Sumerian counterparts in 46 Synthesis that links to the relevant lit note header(s) about Egypt and Sumeria. The title states the insight.
  4. As you learn more about Egypt and Sumeria, this synthesis note might get recruited into a growing index note Comparing Egyptian and Sumerian royalty which links to all relevant synthesis notes (and maybe lit note sections directly as well?).
  5. This interest might spark an article idea Egypt vs. Sumeria: The Role of Princesses in the Fertile Crescent in 92 Seeds into which you pull the relevant sections from your index.

Do I have that right?

Your descriptions of creating and adding to Indexes are generally bottom-up, i.e. driven by interest and emergence. Do you have any top-down ‘rules’? Like some folks require all Zk notes, e.g. contents of your 46 Synthesis, to be linked from an index somewhere and will create one if it doesn’t exist.

What gets added to your broad Egypt index?

Why do you add numerical location identifiers to the header here?

What’s the difference between 44 Insights and 46 Synthesis?

You’ve got the right of it, although the odds of me every making an actual index note for “Comparing Egyptian and Sumerian royalty” are pretty low — it would probably stay a synthesis note unless I somehow do a lot with Egyptian and Sumerian royalty.

But I suppose if it came to that, then yea I would make an index note for it. I might broaden it out to just an index note for Royalty, depending on what else is in my vault. Generally speaking my index notes currently are very single-word in terms of naming structures, it helps me identify them faster.

As far as bottom-up vs top-down, the idea of making extra work for myself just to make sure that everything is linked to an index note feels very artificial to me; I imagine there are some use cases where that makes sense, but for what I do it would be a waste of time, which I am pathologically allergic to.

I use my index notes mostly as a way to keep track of concepts I do a lot with, but I do have a template I use for them.

Somewhat amusingly, because I do have a lot of notes related to Egypt, I haven’t actually bothered to make an Index page for Egypt because I don’t generally sit down and do much noddling about Egypt per se.

Here’s a screenshot that might help demo a little better.

The numerical prepending is because I use the Johnny Decimal system to help ensure folder order. I actually recently aligned it to my physical filing cabinet, which has been surprisingly helpful.

As for insights vs. synthesis, for me, insights are usually a little more … disconnected from a source than synthesis notes. Synthesis notes are usually where I summarize multiple pieces of information on a particular atomic topic, whereas insights are things I realized independently of the text.

It will probably seem like a somewhat artificial distinction just looking at the examples:

but for me it’s the difference between “ah yes, I have noticed that I now have several small notes in my literature notes about a very narrow topic, I should combine them into one top-level paragraph that pulls the information together” (sort of a very mini narrowly focused index I guess?) and “ah! I have realized a thing that was not told to me.” I generally use the insights as fodder for articles like this: 3 Theses, 3rd Thursday: Managing Things » Eleanor Konik

But there’s a fair amount of overlap and depending on what kind of person you are and what kind of writing you’re doing, you could probably just combine them into a folder called “claims” or something.

Hope this helps!

9 Likes

Incidentally, I’m sharing this here because of a moderately amusing conversation I had in the discord with someone, which went roughly along the lines of that person saying they usually start broad with folders and then narrow as needed, and they were sure that someone else out there did it differently, and of course I do –

I start narrow and merge as edge-case grey area situations come up.

40 slipbox is now down to 3 folders:

41 questions
42 zettels
43 references

(I still have subfolders in references, don’t worry :wink: )

2 Likes

@EleanorKonik I’m thinking of implementing a similar structure to my notes, but folders can be a bit rigid sometimes.

While I can add multiple links and tags to a note, I can’t do the same with folders. A book about a topic would go under the “Books” folder (for example), not under this “Topic” folder. With tags I could just add multiple of them to the same note.

Did you find a way to overcome this limitation with folders? Or it was never a problem to you?

This is never a problem for me; I explicitly don’t have “topic” folders; even with tags trying to figure out whether, say, physics counts as “math” or “science” gets to be a nightmare, fast. Is “The Roman Empire” in the “Africa” tag or the “Europe” tag or the “Asia” tag? Oh, all of them, okay then at that point what’s the purpose of an Africa tag?

That way lies madness.

The primary advantage of the Johnny Decimal system for me is that there’s only ever one bucket a given file can go into. The idea is to design your folders so that that’s easy. I don’t ever have something that is simultaneously a note about the video game path of exile and would also fit in my folder of character bios for my novels.

Any time where I’m like “gee, I wonder what I’ve written about cows?” I don’t go searching through my “domestic animals” tag (or should it be animals? or should it be farm animals? or should it be creatures? or should it be things native to the middle east? or or or…) I just use search, and then create a list of all the applicable files, i.e. a LYT-style MOC.

The only time I ever use tags is as todos, i.e. “#atomize this” or “#crossreference with another note” or “#research this claim”

5 Likes

Thank you very much, great explanation! I may be sticking to folders and the Johnny Decimal system after all.

1 Like