Increasingly Atomic Folders: A Workflow

Knowledge management
14

Imagine I’ve been reading a 400 page academic text on, say, Iron Age Syracuase. As I read, I take notes in my annotation app, not Obsidian, because it’s a physical book and I prefer to use Readwise’s OCR instead of literally typing up all of my quotations. But because it’s a long book, it takes me several days, and during that time, I use my resonance calendar to record big-picture impressions that link directly to the source note “Tyrants of Syracuse by Jeff Champion.”

So far, no big deal if I atomize the actual highlights and annotations and stuff later, because I can use the source note as an index and the connections will be maintained.

But then I go ahead and imported all of theq quotes+annotations (from a pdf or kindle or Readwise or whatever; in this case it’s Readwise). I’m left with a document that looks something like this:

quote

Note: Annotation

quote

quote

Note: Annotation

quote

quote

it’s annoying because it’s not really formatted how I like it (and Readwise doesn’t give me the darn page numbers on export, or even separate out which quotes are part of the same section and which paragraphs are on their own, unless there’s an annotation to split them up), which is much more:

ch03p93 descriptive heading

quote

annotation

ch04p103 descriptive heading

quote

annotation

but the process of taking the raw data dump from my pdf or Readwise or whatever and formatting it correctly is significantly easier for me if I just go straight through and fix it without making a bunch of files. I know that there are probably plugins like Workbench and probably Refactor that can split off each section as I go using some sort of hotkey magic, but I’m a teacher, not a programmer, and hotkeys aren’t totally intuitive to me yet. So I go down the list and do it all at once.

But as I go, I realize that my sixth quote and my fifteenth quote address the same topic, so I do create a new note called “unusual uses for greek temples” and embed quote 6 (“ch3p23 greek temples were used as banks”) and quote 15 (“ch4p30 greek temples were used as a base for raiding”) into it.

I do that as I go, synthesizing thoughts and connecting things and then I get to the end and…

… if I use the Note Refactor plugin to turn all of those subheadings into individual notes with descriptive names, none of my synthesis notes will point to the right place anymore and my embeds will be broken.

Now, yeah, I could do it “right” the first time and pull each individual quote and annotation out into their own file, but that’s a lot of extra point-and-click that is the whole reason the Note Refactor plugin exists — because that kind of thing is a bigger timesink than just doing it all in one file. Plus, I kind of like having everything right there in one place for me to look at it as a block while I’m still working with it.

I could also wait until everything is formatted and then refactor it into atomized notes and then create the synthesis notes… but what if I forget one of my ideas? And even if I don’t, now I’m doing the same cognitive work twice — because it’s not like I can “turn off” the part of my brain that notices those connections originally.

So one way I do twice as much work but wind up with more “visible” atomic notes in my file bar, and the other I do half as much work but have to utilize search more heavily to find my notes again.

I decided I’d prefer to rely on search… but in a perfect world, when Note Refactor splits my long litnote, the links to what used to be headers and are now their own notes would get updated. But my understanding is that from a programming perspective that’s a pretty heavy lift.

2 Likes
15

Aha ok yeah I see exactly what you mean now about the note refactor plugin. As a former dev myself I would think the solution from a programmatic standpoint would be for the plugin to (1) check if the selected content contains a ^block-id and if so (2) find all instances of that block ID in the vault and update the links that contain them. (1) is pretty trivial, but changing the links in part (2) could possibly be more tricky.

For highlighting, I’ve experimented with an app that may or may not help you, called Highlighted. It lets me select passages in physical books and OCRs them, lets me tag them, and lets me group them together and attach the page number as well, then export to text or Markdown or PDF. The end result is a large block of text similar to what you describe but I think it is broken down into sections a bit more, because I can essentially pre-chunk a bit in the app. It also lets me type in notes along with the highlights and keeps them in the output.

Example from Ahrens book:

p. 65

It is not only impossible to a focus on more than one thing at time, but also to have a different kind of attention on more than one thing at a time.
…
When it comes to focused attention, we focus on one only, something we can sustain for only a few seconds.
…

Focused attention is different from "sustained attention, which we need to stay focused on one task for a longer period and is necessary to learn, understand or get something done.

#term

I make no claim the page numbers are correct – it tries to automatically assign page numbers and seems to use the kindle or a pdf version which is usually off a bit, and this is where its my turn to be lazy and not bother updating it! :smiley:

The ... (which I manually add as free-text notes in the highlight group) signifies that those are multiple highlights I captured from different places in the book (usually quite close together) and assigned to a single “note” in the app, all attached to the same page (again, maybe the wrong page ¯\_( ツ )_/¯).

The Focused attention is different ... at the end is my own free text note added in the Highlighted app, included in the exported text. #term is a tag I created to denote defined terms in the extract. The app supports tagging as a first class construct.

The ability to add my own notes directly in the same group allows me to also add notes like these, which are attached to another single group of highlights:

N) New ideas must overcone existing rotines to tale hold

N) Silo busting optimizes flow

N) innovation created hidden second order benefits

These were conceptual evergreen note or lit note titles I thought of while extracting the highlighted content so I just typed them directly in the app into the highlighted note as free text. Just to jog my memory when I process it after extraction. They may or may not make the cut when processing.

One weird bug seems to occur though where sometimes the passages in a section with multiple highlights (the ... stuff, which I manually add between captured highlights in a chunk in the app) are sometimes jumbled out of order which is annoying. Not sure what the issue is there.

Also as it is OCR it isn’t perfect and sometimes makes some annoying mistakes that need to be corrected.

But it does seem to be a useful tool, though I haven’t used it much just yet. I’ll take a look at Readwise as well, but you may find Highlighted useful as well.

Thanks for the explanation!

16

I am an android user :sob:

1 Like
17

Incidentally if anyone is curious, I did an interview / live notetaking session with Nick Milo over on the Linking Your Thinking channel where I showcase this process “with the garage door open,” as it were: How to turn your notes into published articles and books using the Obsidian app with Eleanor Konik - YouTube

12 Likes
18

Yes I watched that video Sunday evening. And (speaking as someone who HATES the tendency toward hour-long youtube videos, and has a general rule against ever clicking anything over 20 minutes) it was very good. I learned several quite useful things about Obsidian, and found myself wanting to participate in your conversation. I’ll bet you were a heck of a teacher - you have the natural talent.

7 Likes
19

Some people have asked to see my full folder layout, so here you go, here’s my folder system as of 2021-03-31. Note that my “docs-vault” sits directly in my root for my NAS, and contains functionally all of my files (the only exceptions are my collection of a jillion vacation and baby photos, and the unorganized stuff in my writing folder that I haven’t had a chance to sort through).

Hope this helps someone see one possible implementation of Johnny Decimal:

  • 00 Meta
    • 01 Attachments (sometimes attachments wind up in other places, this isn’t a hard-and-fast “attachments only go here” thing)
    • 02 Pending (I try to use this folder sparingly, mostly for my Tasking file and scratchpads that link to other apps that depend on a ‘default file’ opening, aka Writemonkey. I guess it’s like an Inbox?)
    • 03 Structure (these are my meta-notes about how my vault is structured, and where I put my “example notes” for things like the Palatinate snippets)
    • 04 Templates
    • 05 Tools (I keep “index notes” about external tools for different workflows here, for example my “research tools” has Zotero and Unpaywall and such on it. I also have one for useful Obsidian plugins that get asked about a lot)
  • 10 Dated
    • 11 Daily Roundups (this is where my daily notes — see link for template — go)
    • 12 Feedback (this is where my edit letters, fiction critique obligations, etc go).
    • 13 Monthly Roundups (this is where the raw data dumps for monthly spaced repetition / reflection stuff go)
  • 20 Personal
    • 21 Parenting
    • 22 School (mostly papers I wrote in college)
    • 23 Teaching (I know JD is all about the “no more than 2 levels deep” but let’s be real, that’s a guideline, not a rule, and I could have a whole JD-style vault for JUST teaching materials, so be gentle)
      • global-studies (this is the class I’ve taught most often. I used to have subfolders for every year, but I wound up re-using stuff often enough that I just use the numerical numbering system my curriculum gives to pre-pend filenames and it works fine)
      • hiring-materials (resumes, cover letters, particularly showcase-y things, etc)
      • psych
      • us-gov
      • us-history
    • 24 Taxes
      • 2017
      • 2018
      • 2019
      • 2020
    • 25 Legal (wills, copies of my ID cards, contracts, etc)
    • 26 Medical
    • 27 House
    • 28 Events (Mostly stuff like this Thanksgiving reflection so I can keep track of how much food was useful to have for next year, lol)
    • 29 Activism (records of letters I’ve written to congressmen, reference notes for charities I’m involved with, etc)
  • 30 Interests
    • 31 Programming
    • 32 Games
    • 33 Gardening
    • 34 Writing
  • 40 Slipbox (as in Zettelkasten)
    • 41 Indexes
    • 42 Encyclopaedic (this is stuff that isn’t “properly atomic” because it’s more “overview” style but still heavily paraphrased and mashed up from various sources)
    • 43 References (yeah yeah yeah it’s three layers deep, but you can surely see why)
      • 43.01 Books (literature notes, long but processed, with lots of direct quotations and annotations)
      • 43.02 Journals (literature notes but on journal articles)
      • 43.03 Discussions (honestly these are mostly raw dumps of conversations from Discord that I want to refer back to later)
      • 43.04 Audiovisual (notes on youtube videos and podcasts)
      • 43.05 PDFs (this is where Zotfile puts all of my academic PDFs it downloads from the internet — see this guide for details)
      • 43.06 Zotdumps (this is where Zotfile puts all of my raw annotations, as described in the guide linked above)
    • 44 Insights
    • 45 Questions
    • 46 Synthesis
  • 50 Worldbuilding
    • 51 Verraine (notes for my epic fantasy universe — could easily be a wiki by itself)
    • 52 GeneE (my science fiction universe)
    • 53 Neith (my urban fantasy universe)
  • 60 Characters
    • 61 Character Ideas (this is where I keep useful references like “a list of verbal tics I brainstormed” and "different ways to describe a person’s nose)
    • 62 Character References
    • b. ATA 100-199 (ATA indicates that these are characters from Verraine who were born during that timeframe. Since I write a lot of flash fiction and characters might appear in multiple stories, it’s easier to keep track of this way).
    • b. ATA 200-299 (“ATA” stands for “After the Archivist” aka after the invention of written records)
    • b. CE 1900-2099 (I can tell at a glance that this is my urban fantasy universe)
    • b. AGA 2250 (AG stands for “After Globalization Achieved”)
  • 70 Newsletters (I write my weekly research roundup newsletter in markdown in Obsidian, they’re organized by date. I separate them from my daily&weekly notes by using a 2021.03.21 format instead of dashes or W03 for example, so my navigation searches stay clean)
  • 80 Stories
    • 81 Universes (these are the “master” files for Verraine, which has an overview summary and relies on dataview to pull all stories set in this universe)
    • 82 Indexes (these are basically the “table of contents” pages for individual novels)
    • 83 Pitch materials (queries, blurbs, elevator pitches, etc)
    • 84 Prose
    • 85 Snippets (deleted scenes, scribbled ideas, snatches of a scene, etc)
    • 86 Publication (information about different markets, resources for self-pub, etc)
    • 87 Marketing
  • 90 Articles
    • 91 Images
    • 92 Seeds
    • 93 Recurring (notes for recurring features like my reading roundups or Thesis Thursday)
    • 94 Published
    • 95 Nonfic Markets (mostly indexes that leverage dataview and formatting guides. My “seeds” folder has like eleven billion things in it so I really rely on dataview and these index pages to see what I’ve got going on)
33 Likes
20

This is off-topic but I tried to look it up unsuccessful. Why is a two level limit a guideline in JD (law I’m assuming?)?

21

Heh, as a former lawyer I should have known better than to give that acronym there. “JD” in a personal knowledge management context usually refers to the Johnny Decimal system. It was really designed for multi-user corporate file structures, but for my purposes it’s a flexible guideline and philosophy that lets me have folders without having too many folders, which I like.

2 Likes
22

Ah, I see. I can understand that.

Thanks for the clarity and your contributions to the community!

1 Like
23

Thanks for sharing your process! I’m currently trying to re-design my slipbox as a workflow for producing publications instead of just a personal wikipedia of my ideas. I’ve read and re-read this thread and watched your video w/ Nick Milo and found it super helpful.

I have a few follow-up questions about your workflow, if you don’t mind…

Ok, let me try to play this back.

  1. You have a pre-existing interest in Egypt so there’s an Egypt note in your 41 Indexes.
  2. You read a book and export your highlights and annotations to Women in Egypt by Whoever in 43.01 Books and mark it up with headers and more annotations.
  3. While processing that lit note you make a mental connection (matching) to something you read about Sumeria and create a new note Egyptian Princesses had less work but more prestige than their Sumerian counterparts in 46 Synthesis that links to the relevant lit note header(s) about Egypt and Sumeria. The title states the insight.
  4. As you learn more about Egypt and Sumeria, this synthesis note might get recruited into a growing index note Comparing Egyptian and Sumerian royalty which links to all relevant synthesis notes (and maybe lit note sections directly as well?).
  5. This interest might spark an article idea Egypt vs. Sumeria: The Role of Princesses in the Fertile Crescent in 92 Seeds into which you pull the relevant sections from your index.

Do I have that right?

Your descriptions of creating and adding to Indexes are generally bottom-up, i.e. driven by interest and emergence. Do you have any top-down ‘rules’? Like some folks require all Zk notes, e.g. contents of your 46 Synthesis, to be linked from an index somewhere and will create one if it doesn’t exist.

What gets added to your broad Egypt index?

Why do you add numerical location identifiers to the header here?

What’s the difference between 44 Insights and 46 Synthesis?

24

You’ve got the right of it, although the odds of me every making an actual index note for “Comparing Egyptian and Sumerian royalty” are pretty low — it would probably stay a synthesis note unless I somehow do a lot with Egyptian and Sumerian royalty.

But I suppose if it came to that, then yea I would make an index note for it. I might broaden it out to just an index note for Royalty, depending on what else is in my vault. Generally speaking my index notes currently are very single-word in terms of naming structures, it helps me identify them faster.

As far as bottom-up vs top-down, the idea of making extra work for myself just to make sure that everything is linked to an index note feels very artificial to me; I imagine there are some use cases where that makes sense, but for what I do it would be a waste of time, which I am pathologically allergic to.

I use my index notes mostly as a way to keep track of concepts I do a lot with, but I do have a template I use for them.

Somewhat amusingly, because I do have a lot of notes related to Egypt, I haven’t actually bothered to make an Index page for Egypt because I don’t generally sit down and do much noddling about Egypt per se.

Here’s a screenshot that might help demo a little better.

image
image1639×1021 160 KB

The numerical prepending is because I use the Johnny Decimal system to help ensure folder order. I actually recently aligned it to my physical filing cabinet, which has been surprisingly helpful.

As for insights vs. synthesis, for me, insights are usually a little more … disconnected from a source than synthesis notes. Synthesis notes are usually where I summarize multiple pieces of information on a particular atomic topic, whereas insights are things I realized independently of the text.

It will probably seem like a somewhat artificial distinction just looking at the examples:

image
image522×895 59.2 KB

but for me it’s the difference between “ah yes, I have noticed that I now have several small notes in my literature notes about a very narrow topic, I should combine them into one top-level paragraph that pulls the information together” (sort of a very mini narrowly focused index I guess?) and “ah! I have realized a thing that was not told to me.” I generally use the insights as fodder for articles like this: 3 Theses, 3rd Thursday: Managing Things » Eleanor Konik

But there’s a fair amount of overlap and depending on what kind of person you are and what kind of writing you’re doing, you could probably just combine them into a folder called “claims” or something.

Hope this helps!

11 Likes
25

Incidentally, I’m sharing this here because of a moderately amusing conversation I had in the discord with someone, which went roughly along the lines of that person saying they usually start broad with folders and then narrow as needed, and they were sure that someone else out there did it differently, and of course I do –

I start narrow and merge as edge-case grey area situations come up.

40 slipbox is now down to 3 folders:

41 questions
42 zettels
43 references

(I still have subfolders in references, don’t worry :wink: )

4 Likes
26

@EleanorKonik I’m thinking of implementing a similar structure to my notes, but folders can be a bit rigid sometimes.

While I can add multiple links and tags to a note, I can’t do the same with folders. A book about a topic would go under the “Books” folder (for example), not under this “Topic” folder. With tags I could just add multiple of them to the same note.

Did you find a way to overcome this limitation with folders? Or it was never a problem to you?

27

This is never a problem for me; I explicitly don’t have “topic” folders; even with tags trying to figure out whether, say, physics counts as “math” or “science” gets to be a nightmare, fast. Is “The Roman Empire” in the “Africa” tag or the “Europe” tag or the “Asia” tag? Oh, all of them, okay then at that point what’s the purpose of an Africa tag?

That way lies madness.

The primary advantage of the Johnny Decimal system for me is that there’s only ever one bucket a given file can go into. The idea is to design your folders so that that’s easy. I don’t ever have something that is simultaneously a note about the video game path of exile and would also fit in my folder of character bios for my novels.

Any time where I’m like “gee, I wonder what I’ve written about cows?” I don’t go searching through my “domestic animals” tag (or should it be animals? or should it be farm animals? or should it be creatures? or should it be things native to the middle east? or or or…) I just use search, and then create a list of all the applicable files, i.e. a LYT-style MOC.

The only time I ever use tags is as todos, i.e. “#atomize this” or “#crossreference with another note” or “#research this claim”

7 Likes
28

Thank you very much, great explanation! I may be sticking to folders and the Johnny Decimal system after all.

1 Like
29

Thanks! I actually recently discovered the JD system from your blog post article and decided to try it myself. I am working through it and my biggest issue is how to deal with clients. I’m an attorney, and usually just sort cases by year and number (for example, 2020-01). However, I recently signed several clients that send me various matters every week, and that has me looking for a different structuring option. How do you recommend structuring clients? Thanks!

30

I haven’t done much with clients, but does this page on the Johnny Decimal system not help?

1 Like
32

Thanks for sharing. I actually did read it before posting, but it didn’t make a lot of sense to me, or at least, to my work. I reread it while staring at my “Finder”, and looking at the client files, and I think I found a way that I’ll share just in case someone is as stomped as me. :slight_smile:

I created a folder, 10-19 called “Clients”. Within that folder, the numbers 10 to 19 are divided by “areas” (or matters). For example, one of them is “Immigration”. The other one is “Contracts” and so on.

I tried looking at the 3 number structure that JD proposes. The one that made most sense to me was then start numbering my clients per matter. For example, I have “11” as contract clients. I wrote “1-11 Client Name.” Most of my clients are one-offs, so I then created a sub folder called “1-11-01 Citizenship application” (for example). This way, if the client comes back in the future, I put them as 1-11-02.

In fact, I started moving my active case files, and found this very helpful. For example, I have cases where I have filed applicaitons for siblings, or husband and wife. I put them as “1 case” initially with some subfolders. I’ve never been happy with that structure. I now switched those to, for example, “03-11 Jones Family”, and then I wrote “03-11-01 Spouse 1” and then “03-11-02 Spouse 2”.

I’m sure practice will make it better, but my biggest concern now will be remembering the names and structure.

33

I can’t edit the post, but I did make a subsequent change, to keep with the 3 number project system. I numbered clients, say, 101-11 Client Name, 102-11 Client Name and so forth. Then, the projects are 102-11-01, 102-11-02, etc.

I’ll keep tinkering. :slight_smile:

1 Like
34

That part comes with time, and yeah definitely iterating is the way to go — it’s easier to sense the solutions when it’s your own system.

I’m glad you figured out something and sorry I couldn’t be more help :green_heart:

1 Like