For those with huge vaults: anything you’d do differently or wish you’d known before starting?

Knowledge management
1

I always spend a lot of time figuring out how I want to organize my vault because I’m afraid it’ll be hard to change things once it gets too big.
Right now I rely on notes with indexes to find and organize my content. But pretty often, I feel like tags might be a better choice (in particular when I have notes that would fit in multiple categories) and I’m worried I’ll want to switch to them when it’s already too late and the vault is massive.

So, this question is mainly for people with huge vaults, is there anything you wish you had done, avoided, or known earlier before your vault got out of hand?

2 Likes
2

I have large vaults. I use very few tags and no indexes. I rely mostly on Quick Switcher and folders to keep my vaults organized.

3

From what I’ve seen, it’s just about inevitable that no matter how hard you try to prevent it, at some point you’ll end up wanting to change your system in a way that’s a hassle. What some of us do in that situation is to let the old system persist in old notes, and only change them when you need to use them (and maybe do a useful batch up front).

Probably the biggest, most annoying change I’m making is pulling info out of my daily (and other log) notes. There are a couple of layers to that.

Before Obsidian I used to write all my journal/log stuff plus some other things in a single series of notes, figuring things would be less likely to get lost or forgotten that way, and I could always search or filter it down. But I rarely want to find things by date, and filtering and searching are annoying. I started using a set of themed logs, but merged them together when I started using Obsidian because the built in Daily Notes didn’t support it. Now I’m back on themed logs, and slowly, painfully splitting up the unified ones.

Beyond that, some logs are more particular — the ones for projects and for media that I consume. I have overview logs for those, but the most detailed entries go in the note of the things itself (the project note or book note or whatever). When I want to see what I thought of a movie I want to just go to its note, not comb thru a big general log. So now I’m going thru and pulling the entries for each of those things out.

Sure wish I hadn’t done things the way I did! But sometimes experience is the only way to learn.

3 Likes
4

I find that it is difficult to organize things when creating ideas for creations that are still being brainstormed and developed. Many of the separate ideas have multiple possibly important connections to other ideas. But decisions haven’t been made yet that would assist in establishing the right connections or delineating the correct organization structure.

Putting off the process, on one hand, allows many of these uncertainties to stay unseen and thus less influential on newer ideas going forward. This puts off the actual piecing together aspect of the creative process until later. And in the meantime you are left inventing ideas at the direction of inspiration of recently reviewed content or life itself. This can be good to help move a project in a direction based on your gut, since you are not relying on connections, etc. But, on the other hand, if you don’t put off the process of connecting and organizing, you are somewhat dependent and locked into the assembly you have made so far. And thus each step forward feels exponentially distant from a starting point were you to decide to revert. That being said, I think there are possibly good version control and embed based systems to generate chains and clumps of parts in unison without destructively muddying or disassembly the building blocks, with the use of metadata schemes or well managed modified duplicates.

But both ends of the spectrum tend to result in seemingly flexible systems that, on a higher level, are less flexible if the direction changes. Since this is in relation to creative projects in my case, that is a given. So, I have ended up with a large vault of varying levels of organized and sorted ideas, waiting to be picked apart. I have had to adjust my purpose for using Obsidian from the tool to help put the ideas together as I have them to the tool that I record ideas into only occasionally piecing things together when it feels necessary. The only real use of old notes in this domain is when I feel motivated to comb through them to extract pieces and refresh my working memory of what is there. It really would be nice to find a way to appropriately and efficiently link each reference to other ideas (or parts of other ideas) after the fact. But even the most advanced ai couldn’t solve this. Because only we can read a line and definitely understand what we were referring to and its context, I have come to accept that the only way to put the pieces together is to painstakingly read and link. It is in this process that I wish I could better link between blocks. Like if there were a mode to switch the file explorer to the block explorer and to display things in an unfolding hierarchical manner.

I’ve gone off on a tangent, but in general, I wish I had known what I know now in terms of the futility of creating an organization system for a developing creative project, if there is an expectation it will last. For snapshots, it’s fine. But if, when I started with Obsidian, I had adjusted my hopes for the tool to simply be a place I capture things that can be later organized when I am ready, I would be much further along than I am now. I believe this to be so because by regularly developing systems and then not properly porting the pieces to the inevitable next system, I have ended up with an incredibly complex tangle that mixes the parts with the assemblies, making them indistinguishable. A knot of strings that have knots on them is much more difficult to untie. So when you get to the point where you are just ready to deconstruct everything into its pieces and starting over, you have to contend with all the previous reorganization knots. So, you have to separate those out as individual pieces and the individual pieces they are made up of. In creative projects these pieces are not rigidly defined yet, so you inevitably have many possible duplicates and become disoriented. But the question is, was it worth it? I say it was. But maybe it’s not worth trying to sort things out in what seems like the right way. Maybe the only feasible way is to not try to treat the strategy moving forward as if you must define the logic like a computer making all the right pieces break apart and come together properly. This may prevent you from limiting yourself by trying to solve unnecessary problems and wasting time placing stuff in boxes that go nowhere.

I doubt I was very clear in my explanation here. But it was helpful for me to try to put it into words, for my own purposes.

Thanks!

5

What are the sources of knots, overly complex templates and structures?

6

moving folders is hard. renaming tags, changing links or mass updating metadata is pretty easy.

Using tags is great for compatibility with apps like Zettlr, but too many tags cause pain, and not all apps support subtags like “area/work”. Using Metadata for this proved to be better. Even in less featured programs you can look for metadata with full text search like “type: Idea”. I spend less time sorting things into appropriate places and have no duplicate “ideas” folders for areas.

Using advanced URI Plugin + ID’s is really neat if you regularly reference notes from outside but tend to move them around or renaming them. Keeps the link intact, because it’s tied to a frontmatter ID.

1 Like
7

My vault only got of hand when it was small (< 1000 notes). I find it much easier to organize larger vaults.

Now, after 4+ years experience and a main vault with 10K+ notes and some smaller vaults in the 250 to 1500 notes range, I do some things differently than before.

  1. I use less plugins and I use simpler plugins. I find it much easier to organize large vaults without Templater and without Dataview.
  2. I use every organizing tool that plain Obsidian has to offer: links and tags and properties and folders. I combine them in many variations. I don’t listen to Youtubers who preach that you should not do something because it’s uncool or whatever.
  3. I embrace messiness. I don’t strive for “the perfect system” anymore. Needs change, tools change, everything changes.
  4. Remove obstacles. When some part of my workflow feels like a chore, I drop it or I change it.
  5. Have fun! There’s a difference between getting lost in a rabbit hole and tweaking your setup for fun. Some small details might not matter to others, but if they make my day brighter, why not? I underestimated for a long time, how enjoyable the right theme can be (for me it’s “Primary”).
  6. Experiment, experiment, experiment. When I’m stuck someplace, I try out alternatives. If tags work, I use tags. If they don’t, I don’t use them. I don’t care about pointless discussions anymore. Ten Youtubers have twenty opinions. Why should I care who’s “right”? But I’m always open to new ideas. So when I hear something interesting on Discord or here in the Forum or someplace else, I try it out. Most ideas turn out to be useless for my needs, but some really make a difference. The only way to tell what works for me is personal experience.
  7. Unique filenames and Quick Switcher go a loooooong way.

And as a general rule I’d say, that a vault with less than 100 notes doesn’t need any particular “system”. It’s a nice size to experiment with new ideas, but it’s too small to consider any kind of optimization. If you don’t have a problem, don’t fix it.

3 Likes
8

I disagree. I reshuffle my folders frequently. Obsidian makes it really easy, because links work independently of folders.

The trick? Unique filenames.

9

If you find a folder hard to move by dragging, you can right click or long press it and choose “Move folder”.

I believe renaming tags and mass updating metadata easily both require plugins.

This is more difficult if you have more than one property value (like “type: Idea, Great”) and want to search any value that isn’t the first one.

10

My vault now contains around 6,500 notes. Here is my list of the seven deadly sins that may have helped me: