Ignore/exclude completely files or a folder from all obsidian indexers and parsers

I’m learning about this overlay filesystem. I’m reading that the lower layer is read-only. Doesn’t this prevent you from using Obsidian to update notes?

Nope, the lower directory is only read-only from the context of the overlay mount directory :slight_smile:

Some key points:

  • Use the Obsidian vault as the lower directory
  • Keep using Obsidian via the lower directory directly
  • Create “invisible” files/directories via the overlay mount

In the overlay mount, you have the identical directory structure and the same files as in the Obsidian vault, but you are free to create new files and directories which will only be stored in the “upper” directory

The overlay mount is basically just a merged version of the lower/upper directories, and any changes you make in the overlay mount will only be made in the upper directory

Example:

Let’s say you have a vault, named FooVault at ~/FooVault

You would keep using Obsidian directly on ~/FooVault

Create the following directories for the upper/work dirs:
~/FooVault.diff
~/FooVault.work

These directories are to be used by the overlay mount, but you would generally not access them directly. Just create them for now.

Then you want a directory through which you would be able to create directories and files within the context of the vault, but without actually being stored in the vault. Let’s just go with:

~/foo

After creating the ~/FooVault.diff, ~/FooVault.work and ~/foo directories you run:

sudo mount -t overlay overlay -o “lowerdir=$HOME/FooVault,upperdir=$HOME/FooVault.diff,workdir=$HOME/FooVault.work” ~/foo

Let’s say that within FooVault you have a directory such as:

~/FooVault/Projects/Bar

Within this directory you have a bunch of notes and other things related to the Bar project.

This directory and all the files will also be visible at:

~/foo/Projects/Bar

Let’s now say that you have a GIT repo with source code related to the Bar project, you can now store this in:

~/foo/Projects/Bar/src

Under the hood, this will create:

~/FooVault.diff/Projects/Bar/src

When you are using your terminal/IDE/whatever, you’ll access it via the overlay mount at ~/foo, which will see everything combined from both ~/FooVault and ~/FooVault.diff

But when you are using Obsidian (+ in case you want to edit notes or reorganize your vault using any other tool) you would do it via ~/FooVault directly and not via ~/foo

The main thing to keep in mind, besides doing things that should be “invisible” to Obsidian within ~/foo and making any modifications to the vault itself (adding/editing notes, moving vault folders around) via ~/FooVault, is that in case you want to do something like:

Moving ~/FooVault/Projects/Bar to ~/FooVault/Projects/SomethingElse

From the context of the overlay mount at ~/foo you would now have

~/foo/Projects/Bar/src (mapped to ~/FooVault.diff/Projects/Bar/src) as well as ~/foo/Projects/SomethingElse

So to account for that, in case you’re moving folders around in ~/FooVault you need to move them in ~/FooVault.diff as well

1 Like

Hmm that part seems scary. Not sure I full understand. But good to keep in mind if I ever look at overlays.

Thanks for sharing this clever idea!

Also note that some of the issues mentioned in the following help topic also relate to this FR: Stuck on Loading cache - #76 by FernandoIbrahim
I came across the topic since i already forgot that obsidian indexes everything and was wondering why it was taking so long to start after running experiments that created 125k files with a total of 20GB. Now I either have to clear the cache before starting obsidian and have it gobble up RAM while using a core full time while trying to index all the files in the background or throw my current folder structure overboard…
This is the second use case that I ran into personally where the option to exclude files would be great.

I wanted to throw in supporting this feature request. It feels like a must-have to me for software with the scope of Obsidian.

Logically speaking it’s a bit strange that Obsidian doesn’t have this feature already, considering that it already restricts what types of files it shows you. Why does Obsidian index or otherwise pay attention to files that are not shown within its user interface?

My own use case is that I have a folder for a project, and within that folder I have a number of git repositories. It’s very convenient for me to have the files organized this way, and it’s also very convenient to have the top-level folder be an Obsidian vault. However, whenever I actively work on one of the git repositories, Obsidian eventually hangs, I suppose because too many files are changing too quickly for it. Being able to exclude the folders, or even just the .git subfolders, would fix this issue and allow me to work the way I want.

2 Likes

I also think, this would be a huge feature, which is probably comparatively easy to implement.

This feature is basically already implemented via Files and links > Excluded Files. Maybe just copy this setting, name it Ignored Files and skip those files and directories altogether.

For large vaults this could mean a significant improvement.

1 Like