Ignore/exclude completely files or a folder from all obsidian indexers and parsers

“Excluded files” option doesn’t quite work for me as the most annoying features of having an unwillingly indexed folder for me are not excluded from parsers

Use case or problem

Im currently hosting my obsidian vault in Mega, Mega currently uses a folder called “Rubbish” for local cache, hosted, in this case, inside the vault.

The rubbish folder i cannot deactivate in any way nor rename it to “.rubbish” (for obsidian to ignore it) as the service just creates a “rubbish” folder again

Problem is, that while the native “Excluded files” option takes care of search, graph view and unlinked mentions, the more apparent problems of this peculiar inconvenience stay, as:

  • When opening ‘backlinks’ pane on a certain file (ie. a daily note) under linked mentions appears 10, or 20 notes with the same name as, besides the correct linked note, they are 10+ backups on the unwillingly indexed folder

  • When linking to a note that has a duplicate in the unwillingly indexed folder (almost every note), the link shows the long path (as it needs to differentiate from the multiple backups) thus links appear excessively long instead of just the name of the file

  • The backups, although less noticeable, are still visible in link suggestions, making them more prone to human error linking them instead of the current version

  • And of course, it appears on the files pane, i’ve tried the hide files plugin, but weirldy enough it doesnt work after restarting obsidian

Proposed solution

Either

  • an option to fully ignore a path from all obsidian parsing/indexing
  • an option to designate a backup folder (as the .trash folder natively does for obsidian) to ignore it from all parsing/indexing
  • flags/options for the user to decide which indexers will ignore/ still work/ still work but in a lesser extent, under the excluded files section, instead of a fully curated option
34 Likes

I have a similar problem. I have very large folders than need to fall within the scope of my vault (for other unrelated reasons), but which are never needed in obsidian and just drastically slow down opening and indexing time. Having a proper way of excluding folders or paths from being index would massively improve performances for someone like me (and that’s not helped by the fact that I also already have quite a large note vault with a couple thousand files and figures referenced)

3 Likes

I have a similar problem with a mono repository. Yarn adds a big node_modules folder with 250k files and 6000 markdown files. I can neither delete it, nor rename it, because it’s maintained by yarn. Obsidian chokes under the huge amount of files.

1 Like

I think this issue I created is also basically requesting the same thing: Actually excluding paths from Obsidian entirely - #4 by Zwergziege

This seems to be requested by a growing community of users, mostly devs who want their repos to double as a knowledge base.

1 Like

So I’m aware of the Files & Links > Excluded files feature, and have seen several other threads here discussing specific needs like excluding a path from quick view results, but not addressing a general need to make certain paths invisible to Obsidian entirely.

Use case or problem

I want to use Obsidian to manage / author documentation in my git repos for various development projects. The git repos would exist as subdirectories (submodules) underneath my Obsidian vault. I plan to use the Obsidian Git plugin to sync.

The problem is that in the git repositories, my other working files also exist and Obsidian looks at all of them, everywhere. I really only want Obsidian to deal with markdown files and selected other files. Attachments would specifically be stored under attachments dirs.

The result of this is that Obsidian indexes thousands of files that I don’t care about… like READMEs in node_modules etc… and it ends up unusably slow. To the point where file operations take upwards of 10s to complete.

I imagine that this is a fairly huge blocker for any developer who wants to seriously use Obsidian to support documentation in their working repositories, which is where Obsidian would really shine - documentation is often hard to make sense of in dev repos.

I’m happy to tell Obsidian where to look or not look. I’ve already configured the Excluded files feature with Regex accordingly, but it seems to make no difference to what Obsidian indexes.

Proposed solution

The .obsidianignore file that was recommended in another thread is a great idea.

Another option is to enable a ‘Whitelist’ mode, where Obsidian starts off indexing nothing, and you specifically give it paths in glob syntax to include.

Also, configuring Obsidian to respect .gitignore would be incredibly useful.

Current workaround (optional)

I haven’t found a way around this. I’ve tried the existing features (that I know about) to no avail.

Related feature requests (optional)

9 Likes

I wonder if you could create a symbolic link tree that filters out what you don’t want.
Check out some solutions at bash - Create symlink tree in existing directories - Unix & Linux Stack Exchange

Hmm, that’s actually a really interesting idea…

Off the top of my head, symlinks can’t filter the contents they are symlinking to, or be overlayed, so it would require the duplication of the repo’s directory structure in the Obsidian vault, and the symlinking of each individual file that’s to be included.

Also look at overlayfs which is mentioned in your link, but that also seems to lack the ability to filter paths that are overlayed.

It’s an interesting direction to go…

Still would be far simpler having a .obsidianignore file and respecting .gitignore.

2 Likes

+1 from my side. In my case I’d like to have some 3rd party tools in the vault which require symlinks to system files which require special previleges. This is even worse than long waiting times for commands, since Obsidian simply fails indexing all the files at startup, reports an error and dies. Having the option to truely ignore files and folders would thus be very helpful.
I think both a toggle in the Excluded files list and .obsidianignore files would be good solutions. As a git person, I favor the latter but have to admit that the former better fits the feel of obsidian.

3 Likes

That is only a workaround, which works differently on each platform, think mobile or windows (where SymLinks have to be enabled by an Admin).
We need something built-in that works the same across all platforms.

2 Likes

+1 here; I’d love to use obsidian to connect all of the projects and notes in my PhD work - many of which are repositories themselves with i.e. node_modules/ … I’ve used regex to ignore most of these but obsidian still sucks down 5gb of RAM and has to re-index 14k files each time I open it, then has boggy performance afterwards.

I would second a kind of .obsidianignore file type, that seems like a smart and well-understood solution, though perhaps just by the git-folks community.

3 Likes

Maybe I misunderstood something from that post and this feature but after facing the same issue with docusaurus and hugo projects I took the problem from a different perspective.

With docusaurus where I simply bind my mardown directory (in that case) ./blog ./docs and ./src in my Obsidian Vault which give something like this

  • Obisidian Vault = $HOME/Vaults/*
  • Static Website = $HOME/Public/*
sudo mount -B $HOME/Vaults/website.tld/blog $HOME/Public/website.tld/blog
sudo mount -B $HOME/Vaults/website.tld/docs $HOME/Public/website.tld/docs
sudo mount -B $HOME/Vaults/website.tld/src $HOME/Public/website.tld/src

and so; obviously it is tight to the user (via a script) or the machine (via /etc/fstab) but it works well

1 Like

Out of my head, it is also possible to relay on git and is submodule power
like in both project
Obsidian and Hugo you have a git repo for them and the content of your post are managed as a submodule of them

GitSlave is apparently the new way of doing git submodule with frequently update https://gitslave.sourceforge.net

I want to bump this. My use-case is similar to many other developers - I want Obsidian as my documentation format that lives right there in the repo with the source files.
Problem is, my tools generate tens of thousands of service/cache files, multiple GBs total, and when that happens - Obsidian REFUSES TO LOAD completely.

Of course because the third-party infrastructure is closed-source I can’t just rename the folder with a leading “.” - that will break the project. Dancing around with symlinks, or having project source in one submodule, obsidian vault in another is very unpleasant to work with. I think a setting to exclude folders from Obsidian’s consideration completely is needed - tools are more powerful when they can work together.

5 Likes

I store some code in the same folder as my Obsidian vault and at one point had a build artifact with the same name as one of my notes (but without the .md) extension. This meant that when I tried to open the note via a Wikilink it would open the “Open With” menu, because it thought I wanted this build artifact that Obsidian can’t open anyway. Very strange behavior, and would be easy to solve if I could just exclude the whole folder completely from Obsidian as suggested here.

1 Like

Use case or problem

Some my notes and folders contain a lot of junk text, guides, chapters from books and so on. When searching all this text pollutes search results.
I know about “path:-” and “Settings > Files & Links > Excluded files” but these are inconvenient or not working solutions for daily use.

Proposed solution

Add to “Settings > Files & Links > Excluded files” analogue “Completely excluded files”

5 Likes

I second that! We would like to set up an obsidian Vault in our development project root Folder and we have 90% non-markdown Files in there, which clutters the Folder View (I know I can change the CSS) and makes Obsidian Operations unbearably slow.
I second this idea of excluding specific Extensions (or including only specific ones) using Glob Pattern similar to .gitIgnore.

2 Likes

As far as I understand, this is a duplicate of

[link outdated, the mods merged everything here.]

2 Likes

Honestly, I don’t understand what is standing in the way of being able to exclude directories and files completely from Obsidian’s awareness. There’s a whole, broad use case waiting to be tapped - that of developers, development teams and their markdown knowledge bases across their git repositories.

Literally all that’s missing is being able to ignore dirs like node_modules or dist or .git.

The perfect solution would be to support .obsidianignore and/or perhaps optionally respect .gitignore files.

7 Likes

+1 to this feature, the current implementation completely rules out many use cases of obsidian for developers

2 Likes