Problems with Indexing Crashing Obsidian

Help
1

Hi, I’ve been using Obsidian since about .8 with a massive vault, and multiple plugins, more or less without issue on Windows 10. Occasional troubleshooting with plug-ins or clearing the cache or renaming the vault always succeeded in clearing what minor problems I may have experienced.

However, when I updated to 1.5.12, which was also going to allow me to move to some new plugins that had taken over from things that were no longer supported, I started having consistent problems.

Immediately after updating things seemed to run fine, but the next time I loaded Obsidian would go non-responsive after anywhere from half a minute to 5 minutes. Having had problems with updates without re-installs, I uninstalled obsidian and reinstalled with a clean installer.

Now, obsidian would at least tell me that it needed to re-index my vault, but somewhere after a couple hundred files the screen would go black, and Obsidian would go non-responsive. I tried with the debug window up, and it looked like the omnisearch (plug-in) was having an out of memory error. I disabled it. And then obsidian’s native index method would have the same problem. I disabled all plugins. Obsidian would be stable for about 10 minutes, but would still eventually go non-responsive. If I was watching the indexing pop-up, it would get about a couple hundred files into a target of about a 1000, and then fail.

I tried re-naming the vault, and re-pointing; I tried clearing the cache; I ran with and without plug-ins, but the debugger would no longer note anything before obsidian went non-responsive. Watching the processes, the Archives 1.5.12 sub-process would be chugging along, clearly indexing or snapshotting (unclear) with a large memory pull, and then suddenly and precipitously drop in memory use - exactly when Obsidian would go non-responsive. If I closed, or even force closed, the main obsidian process, the Archives process thread would continue running as its own background process. It would have to be killed separately otherwise re-launching obsidian might not even load.

Since I had not experienced this on 1.1.15 (which is what I was on before the update), I decided to backstep, uninstalling the current version, and installing earlier versions. The problem persisted with every version before 1.5.12 until I got to version 1.3.7. At version 1.3.7 with or without plugins, the native index or omnisearch worked fine again, and Obsidian was stable with or without plugins.

I’m not sure what going on, but it happens reliably for me - and the only thing I can think of is the size of my vault, and that something in the earlier version avoided an out of memory issue that the more current ones don’t, but I have no way to catch a log, since the debugger view appears to wholly crash and be non-responsive.

Thanks

2

My apologies for not saying this explicitly in the original post - but if I make a completely blank, empty new vault, the problem does not occur, nor does it occur with any of the plugins I commonly use or not, which is why I am leaning toward the thought that it is the size/number of files in my core vault causing the indexing crash.

3

I’m moving this to Help, since it doesn’t follow the bug template. (It doesn’t mean we won’t try to help if we can. And it can be moved back if some reproducible bug is identified.)

  1. Since you were upgrading and downgrading, my first question is this: Had you been updating your community plugins? If you downgraded that far, and the plugins began working again, I’m assuming maybe no?

    There’s a chance that updating all your plugins may have corrected some of those issues.

  2. Also, when you had Obsidian upgraded to 1.5.12, do you know what your Obsidian installer version was? If you had a very old installer version, there is a chance that was causing some issues. There are some major versions where it is required to update manually. (Since you downgraded, I assume this did update your installer version to that downgraded version each time.)

  3. Also since you were upgrading and downgrading a bunch, I’m not sure if this is going to be helpful or not, but can you please post your “Show Debug Info”? At least then we’ll know what operating system you’re on. (There’s a template for a reason!)

4

Actually, I had not been updating anything prior to this update - while I was using 1.1.15, I had a stable setup with plugins installed at that time, and thus did not update anything during that period.

  1. When I updated to 1.5.12, I then proceeded to update all the plugins I used. One was no longer supported at all, but had a fork which was supported, and switched to it (Calendarium).

  2. I do not recall the version of the Obsidian installer was at the time - but having had issues before, that is why I eventually uninstalled completely and re-installed from a freshly downloaded version.

  3. During the initial run after update when everything pretended to be stable, all the plugins appeared to be functioning normally, and I had no oddities. It was only once I closed and re-opened obsidian that it began crashing.

  4. When I began backstepping to find a version that wouldn’t crash out, almost all of the plugins I used (and had updated) continue to work without an issue when I got back to Obsidian 1.3.7, except the ones by javalent (all the RPG plugins), which could not load, and I turned them off.

Here’s the current debug info:
SYSTEM INFO:
Obsidian version: v1.3.7
Installer version: v1.3.7
Operating system: Windows 10 Pro 10.0.19043
Login status: not logged in
Insider build toggle: off
Live preview: on
Legacy editor: off
Base theme: dark
Community theme: none
Snippets enabled: 0
Restricted mode: off
Plugins installed: 30
Plugins enabled: 25
1: Waypoint v2.1.0
2: Editing Toolbar v2.4.10
3: External Link Helper v0.8.0
4: File Hider v1.1.1
5: Note Auto Creator v1.4.0
6: Open Gate v1.11.6
7: Tasks v6.2.0
8: Folder notes v1.7.20
9: Sort & Permute lines v0.7.0
10: Fantasy name generator v0.1.0
11: Text Format v3.0.1
12: Wordy v1.3.1
13: Mousewheel Image zoom v1.0.22
14: Text Extractor v0.5.1
15: Omnisearch v1.21.1
16: Advanced Paste v2.7.0
17: Advanced Tables v0.21.0
18: Commander v0.5.1
19: Dataview v0.5.66
20: Enhancing Export v1.10.4
21: Excalidraw v2.1.1
22: Image in Editor v2.2.6
23: Import Foundry VTT journal entries v0.8.3
24: Leaflet v6.0.5
25: Second Window v1.2.0

Thanks again for the help!

5

I tried to get as much out of OP’s overflowing lines as possible.

What I would do (but I’m not an RPG player):

  • Deactivate all plugins that were not updated for a long time (you can start on zero; rename community-plugins.json file to something you can find later).
  • Index vault with Omnisearch disabled (you can try re-enabling it later).
  • Before indexing I would delete the whole config folder (not the personal one such as .obsidian but the one I talk about down below).

In this post I was talking about similar things but here I am hinting at a complete erasing of the contents of C:\Users\<username>\AppData\Roaming\obsidian – start with a clean slate. You can leave your Custom Dictionary.txt in there if you like spelling errors overruled.

You can install 1.5.12 and try out the (newer) plugins that work. I mean, Obsidian will get to 2.8.5 one day and you want to stay on 1.3? Or get a Linux install on dual boot and use a fresh install there.

6

I’m giving this a try now.

Previously I had gone in to the Roaming folder and removed the various cache files, but not the entirety of the folder’s content (id, Local State, etc.).

And no, I don’t want to stay on 1.3.7 (or an earlier build) forever, but I did want to access my data reliably while reaching out for assistance.

Also, I prefer stability for extended periods of time until there is enough impetus to move to a new version, and its attendant plugin support.

Again, thank you for the suggestion, and I’ll post once its done chugging through the process of rebuilding the index (I hope).

7

So, trying a completely removed roaming profile folder with a fresh 1.5.12 install does technically work, but takes about an hour to load my vault.

Even after it has indexed the vault.

So, that’s not a long-term viable solution.

Omnisearch does not appear to crash the program. But having to wait an hour after trying other plugins on a crash, I’ve only been able to verify one likely plugin failure.

8

What takes an hour, the indexing?
How many files?

Are there any files with content such as base64 pics or anything that could halt the process?

More info is needed.

9

Opening vault and loading cache actually go relatively quickly, and then “Loading Workspace” takes about an hour.

The vault has thousands of files in it - my vault is overlaid with my entire personal datastore ever since I started using obsidian - not just md files, but also audio files, video files, text, odt, PDFs. Which ends up being all the files that I might want to crosslink, though that primarily happens with my RPG data.

This is the first time I’ve ever had any sort of problem with it loading the workspace. Previously it might take a minute or two. Five or 10 tops during the pre-v1 era.

For instance, I went to a separate dedicated computer I have where I have the entire datastore and vault synced - but have not updated or synced since before I updated the primary computer.

Same datastore, perhaps a score less files, older version of obsidian, and opens up completely in about 2 minutes.

10

I have 18100 files and have experienced problems with loading the workspace myself but never this long.
See thread:

I am beginning to think it is not a pre-1.5.8 issue (not sure about where any electron changes happened) but it cropped up before 1.5.3. as well. Hard to keep track and I cannot investigate.

The loading of the workspace does take longer when doing a fresh install but a I like to delete all things in the folder specified above (or corresponding folder on other OS) anyway once in a while (every month or so) to have faster searches.

Problem is that if I delete or rename the workspace.json file, it doesn’t seem to make a difference. Possibly the workspaces.json fle would need to be renamed (deleted) as well.

So more investigating is needed but so far I’ve been able to use Obsidian and as you can see from the lack of user responses, I am not exactly being helped by brainstorming together.

I am on Linux (not Wayland) and tried only the debian package and recently the appimage. I don’t quite remember my experience on Windows 10 now as it has been months since I last used the Windows install.

So maybe try to get a 1.4.16 installer – I think that’s still compatible with the latest 1.5.12 version (if not, the 1.5.3 one). Try renaming the workspace and workspaces files to bak or backup or sg as well.

1 Like
11

Well, there is a workspace.json, but not a workspaces.json in the .obsidian folder.

But, I’ll give that a try now.

12

One more thing:
Are there any external files linked or symlinked to the vault? If you don’t have the external drive attached, it can cause a 20-30 minute hang time.

I learned the hard way, and no longer add my symlinked folders with files (some 20000 of them) to my vault. If the external drive is not attached, Obsidian needs to offload these files and takes long.

1 Like
13

No, everything that the vault might look at is in the vault. Which is why it’s so large.

So, no files are linked from external files, nor network accessible drives, nor the internet.

Renaming workspaces does not appear at this point to have had any impact. We are at 8 minutes an climbing on the Loading Workspace page.

14

We need someone with good diagnostics skills to step in and file a proper bug report.
But that could take some time.

BTW, the other day I reproduced with no plugins and the default theme but whenever I mention Linux, I feel like getting into a “meh” situation.

So I’m out of ideas here now.

Did you try the the pre-1.5.3 installer? 1.4.16 I think it was (that’s what I still use on iOS as well, BTW).
Also switch off WiFi and not let Obsidian update and disable the auto-updater.

15

Also, there are I think 3 installers (4?) on Windows: touchscreen one, 32bit, 64bit and all-users where you get some extra functionality (admin rights or sg).
Try each of the latter 3 as well maybe?

16

That will be my next step, but I also have people coming over for a reunion gathering, so, it may have to take a backseat until after.

Also, yes, I never allow automatic updates, that’s just how I roll. So we don’t have to worry about that.

If 1.4.16 (default 64 bit for windows) installer doesn’t work, I may give the all users version a shot, but I may also engage in some shenanigans using the profile and .obsidian folders from the other machine which is still functional when I have a chance.

Thank you for your time and ideas!

1 Like
17

Ok, I think I have it.

And it looks like there may have been three separate issues.

  1. Obsidian loads, but after indexing fails, screen goes black. The black screen issue that appeared first, appears tied to all four of the RPG plugins I was using, when they appear in the right hand side-bar.

  2. Indexing fails and screen goes white, the plugin folders and their attendant additional files are still in the .plugins folder, even though they are off.

  3. Workspaces loads for hour plus. Unknown problem with the Roaming profile folder, the obsidian folder, and the obsidian-installer folder under Local. Possibly connected to errors in the workspace.json as well.

Once I uninstalled, cleared the Roaming folder again (third time), removed the obsidian-installer folder, and removed all the plugin sub folders from .plugins again, and re-installed obsidian with 1.4.16, I directed obsidian to the vault, and it opened everything in about a minute, indexing complete.

Omnisearch & Text Extractor were re-added, and everything worked.

Added plugin folders, had problems. But when removed, went back to functioning normally, was finally able to add folders one at a time and pin down the RPG sidebar plugins as the black screen issue.

Have been able to update to obsidian 1.5.12, and it now appears stable.

I will try to install the RPG plugins completely from scratch in the future, but for now, I have a quick opening, stable obsidian vault again.

Thanks again for the brainstorming!

2 Likes
18

Glad you can drink some beers in celebration.

19

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.