Bug: Opening a vault causes excessive access to IndexedDB (LevelDB), resulting in 100% CPU usage

Bug graveyard
,
1

Steps to reproduce

  1. Open a large vault on macOS
  2. Open Activity Monitor and inspect the CPU usage of Obsidian.app

Did you follow the troubleshooting guide? [Y/N]

Yes.

The issue can be replicated with the Sandbox vault, see the steps in “Additional information” below.

Expected result

CPU usage should return to very low percentages once the vault is fully opened.

Actual result

CPU usage stays near 100% for several minutes. For large vaults, high CPU usage can even remain for over one hour.

Environment

SYSTEM INFO:
Obsidian version: v1.3.5
Installer version: v1.3.5
Operating system: Darwin Kernel Version 22.5.0: Mon Apr 24 20:53:19 PDT 2023; root:xnu-8796.121.2~5/RELEASE_ARM64_T6020 22.5.0
Login status: not logged in
Insider build toggle: off
Live preview: on
Legacy editor: off
Base theme: dark
Community theme: none
Snippets enabled: 0
Restricted mode: off
Plugins installed: 0
Plugins enabled: 0

RECOMMENDATIONS:
none

Hardware: MacBook Pro M2 Pro

Additional information

Steps to replicate the issue with the Sandbox vault

The Sandbox vault does not trigger high CPU load for minutes, but it is possible to observe excessive file access to IndexedDB files that can also be observed for large vaults for durations from minutes to over an hour.

  1. On macOS, open a shell and cd to a suitable directory for writing output to
  2. Run sudo fs_usage -f filesys -w Obsidian | tee obsidian_leveldb.log
  3. Open the sandbox vault
  4. When the sandbox vault finishes loading, stop the fs_usage command
  5. Open obsidian_leveldb.log (from step 2) and search for IndexedDB.

Find massive amounts of stat64 access logs to files under these paths:

~/Library/Application Support/obsidian/IndexedDB/app_obsidian.md_0.indexeddb.leveldb

and

~/Library/Application Support/obsidian/IndexedDB/app_obsidian.md_0.indexeddb.blob

The larger the vault, the more of this indexeddb access happens, and the longer it takes until the CPU returns to near-zero %.

2

There isn’t necessarily anything wrong with this, especially if this happens during the indexing phase. Moved to the help section.

3

@WhiteNoise Apologies, I should have mentioned that I have reached out for help on this issue already (here and here).

I was advised to check the debug steps, which I did.

Please consider this as a bug. It happens every time a vault is opened. That is: close the vault window, open it → 100% CPU for a long time.

This is super annoying and drains the battery when off the grid.

4

It does not happen to me.
When a vault is first opened there is an indexing phase that depends upon the size of your vault and can continue across multiple runs until it has finished. That is normal.

If what you are describing is something else, kindly provide reproducible steps starting with the sandbox vault.

5

I have provided repro steps for the sandbox. Due to the small size of the sandbox vault, the effect is too short to see in the Activity Monitor, but the fs_usage log clearly shows the same behavior of excessively calling stat64 on indexeddb files.

The CPU hogging behavior should start becoming obvious with a large-enough vault.

How large is the vault you tested with?

6

I don’t seen any persistent CPU usage nor excessive IO with the sandbox vault. Also what is excessive to you may be fine for us.

We have test with 20k notes but it’s not just a matter of number of files.

7

One of the vaults that exposes the behavior consumes 32GB of disk space and has 20k items, of which 3k are Markdown files.

The Sandbox vault does not expose the excessive CPU usage to a human observer, but the file access log shows the same pattern of excessive stat64 calls that I observe with larger vaults over the course of many minutes.

8

Again excessive here is your opinion.

If and only if there is vault where the CPU usage doesn’t go down after the initial indexing (which is done one time on each vault), upload it somewhere and dm me the link.

9

Seems I am unable to get my problem across properly.

What I describe as excessive is getting 2,500 lines of file access log in 0.04 seconds..

If and only if there is vault where the CPU usage doesn’t go down after the initial indexing (which is done one time on each vault),

To be clear, I am NOT referring to the one-time indexing of new vaults where Obsidian displays a note saying that the vault is being indexed.

As I have explained before, I observe this EVERY time I open a vault window. Even if the window is closed a few seconds before and there would be nothing new to write to indexedDB.

Every. F***ing. Time.
Minutes. To. Hours. Of. 100. Percent. CPU.

I don’t dare to close Obsidian vaults anymore, out of fear that I might have to to open one while on battery and see the remaining battery time drop from 10h to 3h immediately.

No, it is definitely not an opinion.

upload it somewhere and dm me the link.

What does “it” refer to?

10

This does not happen in the sandbox vault or any other vault I have tried.
If you can create a vault where this happens(base theme, restricted mode on), zip it, upload it somewhere, and dm me a link. I’ll see if I can observe the same problem on that vault.

11

Ok, got it.

There are a few problems associated with your request:

  • I would need a way to generate 20k+ files of meaningful size (and content perhaps). Happy to get any hint on how to achieve that.
  • Since you do not see the issue with 20k of files but I do, I assume the difference lies in the total size of all files. I am not exactly looking forward to uploading 40GB to some online file store through a rather mediocre DSL connection.

This does not happen in the sandbox vault or any other vault I have tried.

Are you referring to the CPU load, or the file access pattern seen with fs_usage?

12

I am referring to “The CPU load at 100 percent for hours, even after the one-time indexing is done.”

The size of non-markdown files is (most likely) not relevant (not the number).

13

I am referring to “The CPU load at 100 percent for hours

Ok, what if we assume for a moment that my observation is valid and the large number of stat64 operations is connected to the 100% CPU load I observe while seeing this pattern.

Do you see this pattern at your end?

Can these stat64 operations be tracked back to a specific task that Obsidian does on every start of a vault?

14

It could be or not. Hard to tell. It could also be a weird file that trips the parsers.
I can’t say much without seeing vault where the 100% cpu forever happens.

15

Ok, so I’ll try creating a repro case

16

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.