Nearly Endless Indexing

Dear Obsidian Team,

I consider your note-taking app to be the best and most stable application currently available on the market. So, congratulations to you. However, I am unfortunately experiencing a frustrating issue. Let me start by saying that I do not use Obsidian Sync because your pricing policy does not account for our local currency and is very high due to the exchange rate. Additionally, I find the Syncthing solution more suitable. But the problem is not related to synchronization; it is related to indexing.

I have over 10k notes, and unfortunately, the indexing process takes a very long time. Maybe this duration is tolerable on the PC side because the Obsidian application can always remain open in the background on a PC. But this is not the case on mobile. I use Android, and from what I can see, the same issue exists on iOS. For the indexing process, the phoneā€™s screen needs to remain constantly on. This requires a very long time, even more than a day, for my 10k (exact number: 12,397) notes. As you would understand, keeping the phoneā€™s screen constantly on is not good for battery health, as it causes the phone to overheat excessively.

Now, letā€™s get to the main important question. Has Obsidianā€™s failure to resolve the background synchronization issue on mobile so far been a manipulation to make us buy Obsidian Sync? Does the indexing process work differently in Obsidian Sync? Or is there really a technical difficulty at play? If so, it is puzzling to me that an application offering such great features to its users has not ensured background indexing so far. I am genuinely curious about the reason.

Thank youā€¦

2 Likes

The screenshot of the indexing process:

Steps to reproduce

I am experiencing a problem related to indexing not occurring in the background on mobile.

Did you follow the troubleshooting guide? [Y/N]

N- Because the indexing issue is a situation related to my own notes. Of course, if I tried it in a sandbox vault or reduced the number of my notes, my problem would be solved. But I want to keep my notes in a single vault and use them without any problems.

Expected result

I am waiting for the application to be modified so that the indexing process will continue in the background on the mobile application as well.

Actual result

This situation has not yet been implemented as a solution by the developers.

Environment

SYSTEM INFO:
Obsidian version: v1.5.12
Installer version: v1.5.12
Operating system: Windows 10 Pro 10.0.22631
Login status: not logged in
Insider build toggle: off
Live preview: on
Base theme: dark
Community theme: ITS Theme
Snippets enabled: 6
Restricted mode: off
Plugins installed: 76
Plugins enabled: 60
1: Auto Note Mover v1.2.0
2: Find and replace in selection v1.0.1
3: HTML Reader v1.0.12
4: Trash Explorer v1.2.2
5: Hyphenation v0.0.3
6: Colored Tags v5.0.0
7: Search on Internet v0.5.0
8: Chronology v1.1.11
9: historica v0.2.5
10: Hider v1.3.1
11: Mind Map v1.1.0
12: Highlightr v1.2.2
13: Recent Files v1.4.1
14: Update time on edit v2.4.0
15: Mousewheel Image zoom v1.0.22
16: Etymology Lookup v1.0.2
17: Smart Rename v1.1.2
18: Media Notes v1.2.0
19: Bible Reference v2.5.3
20: QuranLookup v0.1.0
21: File Explorer Note Count v1.2.2
22: Map View v5.0.2
23: Media Extended v3.2.4
24: PDF++ v0.39.26
25: Wikipedia v1.0.3
26: Translate v1.4.7
27: Multi Tag v0.10.0
28: Edit History v0.1.3
29: Book Search v0.7.3
30: Zotero Integration v3.1.8
31: Automatically reveal active file v1.0.2
32: Second Window v1.2.0
33: Kanban v2.0.48
34: Rollover Daily Todos v1.1.8
35: Tasks Calendar Wrapper v0.2.9
36: Emoji Toolbar v0.4.1
37: Collapse All v2.1.0
38: ReadItLater v0.4.0
39: Advanced URI v1.40.1
40: Open Plugin Settings v1.3.2
41: Meld Encrypt v2.3.7
42: Text Format v3.0.5
43: TagMany v0.2.7
44: TagFolder v0.18.7
45: Tag Wrangler v0.6.1
46: Style Settings v1.0.8
47: Tab Switcher v1.4.0
48: Sync Graph Settings v1.2.0
49: SupSub v1.0.1
50: PDF Highlights v0.0.4
51: Fleeting Notes Sync v0.6.3
52: Convert url to preview (iframe) v0.5.0
53: YTranscript v0.9.2
54: RSS Reader v1.2.2
55: Beautitab v1.6.1
56: Reminder v1.1.15
57: Tasks v7.3.0
58: Archiver v0.26.1
59: Note archiver v0.1.0
60: Image Toolkit v1.4.1

RECOMMENDATIONS:
Custom theme and snippets: for cosmetic issues, please first try updating your theme and disabling your snippets. If still not fixed, please try to make the issue happen in the Sandbox Vault or disable community theme and snippets.
Community plugins: for bugs, please first try updating all your plugins to latest. If still not fixed, please try to make the issue happen in the Sandbox Vault or disable community plugins.


Additional information

Please follow the troubleshooting guide.

I have read the troubleshooting guide, and there is not the slightest thing in the guide related to my topic and the problem I am experiencing! Why should I knowingly waste time disabling extensions for no reason? Also, I am using the latest version of Obsidian (1.5.12). Did you even read my topic before you wrote the answer?

I did. You clearly did not read the guide, your output above still has restricted mode disabled and youā€™re using css snippets and a custom theme. Once again, please read the instructions.

So then, can you explain how my use of CSS snippets and a custom theme is related to the ā€˜indexingā€™ issue? By imposing formality, you are not helping either me or the users. I am writing in the topic that the indexing issue does not occur in the background on mobile, and that the phoneā€™s screen needs to be kept on. But you are still making irrelevant directions and not developing a concrete solution, dear moderator

Itā€™s all in the guide. You should read it.

Like I told you on discord, we appreciate when folks help us out by following the rules. If you canā€™t invest the minimum effort required for us to reproduce, Iā€™m not going to invest my time trying to help you 1:1 without the necessary information. Good luck with your report.

When the indexing takes a long time to finish or is repeated more than one time, it is a sign of:

  1. Some plugin is misbehaving badly. In which case, you should really try to run obsidian with restricted mode (as suggested in the troubleshooting guide and in the bug report template).
  2. You have some file that for some reason (too big, too complex, weirdly formatted not really a markdown file) is indigest to our parser. In this case, I suggest you populate your vault progressively to try to identify those files.

Indexing is process of scanning your notes for links (to construct the graph and populate the backlinks panel). Indexing and Sync are completely separate.

I really donā€™t understand how Obsidian Sync lacking a feature (background sync) can be interpreted as a manipulation to buy it.

1 Like

Hi,

Seems like I was experiencing a similar problem - an infinite indexing of the vault.

First, Iā€™ve followed the guide. Enabled the restricted mode, and seems like the indexing problem was gone. Then Iā€™ve installed ā€œDivide and Conquerā€ plugin, and bisected both snippets and plugins. Strangely, it didnā€™t help. Likely, I was using git for my vault. Iā€™ve started git bisect, and what was surprising for me, it didnā€™t found the first bad commit. It get me thinking that something might be wrong with the obsidian-index. Iā€™ve figured out that the obsidian index was stored at C:\Users\<user>\AppData\Roaming\obsidian on my Windows machine. Now, I did git bisect the second time, but on each bisect step Iā€™ve also removed the Obisidianā€™s roaming folder. And git bisect found the first bad commit this time!

In this commit Iā€™ve found that frontmatter-links was the culprit. Disabling it and resetting obsidian cache/index directory fixed the problem.

What is interesting - bisecting plugins wasnā€™t useful because seems like the index was containing something that prevented obsidian from working correctly again.

Below are steps I followed:

Notice: it can potentially lead to a data loss, backup all your data first.
Do NOT proceed if you donā€™t know what you are doing.

  1. Find the SHA of the first good commit and TEST it first
    and write it down (just pick an old commit e.g. that
    was made a month ago)
  2. Ensure working tree is clean, and start bisect by running git bisect start
  3. Mark the active commit (HEAD) as bad git bisect bad
  4. Mark the first good commit you found on step 1 as good:
    git bisect good <SHA>
    (git will change the active commit)
  5. Remove the obsidian chache folder:
    rm C:\Users\<user>\AppData\Roaming\obsidian
  6. Start obsidian, open your vault, wait till indexing is done,
    then close obsidian, then open it again, and see:
    1. IF the problem is gone, then:
      1. close the obsidian
      2. run git add -A
      3. run git stash
      4. run git bisect good
    2. IF the problem is still present, then:
      1. close the obsidian
      2. run git add -A
      3. run git stash
      4. run git bisect bad
  7. Go to step 5 until git says <SHA> is the first bad commit

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.