0-byte file (again) - note completely destroyed

I know it’s a rare condition, but it’s not unique to my system. Others have reported it too. I would love to find the root cause of course. I suppose I could disable SIP and run dtruss for days, weeks, or longer to try to hit that bug…all the while eroding my system’s performance by tracing everything 24x7.

But I feel like this post I made above would be a better intermediate step and still be useful. Obsidian should be able to tell on its own accord whether file writes are successful. If it can’t even do that, then I don’t see how we can trust it for any serious data storage.

The dtruss command looks useful, it looks like it could be shortened to simply

dtruss -p $(pgrep ^Obsidian$)

or even possibly just

dtruss -n Obsidian

Why is the Renderer process not worth tracing? I thought I remember reading that a lot of the I/O was handled by that subprocess…

…all the while eroding my system’s performance by tracing everything 24x7.

Just dtrace obsidian. You actively use Obsidian 24x7?

Why is the Renderer process not worth tracing?

I never said the renderer process wasn’t worth tracing. Maybe develop a case of curiosity and try it? I’m not seeing what you and some others are seeing, but if I were and I was inclined to keep using obsidian, I would do what I could to figure out what was going on. I’m not sure waiting around for your suggestions to happen is a sound strategy.

I saw in another reply that you tend to make external edits? Even if Obsidian is somehow at fault here (far from proven by these anecdotes) it’s very likely some pattern of use is causing this. I’d want to know what it was.

Had this happen again yesterday to a large/detailed note. 0.14.15. Was able to recover it using file recovery. But there’s still a gremlin lurking somewhere deep below.

I believe I may have a clue now as to at least one of the triggers that can cause this complete destruction of a file.

If I remember correctly, what happened here is I switched to Obsidian and started typing immediately as soon as I saw the window appear.

The sequence I think I typed was ⌘N to create a new note, followed by the keys G-o-o-g-l-e. I was intending to type “Google Drive

I noticed that what actually happened was:

  1. I ended up on a new note with the filename “og” (chars 3 and 4 of what I typed) and no content. Here you can see that in my recent files list:

    File Recovery for og showing 0 B file:

  2. When I switched back to the note that I happened to be on when I pressed ⌘N (filename: “Archiware Pure free backup solution for VMware ESXi.md”) I saw it had been truncated to just the letters Go, screenshot:

  3. When I opened File Recovery to see if I could get the data back, I noticed 2 snapshots of the destroyed note had been created:

    first, this one (11kb) which seems to have all the data, with simply the letter G inserted at position 1, above the YAML frontmatter:

    then this one (2 bytes) which basically deleted all text and replaced it with Go:

    here is this same snapshot scrolled to the bottom

I hope this may be useful to @Licat to provide a clue as to how this could happen? It feels like a race condition.

6 Likes

Another case of data loss, this happened yesterday (Sun Jun 26 at around 10:55am). Obsidian 0.15.2.

I opened a note via URI (this may be a clue? something unique about launching or opening a note via the ?open URI command?). The note’s title is “Python - List Comprehensions.md

Once Obsidian had opened, I was greeted with a completely empty file. I popped the console open to see if there were any errors, clues etc. Only saw this:

Next I went to File Recovery, which looked like this:

scrolled down…

In the end, it appears the file’s contents were simply 100% replaced by null. I used File Recovery to restore the data and went on with my day.

Since then I tried setting up a script on a loop that simply opens this URI, pauses for a few seconds, quits Obsidian, and then repeats. I ran this script for about 10-15 minutes and could not reproduce it. Does this mean it didn’t happen? Obviously not!

@Licat what do you think—anything about it being triggered via a URI action that could lead somewhere?

Please, don’t tag the devs.

Nothing I can think of honestly. Sounds like a race condition but I guess it is hard to repro. The object in console is just a debug statement that we’ve received the URL action.

Regardless, I guess it would be pretty hard to say what’s going on with all those plugins also loaded, but I can definitely look into what’s going on for the previous report with the og later.

Thank you Licat. Of course I don’t know what’s really going on under the hood of Obsidian. But would it be a lot of trouble to add a bit of debug logging around any method that attempts to replace the editor content with null or ''?

If we could even determine whether this behavior is caused by a plugin (I use a pretty small set to be honest) or from within an internal Obsidian function I think that would be helpful in tracking down a fix or workaround.

1 Like

I don’t think it makes sense to add a debug item just for this case and roll it out to everyone, but it should be pretty easy to implement this as a plugin from your side and monkey-patch the vault.modify function.

I’ll see if I can figure out how to write that plugin. Not sure I’m up to it tbh.

In the meantime, I see Liam has joined the dev team (seemingly to help things along with the macOS side, among other things) which is awesome. I saw this in the release notes for 0.15.5:

  • If a plugin view crashes when getting created, the view will properly get cleaned up.
  • leaf.view can now never be null.

Wondering if those are possibly relevant here…

Those were related to a regression in pane management that came in a recent 0.15 change, so unfortunately I don’t think it’d affect this issue.

Looks like I’m not the only one experiencing this rare bug. This is the bug that scares me the most because it can happen unexpectedly. The one I experienced would replace a file with another (exact duplicate). It’s hard to explain this in text. I move back and forth in Obsidian very quick. I haven’t experienced this bug lately but before when I did it was bad.

Now when I create a new file OR create a new link to a local file, I always “go back” just to make sure that the previous file doesn’t get messed up. For example:

Both File A and B has a lot of content. I want A to link to B, so I edit A by putting a link to B. Then I preview A so I can click on that link to B, but when I go to B, all of its contents get replaced by all the content in A, basically B is now an exact duplicate of A. That usually happens when I do things VERY FAST, now I’m paranoid and I slow down.

I set interval to 1 min for file recovery snapshot. I would like to know where the backup files are located. Also when I move the vault around to different location, I notice that I have to start all over with the snapshots. Is there anyway to prevent this? Can’t OBS not start all over?

Note: I store my vault on a usb stick / external HDD. Not sure if that has any affect.

1 Like

Just adding another data point here.

I just experienced another full note erasure, 0-byte file left in my vault. This is with Obsidian 0.16.2. I haven’t had time to try to create the plugin to monkey-patch vault.modify as Licat suggested (yet). If any kind soul feels up to that task it would be wonderful.

1 Like

I’ve just got this again. Today note is empty and File recovery doesn’t contain the latest version of the note, only the initial content that was created by Templater. It’s so strange because I’ve set File recovery to make snapshots every 30 seconds and store for a year because of such data losses. It happens from time to time, I’ve already disabled the most of plugins I use to watch if it helps but it hasn’t helped.

SYSTEM INFO:
Obsidian version: v0.15.9
Installer version: v0.15.6
Operating system: Windows 10 Home Single Language 10.0.19044
Login status: not logged in
Insider build toggle: off
Live preview: on
Legacy editor: off
Base theme: dark
Community theme: Deep Work
Snippets enabled: 3
Restricted mode: off
Plugins installed: 74
Plugins enabled: 9
1: Banners v1.3.3
2: Periodic Notes v0.0.17
3: Templater v1.14.0
4: Hotkey Helper v0.3.15
5: Auto Link Title v1.2.5
6: Homepage v2.2.1
7: Remember cursor position v1.0.7
8: Omnisearch v1.5.2
9: Lapel v0.1.0

And this has happened again but this time File recovery had the latest snapshot fortunately. I have no idea what is the trigger for notes cleaning because I repeat almost the same actions every day with the same set of plugins but it happens without any evident for me reason

1 Like

Just happened to me again— 0.16.3. Can anyone help point me in the right direction on starting a monkey patching plugin to log vault.modify calls?

Had another note destroyed just a minute ago. So frustrating that we made it all the way to 1.0 with this still happening regularly.

Obsidian Version 1.0.2 (Installer 1.0.0)

I experienced this bug multiple times, but there are multiple weeks between when it happens.

  • It only occurs when I close/open the Obsidian vault (or somewhere between when I close and open the vault). The file content was intact when I closed the vault and when I open it again I’m greeted with an empty file.
  • Only occurred to the note that was displayed when the vault was closed. This means that it was easy for me to notice the data loss.
  • The vault is stored locally on my computers disk.
  • The OS timestamp of the file suggests that the file was not modified when I opened the vault. The vault was opened December 31, 2022 1:10 PM as shown in the File Recovery diff, but the OS modified date was December 29, which is the date where I last modified the file.
  • The vault is stored in a folder which is monitored by Syncthing. But all edits was made on the same computer and in the timeframe where the bug occurred, no other computer that shared the vault folder was turned on.

Fortunately I have been able to recover the notes using File Recovery in all the cases.

It seems like the bug is not OS dependent as it occurs on both Mac and Windows. Here is another thread with a Windows user experiencing what seems like the same bug: Complete data loss on last active note when opening vault this morning

Let me know if there is anything more I can do to help resolve the bug.

SYSTEM INFO:
Obsidian version: v1.1.9
Installer version: v0.15.9
Operating system: Windows 10 Pro 10.0.19044
Login status: not logged in
Insider build toggle: off
Live preview: on
Legacy editor: off
Base theme: adapt to system
Community theme: none
Snippets enabled: 3
Restricted mode: off
Plugins installed: 2
Plugins enabled: 2
1: Code block from selection v1.0.7
2: Quick Switcher++ v1.0.8



I checked my backup software which have a version history and can see that the file was wiped on December 29. This means that the bug probably occurred when I closed the vault and not when I opened it.

Just had a fresh 0-byte file incident. :rage:

Simply launched Obsidian, added a few lines of text and pasted in an image, then quit. All within a span of around 4 minutes. File destroyed.

macOS 13.1 + Obsidian 1.1.12 (Installer 1.1.9)

I make a backup (FreeFileSync) with the setting not to backup files with a size of 0 (to not accidently overwrite some precious notes with nothing), because I had the same experience (many times). I guess, the file content is lost by closing obsidian when files are tabbed.

One time I was the problem, so I switched the warning by deleting a file on again.

The problem seems to have something to do with quantenphysics: If you look at it, everything seems fine …:smiley:

A few days ago I had some images horizontally cut (but not completely deleted).

I wonder if I should make a tool that automatically makes a backup history of at least the .md files.

Oh, and I lost files in the plugin directory, too.

BTW: Obsidian hates it (similar to open Excel files), when it runs the vault from an external USB drive which now and then loses its connection - understandably.

The problem is, to get aware of this.

Latest Obsidian, latest Windows 10.

1 Like