# Data loss - 0 byte files (revisited)

**URL:** <https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953>\
**Category:** Bug graveyard\
**Tags:** upstream-bug\
**Created:** [May 10, 2021, 7:06pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953 "2021-05-10T19:06:26Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 10, 2021, 7:06pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/1 "2021-05-10T19:06:26Z")

</div>

- Obsidian 0.12.4 (Installer 0.12.3)
- MacOS 11.4

Continuation from [Data loss - zero byte .md file in vault](https://forum.obsidian.md/t/data-loss-zero-byte-md-file-in-vault/13538)

This is still happening to me. I’m currently only using Obsidian on a single Mac. As suggested by @WhiteNoise in the last post before that thread was locked, I reduced my recovery snapshot time to 2min. This helps in the sense that there might be more recovery points, but does not address the underlying problem.

I don’t have 100% repro steps for this nor do I think I will be able to produce them. I’m more than willing to put in some time and do any sort of testing but I can’t sit here and bang the keys like a monkey until this happens. All I know is it _ **is** _ happening and it’s very unnerving for a “second brain” app to be vaporizing information like this.

In the screenshot below from a few minutes ago, you can see the history of the snapshots as I was working in this note, every 2-3 min. All of a sudden _poof_ there’s a 0 byte version saved.

 ![image](https://forum.obsidian.md/uploads/default/original/2X/6/681b68d58d5e89056f71c18ed31e65ab65452497.png)

I understand the default position is to blame:

1. Plugins
2. The OS / filesystem.

I use very few plugins and they are all widely-used ones like Templater and Advanced Tables. Also, this data loss is not happening _while_ I’m editing the note, it’s happening during either quit or launch of the app - because in this case I was only working on this particular note over the course of a few hours. I had recently quit Obsidian and was just launching it to resume work when I saw the note was blank.

Where is the logic in whatever routine sees a 2.14 KB file suddenly become 0 bytes and decide “yeah that must be right, let’s keep the 0 byte version”. Without any notice/log/user confirmation? It makes no sense to me. Even if this somehow **IS** due to a filesystem bug in macOS, there needs to be some other warning or log produced, otherwise this is simply impossible to track down.

Honestly I am stuck. I’m a huge fan of Obsidian but I am at a loss of how to proceed. What do you suggest I do?

---

<div class="post-metadata">

**Author:** ![WhiteNoise](https://forum.obsidian.md/user_avatar/forum.obsidian.md/whitenoise/32/11_2.png) [@WhiteNoise](https://forum.obsidian.md/u/WhiteNoise)\
**Post date:** [May 10, 2021, 7:25pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/2 "2021-05-10T19:25:56Z")

</div>

do you have other programs running on your vault?

What are the plugins you have enabled exactly?

---

<div class="post-metadata">

**Author:** ![WhiteNoise](https://forum.obsidian.md/user_avatar/forum.obsidian.md/whitenoise/32/11_2.png) [@WhiteNoise](https://forum.obsidian.md/u/WhiteNoise)\
**Post date:** [May 10, 2021, 7:46pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/3 "2021-05-10T19:46:48Z")

</div>

First, I suggest you go plugin-less for 4 days.

If you have no problems, I propose this strategy.

1. Divide your plugins in two disjoint groups A & B with roughly the same numbers of plugins.
2. Run Obsidian with Group A for 4 days

- If the bug doesn’t happen, discard A. Run Obsidian with Group B for 4 days
- If the bug does happen, discard B. Divide the plugins in A in half, say A\_1 & B\_1. Go to step 2.

You should pinpoint or at least narrow down which plugin is causing the issue.

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 10, 2021, 7:49pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/4 "2021-05-10T19:49:57Z")

</div>

Hmm. @WhiteNoise your post made me think. I do have a script that backs up my vault every so often. It’s a simple bash script that just tar.gz’s the whole vault to a timestamped archive elsewhere on the disk. Should be totally non-destructive.

But, the last time it ran was at 1:51pm today. Which I note is right before that file got nuked.

Maybe there’s some sort of file-locking that happens when tar is zipping everything up that causes a race condition vs. the file being saved by Obsidian or something…

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 10, 2021, 7:52pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/5 "2021-05-10T19:52:47Z")

</div>

Sadly, I don’t think that was it.

As you can see from the timestamps in the logs below, my backup script fired at 1:51:31pm, but inside the gzip itself, the file that was nuked has a timestamp of 1:48pm, so the zero’ing out happened minutes before the backup took place.

![image](https://forum.obsidian.md/uploads/default/original/2X/1/166c2949d18bb0a0c7d6b038532fc77da2db620f.png)

![image](https://forum.obsidian.md/uploads/default/original/2X/5/5fc1b00399f002b0bb639ed92efad37904873362.png)

---

<div class="post-metadata">

**Author:** ![pmbauer](https://forum.obsidian.md/user_avatar/forum.obsidian.md/pmbauer/32/10201_2.png) [@pmbauer](https://forum.obsidian.md/u/pmbauer)\
**Post date:** [May 10, 2021, 9:27pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/6 "2021-05-10T21:27:40Z")

</div>

Sorry this keeps happening; I know it’s frustrating.

whitenoise suggestion to binary search your plugins is a good one.  
I know you’ve eliminated your backup script to your satisfaction, but it might be worth trying to run without that as well.

Running `lsof | grep Path/To/vault` and log the results periodically with a little bash script might be good to try as well. I know you think there are no other processes using your vault files, but it would be good to verify.

> Where is the logic in whatever routine sees a 2.14 KB file suddenly become 0 bytes and decide “yeah that must be right, let’s keep the 0 byte version”. Without any notice/log/user confirmation? It makes no sense to me.

Maybe you are not sharing your vault via Dropbox or another Sync service, but what you are describing is not an error case in general. I share my vault among multiple devices and if I zeroed out a file on my laptop, I’d expect that to be reflected on my iPad too without a nag screen.

It’s overwhelmingly likely this is something specific to your setup.  
It’s likely small comfort, but I’d be shocked if this was an issue with Obsidian; this is kind of an outlier report, even less than the panics you get. If all else fails, could try running `dtruss` against obsidian to verify filesystem activity and satisfy yourself that it isn’t over-writing files.

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 11, 2021, 1:14am UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/7 "2021-05-11T01:14:34Z")

</div>

Thanks @pmbauer for trying to help. Really appreciate it. The dude abides! 🎳

I agree that in general, a file that’s being synced among a group of machines going to zero bytes isn’t necessarily an error condition–if there was intent. I will continue the hunt. I am storing this vault on iCloud Drive, but I believe that’s a fairly common setup. Good suggestions on lsof, I will see if that turns up anything.

I also see that 0.12.3 just got released with a bump to Electron 12. So, will re-test all around on the off chance this was some weird Electron bug.

---

<div class="post-metadata">

**Author:** ![pmbauer](https://forum.obsidian.md/user_avatar/forum.obsidian.md/pmbauer/32/10201_2.png) [@pmbauer](https://forum.obsidian.md/u/pmbauer)\
**Post date:** [May 12, 2021, 5:56am UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/8 "2021-05-12T05:56:16Z")

</div>

curious if this might be relevant

> [@iCloud Drive files being "optimized" on Mac and need to be re-downloaded from cloud](https://forum.obsidian.md/t/icloud-drive-files-being-optimized-on-mac-and-need-to-be-re-downloaded-from-cloud/17942):
>
> A week ago or so I moved my vault to iCloud Drive in order to sync with the iOS mobile app. This morning when searching my vault I was not turning up files I knew were in there. I opened the finder and saw that many markdown files in my vault had the little “download” icon, suggesting that they had been deleted locally and needed to be downloaded from the cloud again. This is frustrating because these files were created on the Mac and the hard drive has ample space, so they should not have been …

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 12, 2021, 1:50pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/9 "2021-05-12T13:50:41Z")

</div>

@pmbauer Thanks. True, I’d considered that as well. But, I don’t use that “optimize” feature:

![image](https://forum.obsidian.md/uploads/default/original/2X/d/d11c53d1c4b70ba7ff71cf44ffc09fe7fac4dabf.png)

Not to say there might not be a bug on Apple’s end (I’ve had my fair share of iCloud problems over the years). When I check right now, all of my files say “Downloaded” though.

![image](https://forum.obsidian.md/uploads/default/original/2X/b/bb019a54df57f780a673f9930b87a99c30ee1333.png)

I also have a bash script that periodically checks the iCloud database (directly via SQLite) for “stuck” items (you can see these with `brctl status` also) and reports those errors to me. So I try to stay on top of iCloud shenanigans.

Bigger picture I’m thinking as my vault gets larger and more important, that my best move will be switching to Obsidian Sync and/or a git repo combined with a periodic 1-way sync up to iCloud to be able to view on mobile.

---

<div class="post-metadata">

**Author:** ![GLight](https://forum.obsidian.md/user_avatar/forum.obsidian.md/glight/32/4851_2.png) [@GLight](https://forum.obsidian.md/u/GLight)\
**Post date:** [May 12, 2021, 3:31pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/10 "2021-05-12T15:31:15Z")

</div>

Your experience is a little bit frightening to me. Because I use a script to do the backup: copy the entire vault to a different location and then archive the vault. It is similar to your approach.

Is this method safe?

---

<div class="post-metadata">

**Author:** ![WhiteNoise](https://forum.obsidian.md/user_avatar/forum.obsidian.md/whitenoise/32/11_2.png) [@WhiteNoise](https://forum.obsidian.md/u/WhiteNoise)\
**Post date:** [May 12, 2021, 5:33pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/11 "2021-05-12T17:33:58Z")

</div>

@luckman212 Does the file become empty while still existing or is the file deleted and recreated as an empty (when you click on wikilink)?

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 12, 2021, 5:44pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/12 "2021-05-12T17:44:37Z")

</div>

@WhiteNoise The file becomes empty. It is not being deleted. I confirmed this by looking at the file creation (inode-born) timestamp with `stat`, see below. Also, I am sure that when this has happened, that I’ve just been navigating via the File pane on the left, or through search– not clicking into a Wikilink.

```nohighlight
$ stat -Lf 'modified: %Sm%n created: %SB%n' "Intune.md"
modified: May 11 21:39:46 2021
 created: May 10 12:46:16 2021 <===

```

---

<div class="post-metadata">

**Author:** ![akaalias](https://forum.obsidian.md/user_avatar/forum.obsidian.md/akaalias/32/2109_2.png) [@akaalias](https://forum.obsidian.md/u/akaalias)\
**Post date:** [May 21, 2021, 10:04am UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/13 "2021-05-21T10:04:23Z")

</div>

Hi all, I think I’m having the same issue and I’ve had this happen about 10 times in the last 3 months – MacOS, Big Sur, v0.12.3

To quote @luckman212:

> I don’t have 100% repro steps for this nor do I think I will be able to produce them. I’m more than willing to put in some time and do any sort of testing but I can’t sit here and bang the keys like a monkey until this happens. All I know is it _ **is** _ happening and it’s very unnerving for a “second brain” app to be vaporizing information like this.

> In the screenshot below from a few minutes ago, you can see the history of the snapshots as I was working in this note, every 2-3 min. All of a sudden _poof_ there’s a 0 byte version saved.

Here’s my screenshot:

 ![Screenshot 2021-05-21 at 12.02.19](https://forum.obsidian.md/uploads/default/original/2X/1/1cf43c4ba13b4afbeea643ea1e3ece1f5ab4a006.jpeg)

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 21, 2021, 11:40am UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/14 "2021-05-21T11:40:34Z")

</div>

@akaalias Are you storing your vault on iCloud Drive? There may be a pattern emerging…see [Text disappearing from notes](https://forum.obsidian.md/t/text-disappearing-from-notes/18469/)

---

<div class="post-metadata">

**Author:** ![akaalias](https://forum.obsidian.md/user_avatar/forum.obsidian.md/akaalias/32/2109_2.png) [@akaalias](https://forum.obsidian.md/u/akaalias)\
**Post date:** [May 21, 2021, 2:50pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/15 "2021-05-21T14:50:50Z")

</div>

Good question, actually Yes, I AM storing my vault on my iCloud Drive!

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 21, 2021, 5:17pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/16 "2021-05-21T17:17:07Z")

</div>

Thanks. I’m keeping a close eye on this. Recently updated to 11.4 and have not had the data loss in a little while. Still storing my main vault on iCloud. I have my recovery snapshot interval set at 2min. Also doing hourly backups to a separate gzipped folder, and have a Bash daemon that checks for 0byte files every 10 minutes and pops up an alert if any are found.

Good to know that so far it seems to be isolated to iCloud. Anyone else who is experiencing this please post your setup.

---

<div class="post-metadata">

**Author:** ![WhiteNoise](https://forum.obsidian.md/user_avatar/forum.obsidian.md/whitenoise/32/11_2.png) [@WhiteNoise](https://forum.obsidian.md/u/WhiteNoise)\
**Post date:** [May 21, 2021, 6:43pm UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/17 "2021-05-21T18:43:40Z")

</div>

If this happens and you have obsidian open, please open the console and see if there are errors.

---

<div class="post-metadata">

**Author:** ![pmbauer](https://forum.obsidian.md/user_avatar/forum.obsidian.md/pmbauer/32/10201_2.png) [@pmbauer](https://forum.obsidian.md/u/pmbauer)\
**Post date:** [May 22, 2021, 2:27am UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/18 "2021-05-22T02:27:29Z")

</div>

Have you considered not using iCloud?

---

<div class="post-metadata">

**Author:** ![luckman212](https://forum.obsidian.md/user_avatar/forum.obsidian.md/luckman212/32/3445_2.png) [@luckman212](https://forum.obsidian.md/u/luckman212)\
**Post date:** [May 22, 2021, 3:04am UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/19 "2021-05-22T03:04:32Z")

</div>

I have. See [comment above](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/9).

But, there are some nice things about having the vault on iCloud, especially if you already pay for additional storage there as I do. I’m interested in getting to the bottom of this bug.

My plan now is to try to use Hammerspoon’s `hs.pathwatcher()` function to set up a realtime alert if an outside process modifies a file in the vault. If that doesn’t work, I was also looking at trying to fork [Sinter](https://github.com/trailofbits/sinter) and modify it to track filesystem changes to the vault in realtime using Apple’s Endpoint Security Framework.

---

<div class="post-metadata">

**Author:** ![pmbauer](https://forum.obsidian.md/user_avatar/forum.obsidian.md/pmbauer/32/10201_2.png) [@pmbauer](https://forum.obsidian.md/u/pmbauer)\
**Post date:** [May 22, 2021, 4:31am UTC](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953/20 "2021-05-22T04:31:34Z")

</div>

> **[Brendan's blog » Top 10 DTrace scripts for Mac OS X](http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scripts-for-mac-os-x/)**

[Next page](https://forum.obsidian.md/t/data-loss-0-byte-files-revisited/17953.md?page=2)
