Data loss - 0 byte files (revisited)

[akaalias]

Hi all, I think I’m having the same issue and I’ve had this happen about 10 times in the last 3 months – MacOS, Big Sur, v0.12.3

To quote @luckman212:

I don’t have 100% repro steps for this nor do I think I will be able to produce them. I’m more than willing to put in some time and do any sort of testing but I can’t sit here and bang the keys like a monkey until this happens. All I know is it is happening and it’s very unnerving for a “second brain” app to be vaporizing information like this.

In the screenshot below from a few minutes ago, you can see the history of the snapshots as I was working in this note, every 2-3 min. All of a sudden poof there’s a 0 byte version saved.

Here’s my screenshot:

[luckman212]

@akaalias Are you storing your vault on iCloud Drive? There may be a pattern emerging…see Text disappearing from notes

[akaalias]

Good question, actually Yes, I AM storing my vault on my iCloud Drive!

[luckman212]

Thanks. I’m keeping a close eye on this. Recently updated to 11.4 and have not had the data loss in a little while. Still storing my main vault on iCloud. I have my recovery snapshot interval set at 2min. Also doing hourly backups to a separate gzipped folder, and have a Bash daemon that checks for 0byte files every 10 minutes and pops up an alert if any are found.

Good to know that so far it seems to be isolated to iCloud. Anyone else who is experiencing this please post your setup.

[WhiteNoise]

If this happens and you have obsidian open, please open the console and see if there are errors.

[pmbauer]

Have you considered not using iCloud?

[luckman212]

I have. See comment above.

But, there are some nice things about having the vault on iCloud, especially if you already pay for additional storage there as I do. I’m interested in getting to the bottom of this bug.

My plan now is to try to use Hammerspoon’s hs.pathwatcher() function to set up a realtime alert if an outside process modifies a file in the vault. If that doesn’t work, I was also looking at trying to fork Sinter and modify it to track filesystem changes to the vault in realtime using Apple’s Endpoint Security Framework.

[pmbauer]

Brendan's blog » Top 10 DTrace scripts for Mac OS X

[luckman212]

Found this relevant-looking tidbit today:

Michael Tsai - Blog - iCloud Drive Changing Empty Files

Really seems like this is an iCloud issue. Maybe the way Obsidian saves or updates the files is triggering an edge case - that I don’t know. Are atomic file writes being used? If not, maybe something like this could help: write-file-atomic - npm

Is Obsidian Sync the only way to view vault content with the iOS app aside from storing the vault on iCloud? Just trying to understand what the options are.

[pmbauer]

@luckman212

Is Obsidian Sync the only way to view vault content with the iOS app aside from storing the vault on iCloud? Just trying

Per the help docs, yes. See section on sync solutions
https://help.obsidian.md/Obsidian/Obsidian+Mobile#Obsidian+for+iOS

I have heard of a few using WorkingCopy to locally sync a vault, but haven’t tried it myself.

But if you meant view as in access (Rather than sync, unclear), then you can use the files app on a local, non-iCloud vault, use iaWriter, etc

[ArloZ]

This has happened to me, too! On MacOS, Big Sur, v0.12.3, with my vault in the special Obsidian folder (generated by the mobile app) in iCloud. I had an issue with note deletion or reversion to a previous save of the note that came up every few minutes until I disconnected and reconnected iCloud on my MacBook. Now I have noticed it happening only 2-3 times in the past few days, in a much less intrusive way where the most data that is lost is only around the last sentence I wrote. Made a post about it here: Note sometimes does not save.

When this bug has come up for me 1) Obsidian won’t allow me to switch to another note from the current note, 2) when I quit Obsidian and reopen the note it has either reverted to a previous save (aka a few lines are lost) or been entirely deleted (this has happened very rarely and only before I re-established iCloud on my MacBook — and weirdly enough seems to only have happened when I select the entire note and Command + C copy it in order to paste the content of the note once I restart Obsidian to restore it).

Very strange bug! It does seem to have something to do with the way iCloud syncs — perhaps the Michael Tsai blog that @luckman212 linked to is onto something.

[WhiteNoise]

I am not sure the two issues are connected, because you are reporting two different behaviours.

[luckman212]

I threw in the towel today.

After some success setting up a complex monitoring system consisting of a LaunchAgent that automatically starts an instance of FileMonitor targeting my vault folder whenever Obsidian is launched (tracking all file activity and parent process with millisecond accuracy) plus a Hammerspoon pathwatcher that automatically logs and pops up alerts in realtime if any 0 byte files are detected in the vault… I thought I was finally on course to track this down.

And then Apple’s endless obfuscation bit me in the rear again: seems that files modified in iCloud on other devices (phone, another laptop etc) trigger file changes in a way that bypasses traditional logging methods. I posted a github issue here and a crosspost on StackExchange as well.

I moved my vault out of iCloud and am just syncing with Syncthing for now. Will probably switch to Obsidian Sync when the mobile app exits beta.

TL;DR— At this point I personally consider iCloud unusable for any important data storage.

[WhiteNoise]

thanks for the effort you are putting into figure this out.

[atiz]

I’m following this thread because I also store my vault in iCloud and this is very worrying.

FWIW, I have not had problems, and have been using Obsidian since late last summer (all the time on iCloud). But I only have 1 computer now, so the only other thing I occasionally open my vault with is 1Writer on the iPad. Not sure if that makes a difference.

[luckman212]

@atiz This bug is an edge case for sure, so it may not affect everyone. Still trying to figure out how to trigger it. If iCloud is working for you, consider yourself lucky and you don’t necessarily need to stop using it.

As a precaution, you may want to run this command from a Terminal to be sure you have no 0-byte files lurking in there that may have gone unnoticed:

find "$HOME/Library/Mobile Documents" -type f -size 0

[luckman212]

@Licat thanks for your help on Discord the other day. I’m going to try to set up a simple Node script as you suggest to see if the behavior is the same as I’m seeing from Obsidian.

In the meantime, I did skim over the docs for fs_fspromises_writefile and I saw this:

It is unsafe to use filehandle.write() multiple times on the same file without waiting for the promise to be resolved (or rejected). For this scenario, use fs.createWriteStream().

I almost hate to ask you, but I assume you’re aware of this and it’s not part of this issue.

Another thought: could a 3rd party plugin that modifies the buffer contents possibly be triggering this to happen, e.g. if the timing was “just right” such that Obsidian was executing it’s periodic scheduled save while the user simultaneously triggered a file-altering action such as a Templater script?

[Licat]

I am aware of that note, as I’ve seen when we discussed it over on Discord. However, I believe the vast majority of nodejs and electron apps out there uses writeFile instead of managing their own file handles and write streams. Obsidian’s fs access is serialized; any attempt to write would have to wait for the previous write to finish, so it should be impossible to call write multiple times without having the promise resolve first.

Another thought: could a 3rd party plugin that modifies the buffer contents possibly be triggering this to happen, e.g. if the timing was “just right” such that Obsidian was executing it’s periodic scheduled save while the user simultaneously triggered a file-altering action such as a Templater script?

This should be safe as long as the plugin uses Obsidian’s file system adapter, or the vault API, instead of directly accessing the through the node.js fs package. As far as I know, most plugins don’t use the fs package because it’s not compatible with mobile.

[WhiteNoise]

From a possibly related issue:

Obsdian does not save and get stuck in a limbo state

We are not sure exactly what’s going on here. We believe that on some mac systems node.fs and osx do not play well and the writing queue gets stuck. This is not an Obsdian’s problem but a deeper/lower level issue. In 0.12.5, Licat introduced a periodic safety check on the writing queue and if there is no progress it attemps to reinitialized it.