Attachment File Loss on big initial sync with Obsidian Sync

Steps to reproduce

First sync from Macbook pro (intel) to Iphone using Obsidian sync mechanism (~15000 notes).

Some of the attachments got an “Out of memory” error in the sync log on Iphone. One of them, a 6mb PNG file, turned into a 0-byte file in Iphone and then got uploaded to the sync store thus zeroing the original PNG on the Mac.

I got the alarm from Devonthink which is also indexing that folder, that a file had the 0-byte anomaly, so I just reverted to a backup with Time Machine and all is good :slight_smile:

After that problem I checked the vault for other 0-byte files and didn’t find any. I also checked for modified files and found a new case, another big 6mb PNG file, got corrupted with half the photo showing up - son not 0-byte but cropped size. Fixed it with a backup.

Expected result

Same as the rest of “Out of memory errors” that didn’t suffer from thisissue, just re-download the file.

Actual result

It kept a 0-byte file, and in the other case the cropped size file, detected the change and uploaded it to the sync store essentially deleting the file’s contents in one case and corrupting the image in the other.

Environment

  • Operating system: MacOS 12.4, IOS 15.5
  • Obsidian 0.14.15 (Installer 0.12.19) on Mac / 1.2.0 (50) on IOS
  • Community Plugins on Ios: Advanced Movile Toolbar, minimal theme settings

Additional information

I think you should look into the code that manages the out of memory issue, there might be a condition where the logic thinks that the download was ok but actually the file was 0-byte long.

On the Mac the vault resides in an iCloud Drive folder (not Obsidian’s). Devonthink is indexing the same folder.
On IOS I selected local filesystem for the remote vault (not in iCloud).

I’m sorry I didn’t save the log :frowning: ! But If I remember correctly it was something like:

[Timestamp] - Downloading XYZ.PNG
[Timestamp] - Out of memory XYZ.PNG
(…)
[Timestamp] - Downloading XYZ.PNG
(…)
[Timestamp] - Uploading XYZ.PNG

The out of memory was on logs of the mac or in the logs of iphone?

6mb is not that big of file, considering that we limit up to 100MB.

If it happens again, kindly take a screenshot of the sync logs. Thanks! I don’t recall ever seeing this happening.

On the iPhone log

Of course, I thought the log would stay there for a while but there were so many files I think it got cycled.

I think licat has an idea of what is happening in your case. We’ll look into it.

Now I tested enabling PDF attachment sync, and got 19 0-byte attachments ! Damn. I have the logfile, where should I send it?

you can upload it and link it here or dm it to licat on discord.

Which model is your iphone?

Ok, iphone XS Max 256GB

Sent the log to litcat on discord :heavy_check_mark:

I’m hoping it’s appropriate to piggyback on this, rather than start a new post. I have a similar issue where my files are being deleted from my iPad, which only has 32 gbs. At least 90 files were deleted without me knowing…

Should I delete Obsidian from my iPad to avoid this happening? Or not sync files to the iPad? Some lost items were files but others were critical notes I managed to restore.

EDIT: I think because I can’t store files offline on the iPad, it’s deleting them from there, then reuploading them—sometimes—once I reopen the Obsidian folder.

Still nerveracking, but it helps to understand what’s happening.

You do not have the issue described on this BR for sure.
I am not sure what your problem is, maybe you have no issue at all and you just moved the files and now you have to wait until the sync propagates.

If you find clear steps of what is going on, please open a new bug report and follow the bug report template.

1 Like

Just encountered this bug while migrating to the new regional servers and doing a full re-sync on multiple devices. Mac and Windows re-syncs went smoothly.

On iOS:

  • OOM errors downloading a file from Obsidian Sync on iOS caused files to be truncated (all files were images or zip ~4-10 MB)
  • Obsidian then uploaded those truncated files back to the sync server
  • Obsidian Sync version history did not offer a restore button to roll back those files

The expected behavior:

  • an OOM error would cause the pending file write to abort, so no files would be truncated
  • barring that, if it had to write to disk, it should have appended some sort of warning text to the filename (e.g. my-file.jpg-CONFLICT)
  • at the very least, it should not decide to upload the failed downloads back to the sync server
1 Like

I’m getting OOM errors during syncing of files on the Apple Vision Pro installation of the iPad app.

When I stumbled on this thread, I ended the sync immediately as I was worried about data loss.

I’m not certain if it created any 0-byte files but going to avoid using obsidian sync until this issue is resolved.

I am experiencing the same issue OOM when syncing iPhone from a remote vault which was created on Mac OS. Lots of OOM errors when syncing JPG files (file sized vary in the range less than MB to 7 MB).
The files are still intact on the Mac OS, but I see that the sync has not ran yet here…
What shall I do? This is quite an issue :frowning:

Update: Once the initial sync is complete, it seems that Obsidian on iOS performed another sync, which managed to sync succesfully the image files which produced OOM errors. Checked the files on Mac - there are no corrupted (0-byte) files as a result of the sync process. So probably this is an issue just with the initial sync?

this will be better handled in v1.5.9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.