MacOS Corrupted sync, lost files

Steps to reproduce

  • Create a vault, add some content including some binary files, MP3s, images
  • Sync the vault to a second computer on a bad network where some files fail to download

You can’t use the sandbox to do this because it depends on having a bad network fail to sync some files fully

Expected result

Files that fail to sync should not be marked as deleted, and when selecting to use the remote file to fix the problem the files should not be reported as successflly downloaded when they are in fact corrupted.

Actual result

Files are marked as deleted - if you right click and instruct Obsidian to use the remote file to fix the problem they seem to concat the partially downloaded file: one MP3 I thankfully have a local copy of is 2.4Mb, after choosing to use the remote copy Obsidian has made it 4.1Mb - Obsidian however reports the download a success and marks the files as ‘changed’ - if you then publish you end up with a corrupt local and remote file losing your data.

Environment

Two MacOS computers on a poor quality rural network


Additional information

I’m a paying customer obviously. This is the second time this has happened, the first time I lost a large amount of photos from neolithic sites I’d bagged, it’ll be a lot of work to track down all those photos again and I’ve still not found the time to do it. This second time I’ve lost around a dozen MP3 field recordings.

They are not.

It would be helpful if you provided some sync logs of this happening from both computers involved. Thanks

This is the second time you’ve responded to a user with this issue by doubting the report (see: Sync deleting files inappropriately - #2 by aubreyz).

These MP3 files were absolutely marked as deleted, as were images on a previous occurence of this issue.

A 3rd user reports sync deleting files just an hour ago too: Obsidian Sync occasionally deletes daily notes

I understand your frustration but we are gonna need sync log to understand exactly what is going on.

Let me be clear, It’s possible that there is some problem, but your reconstruction of how it came to be is not compatible with how obsidian works.
So we need the sync logs from both devices to have a better sense of the sequence of events.

Are the logs stored anywhere?

This screenshot shows files deleted that 100% have not been deleted by me, if not me then there must be a bug with Obsidian sync. If I select ‘Use live version’ it will report a success, and then list the files as changed - in fact they will now be corrupted and unplayable. So there’s two problems.

That is not Obsidian Sync, that is Obsidian Publish!
I am confused.

There was a bug in Obsidian Publish (which is not sync solution!) while downloading attachments. It was fixed a week ago.

Yes, publish is showing deleted files which have failed to sync as described. I suspect my very slow internet connection is playing a part - you’re probably developing Obsidian under ideal conditions. We have extremely small bandwidth, upload struggles particularly, it’s a cellular broadband connection which is like going back in time 10 years.

No, the bug that we fixed on Publish wasn’t about bad connection. Anyway it’s fixed now so you shouldn’t have that problem.
If it happens again, open a new bug report and make it clear that you are using Obsidian Publish and not Obsidian Sync. They are two different things.

There is no synchronization in Obsidian Publish, you manually upload and download files.

I don’t agree this is publish - publish is showing the effects of the broken sync. Incidentally I’ve just noticed my custom publish.css has also disappeared and the site has reverted to the default LAF.

I’m going to try Syncthing instead which some people have recommended, and just use Publish from Obsidian, I’ve lost a lot of time to this, and fixing the lost data is going to be a big job.

We have two separate paid services

  1. Obsidian Sync for synchronizing your vault among devices
  2. Obsidian Publish for publishing your vault to a website.

They are completely separate and do no share any code or infrastructure.
Which one are you using?

Clearly both - sync to sync between computers and my phone, and publish to push to the website.

Well, it wasn’t clear to me at all, since you showed a screenshot from Obsidian Publish.

Again, publish is a publishing solution where you manually upload (and download, your screenshot) from the website. There’s no concept of version history or automatic sync. It is a completely separate product from obisidian sync and do not share any data.

Regarding Obsidian Sync, there is a sync history

The logs for the current session are visible by clicking the green sync icon in the lower right corner.

We also have a separate, local-only, file recovery system (separate from sync).

I fully understand what sync and upload are - and that they are separate, I do not understand where this continued rudeness is coming from. Once again - sync has caused issues, which can be seen when I click publish - it doesn’t matter that publish is showing that sync has caused the issue at all - the changed/deleted files showing in publish are a symptom of the issues caused by sync. Let’s finish this now, I was trying to help Obsidian with a lengthy bug report but you have been defensive to the point of being outright hostile from the very first interaction. I like Obsidian, and don’t want to end up rage-quitting over this - I came here in good faith despite losing data on a pretty expensive service and the interactions have been ridiculously unprofessional.

I am sorry I didn’t mean to be rude. I think this whole thread is giant misunderstanding. Feel free to open a new bug report if you have problems again and attach the sync logs.