#2

The way the sync algo is designed, what you are describing can only happen if a new file is simultaneously created and not synced on two devices. Since the file was already created and synced, this should not happen.

Are you sure about the repro step you provided?

#3

Looking at the version history, I’ve heard of one more case of this happening.

I feel like this is due to a race condition between the editor saving and sync downloading the file. If that’s the case, it would mean that you won’t be able to reproduce the issue consistently. In fact, given modern file systems are pretty fast, this should be rarely reproducible.

However, this is a serious problem and I will try to brainstorm ways to fix this.

More info I’d like to request, if you can give it a try - based on your repro steps, how reliably can you reproduce the issue? Is it like a 5/5 reproducible, or 1/5, or is it 0/5 (completely not reproducible, so it must be rare)? Knowing this would help me better predict the causes of failure.

Thanks!

#4

Hi @Licat , thanks for the explanation. I’d say it’s definitely not 100% consistently reproducible, but a maybe 3/5 when the iPhone was on Cellular data. I have encountered this issue multiple times (>3 in the past few weeks), with most of them happened when I go outside to the mailroom and the phone switches to 5G.

And about your theory of editor saving and syncing the same file, I doubt that’s the case. As you mentioned the filesystem write should be instantaneous, so this should be way less than 1/5 reproducible (probably even 1/10000), which is not the case for me. For me it feels more like a problem of determining the newer version between server-side 2pm file and client-side 1pm file plus some additional 3pm updates, but don’t quote me on that, as I have no idea how the Sync works.

And @WhiteNoise, no I’m not creating new files, this happens to the daily note, which is always created early in the morning, way before any of override happens.

Is there anything specific that I can do to help debug this issue? I can probably pull up the sync version history with file content for the last few cases of this. For daily notes, I only append lines to the end of the file, so negative file size update normally won’t happen, unless this issue is triggered.

Here’s another case, which I explicitly wrote about the data loss in the daily note, thus easy to find through search. There’re definitely more, but takes time to locate.

Screen Shot 2022-02-25 at 9.33.14 PM
Screen Shot 2022-02-25 at 9.33.14 PM1920×1427 167 KB

#5

If you’re able to reproduce this issue on purpose, please do that and show me the sync log on the iPhone (which is the device that overwrites the file). I’m looking for everything in the sync log about that specific file.

The sync log can be accessed from Settings > Sync > Sync activity > View.

#6

Sure, but reproducing it on purpose might be difficult, as I’m not 100% sure of the cause / condition for this issue to happen. Meanwhile @Licat can I get a build of Sync that does not clear the sync log on exit (iOS return to home)? Most of the time when I realize the data loss, I’ve already open / quit the app multiple times.

Just tried to intentionally cutting off the internet with Airplane Mode, but Obsidian Sync seems to handle the merge just fine. And in this case there is a notice of two versions being merged: “modified externally, merging changes automatically.” I did not see that notice when the override happens.

#7

The reason I ask is because while you’ve only observed the overwrite a couple of times, it is difficult to actually count how many possible occurrences that did merge properly and did not cause an overwrite issue, to properly calculate the ratio. Here, I want to get an idea of how often you can reproduce the issue specifically when attempting to reproduce on purpose.

Meanwhile @Licat can I get a build of Sync that does not clear the sync log on exit (iOS return to home)? Most of the time when I realize the data loss, I’ve already open / quit the app multiple times.

Unfortunately the sync log is only saved in memory and so when the app closes it is lost. Currently I don’t plan on persisting the sync log to disk for performance reasons and to avoid accumulating disk usage.

I will continue to investigate and attempt to add more safety checks. In the meantime if you happen to spot the issue happening in front of you, please go and make a copy of the sync log and send it here.

1 Like
#8

This happened to me (again) just now.

Sync log shows 3 saves to my “Shopping” file today (at 22:42, 22:43 and 22:43) on my Windows device. Then 40 min later this was overwritten by my Android device with the previous version from yesterday.

I’ve saved the sync log to a .txt file and am happy to share it privately – how do I do that?

#9

DM them to licat on discord or here.

What exactly is you sequence of operation?

#10

Do I need to be on a particular server? Can’t find you.

Not quite sure what you mean, beyond what I posted, sorry.

#11

when was the file created? on which device was the file edited and when? when where the changes synced?

1 Like
#12

The file was created many months ago and has been edited many, many times on at 5 devices (probably just 2 devices in the last few weeks)… the recent changes will show in the log file that I will send.

The problematic changes are those described above:

Sync log shows 3 saves to my “Shopping” file today (at 22:42, 22:43 and 22:43) on my Windows device. Then 40 min later this was overwritten by my Android device with the previous version from yesterday.

#13

This is absolutely not my domain, but FWIW…

  • At this stage of development, good log files must be very valuable. Solving serious issues like this one is probably worth a very slight performance hit.
  • The app is already saving and updating text files constantly.
  • This would be one more text file.
  • Maybe they could be saved in weekly log files, and automatically deleted when they’re more than 2 weeks old. So they never become big.
  • There could be a switch to turn off the saving of log files, for people with older phones.

To do all of that might be a lot of work :man_shrugging: so I’m just putting it out there.

Also, including a diff check tool (or creating a one-click way to export sync versions to a diff check tool) would also help to identify sync bugs and fix any data loss. I would value this a lot. I’m not sure how much work that would take.

Btw, I can’t express how valuable Obsidian is to me :heart:. My suggestions are in the spirit of wanting it to be something I can safely rely on even more, and appreciating how awesome it already is.

1 Like
#14

A change was introduced in 0.13.27 that may affect this.

1 Like
#15

@Licat Happened again with Desktop 0.13.28 and Mobile 1.1.0(38). But this time I realized the data loss while writing on the iOS app and managed to save the Sync Log.

The file involved in the overwrite: log/daily/note/2022/03/2022-03-01.md

Sync Version History

The latest 1.69KB version from iPhone (containing one appended line) was not based on the previous version (1.86KB at 10:03PM), but based on an earlier one (1.64KB at 6:22PM). And Obsidian Sync did not realize the conflict, which essentially ignored the two updates from 8:58PM and 10:03PM.

image
image920×1488 205 KB

Sync Log on iPhone:

Highlighted the log lines related to that file. No errors are reported in the log, and I didn’t get the “modified externally, merging changes automatically” notice either.

 2022-03-01 18:51 - Fully synced
 2022-03-01 22:59 - Connecting to server
 2022-03-01 22:59 - Connection successful. Detecting changes...
 2022-03-01 22:59 - Server pushed [LER0ever-MacMini-2.local] .obsidian/plugins/metadata-extractor/tags.json
 2022-03-01 22:59 - Server pushed [LER0ever-MacMini-2.local] .obsidian/plugins/metadata-extractor/allExceptMd.json
 2022-03-01 22:59 - Server pushed [LER0ever-MacMini-2.local] .obsidian/plugins/metadata-extractor/metadata.json
 2022-03-01 22:59 - Server pushed [LER0ever-MacMini-2.local] .obsidian/appearance.json
+2022-03-01 22:59 - Server pushed [LER0ever-MacMini-2.local] log/daily/note/2022/03/2022-03-01.md
 2022-03-01 22:59 - Downloading file .obsidian/plugins/metadata-extractor/tags.json
 2022-03-01 22:59 - Downloading complete .obsidian/plugins/metadata-extractor/tags.json
 2022-03-01 22:59 - Accepted .obsidian/plugins/metadata-extractor/tags.json
 2022-03-01 22:59 - Downloading file .obsidian/plugins/metadata-extractor/allExceptMd.json
 2022-03-01 22:59 - Downloading complete .obsidian/plugins/metadata-extractor/allExceptMd.json
 2022-03-01 22:59 - Accepted .obsidian/plugins/metadata-extractor/allExceptMd.json
 2022-03-01 22:59 - Downloading file .obsidian/plugins/metadata-extractor/metadata.json
 2022-03-01 22:59 - Downloading complete .obsidian/plugins/metadata-extractor/metadata.json
 2022-03-01 22:59 - Accepted .obsidian/plugins/metadata-extractor/metadata.json
+2022-03-01 22:59 - Downloading file log/daily/note/2022/03/2022-03-01.md
+2022-03-01 22:59 - Downloading complete log/daily/note/2022/03/2022-03-01.md
+2022-03-01 22:59 - Accepted log/daily/note/2022/03/2022-03-01.md
+2022-03-01 22:59 - Uploading file log/daily/note/2022/03/2022-03-01.md
+2022-03-01 22:59 - Upload complete log/daily/note/2022/03/2022-03-01.md
+2022-03-01 22:59 - Server pushed [LER0ever-iPhone] log/daily/note/2022/03/2022-03-01.md
 2022-03-01 22:59 - Fully synced

Sync Log on Mac:

Looks like it just blindly accepted the server side version without any warnings.

2022-03-01 21:59 - Fully synced
2022-03-01 22:03 - Uploading file log/daily/note/2022/03/2022-03-01.md
2022-03-01 22:03 - Upload complete log/daily/note/2022/03/2022-03-01.md
2022-03-01 22:03 - Server pushed [LER0ever-MacMini-2.local] log/daily/note/2022/03/2022-03-01.md
2022-03-01 22:03 - Fully synced
2022-03-01 22:59 - Server pushed [LER0ever-iPhone] log/daily/note/2022/03/2022-03-01.md
2022-03-01 22:59 - Downloading file log/daily/note/2022/03/2022-03-01.md
2022-03-01 22:59 - Downloading complete log/daily/note/2022/03/2022-03-01.md
2022-03-01 22:59 - Accepted log/daily/note/2022/03/2022-03-01.md
2022-03-01 22:59 - Fully synced

Hope this helps with debugging the issue, and please let me know if you need any additional information. I still have no luck in reproducing it on purpose, probably because the condition required to trigger the issue is not what I thought before (version resolving / race condition). This time I had a perfect network connectivity, with both devices in door and connected to WiFi. The Obsidian (desktop) are updated to 0.13.28, so the attempted fix in 0.13.27 is in place, at least for the Mac.

#16

The fix wasn’t specific to this issue and it hasn’t been released for mobile yet.

It’s possible that some plugins or some other software on your devices is changing the file behind your back.

Run obsidian everywhere without plugins and let us know if it happens and think if you have any other software operating on these files (like another sync service).

#17

Our other report from @chriswaterguy mentioned he has the Calendar and Tasks plugin, both of which were installed in @rongyi 's original debug log as well.

Is it possible that one of them (I feel like it could be the Tasks plugin) that’s causing this behavior?

To explain a bit further, the log seen here is identical to the one posted by @chriswaterguy in behavior - Sync seems to download the file when it comes online, then very quickly upload a different version which is smaller.

I’ve gone back to audit the sync code and I don’t see any possibility of the code hanging on to an older version after it has downloaded the latest version. The only possibility here is if a plugin loaded the file in question before Sync downloaded the new version, and then wrote back the old version after Sync has completed the download, which causes the rollback.

1 Like
#18

This post was flagged by the community and is temporarily hidden.

#19

Thanks for the update @WhiteNoise @Licat

It’s possible that some plugins or some other software on your devices is changing the file behind your back. Run obsidian everywhere without plugins and let us know if it happens and think if you have any other software operating on these files (like another sync service).

I can rule out third party software changing the file, since this happens on iOS. And I’m not using any other sync service besides Obsidian Sync.

I probably cannot experiment with no plugins at all, but I can try to disable some of them.

Is it possible that one of them (I feel like it could be the Tasks plugin) that’s causing this behavior?

I’ll start a plugin bisect by disabling some of the plugins that I don’t actively use / need including Tasks.

  • Annotator
  • File Tree Alternative
  • Graph Analysis
  • Highlightr
  • Linter
  • Tasks

I’ve disabled the above plugins, and will report back if the overwrite happens again.

3 Likes
#20

Happened again today with the above plugins disabled… So it’s not caused by Tasks plugin. I believe I have some clue about what’s causing the issue, but have to double-check.

File: log/daily/note/2022/03/2022-03-03.md

Sync Version History

image
image982×1366 112 KB

Sync Log on iPhone

2022-03-03 11:54 - Downloading file .obsidian/plugins/metadata-extractor/allExceptMd.json
2022-03-03 11:54 - Downloading complete .obsidian/plugins/metadata-extractor/allExceptMd.json
2022-03-03 11:54 - Accepted .obsidian/plugins/metadata-extractor/allExceptMd.json
2022-03-03 11:54 - Downloading file .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:54 - Downloading complete .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:54 - Accepted .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:54 - Downloading file log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:54 - Downloading complete log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:54 - Accepted log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:54 - Fully synced
2022-03-03 11:55 - Uploading file log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:55 - Upload complete log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:55 - Server pushed [LER0ever-iPhone] log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:55 - Fully synced
2022-03-03 11:55 - Connecting to server
2022-03-03 11:55 - Connection successful. Detecting changes...
2022-03-03 11:55 - Fully synced

Sync Log on Mac

2022-03-03 11:53 - Server pushed [LER0ever-MacMini-2.local] .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:53 - Merging conflicted file .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:53 - Downloading file .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:53 - Downloading complete .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:53 - Accepted .obsidian/plugins/metadata-extractor/metadata.json
2022-03-03 11:54 - Uploading file log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:54 - Upload complete log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:54 - Server pushed [LER0ever-MacBookAir] log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:55 - Server pushed [LER0ever-iPhone] log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:55 - Downloading file log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:55 - Downloading complete log/daily/note/2022/03/2022-03-03.md
2022-03-03 11:55 - Accepted log/daily/note/2022/03/2022-03-03.md

Enabled Plugins

	Plugins installed: 28
	Plugins enabled: 19
		1: Advanced Tables
		2: Calendar
		3: Quick Switcher++
		4: Natural Language Dates
		5: Custom window title
		6: QuickAdd
		7: Admonition
		8: Excalidraw
		9: Minimal Theme Settings
		10: Advanced URI
		11: Metadata Extractor
		12: Dataview
		13: Diagrams
		14: Style Settings
		15: Open vault in VSCode
		16: Local images
		17: Contextual Typography
		18: Global Hotkeys
		19: Templater

@chriswaterguy do you have the enabled plugin list when this issue happens?

1 Like
#21

@Licat It is now 100% consistently reproducible in my case. Bug report filed to the corresponding plugin repository: [BUG] External file updates lost during Capture prompt · Issue #210 · chhoumann/quickadd · GitHub . Could really use some suggestions on how to fix that either on the Sync side or the plugin side.

In summary, it turns out a plugin (Quickadd) has a long running operation (prompt for user typing). It reads the file before that, and writes the updated file afterwards based on the version it read. The Obsidian Sync updates the same file during that prompt period.

this.app.vault.read(file)
const newFileContent: string = await this.formatter.formatContentWithFile(content, this.choice, fileContent, file);
    // the above function waits for user input. 
this.app.vault.modify(file, newFileContent);

I’m not sure what’s the best practice here, deferring the file read operation to after the user input finishes seems like a solution, but I don’t know much about Obsidian’s API / plugin dev.

1 Like