Obsidian Sync incorrectly duplicates sections of files

Bug reports
,
1

Steps to reproduce

  1. Create and modify a file repeatedly on desktop.
  2. Sync occasionally with iPhone.
  3. Eventually, the sync will take longer than usual and an incorrect sync will occur.

Did you follow the troubleshooting guide? [Y/N]

I cannot reproduce the issue in a sandbox vault as it’s related to Sync, and it only happens occasionally. However, I am not using any plugins or themes, nor are plugins enabled on desktop or mobile. Additionally, I’ve moved my vault out of the C:\ drive entirely in order to preempt any issues, as requested by support. I’ve also ensured that on my iPhone, the vault is stored on the device, not on iCloud.

Expected result

Sync will simply update the iPhone version to the version as modified on my desktop.

Actual result

Sync will create changes that never happened on the iPhone and add them to the file.

Environment

SYSTEM INFO:
Obsidian version: v1.7.7
Installer version: v1.7.5
Operating system: Windows 11 Education 10.0.22631
Login status: logged in
Language: en
Catalyst license: none
Insider build toggle: off
Live preview: on
Base theme: adapt to system
Community theme: none
Snippets enabled: 0
Restricted mode: on

RECOMMENDATIONS:
none

SYSTEM INFO:
Operating system: ios 18.2.1 (Apple iPhone17,2)
Obsidian version: 1.7.7 (172)
API version: v1.7.7
Login status: logged in
Language: en
Catalyst license: none
Live preview: on
Base theme: adapt to system
Community theme: none
Snippets enabled: 0
Restricted mode: on

RECOMMENDATIONS:
none

Additional information

I’m having an issue where once every week or so, notes I modify on only one device fail to sync properly and have sections duplicated. It’s similar to what’s happening in threads like this, but unlike the users in those threads, I’m not using any plugins or daily note features.

For example, a file is modified on my desktop computer. When I open Obsidian on my iPhone, even though the file has not been modified (or even opened) on that device, and sync had successfully completed on that device before the app was closed (I always wait for the green check mark before leaving the app or turning my screen off), a section of the file around what I added on my desktop is duplicated. Originally it seemed to mostly happen on notes that have multiple headers where I add or modify text at headers near the top of the file when there are headers and content below, but I’ve since had it happen in very simple files.

A simple example would be below. Note that I’ve created this as an example of what happens, and this is not an actual note.

File modified on desktop:

Lorem ipsum odor amet, consectetuer adipiscing elit. Libero eleifend justo consectetur nisl magna fermentum.
Congue tempus urna rhoncus luctus purus nec.
Mattis habitasse nascetur nam cubilia fermentum non sed sociosqu.
Aenean dis lobortis arcu rhoncus; sit vel est dis.
Auctor venenatis posuere montes ex et praesent lacus suscipit.
Tortor phasellus hendrerit et fames sodales interdum.
Ac nascetur himenaeos fringilla hendrerit purus semper interdum.

Nostra hendrerit ac senectus vel convallis massa ultricies.
Tristique lectus iaculis vivamus id ut sapien blandit lacus dignissim.
Natoque tristique feugiat orci habitasse maecenas.
Penatibus rhoncus torquent adipiscing velit morbi rhoncus.
Vel ullamcorper eleifend platea quam non natoque.
Vel scelerisque praesent pharetra varius facilisi auctor.

File after sync from phone (no edits performed on the phone; Sync creates this by itself)

Lorem ipsum odor amet, consectetuer adipiscing elit. Libero eleifend justo consectetur nisl magna fermentum.
Congue tempus urna rhoncus luctus purus nec.
Mattis habitasse nascetur nam cubilia fermentum non sed sociosqu.
Aenean dis lobortis arcu rhoncus; sit vel est dis.
Auctor venenatis posuere montes ex et praesent lacus suscipit.
Tortor phasellus hendrerit et fames sodales interdum.
Ac nascetur himenaeos fringilla hendrerit purus semper interdum.

Nostra hendrerit ac senectus vel convallis massa ultricies.
Tristique lectus iaculis vivamus id ut sapien blandit lacus dignissim.
Ac nascetur himenaeos fringilla hendrerit purus semper interdum.

Nostra hendrerit ac senectus vel convallis massa ultricies.
Tristique lectus iaculis vivamus id ut sapien blandit lacus dignissim.
Natoque tristique feugiat orci habitasse maecenas.
Penatibus rhoncus torquent adipiscing velit morbi rhoncus.
Vel ullamcorper eleifend platea quam non natoque.
Vel scelerisque praesent pharetra varius facilisi auctor.

Note that in the above example, the section starting with “Ac nascetur” and ending with the line starting with “Tristique lectus” have been duplicated.

Another issue happens where lines that have been modified exclusively on my desktop will sync lines from a previous version after syncing with my iPhone. To be clear, this file was definitely not modified on the iPhone. This is an actual example.

The file as modified on my desktop, before the incorrect sync:

screenshot 1

The file after sync with my iPhone, showing the lines Sync inserted:

screenshot 2

The changes that Sync inserted in Show Changes:

screenshot 3

I’ve reached out to support about this and I was asked to create this bug report. Additional screenshots of files I don’t want publicly shared are available to QA via Obsidian Support ticket 89488.

2 Likes
2

This is a very intriguing case.

On your phone, are you sure you didn’t select icloud as storage location (and then you enabled sync)?

3

Yes, I’m sure. When the vault was initialized on my iPhone it was not, nor was it ever, stored on iCloud.

IMG_0348
IMG_03481320×2868 98.9 KB

4

Have you ever received “merge conflict” notifications?
Are there copies of these notes in file recovery around the time this happens?

5

No, that was the first thing I checked when this first happened. I assumed it was some sort of merge conflict and that it would show as such in the sync logs. However, I’ve never been alerted to a merge conflict, nor have I ever found one in the sync logs. The logs just show versions being uploaded that are incorrect, as if the changes were made by me when they weren’t.

I also have nothing in file recovery. I just resolve the issues when they happen through restoring the correct version via Version History.

6

I’m experiencing synchronization issues between my Mac and iPhone that match the behavior described in this thread

Specific Problems

  1. Content duplication in files, particularly with tables
  2. File content gets mixed up, with sections being repeated
    • Example: After page 360, content from page 351 gets repeated with headings in the attached file
  3. Merging files during sync appears to be malfunctioning

17.President .md (42.8 KB)

7

@serious Thanks for opening this topic!

I’ve also been having this problem (duplicated content in notes after sync) since at least 2024-03, but I’ve hesitated to post about it because it seemingly happens randomly and I haven’t been able to pin down the circumstances when it happens.

My config:

  • I use Obsidian on the Mac (daily) and on iPad (ca. 1–2 times/week) and on iPhone (1–2 times/month).
  • I use Obsidian Sync. Never used iCloud or any other sync solution.
  • I rarely edit notes on my iPad or iPhone, for fear of triggering sync conflicts which then may go undetected. It’s really not a great experience.
  • 2 fairly large vaults (~7k and ~3k notes, respectively). I have only noticed the duplicated content after sync in my primary vault (7k notes), though it’s possible they also happen in the other vault but I don’t notice them.
  • When I open Obsidian on iPhone or iPad, I always take care to let it fully sync (green checkmark). Also, “Sync > Prevent device sleep” is enabled.
  • I have never once seen a merge conflict notification.
  • I have never once seen a file in the File recovery dialog (though I don’t look often).
  • I do use several plugins, e.g. Blockquote Levels, Code Editor Shortcuts, Dangling links, DataView, Heading Shifter, Linter, Templater.
  • So far, I haven’t noticed any deleted content caused by a sync/merge conflict, only duplicated content. I suppose it’s possible deletions did happen and I didn’t notice them.
  • Since 2024-03-28 (when I started keeping track), I have noticed duplicated content in one or more files (up to 6 files) on 11 separate occasions. It’s likely (I think) there’ve been more, but those are the ones I noticed. When I notice such duplicated content (sometimes weeks/months after it happened), I trace back the day it happened through my vaults’ Git history (which I keep locally on my Mac, not for syncing), but I haven’t been able to deduce a pattern. I tend not to use the version history view in Obsidian, though maybe I should start.
8

+1 I have also been having this problem.

  • Mac Obsidian is typically always open.
  • Markdown files are edited in VSCode typically.
  • iPhone is used numerous times daily

Whole sections are being duplicated.

I have also observed content be removed in some merges. Unsure if that’s related.

9

Could you provide your order of operations for when you use the Mac, vscode, and iPhone? A general “My workflow in a day” overview.

10

This might be playing into it, because the longer the files go unsynced between the devices, the greater the chance to have variations.

  • I have never once seen a merge conflict notification.

Can you also check in the filtered messages of the sync log?

  • So far, I haven’t noticed any deleted content caused by a sync/merge conflict, only duplicated content. I suppose it’s possible deletions did happen and I didn’t notice them.

Is it always a type of duplicated content (headings, lists), or even random paragraphs being merged?

11

I have been experiencing the same problem. Not using iCloud and I have never once seen a merge conflict notification either. It doesn’t happen enough to usually be a concern but it removed text that I had added to a file and had to restore from a version history copy.

My workflow is desktop open most of the day and then use the iPhone app randomly throughout the day. To avoid sync errors I had been waiting for sync to finish on the iPhone before I make any adjustments but it still randomly happens.

12

Similar to my question above; are you using any community plugins?

13

Majority of my edits are made using vscode. Obsidian is open in the background syncing. I almost never use Obsidian as the editor on my Mac.

When I’m away from my computer I use my iPhone to make quick edits and update checklists. Very rarely do I modify large chunks of text. it is usually just single line changes here and there.

Even though these changes are somewhat minimal I can find some massive duplications which may be dependent on headings.

It is rather worrisome and I feel like I could be having information unintentionally deleted when these events occur which seem to be increasing in frequency.