I am only using 1 client actively. All I do to repro this is open a new file, wait a second, modify it, then the file gets deleted. Though Iāve only done it in Obsidian by accident so far. Sounds crazy right? I am going to brain dump here as there are nearly 0 results for any of this on various search engines, and maybe someone else will have ideas.
This script trivially creates conflicted files if ran from ~/Library/Mobile Documents/com~apple~CloudDocs on OSX. It can take a few minutes. For my real use cases I wasnāt going crazy though trying to create races I was typically just doing things ānormalā.
#! /bin/sh
DATA="$(mktemp)"
DST="synctest"
# Just get some data. The contents are irrelevant.
ps uaxwwww > "${DATA}"
while [ ! -e "${DST} 2" ]; do
rm -f ${DST}*
sleep 3
touch "${DST}"
sleep 1
cat "${DATA}" >> "${DST}"
sleep 1
cat "${DATA}" >> "${DST}"
sleep 3
done
# Exits when "synctest 2" is created.
An odd thing is that the āsynctest 2ā file is actually a renamed āsynctestā file since the main file was redownloaded. This matches what I see in my Obsidian .trash folder. That the ā2ā files are the newer ones I expect. This is seen with a more verbose version that shows inodes.
Of course thatās not Obsidian, and it uses the wrong APIs. But itās an example and matches the steps Iāve taken that resulted in deleted files. Specifically for Obsidian when this sort of conflict occurs the file is straight up deleted into the trash, as well as the new conflicted download. In my example neither file is deleted when the script exits.
brctl is useful for monitoring this.
while :; do brctl status com.apple.CloudDocs; sleep 1; done
brctl monitor -i com.apple.CloudDocs
For Obsidian:
brctl monitor -i iCloud.md.obsidian
At the point where the problem happens brctl monitor will claim that the file was modified on the server, āserver edit to downloadā, and needs to be redownloaded. At another point I saw āgeneration counter mismatchā.
I am still looking into this and trying to come up with a concise bug report for Apple, assuming they care about the POSIX API I am using. There are some other oddities involved that I donāt fully understand yet. I am not any kind of Apple developer. I do BSD development for a living though. Right now I am wishing I was working at Apple and could truly debug this. There are maybe 2 hits on any search engine relating to any of the technical details that I can find.
TLDR: There is clearly some kind of race with modifying new files too quickly where the server side thinks there is an edit and causes the iCloud client to redownload into a conflicted state, and rolls back to an older version. Obsidian seems to delete the file in this case which is specific to Obsidian. I do suspect that Obsidian is not properly using the cloud APIs, or some plugin is doing something wrong.
more verbose version
#! /bin/sh
DATA="$(mktemp)"
DST="conflicted"
# Just get some data. The contents are irrelevant.
ps uaxwwww > "${DATA}"
i=0
showmeta() {
DST="$1"
echo "# stat -x ${DST}"
stat -x "${DST}"
echo "# xattr -l ${DST}"
xattr -l "${DST}"
echo
}
while [ ! -e "${DST} 2" ]; do
i=$((i + 1))
rm -f ${DST}*
sleep 3
touch "${DST}"
sleep 1
{
echo "$i" # add version into the file
cat "${DATA}"
} >> "${DST}"
showmeta "${DST}"
sleep 1
i=$((i + 1))
{
echo "$i" # add version into the file
cat "${DATA}"
} >> "${DST}"
showmeta "${DST}"
sleep 3
done
echo "Conflicted '${DST} 2' detected." >&2
echo "i= $i" >&2
showmeta "${DST}"
showmeta "${DST} 2"
And some output near the conflict:
# stat -x conflicted
File: "conflicted"
Size: 291726 FileType: Regular File
Mode: (0644/-rw-r--r--) Uid: (1087209050/ [REDACTED USERNAME]) Gid: (720748206/(720748206))
Device: 1,4 Inode: 4334562453 Links: 1
Access: Wed Nov 10 20:37:32 2021
Modify: Wed Nov 10 20:37:32 2021
Change: Wed Nov 10 20:37:32 2021
# xattr -l conflicted
# stat -x conflicted
File: "conflicted"
Size: 145863 FileType: Regular File
Mode: (0644/-rw-r--r--) Uid: (1087209050/ [REDACTED USERNAME]) Gid: (720748206/(720748206))
Device: 1,4 Inode: 4334562541 Links: 1
Access: Wed Nov 10 20:37:39 2021
Modify: Wed Nov 10 20:37:40 2021
Change: Wed Nov 10 20:37:40 2021
# xattr -l conflicted
# stat -x conflicted
File: "conflicted"
Size: 291726 FileType: Regular File
Mode: (0644/-rw-r--r--) Uid: (1087209050/ [REDACTED USERNAME]) Gid: (720748206/(720748206))
Device: 1,4 Inode: 4334562541 Links: 1
Access: Wed Nov 10 20:37:40 2021
Modify: Wed Nov 10 20:37:42 2021
Change: Wed Nov 10 20:37:42 2021
# xattr -l conflicted
Conflicted 'conflicted 2' detected.
# stat -x conflicted
File: "conflicted"
Size: 291726 FileType: Regular File
Mode: (0644/-rw-r--r--) Uid: (1087209050/ [REDACTED USERNAME]) Gid: (720748206/(720748206))
Device: 1,4 Inode: 4334562603 Links: 1
Access: Wed Nov 10 20:37:45 2021
Modify: Wed Nov 10 20:37:32 2021
Change: Wed Nov 10 20:37:45 2021
# xattr -l conflicted
com.apple.lastuseddate#PS:
00000000 0C 9E 8C 61 00 00 00 00 00 00 00 00 00 00 00 00 |...a............|
00000010
com.apple.metadata:_kMDItemUserTags:
00000000 62 70 6C 69 73 74 30 30 A0 08 00 00 00 00 00 00 |bplist00........|
00000010 01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 09 |..........|
0000002a
# stat -x conflicted 2
File: "conflicted 2"
Size: 291726 FileType: Regular File
Mode: (0644/-rw-r--r--) Uid: (1087209050/ [REDACTED USERNAME]) Gid: (720748206/(720748206))
Device: 1,4 Inode: 4334562541 Links: 1
Access: Wed Nov 10 20:37:44 2021
Modify: Wed Nov 10 20:37:42 2021
Change: Wed Nov 10 20:37:44 2021
# xattr -l conflicted 2
com.apple.metadata:_kMDItemUserTags:
00000000 62 70 6C 69 73 74 30 30 A0 08 00 00 00 00 00 00 |bplist00........|
00000010 01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 09 |..........|
0000002a
The xattrs are probably irrelevant but there is a pattern I noticed that I canāt articulate yet.
% head -n 1 conflicted*
==> conflicted <==
13
==> conflicted 2 <==
15
So conflicted 2 is the newest version here. And version ā14ā is just GONE. Rolled back to 13.