Yes, the plugin syncs every file that obsidian detected in the vault as is.
I didn’t check editing by other software while running obsidian, but if your obsidian reloads the file, it will be synced.
Honestly, I’m not getting to be true friends with Docker yet, I used that to my test server first last month.
But actually, easy for use impressively. I have to learn.

it’s amazing!
really need this !

1 Like

thank you for your great work !

1 Like

On testing 0.1.16 with E2EE enabled, I’m finding that the plugin still gets stuck in a replication loop. This leads to very high CPU usage (20-30% on a Ryzen 5 3600) at idle with fluctuating memory usage.

Here is the log:
11/17/2021, 7:02:02 PM->loading plugin
11/17/2021, 7:02:02 PM->Open Database…
11/17/2021, 7:02:02 PM->Database Info
11/17/2021, 7:02:03 PM->Open Database…
11/17/2021, 7:02:03 PM->Database Info
11/17/2021, 7:02:03 PM->Initialize and checking database files
11/17/2021, 7:02:03 PM->Updating database by new files
11/17/2021, 7:02:03 PM->UPDATE DATABASE
11/17/2021, 7:02:03 PM->UPDATE STORAGE
11/17/2021, 7:02:03 PM->CHECK FILE STATUS
11/17/2021, 7:02:03 PM->Initialized,NOW TRACKING!
11/17/2021, 7:02:03 PM->Pull before replicate.
11/17/2021, 7:02:04 PM->Replication pull completed.
11/17/2021, 7:02:04 PM->Replication activated
11/17/2021, 7:02:05 PM->Replication activated
11/17/2021, 7:02:06 PM->Replication activated
11/17/2021, 7:02:07 PM->Replication activated
11/17/2021, 7:02:09 PM->Replication activated
11/17/2021, 7:02:12 PM->Replication activated
11/17/2021, 7:02:13 PM->Replication activated
11/17/2021, 7:02:15 PM->Replication activated

Disabling E2EE led to the plugin becoming stuck in “NOW TRACKING!”.

Sorry for late, thank you for testing!
Could you enable verbose log?
It’s little mysterious that only ‘replication activated’ shown so many times.
(if really activated, should ‘paused’ is shown too.)
Could you make an issue on GitHub?

I’ve tried to replicate the problem with a new database, but everything seems to be working fine with 0.1.17.
I’ll keep on testing and if I run into any other issues, I’ll make sure to retain verbose logs and create issues on Git.

1 Like

I’m very happy and relieved to hear that!
Thank you for your patience and waiting for improvements.

Yes, please! and of cource, and if yor are not sure that is the bug, please feel free to contact me.

At the moment the plugin relies on CouchDB to sync files. Would it be easier taking an approach similar to Syncthing, where the plugin is able to sync raw .md files and folders to a remote computer directly?
Taking this approach would mean the vault can remain intact without modifications during transport.
Syncthing also uses UPnP which would make setup for those wanting to host on their own hardware for use over the internet much easier than setting up a CouchDB server.
This would prevent the file size limitations and other troubles that come with using databases.

1 Like

Very long.

It may lose some features

  1. LIveSync
  2. Conflict detection and resolution.
    When using CouchDB, We can handle the tree of changeset in the replication, and resolving even in off the line.
  3. Dedupe chunks and keep low transfer amount.

And, of course, we have the limitation of JavaScript. It’s the plugin.

I tried WebRTC for peer-to-peer synchronization but some network denied.

HTTP is the simple and most penetrating protocol.

And I tried to make a plugin that as you said. As the complementary of Official “Sync” and Self-hosted LiveSync. Cloud Backup.
It would be very useful. I thought too.

But there are a few limitations, I realized it’s nonrealistic or may not be simple so much.

OneDrive, GoogleDrive, or other any drive may not accept the connections from obsidian by CORS limitations. If you want to access these hosts, need a proxy server to modify the request. To be secured, require
users to host the server.

A self-hosted WebDAV server can use this without request-modifying-proxies. But still, self-hosting is required.

And, currently, mobile Obsidian has the limitations that it can’t access the non-secure endpoint or self-signed endpoint. So self hosted server should be exposed globally. It’s should not be done in so easy and optimistic attitudes.

So, If I write the server software? It will be work well to synchronize everything.
But very hard to prove there’s nothing vulnerablities.

And, it may be harder than launching docker images of CouchDB and traefik(for reverse proxy).I writing this docs.

1 Like

I see, thank you for your thorough answer! I hadn’t realised there were so many limitations to Obsidian plugins themselves.

It seems like you made the right choice to use CouchDB.

1 Like

Hi vorotamoroz

Thanks for your creative plugin, I do need this for hosting notes on my own server.

I managed to setup the couchdb server with HTTPS, setup the local plugin and forward all notes from laptop to the server, I then setup the mobile app from scratch, the mobile app could download notes from the server.

But the rare thing is I cannot manage them to sync with each other, the notes I modified on laptop or mobile never forward to another peer.

I’m not quite familiar with couchdb, is there a way to check the latest changes on the server? So I may know if the changes are properly pushed to the server or not. I disable the P2P encryption and enable the verbose log on each peer, but there is not much information shown in the log. Or is there a way to check the local database on the laptop or mobile?

Thanks in advance.

1 Like

I cant type the URI, username, password, etc under Android version.

1 Like

Thank you for using this plugin!
But it is something mysterious.
Your vaults have been synced once, but no more sync again…

Please check the remote URI and databases are exactly the same once again?

The latest change sequence no is shown in the sync log, if you enable verbose log. update_seq is so.

Have you enabled any sync?

If any sync is enabled, it could not be able to
change.

Double checked the connection to the database, URI, username, password, database name and all he same. I hosted my couchdb behind the caddy, CORS sets to enabled with “*” or “app://obsidian.md,capacitor://localhost,http://localhost” still acts the same. If the CORS is set to disabled then each peer cannot connect to the database. Since the first time android app could download all the notes, I think it may not be the issue of CORS. I will try to capture network packets on the database and see if anything goes wrong.

I found two files under my couchdb docker:

  • /couchdb/data/shards/00000000-7fffffff/obsidian.1638094448.couch
  • /couchdb/data/shards/80000000-ffffffff/obsidian.1638094448.couch

Would this mean I have two vault to sync with mobile and laptop app separately?

Thank you for checking your settings! And your CORS configurations are right.
Splitting shard is the normal behavior of CouchDB. It’s for clustered Databases. Do not worry about it.

Are there any warnings shown on Settings Dialog?
The warning is shown with the red border. like this:
image

These warnings are shown after being replicated once.

If any warnings are shown, please answer each warning and try to replicate again.

Yes there were two warnings and I have checked both of them on each peer, one is this about incompatibly and another is about lock the database for damage.

It’s quite strange, I’ll check with the public couchdb service and see if my docker (apache/couchdb, v3.2.1) causes the issue.

Never mind, it’s notified from Self-hosted LiveSync.

Sometimes, by timing problem, the plugin detects the differences caused by the migration of local databases (or false positive.)

The plugin stops replications as safely as possible.

When once detected, It won’t be shown by the next incompatible version up.

Please try resolving warnings. Sorry for the inconvenience.

Yes you’re right Thanks. I supposed that the plugin supports webdav sync but I am wrong since I haven’t read your post carefully. :joy:

Never mind. Thank you for your great work.

I tried with two laptops today, and they could sync up with each other, with the same server and same configurations, seems it’s only my Android mobile has the issue. I’ll try to find out why.

Two questions:

  • the “Device and Vault name”, is it just an identifier for different device or is it must be the same on different peer?
  • I see “local database” mentioned in the settings, where is the local database locates? I couldn’t find it on Windows or Android.