[Mobile] Sync with git on iOS for free using iSH

ForceBru · July 14, 2021, 12:38pm

There’s already a post about syncing with WorkingCopy ([Mobile] Setting up iOS git-based syncing with mobile app (using Working Copy)). However, the ability to push is paid, and it costs a lot, but without this there can be no sync!

I propose to use iSH to sync Obsidian with git.

Pros:

It’s free!
It lets you use fully-featured git - the same git that runs on Alpine Linux

Cons:

It requires using the Linux command-line
There appears to be a bug in Obsidian 1.0.3 that doesn’t show new files in the file explorer after you pull them from GitHub. This can be easily circumvented by sorting the notes

Tutorial

Also see the example repository I created to showcase this method.

Also see “Simpler way” below.

Assuming you have an existing repo on GitHub. You can also create the repo right from iSH - the git command you’ll be using is the exact same fully-featured git from Alpine Linux.

Create a new empty local vault in Obsidian
In iSH:
1. First-time setup - install git:
  1. Update Alpine repos: apk update
  2. Install git: apk add git
2. Create a directory called obsidian in your home directory by running cd ~ && mkdir obsidian.
3. Mount your local vault folder into the obsidian folder
  1. Run mount -t ios . obsidian
  2. A file picker will show up
  3. Choose the folder with your local vault
4. Clone your git repository into obsidian
  1. Change directory to obsidian: cd obsidian
  2. Delete the .obsidian folder: rm -rf .obsidian
  3. git clone https://github.com/ForceBru/ObsidianVaultTest . - use your own repository instead. Don’t forget the period . - this is what allows you to clone the repo into the current folder.
  4. (Optional) Run ls -a to see whether your files are there
5. In Obsidian
  1. Restart Obsidian (might not be necessary)
  2. Open the file explorer
    1. Currently (Obsidian 1.0.3) it’s empty (looks like a bug)
    2. Tap the “Sort” button (the rightmost one, with up/down arrows)
    3. Select any sorting order you like
    4. The files should appear
  3. Use your Obsidian vault!
To push your changes, go back to iSH
1. Change directory to obsidian in your home directory: cd ~/obsidian
2. Run git status to confirm that there are modified files
3. First-time setup - tell git who you are:
  1. Set username: git config user.name ForceBruMobile
  2. Set e-mail address: git config user.email "[email protected]"
4. Add files, commit and push. For example:
  1. Add all files into the commit: git add .
  2. Commit: git commit -m "Commit from mobile"
  3. Push: git push
    1. This will ask you to enter your username and password
    2. Git can use your personal access token to login automatically
  4. That’s it! Take a look at this example commit from iSH
To pull changes from GitHub:
1. In iSH:
  1. Change directory to obsidian in your home directory: cd ~/obsidian
  2. (Optional) Check whether there is stuff to pull
    1. Bring remote refs up to date: git remote update
    2. Check status: git status
      1. It should say something like Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded if there is something to be updated
  3. Actually update from the remote repo: git pull
2. In Obsidian:
  1. Go to file explorer
  2. If you don’t see your new notes, do the sort trick again:
    1. Tap the “Sort” button (the rightmost one, with up/down arrows)
    2. Select any sorting order you like
    3. The files should appear
  3. Updates to existing notes seem to be reflected correctly

Simpler way

Mount the entire Obsidian folder which contains all your vaults (as opposed to mounting just the vault).

In iSH:

Make directory to store Obsidian vaults: cd ~ && mkdir obsidian
Mount the file provider: mount -t iOS . obsidian
Choose On My iDevice and select and open the Obsidian folder
Change directory to obsidian: cd obsidian
Clone the repo with your vault right there: git clone https://github.com/ForceBru/ObsidianVaultTest
This will create a folder called ObsidianVaultTest that will be a git repo and that Obsidian will recognize as a vault

Then open the vault in Obsidian (might need to restart it), edit stuff, then push/pull changes as described in steps 3 and 4 in the original post.

To avoid entering login/password all the time, cache or store your credentials: Git - Credential Storage

DummyME · July 15, 2021, 3:36am

Requires a manual push pull?
and is it stored in iCloud or Obsidian app folder?

ForceBru · July 15, 2021, 9:39am

Yes, pushing and pulling is manual via the command line. The data are stored on your device, but it might be possible to mount the iCloud folder in the same way.

DummyME · July 15, 2021, 3:04pm

Could there be something done using Shortcuts? Like to run that command from a widget itself?

Krafty · July 16, 2021, 5:45pm

This is great! Though the git add . command is running very slow for me for some reason

I imagine it is easy to add some simple push and pull scripts to save time.

~/push

#!/bin/sh
cd ~/obsidian
git add .
git commit -m "Commit from mobile"
git push

~/pull

#!/bin/sh
cd ~/obsidian
git pull

ForceBru · July 16, 2021, 9:37pm

iSH emulates an Intel x86 CPU, so yeah, it can be pretty slow sometimes. It’s indeed possible to add your own scripts - iSH runs real-world Alpine Linux, so it can do basically whatever Alpine can do, except for things restricted by iOS, like USB/Lightning connections.

ForceBru · July 16, 2021, 9:40pm

I’m using iOS 12.4, so I’m guessing it can’t do much in the Shortcuts department. Not sure about what’s possible in more recent versions of iOS.

DummyME · July 18, 2021, 7:36pm

The cd Command should include the vault name too right? Since obsidian will not have any git. At least it throws an error for me. git add . is way too slow for me as well.

DummyME · July 18, 2021, 7:49pm

How can I run the script? I am getting permission denied.

DummyME · July 18, 2021, 8:32pm

Okay.
Sadly, it worked for a dummy vault, but as I introduced by own vault, it took ages to execute git add . and simply gets stuck despite keeping the app open forever. I haven’t synced .obsidian folder, only edited one test line in a single MD file. And still I am not able to commit😕

yvonnezed · July 18, 2021, 9:10pm

An alternative worth playing with might be a-Shell, ‎a-Shell on the App Store

It’s not as full featured as a shell, but it’s got more native bells and whistles including Shortcuts support.

The command to play with is lg2. It’s a work in progress, but it’s usable and a lot faster than iSH

DummyME · July 19, 2021, 2:39pm

Also, I got into thinking. If working copy free version can sync a folder into obsidian folder and pull and commit, why not use it for everything except push(paid).
Then simply mount the working copy folder in ish or anything and git push!

Because, the add command etc are a lot slower in iSH.

Edit: It works!!
Everything is managed in WorkingCopy. iSH is utilised only for git push
Maybe a super noob manoeuvre.

ForceBru · July 20, 2021, 9:19pm

That’s pretty cool, actually: WorkingCopy has great GUI, but pushing is paid, while iSH lets you push for free!

DummyME · July 21, 2021, 8:34pm

Yes. That’s exactly what I am exploiting for now.
Even had shortcuts. Pulls when I start obsidian, and commits when I exit obsidian.

Later when I leave ipad, I open ISH and git push.

Atlas · July 26, 2021, 3:10am

Do you mind sharing the shortcut or the commands in a-shell? I find myself not so familiar with shell commands . Thanks!

andrezgz · July 26, 2021, 9:28pm

How did you manage to configure Working Copy (free version) to sync a repo into obsidian folder? Can you detail the specific steps, please?

evergreensnow · July 27, 2021, 3:32am

I strongly dislike the idea of having my notes unencrypted on a hosting service like github so I extended the idea to support end-to-end symmetric AES256 encryption using gnupg on iSH. As the encryption is open source and cross platform you can decrypt your notes anywhere.

Steps:

Make a folders /obsidian and initialize a git repository. This will be a local git repo for rollbacks (see why it is needed later).
Make a subfolder /obsidian/vaults. Mount your ios obsidian folder here. Make sure to delete the .git file so the folder is staged correctly.
Make a folder /syncrepo. This is the repo that pushes/pulls encrypted files to/from the remote.
Initialize a repository in /syncrepo and connect it to your chosen remote repository.

Encryption is done using tar to compress the vaults folder into a single file, and then using gnupg to encrypt the file. As pulling involves overwriting parts of you existing vault, the local repository lets you rollback in case of an accidental pull.

Make sure to install gnupg using apk add gnupg.
Add the push/pull scripts to your root directory.

NOTE: Replace *** with your passphrase in scripts.

Pull script (~/pull)

#!/bin/sh

cd ~/obsidian
git add .
git commit -m "Save before pull"
echo "saved state before pull"

cd ~/syncrepo
git pull
cd ~
echo "pulled repo"

gpg -d --batch --cipher-algo AES256 --passphrase ****** -o - syncrepo/obsidianfile | tar -xzvf -
echo "uncompressed and decrypted"

Push script

#!/bin/sh

cd ~/obsidian
git add .
git commit -m "Save before Push"
echo "saved state before push"
cd ~

tar -czvf - obsidian/vaults | gpg -c --batch --cipher-algo AES256 --yes --passphrase ******* -o syncrepo/obsidianfile

echo "compressed and encrypted"

cd ~/syncrepo
git add .
git commit -m "Commit from Mobile"
git push
cd ~
echo "pushed to remote"

Set both files to run as scripts chmod +x ./pull (and for ./push)
Push/pull whatever content you please

yvonnezed · July 27, 2021, 7:15am

It’s actually pretty similar to the instructions above for iSH. the difference is that instead of git, the command is lg2. Also, instead of using mount to mount the file provider, the command is pickFolder to create what aShell calls a bookmark. then it’s jump bookmarkname to get into the folder.

Try help at the prompt to get a list of all the aShell specific commands.

As for the shortcut, it depends on what you want it to do. You just use the action to give aShell commands, make sure the first command in the list is to jump into your repo folder and then lg2 push or lg2 add or whatever.

Atlas · July 27, 2021, 2:22pm

Thank you! I succeeded! The only flaw is that a-Shell must show up after pushed XD

goncalvesjoao · September 28, 2021, 3:24pm

Guys, thank you so much for this tutorial!

I wanna contribute too and my finding is that the developers of iSH created a new filesystem that allow git add . to not be so slow.

github.com/ish-app/ish

Comment by NoahPeeters - Add support for mounting folders from iOS

ish-app:master ← NoahPeeters:mount_file_provider

I continued working on this. I added file coordination for regular files and rem…oved the `iosfs_mount_data` function by calling the ui functions in `iosfs_mount`. I did a lot of testing with `git` in a mounted folder from Working Copy. I noticed that `git status` can be quit slow even with mid-size repositories because git will check every file in the repository and each of the coordinator calls slows down the process. That's why I added a second filesystem: `ios-unsafe`. That one will simply make sure that the correct permissions for the security scope are requested but the filecoordinators are not used at all. In case of Working Copy, this works very well but there are probably apps which can have a problem with that (that's why I called in `ios-UNSAFE`). Therefore, this pull request now adds the following two commands: ```sh mount -t ios website /mnt/website mount -t ios-unsafe website /mnt/website ``` Finally, someone with more experience in async programming in objective-c should have a look at the usage of mutexes in `iosfs_open`. It does work but I do not know if there is a better/more elegant way of doing this.

Use:

#!/bin/sh
mount -t ios-unsafe . obsidian

and your git status and git add commands should start to run smoothly.

Warm hug!