Obsidian Github Integration for Sync and Version Control

For those that find it useful

so i keep my vault on icloud for storage but on other machines like my fire walled work machine i can only access github so i also sync my vault with github. to keep all my configs and workspace state with exactly what my editor was when i left it on my personal machine here is what i did:

  1. You need git installed already, and this works seamlessly if you have your credentials cached so you dont need to enter your password on git push commands
  2. make an empty repo on github
  3. make my vault a git repo:
git init
git add .
git commit -m “init”
  1. Ignore files that are uneccesary and could cause errors on new workspace
  2. make a .gitignore file with the contents:
.obsidian/cache
.trash/
.DS_Store
  1. Now push everything to the remote repo:
git remote add origin https://github.com/USER/REPONAME.git
git push -u origin master

that will put your files on github ONCE now any updates need to be manually added with git commands etc. but why not automate the crap out it

  1. I make a shell script in my .local/bin/ directory (mac and linux, windows IDK if this works the same with WSL/Gitbash):
touch zk_sync
chmod +x zk_sync
  1. Shell script contents:
#!/usr/bin/env sh

ZK_PATH=”PATH TO YOUR VAULT”

cd “$ZK_PATH”

git pull

CHANGES_EXIST=”$(git status — porcelain | wc -l)”

if [ “$CHANGES_EXIST” -eq 0 ]; then

exit 0

fi

git add .; git commit -q -m “$(date +”%Y-%m-%d %H:%M:%S”)”; git push -q
  1. WHAT THE SCRIPT DOES:

you should never run a script without knowing what it does as it could easily be malicious and nuke your machine so line by line:

#!/usr/bin/env sh
# ^^^^^^^^^^^^^^^ This says find the first instance of a sh (shell) 
# binary and use that shell to execute these commands. 
# There is little to no complexity here and no bashisms so it 
# should work just fine on most systems and instances of shells 
# (bash, zsh, sh, etc.)

ZK_PATH=”PATH TO YOUR VAULT”
# ^^^^^^^^^^^^^^^^^^^^^^^^^^ We are assigning the variable `ZK_PATH` 
# with the (maybe) long string to our vault’s location (mine is super 
# long so this makes the final command look cleaner, 
# it's unnecessary if you care)

cd “$ZK_PATH”
# ^^^^^^^^^^^ cd: Change Directory to your vault’s location

git pull
# ^^^^^^ So if any changes occurred remotely or on another machine
# your local machine knows to pull those changes down instead of 
# having to wait for a local change to run the script

CHANGES_EXIST=”$(git status — porcelain | wc -l)”
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ we are assigning 
# a value to the variable `CHANGES_EXIST`, the value is the output 
# of `git add — porcelain` which outputs a simple list of just the 
# changed files and then the output is piped into the `wc` utility 
# which is “word count” but with the `-l` flag it will count lines. 
# basically, it says how many total files have been modified. 
# if there are no changes the output is 0

if [ “$CHANGES_EXIST” -eq 0 ]; then

exit 0

fi
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The whole if block is saying 
# in plain english: if there are no changes (CHANGES_EXIST = 0) 
# then exit with no error code `exit 0` if there are changes, 
# then continue on with the script

git add .; git commit -q -m “$(date +”%Y-%m-%d %H:%M:%S”)”; git push -q

# git add. = add all current changes in the repo no matter the 
# level of nested folders/files

# git commit -q -m: this saus we are committing changes to 
# our repo, -q says BE QUIET no output prints to terminal 
# if ran manually, -m defines a message for the commit log

## the -m message is “$(date +”%Y-%m-%d %H:%M:%S”)” this 
# runs the command date with the formatting arguments for a 
# date in YYYY-MM-DD HH-MM-SS format as your commit message

# git push -q: push the changes to github and BE QUIET about it

# The semicolons between commands are just saying run each 
# command and then run the subsequent command, they’re just 
# separators

So now that we have the script ready we enter CRON!

  1. Set up cron job (More on cron: https://ostechnix.com/a-beginners-guide-to-cron-jobs/)

on linux i used cronie, mac comes with cron installed already, to get into your cron jobs:

crontab -e
# ^^^^^^^^ -e: edit your crontab file i.e. your list of cronjobs

my cronjob looks like this:

*/30 * * * * /Users/bryanjenks/.local/bin/zk_sync >/dev/null 2>&1

*/30 * * * * the first section is the timers
/Users/bryanjenks/.local/bin/zk_sync the second is what file is it executing (absolute path)
and then to make sure its quiet its sending any script output (shouldnt be any we used -q a lot) to >/dev/null 2>&1

so now any changes i make to anything in my vault is pushed to github every 30 minutes with an ISO time stamp for the files with changes.

i keep the repo private, and this way i always have my work available where ever i go and with git i have version control. plus if there are no changes, the command doesnt do anything.

hope someone else enjoys the workflow :slight_smile:

26 Likes

Did you only exclude .obsidian/cache from the Git repo? I also added .obsidian/workspace to the .gitignore as the current state of the Obsidian window is stored there and changes constantly.

yeah i looked at each of the files in there, and cache was the only one I didn’t want.

i look at it like this: if I’m accessing my vault from work by the time I get there any changes or workspace configurations I’ve made will have been pushed to the remote. Then I can just pull down on the work machine any changes from the prior day so that way even my editors workspace is the same as what I left at home. i also added to my script a git pull after git add just so it also keeps current with the repo when I’m on my way home.

i guess it could basically be considered a poor man’s cloud storage solution :rofl: but with version control.

Ah I see. Depends on the workflow then.
I run Obsidian at work and at home and I only care about ‘lazy data consistency’, i.e. my Obsidian workspace at work and home may look different - as long as at some point in time the data is consistent between both instances. So excluding .obsidian/workspace makes sense for me - but in your case it’s perfectly understandable to include it.

1 Like

Because you said you were using iCloud to store your notes: What Markdown editor do you use on iOS?

I tried out a couple now and the problem I have is that the editor can open files before syncing with iCloud and it’ll then overwrite the current file with the old content.

Did you encounter this problem too or do you have a mobile setup that works better?

@juen iOS user here – I’ve noticed that problem occasionally, so I switched over to using the GitJournal app instead (which makes sense for me because I use a git repository to sync my Obsidian files too).

GitJournal acts as an editor on iOS for your Git repository files.

Prior to using that, my solution was to go to the Files app on iOS and let it refresh updated files before opening an editor app.

Ray

2 Likes

Thanks, will check it out!

i choose to not edit on my phone, I don’t like typing a lot with it and it doesn’t let me access or use my obsidian note templates. i just use desktops and this workflow. iCloud is just for another redundant storage method only for me

1 Like

@tallguyjenks On Mac the backup script doesn’t work, do you have any idea of which could be the problem?

braindump_sync: line 11: [: “��: integer expression expected
date: illegal time format
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] …
[-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
error: pathspec ‘Sync:’ did not match any file(s) known to git
error: pathspec ‘”’ did not match any file(s) known to git

interesting, maybe its because i install the gnu core utils because the bad ones kinda annoy me. but try adding -u after the date function call for the serial date message:

git add .; git commit -q -m “$(date +”%Y-%m-%d %H:%M:%S”)”; git push

to

git add .; git commit -q -m “$(date -u +”%Y-%m-%d %H:%M:%S”)”; git push

let me know if that works and i can update it for others

Strange, now it works (maybe because I removed “Last Sync:” before the date as you wrote on Medium) but the error about the illegal date format remains and on GitHub the commit name is void.

this is after adding the -u flag?

Yes, exactly the same error minus the Sync: part

strange. its all supported POSIX standards for BSD so that should work on all kinds of systems especially other macs…

the error might be caused by the double quote format.

git add .; git commit -q -m “$(date +”%Y-%m-%d_%H:%M:%S”)”; git push

Try adding an underscore or a dash between it. That’s how it worked for me.
(At least the first time, then Git kept crashing in WSL 2, but that had nothing to do with the commit message, but with Git alleging there to be lock files when there weren’t.)

I wrote this powershell script with similar functionality. gist here: powershell script

You can schedule a windows task, similar like a cron job.

1 Like

I’ve been using Github as a sync platform for some time now with good results. I adjusted my link format to ‘Absolute path in vault’ which then gives me the added benefit of navigating (view-only) my notes repository using the Android Github app like a personal wiki. I constantly move from laptop to PC to mobile, so I haven’t tried the automation described above.

2 Likes

I adjusted my link format to ‘Absolute path in vault’ which then gives me the added benefit of navigating (view-only) my notes repository using the Android Github app like a personal wiki.

@Erisred would you mind explaining or breaking down this process for a non-developer? i’m not sure what this means

i’m attempting this obsidian -> github sync process now.
was considering using GitJournal to edit my notes on mobile -> sync to GitHub -> sync to Obsidian local files (?)

git pull
# ^^^^^^ So if any changes occurred remotely or on another machine
# your local machine knows to pull those changes down instead of 
# having to wait for a local change to run the script

@tallguyjenks would this bit enable the GitJournal sync to Obsidian files process?

i’ve never used git journal, git pull in that particular part of the script is so, hypothetically, if I’m at work and i make changes to my git repo through github, when i get home, the script will run and pull any changes made to the remote repo to my vault on my computer using git

1 Like