Converting Roam Daily Notes To YYYY-MM-DD format, with backlinks

[pmbauer]

For the latest instructions see the official Roam Research import guide

Import from Roam Research - Obsidian Help

---

This is a small bash script I used to convert Roam Daily notes so they work with a YYYY-MM-DD format and add some front matter metadata to work better in Obsidian.

Note for MacOS Users

bash sed and date in this script assume a GNU userland. If you are on Linux it should just work. If you are on Mac OS X try brew install bash gnu-sed gnutls and make sure gnu sed date and bash are on your path.

export PATH=/usr/local/opt/coreutils/libexec/gnubin:$PATH
export PATH=/usr/local/opt/gnu-sed/libexec/gnubin:$PATH

I’ve not tried it with Mac OS X’s BSD version of these.

Usage

As always, backup before trying this.

Note if you’ve seeded an Obsidian vault with a Roam Research markdown export, make sure to close that vault before running this script. Obsidian observes changes in the vault folder and tries to update its indexes during the conversion which will slow things down.

From the root of a Roam Research markdown export run:

# convert all daily notes from 2020/2021
ls *,\ 202[01].md | convert_daily_notes

convert_daily_notes Script

#/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

roam_date_to_obsidian() {
  date --date="$(echo "${*}" | sed 's/th//;s/nd//;s/rd//;s/1st/1/')" \
    '+%Y-%m-%d'
}

while IFS= read -r line; do
  roam_daily_note=${line/%.md/}
  obsidian_daily_note=$(roam_date_to_obsidian ${roam_daily_note})

  # add front matter
  echo "---" >> ${obsidian_daily_note}.md
  echo "tags: [daily]" >> ${obsidian_daily_note}.md
  echo "---" >> ${obsidian_daily_note}.md

  cat "${roam_daily_note}.md" >> ${obsidian_daily_note}.md

  # update links
  sed -i "s/\[\[${roam_daily_note}]]/[[${obsidian_daily_note}]]/g" *.md */*.md || true

  # give it a relevant timestamp
  touch -d ${obsidian_daily_note} ${obsidian_daily_note}.md

  rm "${roam_daily_note}.md"
done

Related

Get all uploaded files in Roam Research to Obsidian

You can try these two commands or modify them to your use (run from your vault root). I recommend making a backup first or managing the vault with git. First, download a markdown export of all your notes and extract the zip file. The commands below can be run in that extracted folder. Download all images off firebase and store them in _attachment/img. mkdir -p _attachment/img && \ sed -n 's/.*\[](\(https:\/\/firebasestorage.*\)).*/\1/gp' *.md */*.md \ | sort | uniq \ | sed 's/.*%2F…

[jimw]

Hi,

A complete newbie to both Obsidian and Bash scripts… I’m using Windows 10, could you please point me in the right direction of how to put this together?

Can I simply cut and paste the code into Notepad and he invoke it from Command Prompt?!

Really sorry for the simplicity of this question - this solution looks perfect for my needs!!

Thanks

Jim

[jimw]

OK - so I’ve made some headway with this (thanks to Google!)

I’ve installed Ubuntu from the Windows store and have created the “convert_daily_notes.sh” file within the relevant vault using the nano editor.
I have then tried various versions to try and run the script:

1. As above
2. ./ convert_daily_notes.sh

But I’ve had the following error returned
cat: .md: No such file or directory

or
convert_daily_notes: command not found

Where am I going wrong!!!

[pmbauer]

I’m impressed (I don’t have windows).
Sounds like you have an ubuntu user environment. Great. Make sure you are using a bash shell (type “echo $SHELL”)

If you’ve downloaded your file and named it convert_daily_notes.sh, that’s okay. Try typing this at your command prompt from inside the folder where you have the script:

chmod 755 convert_daily_notes.sh

that will make the script executable
then try running just the ls command from your vault and see what you get

ls *,\ 202[01].md

if everything is working, it should list all your daily notes from 2020 and 2021, assuming they are also all in the vault root as they would be after making a roam markdown export.

Finally try running the full command

ls *,\ 202[01].md | ./convert_daily_notes.sh

(assuming your convert script is also in the vault root)

remember to close obsidian before running that; it could take awhile

[jimw]

Thank you!! The results are mixed!!

chmod 755 convert_daily_notes.sh

Returns “chmod: changing permission of ‘convert_daily_notes.sh’: Operation not permitted”

ls *,\ 202[01].md

Works as expected and lists everything

ls *,\ 202[01].md | ./convert_daily_notes.sh

Then returns “./convert_daily_notes.shL line 15: 2020-04-12.md: Permission denied”
I’m not sure whether this is related to the lack of permission earlier or whether there is something specific to this file (there is no file called “2020-04-12.md”, but there is “April 12th, 2020.md”)?

Really appreciate your help - thank you

[jimw]

OK, I’ve tested a bit more:

• The folder is in OneDrive, so I’ve turned off sync
• I’ve checked all instances of Obsidian are closed (and exited)
• I’ve moved the “April 12th, 2020” out of the vault and now have the following:

sed: preserving permissions for ‘./sedLc9028’: Operation not permitted
sed: cannot rename ./sedLc9028’: Permission denied
touch: setting times of ‘2020-04-13.md’: Operation not permitted

[jimw]

This was completely right!!! The problem was OneDrive (as it always is!) - I needed to make sure that the folder was “Always available on this device”!

100% converted - thank you SO much pmbauer - I’ll write more complete notes on how I did this for future Windows users later in the week

[pmbauer]

I am happy to add any tips you have to make it work in a windows section of the parent post. Great work getting this to run on windows!

[friednslip]

Hi @pmbauer — this is exactly what I was hoping to find. alas i’m on Mac OS X and don’t speak bash— so I understand that you won’t be able to replicate exactly— but maybe there’s a small chance:

1. I replaced sed with gsed and saved the script on the roam markdown export folder —which I’ve renamed as roam

2. ran chmod 755 convert_daily_notes.sh and the test ls command — which worked

3. however, when I ran ls *,\ 202[01].md | ./convert_daily_notes.sh— the response I got was

   date: illegal option – -
   usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] …
   [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]indent preformatted text by 4 spaces

Any chance you might know what happened here/ how I might be able to fix it?

[pmbauer]

from your prompt, what does which date say? it probably needs to point to a symlink to someplace in the brew packages (needs to be gnu date). if it’s /usr/bin/date, probably still using the bsd date utility.

[pmbauer]

hey, I’m pretty sure this is it. Before running the script, try running this:

export PATH=/usr/local/opt/coreutils/libexec/gnubin:$PATH

Or update the script to use /usr/local/opt/coreutils/libexec/gnubin/date

[obsidvicious]

@jimw

I’ll write more complete notes on how I did this for future Windows users later in the week

Hey, any update on that step-by-step Windows conversion guide? Would love to be able to convert these daily note pages with links intact. (And daily more annoyed that Roam devs thought this naming convention would be a good idea in the first place)

[pmbauer]

I believe @jimw 's issues were due to onedrive not having all the files downloaded.
The script should just work if you

I’ve installed Ubuntu from the Windows store and have created the “convert_daily_notes.sh” file within the relevant vault using the nano editor.

So if you install the Windows Subsystem for Linux the script should just work in that environment. You can ignore the MacOS X instructions. (I don’t have a windows install handy to confirm this).

[TheAOR]

Lets say I’m a year in to using Obsidian and have already used this script successfully to convert everything during my initial migration from Roam, but continued using the “MMMM Do, Y” format for my Daily Notes (and their file names). Realizing now that I should’ve stuck with the YYYY-MM-DD format due to the majority of plugins I use referencing the YYYY-MM-DD format.

Could I technically run this convert all of my dates, file names, (and any date shown on any note using “MMMM Do, Y” to “YYYY-MM-DD” format?

[MagnusHultin]

I use MacOS 13.1 and followed the instructions to brew install bash gnu-sed gnutls. I kept running into issues when setting the environment, but managed to get it up and running by specifying gdate, gsed and gtouch in the code.

#/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

roam_date_to_obsidian() {
  gdate --date="$(echo "${*}" | gsed 's/th//;s/nd//;s/rd//;s/1st/1/')" \
    '+%Y-%m-%d'
}

while IFS= read -r line; do
  roam_daily_note=${line/%.md/}
  obsidian_daily_note=$(roam_date_to_obsidian ${roam_daily_note})

  # add front matter
  echo "---" >> ${obsidian_daily_note}.md
  echo "tags: [daily]" >> ${obsidian_daily_note}.md
  echo "---" >> ${obsidian_daily_note}.md

  cat "${roam_daily_note}.md" >> ${obsidian_daily_note}.md

  # update links
  gsed -i "s/\[\[${roam_daily_note}]]/[[${obsidian_daily_note}]]/g" *.md */*.md || true

  # give it a relevant timestamp
  gtouch -d ${obsidian_daily_note} ${obsidian_daily_note}.md

  rm "${roam_daily_note}.md"
done```

Just in case it helps someone else.

[claytonman]

Your approach is neat - doing this conversion in Road Research vault rather than Obsidian vault. Thanks for sharing!

For me I would need a slightly different approach, as I already migrated from Roam Research to Obsidian a few years ago. I have a few thousand legacy daily note files from Roam that follow the annoying month-day-year format, and a few thousand daily notes from Obsidian that follow YYYY-MM-DD format.

This shell script served me well. At least I could rename all the legacy daily note files into a uniform YYYY-MM-DD.md convention, though the links inside those old notes remain unchanged.

duh… this forum doesn’t allow link. That’s a bummer. Well, my shell script can be found on my site, digitalsovereignty.herbertyang.xyz