Import from TiddlyWiki 5 to Obsidian

I migrated around 2,600 entries from TiddlyWiki 5 to Obsidian.

My method was based on this really helpful blog post. These instructions are for MacOS. I’m not much of a programmer so a lot of this stuff is kinda hacky and someone can probably come up with a cleaner version of this. I am happy with the end result though.

Here’s an outline of my steps:

1. First export all the Tiddlers to HTML files

Run this command in your TiddlyWiki directory

tiddlywiki --render [!is[system]] [encodeuricomponent[]addsuffix[.html]] text/plain $:/core/templates/custom-template

A couple notes about that command: 1. I am using the encode URI flag because I have a lot of entries with punctuation and special characters which I wanted to change manually. You may not need it. 2. I created a custom template for myself that has the tags in plain text listed at the top. Here is my template:

title: $:/core/templates/custom-template

`created: `<$view field="created" format="date" template="YYYY-0MM-0DD"/>
<$fields exclude='text bag title tags type created modified' template='$name$: $value$
'></$fields>

<$list filter="[all[current]tags[]!is[system]sort[title]]">#<$view field="title"/> </$list>`
`
<$view field="text" format="text" />

2. Rename the files

I used Hazel on Mac for this but you could use other apps

  • Do a bit of cleanup to remove the URL encoding on the file name. I do this so that I can control exactly how the characters get replaced in the file name.
  • Change creation date and modification date

3. Convert to Markdown with Pandoc

Run this command on the folder that contains all your HTMLified tiddlers.

for f in *.html; do pandoc "$f" -f html-native_divs-native_spans -t commonmark --wrap=preserve -o "-md/${f%.html}.md"; done
  • iterates on all files in a folder
  • puts them in a “/-md” subfolder

4. Cleanup with Sublime Text

Using the batch find/replace on the whole folder I ran a lot of cleanup on the entries.

I couldn’t figure out how to stop Pandoc from escaping all kinds of characters so I ran this (in normal find/replace mode)

Find: `\` Replace: nothing

Also needed to clean up internal links. Pandoc spits internal links out as [Note Name](#Note%20Name) Use regex mode (click the * icon in the find/replace UI) to find internal links and replace them with [[ ]] notation

Find:  `\[([^\[\]]*)\]\(\#(.*?)\)`
Replace:  `[[$1]]`

That’s about it! Did some other minor cleanup with Sublime. LMK if this helps. I think there is definitely room to simplify the process.

8 Likes

I was also using Stroll (based on TW5) and had put aside migrating over folders until I could find an easy method, so glad to find this!

Except I’m so code-illiterate that even though I’ve read through your, and the original link’s step by step instructions, I can’t get past the first step haha (for example, after I open an empty TW, how do I allow markdown? - this was step 1 of the article)

if someone would be so kind to volunteer their time and literally write every single step so someone like me could import their TW files to markdown, I’d super appreciate it <3

totally get if no one does cuz it would be so tedious, but still wanted to comment that this was a very helpful post! (probably since most here seem to know basic coding)
*I am working to learn! just a very slow progress :’(

I don’t think you need to allow markdown, you can just run that command in the folder that contains your TiddlyWiki installation (the parent of your “tiddlers” folder)

1 Like

thanks for the reply!
When you say to “run the command”, is it by using a program? Or is this something that’s done by making an edit within a file?

1 Like

I had the same questions as you initially and am working on writing down a step-by-step workflow. Which OS are you on?

1 Like

How to Export from TiddlyWiki to Obsidian

Note: I am on Mac, so all the steps are specially for that OS. I followed the steps by @kepano and just tweaked/clarified the things that didn’t work for me initially. So this is very much his work. @pikacho, I hope this is helpful for you and others.

Step 1: Set up TiddlyWiki in Node.js

  1. Download and install Node.js
  2. Open “Terminal”, run npm install -g tiddlywiki
    • If an error occurs, run sudo chown -R $USER /usr/local/lib/node_modules (that gives you admin privileges for the specific folder
    • Run npm install -g tiddlywiki again
  3. Check if it works by running tiddlywiki --version. If everything worked it should show you the version of TiddlyWiki
  4. Run tiddlywiki myfirstwiki --init server
  5. Run tiddlywiki myfirstwiki --listen to start the server

Step 2: Import Notes into new TiddlyWiki

  1. Open http://127.0.0.1:8080/ in your browser. It should load an empty TiddlyWiki.
  2. Drag the file that contains the TiddlyWiki you want to export (for me it’s “empty.html”) in the top of the browser window
  3. Press “import”. This imports all of your Tiddlers into the empty wiki so we can export them later. You might need to reload the wiki.
  4. In Terminal, press Control + C to exit the server

Step 3: Export Notes as .html

  1. Navigate to the folder “myfirstwiki” in Finder.
    • For me that’s ~/myfirstwiki/. In Finder Shift + Command + G opens a window in which you can copy my path.
  2. Right click → Services → New Terminal on the “myfirstwiki” folder.
  3. In that Terminal window run tiddlywiki --render [!is[system]] [encodeuricomponent[]addprefix[tiddlers/]addsuffix[.html]]
  4. You can find the exported Tiddlers in the “output” folder in the “myfirstwiki” folder.

Step 4: Change .html to .md notes using Pandoc

  1. Install Pandoc

  2. Navigate to the “output”-folder containing the .html Tiddlers

    1. Open Terminal
    2. Show directory by typing and entering pwd
    3. Move to directory
      • cd makes you enter a folder, pwd shows you the directory you’re in
      • This is how I entered the folder, yours might be slightly different
        • cd myfirstwiki
        • cd tiddlers
        • cd output
        • pwd to confirm you’re in the right directory
  3. Once you’re in the right folder run this command in Terminal: for f in *.html; do pandoc "$f" -f html-native_divs-native_spans -t commonmark --wrap=preserve -o "${f%.html}.md"; done

  4. In Finder, I then copied and pasted the Markdown files into a separate folder.

A note on the original code by @kepano
  • The original code (for f in *.html; do pandoc "$f" -f html-native_divs-native_spans -t commonmark --wrap=preserve -o "-md/${f%.html}.md"; done) that puts all of the files into a seperate folder didn’t work for me. All of the tiddlers that are transferred to Markdown (.md) files are now in the same folder as the .html files.

Step 5: Clean up using Atom

  1. I use Atom, other editors will work just as fine. I loaded the folder with all of the Markdown Tiddlers into Atom and then opened “Find in Project.”
  2. Clean up links.
    • Use regrex mode (click the * icon in the find/replace interface)
    • Find: \[([^\[\]]*)\]\(\#(.*?)\) Replace: [[$1]]
    • This cleans up the links from [Note Name](#Note%20Name) to the [[]] notation.
  3. After that, I didn’t need to do any more cleaning up. Run more find/replace searches if necessary.

Step 6: Import into Obsidian

  1. Navigate to the Obsidian Vault in Finder. Create a new folder “TiddlyWiki Import”.
  2. Copy and paste the Markdown Tiddlers in the folder.
    • Note: A lot of my notes were just empty pages that got created by linking to them. The way Obsidian handles backlinks makes them redundant. In Finder I sorted the converted Tiddlers by size and only imported the ones that were bigger than 1 byte.
  3. Open (if it’s already open then reload with Cmd + R) Obsidian and voilà! Your notes are right in the TiddlyWiki Import folder!

Some final notes
I excluded the use of a custom template in step 3 to minimize friction. I can see how that would be very handy to add though. Also somehow my journaling notes didn’t get exported properly so I will have to manually transfer those.

I am not a coder, so my knowledge comes just from trial and error. If more experienced forum members have things to add/simplify please let me know. All of these steps worked well for me but if you run into issues, let me know and I will do my best to help :slight_smile:

5 Likes

Hi folks!
I wanted to publish my notes on a website, so do you guys know how to go from obsidian to Stroll/tiddlywiki?

@selfire is that for windows?

I’m using MacOS, so no. It might work as well I just haven’t tested it.

Thank you for taking the time to write the instructions. I followed it and it worked for me.

There is a slight error in your instruction in step 3, number 3 - tiddlywiki is missing at the start. So, it should read:

tiddlywiki --render [!is[system]] [encodeuricomponent[]addprefix[tiddlers/]addsuffix[.html]]

Hey @fusionmuck! Glad to hear that it worked for you, even with that error. Thank you for letting me know, I fixed it now!

You’re welcome - the instructions was very well written!

1 Like

These steps by @kepano and @selfire proved really helpful for me in my migration. I would like to add a few pointers to help anyone who may have faced some of the challenges I did.

In Step 5 of @selfire’s solution, I still had html anchor tags in my markdown files. So I used the following regex to fix them. Find: <a[\s]+([^>]+)>((?:.(?!\<\/a\>))*.)<\/a> Replace: [[$2]]

Also, after I was done, I had %20 in my file names because of the spaces I used in TiddlyWiki. This solution: https://unix.stackexchange.com/a/174131 helped me resolve that.

Cheers.

1 Like

There is a post on reddit for that. I had it working for me. But the problem arose when I began using [[ | ]] notation for alias. In tiddlywiki5 its the opposite of Obsidian. If you like to use the basic markdown notation. you will have everything working!!

Let me know if you need the link