From TWiki / FOSWiki to Obsidian

tl;dr: Check out oraculix/TWiki2Obsidian: Convert text files from TWiki markup to Markdown for Obsidian.md. Uses pandoc and GNU sed on Bash. - Codeberg.org

Many years ago I had my knowledge base within TWiki (later in its fork FOSWiki). It is a PERL based Wiki system that uses a syntax that resembles much of the early Wikis and is extendable with macros.
It was long in my bucket list to convert my old notes to Obsidian, but the seemingly easy solution with pandoc turned out to be too basic.

Pandoc (as of version 3.1.3) only does a very basic conversion of TWiki sources and has several shortcomings, like

  • not converting WikiLinkSyntax to Markdown Links (wtf?!)
  • ignoring Metadata (as in %META:TOPICINFO)
  • not dealing with TWiki macros, specifically
    • Code Macros. ==> Syntax Highlighting and sometimes even monospace gets lost.
    • Attachments ==> stays in the output but looks ugly. Furthermore, the existence of %META:FILEATTACHMENT{} slows pandoc down considerably, sometimes it even seems to halt completely.
  • List items in the form of [tab]* are converted to \*, losing all indentation along the way.
  • Piped tables are sometimes not recognized and get converted to escaped text. (not solved by my script yet)
  • Escaped WikiWords get un-escaped. Pandoc simply deletes the leading ! before it. WTF?! (not solved by my script yet)

My first approaches at just a few of these points was to use sed and its powerful regular expressions to pre-process the source files before converting them.
It turned into a sed orgy that solved most of my pain points (turn WikiWords into internal Links, list and embed attachments, turn code macros to code blocks, make the last update date a property).

I’ll leave a link to my TWiki2Obsidian repo on Codeberg here; maybe it is helpful to you.
Feel free to contact me here for any questions regarding this topic.

2 Likes

Thanks so much for putting together this script – you’ve save me a lot of time!

Long story short: Was provided with an old teaching lab’s FosWiki directory that needs to be made available through a wiki again, but I’m avoiding FosWiki like the plague so having this converted to markdown makes it much easier to work with and migrate to a different tool.

So while this isn’t being used for Obsidian, it is being put to good use!

1 Like