Live compiling of notes to PDF via groff


Warning: This is probably only suitable for people who have some experience with the command line. It’s not a full guide, just an inspiration! I still try to write it relatively detailed. I’m not a programmer at all though, and most of it is copy-pasted together. I would be happy if someone with more knowledge looks at this and could improve it! I did this on macOS, but it should work similarly on Linux. I didn’t test it on Windows (WSL). Also: Back up your files before using those scripts! As I said, I’m not a programmer. So, something could go wrong.


Live compilation with groff

Here I would like to write up how I use live compilation from Markdown to PDF via groff. First of all, groff is a typesetting tool from the GNU universe that you probably already have installed, if you use macOS or Linux. You can think of it as similar to LaTeX (or TeX). The advantage is that it compiles faster (in my test around 10 times faster).

I set up a live compilation that uses the fact that Obsidian saves the notes written quite often. We can then monitor a file change with entr that triggers a compilation to PDF via pandoc. (You could also use this with any other text editor, if you want.)

What works

  • footnotes
  • automatic citations and bibliography [@style]
  • Markdown tables
  • LaTeX math (wrapped in $$)
  • write raw ms (using groff to write and to make equations, tables, graphs) (For the basics I recommend this video.)
  • Obsidian links (with a ruby preprocessor)
  • YAML front matter for title, date, author etc.
  • customization via template

What you need

  • groff
  • pandoc
  • entr
  • ruby (optional, for Obsidian links in the PDF taking you back to the respective note in Obsidian)
  • hotkey program like skhd, Alfred, …
  • PDF viewer like with auto refresh on file change like Skim, LivePDFviewer (macOS) or Zathura (Linux)

If you’re on macOS, you probably already have groff installed. But it’s worth updating it to the latest version. ruby you also probably already have. You can install the necessary parts by installing homebrew first. After that just run:

brew install groff pandoc entr

Set it up

It works with two files:

  • obstogroff (Obsidian to groff): the monitor script that tells entr to monitor the file
  • pgroff (Pandoc–Groff): the script that actually compiles

Monitor script

First I needed a script that tells entr which file to monitor. Therefore, we take the content of the clipboard. In Obsidian, you have to use the command “Copy file to path” to copy it into the clipboard. You can set a hotkey for this, e.g. Cmd + Ctrl + C.

#!/usr/bin/env bash

# obstogroff
# Linux parts not tested (please review)!

########## CONFIGURATION ###########
PGROFF="$HOME"/.scripts/pgroff
VAULT="$HOME"/Dropbox/Notes
PDFDIR="$HOME"/Documents/PDF
PDFAPP="LivePDFViewer"
####################################

UNAME=$(uname);
case "$UNAME" in
	(*Darwin*) NOTE="$(pbpaste)" ;; # macOS
	(*Linux*) NOTE="$(xclip -selection clipboard -o)" ;; # Linux
esac;

FILE="$VAULT"/"$NOTE"

if [ "$1" == "-kill" ]; then
	pkill -f "entr $PGROFF" 
	exit;
fi

if [[ "$NOTE" == *".md" ]]; then
	PRE="${NOTE##*/}"
	BASE="${PRE%.*}"
	open -a "$PDFAPP" "$PDFDIR"/"$BASE".pdf || true
	echo "$FILE" | entr "$PGROFF" "$FILE"
else
	exit;
fi

Here, you have to adjust VAULT, PDFDIR, PDFAPP and PGROFF (for the compile script pgroff below) to your liking.

If you save it to e.g. /PATH/TO/obstogroff, make it executable (chmod +x /PATH/TO/obstogroff).

You now could just run that script in a terminal window everytime you want to start monitoring a file. Since it would be cumbersome to open the terminal manually, I set up a hotkey with skhd to run the script. You could also use other hotkey programs, or maybe Alfred or something similar on Linux.

cmd + alt + ctrl - g : /PATH/TO/obstogroff

Then I set another hotkey in skhd to stop the monitoring of the file

cmd + alt + ctrl - k : /PATH/TO/obstogroff -kill

Compile script

Next, we need the actual compile script. You can customize that to your liking with a lot of options. A basic version that enables citation rendering would be:

#!/usr/bin/env bash

#  pgroff

########## CONFIGURATION ###########
BIBFILE="$HOME"/bibliography.bib
CSLFILE="$HOME"/.pandoc/csl/file.csl
PDFDIR="$HOME"/Documents/PDF
####################################

FILE="$1"
PRE="${FILE##*/}"
BASE="${PRE%.*}"

pandoc "$FILE" -t ms --filter pandoc-citeproc \
--bibliography="$BIBFILE" \
--csl="$CSLFILE" \
-o "$PDFDIR"/"$BASE".pdf

You have to specify BIBFILE, CSLFILE and should use the same PDFDIR as above. Don’t forget to make it executable.

Example

My script looks like this:

#!/usr/bin/env bash

# pgroff

########## CONFIGURATION ###########
PDFDIR="$HOME"/Documents/PDF
BIBFILE="$HOME"/Dropbox/biblio.bib
CSLFILE="$HOME"/.pandoc/csl/foerster-gw.csl
LUAFILTER="$HOME"/.pandoc/filter/pandoc-quotes.lua
QUOTLANG="de"
GROFFOPT="-G"
TMPL="n.ms"
####################################

FILE="$1"
PRE="${FILE##*/}"
BASE="${PRE%.*}"

< "$FILE" ruby "$HOME"/.scripts/obsidianlinks.rb | pandoc \
	-t ms \
	--pdf-engine-opt="$GROFFOPT" \
	--template="$TMPL" \
	-M title="$BASE" \
	--filter pandoc-citeproc \
	-M quot-lang="$QUOTLANG" \
	--lua-filter="$LUAFILTER" \
	--bibliography="$BIBFILE" \
	--csl="$CSLFILE" \
	-o "$PDFDIR"/"$BASE".pdf

As you can see, you can add all kinds of options to the pandoc process. One of the extras I added is the following.

Make Obsidian links clickable

With this script (original from radekkozak) Obsidian links in the final PDF work and open the actual note linked in the Obsidian app:

#!/usr/bin/ruby

# Modified for Obsidian WikiLinks by @radekkozak
# Further modified by @phlind
# Original idea for The Archive @mjknght at zettelkasten.de

require 'uri'

VAULT_NAME = 'VAULTNAME' 

def class_exists?(class_name)
  klass = Module.const_get(class_name)
  return klass.is_a?(Class)
rescue NameError
  return false
end

if class_exists? 'Encoding'
  Encoding.default_external = Encoding::UTF_8 if Encoding.respond_to?('default_external')
  Encoding.default_internal = Encoding::UTF_8 if Encoding.respond_to?('default_internal')
end

begin
  input = STDIN.read.force_encoding('utf-8')
rescue
  input = STDIN.read
end

input.gsub!(/\[\[(.*?)\]\]/) do |m|
  match = Regexp.last_match
  if match[1].include?('|') 
    splits = match[1].split('|')
    "[#{splits[1]}](obsidian://open?vault=" + ::VAULT_NAME + "&file=#{URI.escape(splits[0])})"
  else
    "[#{match[1]}](obsidian://open?vault=" + ::VAULT_NAME + "&file=#{URI.escape(match[1])})"
  end
end

print input

Other options

With --template=/template.ms you can customize the ms template pandoc uses. The default one is here.

With --pdf-engine-opt=-G you can pass on options to groff (or actually pdfroff, which is used by pandoc). -G enables graphic processing with grap. So you could put something like this into a note in Obsidian:

```{=ms}
.G1
draw solid
1 20
2 40
3 120
4 30
5 70
.G2
```

Bildschirmfoto 2021-03-19 um 09.02.45

etc.

Usage

Now, when you are in Obsidian and have a note opened (and if you followed my hotkey examples), you could press Cmd + Ctrl + Alt + C and Cmd + Ctrl + Alt + G to start the compiling process, and Cmd + Ctrl + Alt + K to stop it.

Further possibilities

Compile whole vault

Theoretically, you could set this up to automatically compile your whole vault into PDFs and not just one file, since entr could monitor all .md files in your vault folder. I don’t do that currently though.

Alternative tools

Alternatively, if you can’t or don’t want to use entr for some reason, there are other tools to watch files and execute a command on a file change.

Get notifications

You can set up terminal-notifier on macOS—install with brew install terminal-notifier—(or notify-send on Linux) to notify you once you start monitoring a file and once you stop. In my case, the first part in the following examples is part of the kill command in the obstogroff script, the second one I put before the entr command:

case "$UNAME" in
  (*Darwin*) terminal-notifier -title "Obsidian to groff" -message "❌ Stopped compiling!" -remove OBSGR ;; # macOS
  (*Linux*) notify-send "Obsidian to groff: Stopped compiling!" ;; # Linux
esac;

...

case "$UNAME" in
  (*Darwin*) terminal-notifier -title "Obsidian to groff" -message "💜 Live compiling “$NOTE”" -group OBSGR ;; # macOS
  (*Linux*) notify-send "Obsidian to groff: Live compiling “$NOTE”" ;; # Linux
esac;

Respect directories

Note that at the moment PDFs from notes in subdirectories in your vault with the same name as notes in other directories or the root directory will overwrite each other in the PDFDIR. I will try to make it possible to generate different PDFs…


This outlines some basics. There are a lot of things which could be described in detail, e.g. how to customize the template for the PDF. I also would like to improve this to make it easier accessible. So, if you have any suggestions for improvement, I would like to read them.


  • 2021-03-21: edited to improve the scripts
9 Likes

It’s been a while since you posted this, but just curious: Why do you want to compile it to pdf? Inside obsidian, you can switch editor mode on/off independently in different “panels” (i think that’s the terminology). So you can keep a document open in two panels, and have one in viewing mode which would live compile the latex and stuff.

@derekxiao93 That’s true, it’s more of a fun project. But there are some advantages:

  1. It also compiles pandoc citations and the bibliography.
  2. You can use groff to draw graphs etc.
  3. You already have the final document ready, if you, for example, write an essay.

I wanted something like this as I write college notes and having them directly in A4 pdf will be very neat to have.
is it possible to get this inside a plugin that does the preview on a shortcut.