Using Hypothes.is to quickly place notes into my digital notebook

Share & showcase
11

Inspired by all this, and wanting to see if I could come up with something that didn’t use IFTTT, I searched Github to see what kind of Hypothesis export scripts were out there, and I’ve cobbled together a workflow (blogged here ). What follows was created in the context of working on a Mac; PC will be broadly similar.

Basically, if you have python installed on your machine:

Get your annotations

Install ‘Hypexport’ from GitHub - karlicoss/hypexport: Export/access your Hypothes.is data: annotations and profile info . Install it with

pip3 install --user git+https://github.com/karlicoss/hypexport

Then, create a new text file; call it secrets.py and put into it your Hypothesis username and your developer token (which is underneath your username when you have hypothesis open) like so:

username = "USERNAME"
token = "TOKEN"

Save that file.

Now, you can grab all of your annotations with:

python3 -m hypexport.export --secrets secrets.py > annotations.json

Turn json into markdown

Now we need to turn that json into markdown.

( Incidentally, if you want to turn your annotations into a csv, get jq and run something like this

jq -r '.annotations[] | [.text, .tags, .updated, .uri] | @csv' annotations.json > annotations.csv

)

So, here’s a json to markdown script: GitHub - PolBaladas/torsimany: 💡✏️️ ⬇️️ JSON to Markdown converter - Generate Markdown from format independent JSON . Pip install that:

pip install torsimany

but then find where it’s located on your machine (search for torsimany.py) and change this line

data = f.read().decode('ascii', 'ignore')

to just

data = f.read()

and then run

torsimany annotations.json

at the command prompt (in the folder where you downloaded your annotations to), and after a bit you’ll have a file called annotations.markdown. (Incidentally, you can modify the torsimany.py script with your own preferences for how the different elements should be rendered as markdown, if you don’t like the defaults).

Split into individual markdown files

Last thing – we want to split that up into separate markdown files, to drop into the obsidian vault. cpslit, split, awk, etc, all of those things will probably work; here’s some perl (Mac’s have perl installed by default). Copy it into a text file, save with .pl, and if you’re on a mac, change its permissions with

chmod +x split.pl

so you can run it. Here’s the file (sourced from stackoverflow):

#!/usr/bin/perl

undef $/;
$_ = <>;
$n = 0;

for $match (split(/(?=### Title)/)) {
      open(O, '>temp' . ++$n);
      print O $match;
      close(O);
}

then run

./split.pl annotations.markdown

and you’ll have a whoooole lotta files you can drop into your obsidian vault. Ta da!

Fix the lack of file extensions

Now, you’ll have to add the .md file extension, which can be done as a batch with this one liner on a mac:

for file in *; do mv "$file" "${file%}.md"; done

and there we go.

So, not perhaps as elegant as using IFTTT, but you can see every step in the process. I’ll eventually get around to making this all just a single script, but I’m learning as I go. No doubt there are far more elegant ways of making this all work.

Also, it is possible to install your own hypothesis server so that you’re not part of the larger Hypothesis system; I intend to figure that out eventually and will post it on my blog if ever I do. I am personally satisfied with how Hypothesis treats user data, but I understand the concerns of folks who might like more control.

7 Likes