Instapaper Highlights to Obsidian Notes Pipeline

I have recently upgraded my Instapaper account to use it more consistently for highlighting articles on my phone and laptop when I get the chance to work through the large backlog.

I wanted to bring highlights and short comments/notes associated with the highlights from Instapaper into my Obsidian vault to allow me to take further notes and reference article highlights in other Obsidian notes.

No Easy Solution

I looked at two different options but found that neither worked well enough.

IFTT
IFTTT has an Instapaper connector with triggers for both highlights and notes being added to an article in Instapaper.

Unfortunately, IFTTT applets does not allow for a text file to be uploaded to cloud storage, such as Dropbox, with a .md extension which is a must to have it included in my Obsidian vault without manual intervention. A file with other extensions can be uploaded to Dropbox from a URL which was not much help unless I wanted to go in a circle!

Integromat
Integromat has much better functions for uploading dynamic markdown content to Dropbox, but it does not currently have an Instapaper connector (one is planned).

So, I decided to create a process that used both!

The Process

The process involves both integration platforms IFTTT and Integromat with Dropbox as the cloud storage for my Obsidian vault. IFTTT is used to get the data from Instapaper to then send it on to a Webhook in Integromat which processes it and uploads a markdown note to Dropbox.

This should also work for other cloud storage providers such as Google Drive and OneDrive but I have not tested it with those providers.

IFTTT

Two IFTTT applets were setup with different triggers. Both send a request to the same Integromat webhook.

Highlight Applet

Screenshot 2020-10-24 at 21.17.10

Comment/Note Applet

Screenshot 2020-10-24 at 21.18.19

Integromat

I had previously used IFTTT but not Integromat. I found it to be very powerful, allowing me to create a scenario that not only added Instapaper highlights to my Obsidian vault in Dropbox, but also allowed me to created a new note with reference metadata if there was no existing note in my vault.

Steps

Here is a brief description of each of the Integromat scenario steps.

Webhook

The text/plain payload is received from IFTTT. I intially used an application/json payload which was easier to process. However, I came across an issue with the request from IFTTT to the Integromat failing JSON validation due to newlines in either the highlight or in the comment/note. IFTTT does have a feature to escape content using <<<Text to escape>>>> but I could not get that to work consistently.

Text Parser

The payload is parsed using separator characters. Here is the regex I used. I think this could be improved and it relies on none of the separator characters being present in the highlighted text or comments.

~~~(?<title>.*)~~~===(?<url>.*)===£££(?<highlight>.*)£££%%%(?<comment>.*)%%%

An issue that I had with the Regex text parser was that it would fail if there was no (?<comment>.*) capture group with the scenario running for a highlight. I worked around this by including an empty comment in the IFTTT applet for a highlight.

Highlight & Comment Markdown Converters

The highlight and highlight note/comment content is converted from HTML to markdown to remove newlines which were causing issues with the formatting of the content in a markdown blockquotes. These steps could be taken out depending on the markdown note formatting you choose.

Set Multiple Variables

Sets some dynamic variables:

File Path
The path of the target folder in the Obsidian vault on Dropbox e.g.
/Apps/Obsidian/Vaults/MyVault/Inbox

Note Creation Content
The initial content to use for the note in the Obsidian vault if it does not exist. This is what I use with the Article TItle and Article URL both being populated from the Instapaper data.

#instapaper
Source: [Article Title](Article URL)

---

Dropbox Download

Attempts to download the file from the File Path variable using the convention of the Instapaper article title as the filename e.g. /Apps/Obsidian/Vaults/MyVault/Inbox/Some Article.md.

If the download fails because there is no existing file, a Resume error handling directive is used to resume the scenario while setting the file data to be the Note Creation Content variable.

Router

A flow control module that splits the scenario execution into two branches, one for Instapaper Highlights and one for Instapaper Highlights with a comment/note.

The filters between the router and the two Dropbox upload modules filter the execution flow based on the existence of comment data in the payload received from Instapaper and IFTT. If there is a comment/note, the Dropbox Note executes. If there is no comment/note, the Dropbox Highlight module executes.

Dropbox Highlight

File contents are created by appending the highlight data from Instapaper and IFTTT to the contents of the existing file (or the Note Creation Content if the file is being created for the first time).

[[Instapaper Highlight]]
>Highlighted text

Dropbox Note

File contents are created by appending the highlight and comment/note data from Instapaper and IFTTT to the contents of the existing file (or the Note Creation Content if the file is being created for the first time).

[[Instapaper Note]]
Note text
>Highlighted text

Example Output

A screenshot of a note added automatically to my Obsidian vault with two highlights from Instapaper.

I am including the blueprint for the Integromat scenario so you can import and then configure the steps for yourself.

Happy integrating!

Instapaper to Obsidian blueprint.json.zip (2.9 KB)

8 Likes

Does this work for OneDrive too?

1 Like

The process would need to change for OneDrive because Integromat does not support downloading a file from a folder path as it does with Dropbox. A file id must be given. This means you cannot append existing files that have been created with a dynamic file name based on the article name in Instapaper.

One solution would be using a single markdown file in your Obsidian vault on OneDrive, named Instapaper Highlights Inbox.md for example. You can then get the OneDrive file id for that known file and use OneDrive modules in Integromat to retrieve the file and then append it with new highlights by overwriting it with the previous file content plus the newly added highlight content.

The URL for the article could be included with each highlight entry with a combination of an internal link [[Some article]] as well as the original external URL [Some article](https://example.com/some-article) to help you reorganise the highlights later.

Could this be adapted to work with Pocket? I find that Pocket does a better job of capturing images than Instapaper.

I’m fairly certain Pocket’s API is not as flexible as Instapaper’s and there’s not as many options to share/manipulate highlights like in IP. Highlights as a whole seems to be a second class citizen in Pocket :frowning:

Maybe this Pocket plugin for Obsidian can be useful to you:
nybbles/obsidian-pocket: Pocket integration for Obsidian

I think you can use this, too.