Zotero -> zotero2readwise (python library) -> Readwise Workflow

This is about Zotero2Readwise Python library I just released that retrieves all Zotero annotations and notes (using the new Zotero PDF editor and Note Editor), and uploads them to Readwise. The workflow I’m suggesting here is as follows:

Zotero → zotero2readwise Python library → Readwise → Readwise Official plugin

Please NOTE that this is NOT a replacement for the other workflow described in
Obsidian Forum: Zotero → zotfile → mdnotes → obsidian → dataview Workflow.

The main difference here is that the new Zotero (available in Windows, MacOS, and iOS (beta)), you can annotate PDF files without actually saving to PDFs.

The new Zotero PDF Reader and New Note Editor (see Zotero announcement) makes it much simpler to annotate PDF files even on iOS devices (currently in beta).

Important features

  • Tags from a Zotero annotation or note is also passed to Readwise.
  • Updating highlights is also supported. So, if you updating an existing Zotero annotations/notes, the corresponding Readwise highlight will also updated.

You can checkout the library README here to install the library and run it.
You can install the library using pip install zotero2readwise from a shell terminal (refer to README for more instruction).

Minimum Requirements

  • Readwise Access token (from here)
  • Zotero Key (from here)
  • user ID (from here if it’s personal library. If it’s a group library, please check the instructions in here)

Usage

After installing the library, you can do all the conversion in one of the following two ways:

Approach 1 (Run a python script)

After installing the zotero2readwise, download the run.py file from the library GitHub repository, and then just run
python run.py <readwise_token> <zotero_key> <zotero user/group ID>.
You can get more information on how to run the Python script by simply running

python run.py --help

For example, in my case,

Approach 2 (through a Python terminal)

This approach is more flexible as it also allows you to save any failed items.

Feel free to post any suggestion or bug.

1 Like

I’ve developed an automation procedure to run above library on a timely schedule.
You can now use the GitHub repository Zotero2Readwise-Sync to automate Zotero2Readwise sync on a timely schedule (e.g every day) using GitHub Actions tool.

Basically, you can fork Zotero2Readwise-Sync repo and add your Zotero and Readwise tokens to GitHub Actions secrets (the instructions are available in the README). You don’t need to do any other configurations or manual step. Just set it once and you can sync all your zotero to Readwise.
And by using the Official Readwise plugin, all Zotero highlights should be syncing to your Obsidian vault.I’ve developed an automation procedure to run above library on a timely schedule.

Nice, thanks for developing this! A suggestion/idea (maybe a bit redundant with GH workflow existence, but potentially more user-friendly): create an Alfred workflow to do the import (that would just invoke the library and do the sync)

For the life of me I can’t figure out what is going wrong. Admittedly a neophyte at this, and have no expectation of anyone taking the time to dig in, but if you feel like helping here is what I am seeing:

Retrieving ALL annotations from Zotero Database. 
It may take some time...

Traceback (most recent call last):
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/pyzotero/zotero.py", line 411, in _retrieve_data
    self.request.raise_for_status()
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://api.zotero.org/users/bvendsel/items?itemType=annotation&format=json&limit=100

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run.py", line 56, in <module>
    zt2rw.run()
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/zotero2readwise/zt2rw.py", line 45, in run
    zot_annots_notes = self.get_all_zotero_items()
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/zotero2readwise/zt2rw.py", line 36, in get_all_zotero_items
    annots = retrieve_all_annotations(self.zotero_client)
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/zotero2readwise/zotero.py", line 233, in retrieve_all_annotations
    return zotero_client.everything(zotero_client.items(itemType="annotation"))
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/pyzotero/zotero.py", line 178, in wrapped_f
    retrieved = self._retrieve_data(func(self, *args))
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/pyzotero/zotero.py", line 413, in _retrieve_data
    error_handler(self, self.request)
  File "/Users/buckley/opt/anaconda3/lib/python3.8/site-packages/pyzotero/zotero.py", line 1643, in error_handler
    raise ze.HTTPError(err_msg(req))
pyzotero.zotero_errors.HTTPError: 
Code: 500
URL: https://api.zotero.org/users/bvendsel/items?itemType=annotation&format=json&limit=100
Method: GET
Response: An error occurred

I’m pretty sure this error means that Zotero’s server is malfunctioning. I skimmed their documentation and tried a couple of other API endpoints with my Zotero username in my browser and some of it is returning responses other than 500. So their API isn’t down entirely, but it doesn’t seem to want to answer items queries (or collections queries). If today is your first time seeing this error, maybe try again tomorrow and see if Zotero’s server is happier then? If not, maybe someone on the zotero2readwise GitHub repository knows a workaround for the Zotero errors.
Good luck, this is a frustrating type of error!

Hi there,
I love the idea of being able to sync my notes on zotero with Readwise (and in turn with Obsidian) but–and I hate to be this person–I am completely lost. I tried running the python script but I know absolutely nothing about coding or about python for that matter and I am so lost. I entered my readwise token, my Zotero API and library ID in the body of the .py docs in the designated location. I think I got it right but I’m not sure.

Capture d’écran 2022-08-04 à 17.18.10
Capture d’écran 2022-08-04 à 17.19.15

And this is what happens when I try running the init.py file in python
Capture d’écran 2022-08-04 à 17.20.28

I’m sorry for being so tech-challenged.

No worries, I hope this is still helpful. First, make sure you (1) already installed Zotero2Readwise (via pip install zotero2readwise on your terminal), and (2) have found your Readwise token, Zotero key and Zotero ID as instructed in the repo here. Then all you need is to do is (a) download the run.py file here. You can put it in the same directory from where you installed Zotero2Readwise. Once that is done, all you need to do is run a command in the form of python run.py you-readwise-key your-zotero-key your-zotero-id. That will do it!

Thanks for the great work on this!

It seems to be creating duplicate items in Readwise for me. Any idea how to fix it?
I’ve opened an issue here: Duplicate items created in Readwise · Issue #5 · e-alizadeh/Zotero2Readwise-Sync · GitHub

Look at this solution, this is a known issue After the last release all my articles from Zottero got duplicated · Issue #31 · e-alizadeh/Zotero2Readwise · GitHub

Thanks a lot @PR-C, that fixed it. My mistake was looking for an existing issue in Zotero2Readwise-Sync Github instead of Zotero2Readwise.

Another question: anyone knows if it would be possible to pass the citekey from Zotero to Readwise, so it could be imported into Obsidian afterwards?

1 Like

is it possible to install it via Windows OS?