Zotero -> zotfile -> mdnotes -> obsidian -> dataview Workflow

[jadoff]

@Cat

That setting is in extensions.zotfile.pdfExtraction.formatAnnotationHighlight
the default value for this is <p>"%(content)" (%(cite))</p>
I change that to <p>%(color_category) %(content) (%(cite)) <br></p>
Which tells zotfile to use the custom color tags.

Whenever I try to change these default settings and export to Markdown using mdnotes, the markdown files are never made. When I hit the option to export to markdown, I get the popup of where to save it and select it, but then nothing happens.

[Muu9]

How do you think this work flow compares to the one I found here?

[cosmogrammar]

Is it just me, or do all workflows utilizing Mdnotes with Zotero’s highlighting and annotation system in Zotero 6.0 generate some ugly html with the annotations?

Or is it the default Mdnotes template that’s causing this?

I am attempting to implement the (seemingly underrated / underdiscussed) Obsidian Zotero Integration Plugin to get around this, but I’m also having trouble making a template look decent with that.

[Cat]

This workflow doesn’t work with zotero annotations, afaik, only with annotations in the pdf itself. (@argentum could possibly address that better, since I use Juris-m which hasn’t yet updated to version 6).

As I see it, most people are switching to the integration plugin or the Alfred workflow. None of these Workflows is perfect, so it all depends on what your priorities are.

Mdnotes does a good job of separating colours and extracting text accurately with correct page numbers but fails at auto-updating with new annotations (you have to re-export, meaning loss of block ids).

Alfred works well with page numbers and extraction but fails at colour categorisation.

Zotero integration works well at automation, colour categories and updating new annotations but fails at page numbers (but that is apparently taken care of if you use the in-zotero annotations).

You could also join the discord server, where there’s a dedicated thread for the integrator plugin, along with plans for The Annotation Plugin to Rule Them All.

[AugustOvQ]

If you upgraded to Zotero 6.x, you will find that the “Extract Annotations” feature is done by the Zotero built-in functionality. See these two links

kb:zotfile extract annotations [Zotero Documentation]

note templates [Zotero Documentation]

[mikethechap]

I love Zotero and have used it for some years now. I don’t know that I’m going to link it to my Obsidian, but the presence of this functionality is mind-blowing to me. I also learned some things I didn’t know about Zotero.

[Skengman]

Hi all, any updates on this workflow to successfully pull color annotations as of 20220811? I have tried this with the settings given in Cat’s original tutorial and followed through the issues in the thread–nothing is working for me in pulling the colors of highlights. I am able to get all the annotations out no issue (right click on PDF, “Add note from annotations”), but I may be missing something fundamental in settings or there has been a software update in one or more components. If you have any ideas on how to fix, I’m all ears!

[cassiopyas]

Zotfile does not work with Zotero 6. However, Zotero 6 has many of the same functionalities as Zotfile ( see e.g. Zotero 6 compatibility by dstillman · Pull Request #573 · jlegewie/zotfile · GitHub).
I followed the workflow above until section 3.2 and here is my version of 3.2:

Extraction of Notes

When you have added your notes to a paper in Zotero 6 using their annotator, you can create a note with all annotations in the PDF by right-clicking on the parent item in the items list and select Add Note from Annotations. (see pdf_reader [Zotero Documentation])

Following this you can right-click on the parent item in the items list and select Mdnotes -> Create full export note to get the formatted output in Obsidian.

The Create full export note menu exports an item’s metadata and its Zotero notes as a single file. For that it uses the Mdnotes Default Template.md, which you should edit by adding your desired metadata placeholders like done in this tutorial. Zotero notes included in this export will use a new template you need to add to the same root directory as Mdnotes Default Template.md called Zotero Note Template.md. My Zotero Note Template.md only contains one line

{{noteContent}}

Formatting the extraction of notes

The customizing of annotations can be done using note templates in the Config Editor in Zotero, like in Section 3.1. A brief link on how the Config Editor customization works can be found here: note_templates [Zotero Documentation] . Now you need to change

• extensions.zotero.annotations.noteTemplates.title
• extensions.zotero.annotations.noteTemplates.note
• extensions.zotero.annotations.noteTemplates.highlight

3.2.2 Changing the Note Title

In Zotero’s Config Editor, you need to change the setting extensions.zotero.annotations.noteTemplates.title to change the title of the Extracted note. The default is <h1>{{title}}<br/>({{date}})</h1> but as I am not interested in keeping the date, I changed it to <h3>{{title}}<br/></h3>.

3.2.3 Editing the Annotated Comments

I want to highlight my annotated comments from parts of the main text that are highlighted. However, Mdnotes does not yet support italics or bold (In the Changelog (https://argentinaos.com/zotero-mdnotes/changelog/) this feature is listed under “Unreleased”).

In Zotero’s Config Editor, you need to change the setting extensions.zotero.annotations.noteTemplates.note to change the behavior of Notes/Sticky Notes
So for now I have changed it to <p>[[Note]] {{highlight quotes='true'}} {{citation}}</p> <blockquote><p>{{comment}}</p></blockquote> {{if tags}} <blockquote><p><b>Tags:</b> #{{tags join=' #'}}</p></blockquote>{{endif}}

Line by line:

• <p>[[Note]] {{highlight quotes='true'}} {{citation}}</p> This adds the link Note to this comment and tells me where in the document I added this sticky note.
• <blockquote><p>{{comment}}</p></blockquote>: This adds my own note as a blockquote to separate it from the rest of the main text.
• {{if tags}} <blockquote><p><b>Tags:</b> #{{tags join=' #'}}</p></blockquote>{{endif}}: This adds another blockquote with any tags given to this note.

3.2.4 Changing Underlines

This I could not find the setting for.

3.2.5 Highlight Colors

In Zotero’s Config Editor, you need to change the setting extensions.zotero.annotations.noteTemplates.highlight to change the behavior of the highlight.

This is a bit more limited than before as there as of now only are 5 colours in Zotero Annotator : yellow: ‘#ffd400’, red: ‘#ff6666’, green: ‘#5fb236’, blue: ‘#2ea8e5’, purple: ‘#a28ae5’.

My system currently is: yellow is a default highlight, red is notes I disagree with the authors, green marks interesting follow-up material, blue highlights things that confuse me, and purple are important conclusions.

For now I have changed it to this long bit

{{if comment}} {{if color == '#ffd400'}}<p>{{highlight quotes='true'}} {{citation}}</p><blockquote><p>{{comment}}</p></blockquote> {{elseif color == '#5fb236'}}<p>{{highlight quotes='true'}} {{citation}}</p><p> - [ ] {{comment}} #task</p> {{elseif color == '#ff6666'}}<p>[[Disagreement]]: {{highlight quotes='true'}} {{citation}}</p> <blockquote> <p>{{comment}}</p></blockquote> {{elseif color == '#2ea8e5'}}<p>[[Confusion]]: {{highlight quotes='true'}} {{citation}}</p> <blockquote><p>{{comment}}</p></blockquote> {{elseif color == '#a28ae5'}}<p>[[Important]]: {{highlight quotes='true'}} {{citation}}</p> <blockquote><p>{{comment}}</p></blockquote> {{else}}<p>{{highlight quotes='true'}} {{citation}}</p><blockquote><p>{{comment}}</p></blockquote>{{endif}} {{if tags}} <blockquote><p><b>Tags:</b> #{{tags join=' #'}}</p></blockquote>{{endif}} {{else}}<p>[[Highlight]]:{{highlight quotes='true'}} {{citation}}</p>{{endif}}

Again let us go through it line by line:

• {{if comment}}: If a text comment was added to the highlight, do some of the following
  - {{if color == '#ffd400'}}<p>{{highlight quotes='true'}} {{citation}}</p><blockquote><p>{{comment}}</p></blockquote>: If the color is yellow, just add a blockquote note.
  - {{elseif color == '#5fb236'}}<p>{{highlight quotes='true'}} {{citation}}</p><p> - [ ] {{comment}} #task</p>: If the color is green, add a to-do item with the comment and tag the item with the label #task. This makes it possible to do the neat Dataview extraction in Obsidian as explained at the end of tutorial.
  - {{elseif color == '#ff6666'}}<p>[[Disagreement]]: {{highlight quotes='true'}} {{citation}}</p> <blockquote> <p>{{comment}}</p></blockquote>: If the color is red, mark the comment as Disagreement and add the note in a blockquote.
  - {{elseif color == '#2ea8e5'}}<p>[[Confusion]]: {{highlight quotes='true'}} {{citation}}</p> <blockquote><p>{{comment}}</p></blockquote>: If the color is blue, mark the comment as Confusion and add the note in a blockquote.
  - {{elseif color == '#a28ae5'}}<p>[[Important]]: {{highlight quotes='true'}} {{citation}}</p> <blockquote><p>{{comment}}</p></blockquote>: If the color is purple, label the comment as Important and add the note in a blockquote.
  - {{else}}<p>{{highlight quotes='true'}} {{citation}}</p><blockquote><p>{{comment}}</p></blockquote>{{endif}}: If the color is something else (not yet implemented), then just add the note in a blockquote.
  • {{if tags}} <blockquote><p><b>Tags:</b> #{{tags join=' #'}}</p></blockquote>{{endif}}: All tags are added to a list
• {{else}}<p>[[Highlight]]:{{highlight quotes='true'}} {{citation}}</p>{{endif}} If the highlight contains no notes, mark it as Highlight.

Workflow

So the workflow becomes:
Add notes and highlights to your paper,
Right-click paper and Add Note from Annotations
Right-click paper and Mdnotes -> Create full export note

[sam_foucher]

Is there a way to remove the span decorators<span class="highlight" data-annotation="...."></span> around the content in the exported markdown? I understand that it is useful in zotero but it’s too verbose in Obsidian, there is also a <span class="citation" data-citation="...."></span> around each citation.

Thanks!

[Otolith]

You may already know… This issue is mentioned on MdNotes GitHub Issues, but I don’t think it’s a part of the standard code and I don’t know how to do it.

[lnilya]

I would like to offer a heretic question in this thread. Why use Zotero at all?

I can create citations - but i still end up using Obsidian for notes, Zotero for PDFs. And a very complicated process to link them. I found a much easier solution in my own workflow.

1 - Store all your PDFs in Obsidian, highlight in any app that reads PDFs (even zotero)

2 - Make notes in obsidian. I find it helps understanding if you don’t go into the PDF, but do it “fresh” in obsidian, with your own words.

3 - Use DOI as metadata in your notes. This way you can generate citations.

4 - A small dataview snippet can generate you a list of DOIs for any note that links to notes you created in 1.

5 - If you publish then you have to go to bibtex once, copy paste your DOI list and generate a pretty text with citations. That’s all.

I described it with some videos in this Twitter Thread

Burn me on the stake for not using Zotero but I find it so much easier not to use it at all. Prove me wrong?

[Otolith]

Not heretical at all, simply a matter of workflow choices, preferences, and needs. I already do most of what you describe using Obs with Zot. No need to crucify or flagellate yourself.

For example, a big problem I have with Obsidian as a pdf reader is of scaling. Zotero is design to do library management well and Obsidian is designed to do text notes well, which are related but different tasks. My philosophy tends to follow UNIX, ‘do one thing and do it well’. By all means have a different philosophy, but these systems are the foundation of all modern information systems for good reason.

[nhanjkl]

Do you still not use Zotero? Iirc I saw a few of your tweets about using Zotero with Obsidian?

[lottchen_p]

Hello, I am totally new here. I was trying to follow the workflow but my browser tells me it can’t download mdnotes or better bibtex as the files seem to be broken or something.
I’d appreciate any help, probably I overlooked something, I am no tech pro indeed.
Kind regards

[argentum]

Please don’t use Mdnotes anymore. I don’t have time to maintain it and it will be discontinued when Zotero 7 comes out of the beta. I wholeheartedly recommend migrating to Zotero Integrator.

[Feralflora]

I assume it’s because you are using Firefox. Zotero is Firefox based, so if you just click a Zotero addon, it won’t download, but instead, Firefox tries and fails to install the addon in Firefox.

You have to right click and then press “Save link as” or something like that.

Anyway, see @argentum’s message.

[lottchen_p]

Thanks so much to both of you! Can you recommend any introduction to working with zotero integrator? Is this then all I need or do I still have to install the other two plug ins? Please excuse my total lack of competence in these matters (so far).

[gino_m]

@lottchen_p

Can you recommend any introduction to working with zotero integrator?

Here you go:

Using Obsidian and Zotero to write more consistently

[mixpap]

Trying to rekindle this discussion, I don’t think that obsidian alone (as a reference manager without zotero) has a different philolosphy than “UNIX philosophy”. Library managers have a list of objects with metdata, as obsidian has (these objects are notes in this case).
Thats why zotero has also introduced a note tool, as also other reference managers.

I am trying to replace zotero with obsidian only together with a bib file, due to several issues.

1. just keeping up zotero uses at least 1 gb of RAM.
2. Most of the papers I cite are not pdfs I want to read a lot, so I do not to have to download them bloating zotero with tons of pdfs in the local storage. However in order to use them I have to.

So, why using two pieces of software with a memory/storage/time overhead when Obsidian with some very small modifications/plugins (metadata for files, citation management -something as zotero connector) can provide the same result?