Recognise pandoc citations as Obsidian links

Use case or problem

A key part of many Zettelkasten systems is a special category of “literature note.” The essential usefulness of literature notes is that references to a specific piece of literature connect to the relevant literature note. In the case of Obsidian, this would mean that ideally, references to a piece of literature would link to the literature note, and that the literature note would display a backlink to the places it is cited.

The problem is that in academic work, the typical way to refer to a piece of literature is by a citation. The syntax for a citation is different to the syntax for a link.

The problem is not confined to literature notes — effectively, a literature note is a way of summarising an artefact which exists externally, and bibliography managers such as Zotero are able to track more types of artefact than simply literature.

Another category of artefact which often forms part of a note system is archived web pages.

Proposed solution

In org-roam-bibtext (an Emacs package), citations are turned into links, which if opened generate a new literature note. From then on, any other citations to the same item are a link to the original literature note.

This seems like an elegant solution. Literature notes should not be created merely for the existence of a citation. They should only be created when a user requests creation (e.g. a prompt when the citation link is opened).

The key element of a solution is first-class support for pandoc citation syntax as an Obsidian link: [prenote @CiteKey postnote] or @Citekey — see Pandoc citation syntax for the details.

I am not proposing support for citation rendering, just recognition of pandoc citations as valid Obsidian links.

Current workaround (optional)

The obsidian-citation plugin has all the mechanisms for building a Zettelkasten, but it has to choose between citing a reference (using Pandoc syntax) or linking to a literature note.

If Obsidian recognised Pandoc citations as Obsidian links, then the plugin would be able to connect citations back to the literature notes.

Related feature requests (optional)

This requests something similar, but is for actual citations. My feature request is simply for recognition within Obsidian of Pandoc citations as a first-class link to the appropriate note.

11 Likes

I cannot agree more.

1 Like

In my opinion there is an important distinction here between a citation and a link to a literature note, and I don’t agree they are the same.
Making pandoc style citations links should open the original reference, not my notes (often with my own ideas) about it.

Related: Zotero integrations - #44 by argentum

1 Like

You read my mind. Hence why:

3 Likes

What about: when a citation link is clicked, it offers options, which include “Open note”, “Open reference item”, “Open linked PDF”? That preserves the optionality while giving access to the literature note functionality.

The other key thing is backlinks from the literature note to all citations…

1 Like

How would that link be represented in the graph view?

In my opinion, linking and citing are two different things and there should be no ambiguity about which one is which. I wouldn’t use them interchangeably so right clicking doesn’t make sense to me, not to mention that it adds extra steps for opening the notes by having to right-click, making it inconsistent with all the other notes in the rest of the vault.
This also assumes the PDF is stored in the vault and can be identified somehow as belonging to that particular reference. All this makes the link resolution unnecessarily opaque I think, instead of knowing exactly (and explicitly) what I am linking when I write something down.

2 Likes

Interesting and relevant discussion! My workaround for the moment is:

[@AuthorTitleYEAR, 45-56]%%[[@AuthorTitleYEAR]]%%

This way you have a proper Pandoc reference and an Obsidian Link (if you need one). When I render my connected noted into a PDF (I use my own python script to aggregate the final draft from notes before it is sent to pandoc), the Obsidian link is removed and causes no issues, while pandoc references are processed and typeset properly.

Perhaps I am not understanding the goal here but couldn’t one use aliasing to create a link to the literature note that appears like a pandoc cite? A quick test worked for me:
[[test link | [@citekey] ]]

This renders in preview mode as [@citekey] but functions as a link to the page “test link,” which could just be named for the item of literature by whatever convention one desires. The alias could also include whatever pre/postnote, and it would be pretty trivial to fashion an Alfred/Keyboard Maestro/Autohotkey/etc. text replacement to automate the syntax a bit.

Generally speaking, I strongly agree with @argentum on the citation/literature note distinction. In published work, the citation is meant to point to the original source. I use it the same way in my notes and outlines. Having it instead link to my notes on the article seems inconsistent with the typical purpose of citations. Also, I could envision using pandoc-style citation links to particular pages within the item of literature itself, which would only ever be useful as links to the document.

1 Like

You could do this, of course - but that wouldn’t make any sense because the [@citekey] couldn’t be rendered anymore by pandoc, I guess - which is the only reason of using it.

I am quite sympathetic to this possibility, because from a citation in one of my note, I should be able to access easily both the source information and my comments on that work. However, I don’t think it’s incompatible with the view from @argentum and @WhiteNoise.

In preview mode one can add a link next to the bibliographical information to the literature note (if it exists). For edit mode, it’s fine to link directly to the literature note (if it exists): if one follows the proposal from the FR Support Pandoc-style citations with Citeproc-js engine @citekey [@citekey] and relative Bibliography, bibliographical information would be displayed in another pane or preview and there is no meaning to make a link to it. When exporting or using publish, links to literature notes should be omitted (for the reasons argued above).

1 Like

@melsophos I don’t think this addresses the main point: ambiguity between the original source and our notes when writing or reading. I really don’t think we should be using citations and our notes interchangeably. This solution would only remove the “extra steps to open a note” problem, but the main issue is still there.

I don’t understand why you see an ambiguity: if you make clear what is the source information and provide a link on your note on it, where is the ambiguity? Obsidian is supposed to help organize knowledge, which means, giving a work, being able to cite it in context and to write notes on it. When I am giving a citation, I also want to access my notes and ideas on this work. In your mind it’s the same: if you cite a work to someone, it also activates connections to what you know and think of this work, it does not come just as raw information.

And I have never said that citations and notes should be used interchangeably: what I say is that given a citation to something, it should be easy to link to any information I have written elsewhere (especially, in a dedicated note) on that something.

1 Like

At least from your proposal I don’t see a clear separation between notes and citations. An example of how you imagine this looks would probably be helpful.

This makes it sound interchangeable, but if it isn’t an example here too would make this clear.

Maybe this is also a way in which you can clarify how those are different.

@argentum : So let’s assume that the system implemented is the simpler and more conservative FR from @WhiteNoise (as I said in that thread, I would be already very happy if that one is implemented: with the ideas below, it would fit 95% of my needs).

So according to this proposal, bibliographical information is stored in a json or bib file managed outside of Obsidian. For example:

@book{Polchinski:2005:StringTheory-1,
  title = {String Theory: Volume 1, An Introduction to the Bosonic String},
  shorttitle = {String Theory},
  author = {Polchinski, Joseph},
  date = {2005-06-02},
  publisher = {Cambridge University Press},
  isbn = {0-521-67227-9},
  pagetotal = {424}
}

I can now cite this book in any note using Pandoc-style citation, for example some markdown would be (for the moment, forget about Obsidian UI):

# How to get a headache

A famous introduction to string theory is [@Polchinski:2005:StringTheory-1]. Blabla too much complicated maths [@Polchinski:2005:StringTheory-1, page 251], let's instead go for a walk.

Now let’s see how this could be used/rendered in Obsidian.

First, one can (optionally) create a literature note associated to this source, where one can write anything on the source (remarks, computations, errata, references to other works, etc.). Metadata are added/updated in the yaml frontmatter from the json/bibtex bibliography file, following some template (which also defines how filename are created). The important point is to use the citation key (without @) as one of the alias. For example:

---
alias:
  - Polchinski:2005:StringTheory-1
author: Joseph Polchinski
title: String Theory: Volume 1
date: 2005
---

# Polchinski - String theory: Volume 1 - 2005

## Erratta

Author's errata can be found at:
- https://www.kitp.ucsb.edu/joep/links/joes-big-book-string/errata

Currently, this is exactly what the citation plugin is doing.

Second, in Obsidian edit mode: the text [@Polchinski:2005:StringTheory-1] would be hyperlinked to the literature note above, if it exists. If it does not exist, clicking would create a literature note as explained above).
I don’t see to what else one could link: since the bibliography is defined in an external file. Information on the citation can be displayed in a side pane, as suggested by @WhiteNoise (it’s a fantastic idea). The only doubt is how hovering over the link would behave exactly: I would think that the best is to show first the bibliography information, then the preview of the literature note (if the option is enabled).
When starting to write @, auto-completion is done using information from the bibliography file (to account for sources which don’t have a literature note yet).
According to this point of view, the graph view would show a link between any note citing a source and the corresponding literature note (if it exists). I could also imagine a special type of nodes representing citations, so one node for each source in the bibliography file (the node name could be the citation key, and hovering over it would show the detailed information) – very much like the tags.
The point of all this is that I want to retrieve very easily any thought I had on the given source.

Third, Obsidian preview mode. The preview is generated by Pandoc and the list of references is put at the bottom of the file (following pure Pandoc rules). Moreover, if a literature note exists for a given item, the link is added in parentheses at the end of the line in the reference section. For example (I am writing the code in html to show that it’s a preview mode):

<h1>How to get a headache</h1>

A famous introduction to string theory is [1]. Blabla too much complicated maths [1, page 251], let's instead go for a walk.

<h2>References</h2>

<ul>
<li>[1] Joseph Polchinski. “String Theory: Volume 1” (2005). (<a href="Polchinski:2005:StringTheory-1">literature note</a>)</li>
</ul>

The point for which I am not sure of what’s best is what happens when clicking on [1]. A first option is to bring to the bottom of the page to the corresponding item in the reference section. From there, the literature note can be accessed. Another possibility is to show two links on hover, one to the bottom and one to the literature note. Yet another way is to add a symbol next to [1], which links to the literature note (for example [1]📰).

Finally, in export or publish, the result is pure Pandoc (so no link to the literature note, no additional text in the reference section, etc.) because only the citations and corresponding bibliographical matter. The content of the literature note is there only for private purpose and linking information from different notes together (which explains why I think it should be there inside one’s Obsidian vault, but not when sharing).

To summarize, Obsidian is before all a way to link information. For a given bibliographical element, this includes both citations and notes on the element. These are clearly different, but both are necessary to have a complete knowledge of the element in question.

2 Likes

Thank you! This made it clear where our workflows diverge and where the misunderstandings come from: “literature notes” mean different things for us. In your case they seem to contain metadata required for citing and/or perhaps links to other resources.

If I understand you correctly, you’re using literature notes mostly as a substitute for references here, right? And those references have related notes you’d like to see whenever you’ve cited them.

Here is another (very simple) example of a literature note:

---
alias:
  - Polchinski:2005:StringTheory-1
author: Joseph Polchinski
title: String Theory: Volume 1
date: 2005
---

# Polchinski - String theory: Volume 1 - 2005


## Notes 

The book was well written, but unfortunately doesn’t cover [[important concept]] in detail and its explanations about [[modelling Y phenomenon]] are limited to simple examples.

> This might be a quote from the book about string theory. 

The author does (not) mention that there are valid criticisms for [[possible universes]].
On my next paper this could be an #idea to explore, especially when considering the work I’ve done for [[black holes]]. 

> Another quote that I disagree with

And here I go about explaining why I disagree with this, these points are especially useful once I’m writing a paper and want to point out the gaps I’m addressing

> Maybe a quote that points out the main contribution of the author

And here I can have a summary of that in my own words, but I also add some criticisms or gaps I identified later.

> The author points out future work or limitations of their approach 

And here I can have ideas about how to solve it, perhaps using [[specific algorithms]] or maybe something in the state of the art by [@another author] or maybe something I also criticised in my notes about that [[another author]] paper.

## Erratta

Author's errata can be found at:
- https://www.kitp.ucsb.edu/joep/links/joes-big-book-string/errata


When you use in your text [@Polchinski:2005:StringTheory-1] you are referring to the original source (the things in block quotes), but not the notes you made about it. Linking to [[Polchinski2005StringTheory-1]] is useful to link to my notes, which are related to the original source, yes, but are not the original material. For this example above, showing both citation and a link to my literature note makes it ambiguous: to which of these two things am I actually referring to when I wrote that? Is it really a citation of the original source? Did I actually mean both of them? And it gets a bit complicated too, what happens when I share this with other people?

I understand that the idea is to have quick access to your notes when you’re citing, but I think we could find a way to do that without conflating the two, I particularly like this idea:

I still disagree that citing should automatically create a link between notes with citations and my own literature notes automatically, but it makes sense to do it through the tag-like node, e.g. between notes that use citations (e.g. places where I’m just summarising a paper are not about my notes) and the tag-like node, and on the literature note you could add the [@citation] to the blockquotes (if that’s the style of notes you make), or in a sentence (“This are my notes about [@citation]”).

Clicking on a [@citation] could open the search pane just like tags, and show you all notes that have cited this paper/book.
You could filter literature notes from notes with citations by adding tags or any of the other things supported by search operators that currently exist.

1 Like

In my citation plugin template I have added, at the end, the following lines:

## Quoted in:

```query
"@{{citekey}}" file:(-"@{{citekey}}") 
```

This makes sure you have an auto-updating list of all your notes citing the respective literature entry.

4 Likes

From what you say, I think that my example was not clear enough.

First, the metadata in the literature note are there only for having them directly visible when taking notes on the corresponding source. The metadata inside the literature note are populated from the original bibtex/json file, and similarly any citation anywhere in Obsidian would use the bibtex/json file and nothing from the literature note in order to generate the bibliography.

Your literature note is a perfect example of what I have in mind for a literature note and a much better example than my previous message. Indeed here we see comments, connections to other notes, etc. So what I have in mind is to be able to retrieve this easily from any citation.

So I think that there is no ambiguity in the following sense:

  1. If you are using [@Polchinski:2005:StringTheory-1], it’s a citation to the original source.
  2. If you are using [[Polchinski:2005:StringTheory-1]], it’s an internal link to the associated literature note.

Now in case 1), nothing prevents to add a link to the corresponding literature note. However, since the markdown contains only a citation, it is clear that the goal is to cite the original source (and not the corresponding note). Sharing would not contain any information on the literature note (because the markdown does not store anything). The point is to use Obsidian’s UI to add the link only for personal use, such that Markdown makes it clear what is the intended use.

I am not sure to understand completely the objection/explanation in your first paragraph: my impression is that for you citations should be disconnected from what you think of the source (let me know if I did not understand). My point of view is that any information in my notes on the same item should be connected, so if I have citations of a source and a note on the source, it should be directly linked.

Concerning the rest about tag-like node for citations, yes what you describe is what I had in mind. The only point is that I don’t want to go through the search pane to retrieve the literature note for @citation.

I just thought of another way to reconcile both our approaches: the decision of linking citations to the literature note (in Obsidian UI only) could be easily controlled through a parameter in the preferences. If it’s enabled, then it would work as I described above (which should not be too hard, since it’s mostly adding an if condition for adding or not links). All the rest (in particular, the tag-like system) would be identical in both cases.

Good. Then we agree on this.

We still disagree on this. Those two are distinct pieces of information, clicking on 1) and opening the literature note would be confusing.

I think the citation and the notes should be separate, not the citation and the source. And why we probably need to agree to disagree on this:

I’d still argue that allowing citations to behave as either tags or links would generate more confusion, but this might definitely be something worth considering.

I would say it’s not really as “either”: to me, it makes sense to always consider citations to behave as tags. Links would be an additional and optional layer. This would remove the possible confusion you mention in your previous post (if the option is disabled by default, people who enable it will know what to expect when clicking), and this would also reconcile both our point of views: people who want to link everything like me or @lyndondrake can have links, those who want to separate the information can also do it.

I agree that what I’m hoping for is that when I place a citation in a note, I’d like at least the option to have that link to my literature note. One reason is that when I’m reading one of my notes that includes citations of sources, I’d like to be able to jump from the citation to the literature note in question to explore further. This could easily be enabled in a plugin, though, with a hotkey to open the literature note.

Another is that ideally, the backlink list in the literature note would include all the citations. This latter is the real key for me, as the literature note then becomes a great index to all the places where that item is cited. I suppose another way to do this is that I could just write a script which walks through all my note files, and builds index notes for each citation.

2 Likes