Exporting with citations

[PedroGonzalez]

Thank you very much for this insightful reply!

[Feralflora]

Actually, I didn’t use the right Pandoc citation syntax in example number 4. The following is the right way to add a prefix to the citation to get the result you were after.

So, to get this result: “(see, for example, Jones, 2024)”
you cite it like this: [see, for example, @jones2024]

Which then becomes this in word:

In conclusion, all the examples are fully supported using the right syntax.

---

@holroy

It’s so demotivating to give an answer and see they don’t even read the answer, and instead repeats a question on something you’ve already answered.

Yeah, it really is @holroy. It feels pretty disrespectful to everyone’s time and effort. Of course, something can be misread or simply missed, but sometimes it just seems like people going tl;dr, even though some topics just require longer answers. Especially as you dig out from being knee-deep in misunderstandings.

Kudos to you too!

---

@PedroGonzalez You’re welcome!

[AlexG]

[/quote]

@Feralflora

First of all, congratulations!

And thank you so much for your endless patience!

Formatting references

You are right, the other formatting options do become possible when that front-code is added (it’s YAML, right?). I didn’t want to have to paste this code at the start of every file:

[quote=“Feralflora, post:19, topic:75428”]

---
zotero:
  author-in-text: true
  csl-style: apa
  sorted: false
---

So I did a lot of trial and error with the “Extra Arguments” box, and worked out how to get the same command in there:

--lua-filter zotero.lua
--metadata=zotero_csl-style:"apa"
--metadata=zotero_author-in-text:true

No spaces within the commands or it breaks

@Feralflora

So, to get this result: “(see, for example, Jones, 2024)”
you cite it like this: [see, for example, @jones2024]

Fantastic! You are a whizz. I would never have figured that out on my own.

@Feralflora

It seems that you only have one dash at the beginning. You need to have two… I suspect you copied it from @Mathunter’s message above, which a) is missing a code fence and b) probably had some form of (iOS) autocorrect that changed -- into an em dash.

That is exactly where I got it from. I’d been confused about the em dashes in place of hyphens (I’d been experimenting with trying each in turn)… but of course, it must have been an auto-format thing. That makes so much sense!

Where things went wrong earlier

@holroy

@Feralflora

I don’t say this to put you down, but this (and the number of other misunderstandings in this thread) tells me you would really benefit from reading a bit more carefully. Because I really tried to foresee these pitfallls and steer you around them, as you can see above.

Sorry for side tracking, but this is so true for many of the threads in this forum. It’s so demotivating to give an answer and see they don’t even read the answer, and instead repeats a question on something you’ve already answered.

This is actually something that I can speak to as I’ve literally taught an entire university class on it.

Feel free to skip if this isn’t up your alley. I’m not here to force feed anyone lectures; There is no quiz

Psychologists have a name for this problem: “The curse of expert knowledge”. It’s hard for experts to explain things to beginners, because they see intuitively obvious layers of meaning, context, and nuance in the words that they say. It is very difficult for them to see how their words don’t carry all this meaning for beginners and they can be very confused as to why the beginners struggle, or misconstrue their words in such obvious ways.

For this reason, companies that build commercial products (as opposed to gold-hearted volunteers who help the world for free) never let their engineers write their instruction manuals. The engineers will tell people to “adjust the upper flanges until they achieve a 3mm tolerance to the outer manifold”, and then will be utterly baffled when beginners don’t follow along with (what they see as) such clear and precise instructions. Companies with budgets employ technical writers to translate this kind of expert-speak into normal English. That might not sound like a necessary job, or even a hard one, but, unfortunately, it very much is.

Take your explanation above:

@Feralflora

each file format you can output to has its own command template. This enables you to customize the command for that specific format. For example, by adding the zotero.lua filter to the Word command, and only to the Word command, because that’s the only place it is relevant.

That paragraph is easy to parse into a clear meaning, but only if you start with a clear picture in your head of what a command template is, what a lua file is, and presumably also what pandoc does, and how those three things interact to translate documents. That context is immediately obvious in your mind, making the paragraph a clear read. But I’m still not quite sure what it means despite reading it three of four times. This isn’t because I’m dumb or being disrespectful, and it isn’t because you are bad at explaining. It is because it’s legitimately hard for humans to communicate across such different levels of baseline understanding. It’s a well-known phenomenon.

If the Enhanced Export plugin was built by a company with big enough resources (instead of being somebody’s labour of love), they would have spent a bunch of money hiring UX experts (User eXperience) to do dozens of hours of user testing: The UX workers would watch consumers try to interact with a series of prototypes, and then tweak the designs until they finessed away any points of confusion. It’s slow, skilled, and time-consuming work but it means you end up with far fewer people like me flailing through patient attempts to solve their problems. To be doubly clear, this is not a realistic ask of smart hackers who are putting tools together on their own, and then letting people use them for free! I realize this!

Please don’t take ANY of this as veiled criticism, or a demand that you do more - not unless someone starts handing you a budget to hire skilled helpers.

I just saw your frustration and thought it might provide a tiny bit of solace to know that we’re not trying to be disrespectful when we struggle with seemingly basic instructions. It’s to your enormous credit that you keep on rolling through these inevitable frustrations and find ways to help people anyway. It’s extra work for you (and the people you are helping), and kudos to you for that.

My hat is off to you, genuinely, and sincerely!

[Feralflora]

You don’t have to trial-and-error your way to success

@AlexG

You are a whizz. I would never have figured that out on my own.

Thanks! But this serves as an example of what I’m talking about:

You don’t have to figure this out yourself! You got the solutions already in the first response.

See, you could have just referred to the Pandoc citation syntax documentation, which I linked for you in my first message, since you were citing using the wrong formatting:

@Feralflora

See Pandoc’s citation syntax .

Now, I don’t know if you’ll consider this jargon as well, but here’s what those Pandoc docs say:

Citation items may optionally include a prefix, a locator, and a suffix. In

Blah blah [see @doe99, pp. 33-35 and *passim*; @smith04, chap. 1].

the first item (doe99) has prefix see, locator pp. 33-35, and suffix and *passim*. The second item (smith04) has locator chap. 1 and no prefix or suffix.

This is what I referred to when correcting my example for you. So, when you say you would never figure it out yourself, despite having been pointed to the same resource, how can I help but to think that you aren’t reading the necessary resources?

Hidden layers and responsibility

While I acknowledge what are saying about the layers of meaning that we take for granted, I think you also have to take responsibility to a degree where I can assume that you — as a budding Pandoc user — are attempting to familiarize yourself with the fundamental basics of Pandoc.

---

As for your specific example, first I want to emphasize this sentence comes at the conclusion of a lengthy thread. Any clarifications should have been sought much earlier.

Second, Pandoc is a command-line tool, so talking about commands shouldn’t be out of place, and if it is, Pandoc has a “Getting started” page that caters to you specifically:

This document is for people who are unfamiliar with command line tools.

Perhaps it’s the curse at play, but I don’t see what else in that sentence can be considered jargon, besides filters. File formats? Output?

Hope this breakdown helps:

• Pandoc converts text files from one file format to another.
• The output is the results of such a conversion.
• You use commands composed of so-called arguments to interact with Pandoc.
  • Enhancing export conveniently stores and executes these commands for you.
    • Enhancing export comes with default command templates with the most common arguments needed to convert from markdown to various other file formats.
    • For academic purposes (citing and bibliographies) we need to add some additional arguments.
• Pandoc filters are scripts that change the default conversion behavior of Pandoc, giving you a different output.
  • The zotero.lua filter we are using here adds code that enables Word to recognize the citations, as if they were inserted by the Zotero addon for Word.

There’s a limit to how far back we can go with these clarifications.

---

Returning to responsibility, I want to emphasize that at no point did you ask:

• “What is a lua filter?”
• “Can you clarify what an export template / command template is?”
  • “When you say the ‘word template, and not something else’ can you clarify what you mean?”
• “What is the dropdown of file formats in Enhancing export’s settings for?”

---

Framing: From expert jargon to motivated reasoning

Embracing the commentary on communication, I think we also have to consider the phenomenon of motivated reasoning, in particular identity-protective reasoning. Emphasizing the “curse of knowledge” frame is arguably identity-protective and alleviates feelings of cognitive dissonance after having done something “silly”, to quote you. It’s a well known phenomenon.

And the lack of clarification, à la the bullet list of definitions above, is as much a consequence of the time-constraints involved in debugging issues for others during one’s downtime (e.g. while on the toilet) — coupled with an expectation that the other party also does their part with the given resources — as it is a consequence of jargon blindness. Or maybe this is just my motivated reasoning at play…

Nevertheless, I do of course agree that there is plenty of room for improvement in the UX and user guide (or lack thereof) for Enhancing export. This is also due to a language barrier (note the plugin name).

With that said, I think with some self-reflection beyond the frame of “the curse of expert knowledge”, you’ll find that you’ll be more likely to get help in the future without someone as stubbornly persistent as me coming to your aid.

[AlexG]

That’s a very fair and thoughtful take.

I guess it’s about expectation management.

When I tried to pick up basic PHP to customize a website I knew I was in for weeks of YouTube tutorials, web manuals, and StackOverflow threads, and I was happy to do that. It was fun. It’s obviously an easier and quicker challenge to learn pandoc’s command line interface and configuration settings, but it’s also a less widely useful skill, and probably not a challenge that I would have signed up for knowingly.

When I picked up Obsidian I knew it was more customizable than all the slick commercial software that is fully UX tested for raw beginners, so I was certainly expecting a learning curve; Googling recipes and tweaking to implement them, etc. It was only when I stated setting it up for academic writing that I started getting sucked deeper into the pandoc rabbit hole. I was kind of hoping I could just look up a standard command or two, instead of having to learn an entire architecture with its own syntax (even if not a deeply complicated one).

So I never set out to become, “a budding Pandoc user”, or really wanted to be one… If I knew what the learning curve looked like faster I’d probably have cut my losses and just used something else for academic writing.

But here we are now on the other side.

So, all that said, your description of pandoc above is much MUCH more beginner friendly than the standard web manuals. Have you thought about posting it as a resource for beginners (other than a long way down the bottom of a discussion thread that few will want to wade through)?

[Feralflora]

@AlexG

But here we are now on the other side.

I’m glad we made it through! Thanks for clarifying your perspective.

And I agree that there is a need for a beginner-friendly Pandoc guide, tailored to Obsidian users. I actually started working on this a while ago, so this exchange was helpful in shedding more light on what needs clarification. If there are some specific explanations you think this guide should contain, then it would be helpful for me if you could share your thoughts on that.

[AlexG]

Heck, I’d be happy to help you edit it. I don’t know much about pandoc (but that might actually a useful perspective), but I do have some writing skills and I know a bit about UX.

If I were writing it my rough first draft would start with an outline something like this:

Setting

This guide is about writing documents that include academic citations - be that for journal articles, books, your own notes, etc. Making this work withing Obsidian is the easy part. This guide also covers how to export documents into Word (or other formats) that include correctly formatted citations. This is the more difficult part.

Other guides you will find online with “academic workflows” are about how to automatically create neatly formatted notes with Obsidian that contain information drawn from Zotero entries. This is not covered here.

What needs to be done

The basic problem here is getting at least four different things to all work together:

• Zotero has a system for tracking and formatting citations
• Obsidian writes text in markdown, which allows for some basic formatting (that we assume you are fmailiar with). It doesn’t have any native capacity for citations
• Word (or other format, like PDF) has its whole own syntax for handling formatting and doesn’t natively handle citations either.
• pandoc is a program that specializes in translating documents from one format to another. On its own t doesn’t know anything about Zotero either, but there are ways of telling it where to find all the required information

That’s a lot of moving parts and it takes a little bit of plumbing to get them all working smoothly together.

Warning: It will probably take at least an hour to work through all of the steps in this recipe. Possibly longer if anything goes wrong or you need more unusual tweaks

Step 1: Getting the supporting cast into place

Some of those programs need a bit of extra help to give them the tools to deal with each other.

• Pandoc Instructions on:
  • how to install it
  • How to find the path
• Zotero
  • Install better bibtex. Why? It gives more useful internal names to the documents it tracks, that make it easier for other programs to talk to it.
  • Instructions on creating a library file
  • Instructions on where to find the citation format file (for APA7 or whatever)
• Obsidian Install:
  • the enhanced export plugin to send citations anywhere else, and
  • the Pandoc Reference List plugin to show formatted ciatitions within Obsidian . We’ll tell you how to set these up below

Step 2 (optional): Setting up Better Bibtex

Various people have posted instructions for renaming keys into an academic shape (e.g., Jones_1998 whatever). This isn’t required but does make them more intuitive to work with, so is recommended

Step 3: Setting up Obsidian Pandoc reference list plugin

• This formats references when they are presented within Obsidian.
• NOTE: This has nothing to do with them being formatted when you export them to any other format. You can skip this step if you don’t mind the references looking a bit funky when you are working within Obsidian
• I can’t remember where I found the recipe for htis, but it was relatively straightforward. You have to look up the path for where pandoc is, and past it into the right box IIRC

Step 4: Setting up Obsidian Enhanced Export

This is the fiddly bit.

Pandoc is the program that does all the heavy lifting here. Working with it directly takes some skill, so the Export plugin was created to give it a (somewhat) more user friendly interface within Obsidian.

The problem is that even with this more user friendly interface it still takes some doing to get it to coordinate between Obisidian, Zotero, and Word. Doing so still requires finding the right set of commands to feed it. The good news is that we will help you find those commands, and show you where to put them into the plugin so that it remembers you for them, and automates it every time you want to export a new note.

• pictures of which tick boxes need to be ticked how

• It’s best to leave the boxes full of detault text alone unless you really know what you are doing.

That “Extra Arguments” box is where we are going to tell it the commands necessary to manage citations.

The rules for these commands (“arguments” in tech-speak) are pretty simple.

• You can enter as many commands into that box as you like
• Each new command is separated by a space
  • if you want to write yours out in a text editor it will be a lot more human-readable if you separate them with line breaks - those will turn into spaces when you paste them into the Extra Arguments box
• Each command starts with two hypthens, and then can’t contain a space within it (unless it is text like a file address that is encolosed within “quotation” marks)
• It doesn’t matter what order you enter your commands (right?)
• you will have quite a few commands. Don’t panic about this.

Option 1: The basic way

The goal here is to tell Word (or whichever) what a fully formatted output should look like. You need to tell pandoc where to find a full description of your output format, and then give it separate commands to create the in-text citations, and another one to create the bibliography at the end.

• List of required commands, common options, and a demo of what the paths look like in Windows and Linux

Option 2: The fancier way (that let Word and Zotero work together to reformat documents)

MS Word has its own plugin for Zotero that lets you put special placehoulders for citations into your document. If you do this you can ask Zotero to translate the entire thing into whatever format a given journal requires, and you can repeat that translation every time you submit it to a new journal with a different format. That’s pretty handy.
You can ask Pandoc to create a Word doc with these placeholder values. It’s just slightly more complicated.

The bad news: the commands are so complicated that they require dozens of lines of commands sending to pandoc
The good news: Someone has complied most of those commands into a document that you can place in a file called a “lua filter”. All you need to do is point pandoc at this filter file, and it will do the hard work for you.

• Instructions on downloading the lua filter, where to place it,
• required and optional formatting commands

Step 5: Citing with this new setup

• Instructions on entering the various permutations into the Obsidian document [see @Jones_2024], etc
• If you use the option above with a lua filter, then when you have exported into Word it will look like X, at first, but then you click this button and everythig is tidied up.

---

Ok, that was a lot more involved than I intended to start out writing. A lot more involved.

If you do want to cowrite / edit this thing then is there a way you can send a private message and we can work offline? I’m not sure this forum is the most useful place for long-form editing work.