UUID for note links in URL scheme

[thomaskern]

At the moment the obsidian protocol is only (?) used to open files within a specific vault.

Example: obsidian://open?vault=Obsidian%20Help&file=Start%20here

I do not understand why this is done this way. As soon as the vault or file is renamed, the URL is useless. If used in third party apps (which seems to be the entire point of the feature), the third party app is now using broken references. The way it is currently implemented I would be very hesitant to use obsidian links in other apps, which i currently do a lot with Bear and Devonthink notes by adding them to Word docs, google docs, omnifocus, PDFs.

Sensible approach: use a uuid. Other apps are doing this already en-masse, especially in the apple world.

Examples: omnifocus:///task/oDd6JSYNi0N and x-devonthink-item://74995552-B379-4FD7-B0BA-6858E1F956E2

Even meeting folks half-way would be appreciated:
obsidian://open?vault=A&uuid=1389329sdflj2398

[EleanorKonik]

Based on discussion in the Discord, it seems like this would work if Obsidian established something like a .uuidconfig file where it mapped the files on your filesystem to this “internal” uuid, that Obsidian could then use to map that link to the file?

[thomaskern]

Yes, that could be a way. Renaming outside of the app is still a problem presumably but that is a problem with all links. But I don’t want to be too prescriptive.

[pointlessone]

YAML frontmatter can be used for storing UUIDs. Then ids won’t get stale even when files are renamed outside of Obsidian.

[thomaskern]

you mean in the (markdown) note itself?

[pointlessone]

Yes. Exactly.

[nickchomey]

This is crucial for me to be able to use obsidian:// links. I just started trying to implement a system for PDFs and cannot proceed given that external obsidian:// links will all break if the note/file is renamed.

At present, I seem to be left with the option of importing into a program like Zotero, Calibre or Onenote and then simply “attaching” files in Obsidian using their proprietary link protocols that use UUIDs:

• calibre://show-book/_hex_-43616c69627265/4239
• zotero://open-pdf/library/items/D255Z934)
• onenote:https://d.docs.live.net/[redacted]/&section-id={D7FB2C6E-716B-4F46-A75B-DC5AE8F4F58B}&page-id={A7C1AAFE-267B-4762-9911-46708705C96B}&end

This, of course, does not solve the problem of obsidian:// links for notes or files attached to the Vault breaking whenever a filename changes in Obsidian, its just a partial workaround for my particular situation that will surely break/fail me at some point. It also sacrifices the ability to preview the PDF within Obsidian, which is only possible for files embedded in the Vault, not linked externally.

So, yeah, I strongly agree with @thomaskern that Obsidian could benefit greatly from some sort of UUID structure. How this could be implemented and maintain the interoperability of Obsidian with other apps, I would have to defer to the wizards. But I hope that it can be done.

[fanantoxa]

Hi
One more use case to prove this idea may be valuable

Obsidian is great tool for knowledge management and researches.
With combination of Management tools (like Trello, Asana, notion) it can great power to rule the life on another level.

In order to do that it need to have ability to link Obsidian pages (or even nodes) in management tools.
Currently there is a way to copy link to the document and it will look like this:

obsidian://open?vault=Best%20Project&file=wonderful%20Note

One important problem with this approach that if vault or file is renamed there is no way to find it.

Obsidian allow easy create and rethink concepts so files could be renamed quite often.

With this in mind those link might not scale withing a time.
And in a long lasting usage in tasks management apps will lose those life saving links to data and recourses.

I was thinking about a way how to solve it.

I’m not sure atm if I should Make a feature request for it or try to make it work with plug-in.
Please check the idea and tell me which way to go.

Idea is here:

Add to vault, folders and files names unique UUID that will unique identify it but exclude them in editor.

So the vault/folder/name will look like on HDD:

Awesome vault-8d244168-9982-11eb-a8b3-0242ac130003
  |> Some note-90a8285e-9982-11eb-a8b3-0242ac130003.md
  |> Amazing folder-93f81190-9982-11eb-a8b3-0242ac130003
      |> Really cool note-96a1da98-9982-11eb-a8b3-0242ac130003.md

But in Obsidian will see those (and link) like.

Awesome vault
  |> Some note
  |> Amazing folder
      |> Really cool note

And all back-links will not include UUID’s
(that probably could be dome by simple regexp).

And in addition update links to use those:

obsidian://open?vault=Awesome%20vaultProject%2D8d244168-9982-11eb-a8b3-0242ac130003&file=Really%20cool%20note%2D96a1da98-9982-11eb-a8b3-0242ac130003

And update link handler to use UUID to search by both by Filename and UUID at the same time (when UUID’s feature enabled)

Pros

• Looking by both filename and UUID from link at the same time gives many opportunistic
  • Old links will be still relevant
  • if user remove UUID’s from filename new links will work by Filename
  • If original file is renamed there still UUID that will match so link is valid much longer
  • adding UUID’s can be done gradually at the time when User copy link
• If we don’t show UUID’s in the interface so user don’t get mess in names
• It won’t Don’t mess file order both in editor and file system
• Can be reverted by simple script in (PowerShell, bash, etc) if needed.

Cons

• File names in file-system will look a messy
  • But if User use them mostly in Obsidian it’s no that matter
• Links in GDrive/Dropbox will be broken in case of removing/adding UUID’s
• Maybe some tweaks for publisher required

I came from Dynalist and links there were helping me many times. This it only thing that prevent me from using Obsidian atm. But Obsidian looks so cool to use.

Please help me to chose path in which to go:

1. Create a feature request to implement this as a Code plug-in
2. Create a own plug-in that will implement this
3. Create a own plug-in and later that could be merged into Obsidian as Core plug-in

Which option should I chose?

[nickchomey]

@fanantoxa

Thanks for giving this so much thought!

My impression is that it would be undesirable to embed the UUID in the filenames. Both because it is messy, but more importantly because of interoperability, which is part of the core philosophy of Obsidian. If we use these files/folders with other apps - either at the same time as Obsidian, or after migrating away from it for whatever reason - the UUIDs will surely mess things up tremendously.

It was suggested above that a UUID could be embedded in the YAML - that seems to me to be the most preferable way to do this as it wouldn’t really create any clutter at all (it would even be “hidden” in preview mode) and should maintain interoperability with other applications and the file system.

As for your three questions

1. This thread is that feature request. I’m sure the Obsidian devs are focused on many other things right now, so this is unlikely to happen any time soon.
2. and 3. seem to be the same thing to me. If you are able to make some sort of plugin that could implement this functionality within YAML, I’m sure it would be tremendously appreciated by many people. It could also provide a starting-point for the devs to bring something into the core/official code, if that is something they decide to do someday.

[fanantoxa]

@nickchomey Hey! Thank you for response!

First thing I wanted to highlight with this proposal is how essential can be to have permanent links to knowledge.

When you do creative work and have many projects in parallel and many small task to work on it’d be impossible to get back and fix a link when you change them every time it happen.

Some folks at discord already mention YAML front matter

This is really nice way to tackle this problem.

Plugin API a bit restricted atm to make some thing work properly but at the end this is Javascript and I can try to hack around and see what need to make API better.

I’ll update this post if I come up with something!

Tnx for you’re response I really appreciate it

[nickchomey]

I agree that it is an absolutely crucial feature in order to make full use of the obsidian:// links. I’m glad you are willing to try to make something to solve the problem!

I wish I knew how to contribute to helping make this plugin, but I’m glad that I was helpful in some way! If you have any other questions, I will be happy to try to give some useful thoughts and I’m sure others will too. Good luck!

[davecan]

I have a dissenting vote to add.

Appending UIDs to folder and file names like that and not using them in links would mean Obsidian moves further away from the plain-text portability that is one of its distinguishing features from other products. (part of its Unique Value Proposition)

I for one would NOT want a tool that I am using to build a lifelong knowledge base, that is supposed to last longer than any particular tool, to break one of the most fundamental feature it has to offer – the ability to easily create unambiguous links between any two notes in the vault.

Once the UIDs come into play [[Note A]] could link to any of many (for the sake of argument let’s say dozens) of notes that happen to have the same name but unique IDs, and as soon as you try to move away from Obsidian and open your notes in another tool you may be SOL in figuring out which of those notes you were actually linking to.

[fanantoxa]

@davecan Yeah. After some discussion here we got to the same idea. Uuid’s in filenames will make to much mess and not backward compatible.

So current Idea is to store uuid in YAML front matter
That way filenames affected at all.

[nickchomey]

@davecan

I think everyone here already quite well appreciates (and have even already expressed) your concerns - lifelong knowledgebase, interoperability with other apps, etc… - thus UUIDs in filenames was only ever seriously considered by fanantoxa and he quickly realized that it wasn’t the way to go after being made aware of the previous discussion on the issue.

Moreover, it seems to me(/us?) that your concerns are good arguments for a well-thought-out UUID mechanism. As described by OP, the current external link format is utterly useless for interoperable-use outside of Obsidian since renaming files - as will inevitably happen for most people - immediately breaks the links. Any feasible workaround for this - e.g. a system-wide search/replace - is laughable and bound to fail due to user-error. It is for these reasons that UUIDs need to be not just a community plugin (which is perhaps a necessary starting point for the sake of expediency/proof-of-concept), but built-into the core functionality of Obsidian.

Of course, proprietary UUIDs in some hidden database, however they may be implemented - there are many apps mentioned above, like Onenote, Bear, Zotero etc… - cannot realistically be carried forward if we decide to migrate to another app for whatever reason. YAML seems to solve this problem nicely, as it does for metadata generally - the UUID information is just sitting there in plaintext, ready to be parsed and integrated into the other app (something that has to be done anyway with filenames and other linking formats).

Once the UIDs come into play [[Note A]] could link to any of many (for the sake of argument let’s say dozens ) of notes that happen to have the same name but unique IDs

This isn’t an inevitable outcome for UUIDs, especially if done through a plugin since Obsidian (actually, the filesystem) already blocks using same filepaths. If someday UUIDs were to be baked into the core of Obsidian, as should be done, it would have to still adhere to this unique filepath restriction. YAML makes this even easier.

I’m not smart enough to actually design the architecture for implementing this, but hopefully @fanantoxa can get us to a good starting point and if we can get enough support for this crucial issue, perhaps the Wizards of Obsidian will be able to polish it off.

Does this address your concerns? If not, what is missing still?

[davecan]

So I’m not opposed to the concept of a UID in principle, just that it not interfere with inter-note linking as it currently works. That’s all. I do understand the underlying concern about external links.

A UID for external links could be generated and stored in the internal index they already maintain as long as it is exportable to at least one format (e.g. JSON or CSV) when migrating to another app.

I’m not even sure the export would really be needed though, since another app would probably generate its own UIDs anyway. For example, why would Bear jump through hoops to import and honor Obsidian’s UIDs when they may be in very different formats? They would then be expected to do that for many such tools, as would the other tools.

So it should probably be expected that when switching apps then of course the UID would break and the links would need to be updated.

Which implies to me that UIDs could be supported and remain an internal part of the tool rather than exposed in YML where an end user can inadvertently tamper with them and break things accidentally.

[nickchomey]

The point of YAML, however, is that it is interoperable - that if anyone moved away from Obsidian to another tool, the obsidian:// links that are pasted elsewhere - word docs, task managers, zotero, etc… - wouldn’t break. All that would be necessary is a simple global find/replace for obsidian:// to the new protocol.

I agree that Bear or whoever is unlikely to implement that, but maybe that’s more reason to stick with Obsidian or for new apps to adhere to this sort of interoperability standard, which YAML headers seems to be a major part of.

Anyway, whatever the case, I doubt anyone here is at all opposed to it being an internal part of the tool, but I have low expectations of that happening any time soon. So, if @fanantoxa or someone else can figure out a plugin that can handle this in a reasonable way, with YAML seeming like the best way to do that (can a plugin access the internal index?), that would be a great starting point.

I see no reason why note linking couldn’t stay exactly as it is, but obsidian:// links use a UUID that points to whatever the note name happens to be. Again, this is precisely what all of these other apps do, for good reason. Its somewhat bizarre that this was not how Obsidian was designed from the start.

[msteffens]

IMO to achieve a robust note-taking system that can be used in various ways (and also linked to/re-used by external tools), it’s crucial to:

1. have a unique & stable identifier for each note
2. write notes that are as self-contained as possible

“self-contained note” means that the note’s body text (or its metadata) contain all relevant information (including things like citation info and, of course, the note’s ID) – ideally in a format that can be easily searched and/or parsed (see also this related zettelkasten.de discussion).

As also discussed in this reply, if the note’s ID is included in the note itself, it will always travel with the note and will not get lost if you transfer your note into environments where there’s no file path info, like a database, a single text file, or an app that stores files internally using its own naming scheme. Your notes would still remain intact and can be retrieved again w/o loss.

In the same sense, it’s also beneficial if the note ID gets included in the file name itself since this helps with third-party file-based integrations.

[nickchomey]

@msteffens

So, do you have a particular suggestion for how to implement a feature like this? Does a YAML UUID tag, as has been somewhat settled on here, suffice for you?

[msteffens]

IMO, the note metadata are a good place for the note‘s ID, e.g. like this:

---
id: NOTEID
---

Ideally, this ID would be provided by the app itself. But even w/o this I think it‘s good if each note has one, and other tools/plugins can build upon this.

On a sidetrack (about IDs): For my own use case, I wanted to base my note IDs on timestamp-based IDs (and not UUIDs). This is since timestamp-based IDs have the huge advantage that they can be generated manually, w/o the need for a machine. However I’ve wanted millisecond precision to allow for automated ID generation (e.g. on import) w/o the need for manual intervention. Still, the IDs shouldn’t get too long, and stay somewhat readable/recognizable.

To suit the above, my IDs take the note’s creation date (but any date will suffice) with millisecond precision in its reverse base32-encoded form (after Crockford). For comparison, here’s an example:

1) UUID:               9B5D722E-4C58-441F-997F-C086B097E2C7
2) shorter timestamp:  202104130930
3) longer timestamp:   20210413093059456
4) my ID based on (3): 1T5F-CGC0-8PQJ

The encoding keeps the IDs short while still maintaining precision. While these IDs are not as intuitive as pure date-based IDs, I find them more legible than the longer timestamps – the inserted hyphens & the 4char-blocks (together with the fact that it’s digits and letters) actually make them more recognizable for me. In any case, I find them more manageable than UUIDs and sufficient for a PKM use case. And since this system is based on timestamps, these can be also used (if notes are generated manually or externally).

[nickchomey]

Seems reasonable enough and in-line with what has generally been proposed here previously. I wouldn’t be opposed to a date-based ID, though I think I’d prefer something that looks like gibberish (1 or 4) rather than legible (2 or 3). I’m pretty sure Obsidian already handles metadata like Create Date - I wonder if that could be leveraged in some way?

Which tools do you use to implement this currently? It sounds like its not Obsidian