Make file metadata and properties separate concepts

Feature requests
1

I’m not sure how feasible this is or whether it even meets a real need. So please feel free to correct me if, for whatever reason, this is just a bad idea.

Use case or problem

The Properties core plugin is excellent. It’s versatile and can do a lot. However, there are two use cases where properties can come into conflict with one another because the data they seek to track is fundamentally about different things.

The first use case is one where properties are more-or-less the content of the note or very closely related to the content. For example, properties that track character traits in a TTRPG, the details of a movie or book (release date, star rating), or the different stages of a project.

The second use case is one where properties aren’t themselves part of the substantive content, but are about the content or about the file containing the content, such as the date the note was created, when it was last modified, etc. In short, file metadata.

For the second use case, that sort of information can be useful when it comes to journalling or when a note’s content is in flux and it’s beneficial to know when the last change was made. And as I recently learned, there are tools that help to automate the process, obviating the need to rely on the file system which isn’t a reliable way of keeping track of that information.

The properties plugin can accomodate both these use cases quite well but when both are implemented in a single vault it can get a bit messy, creating ambiguity about what a given property is about. Is it about the content (in fact, more less synonymous with the content) or it about the file?

Proposed solution

As mentioned above, no idea of feasibility or whether this would even meet a need; however, one solution might be to seperate Properties as a plugin from file or note metadata, so that Properties would fulfil a more content-centred function and file metadata could be addressed in a different manner, but similarly embedded in (or alongside?) the note.

To my understanding, there may be some issues with embedding metadata about a note within the note itself. Maybe that isn’t even desirable, given that, depending on the level of detail required (down to the second?), it would mean every change would prompt a further change within the note.

So maybe that information could be stored elsewhere (also in plain text), in a seperate file (hidden by default), but with the information it contains made visible within the note by toggling it on in the settings.

Because this separate file would exist alongside the note and essentially only serve to track file metadata, it wouldn’t need to be visible to users, but its presence might also solve problems where, when people move their vaults between file systems, they risk losing that sort of metadata when certain file systems reset the dates. (And if they’ve set up dates to auto-update with the use of a community plugin, this can be a particularly bad result!)

Related feature requests (optional)

Basically, any sort of feature that can allow users to be more confident that their file metadata will remain intact so long as they retain their Vault folder and which encourages users to not delegate the task of keeping this specific information within the note through the use of properties.

3 Likes
2

I like your idea, although the title should be more accurate like “make file metadata and properties separate concepts” etc.

I have previously suggested a new default property for note id: Default template for new note (cltr-n, click to non-existing note) - #67 by blue_emperor. It should be read only. Similarly we could extent this idea further for

  1. show / hidden properties
  2. read only properties that are implicitly generated

These two could be connected so that implicit file properties are hidden by default but it would be beneficial to have custom file properties that are hidden, although that kind of customization is already possible using CSS. So we don’t need anything regarding custom file metadata that can be already set to hidden. However we could benefit of having implicit read only file properties that Obsidian gives us automatically. The benefits are mostly for advanced users but it could be cool to see “note created” or “note modified”. You could then use these implicit properties in Dynamic view search queries.

3

Yes! That’s a better way of capturing what I mean: a clearer separation between file properties / metadata and content properties / metadata, which I believe would be best achieved by tracking file properties / metadata in a seperate (and, by default, hidden) plain text file that feeds the file metadata it’s tracking as input into the user interface when toggled on.

This would allow the properties plugin to retain its versatility and usefulness for more content-focused forms of metadata, while also giving users a more reliable way of tracking the provenance of the note as a whole, should they need to or if they find it useful to do so.

Yes, I imagine if this information were contained in a separate file and hidden by default it would have to be read-only, or rather, only Obsidian would be writing to it; the user would only call upon the information if needed—that is, if for some reason they wanted to extract or display the file metadata without embedding it into the file’s front matter as a property.

4

A little on the side, but have you considered moving your file metadata into inline fields (if that’s acceptable by your plugin combo), or hiding it using CSS?

Regarding the latter you could have a given cssclass to either reveal or hide a given set of your file metadata. This would mean that normally your file metadata wouldn’t show, but by changing the cssclasses you could make them appear again.

The same kind of trickery could be applied to online fields, and here you could also place them in a collapsed callout to reduce the visual clutter of the file metadata which I’m understanding as the essence of your feature request.

5

That certainly would address part of it. However, the other part is about the integrity of the file properties / metadata itself, which is a concern I believe others have raised in the context of moving their vault folders between file systems, resulting in the system changing the metadata.

What I’m proposing would, I think, prevent that issue from arising as Obsidian would retain file metadata within the vault itself, with each note having a corresponding metadata file that only Obsidian reads and writes to.

From the user’s point of view, nothing would really change, but they’d have the additional benefit of being able to call that metadata into a note if they for some reason required it. And if they did require it, they could be confident that, when moving their vault, the metadata being called upon won’t change because Obsidian isn’t relying on the file system when setting those values; it’s relying on the corresponding metadata file within the vault.

I’m not a developer, so I have no idea of what the costs / benefits would be to doing things this way. Maybe it would add too much bloat to a vault as there are suddenly twice as many files (one for each note), and even if the metadata files are small in size, hidden, and only Obsidian has read/write access, maybe the possibility of a user accidentally un-hiding them and editing the contents to break things in unforeseen ways would render the whole proposal untenable. I don’t know. But I’m hoping something can be done to create a clearer separation between properties / metadata that are entirely content-focused and ones that are really just about the file itself.

1 Like
6

I’m not really a technical person, but is this what Logseq does? I experimented a while ago with using Logseq over my Obsidian vault and suddenly these folders full of metadata appeared when I viewed them in Obsidian after having loaded them into Logseq.

7

I think he was referring to the bigger picture, not to only consider his own special use case. Should Obsidian export file metadata and if so how, using the document YAML block or something else? The implicit file metadata is already available using Dataview, the question is should Obsidian provide a setting to save that information into the YAML block or in some other ways. As @serif_type said the feature to auto export file metadata raises a question of the development cost and the potential outcome value of it.

1 Like
8

That’s interesting, @ryadaj. I just installed Logseq to see how that might be implemented there. I don’t know much about Logseq, but my understanding is that they have something similar to properties, with two different types: page properties and block properties. Depending on the details of how these are actually used, this may indeed be along the lines of the distinction I had in mind between properties that contain file metadata and properties that contain content metadata. (Speculating here (again, because I don’t really have any experience with Logseq), but that might be to facilitate some sort of version control functionality?? I imagine that, for such functionality to be implemented, you’d want to not rely upon the local file system for file metadata.)

9

It’s also possible I don’t know what I’m talking about, but thought to mention that example nonetheless since, as far as I can tell, I agree with the FR you’ve posted here :slight_smile:

1 Like
10

I think this can be solved by using more specific property names.

1 Like
11

Such as note_created_date? I agree that that’s a potential solution, and looking at the bigger picture I wonder whether this is a technical problem or a practical one. For instance, in the use cases I mentioned above, there’s no reason why the ambiguity couldn’t be resolved by simply using different vaults for each use case. Similarly, as you point out, if a vault serves multiple use cases, more specific property names may be called for. This would all be a matter of practice, rather than something that calls for a technical solution within Obsidian.

That said, I do wonder to what extent it can become a practical problem for some down the line if they decide to move vaults between file systems or use another application (mentioned here). Unless they’ve taken the initiative to keep track of note metadata in a rigorous way, whether within the note itself (perhaps as properties that automatically update using a plugin like Templater; e.g., <% tp.file.creation_date %> within a note_created_date property) or some other way, they risk losing that information.

Whether that’s a practical problem that perhaps calls for a technical in-app solution isn’t clear to me. I have heard others comment that when journaling, for example, they now ensure they put in a date stamp somewhere to obviate the above-mentioned risk. But it seems that a user would have to know about that in the first place to act to mitigate it.

1 Like
12

Addressing the epistemological problem you seem to be running up against @serif_type, this might make fertile reading for you on your journey of constructing a system of internally consistent reference: Rule-Following and Intentionality (Stanford Encyclopedia of Philosophy) . Hopefully there’s something here that resonates.

1 Like
13

Love it! Admittedly, when I saw the name Boghossian in one of the citations I assumed the worst. Thankfully it’s the other P. Boghossian—the one that’s actually contributed to philosophy.

14

Reflecting further on it, I think it may be asking too much for a technical solution here. I don’t have the development chops to imagine one but, more to the point, I think that as a matter of practice a tool like Obsidian calls for deliberate action; that is, for considering the purpose (or purposes) to which a vault will be used and tailoring properties (and other features, including plugins) to that. (This would be broadly in line with @holroy and @CawlinTeffid’s suggestions in this thread, which achieve some of what is detailed in the original feature request.)

That places a burden on the user, but it’s arguably a burden the user should bear if they want to fully realise the benefits of the software in their own use cases. The only downside is that new users might be expecting something else, particularly when it comes to how the file metadata is managed in-app, where it’s stored, whether it carries over between file system transfers, etc., and they may make usage decisions based on those assumptions. I don’t know what to do about that.

In general though, I am happy to leave this “feature request” (non-specific as it is) as a “nice to have,” but probably not essential, as many of the problems that it would address could be addressed at the user level, provided the user is aware of the issues and considers their implications in their own use case.

15

I am not 100% sure if this solves the problem but I had a similar situation when importing OneNote files and wanted to capture the created date from the file properties into the note itself in case I move the vaults from one computer to the other, I might lose this meta data.

You can see this github issue as I tackle a solution

Here are the steps I took:

  1. Install Templater plugin
  2. Create the /templater/insert file property.md and add the following script
---
Created at: <% tp.file.creation_date() %> 
Last updated at: <% tp.file.last_modified_date() %>
---
  1. Go to Templater Settings and enable Folder Templates, which basically tells Templater to run a specific template whenever a new Note is created inside that folder. Mine is OneNote which I use to import notes into.
    image

Unfortunately, although it works if I create a new note manually, the Obsidian Importer doesn’t trigger “note created” event, hence Templater doesn’t fire on each note created