Properties: Do not strip hashtags in tags

I could tell! :sweat_smile:

I’m a big fan of the simple searchability of hashtags, so I understand not wanting to lose that.

3 Likes

I can imagine that the devs will come up with a solution for this, since there are several people having issues with the tag formatting. Maybe in the settings of the Property feature having the possibility to select different flavors of formatting the yaml.

1 Like

Personally, I don’t mind if Obsidian flags YAML as being invalid (in its opinion), but I don’t like the idea of it actually rewriting the YAML, potentially changing thousands of lines without review or approval.

---
tags: ['#meeting', '#work', third, fourth]
---
`$=dv.span(dv.current())`

Example of valid tags in 1.3.7


EDIT: FWIW, if a user wants to search for a YAML tag not prefixed with an octothorp (#), the line:() search option in Obsidian is good. An example:

line:(tags: dataview)

1 Like

Yes, this is unexpected. Touching the Live Preview Properties UI once reformats everything. I ran into this with quoted aliases.

CleanShot 2023-08-06 at 20.01.17

Not Obsidian’s fault that YAML (in Obsidian) is a mess and needs standardization… but this is jarring.

3 Likes

Thanks for adding the video; that makes it really clear what’s happening.

Touching the Live Preview Properties UI once reformats everything. I ran into this with quoted aliases.

I could even live with automatic reformatting of the YAML code but changing e.g. '#meeting' to meeting is not reformatting but actually changing the value, which is only obscured by Obsidian handling them the same (inside a list called tags).

  1. YAML is not a mess and is much better standardized than markdown. However, it is hard to write properly formatted YAML by hand.
  2. We have decided to take a more active role in the YAML management and preserving exactly what you wrote is non-goal of change. As long as what we rewrite is semantically the same, we are gonna be okay with it.
  3. Regarding this specific BR: yes, we intentionally introduced a semantic change. We deprecated the format “#atag” and when we encounter it we rewrite it without the hashtag.

Please, keep the thread in topic with rants about point number 3. Rants about point 2 should go in a separate thread.

1 Like

Use case or problem

I tested a number of separate approaches for “protecting” formatting on property values (even ones that have been moved into the lineated array syntax) when editing properties through the interface, and found that the only protected syntaxes are '"ex"' and "'ample'" , which for the purposes of tags at the very least are quite cumbersome. The tags will assume a “true” form of #"ex" and #'ample' respectively (henceforth “double-paired” strings).

Additionally, modifying any property or adding a new property will fully reformat all entries excepting those that are “protected” in the manner described above—yet if this is deliberately avoided through only modifying the YAML through Source Mode, single-paired strings will still be appropriately recognized as tags.

Proposed solution

Personally I’d like to see at least one of two options:

  1. The forced formatting should be reduced or made more permissive, such that single-quoted values may be preserved since they are already recognized by the app and function correctly even in the Catalyst build of v.1.4.0.
  2. The YAML Frontmatter should be made to either explicitly disallow single-paired values with regards to native functionality, such that the following would no longer be recognized as valid tag syntax for any native features of the app.
    tags:
      - "#ex"
      - '#ample'
    

Current workaround

The only present solution is to avoid using the Property interface and only edit the YAML Frontmatter in Source Mode or through external text editors / programmatic methods.

Related feature requests

None at present, will be updated as brought to attention.

Thanks and Miscellanea

This issue was opened following and initial observation by @mediapathic , and his insights and suggestions regarding this feature request should be taken with greater priority than mine since it directly affects much of his work.

3 Likes

@WhiteNoise, thanks for commenting but what you’ve written leaves me with some questions.

As long as what we rewrite is semantically the same, we are gonna be okay with it. […] Regarding this specific BR: yes, we introduced a semantic change, […]

That sounds to me like you agree that there is a bug that should be fixed.

[…] we deprecated the format “#atag” and when we encounter it we rewrite it without the hashtag.

This on the other hand sounds like it was a deliberate decision.

So, could you please clarify the current status of this issue? And if there is a decision either not to treat this as a bug or not to implement a solution, I’d be interested in a response to the criticism expressed in this thread and elsewhere.

Edit: As a meta-commentary to this bug report — I find it extremely confusing how these forum threads have now been merged and split and why this one is apparently scheduled to be deleted in 7 days. I understand that any additional system needs maintenance and focus but an actual issue tracking system would be preferable to this slight misuse of a forum.

3 Likes

My bullet points were an attempt to separate the different issues raised in this thread.

Again, specifically for tags, stripping the # is an intentional change not a bug (along with deprecating tag in favour of tags).
As of now, there are no plans of revering this change.

I am sorry for the inconvenience, there is also the FR that @CawlinTeffid posted, which may address what you are facing.

@WhiteNoise, so I understand this correctly — the criticism of this change is ignored, not even given the courtesy of a rebuttal, and to top that off it will be deleted from this forum?

there is also the FR that @CawlinTeffid posted, which may address what you are facing.

If you had read the previous posts, you would know that it doesn’t. I also don’t care for the stance “we broke it but intentionally, so it’s not a bug; perhaps ask for the ‘feature’ of getting it fixed.”

This encounter has left me disillusioned. I like most of what Obsidian has done in the past but using it always came with the risk of any proprietary software. Well, there is no license to cancel, so … have a nice day?

2 Likes

The reason I see the behavior as a problem is that it breaks any workflow outside of Obsidian that relies on finding tags in the files by looking for words preceded directly by a hash sign.

It does because you would be able to to put tags with a hashtag in lists and those would be picked up by your regexp.

It does because you would be able to to put tags with a hashtag in lists and those would be picked up by your regexp

No, you have not understood it correctly, and I’m tired of arguing in what at this point is just support theater (“move along, no bugs to see here”).

1 Like

So then those hashtags would be retained but ones in the tags field would be stripped? That seems confusing and inconsistent.

First, me pointing to that FR was not an endorsment that that feature request will happen.

Second, if that FR does happen (detecting tags in list items), then certainty the hashtag must be present otherwise we couldn’t distinguish a tag from a simple string (in the context of a list of items).

I wanna add that one of the reason for keeping the tags: property without hashtags is for compatibility with static sites generators like Jekyll and Hugo.

1 Like

But why enforce the change on everyone?

Why have all users got to have their YAML rewritten just to support those people who also use Jekyll and Hugo? Those people can learn to write for the platforms they use, can’t they?

From the Obsidian website:

Your thoughts are yours.
Obsidian stores notes on your device, so you can access them quickly, even offline. No one else can read them, not even us.

Your knowledge should last.
Obsidian uses open, non-proprietary files, so you’re never locked in, and can preserve your data for the long term.

These don’t correlate, IMO:

  1. Obsidian can’t read users’ notes but it does overwrite them.
  2. Users can preserve ‘their’ data and yet it is no longer ‘their’ data if it is arbitrarily edited by Obsidian.

These are questions. Just trying to make sense of what looks like significant contradictions between what is being claimed and what is being done.

3 Likes

I would think the compatibility with other text editors is the most crucial feature. There one would search for #foo . The stripping in YAML breaks this.
I don’t know Jekyll and Hugo… I can’t imagine they’re more important.

Additionally it was also the behavior I simply expected from Obsidian. That I would have to type tag:foo is surprising, cumbersome and not easy to spot (one would have to know that results are missing).

This also breaks Quicksearch (the CTRL-F search and the \ search via VIM-mode). None of them show the tag: result. I would consider that broken - unless users are supposed to consider several caveats and inconsistencies when using these basic features.

As for workarounds such as using comments: most importantly it’s counter-intuitive. The logical assumption for most users would be that this is exactly what the properties/YAML are for. Users would have to invent their own additional metadata-system. Either on top - cluttering the start of files, or at the bottom - which splits the metadata and is incompatible with other features. E.g. when one extracts headings to that file. They get put after these comments. One would have to fix this structure each time.

The simplest solution seems to me (as non expert) to make the list array - "#foo" the standard format. Or at least not strip them. It seems compatible with YAML.

PS: Just as a twist, I’ve seen tutorials using - " #foo", which enables auto-completion.

1 Like

“CTRL-F” is for finding text in a single note. Tags in Obsidian apply to whole notes, so there’s not much reason to implement tag search there.

From my point of view, all that’s happening is that the YAML is being tidied and systemised with an expectation that it will reduce the number of issues faced by users who write their own. The possibility of this happening was always there, as evidenced in my post from 2020.

In Python, the simplest solution to handle changing properties is using the JSON module.

To do this, you’d load the JSON module, read the properties from the file, change a value in the returned dictionary, and then use json.dump() to dump the changes to the file.

This is ultimately the simplest solution, which is why the developers went with something like this. I don’t think they considered users wanting to preserve the formatting of their YAML, that requires extra work to allow for that.

I’m expecting that whatever equivalent module they’re using for YAML, the module strips out hashes or treats them differently in some way, so they are ultimately removed in the yaml.dump() process.

If the user does something weird with the formatting, the code that reads the YAML is then expected to notice that pattern and repeat it. That sounds like a lot of extra work, so I can understand the non-enthusiasm for this. It would be very cool though.

It sounds like some custom YAML interpretation and writing is in order here, which ngl sounds like a fun project to work on, though is possibly outside the scope of Obsidian development.

1 Like

Ok, from that perspective then the search is broken, because it doesn’t show all results that are in the file one is searching in.

For the user this is getting too confusing. The quicksearch returns hashtags (#foo), but not tags, right where the user would expect them. So tags and hashtags are not the same … but then when using the other search they are (both tag:foo and tag:#foo return the tag. But then the normal text search doesn’t. The user has to notice by themselves that results are missing in some searches. Then they need to fix it, by inventing their own metadata format … despite having already metadata for that.

I’m not yet clear about the why - because using the quotes don’t break YAML rules, right? I suppose it’s to keep the syntax and implementation simpler? But at the cost of breaking the user expectation and ease of use? Especially for beginners that might not know workarounds.

Editors reformatting text surprisingly in the background also is tricky - especially for Obsidian which aims at Interoperability. I would err on the side of the user having had intend about their input (for Obsidian or external tools), and only modifying if’s invalid YAML.

4 Likes