Film or Recipe Database - future proofing for creating webpage: yaml or dataview?

I’m about to import an awful lot of film and recipe files into Obsidian, and I need to know if I should import the fields as yaml or dataview fields. I have two requirements:

  1. to display info in Obsidian preview mode; and
  2. to displaly on my own webpage at some point.

They seem mutually exclusive in a sense. (By displaying on the web I don’t mean using Obsidian’s “publish” function which I have but is too limited).

yaml

If Ii use yaml and I don’t want all that yaml to display in preview mode then I have to duplicate the info (via dataview or templater) to display it in Obsidian preview mode. I’m not about to copy paste all the yaml fields manually. So I would have to do it via dataview or templater. But if I do that, then converting the md file to html could be tricky because the content of the md file would be filled with inline dataview or non-static templater calls and I’m pretty sure that isn’t html friendly.

Here’s an example of a film page on Obsidian publish where I use dataview to display the data that is in yaml (the rest of the data was entered manullay even though it is in yaml - which is a pain and I have the info twice).

2021-06-29 yaml or dataview for future website

On the other hand I could ‘duplicate’ the yaml with (static) templater which might do the trick, but I would have to update the data manually in the page content everytime I edited the yaml.

dataview

If I use dataview fields throughout instead of yaml, then at least they display in Obsidian both in edit and preview mode (with :: all over the place, though), but I don’t know if I can use the fields in the future once I learn javascript to create html files. (I know it’s js based but can I call the values when using double colon as the key indicator, for example, can I call the runtime when it’s in this format: runtime:: 120 as opposed to being in yaml?)

This is a screenshot from an Obsidian published film page. It renders in html but I don’t know if I could access the data fields in other ways.

2021-06-29 yaml or dataview for future website 2


I’d really like to know how you are dealing with this issue (if at all), or what you would recommend.

I don’t necessarily have an answer but I would say that Obsidian is not best suited to this sort of structured data. Having said that, I totally understand that you may wish to do this because it does have many other strengths which might make it worthwhile.

The biggest question I have faced is if you import this data how can you ensure it’s up to date? How can you add new records? How can you insert (or delete) new fields? These questions become more important if you expect to make changes to the data inside Obsidian but still might like to import changes.

Possible solutions are:

  1. have a reliable way to import the full data set (this might include the option to change which fields become yaml or :: ) and never change the data inside Obsidian. Just reimport it whenever you want to update.
  2. add to the above a way to export the notes into a tool where you can update / modify in bulk easily and then reimport using the approach in 1.

In practise I have used a CSV format for the above - import the CSV creating one note per line in the CSV with column headers being the field names. Then export the notes out when required back into CSV where I can update the notes in a suitable tool like Google Sheets or Excel.

You can alternatively use something like Sublime which can edit notes in bulk but this is not as flexible I have found.

2 Likes

What are you using to publish?

If you plan to develop something, I’ve had very good experiences with 11ty, which converts the markdown into HTML, and uses the YAML data in clean and clever ways.

It has the concept of collections, allowing you to write a (paginate) table of all your films, for instance.

1 Like

@SimonW We have a similar way of getting the info into Obsidian. I exported my existing info into csv from a database I had, transferred to google sheets, added a bunch of columns (including the yaml — type separators) and then I use a script that converts each row into an md file just the way I like it, with headers and all. Most of the data with never change; I mean the film info like runtime and director, etc, doesn’t change so I’m good with that. The things that will change are my personal details, like viewing date, and reviews.

My most recent way around that (still testing) is to keep two film folders: one for the Film Titles (e.g., Die Hard, with the runtime, director, etc) and another for the Personal Details (e.g., Die Hard - Personal with stuff like my rating, viewing dates, reviews). The title page would include the personal details page via ![[ ]] (which is included in my Google Sheets).

That way if I ever have to re-import or change the format of the yaml for the main title page I can do it without interfering with all my other data.

The issue, of course, is getting that into html in the future. Pretty sure an md file with an internal link to the personal details wouldn’t display the personal details, which is what I want thte most, so I have to think on that. Nor am I particularly enchanted with having two pages for each film, but at least it seems like a workable solution, assuming I can get all the info into a webpage.

Edit → I forgot to ask: what do you use to " export the notes out when required back into CSV"?

1 Like

@cristian Thanks for that. I’ll check it out. I hope it is OS agnostic (I’m on Windows).

This is exactly the type of thing I need to know about. I haven’t even started looking at the web output part yet, and I really should because I’m pretty sure it will help determine how I should get the data into Obsidian in the first place! I mean that’s what is holding me back between yaml and dataview.

I wrote a very basic bit of python that simply strips out line returns, puts the text in quotes and separates fields with commas for each note. It then loops through all notes in the directory to create a full CSV file. Basically exactly what you have done in reverse.

With regard to any input inside Obsidian - I add that into a “notes” field at the end of the note in a free form structure (multiple lines). If I need to sync the data fields I do that in Sheets where I combine the new fields and my notes for reimporting (simple copy and paste of columns). None of this is ideal but it’s an OK work around.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.