I had a look, mostly for what you got out of Datacore and how.
I was so impressed I hardly looked around thoroughly enough before coming back here to say thank you.
I’m glad you liked it! I’d be happy to answer any questions you have. Also, the ‘Page Task’ along with Datacore is pretty cool—you can even expand it to run different commands based on properties. The script that changes the icon when you press Ctrl + Space can be found in the Homepage and under SYSTEM > TEMPLATE > CODE.
I’ll probably start tinkering with it tomorrow (today) and ask you about stuff.
For now, just one thing: any major updates on your vault/GitHub repo, especially technical, use this thread to let people know.
On my hobby laptop, I tested your vault for performance. First with 15k files, then your own leaner vault.
I hesitated about being a busy-body about this, but thought I’d mention it anyway, mainly because I’m not a software engineer or web developer and I don’t know the answers.
If in your vault I remove the excluded files (actually, folders) in Files&Links and run a search for path: SYSTEM/TEMPLATE/FORMAT /dat, results are pretty slow coming in with all your CSS enabled.
Once I remove the larger CSS files from .obsidian (and there is also a CSS folder somewhere else), the 200-odd results come in instantly without any stutter of the progress bar in the Search Modal.
I’m just mentioning this because the hefty CSS files in .obsidian are obviously built upon previously made ITS and efemkay snippets and there could be something wrong with them. I suspect issues with the view.css files in the SYSTEM/TEMPLATE/CSS folder as well.
If you don’t see any issues, try adding 5-15k files and try the search.
I just tried much same search for 1586 results in my own vault with all your CSS removed and there was no sign of a lag or stutter.
Tasks also has issues with large vaults, apparently.
So far, based on tests, having Datacore on a large vault doesn’t cause problems.
thank you so much for this. I’ll work on improvements maybe over the weekend. These are very valuable observations. Any idea on where I could get realistic dummy data? I was looking to run test sets to measure performance metrics: 5k, 10k, …, 100k notes.
With this I could hopefully arrive with a solution for very large vaults using the template.
I already have something in mind with tasks and why they are slow on very large vaults but for now focus will be on getting that dummy data.
I never tried it, personally.
Then a regex replace Python script could be used to add YAML to all files, based on filename, creation time, etc.
I have also found that Datacore works with snake_case property keys well but not with camelCase…
Created an account here just to comment about how insane this vault is. really mindblowing. Playing with it today but just want to share my appreciation for the work here.
I’ve seen you finalized your vault and possibly do not take FR’s on your GH.
Can I ask here why in the Map of Content file one can edit the property (cell) values if the changes don’t persist and reflect on the given file’s frontmatter? (the mobile version of M of C doesn’t have this).
Are you planning to add this support?
Thanks
Compare DV implementation here - never tried it but this would be awesome with Datacore.
Sorry, that completely flew out of my head. Anyhow I built a fix for your concerns:
Properties are not being updated
Properties cannot be selected for mobile version
You can download the updated version of the Map of Concent and Mobile Map of Concent
Requirement:
Install and enable the MetaEdit community plugin.
Changes Made:
We created an updateNoteProperty helper function that connects to MetaEdit’s API (app.plugins.plugins[“metaedit”]) to handle the actual note updates.
We built an EditableCell component that manages the editing state and shows a confirmation dialog before making changes, using useState and useRef hooks to track values.
We modified the table rendering to use EditableCell instead of just displaying values, allowing users to click any cell (except file names) to edit properties, which then triggers the MetaEdit update process through our helper function.
Tried it.
Awesome!
This will really rock Datacore users’ experience.
I wonder though:
Why is MetaEdit needed when processFrontMatter is available from Obsidian API?
The confirmation pop-up is nice but could be considered unnecessary as one can re-edit the single cell value if some type was made. When making a lot of changes, this would hinder a good workflow. On pressing Enter, the change would be registered and a notice could be flashed in the upper right-hand corner to let the user know.
I am of course asking these on behalf of the wider community.
I am having issues with the Obsidian API, so I just resorted to Metaedit API which was a suggestion from this article. If you manage to find a solution with Obsidian API, would love to have a look at the code.
There is a small issue with the code, when you edit again the value displayed will be rendering the 1st Update value (but the note property will have the most recent update value). While it is better to just have the notif at the side, this will do as a quick fix to prevent users from making unnecessary/accidental updates. Will send here if I find a solution.
For those of you who are curious with the startup time, here is a breakdown:
If you’re not using todoist, removing the plugin will bring down the time to just 2 seconds
Excalidraw also takes time to load on startup
Note:
From my tesing Lazy loading can bring things down to 1 second startup time even with all the plugins but it causes issues with the Metabind buttons that are nested from tabs
Obsidian start-up time breakdown
Obsidian version: v1.7.7
Operating system: Windows 11
That’d be robot-made code, so I’ll pass.
I may try anyway when I get the time.
Just remembered that MetaEdit has not been updated for quite some time, but pretty sure it uses Obsidian API under the hood, so no biggie. Just want to cut down on installed plugins, is all.
I have found some plugins must be added as Instant loaded, e.g. to counter Commander custom-icons on mobile toolbar issues, etc.
Still, the majority of plugins can be loaded on startup.
