Obsidian Upkeep Note

I created a note in my vault meant to help me find problems in the vault. So far I have a section for finding “Dangling Links” (links to pages that don’t exist), and “Bad YAML” (frontmatter that is empty, missing, or invalid). Both are based on the kind contributions of other forum members. Do you have any similar utilities that might be useful to add? Here’s my note contents if you’d like to try the same:

---
filename: Obsidian Upkeep
aliases:
  - Maintenance
  - Utility
  - Troubleshoot
---

# Obsidian Upkeep

## Dangling Links

```dataviewjs
const unresolved = {}
Object.entries( dv.app.metadataCache.unresolvedLinks )
  .filter( ([origin, unresolvedList]) => 
    Object.keys(unresolvedList).length && !origin.startsWith("Meta"))
  .forEach( ([origin, unresolvedList]) => {
    Object.entries( unresolvedList )
      .forEach( ([newNote, count]) => {
        if ( !unresolved[newNote] )
          unresolved[newNote] = []
          unresolved[newNote].push( dv.fileLink(origin) )
      })
  })

dv.table(["Non existing notes", "Linked from"],
  Object.entries(unresolved)
    .sort( (a, b) => (a[0].toLowerCase() < b[0].toLowerCase() ? -1 : 1) )
    .map( item => [dv.fileLink( item[0] ), item[1] ] )
  )
```

## Bad YAML

```dataviewjs

const result = []
for (let [fname, fcache] of Object.entries(dv.app.metadataCache.fileCache)) {
  // fname is the filename
  // fcache is the entry from metadataCache.fileCache
  // mcache is the actual metadataCache.metadataCache entry
  const mcache = dv.app.metadataCache.metadataCache[fcache.hash]
  
  if ( (mcache && !mcache.frontmatter &&
		!fname.startsWith('Meta') &&
        mcache.hasOwnProperty("sections") &&
        mcache["sections"].some(s => s.type == "yaml"))
     || ( mcache && mcache.frontmatter && mcache.frontmatter.notvalid ) 
     ) {
    const yamlIndex = mcache["sections"].findIndex(s => s.type == "yaml")
    let yamlStart, yamlEnd
  
    // Pull out start and end, if section is found
    if ( yamlIndex !== -1 ) {
      yamlStart = mcache["sections"][yamlIndex]?.position.start.line
      yamlEnd = mcache["sections"][yamlIndex]?.position.end.line
    }
    
    // Determine the cause of the faulty frontmatter
    let cause
    
    if ( mcache.frontmatter?.notvalid )        cause = "Not valid"
    else if ( yamlIndex == -1 )                cause = "NO yaml"
    else if ( yamlStart == 0 && yamlEnd == 1 ) cause = "Empty"
    else cause = "Bad"
    
    result.push([dv.fileLink(fname), cause, yamlStart ?? "", yamlEnd ?? ""])
    // console.log(fname, " » ", mcache) 
  }   
  //console.log(fname, " » ", fcache, "\n  »» ", mcache)
}

dv.table(["File", "Yaml status", "Start", "End"], result)
```
4 Likes

Slightly different use case, but I have a small number of “detector notes” that have a bunch of aliases in each. I use them to sweep up notes that have key phrases in them that I want to avoid appearing in my writing/thinking/note-taking. For example, these are the properties for the “alarm bell phrases” one:

---
tags: [ detectonote ]
created: 20230425.1156
alias:
- is it possible
- is this possible
- its always true
- "it's always true"
- can we just
- nobody does
- noone does
- everybody does
- everyone does
- we already know
- make it visible
- just need to
- just simplify
---

I have others for emotive phrases, wishing, just whatever really. By going to one of these notes and looking at the unlinked mentions, I can find the notes in the vault that have used those phrases and then re-work them as required.

Saying all that, what you’ve done here is a cool idea along the same lines, and I’ve added it into my folder of detector notes - thanks :slight_smile:

3 Likes

Interesting idea, but I’m not understanding how it works. What are ‘unlinked mentions’, and how are they displayed on the page?

Given the following text in a note:

Two fruits that I really like are [[apples]] and bananas.

[[apples]] is a linked mention, clicking the word apples will take you to the note titled apples.

However, if there is also a note in your vault called bananas, then there is an unlinked mention (not linked, but unlinked) between the bananas note and your text. You need the core plugin “Backlinks” enabled, and then Obsidian will show it to you in the sidebar, or at the bottom of the note depending on your settings in that plugin.

Screenshot here shows the bananas note, and you can see the unlinked mention to the “Fruit I like” note on the right side.

Have a look at Aliases - Obsidian Help and Link notes - Obsidian Help in the Obsidian Help for a bit more info around this area.

1 Like

Wow, I have the plugin enabled, just never knew about the ‘Unlinked Mentions’ part. I just took a look at it, pretty cool, thanks!

You may also be interested in the community plugin, Pending Notes: GitHub - ulisesantana/obsidian-pending-notes: Obsidian plugin for searching links without notes in your vault.

1 Like

This is a really neat solution. I’d love to see your other detector notes if you’re willing to share!

1 Like

you can also bookmark searches, so you can add any words (what unlinked mentions turns out is a search result as in the search modal)…
/(ditto|rubbish|duh|trashy|etc)/ you want to remind you to go with a more suitable word
i admit though that adding to the markdown file is easier than updating a bookmarked search

1 Like

Though not as powerful as your solution, if someone wanted a search just for one string, here’s a query that could be used (based on code I found searching the forum):

```dataviewjs
// Query all pages
const allPages = dv.pages("");

// Crawl content and render list
const searchTerm = "is it possible";
const pages = await Promise.all(
    allPages.map(async (page) => {
        const content = await dv.io.load(page.file.path);
        const regexPattern = new RegExp(`.*\\b(${searchTerm})\\b.*`, "gm");
        const regexMatches = content.match(regexPattern);
        if (regexMatches && regexMatches.length > 0) {
            return {
                link: page.file.link
            };
        }
        return null;
    })
);

// Filter out null entries and render the result list
const processedPages = pages.filter(p => p !== null);

dv.list(
    processedPages.map(p => `${p.link}`)
);
```
1 Like

I’d be interested in seeing your other phrases as well. These are really interesting concepts I’d love to use.

Sorry @ryadaj, totally missed your reply somehow (and thanks @Blaidd3104 for creating another notification to pull me back here). Here are the “detectonote” notes I have at the moment, drop them in your vault and they should pick up instances. Just been tidying them up a bit actually.

Note that they’re tailored to my current and desired writing style and topics - YMMV with them. Specifically, you may find you need to remove some terms/phrases because you’re getting too many false positive links in your content. Happy vault tidying! :slight_smile:

Corporate speak.md (421 Bytes)
Alarmbell phrases.md (724 Bytes)
Outrageously Exaggerated.md (303 Bytes)
Tautologies.md (471 Bytes)
Consider and commit.md (304 Bytes)

2 Likes

super awesome! thanks so much :slight_smile:

Not really what I was looking for, but I find it very interesting and useful, thanks for sharing! You might want to start a dedicated showcase note that might get noticed more by writers. I could see a dedicated thread being useful to expand on the aliases you’ve started with!

That’s a great idea, thank you! To be honest, I didn’t really think it was that interesting an idea, but maybe I should give it a go and see :slight_smile:

And to add something more directly related to your original question, I’ve just created these:

```dataview
TABLE WITHOUT ID file.link as "Frontmatter: tag and tags"
WHERE all(econtains(file.frontmatter, "tag"),econtains(file.frontmatter,"tags"))
```
```dataview
TABLE WITHOUT ID file.link as "Frontmatter: alias and aliases"
WHERE all(econtains(file.frontmatter, "alias"),econtains(file.frontmatter,"aliases"))
```
```dataview
TABLE WITHOUT ID file.link as "Frontmatter: cssclass and cssclasses"
WHERE all(econtains(file.frontmatter, "cssclass"),econtains(file.frontmatter,"cssclasses"))
```

The reason for this is I came across a note that looked like this:
image

Looking at it in source mode I found this:
image

Obsidian moved to aliases / tags / cssclasses as preferred property names some time ago now, but it seems I have some notes where that conversion didn’t happen smoothly or isn’t quite right (maybe I have the Linter plugin configured poorly; I’ll have to check). But anyway, a few simple dataview queries to find any notes that have the issue (and there’s probably a better way to do it than I have, would be happy to see more optimal solutions).

Nice! Added them to my page, and luckily didn’t find any results!

1 Like

on the topic of dataview queries…

after a while users will find problems they want to stay clear of

stuff involving illegal characters in certain places…that can break things

with normal dv queries, you cannot use regular expressions to search so you need to use dvjs

e.g. the following looks for illegal characters following block id references and output files with the relevant paragraphs for context:

```dataviewjs
const regex = /((\^[0-9a-z]{6}$)|(\^[0-9a-zA-ZÀ-ŰØ-áàäæãééíóöőüűčžñßðđŋħjĸłß-]{3,25}$))\n(?!-\s|>\s|^\s*$)((?:(?!\s*(?:-|>)).*\n?)*)/gm;

// Query all pages
const allPages = dv.pages("");

// Crawl the raw data content and render the result table
const finalResult = await Promise.all(
    allPages.map(async (page) => {
        const content = await dv.io.load(page.file.path);
        // Split content into paragraphs
        const paragraphs = content.split('\n\n');
        // Find all paragraphs with matches
        const matches = paragraphs.filter(paragraph => regex.test(paragraph));
        
        return {
            link: page.file.link,
            content: matches.join('\n\n')
        };
    })
);

// Render the result table
dv.table(
    [`Note`, `Problematic transclusions`],
    finalResult
    .filter(p => p.content)
    .sort((a, b) => b - a)
    .map(p => [p.link, p.content])
);
```

i found that if there’s - or > chars in the next line, it’s no problem but if i added a comment %%- I don't concur with this.%%, it would break the tranclusion with the error unable to find section ...

so with this query you can find problematic embeds

you can put this query on a homepage_dashboard.md file under a heading and drag it onto a canvas where you narrow to this heading

over time you might even rack up 10 dvjs queries to alert you to problems

downside is the regexes are usually hard to do even with ai help…

note: the current regex supports:

  • obsidian block id’s
  • carry forward plugin generated block id’s
  • does NOT support the new quadro plugin generated id’s

i tried to add european vowels as well but if more german, french, spanish etc. chars are needed, users can add those in the [] array part after ß for instance…

so the janitor work involves looking at the dashboard queries and e.g. add an empty line below the block-id to make the problem go away

1 Like

the plugin dynamic highlights is also interesting

there’s even some custom-made stuff there to try out (i didn’t yet)

e.g. filler words applied some css to:

{
  "Filler-Words": {
    "class": "Filler-Words",
    "color": "#2D801838",
    "regex": true,
    "query": "\\b([Aa] bit|[Aa]bsolutely|[Aa]ctually|[Aa]nd all that|[Aa]nd so forth|[Aa]nyway|[Bb]asically|[Cc]ertainly|[Cc]learly|[Cc]ompletely|[Dd]efinitely|[Ee]ffectively|[Ee]ntirely|[Ee]ssentially|[Ee]vidently|[Ee]xtremely|[Ff]airly|[Ff]rankly|[Ff]requently|[Gg]enerally|[Hh]opefully|[Kk]ind of|[Ll]argely|[Ll]iterally|[Mm]ore or less|[Mm]ostly|[Oo]ccasionally|[Oo]ften|[Oo]verall|[Pp]articularly|[Pp]erhaps|[Pp]ossibly|[Pp]ractically|[Pp]recisely|[Pp]resumably|[Pp]retty|[Pp]rimarily|[Pp]robably|[Pp]urely|[Qq]uite|[Rr]arely|[Rr]ather|[Rr]eally|[Rr]elatively|[Ss]eriously|[Ss]ignificantly|[Ss]imply|[Ss]lightly|[Ss]omehow|[Ss]ort of|[Ss]pecifically|[Ss]trongly|[Ss]upposedly|[Ss]urely|[Tt]he fact that|[Tt]otally|[Tt]ruly|[Tt]ypically|[Uu]ltimately|[Uu]sually|[Vv]ery|[Vv]irtually|[Ww]idely)\\b",
    "mark": [
      "match"
    ],
    "css": ".cm-line .Filler-Words{\n\ttext-decoration: line-through;\n\tbackground: none;\n\tcolor: var(--text-muted);\n}\n\n/* where to disable */\n.HyperMD-quote.cm-line .Filler-Words,\n.pdf-annotations .cm-line .Filler-Words {\n\ttext-decoration: none;\n\tcolor: unset;\n}"
  }
}

Another one that I’ve been experimenting with: finding all unreferenced attachments. I was trying to do it myself, but then found a solution online at List all not referenced attachments - Dataview Example Vault

// add all extensions you want to ignore to the array, i.e. ["md", "js", "css"]
const excludeFiles = ["md", "js", "css", "canvas", "excalidraw"]

const allNonMdFiles = app.vault.getFiles().filter(file => !excludeFiles.includes(file.extension))
const allNonMdOutlinks = dv.pages().file.outlinks.path.filter(link => !link.endsWith('.md'))
const notReferenced = allNonMdFiles.filter(file => !allNonMdOutlinks.includes(file.path));

if (!notReferenced.length) {
    dv.span(`> [!done] No unused attachments found `)
} 

dv.list(dv.array(notReferenced).map(link => dv.fileLink(link.path)))

EDIT: Just found this plugin GitHub - Canna71/obsidian-janitor: Performs various maintenance tasks on the Obsidian vault which does the same task (and a few other things too).

1 Like

Very cool @ShaneNZ , but it doesn’t look for embedded files, so images like ![[My Image.jpg]] come up as unreferenced even though I have it in a note (I think the same would be true of an embedded PDF). Any idea on how to have it notice embedded files, so they aren’t considered unreferenced?