Coalesce headings into one file?

Feature requests
1

Hello all,
I’m a relatively new Obsidian user and would really like to know if there’s a solution to this use case that doesn’t require too much customization. I think it could be helpful to others-

Use case or problem

I would like to use Obsidian as a general daily journaling tool. Something I have been doing in Notion that has been extremely helpful is creating columns for general topics in my life (Health, Work, etc.), and jot down quick notes from the day that I would like to review. After some time, I can simply concat all of the daily notes for a particular topic into one page to get a summary of that section over time.

The daily notes function is perfect for this, however I do not see a way to collect these subheadings over time.

Proposed solution

Functionality that lets me extract all text under headings into one note, while preserving the date of entry.

For example, let’s say there are many daily notes in a folder with ISO formatted dates as file names. Within these files might be sections: # Health, containing some text blocks.

I would like to run a query or command that searches all files in {path}, and extracts all text under specified {heading} into a new note, separating each extraction with the filename.

The results would look like this:

# 2022-12-01

Some text content under {heading}

# 2022-12-13

More text content in another file under {heading}

Current workaround (optional)

I can use the embedded query functionality to get this, however, there doesn’t seem to be a way to extract the query results as markdown content.

Capture
Capture882×347 12.2 KB

Linking to a page with {heading} is not useful, because the linked mentions pane does not allow exporting/pasting either. I’d also like not to use tags, since these are reserved for other purposes.

IMO, this functionality would be a powerful tool to turn unstructured notes into focused bullet journals. Thoughts?

PS - I found two related plugins, but I don’t think any of them match the exact functionality I described

2

You can do this with Dataview.

Here is a working demo vault which you can download to test with:

:floppy_disk: Download demo vault

It will collect the sections under each of your requested daily note headings, and summarise them by day inside a single note:

image
image897×769 21.5 KB

And here’s the how-to guide. The latest version of this guide can always be found here.

Step 1:

Install Dataview, and turn on Javascript queries:

image
image1355×190 13.2 KB

Step 2:

Create a new note and paste this code:

```dataviewjs
// Headings you would like to summarise the text for
const headings = ['Health', 'Mood']

// You can update this to filter as you like.
// Here it is only returning results inside the "Daily notes" folder
const pages = dv.pages('"Daily notes"')

const output = {}
headings.forEach(x => output[x] = [])
for (const page of pages) {
  const file = app.vault.getAbstractFileByPath(page.file.path)
  // Read the file contents
  const contents = await app.vault.read(file)
  for (let heading of headings) {
    // Sanitise the provided heading to use in a regex
    heading = heading.replace(/[/\-\\^$*+?.()|[\]{}]/g, '\\$&')
    const regex = `(^|\n)#+ ${heading}\r?\n(.*?)(\n#+ |\n---|$)`
    // Extract the summary
    for (const block of contents.match(new RegExp(regex, 'isg')) || []) {
      const match = block.match(new RegExp(regex, 'is'))
      output[heading].push({
        title: file.basename,
        text: match[2].trim()
      })
    }
  }
}
Object.keys(output).forEach(heading => {
  dv.header(1, heading)
  output[heading].forEach(entry => {
    dv.header(4, entry.title)
    dv.paragraph(entry.text)
  })
})
```

You’ll need to update the two variables headings and pages at the top to match the location and format of your own files.

3 Likes
3

Thank you for the quick and very helpful response! Your solution worked great and seems that with a little more js knowledge, bet I could add more search criteria and reformat it to whichever need.

Do you think this is worth pitching as a plugin feature? I could see it being very useful for folks who aren’t great with the js api

4

Personally I wouldn’t bother. It’s the kind of thing which is so specific to the individual user case that modifying the Javascript code directly is likely going to produce the best result. There’s also the fact that plugins need to be maintained to keep up-to-date with new Obsidian versions. (Dataview is highly likely to be maintained.)

Plus it’s a great excuse to learn a little bit of Javascript, which is always going to be a valuable addition to your skillset :wink: Codecademy is a great resource.

3 Likes
5

Hi this is fantastic! Thanks for putting it up here. From looking through forums a lot of people have been looking for something like this for ages. I don’t know much about dataviewjs but do you know if it’d be easy to turn the filenames into clickable links like with dataview lists?

6

Very easy :slight_smile: Just replace the current title with a markdown link:

output[heading].push({
  title: '[[' + file.basename + ']]',

This will output [[Note title]] just like any link you’d use in any note. It’s not complex at all to work with Javascript once you know the basics, and I do recommend the course linked above.

You can make it more explicit if you have notes with duplicate titles by using the full path and then the | character to add an alias:

output[heading].push({
  title: '[[' + file.path + '|' + file.basename + ']]',

That will output [[Full/Path/To/File.md|Note title]] just like you would create any normal link in your notes.

4 Likes
7

This is great, thanks a lot! Definitely interesting stuff!

8

This works perfectly still ! Is it possible though to include all lower level headings in the output ?? e.g. if the chosen heading is Heading 2, and is followed by more lower level headings (Heading 3, Heading 4 …), the I want them included in the output as well… in other words all text to be included till next same level heading (Heading 2 here) or a higher level heading (Heading 1 here) …

Right now the output is including only the chosen heading and ignoring other lower level headings …

Please forgive if I am using non-standard language here – I am a new user of obsidian and a noob in Javascript ! Let me know if any more clarity is needed …

1 Like
9

Any pointer towards resolving this please would be highly appreciated !!

1 Like
10

I would be interested in the same thing

11

The code AlanG provided works, except that I get all the headings returned whether there is data under that heading or not. Is there a method to only return headings and data if it exists and if hte section is empty to not show it?

For example on 2023-11-13 I have 2 bullet points under the heading but none under 2024-02-27 yet both are appearing in the query.

12

I asked ChatGPT for help and after trying various cases with checking for “- empty” and/or simply deleting all text and the results it gave worked, though with a useless check for empty now that I am not doing that in my Daily Note template. I don’t know JavaScript and don’t want to break a working code block.

// Headings you would like to summarise the text for
const headings = ['Thoughts and Notes']

// You can update this to filter as you like.
// Here it is only returning results inside the "Daily notes" folder
const pages = dv.pages('"300 Writings"')

const output = {}
headings.forEach(x => output[x] = [])
for (const page of pages) {
  const file = app.vault.getAbstractFileByPath(page.file.path)
  // Read the file contents
  const contents = await app.vault.read(file)
  for (let heading of headings) {
    // Sanitise the provided heading to use in a regex
	heading = heading.replace(/[/\-\\^$*+?.()|[\]{}]/g, '\\$&')
    const regex = `(?<=^|\n)#+ ${heading}\r?\n(.*?)(?=\n#+ |\n---|$)`
    // Extract the summary
	for (const block of contents.match(new RegExp(regex, 'isg')) || []) {
      const match = block.match(new RegExp(regex, 'is'))
      const summary = match[1].trim()
      // Check if the summary is not empty and does not contain "- empty"
      if (summary && !summary.includes("- empty")) {
        output[heading].push({
          title: '[[' + file.path + '|' + file.basename + ']]',
          text: summary
        })
      }
    }
  }
}
Object.keys(output).forEach(heading => {
  if (output[heading].length > 0) { // Only display if there's content
    dv.header(1, heading)
    output[heading].forEach(entry => {
      dv.header(4, entry.title)
      dv.paragraph(entry.text)
    })
  }
})

Hope this helps others.