"Group by" Woes

[Craig]

Ah, OK, thank you for posting this. It helps illuminate the issue.

Generally speaking, Dataview sees data at two levels:

1. The page level, where you can see all the fields associated with that page, and
2. The item level, where fields are associated with bullet list items and tasks (and also inherit the fields from their page).

Because all of your fields are defined in a single note, Dataview sees that as a single row. It has no way of knowing which “Topic” field is associated with which “Source” field, nor what part of the page the #pin/quote tags are supposed to go with.

The easiest way to untangle this knot is to separate your notes so that one note corresponds with one row in your output data. That means your post above should be in two notes, not one.

You could also separate them into list items per @holroy’s good suggestion above.

[romebot]

Just saw this and it’s very interesting, but I’m not sure it will work for my usage, as what I do often is markup existing articles. So as I read/skim the article, I’m quickly flagging & tagging interesting quotes, facts, and terms that I want to access later in dataview tables. I would have to significantly change the article format to do the above and there would be a lot of friction that would slow me down. I could swear it was working yesterday, but maybe I was just cross-eyed by that time. Bottom line is that I’m not there yet, but trying to devise a very quick way to markup an article and then later, sometimes much later, aggregate the results based on topic or person tags. Eg. Show me all quotes related to arguments for or against the existence of God. Or all terms I found interesting related to politics. Does that make sense?

R

[romebot]

Thanks so much, but I thought it was possible w/ inline quotes to make it relate to a block and that each block could be different. The problem, as I answered to @holroy, is that I’m marking up existing articles that I can’t break up (they would have to be broken into dozens by the time I was through. Is there no other way? (also, wasn’t our discussion yesterday more or less along these lines? I swear it was working at some point, unless I was really so tired and made it all up. :))

I really want to come up w/ a system to markup documents on the fly as discussed and later aggregate all those things into a document.

[holroy]

@romebot

Bottom line is that I’m not there yet, but trying to devise a very quick way to markup an article and then later, sometimes much later, aggregate the results based on topic or person tags.

The quickest way to tag something like this then, and have reliable results, would be to select what’s to become your quote, and then hit a hotkey to trigger insertion of a Templater template, which adds the needed syntax. But without turning it into a list item, you’re not going to be able to get all of it.

The list item can also be styled so as not to “disturb” the original article, but either you have to use lists to change the field association level (from page to item), or you need to only have one quote from each article…

The second best, and it’s not close to being as good, is to insert just one field, and having that field be a list of the various parts like the source and topics and so on. But that would be accessible as a block, and it’ll be cumbersome to unpack into other values.

[holroy]

@romebot

I feel like I tried that and didn’t work, but now it does!!! maybe I typed something wrong. Now this is exciting. I will see if I can play w/ grouping now, but this, sorted by source is a really cool way to aggregate quotes or facts from across my database by topic(s). Much appreciated!

If you think this works, then look at how the “XXX Contrary” quote is listed twice, once for Kaufmann, Lisa and one for Sagan, Carl, and similar for the “This is a quote about God” which is also listed twice. I wouldn’t call that working…

[romebot]

You’re right… I guess I was so tired and cross-eyed!

[romebot]

do the lists items have to be consecutive? Can I just put a bullet list in front of the quote at the top of a page, then somewhere in the middle, and then at the end for example?

What about when I want to do an inline quote in the middle of a paragraph?

Maybe I’m not fully understanding.

What about my 2nd example, is that in any way different where I want to enclose the text in single brackets or is that effectively the same as with the quotes.

If you have time, can you reformat part of my 2nd example in a way that would work.

Maybe if I saw it I could then play with it on my own and see if I can find a workable solution? I’m determined to find a way.

Really appreciate the help!

[romebot]

Trying to play with your suggestion, but maybe I’ve bit off more than I can chew. I was SO excited to accomplish what I thought I did, if only I didn’t get the duplicates. Another downside to this is that my quote is now not formatted in a way that I could also use it as a block reference, but it might be an acceptable compromise.

Here is a sample note where I’m imagining that the items are spread throughout the page and not one long bulleted list. I did try it w/ each block as a bullet point (which actually would be easier), but then that resulted in empty rows where I didn’t have the quote, source, and topic. Is there a way around that?

What I’m missing as well as a column with a link to each document that the quote comes from. Not sure how to do that and would be needed as I want to search my whole database as well as sort or group by author.

P.s. I wasn’t able to get the quotation marks to show up even though I formatted it around the quote like you did.

BANNANA TEST NOTE

• “(quote:: I’m not so hot on apples.)”, (source:: Lisa Kaufmann), (topic:: #pin/topic/bannanas)

"But I must explain to you how all this mistaken idea of denouncing pleasure and praising pain was born and I will give you a complete account of the system, and expound the actual teachings of the great explorer of the truth, the master-builder of human happiness. No one rejects, dislikes, or avoids pleasure itself, because it is pleasure, but because those who do not know how to pursue pleasure rationally encounter consequences that are extremely painful.

• “(quote:: Nor again is there anyone who loves or pursues or desires to obtain pain of itself, because it is pain, but because occasionally circumstances occur in which toil and pain can procure him some great pleasure. To take a trivial example, which of us ever undertakes laborious physical exercise, except to obtain some advantage from it?)”, (source:: Tom Nichols), (topic:: #pin/topic/bannanas)

But who has any right to find fault with a man who chooses to enjoy a pleasure that has no annoying consequences, or one who avoids a pain that produces no resultant pleasure?"

• “(quote:: Apples make the world go round.)”, (source:: Carl Sagan), (topic:: #pin/topic/bannanas)

Here is my dataview

Table without id
  item.quote as Text,
  item.source as Source,
  item.topic as Topic
From #pin/topic/bannanas  
flatten file.lists as item 

Here is a screenshot as above:

and a screeshot when every block is a bullet point—those other blocks result in empty rows

[holroy]

@romebot

do the lists items have to be consecutive? Can I just put a bullet list in front of the quote at the top of a page, then somewhere in the middle, and then at the end for example?

The list items can be spread out all over the file. That doesn’t matter.

@romebot

What about when I want to do an inline quote in the middle of a paragraph?

Then you either have to make the entire paragraph an item, or split the paragraph with some text before and after the actual item. Yet again, if you use the (quote:: quoted text) markup, it doesn’t really matter if the actual list item has more text surrounding it.

The only thing you can’t do then is to have multiple quotes in the same paragraph. That does require you to split up the paragraph into multiple list items.

@romebot

What about my 2nd example, is that in any way different where I want to enclose the text in single brackets or is that effectively the same as with the quotes.

Inline fields can be defined using either brackets or parentheses. With brackets the name of the field and the value is shown, and with parentheses only the value of the field is shown.

[VisibleFieldName:: Visible value] and (HiddenFieldName:: Also visible value)

Values are still available: `$= dv.current().VisibleFieldName ` and `$= dv.current().HiddenFieldName`

Unformatted this shows as:

@romebot

If you have time, can you reformat part of my 2nd example in a way that would work.

Reworked term-of-art note

---
Author: "[[Nichols, Tom]]"
Tags: pin/term-of-art/political-test
---

- [!] "(quote:: Quote1. Lorem ipsum)" — (source:: [[Nichols, Tom|Tommy boy]])

Some random text, just for the fun of it!

-  "[pt:: Term of Art1. dolor sit amet]"
-  [pt:: "Term of Art2. consectetur adipiscing elit"]

Some more randomness?!

-  “Quote2. Lorem ipsum dolor sit amet, consectetur adipiscing elit, (quote:: sed do eiusmod tempor) incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.” — (source:: [[Kaufmann, Lisa|Lisa Kaufmann]]), (topic::#topic/test)


## Political terms
```dataview
TABLE WITHOUT ID term as "Political term", Author
FROM #pin/term-of-art/political-test
FLATTEN pt AS term
SORT term DESC
```

## Quotes
```dataview
TABLE WITHOUT ID item.quote as Quote, item.source as Author, file.link as Link
FROM #pin/term-of-art/political-test
FLATTEN file.lists as item
WHERE item.quote
SORT item.quote DESC
```

Reading view, default formatting with the Minimal theme

This shows up for me as:

@romebot

Maybe if I saw it I could then play with it on my own and see if I can find a workable solution? I’m determined to find a way.
Really appreciate the help!

As stated above, formatting can be applied, and just for the example of it, I made the first political term into an important task using [!] (which is part of Minimal theme checklists syntax extension). If one wants to these can be further styled, so that if you use a particular task decoration for your quotes and political terms, you could effectively render these “random” list items as ordinary paragraphs with no indents, possibly with a leading icon (or not).

[holroy]

@romebot

and a screeshot when every block is a bullet point—those other blocks result in empty rows

They can be easily eliminated by adding a WHERE item.quote like in my example.

[romebot]

Thanks for all the effort and time you (and @Craig) have put into this.

I think I understand (mostly), but its definitely not the quick way I was going about it before… still might be worth it, esp. if I can understand what you mean about the where line. This?

where file.name = this.file.name

I think it would def. be easiser to just hit enter to start a new line/bullet then the other way, so would really like to understand how to eliminate the empty rows. It’s not sorting for some reason.

Here is my dataview

Table 
  item.quote as Text,
  item.source as Source,
  item.topic as Topic
From #pin/topic/bannanas  
flatten file.lists as item 
Sort Source ASC

And here is the result coming from multiple documents

that’s because of the blocks w/ no quote

[holroy]

@romebot

esp. if I can understand what you mean about the where line. This?

where file.name = this.file.name

Did I leave that in there, again?! It’s just a line for me limit my query to the test file in my Sandbox vault. Replace it with either your own where clause, or a from clause targeting where your notes reside.

Did you try adding the where item.quote I suggested in the other reply? It should remove lines without quotes.

[romebot]

That fixed it!! I’m not sure what that code means, but it did fix the issue. Last thing then, why wont my table sort by source? I tried lower and upper case “s” in Sort, but doesn’t seem to have an effect.

Table 
  item.quote as Text,
  item.source as Source,
  item.topic as Topic
From #pin/topic/bannanas 
flatten file.lists as item
WHERE item.quote
Sort source ASC

Man, this was some undertaking and I can hardly believe how generous you all are w/ your time and help. Still not sure if this is the right way to go, but I’ll have to try it out on some notes. I’m not sure how to use templater, but maybe I can use some text expander.

One more thing. I noticed in your example that you didn’t always write source. e.g. in the terms of art. yet Tom Nichols was still listed as author… Just before that you had “Tommy Boy” as the source. Wasn’t clear how dataview knew to put “Nichols, Tom” then.

[holroy]

@romebot

I noticed in your example that you didn’t always write source. e.g. in the terms of art. yet Tom Nichols was still listed as author…

You said that for political terms there was only one author, so therefore I specified him in the frontmatter, and then there is no need to respecify him for the political terms.

The quotes could have different sources, so there you would need to specify it, as per your use case.

I’m not sure why you can’t sort on the sources, as it works in my test cases. Try using item.source, and see if that makes a difference.

[romebot]

worked! thanks a million!

[romebot]

I hesitated to bother you again, but when I tried to apply this to an actual document, the quotes worked great, but the terms of art dataview code brings in the quotes in the dataview results. I’m assuming that I can mix quotes and terms of art in the same document (eg. some bulleted blocks with “(quote::)” and some with “(pt::)”, but I tried other documents where there’s only terms of art and no mixing.

This is my dataview:

Table 
  item.pt as Text,
  item.source as Source,
  item.topic as Topic
From #pin/term-of-art/political  
flatten file.lists as item
WHERE item.quote
Sort item.source ASC

And here is the result. why is it bring in quotes w/o anything in the text field and not terms-of-art? Is there some kind of conflict w/ other notes? pulling hair out again.

[Craig]

I don’t see item.pt in your source. Do you mean item.quote?

[romebot]

you just saved me hours more of hair-pulling.
I didn’t realize I had to change item.quote to item.pt!
I guess I dont understand what that line does.
thanks so much!

R

[holroy]

@romebot

I didn’t realize I had to change item.quote to item.pt!
I guess I dont understand what that line does.

It basically checks whether this item has a quote or pt field. It doesn’t care about its, but just checks it it exists.

[romebot]

Ok, I hope this is the last wall I hit on this, but maybe it’s just me being cross-eyed again.
I honestly thought I was done with this… Grrrr.

So I thought that using AND in dataview would limit my results to only those items that have both tags.

E.g.

Table 
  item.quote as Text,
  item.source as Source,
  item.topic as Topic
FROM #pin/class/quote AND #pin/topic/politics/american/gop/anti-government  
flatten file.lists as item
WHERE item.quote
Sort item.source ASC

But the resulting table gives me back an item that does not have the 2nd tag. Is there a reason or is there some other issue?

In my note I have

Politico’s Heidi Przybyla recalled that in his 2021 memoir, former Republican speaker [[Boehner, John]] said of this faction:

• (quote:: "What they’re really interested in is chaos.… They want to throw sand in the gears of the hated federal government until it fails and they’ve finally proved that it’s beyond saving.”), (source:: [[Boehner, John]]), (topic:: #pin/class/quote, #pin/topic/politics/american/gop/anti-government)
• And they are tied tightly to right-wing media:
• (quote:: “Every time they vote down a bill, they get another invitation to go on Fox News or talk radio,” he said. “Its a narcissistic—and dangerous—feedback loop."), (source:: [[Boehner, John]]), (topic:: #pin/class/quote, #pin/topic/media/political-influence/gop)

And my result is this. The 2nd quote doesn’t contain the 2nd tag listed in the dataview code, and I have each item on a different block as discussed above.