Alphabetize/Remove Duplicates

[I-d-as]

I suggest the ability to alphabetize.

I find myself regularly combining lists of links and having to leave Obsidian to re-sort alphabetically. Once, back in Obsidian, I scan the list for duplicates to remove.

Any function that could accomplish both of these goals either simultaneously or separately would be a handy time saver, if you ask me. Thanks.

[malecjan]

Why do you remove duplicates manually? If the application is able to do the sorting, I would expect it to be able to do also removal of duplicates since it is much simpler task. After sorting, duplicates are adjacent.
If basic item is line, then regular expression could be (?<=\n)([^\n]*\n)(\1)* → \1

You can of course first get highlighted or filtered the duplicates for inspection and then decide to remove them all automatically or just some manually or semi-automatically by tweaking the above regex.
I use EmEditor for advanced editing, easily opened from Obsidian by “Open in default app” hotkey. Alternatively I process text within clipboard by AutoHotkey scripts.

There is also Obsidian plugin GitHub - Vinzent03/obsidian-sort-and-permute-lines: Sort and Permute lines in whole file or selection. . For author of plugin it should be easy to add deduplication option if you ask him.

[I-d-as]

Very cool. I appreciate your response. This post was before the more recent discussion on alphabetizing.

Since you know and share so much on the forum and also were part of that discussion and I am pretty sure you already are aware of the quote below. I am including this for the sake of people stumbling upon this who may not.

@I-d-as

And since I posted that request, I have found out you can remove duplicates as a filtering function in LibreOffice calc (and I am sure in Excel and the rest)

Thanks.

[malecjan]

Thanks for quoting it here on relevant place. (I actually did not remember that quote when replying, maybe even skipped reading it. For such reasons, I usually edit original posts of my pending requests, to avoid outdated info and save everybody from reading and replying to it.)

If deduplication gets into the mentioned plugin, then this thread could be archived.

[I-d-as]

Good point about editing. I definitely need to do that.

I once made a request for an ability to download an updated version of forum as an Obsidian vault. Ability to download a synced version of entire forum as vault. That sure would make things easier. Although I do know it is possible to download my own content as a single text file.

Thanks for the suggestion and all the connections you build in the forum. I’m surprised you aren’t considered a Moderator. You have my vote.

[malecjan]

It seems, I will need to study plugins - if their limitations are sufficient for my expectations and how to write/program them. With other duties coming, I do not expect to have/afford time to volunteer as official good quality moderator. I will still help where I feel appropriate.

[Oskiator]

Hi ! I would also enjoy a way to work with duplicates. Not only to remove them but also to find them.

It would be great to have a way to highlight them and then have options to delete them manually or automatically.

I found a way to do that on word with VBA code :

Sub highlightdup()
    Dim I, J As Long
    Dim xRngFind, xRng As Range
    Dim xStrFind, xStr As String
    Options.DefaultHighlightColorIndex = wdYellow
    Application.ScreenUpdating = False
    With ActiveDocument
        For I = 1 To .Paragraphs.Count - 1
            Set xRngFind = .Paragraphs(I).Range
            If xRngFind.HighlightColorIndex <> wdYellow Then
                For J = I + 1 To .Paragraphs.Count
                    Set xRng = .Paragraphs(J).Range
                    If xRngFind.Text = xRng.Text Then
                        xRngFind.HighlightColorIndex = wdBrightGreen
                        xRng.HighlightColorIndex = wdYellow
                    End If
                Next
            End If
        Next
    End With
End Sub

Source : How to find and highlight duplicate paragraphs in Word document?

The idea of highlighting the first occurrence in green and the following occurrence in yellow is cool.

We could also imagine an option to put each group of duplicates of the same line in one color. Exemple : All occurence of {X} in yellow, all occurence of {Y} in green, etc.

[I-d-as]

@Oskiator Very cool! Thanks for sharing this! Something like this would be awesome in Obsidian. If it doesn’t exist already, this seems like a good candidate for a Feature Request or a Plugins Ideas.

[Oskiator]

As this thread is related to the shared idea and already in “plugins ideas”, should I create a new post or here is already ok ? (in order to avoid duplicates in the forum )

[I-d-as]

@Oskiator Ha! It is up to you, but I would guess that it would be better to create a new topic based on the suggestions that I regularly see the moderators making in this regard.

Thanks again!

[Oskiator]

In fact, it seems related to another topic you did create in 2020.

Auto Select/Remove all but one duplicate links and links to self

When building composite notes, it would be nice to have an analyzer plugin that could select all but one duplicate links within a note. It could also select all links to self. This plugin may be able to run in batch mode perhaps with an option to automatically remove these duplicate and self links with the click of a button within all notes or the results of a search. Thanks.

May I post under your idea ?

[I-d-as]

For sure! I forgot about that one. Very cool. I’d be honored.

[PAC]

Hello everyone! Is there any way to remove duplicates from a list in Obsidian yet? Am I missing something? This would be very useful and it seems to me (seeing the dates) that it might have already been implemented, but I cannot find where. Thanks for any help!