Latent Dirichlet Allocation

Hi Folks,

tl;dr anyone working on AI Natural Language Processing?

I’m new to Obsidian and working through my thoughts on how to use this remarkable concept/software.

One thought that struck me was the idea of using Natural Language Processing to discover new connections between documents. I think the structure of Obsidian lends itself well to this idea.

The idea of cross links to create structures is very powerful for ideation. But we are creating this structure with our own inherent biases. This is great for projects or writing but we may be missing an underlying, as yet undiscovered connections.

There is a branch of Latent Dirichlet Allocation modeling called a Correlated Topic Model. Here’s a paper on the subject. (Topic Modeling in Embedding Spaces | Transactions of the Association for Computational Linguistics | MIT Press)

If someone is working on the idea I’d like to connect with them.

-Chris

4 Likes

Hello Chrisbo,
We are exploring some of these ideas here Find similar notes (Python script)

I’m hoping to spend some more time exploring classification models to improve tags and linking together ideas.

1 Like

The DEVONthink plug-in uses DEVONthink’s natural language processing to suggest links to related notes. I have found it to be remarkably useful in surfacing notes I had forgotten.

1 Like