Learning Data Science/Machine Learning and connecting concepts

[Jaybee]

If anyone has any experience of Learning Data Science using Obsidian, please help me.

I am just 3 days into using Obsidian, really great map view but want to know how do you connect ides in Data Science?
Basically everything in my map is just connected so harder to get the bigger picture. Still learning the second brain concept, so will be updating the forums if I find any better ideas.

[ggmate]

I’ve been in the same boat as you. I started using Obsidian when I was starting to study about Machine Learning. And I have to admin that I did not get much value out of the graph view, except for the fact that it looks pretty. Since concepts like Parameters are used in every single note, it now has like 100’s of connections and make the graph view kinda clumsy.

But writing down notes, then looking for places where I can link it to and finally trying to organize them using MOCs really helped me flush out the concepts. So just start taking notes, don’t overthink about process of Knowledge Management in the begining, you will get there. Just start taking notes.

[Jaybee]

Hi buddy,
I can learn from your methods, can we connect on any other platform were we can have a quick chat?

I am currently taking down notes in Evernote and then trying to connect it in Obsidian, I am unable to understand the graph but it looks really pretty, not useful though.

[etothepiiminus1]

Hi,
I’m a mathematician and preparing for a new position in ML validation etc.

Since ML / DSci was not my primary area of research I’ve begun developing a library of useful approaches to common problems that will be useful in my new position — along with relevant examples, links to corresponding libraries, and methods in sklearn etc.

I’d be curious of your structure for dependencies in knowledge, I.e., how you handle the fact that a confusion matrix and decision boundaries are prerequisite to ROC. Also do you use a template for concepts/techniques etc?

I have a structure note corresponding to the problem/task type: “classification performance metrics”. This structure note links to notes each of which are approaches to solve the problem— i.e., I have a note linked to in this structure note called “classification performance via ROC”, and “classification performance via f-beta”

Currently I have three note types and corresponding templates:

1. Problem / Task —a structure note which specifies the task and links to…
2. Approaches / Solutions— a note which codified one approach to solving the problem in the associated structure/problem note. If the approach requires knowledge of ML concepts about which I am unfamiliar then I have
3. Object / Concept note— a simple note with object definition, applications, example, quick facts, and relevant references.

I’m hoping that a core set of tasks/problems with solutions example’s and references will be helpful.

If you have any suggestions for alternative structures for this particular task-solution-reference use-case I’d be grateful to hear your thoughts.

Happy to share my templates if anyone is interested. Please do the same if you are working with a task/problem-approach/solution sort of structure.

[markf]

I’ve been studying a quantitative master’s degree in the last few months and all my notes are in Obsidian, even lots of LaTeX equations. I have a few hundred notes and the graph view has been incredibly useful for me to understand concepts on my course. I’m pleased with results I’ve had from my recent assessments so it’s gone well; if I did ‘linear’ notes I wouldn’t have done that well.

Two of the biggest modules of my degree have been biostats () and epidemiology (). Over time it really became clear when I was taking notes how they related to each other, from the graph view. I have one folder per module in my vault and use those for the colour code.

My longest notes have ~1k words. The typical note has ~250 words. My knowledge base currently totals over 55k words. I wasn’t expecting to write so much… I initially thought it would be very tough to revise that amount of content, but didn’t need to read much for exam preparation. The note taking process definitely helped my comprehension of concepts.

Backlinks are the most powerful thing IMO (not the main graph view), especially when you have dozens of notes. Backlinks are what I used most for navigating between concepts and understanding how things relate to each other, in a rapid way. If a note didn’t have the backlinks that I’d have expected, it was a sign I needed to revisit other notes and add wikilinks to that note.

General summary of how I have done note-taking:

• I do one note per concept. If something sounds like a concept that may come up, I’ll create a wikilink for it. I haven’t felt need to make notes for literature.
• I don’t do MOCs. I wouldn’t even know how to structure MOCs for my knowledge graph! The closest thing I have is a few notes that have many wikilinks, e.g. ‘Study designs’ note has a table of the different study designs in epidemiology, ‘Hypothesis testing’ note has sections for different types of tests e.g. goodness of fit, tests of independence. I created those notes to help synthesise the concepts - they aren’t really tables of content, they do have a fair amount of text.
• If a note has many sections or gets quite large (~750 words), that is an indication I could benefit from splitting out the content into more notes. I have a note for ‘Bias’, which is around 750 words and is focused on the statistical aspects of bias, but early on it also had content on types of bias that affect study design (e.g. ‘Selection bias’). Once I had more detail on those concepts in later lectures, I would rework the notes. It ended up quite organic.
• I do tagging to help me flag which notes I need to rework, where I need to spend more time writing compared to what I managed during lectures. Typically it would be something like ‘#REVISIT’ - I would blast through about 30 of these on a good day. It was actually quite a nice way to gamify my learning. I now have about ~60 of these tags in my graph so I figure I need around 3 solid days of reworking my notes before my next term starts.
• I tend to keep code blocks and equations in notes for concepts. I don’t split it out into separate notes for equations, libraries, etc.

I’m doing modules next term on the machine learning, I can see from the greyed out notes where some of the concepts fit in my current graph, so that also gives me ideas of which notes I should revise ahead of that.

[jonaswanninger]

i would love to to see your templates! You inspired my to try out your note types for my cloud computing knowledge management.

[etothepiiminus1]

Happy to, I’ll add to a github repo
BRB

[etothepiiminus1]

I’ve shared/introduced the templates https://forum.obsidian.md/t/task-solution-oriented-templates-for-reference-during-ml-data-science-tasks-link-included/30318

Direct link:

https://github.com/BradfordFournier/problem-solution-markdown-templates

[Hevel]

Hello there,

Learning data science / ML too. My problem is that most of my notes are inside Jupyter notebooks because I need to explore some concepts interactively and not only with mathematics. Imagine you’re studying NLP. Okay there are tons of maths and concepts to write on… but eventually you want to test it by yourself in a notebook. You want some graphs, to see the process, the results…

And I’m struggling on how to integrate those into Obsidian. What would you recommend?