Obisdian for Engineering Related Research Organization

Hello all! First off, I apologize if this post is not in the correct area.

I am beginning my PhD in Aerospace Engineering in a few months and decided this was a good time to really plan out a good way to organize my notes, thoughts, what I learn and discover, etc. I happened to stumble across Obsidian and have spent a decent amount of time today watching various YouTube videos describing the basic methodology and practices of this type of note taking application.

While I still have a lot to understand about Obsidian and thought organizing in general, I was wondering if anyone has specifically used Obsidian for Engineering related note taking and thought organizing; either for learning or for collecting references for research papers. If so I would love to see some examples of your workflow and any tips you may have for someone who has not yet embarked on their Obsidian journey.

I want to setup a good workflow before I get too deep into my courses where it would be more of a pain to go back and change things!

Thank you in advance!

-Colton

3 Likes

One tip might be to use aliases in your YAML for terms and notations. I remember learning something math and then trying to apply it in physics or like a boundary layer class and the terms they used were different, but were the same concept. And every book has their own form of notation.

As an engineer in industry with notebooks going back years with proofs and derivations and sketches, in not convinced thereā€™s yet a good way to do this on a computer. Simply put, sketching and proofs on a computer suck. All attempts to make it not suck are attempts at copying paper.

In my experience a lot of engineering requires sketches and drawings. And the links between designs are unlikely to form serendipitously, you will have to engineer the link.

I canā€™t see something like this replacing a notebook for engineering. If youā€™re somewhat careful you can reference a decade worth of design notes between different notebooks.

Thereā€™s also a somewhat contradictory requirement here as well; in order to be competent you need to be able to derive and solve the problem from first principles, the concepts need to be ingrained in your psyche. As such if youā€™re forced to come up with a solution you can do it by hand. Eventually thereā€™s no need for notes.

Having said all that, Iā€™d love to know how you solved it and how you end up using these notes

3 Likes

Iā€™m a research engineer in magnetic resonance imaging, and Iā€™m using obsidian for my research.

Iā€™m still trying to figure out the best way to organize my work.

For now Iā€™m mostly reading publication in zotero and extracting the highlights/note with mdnotes.
Iā€™m using the citation plugin to generate a ā€˜sourceā€™ note with a template (the file name is the bibtex key).

Then I put all the citation in that file. After that Iā€™m separating in the source file the main idea/findings with multiple headers. you can link the headers later in obsidian (not sure if it is better to link to only the source note or source note +header)

I put all the source note in a source folder and also tag them #note/source because Iā€™m not sure what is the best things to do later (you can also add a #type/publication to separate source note from book, courses etc

For that part I still have to find a good template, what tag I want. Should I put them in the yaml etc.

After that I create a knowledge database where I create (not really) atomic notes. it feels hard to create note with only a few sentence to describe science stuff in your field. Obviously some part of each note might link for more atomic knowledge note that I will generate later (example : math formula with demonstration).

And at the root I have ā€˜permanentā€™ notes (donā€™t like the term because it is the note that will change the most along times)that can either act as a semi hub. For example, ā€˜mri reconstructionā€™ linked all the reconstruction methods which then linked to more atomic note like noise propagation in this particular methods.

I am only working on that for 1 month so their is a lot of improvement / harmonization to do.

The main improvement I want to do is 1) the template of all these notes. 2) Should I link the source note to their topic or only to knowledge notes (which means I should not add a source note without generating a knowledge note).
3) how to properly use tags in notes
4) separation of knowledge like pure definition / mathematic equation and proof

Good luck with your setup

1 Like

@corsair2014 First off, I want to say good luck on your journey toward a PhD in Aerospace! I completed my Aero PhD some time ago, in the gap between NASA and SpaceX, and occasionally wish I was starting fresh today. Needless to say, there was no Obsidian when I was in grad school and all my engineering notes are in handwritten notebooks more like @Lafarge describes. I would agree with him that a fair bit of how I think is on paper, in the form of sketches and handwritten math derivations.

Iā€™m now an R&D engineer in industry, and I would say that one of the biggest things Iā€™ve learned is that you donā€™t know where your future ideas are going to come from, but they will most likely be combinations of things you learn along the way in different courses and contexts. That is, youā€™ll take classes focused on particular subjects, but ultimately itā€™s the integration of these concepts in new an interesting ways that will enable you to maximize your impact on the world. This is aligned with Asimovā€™s notion of creativity, which I went into more detail on in another thread last year:

With regard to your specific question about how to organize notes, collect references, etc., the cheap answer is to say ā€œyouā€™ll figure it out as you go.ā€ The longer answer is that Obsidian provides you with the opportunity to easily cross-reference notes, concepts, and ideas via links, while also enabling a simple, linear way of entering things. By that I mean that you can create a set of folders, one for each class, and drop class notes into each folder in sequential order. Youā€™ll wind up with your notes naturally grouped by class, year, concept, which is how you think of them as a student.

But then youā€™ll get out into life and start working on some real problem and think to yourself: ā€œI remember looking at a similar problem in class Xā€ or ā€œI know a math technique that was applied to a completely different problem but feels like it could be used hereā€ and you link to it from your new project. I have had great success with this process, applying old methods to new problems, and wish it were as easy as poking through an old folder and linking to some notes as a refresher, rather than digging my class notes out of the basement!

With regard to @Lafargeā€™s comment about the challenge of trying to replace paper with a software tool, I agree with the sentiment but am working hard to find a way! Iā€™ve found that Obsidian pushes me to create clean summary notes of stuff ā€“ what I think some here would consider ā€œEvergreenā€ notes. When I write in my notebook, I sketch, I draw block diagrams, I do math, I write stream-of-consciousness dialogues to myself when Iā€™m trying to work through a particularly hard problem. All these notes are valuable. But then I find the answer, and I want to capture the key insight. I find that the Zettelkasten idea pushes me to spend the time to properly summarize and capture key insights and link them appropriately. I literally will do screen-caps of handwritten notebook pages, add a pointer to a notebook page number, and include them in an Obsidian note. I link as appropriate and move on. This gives me the best of both world: traceability to my raw thoughts, and a concise, linkable summary going forward.

Still working on my workflow, but after almost a year with Obsidian, Iā€™m very happy with where Iā€™ve landed.

3 Likes

@ja_rule Thank you for your insight and best wishes! This may need to be taken elsewhere but I would love to hear about your PhD experience and your industry experience following.

So you use Obsidian in your engineering career now? Would you happen to have a sample vault you could share showing your linking and organization technique? I am just getting started, mostly personal notes right now, but I easily overwhelm myself trying to over-organize and over-notetake things. I want to find a method that is clean and organized, without being 30 folder structures deepā€¦

From my understanding, Obsidian and other similar linked-note type apps (donā€™t really know all the terminology yet) help to find previously unknown links between different thoughts and concepts. This doesnā€™t seem to suit STEM type research very well given the obvious links and depths of the topics, but the interface and simplicity of Obsidian attract me to use it for 1) personal notes and 2) research notes. I think Obsidian can help me stay organized and also bring a fresh perspective to my note taking just by changing the format of entry. I plan to use it alongside One Note for handtaken notes, probably later condensing and transcribing them to Obsidian to organize important information and also make it more searchable in the future.

I think the ultimate next step would be more integrated and more accurate OCR for handwritten notes to LaTEX. MathPix apparently can transcribe entire PDFs to Markdown and Latex format now, so the only gap is getting accurate handwriting recognition which Mathpix can already do I believe. If we could get a reliable workflow there, I could simply export my handwritten notes to Latex and Markdown automatically, which would be amazing. Obviously diagrams would be another storyā€¦

A couple of thoughts here. First, I just answered a question about tree depths in another thread:

Iā€™m not an authority by any stretch on this, so take it as one personā€™s idea about how to organize things.

With regard to this sort of linking not ā€œsuit[ed to] STEM type research,ā€ Iā€™d say itā€™s the opposite. This is exactly the type of linking that drives engineering forward! As an example that has amused me over a career, every field of study thinks that they have invented eigenvalue analysis, which youā€™ll typically see referred to as PCA (Principle Components Analysis), SVD (Singular Value Decomposition), or whatever clever name theyā€™ve come up with. The math is the same, but the context of the class it was taught in makes economists and engineers and statisticians all think itā€™s their own special technique. The Asimov essay I referred to above basically said that there are tons of examples of this underlying innovation. You go to school for Topic A, your first job introduces you to Topic B, you have a lunch conversation with someone who knows B but not A and they refer you to a paper on Topic C. You look deeper and find that the people who understand C could really benefit from A. Blammo, new idea! STEM fields can tend to be siloed, and cross-pollination of ideas can lead to real breakthroughs. The trick to the linking is that this is where your brain comes in. Obsidian is not going to link your ideas for you. It just provides mechanisms for linking, searching, and visualizing that make this process easier.

On the topic of handwritten versus typeset or OCRā€™d notes, I came to the realization after using an app called Nebo ā€” which was known for having very good handwriting to text conversion ā€” that I actually donā€™t want that! I like to see my handwritten notes, as an aspect of my memory system is visual. Once I create a diagram, I hold a rough picture of it in my head, and for my personal notes, Iā€™d want a cropped screen cap of that image in my vault, not some auto-converted mermaid diagram, if that were even a thing. I use GoodNotes on an iPad ā€” havenā€™t used a physical pencil and paper in 2+ years ā€” and it doesnā€™t do OCR, but can search my handwritten text, which is nifty. Iā€™ve become good enough with LaTeX/MathJax that I can quickly type the equations I want to be a permanent part of my notes.

I think of Obsidian as complementing other sources of notes. There are books, PDFs, my own notes, all of which are sources of information. Obsidian gives me a way to summarize my thinking about these other sources, import key chunks, tie things together, all in one place. And if I do this as I go along, Iā€™ll build up a trusted system that will evolve over time and contain pointers to all the topics Iā€™ve ever touched.

I canā€™t show my vault directly, which is mostly full of proprietary examples, but Iā€™ve thought about putting together a stripped down, generic version that illustrates my current approach.

2 Likes

This is what I have found as well. Although, when starting with Obsidian, I didnā€™t know about the whole ā€˜Atomic Noteā€™ and ā€˜Evergreen Noteā€™ etc. methods so I never really tried to get that granular. Some of my notes so far are long and disorganized as I process through different topics, and then some of my notes are short and to-the-point definitions of physical or mathematical concepts that I can link to in my longer notes.

Donā€™t have it all figured out yet but it is fun to see all the links starting to form over time. I am sure it will grow very quickly when classes start up this Fall.

I like this idea and think I will try this moving forward, probably taking handwritten notes in notebooks and digitally in OneNote (OneNote is what I used in Undergrad a few years back and it worked pretty well, and also allowed me to search my handwriting).

That would be awesome! I would love to see it. Thanks for your input so far!

One thing that is an issue for me is the linking to example. Maybe I will create a private github repository for my whole vault, and put each examples in the dedicated Jupiter notebook and just extract the figure + link in my note.

So does your vault include a directory structure containing not only your markdown files but also other formats files?

Yes. I have a folder heirarchy that is a hybrid of topic based and date based. All my daily stuff and the normal work-as-I-go goes into the date based structure. Folders are yyyy/mm, so Iā€™m currently in 2021/06. This folder ā€” and all my month folders ā€” have an ā€œincludeā€ folder, which is perhaps derived from the C coding file structure. I drop PDFs, plots (typically png), and other non-markdown stuff into the include folder, then reference it in my markdown notes. You can drag-and-drop an image straight from the file browser into a note in Edit mode, so this is very easy.

My files all start with the date number, dd, so todayā€™s daily note would be 2021/06/19.md. Longer notes that I create in a day, say project specific stuff that is too big for a daily entry, get their own note, with a name like 2021/06/19 projectX solver optimization for proto1.md. I would then include a reference to this in both the daily note and the main note for ProjectX.

Iā€™ve described this elsewhere in the forum, but this is my way of dealing with the Zettelkasten naming scheme ā€” yyyymmddHHMMSS.md ā€” which I donā€™t care for. This quickly becomes one giant folder filled with a bunch of endless numbers that to my brain doesnā€™t help with organization. By taking the same serial concept and breaking it into yyyy/mm/dd note description.md, I maintain the time axis, which is important to me, while chunking my work into manageable month-sized buckets. And the extra materials ā€” PDFs, figs, etc ā€” get saved in each bucket, so theyā€™re easy to find. Iā€™ll typically rename the included files with a leading dd to key them to the same day as the note, so if I have a long note with a bunch of math and figures, the figures will be named 19 projX pre-opt results.png, 19 projX post-opt results.png, etc. Ties everything together.

3 Likes

Although thereā€™s some overlap in my interests/field, Iā€™m not an engineer, but maybe an outsiderā€™s view could be helpful.

  1. From what Iā€™ve observed, AlphaTerminalā€™s content on Reddit might be of interest. Heā€™s an engineer in cybersecurity and many engineers have found his insights valuable.

  2. This is an index note from Andy Matuschak with note-making principles, useful note-titles as imperatives, strategies for both theory-building and action. AlphaTerminal and others find it useful.
    https://notes.andymatuschak.org/zhmLXArqiCMDr9Q13ViqN3hh3SmrKzjQxWAr

  3. Also, Excalidraw, on the desktop and Obsidian Mobile apps, works quite well in my experience. Itā€™s not as robust as standalone sketching apps like Good Notes, etc., But if writing by hand within one app is preferred, itā€™s worth a shot. For instance, the inter-referencing, not just cross-references, between handwritten and markdown notes is quite powerful. Typed note-titles in Excalidraw next to the handwritten text, diagrams, etc. and stay in sync with their markdown counterparts if you change them.

1 Like

Thanks, Iā€™ll have to check that out if I get a writing tablet again

I am not trying to push you on that, but in case you have pushlished this example vault, could you please attach the link below?

I guess you have a projectX folder which tracks project X. I am curious about the functions of daily notes and project notes. Do daily notes contain everything you input, i.e., act as input folder in othersā€™ vault? And if you have a breakthrough in a project, will you write firstly in dd.md and then link that in projectX.md or write directly in projectX.md? Would you please explain how do daily notes and project notes act as a whole?

@SpiralDong Yes, I may have a project folder for some of my larger projects, but for smaller stuff, I try to stick to the MOC+linking strategy. Because of the type of research I do, some projects span multiple years, and are large enough to merit their own ā€œspaceā€ in my vault. Iā€™m not a fan of multiple vaults and keep everything in one ā€“ work, personal, hobbies, etc. I only want one Second Brain!

With regard to daily notes versus separate notes, itā€™s usually a judgment call. I tend to create my daily notes a week out; that is, on Monday morning, Iā€™ll spend a minute creating the daily notes for the next week or 10 days. This gives me a place to put things: random thoughts, reminders, links to external stuff I may be reading, whatever. Serves as something of a scratchpad, but removes the concern in the moment of where to jot something down. I always go straight to the daily note, which is always there for me.

What Iā€™ve found is that a piece of analysis ā€“ or thoughts about a hobby, doesnā€™t have to be work ā€“ will reach some threshold where I decide to promote it to its own note. So Iā€™m writing some thoughts, or generating figures/plots, adding my commentary (to myself), and it gets big enough that I donā€™t want it buried in a daily. So I factor it out (cut), create a standalone note with the date and a useful title, paste the content, and immediately add a link to the note in the daily. So now my directory has a new file called ā€œ25 projectX awesome new insightā€ sitting right under ā€œ25ā€, the daily file for today. This makes it really easy to find the chunk of note Iā€™m looking for in the browser, since I donā€™t always remember 2 weeks later whether I did the work on the 24th or the 25th. Also makes it easy to add this note to the ā€œproject Xā€ MOC.

With regard to breakthroughs, Iā€™ve also found that a lot of notes are just notes. If it were a lab notebook, a lot of notes are just dedicated to working through the details of a math derivation, which are necessary, but not particularly interesting. Because all these notes end up being relatively atomic chunks in Obsidian, Iā€™ve settled on a paradigm where I have a ā€œChronā€ index for project X that is very much just a linear timeline of the work, like a traditional notebook. Then I have the MOC, which only points to the pages with the key breakthroughs or milestones along the way. Because linking is easy and free, I find this very satisfying. Iā€™ve been toying with using the Dataview plugin to automatically create this Chron index, but I have to actually use my brain to create a good MOC.

1 Like