Organizing Vault Assets Newbie Questions

[Skengman]

Hi all, I’m a research scientist working in cancer biology currently interested in migrating my literature reviews from OneNote to Obsidian. I’ve made a decent start and like this approach so far but I’ve hit a point where I’m running into a few issues that I haven’t found a straightforward answer to yet.

There are 2 topics here:

1. Specific issue with managing image assets with base Obsidian organization.
2. I’m all ears for any tips you may have for my use cases.

#1 Specific Image Org Issue:

I like snipping key figures from references as I go through papers and have been putting them in a specific Vault_assets folder for importing notes into Obsidian. These will be re-used for specific TOCs and MOCs as my vault grows. When I drop a .png or .jpeg asset into an Obsidian note, it creates a new file in the left nav bar. I’d prefer it all live in a folder, but when I set the assets folder in Settings to its own folder, I’m still getting the same issue. Do I need to manually aggregate these to a specific folder no matter what or is there some obvious setting that I’m missing?

#2 General Feedback
I work with several different “targets” of interest in cancer biology, which can refer to the specific gene that encodes that target or its expressed protein form. Different research groups tend to focus on specific targets over time, so there’s continuity over time as new findings emerge for citation as well. I include this because there are a few key things I’m using as fodder for links: target names, biological phenotypes, and authors. Here’s what that looks like for OneNote, which I’ve been using for this since 2016 or so:

Clunky stuff, but I’ve made do with it and Mendeley. However, now that I’ve seen what Obsidian can do, there are several use cases I’m interested in any and all pointers any of you may have:

1. Reference manager (I note there are some threads on .bib integration with Zotero that I need to follow up on) – being able to search by target name, year, topic, etc and see the links to other targets and findings will be really excellent moving forward.
2. Writing tool – writing grants and papers is a slog on reference management, so I’m interested in testing Obsidian as a means of writing out linked chunks as a TOC for first drafts.
3. Jupyter notebook organization & lab notebook

[KD21]

Hi,

I’m a newbie too but…

For question #1. In settings, you can choose to store under a subfolder but you have to enter the actual folder name… this is at odds with some fields which when not filled out the faint grey text is used as a default (like the date format in templates is used) but not for the resources folder.

For the image attachments already in the root of your vault you can move them - one at a time I’m afraid into the new folder (unless you write a script in python/Jupyter to do it for you.)

and then choose a folder

In the screenshot above you can see a sub folder in the attachments folder ‘OneNote to Obsidian - Mac-assets’ so you can move an image file to a subfolder in the assets folder if you want. Obsidian will edit the links in your Markdown files for you.

You can also move a folder of attachments into another folder so in the shot below the ‘OneNote to Obsidian - Mac-assets’ folder is back in the root of the vault

and right-clicking and move lets me move it back into the attachments folder

You may have already gotten everything from OneNote if so stop here. if not this may help…

In another post earlier today I posted a method to get notes out and into Obsidian without directly using any scripts. It also has a good side effect of putting all your assets in a folder by OneNote section, you can then move these folders into the root attachments folder if you wish to keep the root clean for md files. I’ve attached the guide as a zipped md file to this reply. You may find it useful.

OneNoteToObsidian.zip (1.9 MB)

Also as you use jupyter… If you want to use a python script this
https://github.com/Danmou/onenote_export
worked after a fashion. NOTE it only gets HTML so another script will be needed to convert to markdown.

Danmou’s script can be tweaked to handle throttling a little better I think, and I found it more consistent if I imported this
https://github.com/deckar01/ratelimit
from pypi and added rate limit decorators to the get method.

from ratelimit import limits, sleep_and_retry

HOUR = 3600
MINUTE = 60


@sleep_and_retry
@limits(calls=120, period=MINUTE, storage='ratelimit.db', name='hourly_limit')
@sleep_and_retry
@limits(calls=400, period=HOUR, storage='ratelimit.db', name='daily_limit')
def get(graph_client, url, params=None, indent=0):
    while True:
        resp = graph_client.get(url, params=params)

The @ decorators put it to sleep after reaching Microsoft’s throttling point. When it wakes up off you go again. There is also a time out on the webpage you log into the Microsoft account. Just click the back button when the web page gives an error about ‘token missing’ and click on ‘continue’ again and the script will begin to continue (if not still asleep)

I’ve not found any method that hasn’t flaked out over a page in at least one OneNote section yet but both of these got 99% of the notes out for me.

If you do go down this route you need to convert the html to markdown. If you write your own script this could also move the images where ever you want and edit all the link paths. The script I wrote converted all png to jpg as well just to optimise storage as I was moving them.
If you are just going to convert html to markdown then pandoc is said to work well (not used it) and there is a python wrapper for it (also not used it)

There are some odd features in some of the OneNote HTML files - I found alt text entries for images that are dozens of lines long and these made some other methods of converting html to Markdown fall over and the markdownify package for python choked on them as well. So when I wrote my own convertor I ran a method over the HTML to remove these crazy alt tag text values first. If you want the converter scripts message me in here (Click on the K that is used as my Profile Picture) and I can pass them to you, as they are not in git hub as have no documentation or anything like that :-). just some raw code at the moment.

If you have used ink to draw or highlight OneNote pages this is lost and you may see “InkNode is not supported” lines in your markdown files where there was ink after conversion to markdown. I saw some people kept their ink by taking screenshots and then deleting the ink and the original image with the screenshot… Lot of work for heavy users.

[Skengman]

@KD21 thanks for all this info, it’s really great and I’ll work through it! I had already found a way to clean up the image assets: I was already storing them in the same root directory as the other Obsidian .md files and as such I could just do a “show in system explorer” and do a bulk move over to a new directory within my vault (~/vault/x_files) to keep them all organized within the notes! I’ll have to try the OneNote converter another time.