Please help.
I really, really want to use Obsidian as my main PKM/Academic Research Thinking/Note-taking repository (especially because of its bidirectional linking and ability to sync with highlights/notes in Zotero, Readwise etc).
However I already have a very large data set of books (15,000+) stored in Endnote (and Bookends) but also in exportable file formats (RIS, CSV) etc. How do I import this dataset into Obsidian. Surprisingly I found zero help on line. I’m guising this would be easy to do in Notion (which Im avoiding!). The idea would be to create separate pages for each record and include all the fields I have set up (way more than the Zotero limited field set) entered in as YAMAL or front mater metadata. I’m sure this is easy to do, and just can’t figure it out cause im a newbie and just looking into this. Effectively I want to preserve all my years of keywords, tagging, ratings etc stored in my Endnote fields (or as an Excel sheet…) and bring it into Obsidian so I can use dataview or database to make new quires and organize my books (ideally with links to the books cover image attachments).
Can someone redirect or help me figure out how to do this?
Ok well I found the Obsidan CSV/JSon importer plugin, which is able to create separate pages for each record, a lot of faffing around (handlebars etc) but configurable. However if there is an easier method (I don’t just want to create a table of data within a page, like the plugins CSV Table etc) something more like the Readwise notes importer, but for Endnote instead, please clue me in. Or maybe Im going about this the wrong way?
Thanks mbbcam, yes I’ve seen his videos but it’s Zotero specific and still does not show a clear way to import files stored in other systems… however I recently noticed that the Database Folder plugin also has a CSV import feature, so I think that might be the key.
I see you mention Bookends, which is the reference manager that I use, so I assume you are using a Mac. That opens up various possibilities of using AppleScript or other methods for integration between Obsidian and other programs. (As an aside, I haven’t used EndNote for over twenty years – I found it a bit painful on a Mac, though things may have improved.)
@ryanjamurphy has published scripts for integrating Bookends and Obsidian, and though I’ve only dabbled with them so far (too much else to do at the moment) I think they will be very useful. I don’t think he envisaged the bulk import of data when he wrote the scripts, however. But that raises the question of whether it is a good idea to import the whole of your databases into Obsidian. I certainly wouldn’t do it myself. I would keep all my data in Bookends, and link selected items I was working on to notes in Obsidian. That said, I obviously do not know quite what you are hoping to use Obsidian for. (As a further aside, I also have my Bookends attachments folder indexed by DEVONthink, which is an aid to finding material when I need it.)
Database Folder is very good, and so is Dataview. The developer of Dataview is apparently working on an even more powerful plugin called Datacore, which might open up further possibilities.
Thanks again mbbcam… yes I’m waiting on Datacore, … as for Books: it’s a bit convoluted, so feel free to not read my foolish predicament… but here is a brief overview. While I’m now primarily using Zotero as my main book/journal/pdf and reading repository, I really love seeing books organized by and grouped by book covers, and at first many years ago when I was in grad school, I used FileMaker and Endnote… but of course all data entry was manual… fast forward a few years, and then Sente displaced Endnote… then Sente tanked, then I found Delicious (now junk) to simply use the book cover download and display on virtual bookshelf feature (all of this is really pure aesthetics and has no integral connection to actually getting work done)… but I was still desperate to have one system to do everything… custom fields, book cover display and download, metadata extractions, auto pdf download etc… long story short, when Bookends had over 5K references it began to slow down … then I discovered Bookpedia… which handles images better but is not as good at actual citations etc… then in 2018 the project of a unified do it all book database / reference manager / bibliography citation writing tool, floundered and I just went back to basics for a while especially during covid … then I recently decided to start over to see if anything new had come up, and then stumbled on Obsidian (while auditioning Scrivener) … and really saw the power of putting my notes and ideas into that. But Im still somewhat obsessed with making a system that can “display” books I want to prioritize reading as book covers, so I can SEE my shelf, say of books for a seminar, books for an article or book I’m working on, etc. … I just always loved being in Book stores and handing my own physical collections, but now with that entity almost extinct and the decisive move towards ebooks/pdfs, the next best way to organize ones collections and visualize would be with an app like Delicious (which was junk as an actual reference manager) or Bookpedia/Bookends. Anyway so the project has reared its head and Im trying to find solutions for a problem or wish fulfillment of my own making! At this point I want to have just two apps… so the task is to see how I can set up a complete database of Books (not articles) (over 10K, 5K I own, 5K downloaded) that I can play around with and keep as the final repository for the complete data set (while Zotero can remain the initial capture place, metadata, annotation extractor etc)… so I have to figure out how to import images that were stored as downloaded jpeg links in Bookends, or import my data set (from a CSV file) in Bookpeida (which is also good at getting the book covers and then adding them and seeing if I can export from there into Obsidian … all of this is more a fun side project as I mentioned.
However I might just bag the idea altogether if I feel Im wasting too much time on it, or overwhelmed (which is what I feel right now) … and just giving up on the idea of cover flows, gallery views etc and simply use Zotero for reading, citations and just export the notes, like you said, of just books/articles Ive highlighted annotated for further development in Obsidian. But Cover Flow! Seeing my book images and being able to resize them as covers like photographs… I miss that. Just don’t want want to maintain TWO separate databases for it all while using Obsidian also… maybe Notion is better for all of this, but it strikes me as less capable for concept development (links, graphs etc). So in a nut shell that’s it.