What happened
This march I accidentally formatted a local drive containing my vault and was looking for ways to recover lost data, as much as possible. There was no backup, no sync, nothing. Even no record of the vault in AppData file Obsidian.json, that could contain it’s ID and address.
I tried some surface-level stuff like roaming through AppData files, which earned me back a couple of cached .md-files, but not more. I created a post Searching for ways to get my vault back but it only confirmed my fears - the data is probably lost.
Still, I decided to dive as deep as I could and recover anything I could find. In the end, I’ve managed to restore significant amount of data and even more hints that later helped me a lot to reassemble the vault from notes and files scattered across messengers, emails, and other non-systemized notes.
In the process I’ve tried several approaches, some of them were fruitful, others turned to be dead ends, and I thought that my experience might become useful for someone who someday finds themself in similar situation.
Things I have done
My efforts can be divided into two groups - HDD recovery and trying to dig into Obsidian itself in order to gather snippets of data.
HDD recovery
If you are facing similar problems, I’d suggest going for data recovery first - it could be the fastest and the most fruitful way to get your data back in case the vault was on an HDD. (if it was SSD, situation is a lot worse - deleted data gets regularly ‘trimmed’ and in my case would’ve be gone for good by the time I noticed absence of the vault).
Free or paid utilities like TestDisk, DMDE, R-Studio can be used to recover lost filesystems and their content in lots of cases. Even if the filesystem is damaged, you can still try to extract files devoid of their names by running utilities like Photorec (or using respective options in aforementioned DMDE, R-Studio etc).
Unfortunately, it did not work in my case - the original local disc was 500GB in volume, got formatted twice with creation of two EFI-partitions from it’s starting physical sectors, and the only data I was keeping on that local disc was the vault which was less then 250mb. Since data tends to be stored in the starting sectors and that’s precisely where all the storage manipulations were held, both filesystem and files evidently got overwrtitten. Within desired sectors I only managed to recover temporary files used by OS while it was installed, mostly archives and documentation files.
It was even more frustrating to see that software spotted a lot of ‘deleted’ files I did’t need still chilling on other local discs as they were never overwritten.
If not for all the extra-partition creating and doulble-overwriting, if only the local disc was not 99% empty and the desired data was stored in non-starting physical sectors, that would be it - I’d either be able to restore both files and their file tree, or just the files with content but to name. Even in that second case, since files were mostly text, it’d take not more than a couple of hours to open them in series and restore their names, with that restoring the node system.
Hopefully, you’ll be more lucky and this approach will suffice in your case.
Tinkering Obsidian
I didn’t know this in advance, but Obsidian is chromium-based, which allowed me to enter devtools (Ctrl+Shift+I), look into stored data and use console in my endeavours.
-
by going to
Application - > Storage - > IndexedDByou can get a list of stored DBs. These sets have IDs in ther names, so if you used to have multiple vaults, note that DBs with same ID refer to data relevant to the same vault. System DBs are named by vault ID followed by ‘-backup’,‘-cache’ or ‘-sync’. Plugins also have their DBs and those use vault IDs in their names, too. I had DBs containing 4 different IDs, and guessed which of them was the one by presence of dataview and excalidraw DBs as it was the only vault using plugins. You can also get the same list of DBs by running commandindexedDB.databases().then(dbs => console.log(dbs));in the console. -
the following command gets you the number of files present in cache DB relevant to the Vault (you need to paste vault ID instead of [ID] there):
indexedDB.open("[ID]-cache").onsuccess = function(event) {
let db = event.target.result;
let transaction = db.transaction("file", "readonly");
let store = transaction.objectStore("file");
let countRequest = store.count();
countRequest.onsuccess = function() {
console.log("Number of cached files:", countRequest.result);
};
};
Running this command with the suspect ID showed me 249 files, this number fitted my estimation of the total amount including both .md files and attachments. I was hoping that data IS there, it just is encoded. SPOILER:
- After a LOT of tinkering I figured several things. Cache-DB contains list of files, list of headings within those files, their obsidian links, and list of attachments (not the files, just their names). No text. Dataview plugin DB contatins caches some headings (not all of them) and some of the original text.
While on my way to those conclusions I’ve gone through several sycles of “it’s over - we’re back” because it was always the feeling that full version of data is just over the corner waiting to be discovered, decoded, and reassembled.
In the end, I came up with two scripts that transform DB key-value pairs into somewhat readable .md files and recreate a map of vault with all files and links between them in the folder you order. First one creates that map from Dataview DB and files contain approx. 30% of headings and text, the second one creates a map from cache DB, it’s files contain no text, but all the headings, all the attached file names, and list of lines, every one of which represent one paragraph of text, which helps to estimate the size of a note.
Scripts are provided below. You need to paste vault ID and directory paths !
Script for cache DB
let request = indexedDB.open(“[YOUR_VAULT_ID]-cache”); // PLACE YOUR ID
request.onsuccess = function (event) {
console.log(“IndexedDB opened”);
let db = event.target.result;
let fileTransaction = db.transaction(["file"], "readonly");
let fileStore = fileTransaction.objectStore("file");
let fs = window.require("fs");
let path = window.require("path");
let outputDir = "[DIRECTORY]"; // PLACE YOUR DIRECTORY
if (!fs.existsSync(outputDir)) {
console.log(`Creating folder: ${outputDir}`);
fs.mkdirSync(outputDir, { recursive: true });
}
let fileCursor = fileStore.openCursor();
let savedCount = 0;
let queue = [];
fileCursor.onsuccess = function (event) {
let cursor = event.target.result;
if (!cursor) {
console.log(`File records processed`);
processQueue();
return;
}
let filePath = cursor.key;
let fileMeta = cursor.value;
if (!fileMeta || !fileMeta.hash) {
console.log(`Skipping ${filePath} (no hash)`);
cursor.continue();
return;
}
queue.push({ filePath, hash: fileMeta.hash });
cursor.continue();
};
function processQueue() {
if (queue.length === 0) {
console.log(`Restored ${savedCount} files in ${outputDir}!`);
return;
}
let metadataTransaction = db.transaction(["metadata"], "readonly");
let metadataStore = metadataTransaction.objectStore("metadata");
function processNext() {
if (queue.length === 0) {
console.log(`Finished processing all queued metadata.`);
return;
}
let { filePath, hash } = queue.shift();
let metadataGetRequest = metadataStore.get(hash);
metadataGetRequest.onsuccess = function (event) {
let metadata = event.target.result;
if (!metadata) {
console.log(`No metadata found for ${filePath} (hash: ${hash})`);
processNext();
return;
}
let fullPath = path.join(outputDir, filePath);
let fileDir = path.dirname(fullPath);
if (!fs.existsSync(fileDir)) {
fs.mkdirSync(fileDir, { recursive: true });
}
let content = [];
// Extract links
if (metadata.links) {
metadata.links.forEach(link => {
if (link.original) {
content.push(link.original);
}
});
content.push("");
}
// Extract headings
if (metadata.headings) {
metadata.headings.forEach(heading => {
let level = heading.level || 1;
let headingText = `${"#".repeat(level)} ${heading.heading}`;
content.push(headingText);
});
content.push("");
}
// Extract lists
if (metadata.listItems) {
metadata.listItems.forEach(item => {
content.push(`- ${item.text || "Unknown list item"}`);
});
content.push("");
}
// Extract sections (text blocks)
if (metadata.sections) {
metadata.sections.forEach(section => {
if (section.text) {
content.push(section.text);
}
});
content.push("");
}
// Save file
try {
fs.writeFileSync(fullPath, content.join("\n"), "utf8");
console.log(`✅ Saved: ${fullPath}`);
savedCount++;
} catch (err) {
console.error(`❌ Error writing file ${fullPath}:`, err);
}
processNext();
};
metadataGetRequest.onerror = function (event) {
console.error(`❌ Error fetching metadata for ${filePath}:`, event.target.error);
processNext();
};
}
processNext();
}
fileCursor.onerror = function (event) {
console.error("❌ Error reading file storage:", event.target.error);
};
};
Script for DataView DB
let request = indexedDB.open(“dataview/cache/[YOUR_VAULT_ID]”); // PLACE YOUR ID
request.onsuccess = function(event) {
console.log(“IndexedDB opened”);
let db = event.target.result;
let transaction = db.transaction(["keyvaluepairs"], "readonly");
let fs = window.require("fs");
let path = window.require("path");
let outputDir = "[DIRECTORY]"; // PLACE YOUR DIRECTORY
if (!fs.existsSync(outputDir)) {
console.log(`Creating folder: ${outputDir}`);
fs.mkdirSync(outputDir, { recursive: true });
}
let store = transaction.objectStore("keyvaluepairs");
let getAllRequest = store.getAll();
getAllRequest.onsuccess = function() {
let data = getAllRequest.result;
console.log(`Found ${data.length} files in keyvaluepairs`);
data.forEach(entry => {
if (!entry.data || !entry.data.path) {
console.log(`⚠ Skipped an entry without a path`);
return;
}
let filePath = path.join(outputDir, entry.data.path);
let fileDir = path.dirname(filePath);
// Ensure the directory exists
if (!fs.existsSync(fileDir)) {
fs.mkdirSync(fileDir, { recursive: true });
}
// Ensure the file has the correct extension
if (!filePath.endsWith(".md")) {
filePath += ".md";
}
console.log(`Restoring file: ${entry.data.path}`);
let content = [];
// Add links at the beginning of the file
if (entry.data.links) {
entry.data.links.forEach(link => {
if (link.value?.display) {
content.push(`[[${link.value.display}]]`);
}
});
content.push(""); // Empty line for separation
}
// Group text by headings
let structuredContent = {};
if (entry.data.lists) {
entry.data.lists.forEach(item => {
let heading = "Untitled"; // Default heading
if (item.section?.value?.subpath) {
heading = item.section.value.subpath;
}
if (!structuredContent[heading]) {
structuredContent[heading] = [];
}
if (item.text) {
let textLine = item.symbol ? `${item.symbol} ${item.text}` : item.text;
structuredContent[heading].push(textLine);
}
});
}
// Add headings and their content
Object.entries(structuredContent).forEach(([heading, paragraphs]) => {
content.push(`# ${heading}`); // Heading
content.push(...paragraphs); // Paragraphs under heading
content.push(""); // Separation
});
// Write the file
try {
fs.writeFileSync(filePath, content.join("\n"), "utf8");
console.log(`✅ File saved: ${filePath}`);
} catch (err) {
console.error(`❌ Error writing file ${filePath}:`, err);
}
});
};
getAllRequest.onerror = function(event) {
console.error("❌ Error reading keyvaluepairs:", event.target.error);
};
};
Merging these two ghosts of a vault gives you back part of your data and provides good hints on what the missing data might be - you just start recalling it by observing headings attachment names.
In my case it was very useful, as I am a relatively new Obsidian user and this vault was a work-in-progress project to gather notes, documents, and project logs from several sources I’ve been storing them before to one place. Using the restored half-vault as a map, I knew where to look for the missing parts and eventually restored almost all of it.
-
After all these achievements I tried to go further and attempted to create a fake vault log in Obsidian.json file with the original Vault ID and stamps but a new folder with a recreated file tree, hoping that it might trigger parcing back cached text, that lies encoded in some yet undiscovered place. It did not work and stopped further experiments.
-
I am glad that I managed to restore some of the data directly, most of it indirectly, and have some fun in the process, but none of this would have happened if I paid enough attention and/or created backup before gathering all data in one place. For some reason I had this idea in my head that I have to finish building the vault first, and only then proceed to securing it.
I hope this story of mine might inspire someone to not follow my steps and take measures in advance, or help someone deal with existing problems in case things already have happened !