Problem
Some users may store huge files up to gigabytes in their vaults. For some use cases (for example, backup the vault to a cloud service), the plugin needs to read such large files.
The only option is vault.adapter.readBinary(), which reads the entire file at once. The memory usage will surge and Obsidian will definitely crash. Although the plugin can selectively skip large files, this is an obvious UX degradation.
To control memory usage, production level software rely on OS file system, which natively supports this. But in Obsidian, this is impossible.
I saw there’s vault.adapter.appendBinary() added in v1.12.3 that allows me to write a file partially. This API helped me solve a whole range of download-related memory overuse crashes. I hope there’s also a ranged file read to solve upload-related OOM.
Purposed API Shape
vault.adapter.readBinary(path: string, options?: { start?: number; end?:number; })
Accept a small options object in readBinary. options.start and options.end specifies the byte range (inclusive, first byte index is 0) to read.
When options is not provided or either options.start and options.end is not specified, options.start defaults to 0, options.end defaults to file byte size - 1. Throw error on:
options.start >= options.endoptions.start < 0options.end > fileSize - 1options.startoroptions.endis not an integer