Rendering child process dies with "allocation failure"

When opening a larger vault, obsidian eventually freezes, and looking at a process monitor (see below), it seems that the main process is fine but the renderer has died.

My vault has only about 4,000 markdown files, but has 22,000 PDFs, 157,000 PNGs, and 475,000 files of any kind, for a total of 204G. I don’t have issues opening a smaller folder as vault (e.g. my Downloads folder (18 md / 106 pdf / 253 png / 11,000 any / 1.2G).

I actually suspect the “allocation failure” and or various gc messages are unrelated to the renderer crashing, and are benign.

As a clean test, I deleted ~/.config/obsidian and ~/Dropbox/.obsidian and still it crashed.

Environment

  • Operating system: Ubuntu 20.04

  • Debug info:

SYSTEM INFO:
	Obsidian version: v1.0.0
	Installer version: v1.0.0
	Operating system: #58~20.04.1-Ubuntu SMP Thu Oct 13 13:09:46 UTC 2022 5.15.0-52-generic
	Login status: not logged in
	Insider build toggle: off
	Live preview: on
	Legacy editor: off
	Base theme: dark
	Community theme: none
	Snippets enabled: 0
	Restricted mode: on

RECOMMENDATIONS:
	none

Additional information

My vault has only about 4,000 markdown files, but has 22,000 PDFs, 157,000 PNGs, and 475,000 files of any kind, for a total of 204G. I don’t have issues opening a smaller folder as vault (e.g. my Downloads folder (18 md / 106 pdf / 253 png / 11,000 any / 1.2G).

Details

With .deb installer:

[email protected]:~$ sudo dpkg -i ~/Downloads/obsidian_1.0.0_amd64.deb 
[sudo] password for tsbertalan: 
Selecting previously unselected package obsidian.
(Reading database ... 383141 files and directories currently installed.)
Preparing to unpack .../obsidian_1.0.0_amd64.deb ...
Unpacking obsidian (1.0.0) ...
Setting up obsidian (1.0.0) ...
Error in file "/usr/share/applications/org.kde.kdeconnect_open.desktop": "*/*" is an invalid MIME type ("*" is an unregistered media type)
Processing triggers for gnome-menus (3.36.0-1ubuntu1) ...
Processing triggers for desktop-file-utils (0.24-1ubuntu3) ...
Processing triggers for mime-support (3.64ubuntu1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
[email protected]:~$ which obsidian
/usr/bin/obsidian
[email protected]:~$ obsidian
2022-10-19 14:41:00 Loading main app package /opt/Obsidian/resources/obsidian.asar
libva error: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
2022-10-19 14:41:00 Checking for update using Github
[85358:1019/104100.764380:ERROR:sandbox_linux.cc(377)] InitializeSandbox() called with multiple threads in process gpu-process.
2022-10-19 14:41:00 Success.
2022-10-19 14:41:00 Latest version is 1.0.0
2022-10-19 14:41:00 App is up to date.
[email protected]:~$ obsidian --js-flags="--max-old-space-size=16384"
2022-10-19 14:46:08 Loading main app package /opt/Obsidian/resources/obsidian.asar
2022-10-19 14:46:08 Checking for update using Github
libva error: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
[86547:1019/104608.462123:ERROR:sandbox_linux.cc(377)] InitializeSandbox() called with multiple threads in process gpu-process.
2022-10-19 14:46:08 Success.
2022-10-19 14:46:08 Latest version is 1.0.0
2022-10-19 14:46:08 App is up to date.

<--- Last few GCs --->

[86553:0x156a003dc000]    86963 ms: Scavenge 837.1 (1205.8) -> 834.9 (1205.8) MB, 15.7 / 0.0 ms  (average mu = 0.982, current mu = 0.979) allocation failure; 
[86553:0x156a003dc000]    87190 ms: Scavenge 839.9 (1205.8) -> 837.8 (1205.8) MB, 15.3 / 0.0 ms  (average mu = 0.982, current mu = 0.979) allocation failure; 
[86553:0x156a003dc000]    87421 ms: Scavenge 842.8 (1205.8) -> 840.8 (1205.8) MB, 14.5 / 0.0 ms  (average mu = 0.982, current mu = 0.979) allocation failure; 


<--- JS stacktrace --->

[email protected]:~$ 



At the beginning of the obsidian invocation, htop looked like this:

Screenshot from 2022-10-19 10-53-53

The resident memory for the --type=renderer process grows to about 4500M, then it disappears:

Screenshot from 2022-10-19 10-51-01

Note that, for the attempt shown in these screenshots, I’ve actually tried invoking like obsidian --js-flags="--max-old-space-size=16384". The extra flag doesn’t help the problem. I’ve also tried adding --disable-gpu and --disable-software-rasterizer.

However, if I leave it sitting overnight, it will keep checking for updates:

2022-10-19 00:07:26 Checking for update using Github
2022-10-19 00:07:26 Success.
2022-10-19 00:07:26 Latest version is 1.0.0
2022-10-19 00:07:26 App is up to date.
2022-10-19 01:07:26 Checking for update using Github
2022-10-19 01:07:26 Success.
2022-10-19 01:07:26 Latest version is 1.0.0
2022-10-19 01:07:26 App is up to date.
 
 snip
 
2022-10-19 14:07:27 Checking for update using Github
2022-10-19 14:07:27 Success.
2022-10-19 14:07:27 Latest version is 1.0.0
2022-10-19 14:07:27 App is up to date.
[21 zypak-sandbox] Dropping 0x555989fe0270 (3) because of connection closed
[21 zypak-sandbox] Host is gone, preparing to exit...
[21 zypak-sandbox] Quitting Zygote... 

(That overnight run was with the flatpak installer. I have similar problems at 1.0.0 with dep, flatpak, and snap.)

With snap:

[email protected]:~$ flatpak remove md.obsidian.Obsidian


        ID                                  Branch         Op
 1. [-] md.obsidian.Obsidian                stable         r
 2. [-] md.obsidian.Obsidian.Locale         stable         r

Uninstall complete.
[email protected]:~$ sudo snap install obsidian
[sudo] password for tsbertalan: 
error: snap "obsidian" not found
[email protected]:~$ sudo snap install ~/Downloads/obsidian_1.0.0_amd64.snap 
error: cannot find signatures with metadata for snap
       "/home/tsbertalan/Downloads/obsidian_1.0.0_amd64.snap"
[email protected]:~$ snap a^C
[email protected]:~$ sudo snap ack ~/Downloads/obsidian_1.0.0_amd64.snap 
error: cannot assert: cannot decode request body into assertions: parsing assertion headers: header is not utf8
[email protected]:~$ snap download obsidian
Fetching snap "obsidian"
error: cannot download snap "obsidian": snap not found
[email protected]:~$ sudo snap ack ~/Downloads/obsidian_1.0.0_amd64.snap 
error: cannot assert: cannot decode request body into assertions: parsing assertion headers: header is not utf8
[email protected]:~$ sudo snap install ~/Downloads/obsidian_1.0.0_amd64.snap 
error: cannot find signatures with metadata for snap
       "/home/tsbertalan/Downloads/obsidian_1.0.0_amd64.snap"
[email protected]:~$ sudo snap install --dangerous ~/Downloads/obsidian_1.0.0_amd64.snap 
error: This revision of snap
       "/home/tsbertalan/Downloads/obsidian_1.0.0_amd64.snap" was published
       using classic confinement and thus may perform arbitrary system changes
       outside of the security sandbox that snaps are usually confined to,
       which may put your system at risk.

       If you understand and want to proceed repeat the command including
       --classic.
[email protected]:~$ sudo snap install --classic --dangerous ~/Downloads/obsidian_1.0.0_amd64.snap 
obsidian 1.0.0 installed
[email protected]:~$ which obsidian
/snap/bin/obsidian
[email protected]:~$ /snap/bin/obsidian
2022-10-19 14:37:59 Loading main app package /snap/obsidian/x1/resources/obsidian.asar
[84540:1019/103800.243473:ERROR:sandbox_linux.cc(377)] InitializeSandbox() called with multiple threads in process gpu-process.
2022-10-19 14:38:00 Checking for update using Github
2022-10-19 14:38:00 Success.
2022-10-19 14:38:00 Latest version is 1.0.0
2022-10-19 14:38:00 App is up to date.

<--- Last few GCs --->

[84658:0xce2003cc000]    85183 ms: Scavenge 809.3 (1260.4) -> 807.2 (1260.4) MB, 12.9 / 0.0 ms  (average mu = 0.984, current mu = 0.982) allocation failure; 
[84658:0xce2003cc000]    85386 ms: Scavenge 812.2 (1260.4) -> 810.1 (1260.4) MB, 12.5 / 0.0 ms  (average mu = 0.984, current mu = 0.982) allocation failure; 
[84658:0xce2003cc000]    85586 ms: Scavenge 815.1 (1260.4) -> 813.1 (1260.4) MB, 12.8 / 0.0 ms  (average mu = 0.984, current mu = 0.982) allocation failure; 


<--- JS stacktrace --->

[email protected]:~$ sudo snap remove obsidian
obsidian removed

With flatpak:

[email protected]:~$ flatpak install ~/Downloads/md.obsidian.Obsidian.flatpakref

md.obsidian.Obsidian permissions:
    ipc       network       pulseaudio            ssh-auth             wayland
    x11       dri           file access [1]       dbus access [2]      tags [3]

    [1] /media, /mnt, /run/media, home, xdg-run/app/com.discordapp.Discord:create
    [2] org.freedesktop.portal.Fcitx
    [3] proprietary


        ID                                Branch       Op       Remote        Download
 1. [✓] md.obsidian.Obsidian.Locale       stable       i        flathub         5.4 kB / 3.3 MB
 2. [✓] md.obsidian.Obsidian              stable       i        flathub       130.3 MB / 131.8 MB

Installation complete.
[email protected]:~$ flatpak run md.obsidian.Obsidian --js-flags="--max-old-space-size=16384"
[13 zypak-helper] Portal v4 is not available
2022-10-19 15:00:23 Loading main app package /app/resources/obsidian.asar
LaunchProcess: failed to execvp:
xdg-settings
Gtk-Message: 11:00:23.983: Failed to load module "canberra-gtk-module"
Gtk-Message: 11:00:23.983: Failed to load module "canberra-gtk-module"
[13:1019/110024.000195:ERROR:bus.cc(397)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[13:1019/110024.000256:ERROR:bus.cc(397)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
2022-10-19 15:00:24 Checking for update using Github
2022-10-19 15:00:24 Success.
2022-10-19 15:00:24 Latest version is 1.0.0
2022-10-19 15:00:24 App is up to date.
libva error: /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/iHD_drv_video.so init failed
[45:1019/110024.090148:ERROR:sandbox_linux.cc(377)] InitializeSandbox() called with multiple threads in process gpu-process.
[21 zypak-sandbox] Dropping 0x55ac7fffd270 (3) because of connection closed
[21 zypak-sandbox] Host is gone, preparing to exit...
[21 zypak-sandbox] Quitting Zygote...

Update: Tried enabling proprietary intel drivers (a long shot; they should already be active if relevant) like this; no effect:

sudo lshw -c video
sudo lspci -nn | grep -e VGA
sudo apt install mesa-utils
glxinfo -B
sudo apt install -y gpg-agent wget
wget -qO - https://repositories.intel.com/graphics/intel-graphics.key |   sudo apt-key add -
lsb_release -a
sudo apt-add-repository   'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu focal main'
sudo apt update
sudo apt install   intel-opencl-icd   intel-level-zero-gpu level-zero   intel-media-va-driver-non-free libmfx1 libva2

you are right, you may have hit the maximum file size your system can handle.

I don’t know if there are some other system parameters that you can toggle to make it start, maybe you are hitting some OS level or electron level ceiling.

I should mention also–the 1.0.0 release works fine on this Dropbox on my Windows machines, and in fact loads from cold faster than before. The new release did work on the Ubuntu machine a few days ago, and then I realized that I had the disk space to allow Dropbox to sync also my archives folder, which added about 100G. I’m pretty sure that’s when obsidian started crashing.

I think home windows machine syncs the whole Dropbox and still runs obsidian fine, but let me check and get back.

Windows reports that the home desktop vault (Dropbox plus another folder) is 717G, in 642,433 files, 84,508 folders. At this point, I’ve gone to the trouble of leaving work early, clearing a drive in my NAS and pulling it, and flashing an installer USB, so I’m going to just put a big Windows SSD in the work machine and use that for now. But I’ll keep the Ubuntu SSD unmodified, so I can test future versions.

Maybe you know better than me what you are trying to do, but I am pretty sure it is a mistake to make your entire Dropbox an Obsidian vault. It also doesn’t make any sense that you are opening your Downloads folder as an Obsidian vault.

The intention with Obsidian (as far as I know) is to make a separate folder for your vault, and store your notes in there. It isn’t a file system. It’s a note-taking app.

I think you are doing something you shouldn’t be doing. But maybe you have a plan or a reason I’m not grasping.

No, it works fine on all the Windows machines, and worked on the Ubuntu until I added that archives folder. Per my previous postings, wikilinking used to be too slow to use, but it’s also perfectly usable now. Note that that I include that Windows-717GB/650k-files-vault in my “works fine”.

I opened the downloads folder just as an “easy” test, not because it would be of any interest as a vault itself (although it does have a fair few PDFs and a surprisingly large number of markdowns).

This is a bug in the Electron build/packaging pipeline, or maybe just in how I start the program, not a fundamental issue with Obsidian.

1 Like

when Obsidian 1.0.4 is released. Download and reinstall it from the website (appimage or snap). Let me know if the situation improves.

I see only 1.0.3 installers on the website as of 2022-11-23. Should I try from some alternate page?

you miss this part

Ah, even just the when. I’ll watch for it, thanks.