Linux packaging: ship as flatpak on flathub

[Licat]

We will likely be able to build flatpaks once 1.0 releases.

For now, I don’t think we will maintain presence on external hosting sites like Flathub or Snapcraft - there’s a fair amount of them (there’s also brew for MacOS and chocolatey for windows) and with our release frequency (1-2 release/week) it cuts into development time and complicates the release process.

As for automatic updates, we maintain our own update system which is much much optimal than what a platform updater can offer. This is because we only update the .asar files which is less than 5mb as of current latest version. We’ve only made a single full electron update from 8.2.5 to 10.1.1 (in obsidian v0.8.12), which unlocked additional functionality if you did the optional electron upgrade. This update is shipped to the user-preference folder ./config/obsidian/ which is outside of the AppImage already.

Totally understand the user experience though. I think for AppImage, you can use AppImageLauncher (https://github.com/TheAssassin/AppImageLauncher) which is designed specifically to install AppImages with the .desktop files properly.

We’ll think about building flatpaks. Right now I’m just really disappointed in the Linux ecosystem for being so fragmented.

[Licat]

@htuy

OTA update isn’t possible (on windows you can just update the asar itself)

This is exactly what we do right now for all platforms.

Also ouch.

[marcelobritox3]

This would make it much easier to install Obsidian for Linux users.

[htuy]

We will likely be able to build flatpaks once 1.0 releases.

Thanks for considering it.

For now, I don’t think we will maintain presence on external hosting sites like Flathub or Snapcraft - there’s a fair amount of them (there’s also brew for MacOS and chocolatey for windows) and with our release frequency (1-2 release/week) it cuts into development time and complicates the release process.

I’m very much aware of this problem and would hate to let these mundane busy work getting in the way of actual development, but this is exactly why I suggested shipping the flatpak on flathub:

• First of all, no developer account is required for publishing on flathub. Spending time and effort on maintaining presence is not necessary. All you need to do is submitting a pull request, and then do minimal metadata maintenance for new releases, as I’ll elaborate below with examples.
• The basic steps of publishing a flatpak on flathub goes like this:
  1. Submit a pull request here: https://github.com/flathub/flathub/pulls
     • example: Zoom
     • example: GitKaren
  2. Make changes if needed and eventually a dedicated repository will be created
     • example: Zotero
     • The only change that I personally think would be beneficial is to disable built-in OTA by default in this flavor, so updates can be taken care of by flatpak instead. This will not force the user to re-download the static electron releases, because as I’ve mentioned in the main post before, flatpak has delta updates, so essentially this will retrieve a similar amount of data from the internet like a typical .asar-only OTA would.
  3. Flatpak bundles will be automatically built and distributed by flathub
• The basic steps of maintaining this flatpak is also very easy. Suppose you have a .tar.gz or similar release on github, and you have an update to push, all you need to do is to push a commit to change the file hash, file URL and the version info in the metadata, and everything else will be taken care of by flathub. This commit from the Zotero flatpak repo is a very good example to showcase the amount of maintenance effort (or lack thereof ) required: https://github.com/flathub/org.zotero.Zotero/commit/859fc9a388302c760af291beb9c339b85b2b5c26

As for automatic updates, we maintain our own update system which is much much optimal than what a platform updater can offer. This is because we only update the .asar files which is less than 5mb as of current latest version. We’ve only made a single full electron update from 8.2.5 to 10.1.1 (in obsidian v0.8.12), which unlocked additional functionality if you did the optional electron upgrade. This update is shipped to the user-preference folder ./config/obsidian/ which is outside of the AppImage already.

https://blogs.gnome.org/mclasen/2018/08/02/on-flatpak-updates/

Flatpak has feature-parity with your own OTA system.

• Your OTA mechanism provides 2 main benefits: efficiency and stability. Flatpak can provide these benefits equally well, as I’ll address down below.
• OTA is efficient because only the asar is ever updated, and technically flatpak can do this as well, just in a different way. Flatpak does delta updates, which essentially only downloads the asar anyway if the Electron base has not changed. Using flatpak does not imply the user must download the same Electron base over and over again. Most other packaging formats do not provide this feature, or are rather tedious to use/setup (i.e. RPM).
  • Note: as someone who is almost always on a slow and metered connection, I would not have recommended flatpak if it does not support delta updates.
• OTA helps with stability because the new asar will only be used on the next startup, so nothing will be swapped out while the application is running. Typical Linux package managers have an awful habit of pulling the rug out under running applications if you choose to update something while it’s running, which causes a lot of stability issues. For example, firefox gets incredibly upset if you update itself or some important libs while it’s running. Flatpak does not do this, it will not only keep existing deployments around until the last user is gone, and it will also inform the application that it has been updated. Obsidian can simply look for /app/.updated and prompt the user to restart the application, if you decide to take advantage of this feature.

Totally understand the user experience though. I think for AppImage, you can use AppImageLauncher (https://github.com/TheAssassin/AppImageLauncher ) which is designed specifically to install AppImages with the .desktop files properly.

AppImageLauncher is only packaged on Ubuntu (as .deb). This has the same distribution lock-in effect as snap does. It’s true that anyone could install this from source or use the provided appimage to install, but do you really want to set this kind of complex prerequisite for your software to integrate well into people’s systems?

If Obsidian is shipped on flathub, most users with some kind of software store (gnome software for example) can install and update Obsidian with a single click, as long as flathub has been added as a repository on their systems. If not, flathub has detailed setup instructions for pretty much every single distribution now, which is typically one or two commands that just needs to be copy-pasted into the terminal.

We’ll think about building flatpaks. Right now I’m just really disappointed in the Linux ecosystem for being so fragmented.

As a Linux user, I’m equally disappointed as well, and there are good reasons why I did not ask about distribution packaging, debs and rpms. Thanks for your time, and I hope this reply can clarify some things for you. Please let me know if you have any other questions.

[justindirose]

@htuy

This has the same distribution lock-in effect as snap does.

Snap also can have major performance impacts due to what I’ve understood as the app basically running in a virtual machine of sorts.

[justdanyul]

I personally find AppImage > Flatpak. Most Flatpaks I tried, had issues. Slack couldn’t see my home folder without tweaks, XMind kept thinking it was run for the first time, every single time I ran it, and I have more examples, but instead of moaning about the shortcomings of Flatpak, I think the main question here is “when to stop”.

As @Licat mentioned, the Linux ecosystem is horribly fragmented, thus why it’s been failing on the desktop for decades now (i been hopeful since the late 90ies). The effort required for app developers to support “everything” is unreasonable. Yes, the model we have works great for open source software, but alas, not all great software will be open source. And this is what we now have X different projects trying to address, which is kinda funny when you think about it, we have fragmentation in our effort to solve problems caused by fragmentation.

Honestly, AppImages are fine. Spend the effort on developing the app, rather than supporting Snap, Flatpak and the 3-4 other tools bound to land in the near future to solve the exact same problem (this is the Linux way). And let us Linux users worry about how to get the AppImage working, as mentioned above, we can just install appimagelauncher and everything is great. You can even update from your launcher I’m told (i run a boring tiling WM, so I get no such joy, but that’s my choice though).

IMHO, the real pressure should be on the distro makers to just support app image out of the box. All they need to do is install appimagelauncher by default, configured to move your appimages to an “Applications” directory, and hey presto, problem solved.

@htuy I feel the statement about appimagelauncher and distro lock-in is slightly exaggerated. I haven’t looked into to it in anger, but it was no problem to get it running for me on Arch Linux. I haven’t tried anything different, but a quick google showed guides to how to install in on Fedora as well. I’d be surprised if AppImageLauncher itself would be a blocker.

[xawele]

Just 2 days ago, I was also discussing about flatpak in discord.

@ htuy nice explanation.

Hope it comes soon.

[argentum]

I get that we Linux users tend to be opinionated, but please try to keep the discussion focused on the feature request.

[htuy]

I apologize, I’ll disengage from this conversation above and stay on-topic.

[Dor]

I hope all the flagged posts stay.
From my uninvolved point of view, as a low commitment Linux user since the 90s (mostly in Windows atm), they encapsulate much about current circumstances across the communities.

Informative to outsiders. Nothing wrong with disagreement reasonably expressed.

[cabin.mooring]

I would love to see a flatpack release, but understand why the initial setup required to provide it might make it a longer-term thing.

[jstone]

Hey everyone, creator of the experimental Obsidian Flatpak manifest here. I’d be quite happy if the Obsidian developers decided to release Flatpak builds!

One of the things I personally like about Flatpak is that there’s a lot of flexibility in how you can configure a build. It could be submitted to a centralized, community-maintained repository like Flathub which improves discoverability and ensures updates are handled in one place. This is a model which should be familiar to Linux users who’re used to the package management model seen in most distributions.

Flatpak builds could also be distributed as standalone packages if desired, similar to AppImage. For example, the Basemark GPU team has taken the liberty of self-hosting builds for their benchmarking tool as well as submitting it to Flathub:

Basemark GPU | Basemark

Speaking as a developer, I really appreciate the design around containers since dependencies are handled in a truly cross-distro way. By building and running applications in a containerized fashion, the host operating system is no longer a factor. This containerized approach can work in harmony with many CI systems, not dissimilar to Docker/podman. This is in contrast with AppImage where developers have to make assumptions about libraries that would be installed on most distributions.

Getting AppImage to run at all on niche distros like Alpine Linux is still an ongoing task, and there are some fundamental limitations with the current AppImage implementation where the AppImage runtime needs to be dynamically-linked to glibc and FUSE. This really hinders AppImage’s potential as a cross-distro format, and might not be fixed so long as there’s a hard dependency on FUSE. Flatpak on the other hand doesn’t have this limitation – by running everything inside a container, developers no longer have to worry about whether the host is glibc-based or musl-based, has FUSE support, or has libraries designed with static-linking in mind (among other edge cases).

It’s a breath of fresh air that developers can say that they support Desktop Linux instead of having to specifically say Ubuntu, Fedora, openSUSE, Arch, etc. Upstream developers no longer have to maintain packages for specific distribution releases, so download pages can look like this:

https://www.flathub.org/apps/details/org.clementine_player.Clementine

Instead of this:

https://www.clementine-player.org/downloads

I think something that cannot be understated is the amount of inertia Flatpak has gained already. Gnome and KDE – two of the most prolific desktop environments – have integrated Flatpak support for some time now, and many popular desktop distributions including Fedora, Mint Linux, Pop!_OS, elementary OS, and OpenSUSE have enabled support. A lot of work has been done in the past few years to provide seamless integration of desktop shortcuts, mimetype associations, updates, etc, so it can complement distribution package managers without actually being tied to a specific distribution.

Here is an example comparing the UX for AppImage and Flatpak install methods in Gnome Shell:

While troubleshooting the issue in the first screenshot is trivial for experienced users, I think it’s fair to say that it’s not a great first impression for anyone just testing an application on Linux in 2020.

One might be asking – can’t Linux users put in the work to manage applications their own way like they’ve been doing since the 90s? My answer – We don’t have to anymore! We have the technology to bring application management to the same level of convenience that Windows and macOS users have been benefitting from for a long time!

A lot of progress has happened in the past several decades for advancing the state-of-the-art in application deployment on the Linux desktop – in the beginning binaries were distributed as tarballs, then came along .sh/.run/.bin self-extracting files, after that came AppImage, and now it seems like the community is putting work into supporting Flatpak as the latest standard in desktop application management so upstream developers can deploy applications directly to users with as little effort as possible.

If any of the developers have any interest in learning more about Flatpak, then feel free to reach out to me!

[boydo]

First visit to the community here. And first thing I’m looking for is Flatpak support. Thanks for the feature request. Hope this comes soon.

[jstone]

Hey again everyone! I’ve went ahead with creating a pull request on Flathub:

Add md.obsidian.obsidian (Obsidian)

Anyone curious to try testing this build shouldn’t need to run more than:

git clone --branch=md.obsidian.Obsidian https://github.com/joshua-stone/flathub
cd flathub
flatpak-builder --install --user --install-deps-from=flathub --force-clean _build md.obsidian.Obsidian.yml
flatpak run md.obsidian.Obsidian

@Licat @argentum As mentioned in this document, app submissions generally should involve reaching out to upstream developers:

flathub/flathub

Please let me know if you’d be interested in maintaining an Obsidian build on Flathub, or if there are any questions you may have.

[htuy]

The pull request has been merged and Obsidian is now available on flathub.
Thanks to @jstone for doing the heavy-lifting and @Licat for actively participating in this discussion.

Flathub—An app store and build service for Linux

@justindirose @cabin.mooring @boydo

[marcelobritox3]

Amazing! it was perfect on my system (I use Pop! _OS 20.04).
In the Snap version, nautilus’ GTK theme was bugged when it was necessary to create or open a Vault, the launch took longer and had to run a command on the terminal to install.
In the AppImage version, the application launch was fast, however, the obsidian icon did not appear in the gnome menu and needed the AppImageLauncher to integrate Obsidian into the system.
With this flatpak version I don’t have these problems, it made it a lot easier.

[wischie]

Thank you, thank you, thank you! This makes thinks so much easier.

[justdanyul]

I thought I’d let the Flatpak prove me wrong and installed it.

On the 0.9.4 flatpak build, I can’t delete folders. Anybody else seeing this problem?

[jstone]

@justdanyul There are some potential causes of this behavior. Electron apparently has some questionable ways of handling trash across different desktops. I don’t believe it’s a flatpak issue per se, more like the concept of “system trash” isn’t well-defined on Linux and projects like Electron have to hack their own implementations around it, which then might not even behave as expected when run in an environment that has strict permissions checks like in the case of flatpak.

This particular bug can at least be worked around – set Deleted files to Move to Obsidian trash (.trash folder) like so:

The Obsidian devs may eventually add a global option for default trash behavior, but in the meantime I’ve written a workaround that runs every time Obsidian launches.

All you need to do to update and restart Obsidian:

$ flatpak update md.obsidian.Obsidian --assumeyes
$ flatpak run md.obsidian.Obsidian 

I hope that helps!

[htuy]

I’ll start an informal bug tracker here just to keep notes on fixed and current flatpak-specific bugs. The bugs are listed in chronological order. If you see frequent updates to this flatpak, this is why. Flatpak uses delta updates so only a tiny bit of data is downloaded on these bugfix updates.

Current:

Fixed:

• Audio recorder plugin reports “No microphone is connected” commit

  • PulseAudio access is now allowed by default

• Cannot access files on removable drives commit

  • The sandbox is now configured to allow access to /mnt, /media and /run/media by default.

• Trash doesn’t work commit

  • Workaround has been added to automatically set trash behavior to use the local .trash folder instead of system trash. Obsidian doesn’t have a global trash preference so we check each vault on startup and set the trash behavior to local. If you have created a new vault, please either go to Settings -> File -> Deleted files and set this to “Move to Obsidian trash”, or simply restart Obsidian.

• “Open in default app” and “Show in folder” options in the context menu doesn’t work commit

  • Electron relies on xdg-open for these operations, the xdg-utils in the current freedesktop runtime is way too old and doesn’t contain necessary bugfixes required to make this work. The latest version of flatpak-xdg-utils is now bundled as a workaround.