Linux packaging: ship as flatpak on flathub

Basement
1

Hello,

Linux user here. I’ve noticed obsidian isn’t shipped as a flatpak yet, and I think it would be very beneficial for obsidian to consider doing so. The two current available formats are snap and appimage, and they both have issues that flatpak can address:

  • Snap isn’t all that popular or well-supported outside of ubuntu. Even ubuntu derivatives such as xubuntu and linux mint disable snaps by default, because snap is quite an invasive technology in my opinion. Canonical is aggressively pushing those and even went out of their way to change the chromium package to an empty husk that installs the chromium snap to push its adoption.
    Flatpak is typically more distribution-agnostic, its adoption isn’t driven by these kinds of practices with questionable ethics, and it has no daemons running in the background after the application has quit.

  • Appimage is great for “portable” applications that you can just click and run, but they lack the level of system integration that flatpaks and snaps can provide, such as installing icons and desktop files. Without manually installing icons, the obsidian application in my dock (Gnome) is just blank. It’s also slightly slower (1-2s delay) to launch since the appimage will have to be mounted first. Additionally, because appimage is immutable by design, OTA update isn’t possible (on windows you can just update the asar itself), so the user has to:

    • re-download the appimage
    • move it to their desired directory
    • chmod +x on it
    • finally rename it to make it match their existing desktop file they have manually installed in their home directories

  • Here are the benefits of providing a flatpak package and ship it on flathub:

    • Easier installation and updates. Shipping the flatpak on flathub will enable the users to install the latest version of obsidian with one command (3 commands at most if the user did not have flatpak installed and have flathub added as a repository), with none of the aforementioned annoyances that I previously mentioned about the appimage. If they have a software market type application such as gnome software, they can simply click on install in the GUI. The desktop file and the icons are all properly set up during the installation process, and the user don’t have to do anything after the installation to start using obsidian. Updating is trivial as the user just need to run flatpak update or use a compatible GUI frontend such as gnome software, and since delta updates are supported, this will download a similar amount of data (essentially just the asar most of the time) that the windows version would to complete an update.
      Here’s a list of distro-specific setup instructions: https://flatpak.org/setup/
    • Giving the user better control over Obsidian. Obsidian isn’t open source, even though I trust it with my information, some users might not. Flatpak provides some nice sandboxing capabilities without sacrificing much in terms of usability, and the user can easily customize the sandboxing rules by using a GUI application called Flatseal if they find the defaults not suitable for their use cases. For example, I can totally disable network access, or restrict filesystem access to my home folder only. We wouldn’t even run into these problems because the user can just change the sanboxing rules with an intuitive GUI: [Linux] snap version doesn't allow file access outside home folder
    • Flatpak is distribution-agnostic. Unlike snaps, they work equally well on ubuntu (including derivatives) and non-ubuntu derivatives. Flatpak has a runtime concept, which solves the dependency problem like shipping libs inside the appimage does. You can target a runtime with all of the common libs you need, and build your flatpak on top of it. Since this exact runtime is used everywhere across all distributions, it won’t undo the benefit that appimage already provides. This has the obvious disadvantage of incurring a larger initial download, but since the runtimes are reusable across applications, but imho this is an okay trade-off. Users who are not happy with the runtime size can obviously continue to use the appimage.
    • Nitpick: both snap and appimage clutter my system mounts, but flatpak does not.

A friend was kind enough to try building an experimental flatpak with the unpacked appimage, and it runs great: https://gist.github.com/joshua-stone/20e92c9a78f2080b2e9b7edd79c97dc9

There are many other examples available here: https://github.com/flathub/flathub/pulls

Thanks for making Obsidian, and I hope you’ll consider my suggestion. :slightly_smiling_face:

Additional info:

15 Likes
2

Just wanna take the time to appreciate such a detailed feature request, you made a good case, hopefully it’d be considered seriously !

4 Likes
3

I second an easier distribution/install method for Linux like a flatpak — as a recent convert to Pop!_OS the install was not the easiest, though it works.

1 Like
4

We will likely be able to build flatpaks once 1.0 releases.

For now, I don’t think we will maintain presence on external hosting sites like Flathub or Snapcraft - there’s a fair amount of them (there’s also brew for MacOS and chocolatey for windows) and with our release frequency (1-2 release/week) it cuts into development time and complicates the release process.

As for automatic updates, we maintain our own update system which is much much optimal than what a platform updater can offer. This is because we only update the .asar files which is less than 5mb as of current latest version. We’ve only made a single full electron update from 8.2.5 to 10.1.1 (in obsidian v0.8.12), which unlocked additional functionality if you did the optional electron upgrade. This update is shipped to the user-preference folder ./config/obsidian/ which is outside of the AppImage already.

Totally understand the user experience though. I think for AppImage, you can use AppImageLauncher (https://github.com/TheAssassin/AppImageLauncher) which is designed specifically to install AppImages with the .desktop files properly.

We’ll think about building flatpaks. Right now I’m just really disappointed in the Linux ecosystem for being so fragmented.

3 Likes
5

This is exactly what we do right now for all platforms.

Also ouch.
image

6

This would make it much easier to install Obsidian for Linux users.

1 Like
7

We will likely be able to build flatpaks once 1.0 releases.

Thanks for considering it. :slightly_smiling_face:

For now, I don’t think we will maintain presence on external hosting sites like Flathub or Snapcraft - there’s a fair amount of them (there’s also brew for MacOS and chocolatey for windows) and with our release frequency (1-2 release/week) it cuts into development time and complicates the release process.

I’m very much aware of this problem and would hate to let these mundane busy work getting in the way of actual development, but this is exactly why I suggested shipping the flatpak on flathub:

  • First of all, no developer account is required for publishing on flathub. Spending time and effort on maintaining presence is not necessary. All you need to do is submitting a pull request, and then do minimal metadata maintenance for new releases, as I’ll elaborate below with examples.
  • The basic steps of publishing a flatpak on flathub goes like this:
    1. Submit a pull request here: Pull requests · flathub/flathub · GitHub
    2. Make changes if needed and eventually a dedicated repository will be created
      • example: Zotero
      • The only change that I personally think would be beneficial is to disable built-in OTA by default in this flavor, so updates can be taken care of by flatpak instead. This will not force the user to re-download the static electron releases, because as I’ve mentioned in the main post before, flatpak has delta updates, so essentially this will retrieve a similar amount of data from the internet like a typical .asar-only OTA would.
    3. Flatpak bundles will be automatically built and distributed by flathub
  • The basic steps of maintaining this flatpak is also very easy. Suppose you have a .tar.gz or similar release on github, and you have an update to push, all you need to do is to push a commit to change the file hash, file URL and the version info in the metadata, and everything else will be taken care of by flathub. This commit from the Zotero flatpak repo is a very good example to showcase the amount of maintenance effort (or lack thereof :wink:) required: Update to Zotero 5.0.89 · flathub/org.zotero.Zotero@859fc9a · GitHub

As for automatic updates, we maintain our own update system which is much much optimal than what a platform updater can offer. This is because we only update the .asar files which is less than 5mb as of current latest version. We’ve only made a single full electron update from 8.2.5 to 10.1.1 (in obsidian v0.8.12), which unlocked additional functionality if you did the optional electron upgrade. This update is shipped to the user-preference folder ./config/obsidian/ which is outside of the AppImage already.

https://blogs.gnome.org/mclasen/2018/08/02/on-flatpak-updates/

Flatpak has feature-parity with your own OTA system.

  • Your OTA mechanism provides 2 main benefits: efficiency and stability. Flatpak can provide these benefits equally well, as I’ll address down below.
  • OTA is efficient because only the asar is ever updated, and technically flatpak can do this as well, just in a different way. Flatpak does delta updates, which essentially only downloads the asar anyway if the Electron base has not changed. Using flatpak does not imply the user must download the same Electron base over and over again. Most other packaging formats do not provide this feature, or are rather tedious to use/setup (i.e. RPM).
    • Note: as someone who is almost always on a slow and metered connection, I would not have recommended flatpak if it does not support delta updates.
  • OTA helps with stability because the new asar will only be used on the next startup, so nothing will be swapped out while the application is running. Typical Linux package managers have an awful habit of pulling the rug out under running applications if you choose to update something while it’s running, which causes a lot of stability issues. For example, firefox gets incredibly upset if you update itself or some important libs while it’s running. Flatpak does not do this, it will not only keep existing deployments around until the last user is gone, and it will also inform the application that it has been updated. Obsidian can simply look for /app/.updated and prompt the user to restart the application, if you decide to take advantage of this feature.

Totally understand the user experience though. I think for AppImage, you can use AppImageLauncher (https://github.com/TheAssassin/AppImageLauncher ) which is designed specifically to install AppImages with the .desktop files properly.

AppImageLauncher is only packaged on Ubuntu (as .deb). This has the same distribution lock-in effect as snap does. It’s true that anyone could install this from source or use the provided appimage to install, but do you really want to set this kind of complex prerequisite for your software to integrate well into people’s systems?

If Obsidian is shipped on flathub, most users with some kind of software store (gnome software for example) can install and update Obsidian with a single click, as long as flathub has been added as a repository on their systems. If not, flathub has detailed setup instructions for pretty much every single distribution now, which is typically one or two commands that just needs to be copy-pasted into the terminal.

We’ll think about building flatpaks. Right now I’m just really disappointed in the Linux ecosystem for being so fragmented.

As a Linux user, I’m equally disappointed as well, and there are good reasons why I did not ask about distribution packaging, debs and rpms. Thanks for your time, and I hope this reply can clarify some things for you. Please let me know if you have any other questions.

3 Likes
8

Snap also can have major performance impacts due to what I’ve understood as the app basically running in a virtual machine of sorts.

10

I personally find AppImage > Flatpak. Most Flatpaks I tried, had issues. Slack couldn’t see my home folder without tweaks, XMind kept thinking it was run for the first time, every single time I ran it, and I have more examples, but instead of moaning about the shortcomings of Flatpak, I think the main question here is “when to stop”.

As @Licat mentioned, the Linux ecosystem is horribly fragmented, thus why it’s been failing on the desktop for decades now (i been hopeful since the late 90ies). The effort required for app developers to support “everything” is unreasonable. Yes, the model we have works great for open source software, but alas, not all great software will be open source. And this is what we now have X different projects trying to address, which is kinda funny when you think about it, we have fragmentation in our effort to solve problems caused by fragmentation.

Honestly, AppImages are fine. Spend the effort on developing the app, rather than supporting Snap, Flatpak and the 3-4 other tools bound to land in the near future to solve the exact same problem (this is the Linux way). And let us Linux users worry about how to get the AppImage working, as mentioned above, we can just install appimagelauncher and everything is great. You can even update from your launcher I’m told (i run a boring tiling WM, so I get no such joy, but that’s my choice though).

IMHO, the real pressure should be on the distro makers to just support app image out of the box. All they need to do is install appimagelauncher by default, configured to move your appimages to an “Applications” directory, and hey presto, problem solved.

@htuy I feel the statement about appimagelauncher and distro lock-in is slightly exaggerated. I haven’t looked into to it in anger, but it was no problem to get it running for me on Arch Linux. I haven’t tried anything different, but a quick google showed guides to how to install in on Fedora as well. I’d be surprised if AppImageLauncher itself would be a blocker.

2 Likes
12

Just 2 days ago, I was also discussing about flatpak in discord.

@ htuy nice explanation.

Hope it comes soon.

1 Like
16

I get that we Linux users tend to be opinionated, but please try to keep the discussion focused on the feature request.

1 Like
17

I apologize, I’ll disengage from this conversation above and stay on-topic.

18

I hope all the flagged posts stay.
From my uninvolved point of view, as a low commitment Linux user since the 90s (mostly in Windows atm), they encapsulate much about current circumstances across the communities.

Informative to outsiders. Nothing wrong with disagreement reasonably expressed.

1 Like
19

I would love to see a flatpack release, but understand why the initial setup required to provide it might make it a longer-term thing.

20

Hey everyone, creator of the experimental Obsidian Flatpak manifest here. I’d be quite happy if the Obsidian developers decided to release Flatpak builds!

One of the things I personally like about Flatpak is that there’s a lot of flexibility in how you can configure a build. It could be submitted to a centralized, community-maintained repository like Flathub which improves discoverability and ensures updates are handled in one place. This is a model which should be familiar to Linux users who’re used to the package management model seen in most distributions.

Flatpak builds could also be distributed as standalone packages if desired, similar to AppImage. For example, the Basemark GPU team has taken the liberty of self-hosting builds for their benchmarking tool as well as submitting it to Flathub:

Speaking as a developer, I really appreciate the design around containers since dependencies are handled in a truly cross-distro way. By building and running applications in a containerized fashion, the host operating system is no longer a factor. This containerized approach can work in harmony with many CI systems, not dissimilar to Docker/podman. This is in contrast with AppImage where developers have to make assumptions about libraries that would be installed on most distributions.

Getting AppImage to run at all on niche distros like Alpine Linux is still an ongoing task, and there are some fundamental limitations with the current AppImage implementation where the AppImage runtime needs to be dynamically-linked to glibc and FUSE. This really hinders AppImage’s potential as a cross-distro format, and might not be fixed so long as there’s a hard dependency on FUSE. Flatpak on the other hand doesn’t have this limitation – by running everything inside a container, developers no longer have to worry about whether the host is glibc-based or musl-based, has FUSE support, or has libraries designed with static-linking in mind (among other edge cases).

It’s a breath of fresh air that developers can say that they support Desktop Linux instead of having to specifically say Ubuntu, Fedora, openSUSE, Arch, etc. Upstream developers no longer have to maintain packages for specific distribution releases, so download pages can look like this:

https://www.flathub.org/apps/details/org.clementine_player.Clementine

Instead of this:

https://www.clementine-player.org/downloads

I think something that cannot be understated is the amount of inertia Flatpak has gained already. Gnome and KDE – two of the most prolific desktop environments – have integrated Flatpak support for some time now, and many popular desktop distributions including Fedora, Mint Linux, Pop!_OS, elementary OS, and OpenSUSE have enabled support. A lot of work has been done in the past few years to provide seamless integration of desktop shortcuts, mimetype associations, updates, etc, so it can complement distribution package managers without actually being tied to a specific distribution.

Here is an example comparing the UX for AppImage and Flatpak install methods in Gnome Shell:

Screenshot_silverblue32_2020-09-27_12:14:27
Screenshot_silverblue32_2020-09-27_12:14:271920×1080 64.3 KB

Screenshot_silverblue32_2020-09-27_12:15:21
Screenshot_silverblue32_2020-09-27_12:15:211920×1080 292 KB

While troubleshooting the issue in the first screenshot is trivial for experienced users, I think it’s fair to say that it’s not a great first impression for anyone just testing an application on Linux in 2020.

One might be asking – can’t Linux users put in the work to manage applications their own way like they’ve been doing since the 90s? My answer – We don’t have to anymore! We have the technology to bring application management to the same level of convenience that Windows and macOS users have been benefitting from for a long time!

A lot of progress has happened in the past several decades for advancing the state-of-the-art in application deployment on the Linux desktop – in the beginning binaries were distributed as tarballs, then came along .sh/.run/.bin self-extracting files, after that came AppImage, and now it seems like the community is putting work into supporting Flatpak as the latest standard in desktop application management so upstream developers can deploy applications directly to users with as little effort as possible.

If any of the developers have any interest in learning more about Flatpak, then feel free to reach out to me!

6 Likes
21

First visit to the community here. And first thing I’m looking for is Flatpak support. Thanks for the feature request. Hope this comes soon.

1 Like
22

Hey again everyone! I’ve went ahead with creating a pull request on Flathub:

Anyone curious to try testing this build shouldn’t need to run more than:

git clone --branch=md.obsidian.Obsidian https://github.com/joshua-stone/flathub
cd flathub
flatpak-builder --install --user --install-deps-from=flathub --force-clean _build md.obsidian.Obsidian.yml
flatpak run md.obsidian.Obsidian

@Licat @argentum As mentioned in this document, app submissions generally should involve reaching out to upstream developers:

Please let me know if you’d be interested in maintaining an Obsidian build on Flathub, or if there are any questions you may have.

4 Likes
23

The pull request has been merged and Obsidian is now available on flathub.
Thanks to @jstone for doing the heavy-lifting and @Licat for actively participating in this discussion.

@justindirose @cabin.mooring @boydo :wink:

24

Amazing! it was perfect on my system (I use Pop! _OS 20.04).
In the Snap version, nautilus’ GTK theme was bugged when it was necessary to create or open a Vault, the launch took longer and had to run a command on the terminal to install.
In the AppImage version, the application launch was fast, however, the obsidian icon did not appear in the gnome menu and needed the AppImageLauncher to integrate Obsidian into the system.
With this flatpak version I don’t have these problems, it made it a lot easier.

1 Like
25

Thank you, thank you, thank you! This makes thinks so much easier.