Flatpak applications are based on runtimes such as KDE or Gnome Runtimes. Both of these runtimes are actually based on Freedesktop SDK which contains essential libraries and services such as Wayland or D-Bus.
Recently there were a lot of discussion about supply chain attacks, so it might be interesting to ask how Freedesktop SDK was built. The answer can be found in freedesktop-sdk repository:
So it is built using an older version of Freedesktop SDK image. There is now an approved merge request that completely reworks bootstrapping of Freedesktop SDK. It uses another intermediate docker image freedesktop-sdk-binary-seed that bridges the gap between freedesktop-sdk and live-bootstrap.
So what is this live-bootstrap? If you look at parts.rst you’ll see that it is a build chain that starts with 256 byte hex assembler that can build itself from its source and also 640-byte trivial shell that can read list of commands from the file and executes them. Then it proceeds building 130 (as of the moment of writing) other components and in the process builds GCC, Python, Guile, Perl and lots of other supporting packages. Furthermore, each component is built reproducibly (and this is checked using SHA256 hash).
Some caveat: at the moment freedesktop-sdk-binary-seed still uses older binary of rustc to build rustc but in principle one could leverage mrustc to build it. Or possibly rust-gcc will become more capable in future versions and will be able to bootstrap rustc.
So unless your flatpak application uses rust, it will soon be buildable from sub 1 KiB binary seed.
After 1.5 years in development I would like to announce a new version of KDE Partition Manager and KPMcore library.
The main highlight of this release is that GUI does not need to run as root user. Instead we use KAuth framework (note that in the future we plan to use lower level Polkit API directly but this work is not started yet). GUI runing as unprivileged user also makes our Wayland port work perfectly fine. As part of porting Partition Manager to KAuth we also did the following ports:
KPMcore backend was ported away from libparted to sfdisk (part of util-linux). Many thanks to util-linux maintainer Karel Zak who promptly fixed most of the sfdisk bugs that I noticed during porting.
These two ports make KPMcore and KDE Partition Manager more portable. In the near future we might see port to FreeBSD.
Other changes include:
Better support for LUKS2. Now we can resize LUKS2 containers as long as they don’t use advanced features like dm-integrity. At the moment KDE Partition Manager still creates LUKS1 encrypted partitions (LUKS2 creation is not exposed in the GUI) but KPMcore has code to create LUKS2 encrypted partitions, so other users of KPMcore library (like Calamares installer) will be able to implement LUKS2 creation using KPMcore 4.0.
Detection support for Apple’s APFS filesystem and Microsoft’s Bitlocker.
Many of the KPMcore classes now use d-pointers, which in the future releases will help us to maintain binary compatibility (ABI) a bit better.
We now use more modern C++ features. This won’t be visible to users but it’s always good to have a more maintainable codebase.
Last November I reported that running KDE Partition Manager as non-root user is slowly shaping up. Instead of running the whole application as root, KAuth lets us run GUI parts as unprivileged user and some non-GUI helper running as root.
After another 4 months of development, KAuth support is mostly done. There are still a few things that need to be finished, some cleaning up, fixing bugs but major refactoring is complete. Unlike other KDE Applications, KDE Partition Manager uses root for almost any operation, so the way it uses KAuth is quite different from e.g. Kate where the only task the helper has to do is to copy the file to its destination. KDE Partition Manager might need to execute a lot of different commands to do some task (e.g. move/resize partition) and we don’t want user to authenticate 20 times during partitioning operation.
How it works?
When user launches KDE Partition Manager, KAuth helper is immediately started.
The Helper opens DBus system bus interface and listens for requests from KDE Partition Manager.
When KDE Partition Manager needs to run some command, or copy some data on the disks it sends a requests to KAuth helper over DBus system bus.
Only requests from KDE Partition Manager instance that started the helper are accepted. Everything else is ignored.
When KDE Partition Manager exits, it sends requests to KAuth helper to exit. Otherwise (e.g. if KDE Partition Manager crashes), the helper would exit after a short amount of time.
What was done
KDE Partition Manager had a lot of library calls that required root. In particular libparted was used to manipulate disks. I wrote a new sfdisk (part of util-linux) based backend which called sfdisk executable to manipulate partition table. util-linux is obviously GNU/Linux only but so was libparted. Nevertheless, this backend would be much easier to adapt to e.g. FreeBSD. util-linux 2.32 which was just released is required to have a fully functioning sfdisk backend.
While writing a new backend, I redesigned backend, so that its API would not be libparted specific. Since Calamares installer was using some of those calls (and it was not supposed to use them), I worked on porting Calamares away from those and as a result Calamares gained support for manipulating LVM logical volumes. Now Calamares can install into preexisting LVM volume groups.
As part of Season of KDE program Caio J. Carvalho ported away from unmaintained libatasmart to smartmontools. We will require smartmontools 6.7 which is not yet released at the time of this post. This again improves FreeBSD support, as it has smartctl. libatasmart was GUN/Linux only.
Each call to external executables (e.g. fsck, btrfs, sfdisk, etc.) was then sent to KAuth helper. So at this stage we had a new KAuth call each time we wanted to run application. This allowed KDE Partition Manager to run without root but with two severe drawbacks:
When partitions were moved, coppied, all disk data had to be transefered from KAuth helper to the main application and back via DBus. But DBus is not suitable for transfering big amount of data.
Polkit authorization is kept for 5 minutes. So after 5 minutes a new authorization dialog would be shown. It could happen in the middle of operation.
In his Season of KDE project, Huzaifa Faruqui moved the data copying code from KPMcore library to KAuth helper. Thus we no longer had to move disk data over DBus. Initially we had some performance issues but those were solved when we switched from using dd to QFile.
For the second issue, instead of running a new KAuth helper each time we need to execute command as root, I started KAuth helper as a daemon that listens to requests from the main application. Since helper does not quit and is running all the time, Polkit 5 minute timeout does not apply.
What can be improved
All command calls now go through KAuth helper. Some of the commands (such as lsblk) could still be run as unprivileged user.
We can delay starting KAuth helper until it is needed. Then in some cases we might be able to postpone authentication until applying operations.
Caio J. Carvalho and I are still working on improving helper behaviour in case main application crashes (or is killed).
Needs more testing. Feel free to try kauth branches of kpmcore and partitionmanager repositories. I’ll probably release another version before merging this work to master.
While KDE Partition Manager worked on Wayland before, it now works better. If you want to run it via XWayland you no longer need to allow other users to use XWayland server (with xhost +) which is a big security improvement. Previously KDE Partition Manager only ran as a native Wayland client (so you needed QT_QPA_PLATFORM=wayland which Gnome session doesn’t have).
If you think that my work is useful, consider supporting me on
KDE Partition Manager 3.3 is now ready. It includes some improvements for Btrfs, F2FS, NTFS file systems. I even landed the first bits of new LUKS2 on-disk format support, now KDE Partition Manager can display LUKS2 labels. More LUKS2 work will follow in KPM 3.4. There were changes in how LVM devices are detected. So now Calamares installer should be able to see LVM logical volumes. Once my pull request lands, Calamares should also support partitioning operations on LVM logical volumes (although Calamares would need more work before installation and booting from root file system on LVM works. I tested Calamares with KPMcore 3.3 and it successfully installed rootfs in LVM volume and successfully booted). KPMcore library now only depends on Tier 1 Frameworks instead of Tier 3 (although, we will later require Tier 2).
Most of the work is now done in sfdisk branch. Currently, the only functional KDE Partition Manager backend uses libparted but sfdisk backend is now fully working (I would say RC quality). I would have merged in already but it requires util-linux 2.32 which is not yet released.
Yet another branch on top of sfdisk is KAuth branch which allows KPM to run as unprivileged user and uses Polkit when necessary to gain root rights. Everything except SMART support is working. To get SMART working too we would have to port away from (unmaintained) libatasmart to calling smartctl. Feel free to help! It should be fairly easy task but somebody has to do the work. Other than that you can already perform all partitioning operations using KAuth with one caveat. Right now KPM calls KAuth helper many times while performing partitioning operations. It can happen that KAuth authorization will expire in the meantime (KAuth remembers it for about 5 minutes) and KAuth will request a user to enter root password. If the user enters correct password, operation would finish. However, if authorization is not granted we may end up with half completed operation. And of course we don’t want to leave partition half moved, the data will almost surely be lost (half-resized partition is probably okay…). I suppose we can fix this by refactoring KPM operation runner, so that it calls KAuth helper just once with a list of all commands that have to be run. Unfortunately, this KPM Operation Runner refactoring might be bigger than what I would like, as significant changes would be necessary in partition data copying code. Maybe GSoC project then… Or ar there any better ideas on how to prevent KAuth authorization dialog in the middle of partitioning operations?
Today I finally managed to get large part (maybe 90%) of KDE Partition Manager to work from GUI running as unprivileged user. This means better security, better Wayland integration, better theming support, etc. It will still take some time to polish everything and make it ready for release but nevertheless KDE Partition Manager has reached a significant milestone. Unlike most programs that use KAuth, KDE Partition Manager requires root for practically any operation, so it took some time to properly refactor the code and make it ready for KAuth/Polkit.
KDE Partition Manager has also gained another backend that you can see in the video bellow. Previously, libparted backend was used for some of the partitioning operations which for a long time was the only functional backend. Now, KDE Partition Manager can use util-linux tools (mostly sfdisk) to manipulate partitions. In the future sfdisk backend will be the default backend since it supports running without root.
At the moment the code is published in my scratch repository. Eventually this code should reach git master but probably after KPMcore 3.3 is released. Getting Calamares to run root free should now be a bit easier too…
I have just released versions 3.2.0 of KDE Partition Manager and KPMcore library. Note that if you use Calamares installer you need version 3.1.4 or later, earlier versions of Calamares will not compile against new kpmcore.
Detection support for ISO9660 file system.
KPMcore now has lighter KDE Frameworks dependencies, KIO is no longer necessary, KIconThemes dependency moved from kpmcore to Partition Manager.
Our own fstab parser and writer. Before we used to rely on glibc but it had no support for reading fstab comments, so they were lost when writing fstab.
Added support for UDF file system (this change requires util-linux 2.30). While adding support for UDF, the following general improvements were made:
– KPM now supports setting label for file systems where it can only be done during initial formating (e.g. nilfs2 or udf)
– File system label widget (QLineEdit) now validates its input for certain file systems (at the moment only FAT and UDF), not just checks maximum label length.
At the moment I’m slowly experimenting with refactoring kpmcore (not requiring libparted) but it is not part of the current release. The new backend that might appear in later releases can now delete and create partitions but cannot do other stuff (creating partition table, resizing/moving partitions, etc). Hopefully, this will make running KDE Partition Manager and possibly Calamares rootlessly easier. Maybe a GSoC project for next year to finish rootless support if anybody is interested.
There are also a few small portability fixes. In the future even this might be possible (feel free to help). Sorry for non Plasma screenshot :), I just used Live CD, so didn’t bother installing another desktop into RAM.
KF5 port is now more complete than in KTorrent 5.0:
Multimedia, search, scanfolder, ipfilter, stats, scripting, syndication (rss) plugins
are now ported to Qt5. The only missing bits are webinterface plugin and plasmoid.
Also thanks to Luigi Toscano who released took over KTorrent 5.1 RC release
after my laptop screen broke.
Note to libktorrent crashes if both qca is built with botan support and botan is built
with gmp support. Make sure at least one of these of these is not enabled. In fact botan 2
already has gmp support completely removed but most distributions come with botan 1.
KDE Partition Manager and KPMcore 3.0.0 have been released. Here are some of the new features:
Both LVM on LUKS and LUKS on LVM configurations are now supported.
Creating new LVM Volume Groups, adding or removing LVM Physical Volumes from LVM VG.
Resizing LVM Logical Volumes.
Resizing LVM Physical Volumes even if they belong to LVM Volume Group (used extents will be moved out somewhere else)
Added support for online resize. Not all filesystems support this, e.g. ext4 can only be grown online while btrfs supports both growing and shrinking.
Fixed some crashes, Qt 5.7.1 is also recommended to fix crash (in Qt) on exit.
Better support for sudo. Now KDE Partition Manager declares required environmental variables when kdesu uses sudo (e.g. in Kubuntu or Neon), so the theming is no longer broken. Environmental variables for Wayland are also fixed.
Here is a video demonstrating some of these new LVM capabilities. Note this is done directly from my main system, I’m resizing my encrypted rootfs without using any Live CD.
Basic Logical Volume Management (LVM) support was added to KDE Partition Manager this summer. So LVM consists of 3 ingredients:
LVM Physical Volumes. These are partitions on disk devices that are allocated for LVM. The union of LVM PVs is LVM Volume Group.
LVM Volume Groups. These are more or less equivalent to devices that can be partitioned into LVM Logical Volumes.
LVM Logical Volumes. These are similar to like your normal partitions but stored in LVM Volume Group instead of normal disk.
We can have a few different operations on a given LVM Volume Group, e.g. resize (add or remove LVM PVs), deactivate or remove that LVM VG:
These 3 LVM VG operations are hidden in the menu of normal disk devices.
However, there is one more operation: creating LVM VG. At the moment it is listed with the other 3 operations but it is slightly different. It does not act on selected device. So we also can’t hide it for normal devices. So we have two choices:
Keep “New Volume” in the Device menu and show it for all devices.
Move “New Volume” operation into the context menu of LVM PV partition (e.g. where operations like mount, unmount, resize partitions are located) and only make it visible for LVM PV file systems but not other types.
Option 2 will have LVM VG operations split over 2 places, but on the other hand it seems to me that option 2 might make more sense logically. Also, I’m a bit scared that creation of new LVM VG group might be a bit hidden.
I would be very grateful for any feedback (especially from Visual Design Group). E.g. which option would you prefer.
Also, do you think we need another icon for LVM devices? At the moment we are using drive-harddisk but having some other icon for virtual devices might be nice.
KDE Partition Manager and KPMcore 2.2.0 are now released with a proper LUKS support! This is a fairly big feature release but it also got tested more than usual, so a lot of bugs were fixed (including some crashes). Unfortunately there is still one more reproducible crash (bug 363294) on exit when file open/save dialogs are used (and very similar crashes actually exist in some other KDE programs, e.g. kdebugdialog or Marble). If anybody has any idea how to fix it I would be grateful.
Changes in this release:
Much improved LUKS support. We used to just detect LUKS container.
Now KDE Partition Manager can create LUKS volumes and format inner file system. Since default options are used (except for the key size which was increased) we recommend cryptsetup 1.6 or later. At the moment we restrict the choice of new inner file systems to ext234, Btrfs, swap, ReiserFS, Reiser4, XFS, JFS, ZFS and LVM physical volumes when formatting new encrypted partitions but if you create other file systems manually using command line tools they will still work in KDE Partition Manager (other than detection support for LVM PV, the support for LVM is not implemented but this might change soon as a result of GSoC project, there is already some LVM PV resize support in git master). If you think it makes sense to whitelist other file systems and there is a valid use case please leave a comment.
LUKS volumes can be opened/closed.
Resize support for filesystems encrypted with LUKS (obviously you can’t do this while LUKS volume is closed, you have to decrypt it first). To the best of my knowledge, no other partition manager can do this.
To prevent data loss, you can only move LUKS partitions that are closed. A few bugs were fixed in KDE Partition Manager to properly support unmovable but resizeable partitions (i.e. LUKS when it is closed).
Filesystems inside LUKS can be checked for errors, mounted, labels can be set, etc. All other stuff like free space reporting also works (and space taken up by LUKS metadata is taken into the account).
Opened LUKS partitions now cannot be removed, you have to close them first.
Copying LUKS partition works but only when they are closed.
More widespread use of C++11 features.
Fixed a couple of bugs present from KF5 porting. Also new Qt5 signal/slot syntax is used, so moc is used much less often
Clobbering (deleting file system signature) deleted partitions was fixed.
Some other bugs were fixed, e.g. NILFS2 resizing support was fixed.
ntfslabel from ntfs-3g is now used for setting NTFS labels.
A crash when partitions were deleted (mainly extended but not only) was fixed.
Compilation with Clang was fixed.
There is also a slightly older (e.g. now we use KPasswordDialog to unlock LUKS partitions) video demonstrating LUKS support.
Note for packagers: Calamares 2.2.2 will most likely work with KPMcore 2.2.0 after recompilation but Calamares 2.3 will be recommended as soon as it is released. Older versions of KDE Partition Manager are not compatible with KPMcore 2.2.0, so you need to update KPMcore and KDE Partition Manager at the same time. Qt 5.6.1 also fixes one minor NTFS bug in KPMcore but unfortunately it is not released yet.