kpmcore

KDE Partition Manager 4.0

Posted on by Andrius Štikonas

After 1.5 years in development I would like to announce a new version of KDE Partition Manager and KPMcore library.

The main highlight of this release is that GUI does not need to run as root user. Instead we use KAuth framework (note that in the future we plan to use lower level Polkit API directly but this work is not started yet). GUI runing as unprivileged user also makes our Wayland port work perfectly fine. As part of porting Partition Manager to KAuth we also did the following ports:

  • KPMcore backend was ported away from libparted to sfdisk (part of util-linux). Many thanks to util-linux maintainer Karel Zak who promptly fixed most of the sfdisk bugs that I noticed during porting.
  • Caio Jordão Carvalho ported S.M.A.R.T. code away form unmaintained libatasmart to smartmontools.

These two ports make KPMcore and KDE Partition Manager more portable. In the near future we might see port to FreeBSD.

Other changes include:

  • Better support for LUKS2. Now we can resize LUKS2 containers as long as they don’t use advanced features like dm-integrity. At the moment KDE Partition Manager still creates LUKS1 encrypted partitions (LUKS2 creation is not exposed in the GUI) but KPMcore has code to create LUKS2 encrypted partitions, so other users of KPMcore library (like Calamares installer) will be able to implement LUKS2 creation using KPMcore 4.0.
  • Detection support for Apple’s APFS filesystem and Microsoft’s Bitlocker.
  • Many of the KPMcore classes now use d-pointers, which in the future releases will help us to maintain binary compatibility (ABI) a bit better.
  • We now use more modern C++ features. This won’t be visible to users but it’s always good to have a more maintainable codebase.
  • Some bugfixes, in particular related to LVM.

List of all changes can be found in commit log at

https://invent.kde.org/kde/kpmcore

https://invent.kde.org/kde/partitionmanager

Downloads:

https://download.kde.org/stable/kpmcore/4.0.0/src/

https://download.kde.org/stable/partitionmanager/4.0.0/src/

Become a patron Donate using Liberapay Bitcoin: bc1qe2dfqjwgse5v6cl6rhtk352ru90t0hnve45f2c

KAuth support in KDE Partition Manager

Posted on by Andrius Štikonas

Last November I reported that running KDE Partition Manager as non-root user is slowly shaping up. Instead of running the whole application as root, KAuth lets us run GUI parts as unprivileged user and some non-GUI helper running as root.

After another 4 months of development, KAuth support is mostly done. There are still a few things that need to be finished, some cleaning up, fixing bugs but major refactoring is complete. Unlike other KDE Applications, KDE Partition Manager uses root for almost any operation, so the way it uses KAuth is quite different from e.g. Kate where the only task the helper has to do is to copy the file to its destination. KDE Partition Manager might need to execute a lot of different commands to do some task (e.g. move/resize partition) and we don’t want user to authenticate 20 times during partitioning operation.

How it works?

  • When user launches KDE Partition Manager, KAuth helper is immediately started.
  • The Helper opens DBus system bus interface and listens for requests from KDE Partition Manager.
  • When KDE Partition Manager needs to run some command, or copy some data on the disks it sends a requests to KAuth helper over DBus system bus.
  • Only requests from KDE Partition Manager instance that started the helper are accepted. Everything else is ignored.
  • When KDE Partition Manager exits, it sends requests to KAuth helper to exit. Otherwise (e.g. if KDE Partition Manager crashes), the helper would exit after a short amount of time.

What was done

  • KDE Partition Manager had a lot of library calls that required root. In particular libparted was used to manipulate disks. I wrote a new sfdisk (part of util-linux) based backend which called sfdisk executable to manipulate partition table. util-linux is obviously GNU/Linux only but so was libparted. Nevertheless, this backend would be much easier to adapt to e.g. FreeBSD. util-linux 2.32 which was just released is required to have a fully functioning sfdisk backend.
  • While writing a new backend, I redesigned backend, so that its API would not be libparted specific. Since Calamares installer was using some of those calls (and it was not supposed to use them), I worked on porting Calamares away from those and as a result Calamares gained support for manipulating LVM logical volumes. Now Calamares can install into preexisting LVM volume groups.
  • As part of Season of KDE  program Caio J. Carvalho ported away from unmaintained libatasmart to smartmontools. We will require smartmontools 6.7 which is not yet released at the time of this post. This again improves FreeBSD support, as it has smartctl. libatasmart was GUN/Linux only.
  • Each call to external executables (e.g. fsck, btrfs, sfdisk, etc.) was then sent to KAuth helper. So at this stage we had a new KAuth call each time we wanted to run application. This allowed KDE Partition Manager to run without root but with two severe drawbacks:
    • When partitions were moved, coppied, all disk data had to be transefered from KAuth helper to the main application and back via DBus. But DBus is not suitable for transfering big amount of data.
    • Polkit authorization is kept for 5 minutes. So after 5 minutes a new authorization dialog would be shown. It could happen in the middle of operation.
  • In his Season of KDE project, Huzaifa Faruqui moved the data copying code from KPMcore library to KAuth helper. Thus we no longer had to move disk data over DBus. Initially we had some performance issues but those were solved when we switched from using dd to QFile.
  • For the second issue, instead of running a new KAuth helper each time we need to execute command as root, I started KAuth helper as a daemon that listens to requests from the main application. Since helper does not quit and is running all the time, Polkit 5 minute timeout does not apply.

What can be improved

  • All command calls now go through KAuth helper. Some of the commands (such as lsblk) could still be run as unprivileged user.
  • We can delay starting KAuth helper until it is needed. Then in some cases we might be able to postpone authentication until applying operations.
  • Caio J. Carvalho and I are still working on improving helper behaviour in case main application crashes (or is killed).
  • Needs more testing. Feel free to try kauth branches of kpmcore and partitionmanager repositories. I’ll probably release another version before merging this work to master.

While KDE Partition Manager worked on Wayland before, it now works better. If you want to run it via XWayland you no longer need to allow other users to use XWayland server (with xhost +) which is a big security improvement. Previously KDE Partition Manager only ran as a native Wayland client (so you needed QT_QPA_PLATFORM=wayland which Gnome session doesn’t have).

If you think that my work is useful, consider supporting me on

Become a patron Donate using Liberapay Bitcoin: bc1qe2dfqjwgse5v6cl6rhtk352ru90t0hnve45f2c

KDE Partition Manager 3.3 and future work

Posted on by Andrius Štikonas

KDE Partition Manager 3.3 is now ready. It includes some improvements for Btrfs, F2FS, NTFS file systems. I even landed the first bits of new LUKS2 on-disk format support, now KDE Partition Manager can display LUKS2 labels. More LUKS2 work will follow in KPM 3.4. There were changes in how LVM devices are detected. So now Calamares installer should be able to see LVM logical volumes. Once my pull request lands, Calamares should also support partitioning operations on LVM logical volumes (although Calamares would need more work before installation and booting from root file system on LVM works. I tested Calamares with KPMcore 3.3 and it successfully installed rootfs in LVM volume and successfully booted). KPMcore library now only depends on Tier 1 Frameworks instead of Tier 3 (although, we will later require Tier 2).

Most of the work is now done in sfdisk branch.  Currently, the only functional KDE Partition Manager backend uses libparted but sfdisk backend is now fully working (I would say RC quality). I would have merged in already but it requires util-linux 2.32 which is not yet released.

Yet another branch on top of sfdisk is KAuth branch which allows KPM to run as unprivileged user and uses Polkit when necessary to gain root rights. Everything except SMART support is working. To get SMART working too we would have to port away from (unmaintained) libatasmart to calling smartctl. Feel free to help! It should be fairly easy task but somebody has to do the work. Other than that you can already perform all partitioning operations using KAuth with one caveat. Right now KPM calls KAuth helper many times while performing partitioning operations. It can happen that KAuth authorization will expire in the meantime (KAuth remembers it for about 5 minutes) and KAuth will request a user to enter root password. If the user enters correct password, operation would finish. However, if authorization is not granted we may end up with half completed operation. And of course we don’t want to leave partition half moved, the data will almost surely be lost (half-resized partition is probably okay…). I suppose we can fix this by refactoring KPM operation runner, so that it calls KAuth helper just once with a list of all commands that have to be run. Unfortunately, this KPM Operation Runner refactoring might be bigger than what I would like, as significant changes would be necessary in partition data copying code. Maybe GSoC project then… Or ar there any better ideas on how to prevent KAuth authorization dialog in the middle of partitioning operations?

You can grab tarballs from standard locations on download.kde.org server.

KDE Partition Manager 3.0

Posted on by Andrius Štikonas

KDE Partition Manager and KPMcore 3.0.0 have been released. Here are some of the new features:

  • Both LVM on LUKS and LUKS on LVM configurations are now supported.
  • Creating new LVM Volume Groups, adding or removing LVM Physical Volumes from LVM VG.
  • Resizing LVM Logical Volumes.
  • Resizing LVM Physical Volumes even if they belong to LVM Volume Group (used extents will be moved out somewhere else)
  • Added support for online resize. Not all filesystems support this, e.g. ext4 can only be grown online while btrfs supports both growing and shrinking.
  • Fixed some crashes, Qt 5.7.1 is also recommended to fix crash (in Qt) on exit.
  • Better support for sudo. Now KDE Partition Manager declares required environmental variables when kdesu uses sudo (e.g. in Kubuntu or Neon), so the theming is no longer broken. Environmental variables for Wayland are also fixed.

Here is a video demonstrating some of these new LVM capabilities. Note this is done directly from my main system, I’m resizing my encrypted rootfs without using any Live CD.

 

 

LVM support in KDE Partition Manager (feedback needed)

Posted on by Andrius Štikonas

Basic Logical Volume Management (LVM) support was added to KDE Partition Manager this summer. So LVM consists of 3 ingredients:

  1. LVM Physical Volumes. These are partitions on disk devices that are allocated for LVM. The union of LVM PVs is LVM Volume Group.
  2. LVM Volume Groups. These are more or less equivalent to devices that can be partitioned into LVM Logical Volumes.
  3. LVM Logical Volumes. These are similar to like your normal partitions but stored in LVM Volume Group instead of normal disk.

We can have a few different operations on a given LVM Volume Group, e.g. resize (add or remove LVM PVs), deactivate or remove that LVM VG:

LVM Device Menu

These 3 LVM VG operations are hidden in the menu of normal disk devices.

However, there is one more operation: creating LVM VG. At the moment it is listed with the other 3 operations but it is slightly different. It does not act on selected device. So we also can’t hide it for normal devices. So we have two choices:

  1. Keep “New Volume” in the Device menu and show it for all devices.
  2. Move “New Volume” operation into the context menu of LVM PV partition (e.g. where operations like mount, unmount, resize partitions are located) and only make it visible for LVM PV file systems but not other types.

Option 2 will have LVM VG operations split over 2 places, but on the other hand it seems to me that option 2 might make more sense logically. Also, I’m a bit scared that creation of new LVM VG group might be a bit hidden.

I would be very grateful for any feedback (especially from Visual Design Group). E.g. which option would you prefer.

Also, do you think we need another icon for LVM devices? At the moment we are using drive-harddisk but having some other icon for virtual devices might be nice.

KDE Partition Manager 2.2.0

Posted on by Andrius Štikonas

KDE Partition Manager and KPMcore 2.2.0 are now released with a proper LUKS support! This is a fairly big feature release but it also got tested more than usual, so a lot of bugs were fixed (including some crashes). Unfortunately there is still one more reproducible crash (bug 363294) on exit when file open/save dialogs are used (and very similar crashes actually exist in some other KDE programs, e.g. kdebugdialog or Marble). If anybody has any idea how to fix it I would be grateful.

Changes in this release:

  • Much improved LUKS support. We used to just detect LUKS container.
    • Now KDE Partition Manager can create LUKS volumes and format inner file system. Since default options are used (except for the key size which was increased) we recommend cryptsetup 1.6 or later. At the moment we restrict the choice of new inner file systems to ext234, Btrfs, swap, ReiserFS, Reiser4, XFS, JFS, ZFS and LVM physical volumes  when formatting new encrypted partitions but if you create other file systems manually using command line tools they will still work in KDE Partition Manager (other than detection support for LVM PV, the support for LVM is not implemented but this might change soon as a result of GSoC project, there is already some LVM PV resize support in git master). If you think it makes sense to whitelist other file systems and there is a valid use case please leave a comment.
    • LUKS volumes can be opened/closed.
    • Resize support for filesystems encrypted with LUKS (obviously you can’t do this while LUKS volume is closed, you have to decrypt it first). To the best of my knowledge, no other partition manager can do this.
    • To prevent data loss, you can only move LUKS partitions that are closed. A few bugs were fixed in KDE Partition Manager to properly support unmovable but resizeable partitions (i.e. LUKS when it is closed).
    • Filesystems inside LUKS can be checked for errors, mounted, labels can be set, etc. All other stuff like free space reporting also works (and space taken up by LUKS metadata is taken into the account).
    • Opened LUKS partitions now cannot be removed, you have to close them first.
    • Copying LUKS partition works but only when they are closed.
  • More widespread use of C++11 features.
  • Fixed a couple of bugs present from KF5 porting. Also new Qt5 signal/slot syntax is used, so moc is used much less often
  • Clobbering (deleting file system signature) deleted partitions was fixed.
  • Some other bugs were fixed, e.g. NILFS2 resizing support was fixed.
  • ntfslabel from ntfs-3g is now used for setting NTFS labels.
  • A crash when partitions were deleted (mainly extended but not only) was fixed.
  • Compilation with Clang was fixed.

There is also a slightly older (e.g. now we use KPasswordDialog to unlock LUKS partitions) video demonstrating LUKS support.

Note for packagers: Calamares 2.2.2 will most likely work with KPMcore 2.2.0 after recompilation but Calamares 2.3 will be recommended as soon as it is released. Older versions of KDE Partition Manager are not compatible with KPMcore 2.2.0, so you need to update KPMcore and KDE Partition Manager at the same time. Qt 5.6.1 also fixes one minor NTFS bug in KPMcore but unfortunately it is not released yet.

Download links:

KPMcore 2.2.0

KDE Partition Manager 2.2.0

There are already packages for Arch and Gentoo, hopefully other distros will package it too.

LUKS support in KDE Partition Manager

Posted on by Andrius Štikonas

KDE Partition Manager was able to detect encrypted LUKS partitions for some time. I’m glad to report that now support for LUKS volumes is much more complete. Unless I’m mistaken, KDE Partition Manager is the first GUI tool that supports creating and resizing LUKS partitions (provided that file system inside LUKS container supports resizing). This is still work in progress and we still need to implement some checks (e.g. it should not be possible to set labels when LUKS volume is closed). Here is a short video demonstrating current state:

 

KDE Partition Manager 2.0.3

Posted on by Andrius Štikonas

I’m happy to announce new bugfix versions of KDE Partition Manager 2.0.1 and KPMcore 2.0.1.

  • Btrfs used space detection should work without crashing (it was actually cause by crash in btrfs-debug-tree program btrfs filesystem show is used).
  • Improved support for FAT12 partitions. They were not recognized before. For now they are reported as FAT16 (gparted behaves in the same way).
  • Installation path for libparted plugins is not force to be in system prefix anymore. This is consistent with how other KDE Applications work, but cmake might require KDE_INSTALL_USE_QT_SYS_PATHS to be set if you are installing kpmcore to /usr.
  • We know try to find KF5 version of kdesu in libexec even when kdesu is not in $PATH.
  • Fixed visible HTML in one dialog box (#354925).

There is still an open issue that Partition Manager reports itself as 2.0.0 instead of 2.0.1. I tried to bump the version but there seem to be some kind of bug that prevents KDE Partition Manager and Calamares to compile or work. We will continue to investigate this issue but 2.0.1 should work well despite incorrectly reporting it’s own version

Edit: KDE Partition Manager 2.0.3 is now released to fix a crash in settings dialog (it can be used with KPMcore 2.0.x but please use the latest KPMcore 2.0.1).

 

KDE Partition Manager 2.0.0

Posted on by Andrius Štikonas

I’m happy to announce KDE Partition Manager 2.0.0 and the first stable release of KPMcore. This release mostly focused on splitting user interface and partitioning library which will be used in the next release of Calamares. It also brings some bugfixes but unfortunately I wasn’t able to go through all reported bugs in bugzilla yet (but they don’t seem to be regressions).