Last November I reported that running KDE Partition Manager as non-root user is slowly shaping up. Instead of running the whole application as root, KAuth lets us run GUI parts as unprivileged user and some non-GUI helper running as root.

After another 4 months of development, KAuth support is mostly done. There are still a few things that need to be finished, some cleaning up, fixing bugs but major refactoring is complete. Unlike other KDE Applications, KDE Partition Manager uses root for almost any operation, so the way it uses KAuth is quite different from e.g. Kate where the only task the helper has to do is to copy the file to its destination. KDE Partition Manager might need to execute a lot of different commands to do some task (e.g. move/resize partition) and we don’t want user to authenticate 20 times during partitioning operation.

How it works?

  • When user launches KDE Partition Manager, KAuth helper is immediately started.
  • The Helper opens DBus system bus interface and listens for requests from KDE Partition Manager.
  • When KDE Partition Manager needs to run some command, or copy some data on the disks it sends a requests to KAuth helper over DBus system bus.
  • Only requests from KDE Partition Manager instance that started the helper are accepted. Everything else is ignored.
  • When KDE Partition Manager exits, it sends requests to KAuth helper to exit. Otherwise (e.g. if KDE Partition Manager crashes), the helper would exit after a short amount of time.

What was done

  • KDE Partition Manager had a lot of library calls that required root. In particular libparted was used to manipulate disks. I wrote a new sfdisk (part of util-linux) based backend which called sfdisk executable to manipulate partition table. util-linux is obviously GNU/Linux only but so was libparted. Nevertheless, this backend would be much easier to adapt to e.g. FreeBSD. util-linux 2.32 which was just released is required to have a fully functioning sfdisk backend.
  • While writing a new backend, I redesigned backend, so that its API would not be libparted specific. Since Calamares installer was using some of those calls (and it was not supposed to use them), I worked on porting Calamares away from those and as a result Calamares gained support for manipulating LVM logical volumes. Now Calamares can install into preexisting LVM volume groups.
  • As part of Season of KDE  program Caio J. Carvalho ported away from unmaintained libatasmart to smartmontools. We will require smartmontools 6.7 which is not yet released at the time of this post. This again improves FreeBSD support, as it has smartctl. libatasmart was GUN/Linux only.
  • Each call to external executables (e.g. fsck, btrfs, sfdisk, etc.) was then sent to KAuth helper. So at this stage we had a new KAuth call each time we wanted to run application. This allowed KDE Partition Manager to run without root but with two severe drawbacks:
    • When partitions were moved, coppied, all disk data had to be transefered from KAuth helper to the main application and back via DBus. But DBus is not suitable for transfering big amount of data.
    • Polkit authorization is kept for 5 minutes. So after 5 minutes a new authorization dialog would be shown. It could happen in the middle of operation.
  • In his Season of KDE project, Huzaifa Faruqui moved the data copying code from KPMcore library to KAuth helper. Thus we no longer had to move disk data over DBus. Initially we had some performance issues but those were solved when we switched from using dd to QFile.
  • For the second issue, instead of running a new KAuth helper each time we need to execute command as root, I started KAuth helper as a daemon that listens to requests from the main application. Since helper does not quit and is running all the time, Polkit 5 minute timeout does not apply.

What can be improved

  • All command calls now go through KAuth helper. Some of the commands (such as lsblk) could still be run as unprivileged user.
  • We can delay starting KAuth helper until it is needed. Then in some cases we might be able to postpone authentication until applying operations.
  • Caio J. Carvalho and I are still working on improving helper behaviour in case main application crashes (or is killed).
  • Needs more testing. Feel free to try kauth branches of kpmcore and partitionmanager repositories. I’ll probably release another version before merging this work to master.

While KDE Partition Manager worked on Wayland before, it now works better. If you want to run it via XWayland you no longer need to allow other users to use XWayland server (with xhost +) which is a big security improvement. Previously KDE Partition Manager only ran as a native Wayland client (so you needed QT_QPA_PLATFORM=wayland which Gnome session doesn’t have).

If you think that my work is useful, consider supporting me on

Become a patron Donate using Liberapay Bitcoin: bc1qe2dfqjwgse5v6cl6rhtk352ru90t0hnve45f2c

KDE Partition Manager 3.3 is now ready. It includes some improvements for Btrfs, F2FS, NTFS file systems. I even landed the first bits of new LUKS2 on-disk format support, now KDE Partition Manager can display LUKS2 labels. More LUKS2 work will follow in KPM 3.4. There were changes in how LVM devices are detected. So now Calamares installer should be able to see LVM logical volumes. Once my pull request lands, Calamares should also support partitioning operations on LVM logical volumes (although Calamares would need more work before installation and booting from root file system on LVM works. I tested Calamares with KPMcore 3.3 and it successfully installed rootfs in LVM volume and successfully booted). KPMcore library now only depends on Tier 1 Frameworks instead of Tier 3 (although, we will later require Tier 2).

Most of the work is now done in sfdisk branch.  Currently, the only functional KDE Partition Manager backend uses libparted but sfdisk backend is now fully working (I would say RC quality). I would have merged in already but it requires util-linux 2.32 which is not yet released.

Yet another branch on top of sfdisk is KAuth branch which allows KPM to run as unprivileged user and uses Polkit when necessary to gain root rights. Everything except SMART support is working. To get SMART working too we would have to port away from (unmaintained) libatasmart to calling smartctl. Feel free to help! It should be fairly easy task but somebody has to do the work. Other than that you can already perform all partitioning operations using KAuth with one caveat. Right now KPM calls KAuth helper many times while performing partitioning operations. It can happen that KAuth authorization will expire in the meantime (KAuth remembers it for about 5 minutes) and KAuth will request a user to enter root password. If the user enters correct password, operation would finish. However, if authorization is not granted we may end up with half completed operation. And of course we don’t want to leave partition half moved, the data will almost surely be lost (half-resized partition is probably okay…). I suppose we can fix this by refactoring KPM operation runner, so that it calls KAuth helper just once with a list of all commands that have to be run. Unfortunately, this KPM Operation Runner refactoring might be bigger than what I would like, as significant changes would be necessary in partition data copying code. Maybe GSoC project then… Or ar there any better ideas on how to prevent KAuth authorization dialog in the middle of partitioning operations?

You can grab tarballs from standard locations on download.kde.org server.

Today I finally managed to get large part (maybe 90%) of KDE Partition Manager to work from GUI running as unprivileged user. This means better security, better Wayland integration, better theming support, etc. It will still take some time to polish everything and make it ready for release but nevertheless KDE Partition Manager has reached a significant milestone. Unlike most programs that use KAuth, KDE Partition Manager requires root for practically any operation, so it took some time to properly refactor the code and make it ready for KAuth/Polkit.

KDE Partition Manager has also gained another backend that you can see in the video bellow. Previously, libparted backend was used for some of the partitioning operations which for a long time was the only functional backend. Now, KDE Partition Manager can use util-linux tools (mostly sfdisk) to manipulate partitions. In the future sfdisk backend will be the default backend since it supports running without root.

At the moment the code is published in my scratch repository. Eventually this code should reach git master but probably after KPMcore 3.3 is released. Getting Calamares to run root free should now be a bit easier too…

 

I have just released versions 3.2.0 of KDE Partition Manager and KPMcore library. Note that if you use Calamares installer you need version 3.1.4 or later, earlier versions of Calamares will not compile against new kpmcore.

Changes:

  • Detection support for ISO9660 file system.
  • KPMcore now has lighter KDE Frameworks dependencies, KIO is no longer necessary, KIconThemes dependency moved from kpmcore to Partition Manager.
  • Our own fstab parser and writer. Before we used to rely on glibc but it had no support for reading fstab comments, so they were lost when writing fstab.
  • Added support for UDF file system (this change requires util-linux 2.30). While adding support for UDF, the following general improvements were made:
    – KPM now supports setting label for file systems where it can only be done during initial formating (e.g. nilfs2 or udf)
    – File system label widget (QLineEdit) now validates its input for certain file systems (at the moment only FAT and UDF), not just checks maximum label length.

Future directions:

At the moment I’m slowly experimenting with refactoring kpmcore (not requiring libparted) but it is not part of the current release. The new backend that might appear in later releases can now delete and create partitions but cannot do other stuff (creating partition table, resizing/moving partitions, etc). Hopefully, this will make running KDE Partition Manager and possibly Calamares rootlessly easier. Maybe a GSoC project for next year to finish rootless support if anybody is interested.

There are also a few small portability fixes. In the future even this might be possible (feel free to help). Sorry for non Plasma screenshot :), I just used Live CD, so didn’t bother installing another desktop into RAM.

Download links:

https://download.kde.org/stable/kpmcore/3.2.0/src/kpmcore-3.2.0.tar.xz.mirrorlist
https://download.kde.org/stable/kpmcore/3.2.0/src/kpmcore-3.2.0.tar.xz.sig.mirrorlist
https://download.kde.org/stable/partitionmanager/3.2.0/src/partitionmanager-3.2.0.tar.xz.mirrorlist
https://download.kde.org/stable/partitionmanager/3.2.0/src/partitionmanager-3.2.0.tar.xz.sig.mirrorlist

As usual signed with my key: 1EE5 A320 5904 BAA2 B88C 0A9D 24FD 3194 0095 C0E1

Posted in KDE.

As an acting release manager I would like to announce KTorrent 5.1.

https://download.kde.org/stable/ktorrent/5.1/ktorrent-5.1.0.tar.xz.mirrorlist
https://download.kde.org/stable/ktorrent/5.1/ktorrent-5.1.0.tar.xz.sig.mirrorlist
https://download.kde.org/stable/ktorrent/5.1/libktorrent-2.1.tar.xz.mirrorlist
https://download.kde.org/stable/ktorrent/5.1/libktorrent-2.1.tar.xz.sig.mirrorlist

KF5 port is now more complete than in KTorrent 5.0:
Multimedia, search, scanfolder, ipfilter, stats, scripting, syndication (rss) plugins
are now ported to Qt5. The only missing bits are webinterface plugin and plasmoid.

Also thanks to Luigi Toscano who released took over KTorrent 5.1 RC release
after my laptop screen broke.

Note to libktorrent crashes if both qca is built with botan support and botan is built
with gmp support. Make sure at least one of these of these is not enabled. In fact botan 2
already has gmp support completely removed but most distributions come with botan 1.

Also, libktorrent apparently requires Qt 5.7 even though CMakeLists.txt only requires 5.2.
There is a patch to lower Qt requirement in 2.1 branch
https://phabricator.kde.org/R472:bcb17b62ff492a7bc7d65c59a5b0a3513199c65d if you need it
although, right now KTorrent requires Qt 5.7 anyway.

KDE Partition Manager and KPMcore 3.0.0 have been released. Here are some of the new features:

  • Both LVM on LUKS and LUKS on LVM configurations are now supported.
  • Creating new LVM Volume Groups, adding or removing LVM Physical Volumes from LVM VG.
  • Resizing LVM Logical Volumes.
  • Resizing LVM Physical Volumes even if they belong to LVM Volume Group (used extents will be moved out somewhere else)
  • Added support for online resize. Not all filesystems support this, e.g. ext4 can only be grown online while btrfs supports both growing and shrinking.
  • Fixed some crashes, Qt 5.7.1 is also recommended to fix crash (in Qt) on exit.
  • Better support for sudo. Now KDE Partition Manager declares required environmental variables when kdesu uses sudo (e.g. in Kubuntu or Neon), so the theming is no longer broken. Environmental variables for Wayland are also fixed.

Here is a video demonstrating some of these new LVM capabilities. Note this is done directly from my main system, I’m resizing my encrypted rootfs without using any Live CD.

 

 

KDE Partition Manager was able to detect encrypted LUKS partitions for some time. I’m glad to report that now support for LUKS volumes is much more complete. Unless I’m mistaken, KDE Partition Manager is the first GUI tool that supports creating and resizing LUKS partitions (provided that file system inside LUKS container supports resizing). This is still work in progress and we still need to implement some checks (e.g. it should not be possible to set labels when LUKS volume is closed). Here is a short video demonstrating current state:

 

I have bought Orvibo S20 (picture) smart socket. You can find them on Amazon, eBay and probably many other online shops. The socket comes with an app for Android and iOS that is used to control the socket. Unfortunately, the app is proprietary and also not available for normal computers. I wanted to have some free software solution. Also, the original app didn’t work too reliably, and of course you can’t fix it without having a code. Again, this shows why it is important to have free software…

After searching a bit I found some code on GitHub written for Ninja Blocks. It also came with some reverse engineering data (I will refer to this as the original reverse engineering later and will assume that you briefly looked at it). That file might look slightly scary initially if you are not used to that kind of stuff but actually everything is quite simple. It seems that the socket can be controlled by simply sending UDP packets over the network and listening for replies. You can even try to play a bit with netcat but of course it is not too convenient. So I decided to write my own program using Qt 5. It provides a lot of convenient functions in its QtNetwork module and also it would make it easy to write GUI if I ever decide to do so in the future. Also, I’m familiar with Qt because it is used by KDE Applications. Another reason for choosing Qt was it’s support for sockets and slots and I expected them to be useful.

As you can see in the reverse engineering file, there are commands to discover the socket, subscribe to it (which is required before doing anything else), power it on/off and read some tables (Socket Data contains information about the socket and Timing Data stores when to turn the socket on or off). They all follow similar pattern. You have to send

Magic Key+ message length + command id + rest of the message

where Magic Key is 68 64 (hexadecimals) and is used to distinguish these UDP packets from any other packets that are send over UDP port 10000. Every time you send a message the socket replies with another message confirming the action of the first message. Or the socket doesn’t reply. We are using UDP protocol for networking. So there is no guarantee that message is received and later I had to write some code to make sure packets are received. Hence, I implemented message queue and resend every command until I get a proper reply before sending another command. This finally made my program more reliable.

Writing Socket Data

I quickly managed to get some basic stuff working (for example powering it on and off) and soon I implemented most of the commands from the file with reversed engineered commands (reading Timing Table is still not completed but shouldn’t be too hard). Since I wanted to do more than that, I started Wireshark and analyzed a few more packets. I quickly learned how to write Socket Data Table too. Apparently, you send command very similar to what you receive when you request Socket Data but with different Command ID (74 6d instead of 72 74 in hexadecimals).

So to write Socket Data I send the following packet

Magic Key + message length + 74 6d + mac + mac padding + 00 00 00 00  + AA 00 BB + recordLength + record;

where

record = 01 00 /* record number = 1*/ + versionID + mac + mac padding + reversed mac + mac padding + remote password + socket name + icon + hardwareVersion + firmwareVersion + wifiFirmwareVersion + port + staticServerIP + port + domainServerName + localIP + localGatewayIP + localNetMask + dhcpNode + discoverable + timeZoneSet + timezone + countdownStatus + countdown + 00 (repeated twelve times) + 30 (repeated 30 times, note that hex 30 corresponds to 0 in ASCII);

countdownStatus is 00 ff when countdown is disabled and 01 00 when countdown is enabled.

AA 00 BB is actually table number and version flag. E.g. 04 00 01. 4 stands for table number (Socket Data is Table number 4) I don’t completely understand what is version flag, so if you know please tell me in the comments.

Then socket replies with:

Magic Key + message length + 74 6d + mac + mac padding + 01 00 00 00 00;

Now just send an already documented Socket Table packet (see: http://pastebin.com/LfUhsbcS) to update your variables.

Writing Timing Data

Writing Timing Data is exactly the same (even Command ID is still 74 6d) as writing Socket Data but you must specify 03 a a table number

Magic Key + message length + 74 6d + mac + mac padding + 00 00 00 00  + AA 00 BB + record1Length + record 1+ record2Length + record2 + record3Length + record3 + …;

record again contains the same data as what socket sends when you request timing data.

Initial pairing of the socket

The authors of the original Ninja Blocks orvibo-allone driver assume that socket was already paired using the proprietary Android/iOS application. Their original reverse engineering also contains no information how to do that. I expected that this might be a bit tricky to do because unpaired socket is not connected to the router and you have to somehow transmit your wifi configuration into the socket. I think there are at least two ways to pair that proprietary Android/iOS app implements. If you press the socket button for a few second it switches to a rapidly blinking red led mode. Then long press it again and it switches to rapidly blinking blue led and the socket creates an unencrypted wifi network (it was called WiWo-S20).

Then I created the wifi network with the same name on my laptop and tricked the proprietary app into believing that it is the socket’s wifi network. I was able to intercept the following message on UDP port 48899 (everything is in ASCII in the section, not in hex):

HF-A11ASSISTHREAD

So apparently, Orvibo S20 has HF-LPB100 Wifi chip inside. This chip can be controlled by the AT+ commands (you can find them online but I will write a brief summary here) and I was able to do initial socket configuration!

  • Switch S20 to rapidly blinking blue led more. Connect your computer to WiWo-S20 network.
  •  Send “HF-A11ASSISTHREAD” on UDP port 48899 to the broadcast address (don’t include ” in the message).
  •  S20 will reply with “IP address,MAC Address,Hostname”. The socket always replies to the same port as the source port of your message.
  • Acknowledge that you got the previous message by sending “+ok”.
  • Send “AT+WSSSID=ssid\r” where you replace ssid with your WiFi network name. \r is the carriage return (CR) symbol.
  • The socket will reply with “+ok\n\n” (\n in this case is carriage return + line feed) if everything is correct or “+ERR\n\n” if something is wrong.
  • Send your Wifi security settings: “AT+WSKEY=WPA2PSK,AES,PASSWORD\r”.  The socket will again reply with “+ok\n\n”.
  • Switch HF-A11 chip to station mode by sending “AT+WMODE=STA\r”. Again, wait for “+ok\n\n”
  • Reboot your socket with “AT+Z\r”.
  • Connect your computer back to your router. Wait until the socket boots. Now you can find it using normal discovery packet on port 10000, then change socket name, timezone, etc. with Write Socket Data packet…

I did some investigation and it seems that we need to send a slightly modified Table Data and Socket Data packets immediately after pairing to set them to default values. More information will be published later.

Another way to pair the socket from the rapidly blinking red mode. It is slighly less reliable than this method but on the other hand does not require you to disconnect from your wireless. Actually, it doesn’t require your computer to have any wireless at all.

Unfortunately, it seems that either way WPA encryption key is transferred in an insecure way, i.e. the socket is not able to use any public key cryptography. Slightly safer way to do it manually is to first change your WPA key to something temporary and pair the socket. Then use AT+ commands to change WPA password to the real one and change your router’s WPA password back.

Also, note that this socket doesn’t support WPA Enterprise. So if you would like to use it at home then create two Wifi networks: WPA-PSK for the socket and WPA-EAP for everything else (OpenWrt can do this easily).

Code

I released all code under the GNU General Public License version 3, so the code is freely available to everybody. Git repository is available at:

https://git.stikonas.eu/andrius/s20 (feel free to create an account here and fork the code)

Windows binaries

I compiled windows binaries. There is also a very limited GUI that does not yet have all the features of console app but if you prefer GUI then it might be useful.

Console 64-bit 32-bit
GUI 64-bit 32-bit

Donations

If you find this work useful then tips are very welcome. You can send Bitcoin tips to bc1qe2dfqjwgse5v6cl6rhtk352ru90t0hnve45f2c.

I was able to buy the second socket which already resulted in improved multiple socket support. Thanks for the donations!
Become a patron Donate using Liberapay Bitcoin: bc1qe2dfqjwgse5v6cl6rhtk352ru90t0hnve45f2c

Bugs

Bugs are tracked in the Gitlab issue tracker.

See also

Go code by Grayda (basic support for S20 but also supports Orvibo AllOne devices)

PHP code by Fernano Silva (supports most features of S20). See also his technical data file.

Perl code by Branislav Vartik