LWN Headlines

Open sourcing oomd, a new approach to handling OOMs

Over on the Facebook code site, Daniel Xu announces the release of oomd under the GPLv2. Oomd is a user-space "out of memory" killer that was mentioned in our recent article on the block I/O latency controller and it uses the pressure stall information covered in an even more recent article. "Oomd constantly monitors PSI [Pressure Stall Information] metrics to assess whether a system is under unrecoverable load. PSI alone is insufficient, so oomd also monitors the system holistically. This is in contrast to Linux’s OOM killer, which focuses primarily on the kernel’s concerns. Since OOM detection criteria can vary depending on workload, the plugin system supports customization to both the detection and process kill strategies. Thanks to this new ability to monitor key system resource indicators, oomd is able to take corrective action in userspace before a system-wide OOM occurs. Corrective action is configured via a flexible plugin system that is capable of executing custom code. Thus, in addition to oomd’s default process SIGKILL behavior, application developers can customize their plugin with alternate strategies, such as sending a 'back off' RPC to the main workload or dumping system logs to a remote service."

Security updates for Friday

Security updates have been issued by Debian (dnsmasq, linux-base, and openjpeg2), Fedora (libgit2, libtomcrypt, openslp, and perl-Archive-Zip), and openSUSE (gdk-pixbuf, libopenmpt, mercurial, perl, php7, polkit, and rsyslog).

[$] The problem with the asynchronous bsg interface

The kernel supports two different "SCSI generic" pseudo-devices, each of which allows user space to send arbitrary commands to a SCSI-attached device. Both SCSI-generic implementations have proved to have security issues in the past as a result of the way their API was designed. In the case of one of those drivers, these problems seem almost certain to lead to the removal of a significant chunk of functionality in the 4.19 development cycle.

Stable kernel 4.4.142

Greg Kroah-Hartman has released the 4.4.142 stable kernel. It is not an essential upgrade, "but a number of build problems with perf are now resolved, and an x86 issue that some people might have hit is now handled properly. If those were problems for you, please upgrade."

Security updates for Thursday

Security updates have been issued by Debian (ant, gpac, linux-4.9, linux-latest-4.9, taglib, vlc, and znc), Fedora (ceph), Red Hat (fluentd and qemu-kvm-rhev), Slackware (httpd), and SUSE (e2fsprogs, glibc, libgcrypt, mercurial, openssh, perl, rubygem-sprockets, shadow, and wireshark).

[$] Deep learning and free software

Deep-learning applications typically rely on a trained neural net to accomplish their goal (e.g. photo recognition, automatic translation, or playing go). That neural net uses what is essentially a large collection of weighting numbers that have been empirically determined as part of its training (which generally uses a huge set of training data) A free-software application could use those weights, but there are a number of barriers for users who might want to tweak them for various reasons. A discussion on the debian-devel mailing list recently looked at whether these deep-learning applications can ever truly be considered "free" (as in freedom) because of these pre-computed weights—and the difficulties inherent in changing them.

[$] The PEP 572 endgame

Over the last few months, it became clear that the battle over PEP 572 would be consequential; its scale and vehemence was largely unprecedented in the history of Python. The announcement by Guido van Rossum that he was stepping down from his role as benevolent dictator for life (BDFL), due in part to that battle, underscored the importance of it. While the Python project charts its course in the wake of his resignation, it makes sense to catch up on where things stand with this contentious PEP that has now been accepted for Python 3.8.

Stable kernel 4.17.8

Stable kernel 4.17.8 has been released. This fixes the issue with i386 systems that was present in the 4.17.7 kernel.

Security updates for Wednesday

Security updates have been issued by Arch Linux (curl, lib32-curl, lib32-libcurl-compat, lib32-libcurl-gnutls, libcurl-compat, and libcurl-gnutls), Debian (blender, ffmpeg, and wordpress), Fedora (curl), Gentoo (tqdm), Oracle (kernel), Slackware (mutt), SUSE (xen), and Ubuntu (policykit-1).

[$] Kernel symbol namespacing

In order to actually do anything, a kernel module must gain access to functions and data structures in the rest of the kernel. Enabling and controlling that access is the job of the symbol-export mechanism. While the enabling certainly happens, the control part is not quite so clear; many developers view the nearly 30,000 symbols in current kernels that are available to all modules as being far too many. The symbol namespaces patch set from Martijn Coenen doesn't reduce that number, but it does provide a mechanism that might help to impose some order on exported symbols in general.

Stable kernel updates

Stable kernels 4.17.7, 4.14.56, 4.9.113, and 4.4.141 have been released. The 4.17.7 kernel is broken for i386 systems. "I did this release anyway with this known problem as there is a fix in here for x86-64 systems that was nasty to track down and was affecting people. Given that the huge majority of systems are NOT i386, I felt this was a safe release to do at this point in time." Beyond that, these kernels all contain the usual set of important fixes.

Security updates for Tuesday

Security updates have been issued by Arch Linux (thunderbird), Debian (ruby-rack-protection), Fedora (firefox and soundtouch), Red Hat (kernel), Scientific Linux (gnupg2), SUSE (perl and python-paramiko), and Ubuntu (policykit-1).

[$] Python post-Guido

The recent announcement by Guido van Rossum that he was stepping away from his "benevolent dictator for life" (BDFL) role for Python was met with some surprise, but not much shock, at least in the core-developer community. Van Rossum has been telegraphing some kind of change, at some unspecified point, for several years now, though the proximate cause (the "PEP 572 mess") is unfortunate. In the meantime, though, the project needs to figure out how to govern itself moving forward—Van Rossum did not appoint a successor and has left the governance question up to the core developers.

Security updates for Monday

Security updates have been issued by CentOS (firefox, gnupg2, kernel, python, and qemu-kvm), Debian (389-ds-base, cups, imagemagick, kernel, mailman, ruby2.1, sssd, thunderbird, and znc), Fedora (glpi, hadoop, kernel, rubygem-sprockets, singularity, thunderbird, wordpress, xapian-core, and xen), Mageia (cantata and flash-player-plugin), openSUSE (exiv2, libvorbis, nodejs6, nodejs8, openslp, singularity, slurm, and tiff), and SUSE (kernel-azure and openssl).

Kernel prepatch 4.18-rc5

The 4.18-rc5 kernel prepatch has been released. "For some reason this week actually felt very busy, but the rc5 numbers show otherwise. It's all small and calm, and things are progressing nicely."

[$] Tracking pressure-stall information

All underutilized systems are essentially the same, but each overutilized system tends to be overloaded in its own way. If one's goal is to maximize the use of the available computing resources, overutilization tends not to be too far away, but when it happens, it can be hard to tell where the problem is. Sometimes, even the fact that there is a problem at all is not immediately apparent. The pressure-stall information patch set from Johannes Weiner may make life easier for system administrators by exposing more information about the real utilization state of the system.

Security updates for Friday

Security updates have been issued by Debian (cinnamon), Fedora (docker, firefox, jetty, and knot-resolver), Oracle (gnupg2), Scientific Linux (gnupg2), SUSE (gdk-pixbuf, java-1_8_0-openjdk, libopenmpt, php7, and rsyslog), and Ubuntu (dns-root-data, dnsmasq, and thunderbird).

Guido van Rossum resigns as Python leader

Python creator and Benevolent Dictator for Life Guido van Rossum has decided, in the wake of the difficult PEP 572 discussion, to step down from his leadership of the project. "Now that PEP 572 is done, I don't ever want to have to fight so hard for a PEP and find that so many people despise my decisions. I would like to remove myself entirely from the decision process. I'll still be there for a while as an ordinary core dev, and I'll still be available to mentor people -- possibly more available. But I'm basically giving myself a permanent vacation from being BDFL, and you all will be on your own."

[$] Six (or seven) new system calls for filesystem mounting

Mounting filesystems is a complicated business. The kernel supports a wide variety of filesystem types, and each has its own, often extensive set of options. As a result, the mount() system call is complex, and the list of mount options is a rather long read. But even with all of that complexity, mount() does not do everything that users would like. For example, the options for a mount operation must all fit within a single 4096-byte page — the fact that this is a problem for some users is illustrative in its own right. The problems with mount() have come up at various meetings, including at the 2018 Linux Storage, Filesystem, and Memory-Management Summit. A set of patches implementing a new approach is getting closer to being ready, but it features some complexity of its own and there are some remaining concerns about the proposed system-call API.

Pages