Petya Petrova - Fotolia

Linux versus Unix hot patching

We examine the current state of play in the ongoing competition between the Linux and Unix server operating systems

There has always been a debate about how close Linux can get to the real operating system (OS), the core proprietary Unix variants that for two decades defined the limits of non-mainframe scalability and reliability.

But times are changing, and the new narrative may be when will Unix catch up to Linux on critical reliability, availability, and serviceability (RAS) features such as hot patching?

Hot patching, the ability to apply updates to the OS kernel while it is running, is a long sought-after but elusive feature of a production OS.

It is sought after because both developers and operations teams recognise that bringing down an OS instance that is doing critical high-volume work is at best disruptive and at worst a logistical nightmare. Its level of difficulty also makes it somewhat elusive.

There have been several failed attempts and implementations that almost worked, but they were so fraught with exceptions that they were not really useful in production.

What has changed?

Early versions of Linux hot patching have been around for several years, most notably in a company called Ksplice, acquired a few years ago by Oracle. But the real change happened earlier this year when SuSE declared its hot patch capability, kGraft, previously in limited supply, was now available and suitable for all production workloads.

Read more about enterprise Linux

On 25 August 1991, Linus Torvalds released the Linux kernel. We look at how the open source operating system has evolved in the last quarter of a century.

The open source world this week hears that SUSE has developed a new technology called known as kGraft for live run-time patching of the Linux kernel.

This is a bold claim, opening SuSE up to problems if it fails regularly in production. However, in further support of the claim that it can support critical enterprise workloads, at the Sapphire conference in May 2016, SuSE further announced its hot patch capability was certified for SAP Hana. SuSE indicated that the percentage of such patches that could be applied using kGraft after expert review was in the high 90s. While the remaining major Unix operating systems have tremendous proven capability in production and support their own hardware with a number of unique features that Linux still lacks, the importance of this event cannot be over-emphasised.

The open source community, which has proven itself capable of innovating a highly capable fast-follow OS environment, has now proven that it can lead with advanced features ahead of the legacy Unix community. Some versions of Unix have had hot patching for years but never really emphasised it because it was limited in applicability, and those lacking it will catch up. However, the perception of Linux as the lesser option has been permanently shattered.

The reason is probably as simple as it was inevitable – different priorities and possible resource and budget constraints. The number of developers doing core development on proprietary Unix is likely to be an order of magnitude smaller than the number of people contributing to the Linux kernel and other related projects. The remaining revenue streams for the proprietary products, while representing secure and highly profitable cash flow for the next decade, are simply not enough to match the momentum behind Linux, requiring possibly more focused application of resources and, in some cases, representing budget constraints.

Oracle has just come out of a significant development cycle which saw some breakthrough capabilities around hardware acceleration of selected Oracle software operations and major improvements in security features supported by the latest scalable processor architecture hardware, along with expansions of its cloud capabilities.

IBM has continued to invest in AIX, its proprietary  version of Unix, , and has done impressive things with continued RAS improvements for AIX with non-disruptive upgrades (not the same as hot patching) and support for the new Power8 CPU and the OpenPower community.

Hewlett Packard Enterprise (HPE) has been considerably opaque on the future of Hewlett Packard Unix (HP-UX) in recent times, having lost major market share over several years. It has not ported HP-UX to the new x86-based Superdome-X mission-critical servers.

This suggests HPE may intend to give up on HP-UX in the long-term, and focus instead on Linux and Windows.

The hot patch landscape today

While SuSE can be said to have crossed the line first with general availability of kGraft, the hot patch ecosystem is active, and even boasts multiple architectural approaches. In addition to kGraft, Red Hat is offering a tech preview of its Kpatch utility, and Oracle has made the original Ksplice available for its Linux distribution.

The different products operate slightly differently. SuSE’s kGraft  works on a per-thread basis. This allows the OS to run continuously while pointers to the new versus old runtimes are swapped. However, the entire process may take several minutes as the systems continue to operate under the unpatched version and switch transparently to the new environment when the patching process is completed.

Oracle’s Ksplice and Red Hat’s Kpatch are reported to pause kernel execution for somewhere between 10 and 40 milliseconds and then perform all internal juggling at once. This momentary pause may be insignificant to some, but very noticeable in other environments.

The debate, reduced to its essentials, is the choice between getting it all done at once with just a small hiccup that might turn out to be a big burp if your system is doing very high volume transactions, versus having the process take several minutes without interruption. There are also differences in the limitations regarding the kinds of patches.

On the Unix front, suppliers have been discussing and releasing partial solutions for years, and others have hinted this capability for future releases. In 2014, a senior Oracle technologist listed hot patching as a focus for future development, so we can speculate that it is due to be delivered in an upcoming release of Solaris.

IBM has documented and commented on hot patch capabilities since 2007, but the available documentation seems to indicate that it has too many limitations to be considered a mature production capability.

In the latest AIX release, 7.2, IBM has included a live update capability that seems to install a complete new OS image in a parallel logical partition, and transfer over all running processes and their memory.

This approach appears to overcome most limitations on the kinds of patches that can be applied, but the documentation seems to indicate that the process entails a blackout period of some kind. But amidst all of the tentative steps, it appears that only SuSE has made a viable hot patch capability available for general availability production use.

Where is Linux heading?

With technologies competing to solve the same problem, the Linux community is doing the same thing it often does and endorsing both.

In the strange process of cooperation and competition that is at the heart of the open source community, both camps are feeding intellectual property upstream into the kernel development process, and the 4.0 kernel apparently contains code for both approaches.

It is almost certain that a future release of the 4.x kernel will contain production-ready hot patching as a standard feature, placing the burden on the Unix providers to prove they can keep up with Linux..

Richard Fichera is a vice-president and principal analyst at Forrester. This article was taken from his infrastructure and operations blog, Linux vs Unix hot patching – have we reached the tipping point?

Read more on Open source software