What is close-to-the-metal?

Recent discussions with database company Scylla threw up the term close-to-the-metal, or some simply say close-to-metal.

But what does close-to-the-metal mean?

The Computer Weekly Developer Network team has gathered a handful of comments and definitions on this subject and lists them (in part, with full credit and links) below for your reference.

Essentially, close-to-the-metal means database software that works in close proximity to and with knowledge of the actual instruction set and addresses of the hardware system that it is built to run on.

This means that the database (or potentially other software program type) itself can work to ‘squeeze’ out as much power for any given hardware estate (the process of scaling up) before it then needs to expand with further processing and analytics nodes (the process of scaling out).

As noted by wikic2, close-to-the-metal (or close-to-the-hardware) means we’re deep in the guts of the system. “The C [programming language’s] memory management is considered close-to-the-metal compared to other application languages because one can easily see and do mathematics on actual hardware RAM addresses (or something pretty close to them).”

The above-linked definition suggests that close-to-the-metal can sacrifices hardware choice through lock-in and may introduce risk because there is no interface layer to protect against silly or dangerous ranges, settings, or values.

Roger DiPaolo, provides an additional (and much needed) piece of extra colour here when he says that with close-to-the-metal (in programming terms) means that the language compiles (or assembles) all the way down to native machine code for the CPU it is to run on.

“This is so that the code has no ‘layers’ it has to go through to get to the CPU at run time (such as a Virtual Machine or interpreter). A close-to-the-metal language has the facilities for directly manipulating RAM and hardware registers. C and C++ can both do this.”

So trade offs or not, this is the approach Scylla has taken to building its core technology proposition.

The company claims that independent tests show a cluster of Scylla servers reading 1 billion rows per second (RPS) – and so the firm says that this is a performance that ranks ‘far beyond’ the capabilities of a database using persistent storage.

“Everyone thought you’d need an in-memory database to hit [MOPS] numbers like that,“ said Dor Laor, CEO of ScyllaDB. “It shows the power of Scylla’s close-to-the-metal architecture. For 99.9% of applications, Scylla delivers all the power [a customer] will ever need, on workloads other NoSQL databases can’t touch and at a fraction of the cost of an in-memory solution. Scylla delivers real-time performance on infrastructure organisations are already using.”

Bare-metal platform provider Packet partnered with Scylla to conduct the test on 83 of its n2.xlarge servers, each running a meaty 28 physical cores.

The benchmark populated the database with randomly generated temperature readings from 1 million simulated temperature sensors that reported every minute for 365 days, producing a total of 526 billion total data points.

Could close-to-the-metal could be close-to-the-edge of where the art of database computing goes next? Some would say Yes.