The I/O blender means enterprise flash is here to stay

451 Group analyst Simon Robinson explains why enterprise flash is worthy of its hype and will consign HDDs to the background

You’ve probably heard by now that flash is the big thing in enterprise storage, but you may also be overwhelmed by the options when it comes to bringing flash storage into your environment, writes 451 Group analyst Simon Robinson.

This is no surprise; enterprise flash is a hugely hyped market. There are established storage companies and a stream of new start-ups peddling flash products and the wall of hype and tyranny of choice threaten to stifle adoption.

But, for once the hype may be justified; it may take a while, but in a few years we’ll wonder how we coped with bulky, expensive, complex and massively overprovisioned storage arrays crammed with spinning disks to run performance-hungry applications.

Such workloads will run on solid state storage, though whether this is enterprise flash, PCM, Memristor or something else is a different matter that I’ll save for another day. The question for the overwhelmed IT manager for now is, where do I begin?

In my next column I will outline the main approaches emerging around flash-optimised architectures. Here, I focus on why flash is causing a sensation now, and why it’s set to play a starring role in the future of enterprise storage.

Why enterprise flash?

Flash is being adopted in the datacentre because the HDD has become a performance bottleneck.

Disk technology developed at a fabulous rate in the half century since IBM launched the world’s first magnetic drive, in 1956. That is why it is now the dominant means of storing primary, working data in IT. And disk is not going away – enterprise flash isn’t going to elbow disk out of the datacentre, but will augment it.

Though disk capacity has increased tremendously in the last 50 years, disk performance has hit a wall when handling random I/O.

Disk the weakest link

Disk is slow at handling random I/O because it makes disk drives spend far more time looking for data than actually reading or writing. The reasons are directly related to disk’s mechanical nature and there is no fix. Disk is not going to get faster.

But while disk has hit a performance wall, the rest of the IT infrastructure hasn’t. Processors still obey Moore’s law – and even at times exceed it. Network bandwidth is also increasing exponentially, witness 10GbE take-up, and the emergence of 40GbE. The same goes for fibre channel and Infiniband.

One level up, the emergence of server virtualisation has made life even harder for disk. By an effect known as the I/O blender, server virtualisation churns and randomises data access and plays right into the Achilles’ heel of disk. Desktop virtualisation is no friend to disk either, because boot storms create intense peak loads that easily overwhelm disk storage systems.

At the top layer of applications, business users continue to demand increasing performance and more complex applications. For example, new forms of big data analytics are generating intense storage loads.

There is one way to help disk overcome the performance wall and that is a process called short-stroking. This boosts disk performance by cutting back the load on each drive. But it has always been a complex process and one that drives up costs. The bottom line is that disk is the weak link in the IT performance chain and there is no way to change that fact.

Flash to the rescue

Flash has already powered one massive IT revolution in mobile computing and is now powering a second one.

NAND flash met the needs of mobile phone makers that needed non-volatile and low-power storage for their devices. The rest is history. Flash production volumes soared and prices plummeted. In other words, the smartphone in your pocket paved the way for enterprise flash in the datacentre.

So how fast is it? Comparing like for like, the fastest enterprise flash drives are a couple of hundred times quicker than their disk equivalents in terms of IOPS. One enterprise flash drive can deliver tens of thousands of IOPS – the same as an entire mid-range disk array.

Costs are still falling and not just because of production volumes. Unlike disk, flash technology is still developing very rapidly; from the flash drives themselves, through to caching and other management software, as well as flash-powered storage systems.

Better flash cell management is driving up performance and reliability and now enterprise flash is predominantly multi-level cell (MLC), which previously only powered consumer devices, but can now provide enterprise performance, reliability and longevity at an attractive price point.

Manufacturers of enterprise flash arrays claim prices as low as $5/GB. But an equally if not more important parameter is price per unit of performance, most often measured in IOPS. In this, flash is about 40 times cheaper than disk, and this is the defining quality and virtue of flash.

As a major bonus, flash also saves on power, cooling and physical space, all of which are in short supply in many datacentres. Per IOPS, flash uses around 600x less kW than disk. We know of datacentre operators that have seen 90% reductions in power and cooling costs when replacing disk with flash.

Flash - not just for exotic apps

So who’s using flash and what for? At the high end, flash is used by hyperscale web providers such as Facebook and Apple to store the hottest, most frequently demanded data.

In the enterprise space, the financial industry led the charge with its deep pockets and intense need for performance. For that industry, flash has enabled faster analytic, trading and OLTP applications.

But flash use is now well beyond those two sectors and is being taken up across multiple industries.

As well as accelerating mainstream applications such as Exchange, enterprise flash solves the I/O blender randomisation problems created by server virtualisation that can defeat conventional disk storage systems.

It also allows businesses to virtualise the final tier of performance-sensitive database applications that, until now, have remained on physical hosts. Flash is also used as a means of handling the peak storage loads created by desktop virtualisation. 

Now that flash has arrived, the question for many IT managers is how they should implement it. In my next column we’ll look at the main options, and outline the pros and cons of each.

Read more on SAN, NAS, solid state, RAID