Has the storage hardware lock-in had its day?

Two years ago I wrote that the storage industry was apparently ripe for huge change.

The nub of my argument was that storage is a sector of the IT supplier world in which customers are forced to spend money on what are essentially a commodity – ie, drives – wrapped in proprietary software built into hardware controllers.

The argument progressed to take note of the revolution in the server world that had occurred as Linux effectively decoupled proprietary operating systems from RISC chip-based hardware in the previous decade, making open source OSs on x86 commodity hardware a much cheaper option.

The conclusion of the piece looked around at the likely candidates in the world of storage that might do what Linux did in the server world. These comprised storage software that could be deployed on commodity hardware and included GreenBytes and Nexenta as well as open source products such as Red Hat’s and ZFS.

Two years on and it seems the hazy predictions based on a theory and a few small shoots of evidence have been validated by, among others, the biggest name in storage.

This week I spoke with Ranga Rangachari, VP and general manager for storage with Red Hat (not the biggest name in storage), who put forward a similar argument to the above, namely that: “Storage is dominated by ‘tin-wrapped software’ and customers are sick and tired of being locked into silos, with for eg, vendors with three different solutions.”

Rangachari reiterated the argument that what happened in the move from RISC to x86 could happen with storage and that the drivers now are the cloud, the volume of unstructured data and the rise of online analytics platforms such as Hadoop, which requires co-resident storage and processing power, with data moving, as Rangachari put it “East to west not north to south” as in existing server-SAN infrastructures.

The rise of such hyperscale server/storage infrastructures has been pioneered by the likes of Google and Facebook and is exhibit A in the rise of architectures that challenge the existing enterprise storage paradigm.

Instead of shared, but remote, storage in, say, an enterprise SAN, these highly performant Web-serving and analytics stacks comprise converged server and storage hardware, all made of cheap commodity parts with redundancy at the level of the whole unit rather than components within.

Elsewhere – exhibit B – is the emergence of converged storage/server products that ape the hyperscale architectures and are usually geared towards virtual environments. These include Nutanix, Scale Computing and Simplivity.

Exhibit C is the continued rise of software-only storage products that customers can run on any hardware. Virtual storage appliances that will run on virtual or physical machines are available from all the big storage vendors as well as the likes of DataCore, Nexenta.

An important addendum to exhibit C is the plan by VMware to include storage software features in its virtualisation hypervisor products. VMware already has a virtual storage appliance, but it plans to include storage software capability in the form of its Virtual SAN which will allow users to create up to petabytes of capacity from existing unused disk. This threatens to seriously undermine the market of entry-level to midrange storage players.

Finally there is exhibit D – evidence for the prosecution, as it were – and this is EMC’s recent announcement of its forthcoming ViPR storage virtualisation/private cloud/big data software layer.

The 800lb gorilla of the storage market justified ViPR as a response to a changing storage landscape, and is in large part a storage virtualisation platform that will knit together disparate storage systems from any vendor and from commodity drives.

On the surface of things it’s quite remarkable that the biggest disk system vendor should potentially allow users to create storage from any other storage supplier. But, ViPR can give EMC very sharp and well-barbed hooks in a user’s environment, as a software layer that embraces all storage underneath it.

Maybe it should have been called Python, for its ability to smother an organisation’s storage systems, and is apparently the antithesis of the move to more openness in systems that I’m arguing is a trend here. So, why it is evidence for my case?

Because it is a recognition by EMC that storage will henceforth no longer solely reside on the enterprise storage array as such; that it will be distributed in traditional storage environments, converged hyperscale datacentres and edge devices and that these must be linked by a software layer that virtualises the capacity underneath it.

So, it seems the biggest player in the storage market has recognised that the dominance of the traditional storage array is a thing of the past. Having made that concession it will be interesting to watch whether the likes of EMC can transition to the new reality against its rivals that offer more open storage software.

Let’s check back in another couple of years.