Storage array makers not following EMC's lead on flash solid state drives

Storage array manufacturers are copycats, except when it comes to flash solid state drives. In this area, storage array makers haven't yet followed the innovator, EMC. Why?

Storage array manufacturers are copycats. They each have their dual-controller modular arrays and they have their high-end monolithic arrays and low-end entry-level arrays. When one storage array maker makes a significant innovation, the rest of them follow.

Except, that is, when it comes to flash solid state drives (SSDs). In this area, EMC has innovated, and the rest of the storage array makers haven't.

In January, EMC announced its enterprise flash drive (EFD) storage for its Symmetrix DMX4 arrays. The SSDs formed a very fast access Tier Zero storage with lightning-fast access to data. Yes, they were expensive, but they were fast. Customers short-stroking Symmetrix Fibre Channel drives for speed -- that is, only writing data in the fastest-accessed tracks and ignoring the others -- found that these flash drives delivered better performance. Plus, that performance boost came at a reasonable cost, as the short-stroking had artificially inflated those users' cost/GB.

SSDs have lightning-fast reactions and can wrong-foot slow controllers.
The storage world went "Wow!" and saw the logic and learned about STEC, the flash drive supplier to EMC, and its unique Fibre Channel interface SSDs. STEC said it was talking to other storage array manufacturers, and we all waited for product announcements from vendors following in EMC's footprints. Yet, there hasn't been a single one, just vague expressions of intent.

EMC has since rubbed salt in this wound and added SSD storage to its modular Clariion CX4 product line. So the question is, why are all the other storage array makers being so tardy?

There was talk of EMC having a restrictive contract with STEC which, supposedly, runs out this month. There was also the realisation that adding SSDs to storage arrays is difficult because the arrays have interfaces and controllers geared to hard drive I/O rates and access patterns.

Storage array vendors rebalancing networks for SSDs
SSDs have lightning-fast reactions and can wrong-foot slow controllers. An array's internal network links and its controller resources would need rebalancing, which could take months of effort. Imagine if the taps in your home had to deliver water from the normal mains and from a very high-pressure water source as well; you would need serious plumbing modification to cope. That's what the array vendors are having to comprehend and work out how to do.

Hitachi Data Systems has said it will add SSD storage to its arrays, meaning, we expect, the high-end USP-V. So the HP XP24000 and Sun 9990, both rebranded USP-Vs, will get them. It's been reported that HP has been working on adding SSDs to its modular EVA line. Sun CEO Jonathan Schwartz has blogged that in the next few months, Sun will make storage announcements which will include flash.

This leaves Dell, IBM and NetApp as the remaining majors not committed and, in the next tier of storage suppliers, 3PAR, Compellent Technologies and Pillar Data Systems being uncommitted. Dell, like Fujitsu Siemens Computers, resells EMC kit and each get access to SSD storage that way.

IBM follows a different SSD path
IBM has been as radical as EMC but in its own way. Thinking that adding low latency flash drives to hard drive arrays could quickly overwhelm the array controller and I/O resources, IBM built a huge flash drive out of Fusion-io SSD components and arrived at a 4TB SSD array hooked up to the PCIe bus structure of a specialised server, the SAN Volume Controller (SVC), which then delivered the resultant 1 million IOPS to application servers as a SAN resource. This is Project Quicksilver.

Project Quicksilver shows that IBM is taking SSD storage quite seriously. It's not exactly hurrying though.
Yes, it is a Tier Zero enterprise flash drive resource but it happens to be outside IBM's DS8000 arrays. Normally the SVC deals with hard drive arrays across a Fibre Channel fabric. This 4 TB lump of SSD storage was directly connected to the SVC's own bus and, in effect, the SVC was the SSD controller. Could this be added to an IBM SAN as an SSD array? IBM says it could take 12 months to productise Quicksilver, but the project shows that IBM is taking SSD storage quite seriously. . .even if it's not exactly hurrying.

What is NetApp doing? Its chief technology officer, Jay Kidd, has blogged: "NetApp is in the process of certifying enterprise-grade SSDs that you can use in our existing storage shelves."

3PAR CEO David Scott says that because data in 3PAR's Inserv arrays is striped across all the drives, 3PAR doesn't have the same need to deliver more IOPS as the narrow striping major storage vendors (meaning EMC and the others). When the time is ripe for SSDs to be used, then 3PAR will adopt the technology.

EMC VP Chuck Hollis blogs that any storage array which stripes its data cross all available spindles (hint: 3PAR) has a novel problem with flash. The array software has to alter the way it deals with data as it needs to ask if incoming data is for the SSD or hard disk drive storage and deal with it differently in either case. In effect, the array becomes two logically separate sub-arrays. Modifying array software to do this takes time.

Hollis adds that flashed Symmetrix arrays just love database hot spots, those situations loathed by database administrators because too many I/Os are queued up waiting to access a portion of a drive array unable to cope. Since SSDs just eat this problem and spit it out completely sorted, EMC sales reps love a hot spot-stalled drive array -- it's a perfect EFD Symmetrix opportunity. They can make flash hay while the sun shines and competitors vacillate.

Of course, this is a short-term thing. SSD validation processes will come to an end and SSD capacity increases and cost decreases (via multi-level cell technology and enhanced write endurance) will accelerate adoption. But it won't be a hard drive wipeout -- far from it, with IDC expecting only 2% of hard drive shipments to be replaced by flash in the next few years. But there will be a halo effect with a flash tier safeguarding HDD array sales (as well as preventing EMC poaching customers through a hot-spot problem solving approach).

We are in a calm period before a relative storm. EMC planted a seed and the germination period is coming to a close. Expect a flurry of announcements in the next few months, with Sun probably leading the way.

About the author:Chris Mellor is Storage Editor for The Register.

Read more on SAN, NAS, solid state, RAID