Has EMC missed out on the hybrid flash array?

EMC has announced a three-pronged entry into the world of the flash array, but why hasn’t the giant vendor adopted the hybrid flash array pioneered by some innovative startups?

EMC has plunged into the flash array market with a three-part dive. Has it made a clean entry?

Part 1 is its server flash cache card, the VFCache card formerly known as Project Lightning. At the recent EMC World conference in Las Vegas, it revealed more about its two coming networked flash arrays.

Project Thunder is an enclosure, presumably rackmounted, containing between one and 10 VFCache cards, with an InfiniBand connection to servers that runs at 56 Gbps. The whole thing is dubbed a server area network and will provide the servers with a low-latency link. Apparently 40 Gbps Ethernet may also be a server networking option.

The intention is that Thunder will feed the VFCache cards with data so they can carry out their read caching work. We don't know if Thunder will provide the write caching that the VFCache cards cannot do, nor if it will enable the automated vMotion of virtual machines from server to server, again which VFCache cannot do.

Thunder will be fed in turn by back-end VMAX or VNX arrays and possibly Isilon scale-out filer arrays too, with FAST, EMC's Fully Automated Storage Tiering array software, being used to move hot data blocks from the array to Thunder and then on, possibly, to the VFCache cards.

FAST will move cooler data blocks back from Thunder to the backing array if -- and this is an important “if” -- Thunder is designed as a tier of storage, but it needn't bother doing this if Thunder is seen as a cache. In that case, incoming hot data blocks will simply overwrite the oldest and least-used blocks in the Thunder array.

The second EMC flash array will use technology acquired with the Israeli startup XtremIO, which it has bought for a rumoured $430 million. This technology produces scale-out all-flash arrays that can be clustered to a maximum of eight nodes with a capability of 2.3 million IOPS. EMC calls this development product Project X. Its technology includes data deduplication and compression so as to increase the raw capacity of the product three to five times, possibly more.

Project X will be connected to servers via a traditional storage area network (SAN) -- Fibre Channel, in other words -- and we can assume 16 Gbps links will be supported. This means servers will still endure network latency, but Project X will be able to support lots and lots of servers, or desktop endpoints in the VDI scenario.

EMC told EMC World attendees that Project X has a 5 GBps bandwidth when cloning VMs, can create 100 10 TB data volumes in seconds, has 150,000 write IOPS, 300,000 read IOPS and 180,000 mixed read/write IOPS.

No other mainstream storage vendor has such a determined or product-rich flash storage strategy, as far as we know. We have VFCache for server flash caching, Thunder for very high-performance flash array storage and Project X for high IOPS flash storage but with network latency.

There are two classes of storage array startup that EMC and the other storage vendors will face. One is the all-flash array like those from Nimbus, Pure Storage and Whiptail, plus Kaminario and Texas Memory Systems. These will migrate to the SMB space as solid-state becomes cheaper with three-bits-per-cell flash and as data deduplication and compression provide more effective flash capacity. Thunder can compete with the high-performance ones, but it has no compression and no deduplication so its effective capacity will be its raw capacity. Its cost per gigabyte will be higher than that of deduplicated and compressed competition.

Project X should be able to compete with the deduplicated and compressed all-flash array startup products and have the advantage, to EMC customers, of back-end array integration via FAST.

The other flash array startups are producing hybrid arrays, ones with a flash tier or cache and a layer of hard disk drive storage behind it. Example suppliers are GreenBytes, Nimble Data and Tintri. They optimise their systems for performance and capacity, and it appears that EMC has no direct answer. Neither does any other mainstream storage vendor.

When combined with data deduplication and compression of flash tier data, these suppliers provide an even better combination of capacity and performance, and, again, no mainstream storage vendor can match them.

Has EMC got its flash layer cake correctly layered by leaving out hybrid arrays with ground-up designed software? Are the flash-assisted VMAX and VNX arrays hobbled by old software designed in the disk era, leaving EMC open to attack by hybrid arrays that provide a compelling mix of performance, capacity and low cost per gigabyte or cost per VM that it simply cannot match?

Are hybrid arrays compelling? That is going to be decided by customers choosing to buy or not buy, and we'll have to see. EMC has struck a bold course, and the other vendors will have to follow suit. It's an exciting time in storage array land, and there's much to look forward to.

Chris Mellor is storage editor of The Register.

Read more on SAN, NAS, solid state, RAID