Why are EMC, HP and IBM so tentative about clustered NAS filers?

The biggest storage vendors obviously see a huge opportunity in the rise of unstructured data. Yet for some reason, they're not taking aim at the leaders in HPC and clustered filer storage market, DataDirect Networks and Isilon.

High-performance and scalable storage technology is hard to do. Just ask Alex Bouzari, the co-founder and CEO of DataDirect Networks. But when you do get it right, customers flock to your door.

Clustered filers are hard work. Just ask Sujal Patel, CEO of Isilon Systems. Even when your firm has a winning technology, mistaken senior managers can put your very existence at risk. But Patel came back as CEO and the rescue of Isilon is proceeding apace, as customers buy the company's fast filer products.

It may appear that Isilon more or less invented clustered filers, although it shares supplying highly scalable and very fast storage to the digital media and HPC markets with DataDirect Networks, which does block storage as well as files in its S2A storage appliances.

For both Isilon and DataDirect, their operating systems and system software sets are mature but still being further developed, and their hardware architecture is also on a development roadmap. Both companies have a steady stream of customer wins, and in their areas of clustered and/or highly scalable storage at petabyte-plus levels, as well as high-performance storage, they have established themselves as worthwhile suppliers.

Which begs the question: Why haven't EMC, HP, IBM, NetApp and Sun been able to match the products of DataDirect and Isilon?

It's not because the major storage vendors don't see a huge opportunity with the rise of unstructured data. (Witness EMC's sponsoring of the IDC Digital Universe study for two years in succession.) They also see the opportunities inherent in the digitisation of movies and the associated special effects industry.

Yet when it comes to EMC, HP, IBM, NetApp and Sun playing in this market, either they don't or else their offerings are either tentative or sidestep the two main players.

Five years experience suggests that retrofitting Isilon IQ/DataDirect S2A technology smarts onto an existing product is a very hard thing to do.
Take Sun. Although Sun's X45000 Thumper hybrid storage/server product is reputedly a $100 million a year run rate product, it doesn't seem to be cramping DataDirect's or Isilon's business. Data warehousing is more Thumper's forte, as shown by Sun's strategy of selling the X45000 with application software partners such as database software maker GreenPlum.

NetApp saw the need to cluster its FAS block and filer products and so in 2003 purchased Spinnaker Networks. Five years later, NetApp has a clustered version of its Ontap operating system – GX -- separate from its mainstream Ontap 7G. GX has good SPEC benchmark results but it's not cramping Isilon or DataDirect's style any more than Sun is. Perhaps when Ontap 8 (the presumed name of the merged 7G and GX product) appears -- hopefully next year -- it will prove stronger competition.

Five years experience suggests that retrofitting Isilon IQ/DataDirect S2A technology smarts onto an existing product is a very hard thing to do.

EMC and HP have chosen to build their own technologies. EMC's Atmos was announced only this month (six months later than originally intended), but EMC has been shipping to beta-type customers since June. Atmos runs on InfiniFlex 10000 hardware -- a server running the Atmos software with, essentially, SAS-attached JBODs. Atmos is intended for cloud computing with service providers storing files in object format in large data centres and protecting them by copying them to other data centres, thus also providing local access, faster than fetching files from halfway around the globe.

There is much, much more to Atmos than this sketchy overview suggests. But we know enough about Atmos to know that it is no Isilon or DataDirect competitor. Movie special effects houses and high-performance computing (HPC) IT buyers won't be queuing up to buy Atmos. EMC would say that the cloud storage (information handling) market will be many times larger than SFX superfilers. EMC is probably right, but it has no effective answer to either Isilon's or DataDirect's products. The non-clustered Celerra is not in the same league.

HP has its Extreme Data Storage 9000 system. It's a curious product, developed in stealth with secrecy about its SAS fabric linking the processing units and the storage units. Storage suppliers are adopting SAS instead of FCAL quite widely, so why is HP clamming up?

Isilon and DataDirect are seemingly ready to take a beating from the famous five of the storage industry. But the five are refusing to play.
There is no good ExDS9100 performance information. The product hasn't really been announced yet -- HP just drops hints. So now HP runs the risk of presenting it to an underwhelmed world, the same fate that befell IBM when it finally did announce its XIV block storage product and its Scale-Out File System (SOFS) a couple of months ago.

The acquired XIV technology was a disappointment. No clustering, limited scalability and all data mirrored, halving the realisable storage capacity. It seems to be a product in mid-development still and not one to give DataDirect the slightest cause for concern. The SOFS offering is a services-led one and IBM has not been generally open about the hardware. On the face of it, the SOFS technology has multi-petabyte scalability and good performance, with CIFS running at 577MB/sec and NFS access at 880MB/sec across a 10Gbit Ethernet link. So why isn't it making more of an impact? Why is IBM so coy about it?

So, there they are, Isilon and DataDirect, two prime targets seemingly ready to take a beating from the famous five of the storage industry. But the five are refusing to play. EMC has sidestepped into the cloud. HP is being coy. Ditto IBM. Sun is more focussed on its open storage opportunity, perhaps, like the way EMC views the cloud, seeing it as a far bigger market than movie special effects and HPC. Whatever, Sun just isn't looking in Isilon's and DataDirect's direction.

NetApp is deep into merging 7G and GX and may in any event cede the high ground to DataDirect and Isilon. NetApp is content with the larger middle ground where its FAS customers can add a layer of clustered storage to their existing NetApp kit and use their existing IT admin NetApp skills and procedures to absorb and manage them, a much-valued aspect of NetApp's product strategy.

So it looks as if DataDirect and Isilon are going to have a clear and unimpeded run at their markets in 2009. If their customers continue to need more kit and new customers, seeing the spread of these products in their industry, join them, they should survive the downturn quite nicely. They already have low-end products and this product-offering tactic can be extended.

Will HP and IBM get their fingers out and start offering serious competition next year? Maybe yes, maybe no. Both could be mostly occupied with preserving sales of their bread-and-butter products rather than launching untried products into markets already dominated -- if that's not too strong a word -- by Sujal Patel and Alex Bouzari.

Chris Mellor is storage editor for The Register.

Read more on SAN, NAS, solid state, RAID