Frenzy in storage market as data growth casts shadow

It was flash, bang, wallop in the first three months of this year. The major storage firms exploded bombs en route to making sure they grow and prosper as crazy file storage growth threatens to overwhelm datacentres.


It was flash, bang, wallop in the first three months of this year. The major storage firms exploded bombs en route to making sure they grow and prosper as crazy file storage growth threatens to overwhelm datacentres.

What made it so frenetic was a sudden surge of activity in flash storage as well but the terrible twins of storage so far this year have been files and clouds. Everyone has been kissing cloud computing and promising to get the rampaging file elephant herd under control.


The scene was set recently by IDC coming out with version 2 of its EMC-sponsored data explosion white paper. We now have digital shadows and the world’s digital data total is approaching several squillion whateverbytes, enough to send EMC marketing flacks into dances of information management glee.


There are two approaches to the file deluge: sidestep it by having files in the cloud; or confront it with clustered de-duplicated file stores. IBM bought XIV for its grid or clustered file storage technology. EMC bought PI and put its boss Paul Maritz in charge of its cloud computing activities. Brocade launched its massive datacentre switch and added file management to the switch. Brocade, HDS and BluArc agreed to take each other’s super filer box, file management software and file virtualisation software.


Brocade is rapidly strengthening its file area network products and has possibly the most rounded networked file management product set of any vendor. Other pieces of the file puzzle include eDiscovery and content management with EMC’s document sciences acquisition bedding in nicely with documentum and Autonomy zooming up the awareness charts with its Zantaz eDiscovery and legal hold software.


There is also archiving with every supplier in the field and their brother putting out updated file and email archiving products. The Holy Grail of a single unified mega-archive covering files, emails, compliance and legal discovery concerns looks to be a mirage though. It just will not happen.


Besides this frantic activity, disk arrays looked mundane with regular but predictable array upgrades, until EMC lobbed its flash grenade and added solid state drives to Symmetrix.


Apple launched its MacBook Air with flash option and suddenly flash substitution for hard drives was on everyone’s hot topic list. It is not going to happen except for absolutely-must-have-performance customers until 2010 or so but happen it will. The technology is starting to look that solid.


Consumer storage matters are driving suppliers with Seagate and Western Digital focusing more and more on this sector. EMC mounted a surprise acquisition raid on Iomega for its consumer and Soho/SMB removable disk Rev technology and China market connections. Drobo introduced its lovely back-up box and cloud back-up services for consumers and SMBs multiplied.


Green stopped being exciting and retreated into the taken-for-granted background. Tape didn’t get as much of a boost as vendors might wish from tape being the greenest storage of all. Tandberg Data nearly crashed and burned, losing tons of money but stopped on the brink from which state it may recover. PivotStor introduced three relatively standard tape automation products but appears to have such a solid grasp of channel fundamentals that it may well profit as other tape vendors are still trying to build great business models involving disk and tape combinations.


The big galactic force exerting its huge gravitational pull on storage was VMware. Every storage vendor rushed to put a VMare-friendly face on their products, some even encapsulating their products as VMware virtual machines and you can’t get any deeper into VMware’s bed than that but remember, VMware doesn’t do monogamy. And neither should you. Hyper-V is coming.


De-duplication spread and spread but vendors found that one de-dupe size does not fit all. The technology is absolutely necessary to deal with rampant file bloat but storage system vendors feel the need to differentiate their technology, not wanting to be FalconStor clones. So much so that IBM is now reported to be trying to buy Diligent.


Whether you store files in your datacentre or in the cloud there is absolutely no point in duplicated information. When you’re storing petabytes, even exabytes of files, then de-duping can cut your costs like nothing else.


Flash is nice and promises faster access to data but the big, big need is capacity. Just as well as Seagate is revealing its HAMR and bit pattern media plans with 45TB 3.5-inch drives possible in a few years time. Without that and de-dupe we could all drown under a file flood.



Chris Mellor is editor at

Read more on Storage Virtualisation Solutions and Services