All-flash arrays: Will time run out for mainstream acceptance?

Antony Adshead | No Comments
| More

The all-flash array has been the flavour of recent times in storageland. But, has the hype exceeded the reality?

It may well have done. Or maybe it's just its timing is off.

If we look at our recent Purchasing Intentions survey there's no doubt that flash storage is popular. More than half of respondents indicated they had flash in use (36%) or planned to implement (7%) or evaluate it (25%) this year.

That's a fair amount of traction for flash, but not so much of that kudos can stick to all-flash arrays, according to another survey we ran this summer on, this time from 451 Research.

Its survey gained more granularity on the flash question and found for most respondents (67%) flash in use now is installed in existing SAN/NAS storage arrays while 25% have put it in servers. A mere 8% reported having deployed an all-flash array.

What that shows is that for the most part IT departments see the addition of flash to existing storage or to servers as the best way to accelerate I/O performance for key applications. That should be no surprise - all-flash arrays don't come cheap and with constrained budgets it's clearly best to target fast access media where you need it.

But it is in contrast to storage industry hype, and perhaps more importantly, the billions spent to develop or buy all-flash arrays, such as EMC's purchase of XtremIO, IBM's Texas Memory Systems acquisition and its pledge to invest $1 billion in flash.

The survey results also show that for many applications right now, disk is quite adequate. Compare the percentage of those in the survey that have virtualised servers (87%) with the numbers that have flash in place (36%) and it looks like there isn't a rigid driving shaft between the deployment of virtualisation and the need for flash.  

And so, the all-flash array could turn out to be something that takes its time to become a must-have. There's little doubt that disk will one day be superceded by solid state media, but in the meantime alternatives to flash are being developed. The hope must be in Big Storage Towers that flash is still the solid state media of choice when disk has finally had its day.

Death of the LUN: Another nail in the coffin from Gridstore?

Antony Adshead | No Comments
| More

Has the LUN had its day? It has been the de facto method of creating logical volumes on physical storage for decades, but in this era of virtualisation that may be becoming a thing of the past.

In VMware or Hyper-V environment the LUN still exists but only as a single large pool in which the virtualisation platforms' virtual drives are created.

But Gridstore, which substitutes so-called vLUNs for LUNs while providing storage for Microsoft Hyper-V virtual machines, claims to have done away with the LUN altogether, in something like the way Tintri does for VMware environments.

Gridstore combines storage arrays largely comprising cost efficient 3TB or 4TB SATA drives and MLC flash (500GB or 1TB in its performance nodes) with software that lives in Microsoft System Center

That software comprises a "vController" that matches Hyper-V virtual machines to vLUNs and provides quality of service (QoS) on storage provision. The vController, Gridstore says, emulates the single app-single server-DAS setup of the physical server world with data put in queues and sent in bursts rather than randomly as they occur.

"Virtual environments and the LUN are an architectural mismatch," says George Symons CEO at Gridstore, which makes scale-out array nodes in 12TB and 48TB base units, expandable to petabytes.

"A LUN must cater to many servers of different types of workload and are the site of the I/O blender effect", he says, referring to the way many and random I/O requests from virtual machines can overload physical storage.

In Gridstore arrays the vController can match storage performance to the needs of the VM, should that be sequential or random and can make sure "noisy" virtual machines do not disrupt others, alerting the admin if a VM isn't getting the gold, silver, bronze levels of performance set.

Gridstore claims 40,000 IOPS for the minimum three-node configuration of its 12TB devices. It doesn't sound a lot, when you think of the 500,000 and 1 million IOPS boasted by the all-flash provider.

But you don't need that, says Symons. "They talk of one and two million IOPS but 40,000 IOPS covers the needs of most people. To be honest most customers don't know what they need and most are in the 1,000 IOPS to 5,000 IOPS range. But in any case we can scale to 100,000 IOPS on nine nodes.

EMC's DSSD introduces the PCIe flash appliance

Antony Adshead | No Comments
| More

It's always interesting when a new storage technology comes along, not least because we have to figure out what exactly we're looking at.

Under the microscope this time is the fruit of EMC's acquisition of DSSD, a Silicon Valley startup bought last month.

EMC calls it "rack-scale flash" and Dan Cobb, CTO of the flash division at EMC, told me DSSD had tried to do three things in its short existence.

These are, he said: "Achieve massive storage density, using flash and other components to build an all-flash appliance - as distinct from an array - in terms of the number of chips that can be co-located."

"Build a connection to hosts that is massively parallel - using the Gen 3 PCIe connect - to connect between 8,000 and 16,000 independent flash dies directly to the host with tremendously low latency, compared to one drive that that typically holds 16 dies via a single SAS or SATA interface."

"With all flash management, wear levelling, garbage collection etc integrated into system software to achieve incredible effectiveness."

What we have here is a PCIe-attached flash appliance with capacity of "hundreds of TB, approaching petabytes" that can operate as direct-attached storage (DAS) or, it is claimed, as an extension to RAM.

EMC is aiming it at in-memory database use and big data for real time operations.

Cobb said EMC would be working on three forms of connectivity for DSSD. These would be:

·         As a traditional block interface using NVME (non-volatile memory express) to connect via PCIe.

·         Via custom APIs. DSSD will have developed for it new API primitives, for example a plugin for HDFS low latency operations tailored for specific applications.

·         In-memory database use - for example with MongoDB - that will result in virtual memory primitives that allow the database to see one giant memory store in DSSD.

So, it looks like we have a new beast on our hands, a PCIe-connected flash appliance for use as extremely low latency DAS and/or as a RAM extension.

It's a bit like a server-side flash store but with capacity that massively outscales existing products, which will be able to, as Cobb put it, be used as "a very fast failover, for example running multiple SAP HANA instances."

And at the same time it might be something like the new memory channel storage products now emerging. Sure, it's not DIMM-connected but the very low latency claimed by EMC may allow it to become a RAM extension.

Anyway, more will certainly become clear over the next year, with EMC planning to "harden" the product and go for some kind of product launch this time next year.

Actifio gets funding, but what's the future for a good idea in storage?

Antony Adshead | No Comments
| More

Actifio this week announced it had gained another $100m in funding, adding to a previous round of $107m, and according to Ash Ashutosh, founder and chief executive of Actifio, that pushes its market valuation to $1bn.

Actifio's Copy Data Storage Platform is the latest iteration of what we might call file virtualisation.

In the Actifio scheme numerous, isolated, many-times-duplicated versions of files are rationalised into the smallest number of copies required for the various requirements of the organisation - file access, backup, archiving and/or disaster recovery.

Whereas most businesses suffer the unwanted and unplanned multiplication of files as users copy, email, etc information between them, Actifio slims data down to a "golden copy", which is in practice the nearest one to the application that created or updated it.

Other copies are held elsewhere. They may be needed in production by other geographically located datacentres, or may be at different stages in their lifecycle, being backed up or archived, for example, and are updated from the golden copy so that all are eventually synchronised. Copies are retained with snapshot functionality, ie they can be rolled back to any point in time where changes were made.

Actifio targets the data protection and disaster recovery market and hopes to replace existing replication products, including at the storage array. It supplies the product as software or as an appliance on an x86 server. When customers deploy it Actifio discovers all the apps in the environment and policies can be set for their data - how many copies, on what tier of storage media it should be kept, etc.

It all sounds like the way you'd do file storage if you were thinking it up from scratch.

But there could be obstacles.

For a start, with 300 customers gathered over five years it hasn't exactly set the world on fire. And while the Actifio scheme is a clever one that can save a lot of disk space, re-architecting an existing environment might be a big ask for a lot of customers and a nerve-jangling prospect.

Perhaps that's why more than half its customers have deployed Actifio where data is clearly separated from production data - 6% use it for analytics and 17% for test and dev - or into relatively new, greenfield, environments at the 30% of its customers who are service providers.

Then there's the fact that there are many vested interests in storage that work against the idea of reducing the need for disk capacity. Ashutosh says the market it is playing in is worth $46bn but how much of that will take a swipe at disk vendors' revenues?

Whatever happens, the future for Actifio looks like one of going public with an IPO or being bought. Let's hope if it's the latter that it's not bought by a disk array maker that puts it out to pasture. 

Enhanced by Zemanta

BYOD backup: A looming Bring-Your-Own-Disaster?

Antony Adshead | No Comments
| More

Recently I blogged my thoughts on why mainstream backup products don't protect BYOD devices - laptops, tablets, smartphones etc - and came to the conclusions a) BYOD backup is a different beast to mainstream fixed source backup and is only provided by some specialised suppliers, and b) many BYOD users are probably using Dropbox et al.

However, it turns out I was off the mark, especially on the second point. In fact most organisations are not backing up these portable devices at all. That's the conclusion to be drawn from last spring's purchasing intentions survey. It found that more than two thirds of tablets and smartphones and nearly half of all laptops are not backed up (see chart below).

I banged on about potential BYOD compliance risks to this in my previous blog. That, you would have thought, would be sufficient impetus for users to rectify this situation. Maybe they want to. But, even if the desire is there enterprise and midrange backup products simply don't protect these types of device.

All of which leads to the conclusion that the backup suppliers really are missing a trick. It is quite literally a huge unfulfilled market. If the backup software makers had BYOD backup functionality in their products they could deploy one of IT marketing's greatest weapons - fear. So, why they don't remains a mystery.


Enhanced by Zemanta

Cold Storage, Helium and HAMR. Can they save the spinning disk HDD?

Antony Adshead | No Comments
| More

While super-fast flash storage has hogged the headlines, recent months have seen the available capacity of spinning disk HDDs increase to 6TB with the shipping since last September of HGST's helium-filled SAS and SATA 3.5" HelioSeal drives. This is a 50% increase on the previously available 4TB drives.

HGST has been able to do this because it has got a jump on its rivals by patenting a method of sealing helium into drives instead of air. Helium, famously, due to its ability to produce a funny voice when sucked from a party balloon, is about 1/7th the density of air.

This reduces friction against spinning components in the HDD, when they start up and as they run, and brings, says HGST - which thinks it has an 18 month/two year lead on the competition - a 36% decrease in power usage plus, crucially for capacity, the ability to run seven (thinner) platters in the drive rather than the usual five.

As if a 50% boost in capacity was not enough, we're looking at the possibility of HDDs shipping with between 7TB and 10TB (with helium) by the end of this year and into 2015.

That's down to the adoption of new ways of writing data to the surface of platters and consequent increases in areal density as the HDD makers move from the current standard of perpendicular magnetic recording (PMR) to the next generation shingled magnetic recording (SMR).

Then, two or three years down the road we're looking at a tripling of current HDD capacities to around 12TB (with helium) with heat assisted magnetic recording (HAMR), which does what it says on the tin really, by heating up the surface of the drive and increasing the density of its storage capabilities.

It all sounds like great news until you think of the RAID rebuild times. These can currently stretch to days for 4TB drives that use the parity-based RAID levels (5 and 6) and will only get worse with capacities that double or triple that.

"It's not good," says HGST's EMEA sales VP Nigel Edwards. "As capacities increase so will RAID rebuild times. It is an issue, but we are seeing huge demand and are being pushed for larger capacity drives."

According to Edwards the future of such ultra-high capacity HDDs is in "cold storage", ie archiving that sacrifices access times for ultra-low cost per TB. Here, if HDD makers can bring the cost per TB price of spinning disk down to that of tape, service providers will offer data archive services using vast amounts of disk drives that are spun up as customer access needs dictate.

It's a plausible case. And it'll be interesting to watch how it plays out. Because, as the HDD makers drive for ever-higher capacity disk the tape makers too - with a head start in terms of capacities/densities - are also looking at more archive-friendly technologies, such as LTFS and SpectraLogic's Black Pearl implementation.

Oft-heard soundbite used to proclaim "tape's is/isn't dead". Now it seems there's a current of "disk isn't dead" emerging and finding use cases to ensure its survival.

Why don't backup products protect mobile/BYOD?

Antony Adshead | No Comments
| More

It's a contemporary mystery. Why, when the trend towards ever-greater mobile device use is such a prominent one, do the mainstream backup software suppliers almost all fail to provide for such use cases?

You'd need to have lived in a cave (without a broadband connection) for two years not to have seen the rise of mobile, tablets, smartphones and the BYOD phenomenon. And even if you had dwelled in such a place you could still learn that, for example, of the top 5 IT projects planned by readers in 2014, three of them are "tablet PCs", "smartphones" and "mobility".

Of course, there's a huge need for these devices to be protected, and for a variety of reasons. They may contain data that's valuable in business terms or sensitive for data protection reasons. Meanwhile, many types of data held or generated on mobile devices can be the subject of legal e-discovery requests.

Businesses go to great lengths to ensure virtual and physical server data is protected and a relatively small number of backup software suppliers make a healthy living from it, but curiously, almost none of them includes mobile device support in their products.

You can, if buying enterprise or midrange backup software from Symantec, CommVault, EMC, IBM, HP, Acronis, CA or Microsoft, ensure various levels of granularity of backup with at least the two biggest virtual machine hypervisors, probably integrate with their management consoles and with numerous applications as well use the cloud as a backup target.

But one thing you won't be able to do with most of those products is to backup tablets, smartphones, and possibly not even laptops.

There are specialist suppliers, however, such as Druva, that do make specialised backup products for mobile and laptop devices. Meanwhile, midrange backup supplier Acronis has indicated it wants to travel in this direction with the purchase of file sharing and collaboration tool GroupLogic, but this is more a DropBox-type tool than backup. In this space too HP has its Connected Backup (that's not integrated with HP's Data Protector backup product).

There are also of course DropBox itself and Box and other file sharing and collaboration tools aimed at mobile users.

So, we have a yawning chasm. On the one side we have the backup products aimed at virtual and physical server estates and on the other we have some specialist mobile/laptop backup products and the file sharing tools.

That can't be good for users. It is surely preferable to be able to deal with backups for all devices from one product that covers fixed and mobile.

And you would think vendors are missing a trick too. After all, with the proliferation of mobile devices that's a huge potential pool of licence sales to be tapped.

Perhaps it boils down to the nature of backup and data protection in the two spheres. On the one hand larger SME and datacentre backup needs an application that can schedule, manage and monitor the movement of large amounts of data on a regular basis. Meanwhile, mobile device use patterns dictate more atomised, individual levels of service on an irregular basis and of relatively small amounts and simply don't require the need to deal with scale in the same way.

So, perhaps never the twain shall meet and the world of backup is destined to remain a fragmented one. For me, it's a puzzle. If you have any clues what's holding the world of mainstream backup products back from sweeping up all those new mobile/BYOD users then please feel free to comment.

Startup watch: Gridstore targets Hyper-V storage market

Antony Adshead | No Comments
| More

This week I spoke to Gridstore, a startup that offers storage arrays that integrate with the Microsoft Hyper-V virtualisation environment and with on-board smarts it claims boosts I/O for Hyper-V traffic.

Gridstore is delivered as a hardware appliance with its software configured on Dell servers it calls nodes. You can get two types of node - in a minimum of three - the H-Class high performance version with a mixture of PCIe flash and 12TB of spinning disk and the C-Class, aimed at less-demanding I/O applications that comes with spinning disk only. Building out these nodes can take you to a maximum capacity of around 3PB.

So far so box-standard array, but where the smarts come in is that Gridstore applies intelligence to the I/O queues. Normally, of course, virtualisation traffic creates the so-called I/O blender effect, which is when numerous virtual machines try to access disk and the potentially large number and random nature of the calls overwhelms the media.

What Gridstore does is to examine I/O coming in from different VMs and put it into queues and make that access sequential rather than random. The user can define the priority of access to storage for different VMs in a quality of service (QoS) gold, silver, bronze approach.

George Symons, CEO at Gridstore, says it "cleans up contention", and agrees its something like what Virsto (acquired by VMware in Feb 2013) does, but at the storage not the host.

What's also interesting is that Gridstore's controller software is integrated with Hyper-V at the level of its operating software, working like a driver specific to that OS, so can't be used with VMware.

"To be able to clean up I/O we need to operate at that level. When Hyper-V goes to write to the drive, we intercept it and deal with it. Hyper-V thinks it's dealing with a local drive," says Symons.

Due to this tight integration with Hyper-V, Gridstore can use advanced features such as data deduplication, thin provisioning, replication and snapshots in Windows Server 2012 that are usually the preserve of high-end storage hardware.

But being so tightly integrated into a specific hypervisor also means that a VMware ESX version is some way off, at least 12 months, says Symons.

The company (California-based) is making its first European push with the appointment of former NexSan man Andy Hill to VP for EMEA sales. The company is at B round in its funding cycle and late last year held a beta product programme of the block-based Hyper-V storage hardware with around 20 customers after switching its focus from scale-out NAS.

All of which means it's early days yet for Gridstore, but it's certainly bringing an interesting product to market, although tightly targeted at users of the, for now, minority hypervisor.

Pure Storage's Dietzen - we'll be up there with the storage giants

Antony Adshead | No Comments
| More

It's not often you get the CEO of a tech startup tell you his company is going to be a market leader, in the tens of $billions of revenue category. That's what flash array maker Pure Storage CEO Scott Dietzen told me this week, so I thought it worth recording. After all, achieving such an aim would put Pure alongside the likes of EMC, NetApp, IBM etc in storage revenues.

Currently, Pure Storage has around 400 employees and "hundreds" of customers worldwide. Those magnitudes place it firmly in the startups camp, but it does have a healthy balance sheet and prospects.

It has, for example, attracted possibly the biggest funding injection in enterprise storage history. Talk is that it is IPO-ready, on the basis of a valuation, "north of $1 billion", said Dietzen and it has seen several quarters of growth in the region of 50%. The company could have gone for an IPO but decided to stay private, said the CEO.

Why would it do that? A company in such rude health must be getting offers it can't refuse. And it must face questions from customers concerned it could be snapped up by big six player, or that it could have a less-than-successful IPO, like Violin did.

Dietzen said Pure had been approached by half a dozen potential buyers, but it has decided to hold out. He believes that Pure has the strength to stay the course to achieve a place alongside the giants of storage.

"Why? Our numbers are really good - we've been growing at 50% a quarter sequentially - and we have a desire to stay independent for the long term. All storage is going to shift to flash and if you're not growing at 40% to 50% a quarter you're not going to maintain market share there."

"I'm frequently called on to answer questions about why we're not entertaining M&A conversations. We're a private company that has shareholders to look after. But that said, our shareholders see that ultimately a market leader in the flash array space is worth tens of billions of dollars and that we are on track to achieve that."

"Ultimately, companies get to be as big as they deserve to be. If you do what you do as well as we have you are not available as an acquisition target. We've experienced huge growth and see the same in front of us. This is how market-leading companies get built; they're the ones that outpace the others."

And finally, Dietzen stuck the boot into hybrid flash arrays, which he characterised as a poor substitute for all-flash.

He said, "Hybrid flash is marketed as providing the performance of flash with the cost of disk. But often what you get is the performance of disk at the cost of flash. Disk is so much slower than flash so if 95% of I/O goes to flash and 5% to disk the overall performance will be 5x slower."

Whatever happened to Veeam's V-Index of virtualisation trends?

Antony Adshead | No Comments
| More

We live in the Information Age, apparently, but sometimes it's far too easy to form the impression that decent information is at best hard to come by and at worst purposely withheld.

This week I spoke to Veeam, the virtualisation backup software specialist. The briefing was mostly a recap of some of the new features announced in version 7 of the Veeam product earlier this month, including WAN acceleration, array-based replication and tape support.

Also discussed was Veeam support for the latest versions of the VMware vSphere and Microsoft Hyper-V hypervisors; a result, said product strategy specialist Mike Resseler, of the company closely following virtualisation platform trends.

Now, if I'd wanted to check the relative penetration of the various hypervisors and the overall percentages of physical to virtual servers there was a time I could have quickly accessed Veeam's own V-Index surveys, carried out by Vanson Bourne, of more than 500 organisations in the US, UK, France and Germany.

There I could have seen stats for numbers of virtualised servers vs physical, the degree of server consolidation resulting from virtualisation and the relative penetration into datacentres of the various virtualisation hypervisors.

Sadly, however, V-Index was a very short-lived programme, lasting only, it appears, for one iteration of the survey.

Naturally, my suspicious journalist's mind suspected the results were not what Veeam wanted to see; the V-Index, for example, showed only about 35% of servers in the UK were virtualised in the last quarter of 2011. And that might be an argument for not buying Veeam, which only backs up virtual servers, and instead looking at a product from one of the larger incumbents that backup virtual and physical devices.

Veeam's public relations company reassures me my suspicions are wide of the mark, however, and that the company decided to concentrate its efforts on its annual Data Protection Report and to leave the kind of reporting done by the V-Index to the likes of Gartner and IDC.

But, Veeam's Data Protection Report clearly doesn't give the same metrics at all and you try finding Gartner or IDC figures that give the same information as the V-Index did. Oh, I'm sure they exist, but in nothing like the easily accessible format of Veeam's creditable efforts, which potentially could have provided a great resource that combined regular snapshots into a picture of virtualisation trends over time.

And that was Veeam's aim. In that July 2011 blog post Veeam VP for product strategy and chief evangelist Doug Hazelman told us about V-Index and how "very excited" he was about it. But yet by the end of that year the programme appears to have bitten the dust.

So, there's just one more thing, as I hover near the door Colombo-like, why was V-Index such a great idea in July 2011 but ditched less than half a year later? I can't help thinking I was right to be suspicious.

Supplier hype-watch: Violin and DataCore at SNW Europe

Antony Adshead | No Comments
| More
IT trade events like SNW Europe this week are a great opportunity to study the techniques employed in IT suppliers' marketing messages.

Some use out-and-out distortions of commonly-understood technical terms.

Violin Memory, for example, loves to emphasise the second word in its name, ie "memory". A strapline it uses is, "Run business at the speed of memory" and we're asked to think not as storage admins but as "memory architects" using its "persistent memory architecture" etc etc.

But how does all that stack up? Memory traditionally is the media on the motherboard closest to the CPU where a portion of the data and the application reside during working operations. Now, Violin may produce some very fast all flash arrays but are we really talking "the speed of memory" here?

Its high-end 6000 Series, for example - "Primary storage at the speed of memory" - product specsheets don't distinguish between reads and writes and offer latencies of "under 250 μsec" for SLC variants and "under 500 μsec" for the MLC variants.

I asked Violin CTO Mick Bradley how they could call it "memory" when it doesn't appear to conform with the commonly understood meaning of memory either architecturally or in terms of performance. His reply was: "We call it memory because it's silicon and it is addressed in pages of 16K."

Hmmm, such a definition doesn't cut much ice, especially now there is flash storage that does operate at the speed of memory. An example of this so-called memory channel storage is Smart's UlltraDIMM, launched earlier this year. Such a product could claim to operate "at the speed of memory" with write latency of less than 5 μsec and being actually located in motherboard DIMM memory slots.

Meanwhile, others change the way they describe their product according to which way the wind is blowing.

Storage virtualisation software vendor DataCore is a great example here. At SNW Europe this week, DataCore EMEA solutions architect Christian Marczinke told us how the firm had been the pioneer of "software-defined storage".

Err, hang on there. DataCore's use of software-defined storage to describe its products is dates back less than nine months and is clearly a response to the use of the term by VMware with its overall software-defined datacentre push and EMC's ViPR hype.

In fact, until around a year ago DataCore referred to its product as a "storage hypervisor", clearly bending with the wind blown from VMware's direction. I dealt with that issue of nomenclature here.

Does all this quibbling over terminology matter? Well, on the one hand, it obviously matters to the likes of Violin and DataCore, who despite having great products, clearly feel the need to over-egg their puddings. And on the other hand to IT journalists it matters because it's our job to ensure customers get a clear view of what products actually are.

To be continued . . .


The VMworld ecosystem: Avatar's Pandora or Total Recall's Mars?

Antony Adshead | No Comments
| More

Here at VMworld Europe in Barcelona the term ecosystem is being thrown around with gay abandon. It's a lovely-sounding word. It evokes life, the planet, lush green rainforests, myriad plants and animals living in harmony etc etc.

IT vendors like to use it for those reasons and all its positive associations.

VMware is particularly keen on it, and it seems most apt. The layers of virtualisation they have laid onto physical servers are now being joined by levels of abstraction above the networks and storage infrastructures and into those hypervisor(s) they are gathering the intelligence to run nearly all aspects of the datacentre via ever fewer screens.

But stop for a second to think about what it means to step outside your ecosystem. Or alternatively, think about the movie Total Recall where the governor of Mars, Vilos Cohaagen, exercised his power through a total monopoly on breathable air.

Now, of course I'm not likening VMware's gathering of datacentre functionality to Cohaagen's tyranny, but look what happened when Cohaagen got sucked out of the safety of his ecosystem and onto the Martian surface.

Obviously this won't happen to you just because you deploy VMware in your datacentre, but there are good reasons to think deeply about what you're getting into.

Not least with storage, probably the area most affected by virtualisation. It accounts for something north of 50% of datacentre hardware costs and Gartner has predicted those rise by 600% upon virtualising your infrastructure. That's because packing lots of virtual servers into relatively few physical devices makes direct-attached storage unsuited to the massive and random I/O demands, and almost always means an upgrade to shared storage SAN or NAS arrays.

The day-to-day consequences of this are that storage will become more difficult to manage - masked by the VMware ecosystem - as it fills up more quickly, requires more rapid provisioning and generates ever more complex and potentially rapidly changing and easily broken dependencies and access profiles. And that's before we get to replication, mirroring, backup etc, all of which also presents a massively complex and dependency-heavy payload on the VM infrastructure.

All of which goes to show there's a downside to the concept of an ecosystem. VMware et al like to portray themselves as the Na'vi in Avatar, as guardians of their idyllic world. But the reality can end up more like Total Recall, where breathing the air is costly but stepping outside is even more difficult and dangerous.

For that reason it pays to exercise due diligence over the consequences of datacentre virtualisation, the likely costs and knock-on effects into storage and backup and to be sure to you have surveyed all the alternatives available in the market.

Simplivity converged storage converges with the hyperscale

Antony Adshead | No Comments
| More

If you could build a datacentre - and more importantly its contents - from scratch chances are it wouldn't look much like many of them do now. Technologies have come along, have served their purpose as an advance on what went before, but later become the next generation's roadblock to efficient operations.

Take the x86 server. It replaced the mainframe or RISC UNIX server. In comparison to them it was cheap; you could put one app on each and keep adding them. But then, of course we ended up with silos of under-used compute and storage. And latterly, to this was added shared storage - the SAN and NAS - that solved many problems but has challenges of its own.

How would the datacentre be designed if it was built from the ground-up now?

Well, there are two answers (at least) to that one. The first is to look at what the likes of Amazon, Google et al have done with so-called hyperscale compute and storage. This is where commodity servers and direct-attached storage are pooled on a massive scale with redundancy at the level of the entire compute/storage device rather than at the component level of enterprise computing.

The second answer (or at least one of them) is to look at the companies bringing so-called converged storage and compute to the market.

I spoke to one of them this week, Simplivity. This four-year-old startup has sold Omnicubes since early 2103. These are 20TB to 40TB capacity compute and storage nodes that can be clustered in pools that scale capacity, compute and availability as they grow, and all manageable from VMware's vCenter console.

Omnicubes are essentially a Dell server with two things added. First is a PCIe/FPGA hardware-accelerated "Data Virtualisation Engine" that sees data on ingest broken into 4KB to 8KB blocks, deduplicated, compressed and distributed across multiple nodes for data protection as well as being tiered between RAM, flash, HDD and the cloud.

Second is its operating system (OS), built from scratch to ensure data is dealt with at sufficient levels of granularity and with dedupe and compression built in plus its own global, parallel file system.

With all this, Simplivity claims in one fell swoop to have replaced product categories including the server, storage, backup, data deduplication, WAN optimisation and the cloud gateway.

And to some extent the claim rings true. By dealing with data in an optimum fashion from ingest onwards, parsing it in the most efficient way and distributing it according to what's most efficient and safe, it has come up with something like how you'd deal with data in the datacentre if you were to design its parts from scratch right now.

That's not to say it's without limitations. Currently Simplivity is only compatible with the VMware hypervisor, though KVM and Microsoft Hyper-V variants are planned. And it is of course a proprietary product, despite the essentially commodity hardware platform (except the acceleration card) it sits upon, and you might not put that on your wishlist of required attributes in the 2013 datacentre.

Still, it's an interesting development, and one that demonstrates a storage industry getting to grips with the hyperscale bar that the internet giants have set.

EMC VNX: New OS + PCIe Gen 3 = a flash array turning point

Antony Adshead | No Comments
| More

EMC's refresh of its VNX line of unified storage arrays is largely based on an almost complete re-write of its near 20-year-old Flare operating system. Flare was on its 32nd release and has been replaced with a new OS, MCx, but what exactly has changed under the bonnet?

In short, MCx has been developed to take advantage of Intel multi-core processors where Flare was completely unable to do so. In addition VNX controllers also now use the latest Gen 3 PCIe cards and so physical bandwidth is also hugely increased.

All this amounts to unified storage arrays with much-boosted capabilities when it comes to exploiting the speed of flash storage.

Flare was originally developed by Data General (acquired by EMC in 1999) for the Clariion brand of arrays, and like storage operating systems from all vendors way back when, was written for single-core processors.

When multi-core processors arrived Flare was rewritten to allocate different functions - eg, RAID, memory, data placement, data services - to different cores, but with one function assigned to one core it was easy to max out.

But at the time with spinning disk the norm there was no great need to overcome this bottleneck. That all changed with flash storage, however.

Flare as-was, simply couldn't support the I/O and latency requirements of flash and this was where EMC fell behind the flash startups that built systems from the ground up.

So, MCx was rewritten for multi-core processors - the new generation of VNX uses Intel Xeon 5600 CPUs - and all processing functions are parallelised across the 32 cores and EMC claims something like 40,000 IOPS per core and into the hundreds of thousands per controller up to 1m IOPS in the VNX 8000 with latency staying below 2 microseconds.

Another key advance is the use of Gen 3 PCIe cards.

While the processors could be a bottleneck in legacy arrays so could connectivity in and out of the array via PCIe. Gen 3 boosts bandwidth and lane count between the processor and the storage as well as front (Fibre Channel, Ethernet) and back end (SAS) port bandwidth.

All of which helps put the new VNXs in a similar performance ballpark to the all-flash vendors. According to EMC mid-tier business director, Sean Horne, that means customers can now buy a midrange VNX array and look at having enough headroom for four or five years of growth in virtual machine count and performance requirements.

What EMC is hoping is that customers who have been, rightly, impressed with the performance of flash arrays from the startups will now be able to get similar performance for their virtual environments from EMC.

It certainly puts EMC at the forefront of the big six storage vendors; re-engineering an existing array family for flash, rather than throw flash at legacy OS and controller hardware. And for that reason it is a turning point in the flash storage story.

Enhanced by Zemanta

Storage virtualisation vs software-defined storage

Antony Adshead | No Comments
| More

The aim of this blog post is to try to iron out some misunderstandings in two common terms in storage. Two terms that are actually really rather connected; storage virtualisation and software-defined storage.

First let's deal with storage virtualisation. Here at we're pretty certain that there's a good deal of confusion about this term. In our yearly poll of readers we keep hearing that "storage virtualisation" is a key priority on IT department to-do lists for the coming year. This year that figure was 36%.

That figure seems unusually high. It's an un-scientific measure, for sure, but as a storage journalist I get a fairly good idea of the type of projects that are hot by what comes across my desk, by speaking to customers, and to vendors, and I just don't hear much about storage virtualisation.

So, when those questioned in our poll ticked "storage virtualisation", what many probably thought we were asking was "is storage for virtualisation a priority?" Why? Because server and desktop virtualisation is a big priority for a lot of organisations right now and implementing storage and backup to support it is a key part of that process.

Meanwhile, storage virtualisation products allow organisations to build storage infrastructure from multiple vendors' hardware. Storage suppliers, of course, would prefer that they provided all of your disk systems. Consequently, while the key storage vendors have storage virtualisation products, it's not something they push particularly hard in marketing or sales terms.

Storage virtualisation products include EMC's VPLEX, IBM's SAN Volume Controller (SVC), NetApp's V-Series and Hitachi's VSP.

There are also the smaller storage virtualisation vendors and products, such as DataCore's SANsymphony, Seanodes' SIS, FalconStor's NSS and Caringo's CAStor.

These are all reasonably well-established products that allow users to create single pools of storage by abstracting the physical devices upon which they are layered to create a virtual storage array.

More recently, we've seen that capability emerge in the form of products at a higher, environment level.

Here, I have in mind, for example, VMware's plans for Virtual SAN, which will allow pooling, from the hypervisor, of server-attached disk drives, with advanced VMware feature integration, such as high availability and vMotion. It will scale to petabyte levels of capacity and will put some pressure on existing storage vendors playing in the SME up to small enterprise levels when it come to fruition.

And there is EMC's ViPR environment, announced at EMC World 2013, which merges storage virtualisation with big data analytics. Key to this discussion is ViPR's ability to pool storage from direct-attached hard drives, commodity hardware and other vendors' arrays into one single reservoir of storage that's manageable from a single screen.

These initiatives contain a large dose of what has for a long time been called storage virtualisation but are billed as software-defined storage.

So, to what extent are either of these terms accurate reflections of the technology they represent?

Well, of course both terms could be said to be so vague to be almost meaningless. After all, all storage is based on the retention of data on a physical drive, but that would be nothing without software that abstracted/virtualised for example, blocks and files to physical media, RAID groups and LUNs. In other words storage never exists without being defined by software or being virtualised in some sense.

So, how do we make sure we're using these terms clearly? Well, on the one hand it seems reasonable that storage virtualisation should refer to the abstracting of multiple vendor systems into a singly-manageable pool of storage. If there's anything such as historical usage in storage and IT then those systems ranging from IBM's SVC to the likes of DataCore seem to fit that billing and have done for some time.

Meanwhile, while we can recognise that VMware's planned Virtual SANs and EMC's ViPR are heavily based on storage virtualisation capability as defined here, they also go beyond this, to incorporate much higher level features than simple storage functionality, such as vMotion and big data analytics respectively.

Despite the efforts of some vendors, notably DataCore, which has gone from dubbing its products a "storage hypervisor" to software-defined storage according to the whims of the IT marketing universe, it seems reasonable to define storage virtualisation as quite narrowly as centring on the ability to pool heterogeneous media into a single storage pool.

Meanwhile, software-defined storage can be reserved for higher level function and environment-type products that also include storage virtualisation.

It's always a battle to define terms in an area so fast moving as IT, and with multiple vested interests and active marketing departments, but it's certainly valid to try and define terms clearly so the customer is able to see what they're getting.


Enhanced by Zemanta

Atomic Writes at the leading edge of flash in the datacentre

Antony Adshead | No Comments
| More

A rather tiny bit of storage news this week illustrates the changes taking place as part of the flash revolution, and also where its leading edge lies.

The news is that Fusion-io has submitted proposals for standardised APIs for Atomic writes to the T10 SCSI Storage Interfaces Technical Committee.

Why this is interesting is that it's is all about the interface between flash memory/storage and some of the most business critical database apps.

Atomic operations are database operations where, for example, there are multiple facets to a single piece of information and you either want all or none of them read/written. Completing only one part is highly undesirable, such as a query for credit and debit in a banking system.

Until now, with spinning disk hard drives, supporting MySQL, for example, because of the possibility of disk drive failures writes took place twice before acknowledgement, as a failsafe. Clearly, such a doubling of operations, is not optimum in terms of efficiency.

What Fusion-io has done is to eliminate that duplication of effort with APIs that build in management of Atomic operations to flash memory.

The flash pioneer claims its Atomic Writes capability provides performance throughput increases of up to 50%, as well as a 4x reduction in latency spikes, compared to running the databases on the same flash memory platform without it.

Gary Orenstein, marketing VP, said: "The background is that many have adopted flash as a direct replacement for the HDD. But Fusion-io believes flash should do more than that and that we should be moving away from the last vestiges of mechanical operations in the datacentre."

"What we're looking at are new ways to break new ground that are faster, with fewer instructions," he added.

Currently these capabilities only come with Fusion-io flash products and are already supported in MariaDB and Percona MySQL distributions but upon T10 standardisation they will be open to all vendors.

Stepping back to take a big-picture view what this also illustrates is the contrast between the extremes of flash implementation in the datacentre.

One the one hand there is this type of work at the leading edge of flash storage use, squeezing ever-greater efficiencies from the interface between flash and the apps/OSs etc that it works with by use of software.

At the other there are the legacy arrays into which flash drives act as a pretty much dumb replacement for the spinning disk HDD.

Enhanced by Zemanta

Has the storage hardware lock-in had its day?

Antony Adshead | No Comments
| More

Two years ago I wrote that the storage industry was apparently ripe for huge change.

The nub of my argument was that storage is a sector of the IT supplier world in which customers are forced to spend money on what are essentially a commodity - ie, drives - wrapped in proprietary software built into hardware controllers.

The argument progressed to take note of the revolution in the server world that had occurred as Linux effectively decoupled proprietary operating systems from RISC chip-based hardware in the previous decade, making open source OSs on x86 commodity hardware a much cheaper option.

The conclusion of the piece looked around at the likely candidates in the world of storage that might do what Linux did in the server world. These comprised storage software that could be deployed on commodity hardware and included GreenBytes and Nexenta as well as open source products such as Red Hat's and ZFS.

Two years on and it seems the hazy predictions based on a theory and a few small shoots of evidence have been validated by, among others, the biggest name in storage.

This week I spoke with Ranga Rangachari, VP and general manager for storage with Red Hat (not the biggest name in storage), who put forward a similar argument to the above, namely that: "Storage is dominated by 'tin-wrapped software' and customers are sick and tired of being locked into silos, with for eg, vendors with three different solutions."

Rangachari reiterated the argument that what happened in the move from RISC to x86 could happen with storage and that the drivers now are the cloud, the volume of unstructured data and the rise of online analytics platforms such as Hadoop, which requires co-resident storage and processing power, with data moving, as Rangachari put it "East to west not north to south" as in existing server-SAN infrastructures.

The rise of such hyperscale server/storage infrastructures has been pioneered by the likes of Google and Facebook and is exhibit A in the rise of architectures that challenge the existing enterprise storage paradigm.

Instead of shared, but remote, storage in, say, an enterprise SAN, these highly performant Web-serving and analytics stacks comprise converged server and storage hardware, all made of cheap commodity parts with redundancy at the level of the whole unit rather than components within.

Elsewhere - exhibit B - is the emergence of converged storage/server products that ape the hyperscale architectures and are usually geared towards virtual environments. These include Nutanix, Scale Computing and Simplivity.

Exhibit C is the continued rise of software-only storage products that customers can run on any hardware. Virtual storage appliances that will run on virtual or physical machines are available from all the big storage vendors as well as the likes of DataCore, Nexenta.

An important addendum to exhibit C is the plan by VMware to include storage software features in its virtualisation hypervisor products. VMware already has a virtual storage appliance, but it plans to include storage software capability in the form of its Virtual SAN which will allow users to create up to petabytes of capacity from existing unused disk. This threatens to seriously undermine the market of entry-level to midrange storage players.

Finally there is exhibit D - evidence for the prosecution, as it were - and this is EMC's recent announcement of its forthcoming ViPR storage virtualisation/private cloud/big data software layer.

The 800lb gorilla of the storage market justified ViPR as a response to a changing storage landscape, and is in large part a storage virtualisation platform that will knit together disparate storage systems from any vendor and from commodity drives.

On the surface of things it's quite remarkable that the biggest disk system vendor should potentially allow users to create storage from any other storage supplier. But, ViPR can give EMC very sharp and well-barbed hooks in a user's environment, as a software layer that embraces all storage underneath it.

Maybe it should have been called Python, for its ability to smother an organisation's storage systems, and is apparently the antithesis of the move to more openness in systems that I'm arguing is a trend here. So, why it is evidence for my case?

Because it is a recognition by EMC that storage will henceforth no longer solely reside on the enterprise storage array as such; that it will be distributed in traditional storage environments, converged hyperscale datacentres and edge devices and that these must be linked by a software layer that virtualises the capacity underneath it.

So, it seems the biggest player in the storage market has recognised that the dominance of the traditional storage array is a thing of the past. Having made that concession it will be interesting to watch whether the likes of EMC can transition to the new reality against its rivals that offer more open storage software.

Let's check back in another couple of years.

Forget MLC vs SLC; it's going to be TLC vs MLC soon

Antony Adshead | No Comments
| More

The NAND flash memory scene is already an alphabet soup, with SLC, MLC, eMLC and now TLC to contend with.

Flash is flavour of the month/year in enterprise storage, because of its ability to rapidly deliver the likes of virtual desktops and servers, as well as processing high-performance transactional databases.

You may have recently got to grips with the distinctions between MLC and SLC. In fact, we know that many of you have because our explainers on MLC vs SLC are among our most-read pages month after month.

That may be about to change, however, as the flash market evolves.

Namely, SLC seems to be all set to effectively fade from the flash acronym lexicon, while TLC enters it.

SLC - or single level cell - is the best-performing and most durable of the NAND flash types. It's also the most expensive per GB. And while many flash storage system vendors offer SLC, take-up seems to be far slower.

That is, admittedly, from the decidedly non-scientific viewpoint of a storage journalist to whom vendors are keen to trumpet customer wins. But what I see on a regular basis is the use of MLC/eMLC flash, which has had its shortcomings addressed by clever software error correction etc.

Meanwhile, there is evidence that TLC - triple level cell flash - is creeping up as an enterprise flash option.  What's the evidence?

Samsung Semiconductor launched TLC-based flash products late last year. And speaking to the CEO flash array maker Pure Storage, Scott Dietzen, last week, he indicated it was only a matter of time (or more precisely, cost) until TLC makes an impression on the enterprise storage market.

The read latency of TLC is now nearly as good as MLC. Samsung pitches its TLC products for heavily read-intensive use cases, such as streaming media, for example. Dietzen expects TLC and MLC to be used in a tiered fashion in enterprise storage when the price of the former reaches two thirds of the latter.

That might not be too long. A quick look at flash market analyst sites such as the Taiwanese inSpectrum show the contract price for 128GB of MLC at an average of $8.72 while the same capacity TLC is about 75% of that at $6.60.

As the proportion of TLC flash manufacturing increases that price will decrease. Perhaps we'll see that 66% hit this year and TLC-based storage products emerge.

I think it's time to get writing that TLC vs MLC article.

Enhanced by Zemanta

Egnyte: 5G the tipping point for cloud storage

Antony Adshead | No Comments
| More

What are the limits of cloud storage right now? We've examined it elsewhere. And you wouldn't necessarily ask the CEO of a cloud storage service provider that question. They have too many reasons to come a little fast and loose with the facts, purely in their commercial interest, of course.

But this week I asked those questions of the CEO of Egnyte, a US-headquartered cloud storage provider that focuses on providing file sharing and synchronisation that is breaking into Europe.

Egnyte has two US datacentres and one in Amsterdam and holds about 12PB of customer data in a hyperscale storage environment; ie Super Micro server chassis with direct-attached storage on 4TB commodity drives. It's all held together by a home-grown object storage file system with redundancy at server level rather than that of the components within. Added to this is a dash of Fusion-io and Intel PCIe flash for rapid caching of customer data.

Egnyte offers cloud storage to its customers, with data kept in its datacentres plus access to Amazon, Google, Microsoft and NetApp clouds.

It also offers customers a hybrid of on-site storage alongside the cloud and herein is the recognition that for most types of production data the cloud is simply not yet ready. That's because network latency is still too great for access to data to be swift enough for the most business-critical applications.

So, when will the cloud really break through as an option for production data storage?

Egnyte CEO Vineet Jain sees a tipping point when 5G mobile networks are established.

Jain said: "Today 47% of our users access Egnyte by mobile, and currently we have 4G networks that have a maximum of 100Mbps bandwidth. That's nowhere near what's needed, but 5G is expected to be 1,000 times faster than that. Until then the cloud will be good for some things but it will be hybrid [ie, with disk storage at the customer site] until bandwidth is reliably available with no chokepoints."

Of course, reliable bandwidth isn't the only obstacle to cloud adoption. Security and compliance are the other key concerns, which, says Jain, could be overcome if businesses think realistically about what the cloud is good for.

"Like the mythical paperless office, there's been too much cloud hype," he says. "There will be an increasing amount of data put into the cloud, but we'll see it skewed towards that large proportion of data that businesses must keep but is infrequently accessed."

It's good to hear a realistic view of the cloud. And it'll be interesting to watch how cloud develops over coming years. Ultimately, the onset of usable could storage could shake up the entire storage industry as we know it, with the current incumbent vendors needing to adapt to survive as hyperscale storage-driven service providers offer increasingly usable remote storage services. But that's a musing for another blog sometime.

Enhanced by Zemanta

NetApp's FlashRay; an OS waiting for some hardware?

Antony Adshead | No Comments
| More

It's been an interesting week for NetApp-watchers. On Tuesday we learned of the company's latest moves in the flash array space; the EF540 flash array and FlashRay, a new operating system (OS) that's optimised for flash. All of which screamed (to investors and potential customers), "We have a flash array too now! You can get it from us, not the upstart startups or our larger competitors."

Having said that, there's no doubt these represent progress for NetApp, which has had an odd relationship with solid state. For a long time NetApp's stance was that cache was the only place for flash, and it would not form a distinct tier in your storage infrastructure.

Tuesday's announcements are the manifestation of a Damascene conversion on the flash question for NetApp that had taken place over the past year. Now it seems flash is built firmly into the future of the company. Well, reasonably firmly; the EF540 actually looks like a rush to get something to market to position NetApp against the competition.

Let's deal with this first. Late last year there was some confusion at NetApp towers about whether the company would develop and all-flash array at all. First, CTO Jay Kidd told me NetApp wouldn't be playing in the all-flash market because it wasn't a big market and there were other ways to introduce flash into the server-storage infrastructure.

After that article was published you could hear the sound of NetApp backpedalling from several thousand miles away. I don't know whether Kidd was off-message, still expounding the no-flash-tier mantra, or simply had demonstrated some extreme failure to communicate properly.

Still, things were put back on-message three weeks later in this interview with our sister publication, where Kidd said NetApp had plans for an all-flash array in 2013. Although, why Kidd didn't mention the EF540 in November is a mystery, especially as it had been sold in a limited release prior to that.

Such twists and turns aside, questions must be raised about the EF540. Sure, it's an all-flash array, and it can give up to 300,000 IOPS, but it uses the existing E-Series operating system, SANtricity, which is not built for flash and neither is the controller hardware built for flash.

Is this a problem? More than likely not in the short-term. NetApp's all-flash start-up competition, like Violin Memory and Whiptail boast huge IOPS, into the millions, which is probably silly and un-necessary for all but the most extreme use cases on the planet. I mean, with virtual desktop I/O requirements of something between 10 and 200 IOPS per seat the EF540 has plenty to give.

But in the longer term the EF540 must have limited life. Leaving aside whether it relies on throwing sheer TB at getting the throughput it does, its OS and its controller hardware are not built for flash. The OS doesn't optimise operations for the vagaries of flash and its wear characteristics and the controller hardware/backplanes etc are not built for the speed of flash.

That NetApp has announced the flash-optimised FlashRay is a tacit admission of those points.

But what is FlashRay and what will be its significance for NetApp? Well, facts about FlashRay were thin on the ground in Tuesday's announcements, but it appears to be a flash-optimised storage operating system. And apart from the flash-optimisation bit it sounds almost exactly like NetApp's existing OS, Data ONTAP. Indeed, Lawrence James, UK products, alliances and solutions manager for NetApp, told me that FlashRay is "ground-up developed, but will inherit features" from ONTAP. Hmmm.

Leaving that bit of speculation aside, NetApp's launch of FlashRay has potential implications for its existing storage hardware range. FlashRay undoubtedly represents a progressive move by NetApp, but what hardware will it be allied with? After all, the new OS may well be flash-optimised, but the controller hardware on the FAS and E-Series families are not. So, will we see a new family of NetApp arrays, or will FlashRay be ported to FAS arrays with upgraded hardware?

They are interesting times indeed for NetApp watchers.