What you should know about flash memory in storage


What you should know about flash memory in storage

Chris Mellor, Contributor
IBM recently released a record-breaking benchmark result. An IBM Power 595 server, using 48 of its 62 cores and 84 STEC ZeusIOPS flash solid-state drives (SSD) achieved 300,993.85 SPC-1 IOPS. There was 5.8 TB of flash memory in this setup and the total system cost was $3.2m. That means each SPC-1 IOPS cost $10.77.

That's impressive, but it's a very expensive way to get 300,000 IOPS.

Compellent, meanwhile, puts STEC ZeusIOPS flash drives at the top of a storage pyramid. There would be, say, three tiers of storage in such a SAN pyramid: a small amount of the fastest-performing SSDs at the top; then mid-level performance and capacity SAS drives, and; bulk capacity and relatively low-cost SATA drives forming the base of the pyramid. Compellent has submitted its storage array to an SPC-1 benchmark but we can be pretty sure it wouldn't come in at 300,000 IOPS mark; possibly it would rate somewhere around the 100,000 IOPS to- 150,000 IOPS level.

The benchmark results we need to see are for tiered storage arrays equipped with tier zero flash, tier one Fibre Channel or SAS drives, and tier 2 SATA
In the Compellent scheme of things not all IOs are equal and some go repeatedly to the same small set of data, the most highly active. So, they put this in the SSD storage, where it gets the lowest-latency response while the least active data, which will be infrequently accessed is located on the SATA drives, with medium-access level data in between. The array software tracks block-level access rates and automatically moves blocks between the tiers of storage to maximise performance on the one hand and capacity and cost on the other.

In Compellent's case, SSD is being used surgically, like a scalpel; precisely and positioned accurately for the small proportion of the multi-terabytes of data stored that actually needs lightning-fast access.

IBM, on the other hand, uses its flash like a sledgehammer. Assuming a dataset that needs fast access to any item, it wields a 5.8 TB flash hammer and blows IOPS bottlenecks away. Every I/O is equal and they all need accelerating in that way of thinking. Of course, there is a cost - $10.77 per IOPS, in fact.

This is the method espoused by in-memory database device suppliers such as Texas Memory Systems, with its RamSan DRAM solid state systems. They don't use the SSD as a tiny store for identified hot data, but instead put the whole database into RAM because you don't know which is the hot data. And, even if you could work it out, moving it between SSDs and hard drives would take too long and the opportunity for a currency price arbitrage deal you're working on will have passed.

There is another way of getting 300,000 IOPS at a third of the price. Take a Xiotech Emprise 5000, the storage array that uses sealed canisters of drives, and buy 51 of them. One 5000 with twenty 146 GB hard drives runs at 5,892.6 IOPS. The per-IOPS cost is $3.53 and the total system cost $20,820. Fifty-one will get you 300,522.6 IOPS and cost $1,049,682. The total system will store 148,920 GB, around 149 TB in total. Of course, it will occupy more space than the 5.8 TB Power 595 and consume more power but you save over $2 million on the purchase price.

Traditionally storage vendors have SPC-1 tested homogenous arrays, those using only one type of hard drive. For example, HP achieved 200,245.73 IOPS with $3,525,389 worth of XP2400 fitted with 1,024 147 GB disk drives (total 151 TB) at a cost of $17.96 per IOPS. The IBM result stands up well compared with this but is it the right way to use flash? Perhaps a TPC benchmark would be more appropriate for a system where all the data is held in flash memory? Indeed, IBM tops the TPC rankings with a Power 595 server result of 6,085,166 tpmC at a cost of $2.81/tpmC.

A tiered storage array with flash at its apex would deliver a much better balance of capacity, performance and value than flash sledgehammer systems devised to exert brute storage performance force on all I/Os. But, it seems neither SPEC or TPC benchmarks have been designed to take this into account. Submitting a tiered storage array with flash at its tip into a benchmark designed to get the most raw performance is not ideal.

We can use the per-IOPS or per-tpmC cost, however, to compare how cost-effective different systems are. In this sense, the 300,000 Power 595 SPC-1 result, at $10.77 per IOPS, is three times more expensive than using a Xiotech Emprise 5000. One commentator said to me of the IBM result: "It's not only high, it destroys the entire value proposition for SSD. SSD is equated as high value today precisely because [it has] superior $/IOPS. Nothing else matters. Absolute I/O is one thing, dollars per I/O is quite another. This is why SPC-1 is so valuable; it normalizes results between disparate systems... This result - over $10 - is horrible."

That's a comment from the use-flash-as-a-scalpel viewpoint. Texas Memory Systems has built its entire RamSan business on putting all the low-latency access data needed into solid state. It fits into the in-memory database paradigm, and would have something different to say, extolling the virtues of absolute I/O. Where the opportunity cost of missing a transaction is high then the absolute cost of putting an entire dataset into memory can also be high but still be reasonable in the circumstances. Texas Memory Systems wouldn't exist if that were not the case.

But, for the vast majority of storage applications having the full dataset in memory or flash is going to be a ridiculous waste of money. For them, the Power 595 flash result is overkill. Yes it is very, very fast, but so is a Lamborghini Diabolo. So what? I don't need a supercar when I'm delivering small goods and that's why there are millions of white vans on the roads and probably less than 500 Diabolos.

The IBM STEC flash result isn't a disk drive array storage killer; it's not even a server direct-attached disk storage killer. It can eat into DRAM SSD sales, as 5.8 TB of flash is cheaper than 5.8 TB of DRAM - but that's all. The real benchmark results we need to see are for tiered storage arrays equipped with tier zero flash, tier one Fibre Channel or SAS, and tier 2 SATA drives, and a measure balancing raw IOPS to capacity and cost.

It doesn't help that EMC doesn't submit its storage arrays to SPEC benchmarks and that means there is a huge blind spot in comparability. For the moment we can say that comparing a Power 595 flash-equipped server with a tiered storage array with some SSDs in it is inappropriate. They are different tools for different jobs. Sledgehammers and scalpels each have their places in the world, they have their roles to play, but not in the same toolset.

Chris Mellor is storage editor with The Register.

Email Alerts

Register now to receive ComputerWeekly.com IT-related news, guides and more, delivered to your inbox.
By submitting your personal information, you agree to receive emails regarding relevant products and special offers from TechTarget and its partners. You also agree that your personal information may be transferred and processed in the United States, and that you have read and agree to the Terms of Use and the Privacy Policy.

COMMENTS powered by Disqus  //  Commenting policy