This article is part of our Essential Guide: A guide to flash storage

Cern cuts latency in move from NetApp PCIe flash cards to drives

Large Hadron Collider operator Cern used PCIE flash cards for databases but found their limits and deployed flash drives in NetApp filers

Cern, the European Organisation for Nuclear Research and home of the Large Hadron Collider (pictured), has achieved sub-millisecond latency and cut disk purchases by 50% following the deployment of flash storage in its estate of clustered NetApp filers.

It initially made gains across its 2PB of NetApp clusters by adding FlashCache PCIe flash cards – but then increasingly by adding flash drives to its arrays.

The research establishment near Geneva first deployed flash in PCIe format in 2010 on NetApp clusters that run Oracle databases for engineering, control and management data on the Large Hadron Collider.

Its NetApp clusters – running Clustered ONTAP 8.2.2P1 – comprise 14 nodes of FAS6220 filers, plus four nodes of FAS6220, another with two nodes of FAS6220 for development and test, four nodes of FAS8060, one with two FAS6250 for backup data and a four-node FAS8040 cluster in Hungary for disaster recovery.

Cern started using PCIe flash to accelerate some database applications. By providing faster access times to hot data, it meant Cern could use fewer HDDs. It had previously added spinning disk HDD capacity to boost I/O performance – a practice known as short-stroking, where disks are added to speed access times to data but not necessarily for capacity.

Rapidly growing databases

Group leader for database services, Eric Grancher, said: “Data growth had been very fast with some databases growing by 100TB a year. We were adding more spindles to get capacity but also to maintain performance.”

So, Grancher’s team added NetApp FlashCache PCIe flash to its filers to cut latency on key applications. PCIe flash – usually added to servers but also to arrays in some cases – is a key method of boosting performance by adding solid state storage. But, Cern soon reached the limits of what PCIe flash could offer, said Grancher.

He said: “We moved to flash drives from flash cache as it is more extensible – you can add to it more easily. PCIe flash is limited; there just aren’t the slots available.”

Reduced latency

So, in 2012 Grancher started to add NetApp FlashPool flash drives to the filers’ disk shelves. These are standard format flash drives, but allowed Cern to achieve sub-millisecond latency on database operations, said Grancher.

He said: “One database on the 14-node cluster has an I/O profile with >71% random 8kb IOPS completed in less than 1ms, which is very good for random I/O.”

Summing up the benefits of moving to flash, Grancher said: “It’s all about latency – we’re getting sub-millisecond performance from flash and that reduces the number of spindles.”

Spinning disk applications

But not all operations need flash, said Grancher. “Some volumes have flash and others don’t. For example, we don’t need flash for log data, as it is sequential and there’s no real gain in flash over HDDs.”

That’s because flash excels at random I/O while spinning disk HDDs show their shortcomings during such operations. Conversely, however, spinning disk – with its mechanical seek heads – operate well when data is read sequentially, rather than from all over the platter.

In the Cern databases cluster flash is used as cache. That is, hot data is moved automatically to flash while a copy also resides in bulk SATA storage on the filers.

Read more on SAN, NAS, solid state, RAID