Gordon Moore observed in 1965 that transistor density on integrated circuits doubled every two years, and since then we have seen massive increases in processor and memory speeds.
But storage has failed to keep up with the memory and processors following Moore's Law. Even flash storage is failing to deliver the performance demanded by certain applications.
A new solution, memory channel storage, looks to solve this performance problem by bringing compute and storage even closer together.
What is memory channel storage?
Memory channel storage is a form of non-volatile DIMM (NVDIMM) memory that retains its contents when powered down and delivers low latency, high-performance persistent storage directly on the processor bus.
NVDIMMs based on DRAM have existed for some time and have used either a backup battery unit or large capacitors to retain their contents when power is lost. By contrast, memory channel storage uses NAND flash deployed in SSDs and PCIe SSD devices, which results in persistent data retention without the need for additional backup components.
Memory channel storage benefits
Memory channel storage puts persistent storage closer to the processor by placing data directly on the processor memory bus and bypassing the PCIe bus and any associated storage protocols used to control solid state drives.
The result is the achievement of latency figures as low as five microseconds for write I/O. This figure is at least three times faster than the fastest PCIe SSD products on the market today and an order of magnitude faster than solid state drives.
More on memory channel storage
- Solid-state memory channel storage takes on latency issues
- IBM rebrands SanDisk and Diablo's ULLtraDIMM memory channel storage
- Smart and Diablo launch Ulltra DIMM memory channel storage
- Comparing DIMM-connected flash vs. PCIe flash benefits
- DIMM-connected flash further blurs the line between storage and memory
- Smart Storage goes DIMM with memory channel storage
Device capacities are comparable with those of SSD flash drives with 200GB and 400GB products available today, and some predict this will double in the near future.
When deployed in servers with many DIMM slots, memory channel storage can deliver terabytes of storage capacity, in many cases more than could be achieved with performance-focused PCIe devices.
Memory channel storage performance scales with the number of devices deployed on the server, so as more devices are added, the performance per GB of storage deployed remains constant. Power consumption of memory channel storage devices is typically around 12W per module, around half that of similar PCIe SSD devices.
Memory channel storage disadvantages
Like PCIe and some SSD flash drives, memory channel storage is installed inside the server, which results in a number of issues.
Firstly, data on the devices is isolated from direct access by other servers. So, although data is persistent on the NAND cells, some additional data protection process is required to mitigate module or server failure in the same way as PCIe SSD devices must be managed.
Second, the server needs to be powered down when failing modules need to be replaced or to expand capacity. This reduces the availability of systems unless some form of clustering is used.
Because memory channel storage devices are based on flash, they have the same lifetime wear characteristics as flash drives
Also, because memory channel storage is deployed into slots normally reserved for DRAM, BIOS modifications are required for the server to correctly identify memory channel storage as solid-state storage and not DRAM memory.
At the operating system level, driver software is needed to ensure memory channel storage devices are correctly identified to the system. This means memory channel storage devices can’t simply be plugged into existing servers and expected to work seamlessly in the same way flash drives would.
Because memory channel storage devices are based on flash, they have the same lifetime wear characteristics as flash drives. The UltraDIMM product from Smart Storage Systems and Diablo Technologies, for example, uses endurance-enhancing features such as Smart Storage’s Flash Guardian technology to improve product lifetime.
Memory channel storage use cases
Memory channel storage will provide another tier of storage as well as spinning disk and flash storage. The low latency and consistent performance characteristics of memory channel storage means the technology is suited to environments where application response time is critical, such as high frequency financial trading systems.
Memory channel storage is also suited to other applications that use in-memory techniques to process data, but have to regularly checkpoint to external storage – for example high-performance databases or big data solutions.
For virtual servers, memory channel storage will be applicable to hyper-converged platforms
Because memory channel storage is persistent, there is less need to write to external storage to track transaction updates. Some code changes may be required from the application and database suppliers to obtain the best performance from memory channel storage-enabled systems.
Memory channel storage devices are also a good fit with virtualisation for VDI and virtual server implementations. For virtual servers, memory channel storage will be applicable to hyper-converged platforms, such as those from Nutanix and SimpliVity, to improve the performance of systems that use PCIe SSD today.
Memory channel storage devices will also find beneficial use cases in software-defined storage deployments, with products such as Maxta MxSP, EMC’s ScaleIO, HP StoreVirtual and Nexenta’s NexentaStor. It’s possible to envisage designs that use commodity servers and memory channel storage to create scale-out storage architectures.
Memory channel storage market view
Smart Storage Systems (now part of SanDisk) offers memory channel storage under the UltraDIMM brand name. The company claims performance figures of 1000Mbps for sequential reads and 760Mbps for writes with 150,000 and 65,000 IOPS respectively. This is achieved at less than five microseconds of write latency and around 150 microseconds for read.
IBM has been the first server supplier to offer memory channel storage in its servers. eXFlash (pictured) is based on technology from the Diablo Technologies and Smart Storage Systems partnership and is available in either 200GB or 400GB modules for use in their latest X6 server range.
The UltraDIMM and eXFlash products claim endurance levels of 10 complete module writes per day
IBM claims eXFlash performance figures as low as 3.3 microseconds for writes and throughput up to 515MBps with 43,000 (4KB) IOPS per device. Read operations are a little slower at 144 microseconds and performance of 645MBps or 125,000 4KB IOPS.
Both the UltraDIMM and eXFlash products claim endurance levels of 10 complete module writes per day and an unrecoverable bit error rate of 1017. This means memory channel storage devices fare well in terms of reliability compared to traditional hard disks.
Smart Storage Systems and Diablo are not the only companies working on NVDIMM products. Viking Technology has released the snappily named ArxCis-NV module, which uses NAND flash as backup for DRAM. This has the benefit of much better write endurance as flash is only ever written periodically. However, the product still needs some form of capacitor to save data when power is lost.
Many companies are signing up to the SNIA NVDIMM Special Interest Group, so we can expect to see many more products in this category brought to market in the coming months.