The undeniable move in storage is through to the use of flash-based systems. Some companies, such as Pure Storage and Violin Memory are pushing complete flash-based arrays, whilst others, including the big guys of EMC, NetApp and IBM are currently working with tiered systems of spinning magnetic disks with flash-based disks for dealing with "hot" data - i.e. the data that is most often used.
The arguments for going for flash memory are apparent - no moving parts, lower thermal output, lower energy requirements, dramatically faster storage operations. However, it has long been known that flash memory works in a different way to magnetic hard disks, and unless the different operation modes of flash are understood, any move to flash based storage for enterprise storage could be a major mistake. The failure of a magnetic hard disk can be caused by mechanical failure, and will generally be catastrophic - for example, a disk head hitting the platter. Standardised approaches to dealing with this, such as through RAID or, in extremis, backup and restore will allow a hard disk to be replaced. General wear and tear of a magnetic disk will lead to SMART systems being able to monitor increases in sector failure, increases in energy being drawn or temperature increases at the disk casing surface and advise in advance that a disk should be replaced.
Flash, however behaves differently. In essence, flash has a pretty well defined life - the cells involved in storing the data can only be written and erased a certain number of times before they fail. The perception has often been that this means that flash array failure will be predictable - but will also be pretty dramatic over a short period of time.
However, let's just look back a little at flash as a memory system. The first flash memory was developed by Toshiba in 1980. It made its commercial debut in 1988 through Intel, and grew to rapid acceptance as prices fell and the usage of consumer devices dependent on flash memory, such as PDAs, MP3 players, digital cameras, smartphones and tablet computers grew. Even though these devices are predicated on the use of "cheap" flash memory with little to no intelligence behind how the storage is managed, many of them have been around for a good few years without major issues. Sure - many have encountered storage failures - but as against the number of mobile devices that had a built in spinning disk? The relative number of failures are probably reasonably similar - or even weighted in the direction of flash.
Indeed, many employees will be using flash-based memory sticks to carry around work items and will rarely think twice about it. Yet, there is still a built in resistance in many parts of the enterprise in using flash for enterprise storage.
To counter this, those working on flash memory systems have tried to create "enterprise-class" flash. Consumer-based flash is generally based on multi-level cell (MLC) technology. This can support a few thousand write/erase cycles per memory cell, with adequate performance. Single-level cell (SLC) flash provides very high performance along with the capability to support hundreds of thousands of write/erase cycles - but it is very expensive. In the middle lies eMLC (enhanced MLC), which pushes MLC's endurance to a few tens of thousands of write/erase cycles, while keeping costs reasonably low.
What vendors are finding is that the speed performance of SLC is not a major issue at the moment - flash is so much faster than magnetic disk that the perceived extra performance of SLC is only noticeable in certain cases. Also, most vendors have been working on how to manage the longevity of flash, and with a combination of intelligent management and advanced heuristics in how the data is placed on the flash system, they can create systems that can last over a suitable period for enterprise usage.
So, for those looking toward the use of flash memory, the first thing is to understand is that vendors who try to persuade you that SLC is for enterprises and MLC/eMLC is only for consumers are only playing to their own agendas. The next thing is to consider what you are trying to achieve - and see whether a hybrid flash/magnetic disk primary storage approach is right for you or whether you should go straight for a full flash-only based primary storage system.
This area will be picked up on in a later post...