Storage now cheaper than ever

Nothing is more annoying than having your C drive so full that Windows grinds to a halt. These days, storage is so cheap that...

Nothing is more annoying than having your C drive so full that Windows grinds to a halt. These days, storage is so cheap that there is no excuse not to have plenty of it. Nevertheless, suppliers are doing their best to develop even better storage technologies that are faster, cheaper and more dense.

In the magnetic storage field, companies have consistently refined techniques to make the reading and writing of individual bits on the magnetic surface more precise, enabling them to pack more data into a square inch of disc space. As the bit density increases, the size of the magnetic grains on the disc must decrease to accommodate the smaller size of each bit. The problem is that, as density approaches 100 gigabits per square inch, the physical law of superparamagnetism comes into effect. Reducing their size beyond certain limits can cause grains to alter their magnetic state at will and therefore renders discs unreliable.

IBM has been researching methods of combating superparamagnetism, such as using different types of magnetic material and preprocessing it into patterns to reduce interference between magnetic grains on the disc. It has also examined the idea of perpendicular recording, creating "pipes" of magnetism that extend down into the disc which, it says, form a closed magnetic field and make it more difficult for different magnetic particles to interfere with each other across the surface.

IBM's proposed perpendicular recording method uses a vertical magnetic channel through the disc to record a single one or zero. But in the optical field, suppliers are exploring the possibility of writing multiple layers of information throughout the material of the disc, creating a multi-layered "wedding cake" of information, with different data stored at various levels within the media. Lasers then focus on the layers to read back the data.

One company that has been making significant advances in this area is Constellation 3D. The company, founded in 1995, has been researching fluorescent media, which it claims enables lasers to read the various layers of a disc without any interference from other layers. The technology will enable it to store up to 100 layers of data, each holding 4.7Gbytes (the capacity of a single-layer DVD disc) within a single CD-sized disc, embedded within the material. It aims to release multiplayer CD-sized discs with more than 100Gbytes storage (compared to the 17.4Gbytes capacity on a dual-layer, dual-sided DVD). It claims that this will allow up to 20 hours of compressed high definition TV recording. It will also release card-based products with up to 10Gbytes storage. Both these product families will be available in Rom format first and then write-once-read-many format later on.

Apart from enhancements to optical storage by changing the media, companies are also exploring the possibility of increasing storage density by changing the read-write mechanism. Research scientists are experimenting with blue laser technology as a means of cramming more information onto a disc. Its shorter wavelength allows it to read and write information more densely. The biggest challenge for blue laser is its expense, which will restrict it to high-end drives initially.

Meanwhile, in the established world of DVD technology, the read-only technology is well accepted. Read-write technology is less established, thanks to competing standards from rival factions. According to IDC analyst Simon Shepherd, there are three standards vying for dominance: DVD+RW, DVD-RW and DVD-Ram.

"Currently, it is unlikely that these technologies will converge, although technologically it is not a problem," Shepherd says. "It could take over three years for the issue to be resolved."

The DVD world is exciting because of its multimedia connotations, but the market for tape technology continues to develop, with new innovations coming down the pipeline. Tape technology, which is suited to storing large amounts of archived material, has traditionally been based on technologies such as digital linear tape (DLT), a popular standard originally developed by Digital Equipment (now part of Compaq).

This technology provided storage capacity of 70Gbytes, and transfer rates of up to 12Mbytes per second. Super DLT technology, which has been in preparation for the last couple of years, is now shipping to customers and in its first iteration offers a capacity of more than 200Gbytes (compressed) with a transfer rate of more than 20Mbytes per second.

Another group of suppliers including HP, IBM and Seagate is supporting a rival technology, the linear tape open (LTO) standard, available in both the Accelis format for fast access and the Ultrium format for high capacity storage. HP released its first LTO products in November 2000 and at this early stage, the LTO and Super DLT technologies are neck-and-neck in terms of acceptance.

Today the emphasis in the storage market is not so much on what type of technology you use as on how you use it. Buying an array of magnetic or optical discs with a certain capacity and access speed is easy because it involves commodity products. Putting such storage devices on your network and linking them to servers and to other storage devices is a more difficult problem.

Storage area networks (Sans) have been on the market for three years. Sans are networks of storage hooked together using high-speed connections either through a hub or, more likely, a switched configuration. The server infrastructure then connects to this network at strategic points. The benefits of Sans are reliability, because data can be backed up easily between San devices; and flexibility, because a single storage device is no longer "owned" by a directly connected server. Fibre channel has become a popular means of hooking San equipment together.

So why discuss Sans in an article about the future of storage? This technology's time has yet to come because many user companies have failed to exploit it, according to James Henderson, an analyst at specialist storage analysis company Macarthur Stroud.

"A lot of suppliers say they can offer a complete San but there are still problems hooking together components," Henderson says. "The open systems market is something that companies are working towards."

Companies that have maintained expensive, difficult-to-implement fibre channel links will slowly move to TCP/IP, although this will take a couple of years because very high-speed links are difficult to implement over TCP/IP. IBM is set to release low-end San products this year running on TCP/IP over iSCSI, which is a draft standard from the Internet Engineering Task Force.

The Storage Networking Industry Association (SNIA), which is driving standards development in this area, formed the SNIA IP Storage Forum last month to further exploration of this technology.

While San technology is growing, there are some technologies in the labs that will not be available for some time. They are promising successors to conventional magnetic and optical storage when these technologies reach their limits, although for magnetic storage this is unlikely to be for seven years or longer, according to Claus Egge, another IDC analyst specialising in this sector.

Holographic storage is an exciting, if far-off, technology. Whereas many storage mechanisms write data only on the surface of their media, holographic storage works by storing pages of data all the way through, at very high-density rates. In this sense, it has some of the qualities of Constellation 3D's multi-layer technology, but data is stored and read in a different way. Using refractive physics, lasers shone at holographic material at different angles can read many pages. IBM claims to be able to store 10,000 pages of information, each with 1Mbit of information, in material the size of a sugar cube.

Future applications could include the storage and playback of high-quality video information, thanks to the high data retrieval rate associated with the technology - it can read over one billion bits per second in laboratory conditions, due to the lack of moving parts and the ability to read whole pages of data at once.

Other companies are also becoming interested in the technology. At the end of January this year, Lucent set up a company called InPhase Technologies specifically to research holographic storage.

HP has been approaching storage from an even more futuristic direction, with its atomic resolution storage project. This works by manipulating atoms using ultra-sharp heads, meaning bits can be stored at the most dense, high-capacity level physically possible. Should this become commercially feasible in the future, it would eradicate capacity problems for those companies which could afford it.

Storage has come a long way since the early days of computing, when hard discs had not even been considered and Ram would have been thought revolutionary. It still has a long way to go, but as with processor technology, we are unlikely to see any quantum leaps in physical storage products until we have exhausted the opportunities in the magnetic sector.

A day in the life

Jim's personal digital assistant fits on his wristwatch and it has a holographic storage mechanism built into it, which enables it to store huge amounts of data. He uses it to play a high-quality movie downloaded from his PC while waiting for his dentist appointment. Then, when he emerges three fillings and an extraction later, Jim uses it to play soothing digital music - selected from more than 100 albums, stored in the size of a sugar cube. When he gets to the office, he uploads his sales figures for the previous day to the corporate network. One device on the storage area network is down, but it does not matter - the system reroutes the information through to a back-up device that stores the data, ready to be distributed to other storage devices on the network when ready.

In the not-too-distant future

Peer-to-peer storage is almost upon us. Endeavour Technologies, a specialist peer-to-peer software firm, launches its peer-to-peer file access product this year, while Microsoft is working on a serverless file distribution system called Farsight. Here client systems form a logical, virtual fileserver between them, using a global name space for files and sharing multiple encrypted replicas of their files between different machines. Consequently, if your disc crashes and wipes your data, you should still be able to access files when you fix your system and bring it back online.

Click here to learn more about Farsight.

A timeline for storage technologies

2001: LTO and Super DLT tape drives shipping Constellation 3D ships 5Gbyte and 100Gbyte read-only disc and drive

2002: Storage service providers (ASPs for storage) become more popular among user customers

2003: IP-based Sans begin to hit the market in volume, capturing public attention

2004: DVD standards battle solved, one standard becomes dominant

Read more on Data centre hardware