Keeping a lid on data storage

It is a rare mantelpiece that does not have at least one photograph of a wedding, graduation or family gathering. So when launched a service allowing customers to store photographs of important events online, the company knew the biggest challenges would be meeting demand and offering total reliability.

It is a rare mantelpiece that does not have at least one photograph of a wedding, graduation or family gathering. So when launched a service allowing customers to store photographs of important events online, the company knew the biggest challenges would be meeting demand and offering total reliability.

"If you are storing pictures of really important events in people's lives, like weddings, you cannot afford to lose a single image, and people expect to be able to access their images instantly," says Simon Wardley, Fotango's chief operating officer. "That is the same whether they are accessing their holiday snaps or images of a big event like Live8."

The challenge for Fotango was to create a storage infrastructure that would provide reliable access to several terabytes of image data, without spending a fortune on storage hardware. The firm also wanted to be able to move data around so that if demand for some images suddenly spiked, those images could be placed on a storage medium that would provide faster, high-performance access.

After a three-month consultation process, Fotango invested in storage virtualisation technology from IBM that allows IT staff to view all available storage as a single pool and move data easily between storage platforms.

"With virtualisation, we can add capacity easily where we need it, and reprioritise high-performance hardware when necessary," says Wardley. "We have very high volumes of data and high rates of change, but virtualisation still means we can make the very best use of our resources."

Many enterprises are also struggling to tame the storage monster, says Bob Zimmerman, an analyst with Forrester Research.

"As inexpensive as storage may be, the volume of corporate data is still increasing by about 40% a year," says Zimmerman. "Ultimately, the cost of storage hardware, access and management will outweigh the savings you are making from disc price erosion."

Storage management is also an issue because few companies work in a single-supplier datacentre, says Paul Griffiths, technical marketing manager at storage supplier Brocade.

"In most enterprise datacentres there will be some legacy mainframe storage and you may have a mixture of Unix and Wintel servers from various suppliers," he says. "That is because most companies have chosen their hardware for technology or financial reasons, rather than ease of management."

This complexity can dramatically increase the cost of storage management and, in some cases, mean that management is not even attempted, says Simon Pennock, managing director of The Networked Storage Company, an independent storage consulting firm.

"It is a big headache because each point solution has its own management tools and its own training requirements," he says. In many cases, companies do not have time to implement the various management systems and end up managing storage manually. "An awful lot of spreadsheets are being used to manage storage in enterprises today," says Pennock.

Storage virtualisation helps companies address these problems because it allows IT departments to present heterogeneous storage hardware from multiple suppliers as a single pool of storage.

The first important advantage of this is that companies no longer need to individually manage a dozen different hardware platforms. Instead, they use a single interface to control the entire storage pool.

Secondly, virtualisation can help companies increase the efficiency of their existing storage systems by increasing utilisation.

"With storage virtualisation, you can pool all your storage and parcel it out to applications and servers that need it, as required," says Guy Bunker, senior director, software engineering at supplier Veritas. "Not only does this mean better utilisation, but we are also seeing block-level back-ups and intelligent replication across storage networks, which would be impossible without virtualisation."

"It is not unusual for companies to only use 20% or 30% of the capacity on some of their storage hardware," says Lawrence James, information lifecycle management (ILM) solutions business manager at supplier StorageTek. "With virtualisation, you are presented with an image of the entire pool, with all the spare capacity pooled together, so you can see utilisation rates of 80% or 90% quite easily."

Better utilisation means companies can wait longer before buying new hardware, generating big savings. One high street bank that implemented virtualisation technology saved 20m in storage hardware and associated costs over three years, says James.

Finally, virtualisation can reduce complexity, allowing many routine storage maintenance tasks to be simplified or automated. "One of the biggest benefits of virtualisation is that it removes the need to individually manage and control a dozen different mainframe, Wintel and Unix storage servers," says James. "For many of our customers, using virtualisation means that staff no longer have to come in at weekends to manage the servers."

Virtualisation is not a particularly new idea, but the technology has advanced enormously in the past couple of years, says Claus Egge, programme director at analyst firm IDC.

"The concept of having a virtualisation layer between storage hardware and applications has been around for years, but whereas it used to focus on what was happening in the storage array, now it can operate at a network level across various platforms and environments," says Egge.

Companies can choose from a range of virtualisation technologies, which operate at a hardware, software or appliance level (see box). Each approach has pros and cons, so it is important that users consider what they want to achieve before investing in any new technology, says Egge.

Forrester recommends that companies with large datacentres implement virtualised storage on a fabric-based model, but this approach is not suitable where companies are consolidating a large number of homogenous arrays. For this an array-based model would be better, says Zimmerman.

In some cases, enterprises may be wise to wait for anticipated developments in silicon, which should make virtualisation easier in heterogeneous environments. Intel and AMD are both adding new capabilities to processors that will make it easier to run multiple operating systems on a single platform. Intel, for example, plans to add virtualisation technology to its Pentium line from the end of 2005, and to its Itanium and Xeon lines in 2006.

Some analysts say the latest virtualisation technology could enable companies to implement ILM, another hot topic for IT directors.

"ILM would be practically impossible for most companies without virtualisation," says Egge. "However, with virtualisation becoming more mainstream, we expect to see a lot of companies also rolling out ILM projects."

ILM is based on the idea that the value of information (and therefore data) varies over time, explains Bunker. The most valuable data should be stored on fast/expensive disc storage and less valuable data moved to near-line CD/DVD storage or tape, and ultimately archived. "With virtualisation, this process can be completely automated based on administrator-set policies," says Bunker.

Certainly, early adopters of storage virtualisation have reported impressive results. According to research by the US-based Enterprise Strategy Group, virtualisation delivers average cost reductions of 24% in hardware, 16% in software and 19% in San administration.

Today, about 30% of StorageTek's enterprise customers are looking at virtualisation, says James, and many have already deployed the technology. "I expect that to grow further," he adds. "Many of our customers could expect to reduce storage management costs by a factor of 10 if they use virtualisation."

Read more on Integration software and middleware