Coping with the storage explosion

You may have ten times more data to store in five years' time than you do now. Jane Dudman looks at your management options

You may have ten times more data to store in five years' time than you do now. Jane Dudman looks at your management options

The price of storage per Gbyte has plummeted in the past few years, but not the cost of managing it. Volumes of stored data are set to rise - as more business is conducted over the Internet, the volume of information being captured soars. Analyst company Forrester Research says that most companies will have 10 times more storage in five years time than they do now. Forrester thinks that will push up the amount being spent on storage and the administration to run it, from 5% of today's average IT budget to 17%.

"The cost of managing storage is anything from four to eight times more than the cost of buying it," says Paul Gunstone, director of e-continuity at disaster recovery specialist Safetynet, a major user of Legato storage software. Gunstone believes the real costs of managing storage is bringing about major change, as companies re-assess their storage strategies in the face of new technologies.

As a result of the reduced cost of storage media, the cost of building mirrored systems has fallen dramatically. This can have a big impact on overall costs by allowing changes to be managed within normal working hours. Gunstone cites one company that saved £20,000 in overtime costs after installing mirrored systems."With mirrored systems, rather than having to wait until out of hours to carry out upgrades, the IT team can take down one system and resynch it, then do the other."

Another way to cut down the cost of managing storage is to consider outsourcing, which has not yet impacted storage, except as part of an overall IT contract. For example, US firm StorageNetworks, whose service is based on providing fibre-based connections from its storage centres into customer premises, has built its first UK centre and is building a second, with the infrastructure due to be in place by the end of March.

Safetynet is working closely with StorageNetworks and Gunstone says outsourced storage will provide customers with much greater financial and technical flexibility. "This is a managed storage service," he comments. "At its lowest end, it's disc on demand. You buy a processor, phone the storage management company and say you need an extra 50Gbyte of storage. All they do is turn up the tap."

The financial advantage of a storage service is that it turns the capital cost of buying storage into ongoing revenue costs. In the US, StorageNetworks charges around $65 (£41) per Gbyte of storage per month, so it will pay IT managers to work out what they spend on purchasing and managing storage. Initially, though, this kind of service will be economical only for fairly large customers, acknowledges Gunstone.

"This service is based on dark fibre and that is still expensive," he admits. There are other limits too: customers will have to be within 25 miles of a StorageNetworks centre to use the dark fibre links, although the facility can be used over a WAN from anywhere in the country.

Internet-based online storage is based on a similar concept to managed storage services, where the storage facility itself is off-site and run by a specialist company. Several companies, such as NetStore, now sell online storage services and claim to be able to provide services for both large and small customers.

Most IT managers still run their own storage facilities in-house and are looking to make the best use of new technologies, particularly the much-hyped concept of storage area networks (Sans)tohelp them face the familiar challenge: how to enforce a regular, efficient back-up policy for all systems, particularly when it comes to remote and mobile users, and how to ensure users can access stored data quickly when they need it.

Ian Batten, principal systems engineer at Fujitsu Telecoms Europe, which uses storage systems from Auspex, says his company's storage requirements have soared over the past decade. "The main requirement when we first bought storage systems was performance," comments Batten. "Back then, we had about 30 or 40 users on disc-less Sun workstations and we wanted to get data onto the desktop in the minimum time. Now we have more than 1,000 users and a more heterogeneous environment, so the emphasis of our storage strategy now is on space rather than speed."

Batten says today's combination of communications and storage systems should be able to provide the performance levels required by most businesses. "There is a finite amount of data people need at any one time and that has not expanded as fast as you might think," he says. "You can buy any fileserver and get a reasonable performance, unless you have extreme requirements like the oil and gas industry. But what you do need is capacity."

Fortunately, he points out, the cost of storage has plummeted as demand for capacity has soared. The first storage system Fujitsu Telecom Europe bought from Auspex was a 14 Gbytes system. The company pays the same price for 2Tbytes storage now as it paid for 14Gbytes then. "The price per Gbyte has dropped to not much over £100," points out Batten. This makes it easier to buy larger systems, but gives managers like Batten bigger headaches about how to manage such large facilities.

In the debate about whether storage should be split up and kept close to the systems it is supporting, or kept as a central storage facility, Batten is firmly on the side of centralisation. "It is good to keep it in one place because it makes the back-up transfer easier and, in practice, the meantime between failures is better because the chances are that most of the users need most of the servers most of the time," says Batten.

Other organisations feel differently. The British Library, for example, has installed a storage system based on multiple storage servers partly because supplier Mosaic feels this reduces the potential for a single point of failure.

Whether IT managers opt for centralised storage or a more dispersed approach, the real key to managing storage effectively lies in software. Hence the rise of storage software specialists such as Legato and Veritas, whose products are used by many of the leading storage suppliers.

The importance of storage software is acknowledged by Michael Tonge, technology project manager in PruTech, the IT services division of insurance giant Prudential Assurance. Prudential has installed systems from storage specialist EMC to back up its huge Oracle customer contact management database, available online to the company's 90 regional UK offices. The software it uses includes EMC's SRDF and TimeFinder, which help create real-time mirrored volumes of the Oracle databases, and Enterprise Data Manager software, which backs up mirrored data from an EMC Symmetrix 3930 server to StorageTek tape libraries.

"The resilience provided by the Symmextrix hardware, in tandem with the SRDF and TimeFinder software, ensures bullet-proof, non-stop operation," comments Tonge. Not only is the Prudential looking for full resilience, it also wants to be able to carry out its back-up procedures with the least possible interruption to its business operations, which is a challenge when running a 1.5Tbyte customer database.

TimeFinder and SRDF are used to run point-in-time back-ups, enabling business-critical activities to continue while backups and restore operations are being carried out. Mirrored volumes of the database are created, then split from the production system and used for non-disruptive point-in-time back-ups. After the back-ups are completed, the mirrored volumes are synchronised with the production system. By carefully timing back-ups, data can be restored almost instantaneously from the mirrored system for more than half the working day, while online database transactions run uninterrupted.

Not many businesses have to cope with the huge storage challenge faced by the Prudential. But whatever their size, organisations have to ensure they have the right systems in place to back up and access all that information. If storage requirements grow as fast as many predict, the hitherto overlooked cost of managing storage is going to become a hot potato in many IT departments.

Cutting the cost of storage management

  • Media is cheap, management is not. Mirrored systems enable more work to be done in office hours and cuts expensive IT staff overtime bills

  • Managed storage services and online storage are emerging as a potential alternative to running in-house storage and turn capital costs into revenue costs, but need careful assessment

  • Watch the footprint: large central storage systems may take up large amounts of premium price floorspace; but with dispersed systems, watch out for the comms costs

  • Watch out for hidden hardware upgrade costs, such as moving from CD to DVD

    Centralising storage

    There are a number of storage management benefits to be achieved by moving to a large centralised enterprise class storage subsystem, says Tony Reid of Hitachi Data Systems.

    Raw storage costs - By consolidating storage requirements into an enterprise storage subsystem the spare storage pool can be allocated to the busy servers, leading to better utilisation of storage resources and reduced costs

    Reduced management costs - One centralised enterprise storage system means only one management structure. Research group Dataquest claims that one person can manage three times more storage if it is centralised versus distributed

    Availability - Centralising storage with an enterprise system from one supplier allows the user to take full advantage of their business continuity features

    Scalability - To achieve this benefit an organisation needs to link its centralised storage to a storage area network (San), freeing up network resources by taking back-up traffic off the Lan.

    Managing storage at the British Library

    One of the benefits of the British Library's move to its new site at St Pancras has been a decision to consolidate its store of book titles into a single central database of titles on CD and DVD disks, enabling the public to access digital reference material in the library.

    The library wanted a reliable system for the half-million researchers who use its facilities each year and, equally important, the ability to add new titles. Each year, the library adds more than 500,000 new items to its collection and expects that number to rise sharply as a result of the growth in new media and digital reference titles. It wanted a storage system that was physically small because, despite the move to the new site, there's still a shortage of space, and was cost-effective to buy and manage.

    The library opted for a system from Mosaic Systems, based on a multiple server approach. It has installed ten storage racks, each containing an Axis StorPoint CD E100 server, six CD-Rom drives, one DVD drive and a 36Gbyte hard disc drive and is using mirrored caching. Not only does this ensure almost instant access to the stored titles, but also ensures compliance with CD and DVD software licence rules, which could be potentially infringed by copying data to a hard disc and physically removing the media.

    Mark Davies, digital media development manager at the British Library, says the result is a marked improvement on access to the CD-Rom titles. One particularly useful feature has been information from the system about CD and DVD use, enabling managers to ensure they have enough licences for the titles in most demand.

  • Read more on Integration software and middleware