Squeezing it all in

Are your servers bursting with months-old data that you dare not delete? Is your basement bulging with old tapes? Karl Cushing...

Are your servers bursting with months-old data that you dare not delete? Is your basement bulging with old tapes? Karl Cushing looks at options to ease the pain

Storing data is costing you three times what it cost to acquire that data, according to Andrew Lament, head of infrastructure architecture at KPMG Consulting. This means that organisations have to increase the efficiency of how they deal with data by some 66% each year just to stand still. Finding how to best to manage storage and ensure recoverable, upgradable and testable data whilst reducing total cost of ownership is a key requirement for every organisation, regardless of size.

The great storage hopes, according to the pundits, were supposed to be storage area networks (San) and network attached storage (Nas). However, the fact remains that most storage is still attached directly or via the server. A recent report from research and consultancy firm Macarthur Stroud, found that just 30% of organisations in Europe have adopted one or more fibre-based Sans. "That needs to change over the next few years," warns Andrew Batty, chairman of the Fibre Channel Industry Association Europe. He predicts that direct-attached storage will become increasingly problematic, very expensive and will require even more people to manage it over the next three years.

John Mills, vice-president and European general manager for storage firm Auspex Systems, is even more forthright. "Organisations that still have direct-attached storage run a business risk," he says. On top of the increasing costs and difficulties involved in managing your data using this kind of storage, the key problems are the lack of flexibility and the single points of failure. As Mills points out, if a server goes down and some your storage is directly attached to that server then you can't access that data.

Organisations also run the risk of getting locked in with direct-attached storage, he says, as the only place you can buy more capacity from is your server supplier and generally you will have to upgrade your server as well. According to Mills, although direct-attached storage is cheaper in terms of purchasing and implementing it and the infrastructure needed, it is becoming something of a false economy.

Daniel Sazbon, European manager of storage network solutions for IBM, also warns that using direct-attached storage presents a very real danger that you will not be able to back up and protect that data effectively.

"The imperative is to network your storage," says Nigel Williams, vice-president for software operations in Europe for Legato Systems, which produces storage management software. "Yes, there's an outlay but you'll save money in the long run." He says one of the best ways of saving money is consolidation - bringing all your storage into one place - and it is this goal that is currently driving the San market.

The argument goes something like this. By consolidating your storage environment and managing it from a central operations console you can use fewer people and also less skilled - ie cheaper - staff. Williams says that so far organisations have focused on discs but they should also consider tape libraries, as several major manufacturers offer large tape libraries for this purpose. You also need to look at automation software. "That's where you can realise some of the potential gains," he says. "Prioritise and consolidate network storage so you can share it and then apply automation software on top."

Batty agrees, "Consolidation and centralising your San will save your business money through better management, with fewer people needed to manage larger amounts of data."

The general idea is that you can utilise your storage capacity more efficiently if you network it. Lament says that, whereas utilising about 50% of storage capacity is a good figure for an organisation with direct-attached storage, for an organisation using a San that figure would reach about 75%.

But even here there is a real danger that your San will be getting under-utilised. Mills says that research done by Auspex suggests that most companies are under-utilising their San by about 30%, with some analysts putting the figure even higher. Whilst storage managers will naturally only want to use 70% of their storage capacity to allow for slack space, he says, they are only using about 70% of that 70%. This is where virtualisation software steps in.

Virtualisation software suppliers promise yet more potential efficiency gains from networked storage by allowing an organisation to view and manage its different pools of data as a single entity. Lament says virtualisation has replaced hierarchical storage management, allocating space where required, prioritising data and putting it in the most convenient place, and building on the improved storage management abilities of the network storage model. "Virtualisation allows you more flexibility with what to do with your data," he says.

According to Batty, whereas Sans can increase the efficient management of your data by six or seven times, with virtualisation one person can manage 10 times more storage. Other spokesmen, including Sazbon, believe that this figure can be much higher.

Another benefit is that you can work on copies and thus make changes to your systems without taking them down. When you've finished working on the copy you need only to allow for a few minutes or even seconds of downtime to swap over, says Batty.

Not everyone is so convinced however. "All [virtualisation software] is doing is giving an application view of the storage instead of giving a physical view," says Williams. "It already exists and has done for quite some time. Eventually it will just become apart of storage networking."

Williams says people shouldn't wait for some kind of fancy virtualisation product, they should be networking their storage - and realising the benefits of doing so - today.

Return on investment can be swift. Sazbon says that, in his experience, most Sans pay for themselves within the year. However, when measuring total cost of ownership, it is important to factor in elements such as downtime.

"What users have to do is examine the business and the extent that it will be affected by the data not being available; look at the cost of downtime and do a calculation based on the cost of implementing a San or Nas," says Mills.

Cost is also one of the biggest inhibitors for companies considering implementing a San as the initial costs are high. Sans run on fibre channel technology which has no standard and is expensive, although it is getting cheaper. There are developments in the pipeline, however, that will make Sans cheaper and address the issue of standardisation, which should bring down total cost of ownership. The key technology here is ISCSI, a new networking standard being developed by the Internet Engineering Task Force. ISCSI links data storage facilities by carrying SCSI commands over IP networks. "I wouldn't wait for ISCSI though," says Lament. "For companies that are reaching bursting point, getting a San implemented now is a very good idea."

Companies should have at least one eye on the future though. Mills stresses that organisations need to make sure they are buying from a supplier whose roadmap allows the user to take advantage of these technologies when they become available.

For smaller companies with limited storage needs, Nas, which is relatively cheap to implement, might be a viable alternative to a San. Batty says the file-based Nas model - as opposed to the block-based San model - can be a good option for smaller companies, acting like a file server except cheaper. "At entry level, it's a quick and easy way to add storage space to network," Batty says. "It'll be OK until you get to a certain size where management starts to be an issue and you need to be operational 24 hours a day, seven days a week."

Companies should consider going down the San route if they have six or seven servers, need higher availability for some data or are seriously considering formulating a disaster recovery plan, Batty says. Another option is outsourcing. "There is still a case for outsourcing storage to companies to remove the administrative overheads that most companies are suffering from," says Lament. He believes that this model is particularly useful for companies with e-commerce operations.

Backing up - specifically the question of when to back up - is another big storage management issue. Scheduled downtime is becoming less and less acceptable so doing backing up during downtime is becoming increasingly difficult.

The two most common methods for backing up data are to disc and tape and the latter is still the most popular medium for archiving. Although recovering data from tape is a lot slower than from other media, it is useful for storing data that does not have to be accessed often or must be kept to comply with regulations.

Some organisations take snapshots of their data to disc throughout the day and use them to help back up more quickly overnight. Data mirroring and replication are effective ways to back up and recover data but are expensive. Organisations should use different methods for backing up and storing data depending on the criticality of that data, the organisation's availability requirements and the length of time they are willing to wait to recover it. As Williams puts it, "It's a case of matching technology to business need."

The important thing is to act now and start making your storage strategy as efficient as possible, building a robust storage infrastructure that is scalable, more efficient and leaves your options open for the future. As Batty says, "If you bite the bullet now you'll start making savings now." So don't delay, act today - time is money and right now that's a commodity that is in very short supply.

Five ways to achieve cheaper storage management
Jason Phippen, European head of solutions marketing at Veritas, offers the following top tips for managing storage:

1. Base your storage infrastructure on software rather than hardware to ensure freedom of choice and avoid proprietary hardware supplier lock-in. A good storage software infrastructure enables more terabytes to be managed by each network administrator

2. Deploy the highest levels of automation so that manual processes like tape archiving and duplication are automated. This will reduce the number of skilled IT staff doing repetitive manual tasks and improve recovery times

3. Leverage capabilities for application and back-up servers by using technology such as bare metal restore. Bypassing operating system re-installs and configuration can shave many hours from the recovery process and so applications and servers can be restored quicker

4. Avoid deploying high availability solutions from the operating systems suppliers, which typically only deal with their own operating systems. Adopting a common approach to high availability brings many cost savings, ranging from reduced training requirements, lower maintenance costs and hardware savings to an overall increase in the availability of the company's application services

5. Management of the storage network or San should be considered as part of the infrastructure definition and planning process and should not be an afterthought. This will enable quicker and more effective storage network deployment.

  • Don't be myopic: networking your storage might be expensive but you will probably save money in the long term

  • Sticking with direct-attached storage can be a false economy and your organisation could run a business risk as it might not be able to back up and protect data effectively

  • There is a real danger of getting locked in to one supplier with direct-attached storage

  • Include tape libraries in consolidation plans as well as disc-based storage

  • Look at using automation software and virtualisation to leverage the potential benefits of networked storage

  • When measuring total cost of ownership, factor in elements such as downtime and the potential costs to your organisation of data not being available

  • Match technology to business need: use different types of storage depending on the criticality of the data in question

  • Although you shouldn't wait for "fancy" virtualisation products or developments in ISCSI, you should start networking your storage today, make sure your network is scalable and flexible so you can take advantage of these developments.

Guest editor's comment
When I worked in the legal industry, it was the cost of the time to make decisions that was the limiting factor. I found that, provided you had the basics right, it was cheaper to use technology to keep all the data and manage it than to ask the lawyers to spend the time away from billable activities to decide what to keep and what to delete. So everything was kept using a storage area network and a document management system. But, as the cost of technology falls and the cost of staff rises, I expect storage solutions will become economically compelling in other business sectors.

Storage is a horizontal technology so businesses should be able to gain benefits across the board by better storage management. However, some businesses account for their technology in vertical business units. In this case, it may not be as easy for companies to reap the benefits because the cost of the time required to reorganise internal cost allocation systems should be considered as part of the cost of adopting such solutions. Other businesses run distributed systems and the cost of adopting storage solutions may involve considerable development costs. This may not be practical in the short term, even if it is seen as desirable. There may, for example, be some fear of putting all one's eggs in the same basket. But planning can avoid this, especially if it falls in line with cyclical investment in technology.

Owen Williams is head of IT at Knight Frank

Read more on Networking hardware