IT directors looking for an easy way to manage increasing volumes of data by buying more hardware could be storing up problems for the future.
When it comes to tackling burgeoning storage volumes, most IT directors admit they tend to choose a short-term solution instead of a long-term strategy. In fact, 80% say they simply purchase more hardware rather than investigate ways of getting more out of existing resources, according to a recent survey conducted by market research company Vanson Bourne.
In the survey, commissioned by systems integration company Computer Systems and Finance, many IT directors admitted that plummeting hardware costs made buying extra storage seem like a "relatively pain-free" option.
But in addition to storing corporate data, they may also be storing up long-term problems for themselves, says Mark Sweeney, business development director at Computer Systems and Finance. "The policy of adding more storage to an infrastructure that is not designed to support it can create its own problems,” he says. “Such organic growth quickly becomes costly to maintain and manage – especially if the hardware has been bought from different suppliers to capitalise on tactical cost savings."
Such an approach can introduce much inefficiency. Storage management and administration software tools may be duplicated across the infrastructure; storage administrators have to retain a wider base of product knowledge; and parts have to be bought from separate suppliers so the company has to forgo volume discounts from individual suppliers.
And there is another problem: although organisations continue to purchase additional storage hardware in response to tactical business demands, it is likely that much existing storage space is sitting idle. "It is accepted throughout the data storage industry that many organisations use only 40% of their storage capacity," Sweeney says.
What is needed is a new approach in which IT managers set out a long-term strategy for their storage architecture. To do so, they must take the time to evaluate existing levels of use and the workload their storage infrastructure is forced to bear.
The larger IT companies say they help their customers to do this. "It is vital to analyse current storage usage before embarking on any storage implementation or upgrade,” says Nigel Ghent, UK marketing director at storage supplier EMC. “You need to know what you are holding, where you hold it and what data has specific compliance requirements. That is a huge job in itself.”
Steve Cliff, sales manager for the northern region at IBM, says a company should then look to implement storage strategies such as information lifecycle management. "Once you know what data you need to store, you can put in place a variable cost storage architecture where you match storage systems to data, according to its value.”
The next step along this path is to network the storage architecture. In recent years, the evolutionary theme in data storage has been the shift away from direct-attached storage (DAS) to storage area networks (Sans) and network-attached storage (NAS) systems. Although DAS still persists in most datacentres, and will continue to do so for as long as the legacy applications it supports remain in service, new storage investment has largely gone to building San and NAS systems.
"Traditional storage topology - pairing a storage array with a single server, or statically partitioning an array to meet capacity needs for several servers - fails to provide the flexibility to dynamically reallocate resources to meet unpredictable demands, isolates data to a single server and, given data storage growth rates, is almost impossible to manage," says Bob Zimmerman of IT market research company Forrester Research.
Networked storage, by contrast, circumvents this problem, in theory at least, by organising individual storage resources into a unified "pool" that can be accessed by a variety of applications across the company.
This also enables companies to implement storage systems from different suppliers, thanks to standardisation efforts in this area (see “Raising The SMI-S Standard” below). "With a good, solid San in place, I think it is safe to say that most organisations can comfortably continue to invest in storage systems from a number of different suppliers," says Martin Davies, principal consultant at systems integration company Morse.
Most organisations have implemented a blend of San and NAS technology. San systems provide the fined-grained, so-called block-level access to data, which is needed to optimise the performance of databases and database-centric applications such as customer relationship management and enterprise resource management. NAS, on the other hand, better supports the kind of file-level access needed for document- or file-centric applications such as office, workflow and collaborative environments.
Thanks to San-NAS convergence, all data can be held on a single physical San, but the system's capacity can be logically partitioned to accommodate block-level access over Fibre Channel, SCSI or iSCSI, and file-level access over IP.
Users must also choose an appropriate communications protocol based on price/performance parameters. Fibre Channel still offers the best price/performance balance, and can operate across distances of about 100km. To link more distant datacentres, Fibre Channel can be encapsulated in IP packets, and where global reach is needed, users can deploy Fibre Channel over wave division multiplex links.
The drawback of Fibre Channel is cost. Fibre Channel host bus adaptors are expensive and, where their expense is not justified by service level requirements, slower, shorter distance links can be achieved using commodity disc interfaces such as serial advanced technology attachment (SATA), or the economy-class technology of SCSI over IP (iSCSI).
iSCSI is increasingly spoken of as a cheaper, mainstream alternative to Fibre Channel. However, most tape and disc devices still do not have native iSCSI connectivity, so any perceived cost saving will normally be cancelled out by the need to install an expensive bridging device.
SATA, a more robust version of the ATA interface built into commodity, PC-class discs, offers a cheaper, near-line alternative to Fibre Channel disc. Back-ups can be sent from SATA to tape without disruption to the production environment.
Similar price/performance issues come into play when archiving to tape. DAT, VX1 or AIT-1 are all popular, economic formats for backing up local servers, but they do not offer the performance required for larger systems. Here, DLT (digital linear tape) devices supporting transfer rates of 2.5Mbps are a better option and are moderately priced.
However, DLT has reached the physical limit of its capacity (80Gbytes), forcing larger users to invest in high-speed, high-capacity storage such as SuperDLT, LTO and SAIT. Using 2:1 compression, these formats can handle up to 600Gbytes, 400Gbytes and 1.3Tbytes of data respectively.
Managing storage resources effectively is also a software issue. Storage resource management software tools aim to automate much of the labour-intensive and error-prone processes associated with allocating storage resources, and virtualisation enables the servers and their applications to treat the devices on the storage network as a single pool of capacity.
By allowing more efficient and flexible use of storage resources with less manual involvement, virtualisation and storage resource management promise improvements in IT staff productivity.
Given the complex questions that surround the purchase and implementation of company-wide storage architecture, it is hardly surprising that some IT directors prefer to turn storage management over to a third-party services company. Despite that, the storage service provider model that was touted three or four years ago by a number of companies heavily backed by venture capital failed to take off, for a number of reasons.
Many customers did not want to give up control of their data and the customers that did wanted dedicated resources, thus negating any potential benefits that storage service providers got from the economies of scale offered by sharing resources between clients. The storage management tools that enable IT directors to allocate storage resources to data were not available at that time so storage service providers were forced to write their own.
But most of all, many clients raised the question, "Why outsource storage in isolation?" In other words, if they decide to outsource storage, it made more sense to outsource servers as well. For that reason, storage is now usually outsourced along with servers and other infrastructure elements as part of a wider managed services deal.
For example, in November 2004 French electrical group Schneider Electric signed a £1bn 10-year outsourcing deal under which IT services company Capgemini would take responsibility for the support of Schneider's entire IT infrastructure - including storage systems.
There is, however, a booming market for specific storage services such as back-up and recovery and remote mirroring. In mid-2004, broadcaster GMTV signed a three-year contract with hosting company PSINet Europe to manage its disaster recovery and data back-up systems. GMTV's production and broadcasting systems are archived and stored off-site to ensure a programme can go ahead even if the broadcaster's London studios are evacuated.
These kinds of services may be attractive to some companies, but Vanson Bourne's survey found that, despite ever increasing volumes of corporate data and a slew of legislation about their retention, 40% of IT directors still prefer to keep storage in-house. To do so, they must plan for the long term, not look for a quick fix.
Raising the SMI-S standard
In 2004 the major players in the storage industry finally agreed on a standard application programming interface that allows storage management software to link to any disc array, from any supplier. The Storage Management Initiative-Specification (SMI-S) may have limitations in its current version (1.01), but promises true multisupplier interoperability in the future.
Five years in the making, SMI-S has come on "in leaps and bounds over the past 12 to 18 months", says EMC’s Ghent. "Many people do not understand the full extent of the work that has gone on behind the scenes - but a standard must have teeth, and SMI-S does."
SMI-S is based on the Distributed Management Task Force's common information model and web-based enterprise management standards. In the storage world, SMI-S is similar to the simple network management protocol, the standard for managing and monitoring network devices on a Lan or Wan. More than 100 products - from suppliers such as EMC, Hitachi, IBM, Hewlett-Packard, McData, Brocade, Network Appliance, StorageTek and Sun - are certified as SMI-S-compliant, and the number is growing.
That broad support for SMI-S is vital because it guarantees IT decision-makers a common schema for choosing storage management applications and hardware. An IT director can order a Hewlett-Packard disc array now and an EMC SMI-S-compliant disc array in 18 months’ time in full confidence that the same software can handle such tasks as the provisioning and monitoring of both.
SMI-S 1.01 provides common schema for the provisioning of a storage network, the monitoring and provisioning of storage subsystems, and for creating, mapping and masking logical unit numbers - a unique identifier used on a SCSI bus that enables it to differentiate between up to eight separate devices. Key features still to be addressed are snapshot management, replication, mirroring and support for NAS devices and tape libraries. These are expected in SMI-S 1.1, due in 2005.