The growth in e-commerce and other Web-based activities, together with data warehousing and increased use of multimedia data in applications such as digital imaging, has led to a phenomenal growth in storage requirements.
Strategic Research, a market research company which specialises in tracking storage trends, estimates that storage demands are doubling roughly every two years. On top of that, the move towards customer-focused marketing and supply chain integration means that different applications often need access to the same data at the same time.
The sheer volume of this data - and the fact that it increasingly needs to be available around the clock to support 24x7 operations such as Internet sites - presents a new set of management challenges to IT directors. Storage area networks (Sans) aim to address these issues.
Almost all storage today is "bus attached" - that is to say it is connected directly to a particular server using SCSI or IDE (integrated drive electronics). Network-attached storage (Nas) and Sans both evolved as a way of consolidating storage and bringing it out from behind particular servers whose capabilities and loading can limit access to the data. The difference between Nas and a San lies in how that storage is made available to users.
San vs Nas
Nas attaches a disc array directly to the main communications network, which links clients to servers via a processor running an operating system which deals with file I/O protocols such as NFS (network file system).
By contrast, a San comprises a dedicated high-speed network sitting behind the servers and connecting multiple storage devices to the back ends of multiple servers. It uses a storage protocol, such as SCSI or the newer fibre channel standard, to transfer data. A San also incorporates a management layer which organises the transfer of data, allows storage devices to be controlled and handles tasks such as back-up and replication.
If you believe San suppliers' marketing, you might think it equals fibre channel. However, Michael Peterson, president of Strategic Research, says Sans are not a new idea. Those based on a network storage interface called Escon have been connecting mainframes to multiple storage systems for years, he explains. What is new is that San architectures are now being applied to distributed systems such as NT and Unix.
According to Andrew Rowney, sales and marketing director at Storage Area Networks, fibre channel has become associated with Sans partly because, running at 100mbps, it initially offered a considerable performance advantage over SCSI running at 40mbps. However, Rowney suggests it is perfectly possible to create a San using an established storage network protocol like SCSI, especially given the development of the new backward-compatible SCSI standards running at 80mbps and 160mbps.
Whichever protocol you use, San suppliers claim the architecture offers a number of benefits. First, consolidating your storage in one place should help you to cut the cost of managing it. Peter Coleman, technical manager at San hardware supplier Gadzoox, points out that IDC estimates that a single technician can look after about 100Gbytes of server storage, with the result that management costs eat up half the storage budget. When storage is centralised, the same person can manage 750Mbytes of storage, reducing the management overhead to just 15% of the overall cost.
On top of that, consolidating your storage devices allows you to deploy them more flexibly - re-allocating spare capacity on one system to another server, for example. This should result in a better return on your investment in existing storage devices and may help you avoid unnecessary expenditure.
Although the capital expenditure associated with deploying Sans can be quite high, Coleman believes these costs can be recouped relatively easily. However, Gregg Ormsbee, product development director at Tandberg Data, warns that current San solutions have only begun to scratch the surface in terms of consolidating storage.
At present, each application is still likely to have its own dedicated data store and discs. Ormsbee expects to see increasing synchronisation of data between applications, leading eventually to the development of a single data store containing a superset of all the data in the organisation, together with metadata describing how it should be used by different applications.
One of the drawbacks of consolidating your storage devices in one place is that it could make your data more vulnerable. But it also makes it much easier to replicate that data, and one of the most prominent features of the new fibre channel protocol is its ability to link devices across a distance of 10km, allowing remote vaulting of data and high-speed transfer of data between sites.
Rowney points out that to take advantage of this feature you need access to a fibre channel link between the two sites. "Even today, you can't buy fibre channel services from [telecoms and Internet service providers], and most companies don't have the opportunity to lay their own cable," he says.
However, carriers do offer asynchronous transfer mode (ATM) backbones which can provide high-speed links over much greater distances than fibre channel. Not surprisingly, Rowney's firm has developed a product which allows Sans to be connected through ATM links.
Apart from helping IT managers to protect their data through improved back-up, San technologies lend themselves particularly well to use with other high-availability techniques such as clustering. Coleman points out that operating systems such as Windows 2000 and Netware 5 have been specifically designed to offer clustering of standard Intel platforms connected to Sans. Multiple pathways between storage media and servers in a well-constructed San can also help to ensure high availability of data.
According to Coleman, another benefit of attaching all your storage devices to a dedicated network is that it offloads traffic from the Lan. As a result, users may see significant improvements in the performance of applications, given that up to 30% of all Lan traffic is due to back-up. This will become increasingly important as 24x7 operations such as e-commerce become more widespread and overnight back-up windows disappear. Back-up via a San will also minimise the impact on users when a system failure means data needs to be restored during the normal working day.
Coleman says the ability to shift traffic off the Lan makes Sans particularly suitedto applications which involve moving largefiles around - such as video streaming or pre-press work in the publishing industry - because storage protocols are designed to offer much higher sustained data rates than Lan protocols.
However, implementing a San to achieve these benefits is still far from straightforward. According to Chris Boorman, Euro-pean marketing director at storage software specialist Veritas, the quality of San offerings can vary between suppliers, and even within a supplier's turnkey solution. He believes that good San solutions will only be achieved by carefully selecting and integrating best-of-breed products for each of the technologies which make up a San - San hubs and switches, host bus adapters which connect servers to the San, management software and the storage devices themselves.
Ormsbee says unified standards for San, allowing plug-and-play interoperability, are still six months away. He and Coleman agree that integration skills will be essential to the success of San implementations. "You need an integrator which can bring together best-of-breed products to meet your business needs, rather than going to a single supplier who might be able to offer a turnkey solution but not one that is best for your business," Coleman argues.
Rowney warns that such skills may be in short supply in the immediate future as storage suppliers - which have not been noted for their consultancy and integration skills in the past - struggle to develop channel partners with the right expertise.
IT managers will find that storage management software is only just beginning to incorporate some of the features needed to implement Sans supporting more than a limited number of servers - controlling which servers have access to which data, for example. Early adopters of Sans should prepare themselves for a rocky road before they reap the benefits.
Fibre Channel Association - http://www.fibrechannel.com - 01635-874330
Storage Network Industry Association - http://www.snia.org - 001-650 949 6750
Brocade Communications - http://www.brocade.com - 01344-750190
Computer Associates - http://www.cai.com/arcserveit - 01753-577733
Dot Hill Systems - http://www.dothill.com/products/sannet/san_net.htm - 020-8874 5252
Exabyte - http://www.exabyte.com - 0800-966172
Gadzoox - http://www.gadzoox.com - 0118-988 0262
IBM Numa-Q - http://www.sequent.com - 01932-851111
Legato Systems - http://www.legato.com - 01628-511811
M4 Data - http://www.m4data.com - 01252-864600
Mylex Corporation - http://www.mylex.com - 01256-334000
Network Appliance - http://www.netapp.com/products/filer - 0771-445 6233 Overland Data - http://www.overlanddata.com - 0118-989 1891
Storage Area Networks - http://www.san.com - 01223-566111
StorageTek - http://www.storagetek.com - 01483-737333
Tandberg Data - http://www.tandberg.com - 01582-769071
Tivoli Systems - http://www.tivoli.com/products/index/storage_mgr/ - 01753-780000
Veritas - http://www.veritas.com - 01932-876876
Before you buy
- Decide which data and applications warrant the cost and complexity of a San
- Find a partner with integration skills who is willing to take responsibility for delivering an end-to-end solution
- Look for a solution which is scalable and can incorporate legacy storage devices
- Before buying, ask for the particular combination of products you want to use to be tested in an interoperability lab
- Make sure you understand which protocols your supplier is using and how they plan to migrate to unified industry standards once they are agreed
- Check support for management activities such as dynamic re-allocation of resources
Case study: Nationwide Building Society
Nationwide, the UK's largest building society, has pursued a policy of centralising its data for a number of years. However, the advent of Windows NT-based systems, which typically involve multiple servers, meant that traditional connection protocols could not keep up with the number of servers which now need to be connected to central data stores - which are themselves growing larger.
"You can now get devices with seven or eight terabytes in a box which can support a hundred servers, but you can't connect that many via SCSI," explains Martin Conway, a technical consultant at the building society.
On top of that, Nationwide had a number of other goals: connecting servers to storage over greater distances; implementing dual-port connections to improve resilience and load balancing; improving performance in its data handling activities and achieving greater flexibility in using and sharing devices such as tape drives.
The building society quickly realised that San technology provided an answer to all these challenges. However, it decided to implement a San only when suitable management software became available. "The physical technology has been there for some time, but not the software needed from a security perspective to allow us to control which servers see which discs," Conway explains.
Nationwide now has a San based on StorageTek's Access Hub, which is connected to disc storage from EMC as well as products from Digital, Compaq and Emulex. Connection to tape storage is currently being investigated. Nationwide chose StorageTek because of its long relationship with the supplier,and StorageTek's ability to provide a suitable device backed up by high levels of implementation support,which sawthe supplier underwrite the provision of an end-to-end solution.
The key decision was whether to go for hub or switch technology. "Switch technology offers the advantage of any-to-any connectivity, but it's costly," Conway says. "In the NT space, you need a lot of ports, so the cost-per-port becomes an issue. A solution based on intelligent hubs - where you can change loop configurations on the fly - offered us the best compromise of cost and functionality. However, switching suppliers are driving their costs down and later this year we will be looking at whether to intro-duce switches between the storagemedia andthe hubs."
Case study: BSkyB
The award-winning 3D and special-effects department at satellite broadcaster BSkyB is responsible for producing animations and promotional segments for channels including Sky One and Sky News. A minute of video takes up 1.5Gbytes, handling and sharing data is a major issue for the department, but a San installed last year has enhanced productivity by making it easier for users to share data. Because it is based on a Raid array, the San has also increased the security of the data.
Previously, data was stored on individual workstations connected via Ethernet. Dave Sedgwick, BSkyB's visual effects supervisor, explains, "Before the San installation, if someone needed to work on a different workstation, it would take longer to move the data than it would to complete the project".
The new set-up, which was implemented by storage integration specialist Sagitta, links seven NT workstations to a 144Gbyte Raid array using a Gadzoox Gibraltar fibre channel-managed hub. One of the key reasons why the company chose Gadzoox for the project was because it has a strong reputation in the storage community for this size of San installation. BSkyB did consider using gigabit Ethernet rather than a San but decided it was not mature enough to meet its needs.
Installing the San was not completely straightforward. According to Sedgwick, there were a number of interoperability issues to be resolved during the implementation, particularly when connecting Silicon Graphics workstations. And there are a few bottlenecks which still need to be addressed.
If these issues can be resolved satisfactorily, BSkyB is considering extending the San to support more storage and up to 80 end-users.