News

Three ways to create clustered storage

Clustered storage systems run on storage servers, NAS gateways and hosts. Here's how to determine which clustered file-system architecture is best for your needs and storage environment.

Published: 15 Sep 2006 1:00
Clustered storage systems run on storage servers, NAS gateways and hosts. Here's how to determine which clustered file-system architecture is best for your needs and storage environment.

Clustered file systems (CFS) offer a practical way to respond to big storage problems such as the proliferation of low-cost servers, application data growth and the need to deliver better application performance. A CFS pulls together and shares the excess storage capacity that's often available but hidden on storage networks. In doing so, a CFS increases storage utilization rates, delivers performance typically found only in high-end arrays and gives users an economical way to scale their architectures.

There are three ways to deploy a CFS: on storage servers, NAS gateways and hosts. Any server in the cluster can access any block of storage managed by the cluster. Most CFS also integrate the volume manager with the file system. This allows the CFS to break large files into blocks called extents, and to stripe those extents across different storage arrays to improve I/O performance.

There are several key questions that need to be answered before selecting a CFS:

  • Can the CFS make use of existing storage and network resources?
  • How difficult is it to install and configure?
  • How does the CFS manage data integrity?
  • Can it scale performance and capacity linearly and independently?
  • What problem is the CFS best suited to solve?

Clustered storage systems

Clustered storage systems are composed of bricks (servers preconfigured with set amounts of CPU, cache and storage) or blades. Each brick is loaded with the vendor's CFS software that controls and shares the processing memory and storage resources of the bricks in the cluster; blades are managed by an external server that contains the CFS.

Isilon Systems Inc.'s IQ storage clusters use storage bricks and its CFS called OneFS, which combines four layers of storage management software--file systems, volume management, data protection and high availability--into one logical file system. This integration allows OneFS to configure storage on any of the up to 88 bricks it supports in its clusters and to create volumes up to 528TB. Isilon also gives users the ability to choose between bricks of different sizes, ranging from 1.9TB to 6TB raw. While each brick supports only 12 serial ATA (SATA) disk drives, by offering bricks with different size disk drives, Isilon lets users select bricks that meet specific app performance requirements.

Terrascale Technologies Inc. also uses storage bricks, but places its TerraGrid Cluster File System (CFS) on the clients accessing the bricks. Terrascale built its TerraGrid CFS based on the open-source XFS file system; it's a parallel file system that allows apps running in parallel to simultaneously access the same files. TerraGrid CFS scales to support hundreds of nodes, and lets a server read or write data to any node. However, TerraGrid CFS is available only for Linux servers. Windows or other Unix servers that need to access the storage pool have to go through a Linux NAS gateway that contains TerraGrid CFS.

Panasas Inc.'s ActiveScale Storage Cluster is architected in a manner similar to that of TerraGrid CFS, but it also has some unique characteristics. Like Terrascale, Panasas places agents on all clients accessing its storage, directly supports only Linux servers and allows multiple clients to access back-end storage. But Panasas uses Panasas StorageBlades that hold two 400GB SATA drives each. These drives are virtualized by Panasas DirectorBlades that stripe the data across the StorageBlades. DirectorBlades cluster together to create one "virtual" NFS and CIFS server that can scale I/O at high performance levels.

But problems with the clustered storage systems architecture may surface as more bricks are added to the cluster. It's the responsibility of the CFS to manage each additional module's processor, cache and storage capacity. Failing to keep the cache coherent across the bricks can result in file corruption; however, keeping the cache coherent among all of the bricks generates a lot of chatter and degrades the overall performance of the cluster.

Isilon deals with this issue by designating two or more of its bricks as "owning" bricks for each specific file. Keeping the cache consistent in only a few bricks eliminates much of the chatter among bricks. If the request for a file is received by a brick other than the owning brick, the CFS redirects the request to the owning brick. Once the owning brick receives the request, it directs the CFS to distribute the data writes evenly across all of the storage bricks instead of just the disk drives in the owning bricks.

Isilon's approach meets most application requirements when just a few servers need to access large files sequentially, but this technique falters as the number of servers that need to access data concurrently on multiple bricks grows. In that scenario, the owning bricks wouldn't be able to expeditiously handle all of the redirects coming from the other bricks and performance would degrade.

To avoid this problem, Terrascale's TerraGrid CFS allows any server in the compute cluster to access any data block directly on any brick at any time. This approach eliminates the need for cache coherency among the bricks or for the CFS to add any meta data to the file because the file is locked while the server is directly accessing the blocks of the file.

But none of these products overcomes the two main problems of CFS platforms. First, although SATA drives are well suited for the sequential data access required by applications with large amounts of digital content (such as audio, video and graphics), when used in environments with large amounts of random reads and writes, SATA drive performance is significantly lower than that of higher performing Fibre Channel (FC) drives. The other problem is that these systems don't let users redeploy storage they already own. If you have installed storage you want to use in a cluster, CFS architectures that reside on NAS gateways or client servers should be considered.

Click here to continue reading Three ways to create clustered storage, page 2.



Read more on Integration software and middleware