Rein in NAS with file virtualisation

File virtualisation can reduce storage TCO and increase availability, as we explain in this feature on the technique.

Once considered an afterthought, corporate network file servers are becoming major management headaches. Complicated file migrations, heightened compliance concerns and stranded storage capacity are just a few of the file storage management issues forcing companies to re-evaluate how they handle this critical piece of storage infrastructure.

Forward-looking companies have embraced file virtualisation appliances to recapture stranded storage capacity on network file servers and to perform file migrations among them. However, using file virtualisation appliances as permanent corporate file managers is a big step that some corporations are unsure they want to take.

File virtualisation appliances provide the following major benefits:

  • A global namespace that indexes files on network file servers
  • Excess storage capacity can be shared among network file servers
  • Data migrations that are transparent to end users and applications
  • Support for tiered storage infrastructures

Operating in front of network file servers, file virtualisation appliances create an abstraction layer between file servers and the clients that access them. The file virtualisation layer catalogues files and file systems across multiple network file servers, and enables administrators to present a single, logical file mount point for all network file servers that's accessible by all client servers. Once in place, network file servers continue to host file data and metadata while the file virtualisation appliance provides advanced file management capabilities.

File virtualisation appliances change the dynamic of how organisations manage their network file servers. For example, the appliance aggregates stranded storage capacity on network file servers, simplifies file migrations among the network file servers and virtualises network file shares. However, these benefits depend heavily on the underlying architecture of the file virtualisation appliance. Key attributes such as high availability, the ability to scale capacity and performance, and integration with third-party indexing and search engines are handled differently by the various file virtualisation appliances.

Typically, organisations initially bring file virtualisation appliances into their storage environments to manage file servers more efficiently. Approximately three years ago, Ubicom, a developer of communications and media processor and software platforms, CA, was running out of space on its Network Appliance (NetApp) Inc. filer. However, Jim Poehlman, Ubicom's chief information technologist, didn't have the $100,000 required to upgrade the filer. While he had other network filers available with excess capacity, he needed to migrate files in real-time without disrupting production applications because downtime would cost the company tens of thousands of dollars per hour in lost productivity and missed service-level agreements.

Poehlman deployed NeoPath Networks' File Director, which "didn't require us to reconfigure our network or bring our applications down," he says. "NeoPath Networks also gave us a price break by selling it to us for $15,000, though at the time we bought it they were still in beta."

Kevin Hayes, technology security manager for the City of Jacksonville, also needed to recapture stranded storage capacity. Hayes had approximately 15 Microsoft file servers with varying amounts of excess capacity. As part of the reconfiguration, Hayes needed to improve application file performance, an issue he couldn't resolve with a simple file migration. To address these issues, he chose Maestro File Manager from Attune Systems Inc., which allowed him to create one logical file share that aggregated and virtualised filer volumes across different network filers. "Migrating files to one virtual file share allowed me to stripe files across my entire network file server farm ... and increase application file performance," says Hayes.

Ibis Consulting , which provides electronic discovery and compliance solutions for corporations involved in litigation, had a different set of requirements. As part of its discovery process, the firm receives large amounts of data from multiple clients in different file formats that need to be uploaded to its NAS storage. Ibis then culls and classifies files from its different clients and moves the files to different storage tiers for attorney review.

Shane Lennon, vice president of strategy and market development at Ibis Consulting (now a Pitney Bowes company), selected an Acopia Networks Inc. ARX6000 because it helped to "track and place files across multiple NAS systems and a large number of CIFS shares without causing a bottleneck to the Ibis process."

File virtualisation architectures

File virtualisation appliances are available in four different architectures: out-of-band, a combination of out-of-band and in-band, in-band and split-path. Brocade’s StorageX is the only file virtualisation appliance that stays exclusively out-of-band or out of the data path. It resides on a Windows 2000/2003 namespace server and creates a global namespace by discovering and indexing files on network file servers.

Client servers then access and use the StorageX global namespace in the following ways. Windows client servers discover the StorageX global namespace using the Distributed File System (DFS) client included with the Windows operating system. For Linux and Unix operating systems, StorageX updates the Network Information Service (NIS). These client servers then access files by first contacting the StorageX global namespace server, which provides the client server with the physical locations of the requested files. The client server then uses that response to directly access files on the network file servers.

While this architecture may work well for organisations aggregating file views across multiple data centers, some administrators may have a problem with how StorageX handles file movement. For Windows-based clients, StorageX's Replication Manager uses a native Windows-based replication agent to move files. However, on Unix and Linux hosts, StorageX deploys and installs its own replication agent that uses the NFS protocol to move files between different network file servers. This means StorageX has to have the appropriate security access to the Unix and Linux operating systems to install this agent, which may be a concern to IT shops that don't want to introduce another agent on their hosts.

EMC’s Rainfinity Global File Virtualisation uses a combination of out-of-band and in-band techniques. Rainfinity stays out-of-band during the initial file discovery, but moves into the data path when virtualising files. This requires administrators to create a virtual LAN (VLAN) in the network that allows Rainfinity to move into the data path between client servers and network file servers to virtualise files. When it does so, the Rainfinity appliance assumes the IP address and DNS name of the network file server. This causes the network to reroute traffic normally directed to the file server to Rainfinity, at which time Rainfinity takes control of file placement and movement among network file servers.

A feature that distinguishes EMC's Rainfinity from other file virtualisation appliances is that it doesn't require the creation of its own namespace. The absence of a namespace was a determining factor for Brian Peterson, a storage architect with a Midwest food packaging company, who chose to use Rainfinity as part of his company's NAS migration and consolidation efforts. Because Peterson had previously implemented a NAS namespace within his company, bringing in another file virtualisation appliance and converting to its namespace would have meant taking a single huge outage requiring the coordination of almost 500 people.

Because Peterson's corporation uses NAS so extensively, he needed protection against Rainfinity failing while operating in-band to ensure that enterprise applications wouldn't come to a halt. To prevent that, Peterson's team wrote what they called the "Big Red Button" script. The script allowed the operations staff to quickly take Rainfinity out-of-band and out of the data path of the servers, allowing client servers to directly access data on network file servers.

"The script provided the insurance we needed to insert the virtualisation device into such a critical data path," says Peterson.

Moving in-band

Because of user concerns about putting devices in-band, some in-band file virtualisation appliances give users the option to first deploy their appliances in an out-of-band configuration. Attune Systems' Maestro File Manager can be operated in three modes: discovery, native and extended. In discovery mode, Maestro File Manager allows users to initially discover, analyse and report on storage usage and utilisation on their network file servers without putting it in-band. In that mode, it generates reports on information such as CPU and storage utilisation, file-access patterns and throughput on specific network ports. These reports let organisations determine the true benefits they'll realise from deploying file virtualisation, and appropriately configure and size the file virtualisation appliance.

Another key consideration when selecting and configuring in-band appliances is their hardware configuration. Both Attune Systems' Maestro File Manager and NeoPath Networks' File Director rely on clustered configurations available with server operating systems: Windows in the case of the Attune Systems product and Linux for the NeoPath device. In both cases, they're configured for active-passive failover, although there may be a delay of up to a minute while the passive server takes over for a failing active server.

Enterprise NAS environments with high availability, transaction or throughput requirements may find the latencies associated with these in-band configurations unacceptable. Only Acopia Networks' ARX6000 addresses those concerns in its software and hardware. On the software side, the ARX6000 uses a split-path architecture that separates the data and control paths. The split-path architecture allows the ARX6000 to better support high transaction rates by assigning more frequent file operations, such as reads and writes, to the data path while the control path handles the more infrequent file requests, such as create and delete operations on the file server directory.

The ARX6000 also supports an active-active configuration that allows dynamic failover between the two nodes in the cluster. Ibis Consulting found that with the ARX6000, its general file server availability improved because the firm could migrate files from a network file server that needed maintenance to other file servers still in production. "Before we deployed the ARX6000, maintenance outages could run up to a day," says Lennon at Ibis Consulting. "Now outages last at most four hours on a weekend once a quarter."

To support the higher transaction and throughput rates required by most enterprises, Acopia Networks uses proprietary hardware in the ARX6000. Adaptive Services Modules (ASM) speed up the control path functions of directory changes, while Network Services Modules (NSM) accelerate file read and write operations.

F5 networks has an appliance-based file virtualisation approach, which it says enables faster overall throughput.

But despite the immediate benefits derived from file virtualisation appliances, some users are still uncertain about the role file virtualisation appliances will play in the long term. Storage architect Peterson only uses EMC's Rainfinity when he needs to perform file migrations within his corporate NAS infrastructure, while the City of Jacksonville's Hayes anticipates removing Attune Systems' Maestro File Manager once he finishes consolidating on a NAS gateway. Given situations like these, vendors are trying to ensure that their file virtualisation appliances become more than just one-time file migration tools.

Next steps

To transition their products from virtualisation/migration engines to "corporate file managers," vendors are adding key features that will keep file virtualisation appliances relevant regardless of how a corporation's NAS infrastructure evolves. File virtualisation appliances, for example, allow organisations to index their corporate files and to immediately know when changes are made to specific files. Attune Systems recently established a relationship with Google to allow users to perform more sophisticated indexing and searching across their corporate file servers.

File virtualisation appliance vendors also recognise that user storage environments will remain heterogeneous. "In these environments, our file directors act like an umbrella, allowing the free movement of data between different brands of network file servers," says Ali Zadeh, chief operating officer at NeoPath Networks.

Acopia Networks recently released the results from an independently validated lab test that demonstrated the ability to create a consistent point-in-time snapshot across a virtualised volume whose physical files simultaneously reside on both an EMC Celerra NS700 filer and a NetApp FAS3050 filer. The Acopia Networks snapshot was also used to create a self-consistent backup across multiple file servers.

"The ability to do snapshots across the proprietary boundaries created by many enterprise storage vendors is an important step toward the deconstruction of monolithic storage arrays," says Jon Toigo, chief executive officer and managing principal at Toigo Partners International LLC, an analyst firm. "Heterogeneous snapshot holds out the possibility of driving down the cost of array products by eliminating a consumer lock-in," he says.

EMC anticipates users will deploy Rainfinity for a broader range of applications. While migrations are now the primary use for Rainfinity, companies may also use file virtualisation appliances for capacity, file, tiered storage and performance management. But according to Jack Norris, EMC's director of virtualisation marketing, the real value of file virtualisation appliances is that "virtualisation will drive the intelligent placement and classification of data."

File virtualisation appliances are a corporation's best bet when migrating files or optimising file placement on network file servers. However, key differences in their architectures may narrow users' choices IT shops with centralised, high-volume NAS implementations will largely start and stop their product search with Acopia Networks' ARX6000. Department-level and smaller IT shops will have the luxury to choose between in-band and out-of-band architectures. And if you're looking to use file virtualisation appliances on a more permanent basis, you should give preference to products that are further down the road toward delivering a standardised approach of enterprise file indexing and search capabilities.

File virtualisation checklist

Five items to consider before you select and implement a file virtualisation appliance.

  1. In-band or out-of-band? In-band file virtualisation appliances present a common, single interface for all network file servers. While they can simplify management and file migrations, in-band appliances can become a bottleneck if not sized appropriately. Administrators can usually deploy out-of-band file virtualisation appliances with little or no initial disruption to their environment, but they may need to push out agents to servers or move in-band to perform file migrations.
  2. Global namespace. A global namespace is a central catalog of the files that reside on network file servers. Organizations that have NAS namespaces may need to switch from their current namespace to the one introduced by the file virtualisation appliance.
  3. Appliance availability. If you plan to use a file virtualisation appliance strictly to perform file migrations, availability isn't a concern because both in-band and out-of-band appliances permit failures without data loss. But if you're planning to permanently virtualise the environment and also take advantage of advanced functions like striping files across different file servers or aggregating network filer volumes, make sure the appliance's level of availability matches your company's service-level agreements.
  4. Filer volume sizes. File virtualisation appliances are often used to consolidate file systems residing on multiple network file servers onto one file server during migrations. However, it's important to remember to keep file-system volumes at sizes that can be backed up within specified backup windows.
  5. Support for network filer APIs. File virtualisation appliances don't diminish the value that NAS operating systems like EMC Corp.'s DART and Network Appliance Inc.'s Data Ontap deliver with such features as snapshots and file locking. When using these features, look for file virtualisation appliances that integrate with these devices so policies may be set on the appliance to centrally manage these functions.


Read more on SAN, NAS, solid state, RAID