SAN consolidations, blade servers, virtual servers and storage, data encryption and requirements for more secure fabric services are prompting enterprises to bring more application services into their SAN fabric. Fibre Channel (FC) switch vendors Brocade Communications Systems and Cisco Systems are adding new features to their FC directors' core operating systems or on specialised FC director blades that support these new applications.
For Brocade, the future of its FC directors clearly lies with its 48000 Director model. Though Brocade plans to continue support for its other FC director into the foreseeable future, its 48000 Director is clearly the flagship model for new switch services. "The 48000 will sit at the core," says Doug Ingraham, Brocade's senior director of SAN product management. Application blades such as the FR4-18i Director Blade, FC4-16IP Director Blade and the FA4-18 Application Blade are only available for the 48000, and Brocade doesn't plan to port these blades or services to its other FC directors.
Cisco is further down the road in offering fabric services in its MDS 9513 Multilayer Director SAN-OS, and on its Multiprotocol Services (MPS-18/4) and Storage Services Module (SSM) blades. (The SAN-OS term is commonly used across all Cisco FC switches and directors. However, there are differences in SAN-OS functionality on the various models, so we use the term MDS 9513 SAN-OS throughout this article.) However, services such as continuous data protection, replication, volume management, virtualisation as well as Cisco's new Data Mobility Manager (DMM) are only available if the optional SSM blade is installed on a MDS 9500 Series Multilayer Director. (Encryption services are supported on Cisco's MDS 9222i Multilayer Fabric Switch or MPS-18/4 director blade.)
The directors from Brocade and Cisco are at different stages in their ability to help companies consolidate, control, and share their local and remote FC SAN resources (see "Key considerations to selecting a Fibre Channel director," PDF). For companies with highly distributed management environments that don't want central control, Brocade's architecture lets users introduce fabric services that preserve local administrative control while sharing SAN resources. Companies that need to centralise and standardise their FC infrastructure across the enterprise will be better served with Cisco's FC director architecture.
Consolidation and control
Consolidating remote SAN islands includes technical challenges and political issues about who will control the resources after the consolidation is complete. FC directors currently provide two methods to perform SAN consolidations: fabric mergers and fabric joins.
Brocade's 48000 Director and Cisco's MDS 9513 Multilayer Director allow companies to merge SAN islands into one large, logical fabric. But fabric mergers mean turning control over to one party--usually the department with the larger FC director. The merger also comes with countless technical intricacies. Verifying fabric settings and ensuring they're correct can exceed the time and risk thresholds of many companies. Rather than forcing companies to deal with these issues, Brocade and Cisco permit companies to join remote SAN islands to their large FC directors while keeping them logically separate. Users can share resources among virtual or logical SANs; if a tape drive or disk ports reside in one virtual SAN, servers in another virtual SAN can still access them.
That's where the similarities end. Cisco's virtual SAN (VSAN) technology is part of its native SAN-OS operating system found on its MDS 9500 Series Multilayer Directors. This permits FC switches, or directors of separate and potentially remote SAN islands, to connect to any FC port on the MDS 9500 and remain logically separate from other VSANs that exist on the MDS 9500. Brocade's logical SAN (LSAN) implementation is available only as part of its 48000 FR4-18i Director Blade or on its external 7500 SAN Router.
Which approach is better depends on who'll control what resources after the consolidation. Cisco's FC director architecture assumes companies will want to:
- Consolidate or connect remote SAN islands onto its MDS 9500 Series Multilayer Directors.
- Centralise the administration of user and administrator accounts in the FC SAN.
- Centralise the control and sharing of SAN resources across virtual and remote SANs.
With VSAN functionality a native part of the MDS 9513 SAN-OS, users don't have to buy specific blades or switches to obtain this functionality. Administrators may grow VSANs logically on a port-by-port basis instead of adding a new blade into a 48000 or introducing an FC switch into the fabric as Brocade requires. Using director blades or switches adds the step of moving the physical FC connections of resources (servers, tape drives and storage arrays) to the ports on the director blade or switch.
The MDS 9513 eliminates these steps by allowing admins to configure any MDS 9513 FC port as an "E_Port" and connect it to any vendor's remote SAN switch. VSANs are then created on the MDS 9513, which admins may define to just include the FC ports connected to the remote SAN island. Specific resources among VSANs are shared using Inter-VSAN zones (IVZs).
This is where the issue of control enters. The MDS 9500 Series Multilayer Director supports up to 1,024 VSANs in a single fabric and each VSAN supports the creation of individual admins and users. But sharing resources requires the creation of an IVZ by an admin with superuser privileges and access to all VSANs. The main concern with this approach is that superusers may create IVZs between VSANs without informing the individual VSAN admins. This allows someone external to the VSAN to control how and when specific VSAN resources are shared.
Preserving SAN autonomy
Brocade's director architecture assumes companies will want to consolidate FC ports and share resources among LSANs, but leave control in the hands of specific business units. This approach avoids some of the bureaucratic haggling that Cisco's consolidation approach may cause. Martin Skagen, Brocade's director and chief architect, office of the CTO, finds the benefits of consolidation and partitioning great, but they create an extremely strict change control environment. Once consolidation occurs, "the moons have to align to change anything," says Skagen.
To prevent this and to keep remote SANs separate, Brocade's 48000:
- Connects remote SAN islands to its 48000 through a 7500 SAN router or the FR4-18i blade
- Allows each LSAN to retain control of the creation of user and administrator accounts
- Allows each LSAN administrator to retain control of sharing of LSAN resources
Both the 7500 router and FR4-18i Director Blade support a special type of E_Port for connecting different fabrics called an EX_Port. This allows Brocade to introduce its Fibre Channel Routing (FCR) feature into these two devices that allow different LSANs to communicate with one another.
FCR creates a virtualised switch called a "translate domain," which imports all of the nodes of the edge fabric. Resources are then accessed between different edge fabrics by creating a zone name starting with "LSAN_" that contains the port WWN of the nodes that will access each other. Brocade's FCR recognises these new LSAN zones in the respective LSANs, automatically creates the appropriate routes and then allows communication between selected nodes in the two zones.
Brocade more fully preserves admins' control in separate and remote SAN islands by eliminating the need for a superuser. Admins of each remote SAN island or logical SAN first create an LSAN zone containing the nodes to be shared between LSANs. If an LSAN zone is created in only one LSAN, members defined in the zone aren't shared across LSANs by the FCR. This allows firms to preserve existing political boundaries and gain consolidation benefits.
Cost and complexity
As more services are added to the fabric, costs and management complexity increase. Costs may rise because each new service requires the addition of the appropriate switch to the fabric or director blade to the FC director. Because each new service introduces additional hops into the fabric, it may be prudent to add only one or two fabric services at a time.
Deepak Munjal, Cisco's data centre solutions senior marketing manager, says some of the different fabric services will eventually move off switches and blades and become part of the MDS 9513's SAN-OS. Specialised ASICs on every FC port will eventually take over the processing for each service and let users select which services they want to offer to each app on a port-by-port basis. Munjal says cost and demand will drive these changes, but "it is not realistic to do this [now] since the price of putting an ASIC on every port is still not affordable."
It took three to five years for encryption to move from an appliance to a chip that sits on a router port in Ethernet networks, says Munjal. "Fabric services will likely follow the same path," he adds.
The growth of blade servers creates a problem for FC directors: port preservation. A fully populated IBM BladeCenter Chassis can use up to 28 FC director ports assuming two paths; a Hewlett-Packard BladeSystem c-Class can consume up to 32 FC director ports, again assuming redundant paths. In addition to using up ports, blade servers often can't use the bandwidth available to them on FC director ports.
To mitigate these issues, Brocade and Cisco offer blade switches that act as gateways from the blade servers to the FC director. Blade servers connect to these gateways as they normally would to an FC switch with an N_Port login. However, the switch connects to the FC director and presents itself as an N_Port or host ports instead of as an E_Port as FC switches normally do.
This feature allows the gateway to take advantage of the virtual N_Port ID Virtualisation switch standard and re-present each blade server N_Port login as multiple virtual N_Port logins to the FC director. Once the virtual blade server N_Port is logged into the FC director, the FC director can treat it as a server logged into the SAN, including zoning it to specific ports and assigning qualities of service to traffic coming from that blade server.
But there are differences with how the two vendors implement their gateways, including which vendors' blade servers are supported. Brocade's blade server SAN switch supports blade server chassis from more vendors, including Dell, Fujitsu, HP, Hitachi, IBM, Intel and NEC. Cisco's Fibre Channel Blade Switch currently works only in HP and IBM blade server chassis.
The gateways also scale back on certain software features their switch OSes typically support. For instance, Cisco doesn't support VSANs and IVZ on its Fibre Channel Blade Switch, while Brocade disables nearly all features its FC switches support such as ISL trunking, FC-AL and Brocade Fabric Manager. Disabling switch features such as ISL trunking happens because the switch presents itself to the FC director as an N_Port instead of an E_Port or EX_Port, so the FC director recognises the gateway as a storage device and not an FC switch.
Changing this requires the introduction of a feature called F_Port trunking into FC SANs. F_Port trunking is similar to interswitch link trunks between switches except that it's designed