Are you ready for a move to director switches?

Director switches offer greater port density, high availability and enhanced services. If your SAN fabric is becoming unwieldy, it might be time to upgrade to a director.

If your Fibre Channel storage-area network (FC SAN) fabric has grown to the point where it's becoming hard to manage or map, you might want to consider director switches. Pricey but port-dense, these modular devices are intended to fill the same role in a SAN as high-powered chassis switches take in an Ethernet network: core aggregation, in-line services and high availability.

Director switches are chassis based and modular, which makes them expandable. They also have high port counts, and are highly available and fault-tolerant. They usually have telco-grade capabilities such as nondisruptive firmware updates, hot-swap components and no single point of failure. They may also be able to run applications or accept slot-in boards that handle specialist tasks such as encryption, or protocol conversion for long-distance data replication.

Consolidation onto the Brocade 48000 Director in both the mainframe and open systems environments has brought substantial consistency to our operations across both architectures.
Markus Echser
storage management team leaderSwiss Re
Fixed switch vs. director switch

Most Fibre Channel switches aren't directors. Instead, they're fixed switches with a set number of ports and more limited fault tolerance. Fixed and director switches have the same FC switching functions, but fixed switches include less flexibility. Fixing a failed port means replacing the whole switch, whereas on a director you only swap the line card.

There are several ways to choose between fixed and director switches. A simple rule of thumb is port or switch count, according to Bob Passmore, research vice president at Gartner. He stresses that even with directors you must have redundant parallel fabrics, and suggests that once your storage network is big enough to require more than two fixed switches on each fabric -- which with fixed switches at 32 or 64 ports each means 100 or so ports per fabric – it's worth considering a move to directors, even though they cost approximately twice as much per port.

Passmore added that just as organisations build two- or three-tier Ethernet local-area networks (LANs), it's also possible to build a core/edge SAN that combines both types of switch. Indeed, this is pretty much inevitable once blade servers enter the picture. These typically have a fixed switch within the blade chassis that aggregates the server connections and then routes them to a core director switch.

Re-insurance company Swiss Re added director switches to the SANs in its two data centres for consolidation purposes, said Markus Echser, the company's storage management team leader. "Consolidation onto the Brocade 48000 Director in both the mainframe and open systems environments has brought substantial consistency to our operations across both architectures," he said. "This means that we've cut our administrative outlay and are also well prepared to cope with future data growth."

Directors bring greater scalability and easier management

It's important to remember that directors aren't merely big switches. In addition to higher availability, they bring added capabilities such as port and SAN virtualisation, data encryption, inter-switch link (ISL) aggregation, and larger port buffers to support long distance replication and SAN bridging.

"You can't decide just on port count now, especially when you consider storage pooling and virtualisation technologies are being layered on, some in the switches, some in appliances, some at server level," argues Tony Lock, programme director at industry analyst and research organisation Freeform Dynamics.

Lock said that while SANs have brought many extra capabilities, such as snapshots and data replication for disaster recovery, they've also brought much more infrastructure and management complexity. That means organisations may need storage consultants and managed service providers to help them make sense of it all.

"The decision is much more to do with the mind of the buyer and the use they want to put the switch to," he said. "For example, what are the business requirements from the storage? From that you map out the storage function and then you talk to your suppliers or, increasingly these days, to some storage specialists."

It was those extra features -- along with greater scalability and easier management -- that persuaded logistics firm Panalpina to use two Cisco Systems' MDS 9509 Multilayer Directors as the core of its new SAN, which has dual redundant fabrics. The SAN includes 20 TB of enterprise storage, and connects to servers, including Citrix Systems and Microsoft Exchange systems, and IBM AIX systems for in-house applications.

"With the modular design and future roadmap of the Cisco directors, we get investment protection because we can easily upgrade with new capabilities in the future," said Kevin Ball, Panalpina's network group manager. "We also like the fact that the Cisco products are designed to integrate and interoperate well together, and we have similar software platforms to manage." He added that Panalpina also uses Cisco VSAN (virtual SAN) technology to connect to its disaster recovery site over a shared long-haul link.

So which of those extra functionalities should you look for when choosing a director? This is where you need to know what you want to do with the device, said Tam Dell'Oro, president and eponymous founder of research and consulting firm Dell'Oro Group.

"Everyone has their own opinion on what 'advanced capabilities' are cool and sexy," she said. "Usually end users only care about one or two features. Manufacturers make claims of being able to do this and that, but the real test is how well it performs this or that. Only a lab test can determine whether a product performs at a top rate or at a real lousy level."

Dell'Oro added: "I've heard that virtualising traffic streams in and out of the server through the adapters, through the switches, to the storage array is very good. Both Cisco and Brocade claim this capability, but who does it better? I can only guess by sales performance."

Director switch players: Brocade, Cisco and QLogic

Fortunately, it's pretty simple to draw up a shortlist when you want to buy director switches. A few years ago, you might have had a dozen suppliers to choose from, but now there are just two big players left -- Brocade and Cisco -- and QLogic.

While some switch vendors have disappeared (such as Sandial Systems), most have been acquired and many have ended up owned by Brocade. For example, director pioneer Inrange Technologies, which introduced the first 256-port director, was bought by CNT, which in turn was acquired by McData. McData also bought high-end director startup Sanera before McData was finally taken over by Brocade.

In contrast, Cisco's MDS director technology came from Andiamo Systems, a start-up Cisco funded and subsequently acquired.

"Brocade has a huge inherited user-base from all those other vendors," Gartner's Passmore said. "Most of the Inrange kit has gone now, but there's still lots of McData and classic Brocade [available]. Most of those customers are comfortable with Brocade," so they only glance at Cisco when it comes time to buy new gear, he said.

"Cisco is still using its original technology, so it's around two years behind Brocade, which is why Brocade is gaining share," he added. Part of the problem, Passmore said, is that Cisco bet on Fibre Channel over Ethernet (FCoE) taking off a lot faster than it actually did, so it missed out on an FC market that continued to grow.

Gartner's figures show Brocade with more than 50% of the director switch market and Cisco with most of the remaining market share. However, Passmore said Cisco has focused its activity on directors and sells far fewer fixed Fibre Channel switches than Brocade. It therefore has only 20% of the overall Fibre Channel market, while Brocade has more than 70%, leaving less than 10% for other vendors such as QLogic.

According to Dell'Oro, Brocade's director-class switch sales have climbed away from Cisco in recent quarters, and she attributes this to both product and business issues. On the product side, she pointed out that Cisco's directors (or modular switches, as she prefers to call them) are oversubscribed, whereas Brocade's aren't, so Brocade's ports can all run at line rate whereas Cisco's can't. Line-rate director ports make it more practical to build a core/edge network, using cheaper fixed switches at the edge to push the overall cost down.

"The key aspect to high-end Fibre Channel switches is the capability of supporting 8 Gbps Fibre Channel, scaling to 16 Gbps, and being able to integrate and support FCoE," Dell'Oro said. "Brocade's DCX has 256 Gbps bandwidth per slot compared to 96 Gbps on the Cisco MDS 9500. We attribute this lag to Cisco's focus on its data centre Ethernet platform, Nexus, not the MDS 9500. Cisco's focus is on converging the Ethernet LAN and SAN with FCoE and its proprietary version of Converged Enhanced Ethernet (CEE), named Data Centre Ethernet."

On the business side, Dell'Oro suggested that Cisco's partners may be shifting their allegiances following the strategic changes at Cisco that saw it introduce rack and blade servers, as well as converged storage and network switches. Perhaps the best thing is to think of director-class switches as the SAN equivalent of an IP core switch or router. Just as core routers are expensive, but are needed to provide backbone capacity, so are directors. And while the two will eventually converge, there's probably another three to four years before that becomes a practical proposition for the mass market.

Read more on Storage fabric, switches and networks