Switch upgrades and replacements

Switches are routinely expanded or replaced to accommodate more storage platforms or host servers in the fabric. Switching is also updated to achieve better performance (e.g., faster ports), enhance fabric availability, restrict network traffic and bring intelligent features, such as virtualization, to the fabric. This section of the guide highlights some of the most important best practices and goals for switch expansion or replacement.

Best Practice No. 1: Understand the physical limitations of your switches

A switch is usually sized based on a finite number of ports so you should know how many ports are available for use, as well as the total number of ports that can be added to your current switches, if any. Switches intended for small and many medium-sized organizations offer fixed-port counts and cannot be expanded. The Catalyst Express 500 product family from Cisco Systems Inc. and the SANbox 1400 from QLogic Corp. are examples of fixed-port switches.

When you need additional ports, simply buy another switch and connect the switches together through an interswitch link (ISL). Switches intended for large business and enterprise use, aka director-class switches, are far more upgradeable, offering as many as several hundred ports through modular upgrades. For example, the 48000 director from Brocade Communications Systems Inc. can handle up to 384 ports through up to eight 48-port blade modules. Experts recommend using fewer, larger switches. "A company that needs 100 ports would probably be better served buying a single director-class switch instead of buying three or more smaller switches and tying them together," says Stephen Foskett, director of data practice at Contoural Inc.. "They just perform better than the smaller ones."

Best Practice No. 2: Watch port speeds and look for 8 Gbps Fibre Channel switches

Beyond the sheer number of ports, storage professionals should also consider port speed. Common Fibre Channel port speeds include 1 Gbps, 2 Gbps and 4 Gbps. You can find ports running at 10 Gbps, but 10 Gbps ports are not backward compatible with the slower port speeds. Consequently, 10 Gbps ports are typically used as ISLs to interconnect Fibre Channel switches within the data center. You don't really need to buy 10 Gbps ports unless there is a specific need for them. The next big push for port speeds is 8 Gbps, but it's not quite ready yet. "You can buy the [8 Gbps] HBAs but not the switches," Foskett says. "Eight [Gbps technology] is off the table right now."

Ethernet switches for iSCSI SANs commonly offer 1 Gbps ports. Switches with 10 Gbps ports are readily available, but the actual number of 10 Gbps ports is limited. For example, the Cisco Catalyst 4948 10 Gigabit Ethernet (GigE) switch provides only two 10 Gbps switch ports.

Best Practice No. 3: Understand the functional limitations of your switches

Today, switches incorporate features that go beyond simply managing network communication, such as data migration and storage virtualization, which are common in the large director-class switches. The need for intelligent features, such as these, is frequently the motivation for switch upgrades and replacements, and storage administrators should consider intelligent features and their role in the data center. For example, it may be possible to justify a Brocade Mi10K director for VSAN support in the organization. Conversely, experts warn against acquiring unneeded features and functionality. Not only is this a waste of money, but the switch itself is underutilized.

Factor emerging technologies into your upgrade plans. The FCoE connectivity that is on the horizon will affect switches and host bus adapters (HBA), requiring a forklift upgrade to implement. As a result, some users may forego switch upgrades or changes until FCoE-compliant switches and storage systems are available. "Do I want to make a big 4 gigabit Fibre Channel SAN investment when EMC tells me I'll be using a totally different protocol in three years?" Foskett asks.

Best Practice No. 4: Standardize on one switch vendor in the enterprise

Experts recommend standardizing on a single switch vendor to ensure full compatibility and adequate security between switches, especially if the switches include intelligent features. "Having more than one vendor in your fabric can compromise your security." says Phil Goodwin, president of Diogenes Analytical Laboratories Inc. If you must use switches from more than one vendor, thorough compatibility and security testing is highly recommended.

Best Practice No. 5: Architect the switching infrastructure for high availability

It's important to architect a SAN so that redundant connections use multiple pathways to separate switches, eliminating single points of failure within one switch. However, the cost of multiple directors is often prohibitive, while the use of additional switches is typically discouraged. Storage architects are forced to perform a balancing act to achieve a suitable mix of cost, simplicity and reliability, and that balance is different for every organization. "You have to decide what your tolerance for risk (and outage) is," Foskett says. "We always recommend building dual redundant SANs."

Best Practice No. 6: Watch for unexpected failure rates on older switches

Older switches (circa 2002) may be subject to surprisingly high failure rates. "If you've got a four- or five-year-old switch with 16 to 24 ports, consider replacing the whole thing," Foskett says. Budgeting a replacement switch at the first sign of port failures offers an opportunity to purchase a larger switch with more ports to accommodate network growth and allow consolidation with other small switches in the environment. The move to a new switch can also introduce useful switch features that may not have been available when that original switch went into service.

Best Practice No. 7: Look for opportunities to deploy iSCSI

ISCSI technology is being deployed by more and more small and medium-sized businesses (SMB) , as well as within the departments and workgroups of large enterprises. While iSCSI is still not an ideal choice for all SAN deployments, its use is increasing, bolstered by its relative simplicity, low deployment costs, good overall performance and reliability, and broad user knowledge of Ethernet. When expanding a SAN, it's now valid to consider the potential role of iSCSI in your own enterprise. "Even though iSCSI is perfectly acceptable for any operating system, if you have 100 Windows systems that you'd really love to hook to the SAN, strongly consider getting an iSCSI SAN instead of a Fibre Channel SAN," Foskett says.

Best Practice No. 8: Avoid port sharing in iSCSI

One potential problem with Ethernet performance is the common practice of port oversubscription. Many Ethernet servers do not depend on high performance, so almost all Ethernet switch ports are oversubscribed. This allows more than one device to use the same switch port, thereby maximizing each port's bandwidth utilization. Experts note that a port may be oversubscribed as much as 10 to 1. Oversubscribed switches may not be able to handle the excess load in a high-performance iSCSI SAN, so avoid configuration practices that lead to switch port oversubscription, or select high-end Ethernet switches for deployment within an iSCSI SAN.