iSCSI SAN: Reliable shared storage without Fibre Channel's costs

The iSCSI SAN is a block-level shared-storage alternative to Fibre Channel that is becoming increasingly popular among SME organisations.

By Danny Bradbury, Contributor

The iSCSI SAN is a technology worth considering for many organisations in a post-recession, cash-constrained world. iSCSI SANs can be a good way for many small and medium-sized enterprise (SME) organisations to move from direct-attached storage (DAS) and deploy a SAN for shared storage without breaking the bank. But does an iSCSI SAN offer sufficient performance?

iSCSI is most often compared to Fibre Channel because of the tendency for both to be used as block-level shared storage. The two have one thing in common: They're both used to exchange SCSI commands between storage and server nodes. However, they also have some significant differences.

The core difference between Fibre Channel and iSCSI is the underlying transport. Fibre Channel has its own networking stack, which doesn't use Ethernet. The exception to this is Fibre Channel over Ethernet (FCoE), which runs Fibre Channel commands over a specially designed converged enhanced Ethernet (CEE) link.

Conversely, iSCSI uses Ethernet and TCP/IP as its underlying transport layer, making it possible to use standard Ethernet equipment for connectivity. Many iSCSI SANs today are built using standard network interface cards (NICs).

Fibre Channel's performance has been set by the people that build the point-to-point networking technology. Fibre Channel bandwidth currently tops out at 8 Gbps, although a 16 Gbps version was ratified last year. Meanwhile, iSCSI's performance is tied to the performance of the underlying Ethernet network, meaning that every increase in Ethernet speed offers it a performance boost. With 10 Gigabit Ethernet now in widespread use, the technology is a relatively high-performance shared-storage option.

That's all at the level of the fabric/network, of course. But, iSCSI has also improved over time in terms of disk drive performance. While early iSCSI products almost exclusively relied on SATA drives, the better-performing and more reliable SAS drives are now becoming more commonplace.

That said, there are drawbacks to iSCSI, warned Hamish Macarthur, founder of UK-based storage analyst company Macarthur Stroud International. It isn't as resilient as Fibre Channel, he pointed out, because it operates over a lossy network prone to packet collision. At least Fibre Channel over Ethernet has been designed for lossless operation.

Although non-CEE Ethernet networks will always be lossy, there are ways to improve iSCSI performance at the server-connect level. Buying an iSCSI host bus adapter (HBA) for an iSCSI SAN (also known as a TCP/IP Offload Engine -- or TOE card) will enable you to offload the task of wrapping and unwrapping SCSI packets in Ethernet frames from the server's CPU. This could become particularly important in a virtualised server environment where every ounce of the CPU's power is being squeezed.

Macarthur also warned that iSCSI over a shared network is a bad idea. It should be installed on a separate Ethernet network segment of its own to protect the traffic, he said. "Otherwise, if you have something that messes the shared network up, you're going to get a very staccato type of service from the system, which is not exactly going to be acceptable," he warned. "So you still use Ethernet cabling and switches, but you have them as dedicated lines between the storage and the CPU, as opposed to just using the open Ethernet connection."

On the upside, this means that the effect on network and application teams should be minimal. Restricting the traffic to its own LAN segment means that, for example, those dealing with the corporate VoIP system, or horizontal applications such as email and groupware, shouldn't be adversely affected by the introduction of an Ethernet-based SAN. The lack of political fallout will be worth the extra switching equipment necessary.

None of this rules out iSCSI as a mechanism for SAN, Macarthur pointed out -- it's just that if you want to support mission-critical apps you need to get the networking side of things dead right.

Having said that, the cost of constructing an iSCSI SAN as opposed to a Fibre Channel network will be lower for two main reasons. Firstly, there is no need for specialist equipment: you can set up an iSCSI SAN using an off-the-shelf Ethernet kit. Secondly, the cost of human capital is also lower. "Even if you're talking about adding a different network, it's still the same Ethernet gear," pointed out Henry Baltazar, senior analyst for storage and systems at the 451 Group. "You won't have to learn a totally different type of technology. Conversely, Fibre Channel specialists won't be cheap."

"We would have had to have hired specialist staff," said Jonathan Burden, technical services engineer at University College Birmingham. The college, which needed to upgrade its storage, installed three SATABeast storage arrays from Nexsan in a project costing about £100,000. The storage arrays, which were initially purchased purely for backup purposes but are now storing all of the organisation's data, are configured for both Fibre Channel and iSCSI. Burden chose the latter to integrate the units into a SAN spread across three physical locations, each 10 minutes apart, using fiber optic cable that was already installed from a previous project.

"We're not really pushing the network that much. We don't need to have really good, solid network activity all the time so we can afford to run on iSCSI," said Burden, adding that if the organisation really began to use its Gigabit Ethernet network's capacity to the fullest and experienced data dropout problems, it would consider switching to Fibre Channel immediately.

Its lower cost makes iSCSI a suitable technology for organisations that have no prior experience with SANs. But, those with Fibre Channel-based SANs are unlikely to want to move away from that technology, said Andrew Reichman, senior analyst at Forrester, because they will not want to relinquish control of their networks to conventional network administrators.

However, Fibre Channel-competent organisations may want to form hybrid Fibre Channel/iSCSI SAN infrastructures, mixing in iSCSI for non-mission-critical applications where it makes sense. This also makes it possible to connect smaller Windows-based servers to a SAN at a lower cost than would be possible with Fibre Channel, potentially opening up storage virtualisation to more parts of the computing infrastructure.

A couple of years ago, Reichman published a paper suggesting that the cost of a director-class Fibre Channel port was around $1,500, with non-director-class ports selling for around $600. IP switches suitable for SAN functionality were roughly $300.

"While the individual numbers have changed (decreased in price across the board), the relative differences haven't changed that much," said Reichman. "This report points to a roughly 4X cost premium on Fibre Channel per server networked. Since Fibre Channel costs are up these days with the transition to 8 Gbps, I'd say that the delta remains close to that number today."

Perhaps this is why in the United States, iSCSI is slowly gaining traction against Fibre Channel, especially as organisations with reduced budgets thanks to the financial crisis find ways to cut financial corners. IDC found last year that iSCSI had gained market share, growing from 6% of the networked storage market in 2007 to 13% for the first three quarters of 2009. Fibre Channel lost almost the same percentage, dropping to 61% from 72%.

In the UK recently, the 2010 Purchasing Intentions Survey found that 41% of people reported iSCSI deployments in 2010, compared with 27% in 2008.

It may be several years before iSCSI achieves the same performance and resilience as Fibre Channel enjoys today, but for an increasing proportion of scenarios, including departmental computing, it could easily suffice as a low-cost shared-storage alternative. As such, it is well worth considering for your next storage purchase.

Read more on Storage fabric, switches and networks