IP extends its reach in data storage

It is 15 years since internet pioneerVint Cerftore off his shirt to reveal a T-shirt bearing the slogan "IP on everything" at a meeting of theInternet Engineering Task Force. But while Internet Protocol has become a de facto standard for voice and data traffic in general networking, it has failed to make an impression on the ramparts ofstorage networking.

Now, this could be starting to change, with IP- and Ethernet-based protocols increasing their tiny share of the storage networking market in the past year.

According to analyst firm IDC, the western European storage market was worth £3.7bn in 2006, of which £38.9m was accounted for by iSCSI (Internet Small Computer System Interface) storage area networks (Sans). Average annual growth in total storage spend is expected to be less than 2% a year between 2005 and 2010, but IDC predicts a 69% compound annual growth rate for iSCSI Sans in that period.

By 2010, IDC estimates that iSCSI will account for roughly 8% of total storage spend and 15% of spending on Sans in western Europe, compared with 1% and 3% in 2006.

iSCSI is the most prominent of the IP-based storage networking protocols, accounting for more than 95% of the IP storage market.

Fibre Channelversus IP

The gorilla in the storage networking market is still Fibre Channel. It is, as the name suggests, a hardware-intensive "channel" method of communicating. This means it is eminently suitable for storage traffic, because it is a more direct means of transport than networking, being to a greater extent point-to-point.

That is important because it means Fibre Channel can provide faster network speeds and greater integrity of data with fewer overheads than IP, so it is well suited to the transfer of large amounts of valuable data.

However, Fibre Channel does have its drawbacks. The key one is that there are limits to the physical distance over which a storage network can be connected.

In an age when businesses are becoming increasingly global, there is a growing need to be able to link storage resources across thousands of kilometres. ­Fibre Channel has a limit of between 10km and 100km.

Distance also comes into play where firms want to maximise utilisation of storage resources by making branch office systems available to the wider network.

That is where IP came in - allied with the ubiquitous SCSI device connection method - to create iSCSI, says Ian Bond, senior consulting systems architect at network supplier Cisco.

iSCSI takes SCSI commands, which are usually encapsulated in Fibre Channel frames, and puts them inside IP packets.

"iSCSI arose from the question, 'Can I run storage protocols but instead of using Fibre Channel can I use IP?' If you have servers that are remote from existing storage networks and you want to extend the San, but perhaps distance or the cost of Fibre Channel host bus adapters is a factor, iSCSI is a natural choice," says Bond.

"Instead of buying expensive Fibre Channel, you can opt for Gigabit Ethernet and a network interface card. You typically find it where people have distributed storage resources, without massive traffic, that they want to access data from."

Extending Sans with iSCSI

As the IDC figures suggest, iSCSI is a good choice for businesses that want to extend the reach of their storage networks at a cost that is a fraction of using Fibre Channel, in terms of both equipment and skills.

Dennis Szubert, principal analyst at Quocirca, says, "iSCSI provides greater benefits than just running over Ethernet. iSCSI and the IP are much more virtualised than Fibre Channel, making it much easier to manage and scale."

Aad Dekkers, director at the Storage Networking Industry Association (Europe), says, "iSCSI allows you to build Sans with commodity components. It is good for those companies that want to deploy a solid solution but do not want to invest in the skills necessary for Fibre Channel. The benefit is that storage resources can be shared more easily."

One company that has benefited from iSCSI storage networking is Foster's EMEA, which distributes more than 20 Australian wine brands across 32 countries in Europe. The company has a rapidly growing mountain of data to store, and using storage within individual servers was proving difficult. As more and more remote workers were equipped with Micro­soft Exchange and Blackberry e-mail services, the business needed to expand its storage resources.

Technology services manager Ken Kaban says Foster's wanted to go down the San route, but it was concerned about the cost of Fibre Channel.

"To support these workers effectively, we needed to expand our storage considerably. At the same time, any system we installed would have to be easy to manage and expand in future. We knew that Fibre Channel would require a huge level of investment from us to get the right results. The training and additional network expenditure would have made it prohibitively expensive," he says.

Kaban opted for iSCSI-based San arrays from EqualLogic. Besides savings on equipment and skills compared with Fibre Channel-based products, Foster's has cut down on storage management time.

"Overall, I estimate that the EqualLogic array saves about 10 hours of staff time per week, which would otherwise be dedicated to simple management tasks. Removing this overhead has meant that we can get on with providing higher value to the business as a whole," he says.

FCIP and iFCP

Beyond iSCSI, other applications of IP to storage networking are also making important inroads. iFCP (Internet Fibre Channel Protocol) and FCIP (Fibre Channel over IP) allow users to do radically new things with Fibre Channel-based storage networking.

By terminating the Fibre Channel session at the iFCP gateway and running storage traffic over protocols designed to work at global levels, iFCP breaks the distance barrier of traditional Fibre Channel networks and improves scalability and manageability.

Both protocols encapsulate Fibre Channel frames into IP packets, but the operation and application of the protocols is different.

FCIP is a tunnelling protocol that allows a physical Fibre Channel connection to be extended across an IP network, which in effect enables two separate Fibre Channel fabrics at different locations to form one large fabric. As you add more sites, the fabric is stretched further.

Bond says, "FCIP is being adopted to move data in production environments where you need to link a number of sites globally. More specifically, if you want to move storage between datacentres that are, say, 150km apart, you can run into problems with native Fibre Channel, which does not have the range. Even at distances below this, Fibre Channel may not be an option if the optical plant is not there, as is the case in some parts of the world."

Cisco's IT organisation uses FCIP to move data between three sites - west coast US, east coast US and Amsterdam.

"Those are distances of 3,000km or more, but using FCIP we can asynchronously copy data between sites - it is a classic case. It is too far to run anything but IP," says Bond.

By contrast, iFCP maps unique IP addresses to individual Fibre Channel devices and, because each device has its own identity in the IP network, it can individually send and receive storage traffic to and from other devices on the network. Each iFCP gateway domain is effectively an autonomous system that is invisible to the IP network and other iFCP gateway domains.

Because each such Fibre Channel fabric is independent, the system can be built from different suppliers' equipment.

Greg Schultz, founder and senior analyst at analyst firm Storage I/O, says, "iFCP is similar to FCIP but with two key differences. With iFCP, you keep separate, logical Sans unlike FCIP, where you consolidate separate resources into one giant San. Also, iFCP is compatible with open systems but not mainframes, which FCIP is."

Fibre Channel over Ethernet

IP, as a network technology, is software-intensive and therefore comes with overheads that the channel-like Fibre Channel does not. Such overheads can be a concern, and to obviate this a new standard - Fibre Channel over Ethernet (FCoE) - is being formulated.

FCoE, which is about 18 months from standard ratification, misses out the network and transport layers used by IP and TCP and encapsulates Fibre Channel frames directly into Ethernet at layer 2.

As well as giving the advantage of distance lacking in Fibre Channel, FCoE could also save datacentres a lot of time and money by doing away with the need for separate Ethernet and Fibre Channel physical connections on devices. Also, the potential is there for the same network to host multiple forms of storage and other network traffic.

FCoE also offers the possibility of deterministic networking (a feature of Fibre Channel, but not IP) in which packet transfer is guaranteed within a timeframe - an essential feature for mission-critical traffic.

Simon Pamplin, senior systems engineer at network supplier Brocade, says, "FCoE is defining a new standard for storage, putting it over Ethernet and using its deterministic nature.

"IP's approach is to send lots of information and to keep resending it if some does not get through. This is OK for much general networking, but not for storage, where entire read and write sequences must be sent correctly."