Jack of all trades, master of none?
- Posted:
- 15:09 13 Aug 2002
Back in the early 90s, an Informix marketing executive made a presentation to some gathered journalists predicting the death of the mainframe by 1999. In fact, not only is the mainframe alive and well, but some technology concepts that emerged from the mainframe market are now being proposed for use in more distributed data centre applications. Clearly, new dogs are learning old tricks.
One of the most high-profile technologies using mainframe-like principles is Infiniband, a server connectivity architecture which vendors are hoping will dramatically improve the way that servers talk to other equipment in the data centre. Although this technology is relatively new (the Infiniband Trade Association was formed in October 1999), it uses the old mainframe principle of divorcing I/O processing from a server's central processing unit.
ADVERTISEMENT
The companies steering the ITA - Compaq, Dell, Hewlett-Packard, IBM, Intel, Microsoft and Sun Microsystems - wanted a new technology that would enhance the way that servers connected with peripheral devices such as storage arrays and with other servers. Previous data centre connectivity architectures have significant drawbacks, according to the ITA's collection of white papers. One such drawback is that many of them were bus-based, whereas Infiniband is fabric-based. Bus-based architectures use a single data 'highway' that all devices share. Such data connections can become congested, meaning that the number of devices using them can be limited and bandwidth is constrained. With server processing power increasing exponentially, this can cause problems.
Switched fabric
Infiniband works by using a switched fabric technique, in which a switch or collection of switches connects devices into the server on a point-to-point basis. The necessary I/O processing power takes place outside the server, rather than inside the box, and this has several other advantages. Instead of sending all information to a single server, an Infiniband switched fabric can take input from multiple external devices and send them to any servers attached to the Infiniband fabric.
Two key components of the Infiniband fabric are the host channel adaptor and the target channel adaptor. These are hardware interfaces used by the server and by external devices to connect to the Infiniband fabric. The HCA sits inside the server, generally on the motherboard, whereas the TCA sits on the external device's I/O controller on the other side of the fabric. They talk to each other using packets defined in IPv6, the latest version of the Internet protocol.
Infiniband networks provide a bandwidth of between 2.5Gbit/sec and 30Gbit/sec, depending on whether you use a 1x, 4x or 12x link as defined in the Infiniband specification. An Infiniband link can stretch for up to 17m using copper cable, and the distance rises to kilometres when running Infiniband over fibre.
Varied applications
The applications for Infiniband are varied, according to Brice Clark, director of strategy and business planning for Hewlett-Packard's Procurve range of switches and hubs. One attractive possibility is to use Infiniband as a single connectivity architecture to handle I/O between the server and other devices using other protocols. He envisages a scenario in which an Infiniband fabric is used to connect a server to other devices that are using a mixture of fibre channel and gigabit Ethernet connections, for example. This would give the other devices the benefit of the switched architecture, while also minimising the amount of cabling spaghetti flowing directly into the server.
The 17m limit for copper-based Infiniband connections is also enough to make it viable for storage applications, according to Clark. A white paper from Mellanox, a member of the ITA, also highlights storage area networks (SANs) as a suitable application for Infiniband. Generally, fibre channel or high-speed Ethernet connections are used to build SANs, which are collections of storage arrays communicating through their own high-speed matrix of switches in much the same way as an Infiniband fabric is structured. An Infiniband SAN fabric could increase bandwidth over fibre channel, which peaks out at around 4.24Gbit/sec.
Clustering potential
Clustering is the other area in which the technology shows some potential. Geoff Poskitt, a systems architect at Fujitsu Siemens, likes the concept of Infiniband in a clustering environment because of the technical qualities of the architecture. Ethernet clusters can run into problems because of Ethernet's carrier sense with multiple access/collision detection (CSMA/CD) framework, in which packets are sent across the network in the hope that they don't collide with other packets. If they do collide, then they are resent. The more traffic on the network, the more likely these collisions are to occur and the slower network traffic will be.
"Clustering doesn't really have an industry standard at the moment," Poskitt explains. "There are some loose clusters that work over fast Ethernet. But there are other clustering configurations that carry low message latency requirements. With Ethernet, you can get high latency."
Not everyone agrees with him. Geoff Barrall, CTO for network attached storage vendor BlueArc, explains that his company was planning to roll out Infiniband-capable products, but that it decided to put its implementation of the technology on ice following what it perceived as a general cooling-off among vendors. "We've put it on hold now until we can see survival for Infiniband and until we can see some form of adoption for it," he explains, adding that while Infiniband may be a low-latency product, Ethernet is getting so fast now that such considerations are often irrelevant. "It would be possible to do the same things in [the clustering] space with Ethernet. Infiniband has a lot of strong competition now from Ethernet. Historically, Ethernet has always done well. It beat off Token Ring, FDDI, ATM and more recently it's been competing with fibre channel."
Blade servers
With clustering and SANs taken care of by technologies such as fibre channel and Ethernet, the most promising opportunity for Infiniband is in a new area - blade servers. This product category, which we discussed in the 9 July issue of MicroScope, focuses on server density and offers an alternative to traditional rack-mounted servers at the low end of the market. A blade server has a tightly packed chassis in which tens of 'blades' - essentially servers on cards - can be slotted, increasing the processing power per square foot ratio. Vendors argue that this is an increasingly important issue for companies which have to pay steep rental costs for space in specialist data centres.
Several companies are planning blade servers, although it's not clear that all of them will be using Infiniband, at least in the first instance. Fujitsu Siemens, for example, has launched some products in this category, but its products are aimed at front-end applications such as e-mail servers and are based on gigabit Ethernet. Poskitt says the company will expand into the middle tier next year, and that the next generation of blade servers will be "full performance". "A lot of people are saying that Infiniband products will start appearing in the second half of next year, or 2004," he predicts, adding that blades will have an Infiniband chip on them that plugs into a backplane. An Infiniband switch will then connect a group of blades together, bringing us back to the clustered scenario again. "The key thing to remember is that this doesn't make sense in a single scenario. You'd need a lot of shared servers to want to do that," he argues.
Too risky
None of this makes things very easy for companies such as BlueArc, which doesn't want to take a leap of faith into Infiniband until it's sure that it has a supplier that's going to stay the distance. Barrall doesn't want to start using a vendor which then pulls out of Infiniband or goes out of business, leaving it unable to get parts. The problem is that he doesn't think the market for blade servers is enough to support all the Infiniband vendors, making the whole thing too risky for him at present.
Barrall isn't the only person who's concerned about Infiniband. Nigel Lambert, marketing manager for storage equipment distributor Zycko, is equally cynical. "No mainstream product has its eye on Infiniband. It's going to be high-end and niche," he says, adding that his company understands gigabit Ethernet because it's built on well-established standards, and that it's also working with fibre channel, another tried and tested technology. "The channel is there to serve the needs of the mainstream."
If it were just small players such as BlueArc and Zycko that were anti-Infiniband, perhaps the ITA wouldn't have quite so much to worry about, but Infiniband has suffered more significant knocks. In particular, Intel announced in May that it was not going to be making host channel adaptors for the volume market, contrary to earlier plans. Instead, it would leave that to third parties such as Mellanox. This did nothing to encourage enthusiasm in the market.
Diluted message
Infiniband sounds like a no-brainer, technically speaking, so what gives? One problem, according to HP's Clark, is that Infiniband tried to be all things to all men, and has diluted its message in the process. "It's better to focus on a technology than trying to make it all things to all people," he says, arguing that people have already been proposing that the technology be used as a LAN transport mechanism before it has even met its primary objective. "Infiniband hasn't fulfilled anything yet, even as an I/O system. In fact it's taken so long that other I/O alternatives have shown up. The way it ends up when you try to do everything at once is that nothing happens."
He says that Infiniband could end up suffering the same fate as ATM, the technology that tried to serve the workgroup, LAN backbone and WAN backbone markets in the latter half of the 90s, and which he says is now being replaced with gigabit Ethernet installations across the board. The reason that Ethernet has won out over so many other technologies is that it builds on strong foundations, establishing itself in one market before attacking another, rather than trying to capture all markets at once, he argues.
Moving goalposts
Perhaps one of the biggest problems for Infiniband is that because the lead time for the technology has been relatively long, the market goalposts for the technology are moving. Infiniband was originally designed to offer an alternative server connection to the PCI bus, which has been the primary means of connecting external devices to servers and PCs for the past decade. Initially, host channel adaptors would be based on PCI cards, but eventually they would be provided as native implementations. The problem is that since then, PCI technology has evolved, leading to an extended lifespan for this veteran bus technology.
The PCI-Express and PCI-X 2.0 specifications that were ratified on 23 July by the PCI-SIG - the consortium that develops and promotes the PCI standard - dramatically enhance PCI's speed. The most significant of these two standards for the Infiniband market is PCI-X 2.0, because it focuses more on external connectivity, whereas PCI-Express (formerly known as 3GIO), is concerned primarily with connecting components that reside inside the PC, such as graphics subsystems.
PCI-X 2.0 is the latest in a series of evolutions of the PCI standard. Literature on the PCI-SIG Web site explains that the latest version of the specification offers speeds of up to 533MHz, compared with the original PCI's 33MHz specification. The PCI-SIG explains that it will be a complementary architecture to Infiniband and will be able to connect to Infiniband fabrics outside the box.
What's interesting is how PCI-X 2.0 is putting the squeeze on Infiniband at the server I/O end of the equation. Whereas Infiniband was intended to replace PCI, companies now have an alternative technology based on a well-established I/O mechanism. This puts Infiniband under the same pressure at the server I/O level as it does at the external connectivity level, where it faces similar pressure from gigabit Ethernet.
Lukewarm support
Given this reduction in the scope of Infiniband at all levels, the future of the technology is far from certain, and the lukewarm support from major players such as Intel isn't helping. Systems integrators and VARs, faced with customers who simply want to get the job done, may find Infiniband a hard sell, especially as many products haven't even appeared yet. Moreover, they will likely have more experience with technologies such as gigabit Ethernet and perhaps fibre channel.
Infiniband has an impressive name, but it could end up being a niche technology for very high-performance applications. There is doubtless an opportunity for vendors and channel partners to make a profit, but it isn't as large as the ITA may initially have hoped.
Further information
www.bluearc.com
www.fujitsusiemens.co.uk
www.hp.com
www.pci.sig.com
www.zycko.com
