Oil giant Shell has already deployed a Linux supercomputer cluster, as has counterpart WesternGeco, indicating the open source operating system's arrival in the petrochemical arena. Industry experts are predicting further growth in Linux's popularity and that it will become a serious supercomputing contender in the not-too-distant future.
Rob Gear, an analyst at research organisation Ovum, also thinks Linux will adopt a more dominant position at the high-end this year.
"This is supposedly the year where Linux will make more of a challenge on the enterprise," Gear explains. "And I can't see any reason why it would create any problems. It's not really that hard a task to put together a computing cluster. Essentially it's a group of computers linked together with an Ethernet cable."
IBM is one of the IT companies that have invested massively in Linux. Andy Hoiles, Linux business manager for EMEA Northern region at IBM, believes the Linux supercomputer cluster is on the verge of explosion. "As we go through the year, we'll see more and more of the world's top 1,000 supercomputers running Linux," he says.
The supercomputer can take two forms. It can either be one very large box housing myriad processors and computing power or a cluster of smaller machines, linked together. The Linux supercomputer takes the form of a cluster and generally runs on Intel processor-based machines in a parallel environment.
The main advantage of opting for a Linux supercomputer over a traditional Unix - or similarly high-end-system is cost. IBM's Hoiles estimates the average cost saving to be around 50% but points out that figure could be higher in some instances.
It has been argued that a Linux supercomputer will not perform as well as other non-Linux-based systems because the operating system has not been designed for high intensity computing environments.
This was not found to be a problem at Inpharmatica, a drug discovery organisation that has successfully deployed a pure Linux supercomputer cluster. Describing the setup at Inpharmatica Pat Leach, chief information officer at the company, says: "With cluster computing you have a very large number of cheap computers; you design the architecture so it doesn't matter if some of them break down."
Inpharmatica has two supercomputer systems, including a pure Linux cluster residing at its central London headquarters. The Linux cluster is used for drug discovery and protein analysis of the huge amounts of new data generated by the Human Genome Project. The Linux supercomputer cluster, called Biopendium, was designed, built and implemented by Inpharmatica and systems integrator Northgate. It consists of 1,200 processors, which includes 500 dual Pentium III 700MHz processors, 1Gbyte of RAM and a Sun E4500 server, running Debian Linux, supported by 15Tbytes of storage.
Leach estimates that the system, including storage, cost between £3 and £4m. "The primary reason we went with Linux was one of cost," Leach says. "We could have bought a supercomputer from IBM or Sun or Cray, but the economies didn't make sense."
When Inpharmatica considered investing in the Linux supercomputer three years ago it had no real-life scenarios to follow. But it ploughed ahead relying on its strong academic background and technical abilities. In fact 70 of Inpharmatica's 100 staff have technical expertise, and the company was confident it could build a successful Linux supercomputer cluster.
"Linux has a significant benefit [over other operating systems] because it uses free and open source code, so any problems can be solved very quickly," Leach says. But the deployment was largely problem free. "The biggest problem we had was one of scalability; how do you get 500 computers to link together?" However, Leach adds, other problems arose relating to the physical practicalities of the cluster such as how to keep the machines cool enough
Leach believes we will see many more Linux supercomputing systems coming into the spotlight this year. Financial services organisations are likely to be among the first commercial companies to adopt such systems, as are organisations with large data mining requirements.
Undoubtedly, the opportunity to run Linux in a supercomputer configuration has arrived. However Inpharmatica's Leach points out that it is not something to be rushed into:
"Find someone who has done it before, and when you go to a vendor make sure they know what they're doing. And make sure you examine the experience of the end user," he advises.
This was first published in January 2002