How grid power pays off for HSBC

Grid technology has broken out of the scientific and academic communities' labs and entered the enterprise.

Grid technology has broken out of the scientific and academic communities' labs and entered the enterprise. Global banking group HSBC is a good example of a commercial organisation that is wrestling with grid technologies, while harnessing their power for everyday use.

Grid computing links together hundreds or thousands of PCs or servers so they can carry out massive simultaneous computations.

The grid operator can view the grid as a pool of computing resources regardless of the location of the servers. This allows operators to schedule or allocate jobs according to which areas of the grid are free.

Over the past year, HSBC has been standardising its grids on Microsoft server software and commodity server hardware. It plans over coming years to connect up the disparate grid projects run by its IT departments across the world.

Craig Carter, HSBC's global architect for enterprise systems, has driven the agenda to create a standardised grid architecture that can be replicated across its main locations, to simplify the management side.

HSBC uses its grids to carry out derivatives trades, which rely on making numerous calculations based on future events, and risk analysis, which also looks to the future, calculating risks based on available information.

By their nature, these financial instruments draw heavily on mathematical computations.

Derivatives traders and analysts were demanding more raw processing power so that they could calculate their trades even more quickly.

"Our demand for grids came from the business, and we had to react to the need to compute huge amounts of data, increasingly due to regulatory compliance," said Carter. "Derivatives trading is very computational, and was running on big and expensive bits of iron, and the quality of the data wasn't very good."

The servers that HSBC was using before it established its grids tended to be high-end IBM and Sun servers. But even these were unable to match the demand from the traders and analysts, despite the fact that they could deliver huge amounts of processing power.

The company began to consider using the 14 grids it had been developing as separate projects in Hong Kong, New York, London and Paris.

One of the first of these was set up three years ago in Paris, designed for interest rate derivatives.

It ran a batch processing application called LSF (Load Sharing Facility) from Platform Computing and linked to HSBC's home-grown Summit trading system. HSBC had to do a lot of the development work, writing an application programming interface (API) to allow its risk calculation program to run on the grid.

The New York grid was developed next, two years ago, with the Hong Kong and London grids emerging a few months after that. However, each of the four IT systems used different grid application platforms, with the sub-grids even running different versions of the same platform.

"The local developers came up with a solution independently," said Carter. "We had siloed grids running on Platform Symphony and DataSynapse's Grid Server, and also Platform LSF. In Hong Kong there were two grid outbreaks that didn't talk to each other, and in Paris there was a mix. There was a lot of non-communication."

When Carter came on board at HSBC a year and a half ago, HSBC paused to consider its options for handing the traders and analysts more computational power.

The options included buying much bigger HP or IBM servers, expanding the servers they already had, or developing their grids by adding hundreds or thousands of small commodity servers.

Going the grid route was attractive for several reasons, said Carter. HSBC was restructuring its IT organisation and allocating its IT managers, developers and other IT professionals to different business groups, such as risk or derivatives. Each business unit chief was given an IT counterpart.

At the same time, over the past two years, HSBC has spun off its IT operations into a semi-separate company, so IT could charge the business for the IT operations it uses.

The US wing of HSBC went live with this in 2004, with the UK going live at the start of this year, and France planning to adopt the model next year. "That affects the landscape - the business wants to know what it is spending on, and to see where it can combine IT resources," said Carter.

The idea of charging the business back for its IT operations sat well with the model of the grid, which is based on individual processing jobs.

Secondly, with the IT staff restructuring and arrival of the grid standardisation programme, HSBC had the opportunity to free its developers. They had been tied up operating, managing and troubleshooting the grids, rather than focusing on development work. "Developers are an expensive resource," said Carter.

Carter brought the grid teams together in 2005 to talk to each other. "They found they had a lot in common," he said, and the developers bought into the idea of sharing their grid knowledge for the common good.

Working towards a common grid server build through 2005, the developers settled on x86 processor-based servers running Microsoft Server 2000, which offers a "light footprint" for each compute node.

Unusually, 95% of HSBC's production grid runs on Windows rather than Unix, or Unix flavours such as Solaris or Linux, as do many academic and scientific grids.

The main reason for choosing Windows was that the development and operations staff, as well as the traders, were more familiar with Windows. Also, the main grid applications HSBC used - Sophis, Summit and Microsoft Excel - ran mainly on Windows.

Using commodity servers with a standard build and the same API has made server manageability straightforward because HSBC could apply tools such as automatic provisioning to quickly ramp up the number of ­servers on the grid.

The ability to scale up the size of the grid has enabled HSBC to offer more computing power to its traders and analysts, said Carter. "Grid enables us to do things we have not been able to do before, within a timescale. We have not saved money but we have made more money because we can do more things now.

"Grid is perceived as difficult by our internal people. We have got to a point where we are trying to present it as pervasive. It is still relatively difficult." For example, troubleshooting problems in the grid takes time and expertise because of the size of the computer logs.

"We have had issues with some of our independent software suppliers, who do not produce cluster-aware software," said Carter. "Up until the last few years, they couldn't help. When they have said they can do it, they have come up with their own version, which has not worked for us."

HSBC has also come up against some unexpected non-technical issues in extending its grid, said Carter. Cross-border taxation is one conundrum that is slowing down HSBC's attempts to link the grids across countries.

The problem is that if a UK trader is using a UK-based grid application internally, no VAT is chargeable. But if a French trader is using a UK-based grid application, because the French bank is a different legal entity, it is almost as though the trader is using a third-party application.

The company is therefore subject to sales tax. Banks cannot recover VAT because they do not charge VAT to operate a current account, said Carter.

Secondly, there are data issues when sending certain data across national boundaries, he said. For certain countries, such as Malaysia and Switzerland, data cannot be sent out of the country using a grid. "Regulations are a relatively new and unpleasant requirement, though the UK is pretty free and easy," he added.

"We were trying to do a one-size-fits-all amorphous blob. The goal is not to worry where your compute resource is. We can do this in the next 12 months, but we are let down by the regulations and sales taxes we have discovered."

Nevertheless, Carter is planning the next stage of the grid programme, which is to concentrate on developing a uniform grid in each country. The UK and France are on track, and are the most likely to be linked together first.

Carter is also evaluating multicore processors, saying that grids make good use of them. Multicore chips have two or more processing units per chip. A server with two single-core CPUs typically runs two instances of a grid agent - one for each chip.

With dual-core chips, four instances can be run, using hyperthreading technology. "This pretty much quadruples the workload. The ability to use those threads is the difficult bit - but we are developing it. It is very complex," said Carter.

Moving forward, the bank plans to extend the use of its grids more widely, to calculate more interest rate, equity and credit derivatives, and credit risk management and foreign exchange options. The demands of the business will continue to drive the need to grow the grid, said Carter.

Grid computing is no longer the preserve of scientific number crunchers. Arif Mohamed investigates how business needs are driving grid development at international bank HSBC

The technology behind HSBC's grids

HSBC has grid-enabled more than 3,500 CPUs in server grids in four countries and it is working to unite its grids across the world.

During 2005, HSBC grid-enabled 750 CPUs for interest-rate derivatives and FX options in the City of London. It also grid-enabled 1,200 CPUs in New York.

Each of the grid networks is dedicated to a single process, or a third-party software application such as Sophis, Summit, Microsoft Excel or Fast Crisp.

The applications mainly run on Windows Server 2000, running grid applications from Platform Computing (LSF and Symphony) and DataSynapse (Grid Server).

Some 95% of the production grid (HSBC also runs test grids) runs on Windows, but 5% - a couple of hundred servers - run on Linux servers.

These machines run a risk management application that runs only on Linux.

What is a derivative?

A derivative is a financial instrument whose value depends on another asset such as a share. A derivative can be traded and could take the form, for example, of an option to buy or sell a share at a future date for a specified sum.


Vote for your IT greats

Who have been the most influential people in IT in the past 40 years? The greatest organisations? The best hardware and software technologies? As part of Computer Weekly’s 40th anniversary celebrations, we are asking our readers who and what has really made a difference?

Vote now at:

Read more on PC hardware