Grid technology has broken out of the scientific and
academic communities' labs and entered the enterprise. Global
banking group HSBC is a good example of a commercial organisation
that is wrestling with grid technologies, while harnessing their
power for everyday use.
Grid computing links together hundreds or thousands of PCs or
servers so they can carry out massive simultaneous
computations.
The grid operator can view the grid as a pool of computing
resources regardless of the location of the servers. This allows
operators to schedule or allocate jobs according to which areas of
the grid are free.
Over the past year, HSBC has been standardising its grids on
Microsoft server software and commodity server hardware. It plans
over coming years to connect up the disparate grid projects run by
its IT departments across the world.
Craig Carter, HSBC's global architect for enterprise systems,
has driven the agenda to create a standardised grid architecture
that can be replicated across its main locations, to simplify the
management side.
HSBC uses its grids to carry out derivatives trades, which rely
on making numerous calculations based on future events, and risk
analysis, which also looks to the future, calculating risks based
on available information.
By their nature, these financial instruments draw heavily on
mathematical computations.
Derivatives traders and analysts were demanding more raw
processing power so that they could calculate their trades even
more quickly.
"Our demand for grids came from the business, and we had to
react to the need to compute huge amounts of data, increasingly due
to regulatory compliance," said Carter. "Derivatives trading is
very computational, and was running on big and expensive bits of
iron, and the quality of the data wasn't very good."
The servers that HSBC was using before it established its grids
tended to be high-end IBM and Sun servers. But even these were
unable to match the demand from the traders and analysts, despite
the fact that they could deliver huge amounts of processing
power.
The company began to consider using the 14 grids it had been
developing as separate projects in Hong Kong, New York, London and
Paris.
One of the first of these was set up three years ago in Paris,
designed for interest rate derivatives.
It ran a batch processing application called LSF (Load Sharing
Facility) from Platform Computing and linked to HSBC's home-grown
Summit trading system. HSBC had to do a lot of the development
work, writing an application programming interface (API) to allow
its risk calculation program to run on the grid.
The New York grid was developed next, two years ago, with the
Hong Kong and London grids emerging a few months after that.
However, each of the four IT systems used different grid
application platforms, with the sub-grids even running different
versions of the same platform.
"The local developers came up with a solution independently,"
said Carter. "We had siloed grids running on Platform Symphony and
DataSynapse's Grid Server, and also Platform LSF. In Hong Kong
there were two grid outbreaks that didn't talk to each other, and
in Paris there was a mix. There was a lot of
non-communication."
When Carter came on board at HSBC a year and a half ago, HSBC
paused to consider its options for handing the traders and analysts
more computational power.
The options included buying much bigger HP or IBM servers,
expanding the servers they already had, or developing their grids
by adding hundreds or thousands of small commodity servers.
Going the grid route was attractive for several reasons, said
Carter. HSBC was restructuring its IT organisation and allocating
its IT managers, developers and other IT professionals to different
business groups, such as risk or derivatives. Each business unit
chief was given an IT counterpart.
At the same time, over the past two years, HSBC has spun off its
IT operations into a semi-separate company, so IT could charge the
business for the IT operations it uses.
The US wing of HSBC went live with this in 2004, with the UK
going live at the start of this year, and France planning to adopt
the model next year. "That affects the landscape - the business
wants to know what it is spending on, and to see where it can
combine IT resources," said Carter.
The idea of charging the business back for its IT operations sat
well with the model of the grid, which is based on individual
processing jobs.
Secondly, with the IT staff restructuring and arrival of the
grid standardisation programme, HSBC had the opportunity to free
its developers. They had been tied up operating, managing and
troubleshooting the grids, rather than focusing on development
work. "Developers are an expensive resource," said Carter.
Carter brought the grid teams together in 2005 to talk to each
other. "They found they had a lot in common," he said, and the
developers bought into the idea of sharing their grid knowledge for
the common good.
Working towards a common grid server build through 2005, the
developers settled on x86 processor-based servers running Microsoft
Server 2000, which offers a "light footprint" for each compute
node.
Unusually, 95% of HSBC's production grid runs on Windows rather
than Unix, or Unix flavours such as Solaris or Linux, as do many
academic and scientific grids.
The main reason for choosing Windows was that the development
and operations staff, as well as the traders, were more familiar
with Windows. Also, the main grid applications HSBC used - Sophis,
Summit and Microsoft Excel - ran mainly on Windows.
Using commodity servers with a standard build and the same API
has made server manageability straightforward because HSBC could
apply tools such as automatic provisioning to quickly ramp up the
number of servers on the grid.
The ability to scale up the size of the grid has enabled HSBC to
offer more computing power to its traders and analysts, said
Carter. "Grid enables us to do things we have not been able to do
before, within a timescale. We have not saved money but we have
made more money because we can do more things now.
"Grid is perceived as difficult by our internal people. We have
got to a point where we are trying to present it as pervasive. It
is still relatively difficult." For example, troubleshooting
problems in the grid takes time and expertise because of the size
of the computer logs.
"We have had issues with some of our independent software
suppliers, who do not produce cluster-aware software," said Carter.
"Up until the last few years, they couldn't help. When they have
said they can do it, they have come up with their own version,
which has not worked for us."
HSBC has also come up against some unexpected non-technical
issues in extending its grid, said Carter. Cross-border taxation is
one conundrum that is slowing down HSBC's attempts to link the
grids across countries.
The problem is that if a UK trader is using a UK-based grid
application internally, no VAT is chargeable. But if a French
trader is using a UK-based grid application, because the French
bank is a different legal entity, it is almost as though the trader
is using a third-party application.
The company is therefore subject to sales tax. Banks cannot
recover VAT because they do not charge VAT to operate a current
account, said Carter.
Secondly, there are data issues when sending certain data across
national boundaries, he said. For certain countries, such as
Malaysia and Switzerland, data cannot be sent out of the country
using a grid. "Regulations are a relatively new and unpleasant
requirement, though the UK is pretty free and easy," he added.
"We were trying to do a one-size-fits-all amorphous blob. The
goal is not to worry where your compute resource is. We can do this
in the next 12 months, but we are let down by the regulations and
sales taxes we have discovered."
Nevertheless, Carter is planning the next stage of the grid
programme, which is to concentrate on developing a uniform grid in
each country. The UK and France are on track, and are the most
likely to be linked together first.
Carter is also evaluating multicore processors, saying that
grids make good use of them. Multicore chips have two or more
processing units per chip. A server with two single-core CPUs
typically runs two instances of a grid agent - one for each
chip.
With dual-core chips, four instances can be run, using
hyperthreading technology. "This pretty much quadruples the
workload. The ability to use those threads is the difficult bit -
but we are developing it. It is very complex," said Carter.
Moving forward, the bank plans to extend the use of its grids
more widely, to calculate more interest rate, equity and credit
derivatives, and credit risk management and foreign exchange
options. The demands of the business will continue to drive the
need to grow the grid, said Carter.
Grid computing is no longer the preserve of scientific number
crunchers. Arif Mohamed investigates how business needs are driving
grid development at international bank HSBC
The technology behind HSBC's grids
HSBC has grid-enabled more than 3,500 CPUs in server grids in
four countries and it is working to unite its grids across the
world.
During 2005, HSBC grid-enabled 750 CPUs for interest-rate
derivatives and FX options in the City of London. It also
grid-enabled 1,200 CPUs in New York.
Each of the grid networks is dedicated to a single process, or a
third-party software application such as Sophis, Summit, Microsoft
Excel or Fast Crisp.
The applications mainly run on Windows Server 2000, running grid
applications from Platform Computing (LSF and Symphony) and
DataSynapse (Grid Server).
Some 95% of the production grid (HSBC also runs test grids) runs
on Windows, but 5% - a couple of hundred servers - run on Linux
servers.
These machines run a risk management application that runs only
on Linux.
What is a derivative?
A derivative is a financial instrument whose value depends on
another asset such as a share. A derivative can be traded and could
take the form, for example, of an option to buy or sell a share at
a future date for a specified sum.
Vote for your IT greats
Who have been the most influential people in IT in the past 40
years? The greatest organisations? The best hardware and software
technologies? As part of Computer Weekly’s 40th anniversary
celebrations, we are asking our readers who and what has really
made a difference?
Vote now at:
www.computerweekly.com/ITgreats