Failures in an IBM DB/2 database have caused a major crisis at a leading Danish bank.
Key trading systems at Danske Bank ground to a halt on 10 March and the bank was not fully operational again until 17 March, it said in a statement released on 3 April.
IBM was unaware of four bugs in the DB/2 database, which mainly affected payments and the trading and settlement of currencies and securities, said Danske Bank.
The first of four IT problems the bank faced emerged after an electrical outage during routine disc maintenance on one of the bank's IBM disc system at its data operating installation in Ejby. This caused data inconsistencies in its DB/2 database.
In the statement Danske Bank said, "This first software error in DB2 database software had existed in all similar installations since 1997, without IBM’s knowledge."
The statement went on to say that a second software error delayed the recovery process, as several DB/2 tables could not be started, and a third error on the system prevented recovery jobs from being run simultaneously.
Yet another software bug stopped the corrected data from being reloaded back into the databases. In the statement Danske said, "This last error, which appeared on Thursday, 13 March, resulted in new episodes of inconsistent data that had to be recreated by other methods."
To avoid further delays in restarting the systems, the bank decided not to wait for IBM to issue patches and used back-up data from its operating centre in Brabrand to restart the systems.
The system was running four days after the initial failure. However, it took Danske until 17 March to clear all transactions that had accumulated while the system was not operational.
Danske said it would upgrade its GDPS emergency security system (Geographically Dispersed Parallel Sysplex) mainframe system next month which, it said, would improve disaster recovery.
The bank implemented GDPS since the autumn of 2002, which provides mirrored discs to supplement its existing two-centre IT operations set-up.
In its statement Danske said, "The group is of the opinion that the next version of GDPS, to be released in May 2003, would have prevented the effect on operations caused by the hardware error."