RBS splits problem systems to reduce IT risks

RBS has split the IT system behind its high-profile outages into three to reduce the risks of a single failure impacting operations

Royal Bank of Scotland (RBS) has split the IT system behind its high-profile outages into three to reduce the risks of a single failure impacting its operations - part of a £750m investment to improve its troubled IT.

In its financial statement for first quarter of 2014, where the state-owned bank announced a pre-tax profit of £1.6bn, the bank described the action it has taken to avoid a repeat of the disastrous IT glitch that left customers unable to access accounts.

In July 2012, RBS, NatWest and the Ulster Bank customers could not access their accounts for days, as a result of software issues. The glitch in the CA7 batch process scheduler ended with 12 million customer accounts being frozen. Customers could not access funds for a week or more as RBS, NatWest and the Ulster Bank manually updated account balances.

RBS has split the single batch scheduler that supported banking services at NatWest, Ulster Bank Northern Ireland and Ulster Bank Republic of Ireland onto three dedicated versions. It added that RBS already runs in a separate scheduler environment.

The bank stated: "[Splitting]...the batch schedulers means that, if a problem occurs with transactions on one of these brands, it will not impact the activity taking place to support the other two, avoiding a repeat of the 2012 system."

RBS said that the work splitting the batch schedulers was part of a wider a programme to simplify the organisation. "This includes investment of around £750m over a three-year period to improve the safety, security and resilience of our IT systems."

In February RBS said it will reduce the number of technology platforms by over 50%, reduce the number of core banking systems from 50 to 10 and cut the number of payment systems it uses from 80 to 10. 

Before the credit crunch RBS went through a stage of growth through acquisition and, as a result, the bank has many IT systems with significant overlap.

"This simplification is intended to deliver significant improvements to services delivered to our customers but, at the same time, serves as the cornerstone of a programme designed to bring our cost base down from £13.3bn in 2013 to £8bn," said RBS.

In December 2013 RBS CEO Ross McEwan said the failure of systems was unacceptable and blamed years of underinvestment in IT for the issues.  "For decades, RBS failed to invest properly in its systems. We need to put our customers' needs at the centre of all we do. It will take time, but we are investing heavily in building IT systems our customers can rely on," he said.

One IT professional, with good inside knowledge of the RBS technology, said outsourcing was at the heart of the the glitch in the CA7 batch process scheduler.

"The CA7 incident happened barely a week after the UK experts had been laid off and [concerned] a standard system upgrade," he said. "In the end it was such a mess that the original experts had to be recalled to sort it out on high pay rates."

The expert did not blame the Indian outsourcer's staff, but the fact that there was a tight timescale to hand over IT operations: "It was impossible to devolve 30 years of experience in the few months of hand-over. Cross-training was attempted, but with the throughput of people in India [as fast as they were trained, many went off to get better jobs somewhere else], all the intellectual capital was lost."

Read more on Offshore IT services