British Airways has revealed the full extent of the IT problems that marred Terminal 5's opening on 27 March this year.
Written evidence submitted to the Transport Select Committee shows that a myriad of problems struck during the terminal's first few days, and these were exacerbated by the way in which BA's IT staff handled the problems.
Terminal 5 is one of the most technologically advanced airport terminals in the world, but MPs described its opening as a "national humiliation". During the first five days, BA misplaced more than 23,000 bags, cancelled 500 flights and made losses of £16m.
The Transport Select Committee called in Willie Walsh, BA's chief executive, Colin Matthews, airport-owner BAA's chief executive, and Nigel Rudd, a non-executive chairman of the board, to face some tough questions earlier this month.
Willie Walsh revealed that IT problems and a lack of testing played a large part in the trouble. But he said the airline could have coped if IT had been the only issue.
He reeled off a list of failures. Staff had not been trained properly, they were unable to park when their car parks became too full on the day of opening, staff security searches were delayed, and construction work on parts of the building was not finished when the airport opened. Out of 275 lifts, 28 were not working - 17 are still broken, with 11 still to be fixed by the end of May.
"It was a combination of factors," Walsh said. "We could have coped with a couple of the problems. But all of the problems hitting us led to a cascade."
British Airways' written evidence showed how many IT problems staff had to contend with. To begin with, loading staff could not sign on to the baggage-reconciliation system. They had to reconcile bags manually, causing flight delays. Problems with the wireless Lan at some check-in stands meant that staff could not enter information on bags into the system using their handheld devices.
During testing on the baggage system, technicians installed software filters in the baggage system. Their job was to prevent specimen messages generated by the baggage system during the tests being delivered to the "live" systems elsewhere in Heathrow. But they were accidently left in place after the terminal opened.
As a result, the Terminal 5 system did not receive information about bags transferring to British Airways from other airlines. The unrecognised bags were automatically sent for manual sorting in the terminal's storage facility.
An "incorrect configuration" stopped the feed of data from the baggage-handling system to the baggage reconciliation system. On Saturday 5 April - a week and a half after opening - the reconciliation system failed for the whole day. Bags missed their flights because the faulty system told staff that they had not been security screened.
There were errors in the transmission of BA flight data between BAA and communications and IT contractor SITA. As a result, the system did not recognise a proportion of the bags, which were held within the T5 baggage system for manual processing. A lack of server capacity at the terminal excacerbated the problems, Walsh told the inquiry.
As these errors built up, more bags went unrecognised by the system, missed their flights, or had to re-booked on new flights. The baggage-handling system froze after becoming unable to cope with the number of messages generated by re-booking flights, forcing managers to switch off the automated re-booking system.
By 5pm on the first day of opening, British Airways could no longer accept checked baggage. It told passengers in the departure lounge they would be leaving without their luggage. Anyone who had not yet checked in could choose between travelling without baggage or re-booking their flight. Staff took unrecognised bags out of the system and sorted them manually - this happened every day until 31 March, during which time a total of 23,205 bags had to be manually sorted.
Walsh said BA's IT staff finally removed the software filters on 31 March, four days after opening. This was not the only IT problem to continue for a few more days. The airline was forced to cancel hundreds of flights as it struggled to clear the baggage backlog and work out why the systems were failing.
"I believed at the time of the opening that the move would be successful," Walsh told MPs. "I do regret it, but I have not actually met anyone who was able to predict the particular problems we would encounter."
BA puts the failure to spot the IT issues down to inadequate system testing, caused by delays to BAA's construction work. Construction work was scheduled to finish on 17 September last year. The delays meant BA IT staff could not start testing until 31 October. Several trials had to be cancelled, and BA had to reduce the scope of system trials because testing staff were unable to access the entire Terminal 5 site.
BAA was less keen to attribute blame, or explain in detail what went wrong. Matthews told MPs that he had not made the time available to investigate the problems surrounding Terminal 5's opening because he had spent his time sorting them out. His answers did not go down well with the MPs, who accused him of complacency.
"I made a very specific decision that, rather than take the time for operations and technical experts to brief me on this appearance, I concluded that my responsibility was to fix issues with passengers," he said.
Most of Terminal 5's problems have now been ironed out, and BA staff are working to resolve those that still exist. BA has postponed the transfer of long-haul flights from Terminal 4. Originally planned for 30 April, the move now will not happen until at least early June.
"Clearly our reputation has been damaged, but I am satisfied that we understand around 95% of the issues that led to our problems," Walsh said. "We are now working very hard to demonstrate that Terminal 5 is and can be a fantastic success."