A new report has highlighted the dangers of accident investigators
relying on suppliers to establish the culpability of their own
software. Tony Collins reports
Tony Cable's task as an independent investigator seemed
perfectly straightforward. His job was similar to that of a
forensic IT auditor. He was to establish what role, if any,
computer problems had played in a major accident. .
Fadec, a safety-critical engine control system had recently been
installed on Chinook military transport helicopters. The question
was: had any of the known problems with Fadec affected the last
flight of a Chinook helicopter which crashed on the Mull of Kintyre
in June 1994, killing all 29 people on board?
It was an apparently straightforward task because Cable had only
to ask the right questions and identify any software flaws with the
help of the Ministry of Defence and Fadec's supplier and
sub-contractors.
At that time, there was a vast amount of information that could
have been made available on the Chinook Fadec because the customer
and the supplier were at an advanced stage of legal dispute over
Fadec.
Cable was unaware of the litigation. So he was also unaware that
the MoD was claiming that the design of the Fadec contained
"fundamental flaws". He was also unaware of the MoD's claim that
the Fadec system was not airworthy.
Other information was kept from Cable. He was not told the
Fadec's full history of problems. He was not given access to
Squadron Leader Robert Burke, the Chinook community's most
experienced unit test pilot, who had a specialist in-flight
knowledge of Fadec. Burke said that he was instructed not to talk
to Cable.
Cable was also unaware of the existence of the MoD's expert
witness in the litigation over Fadec, Malcolm Perks. Perks could
have told Cable that a flawed Fadec was capable of causing a
catastrophe. In the end, however, Cable had only a limited
knowledge of the system.
The issues facing Cable were similar to those that confront
independent forensic IT auditors during routine investigations. The
auditors are not experts in the software that is in use at the
companies they are investigating. They can identify the cause of an
incident and any related computer problems only with the help of
the supplier, the customer or both.
But sometimes suppliers and customers are unenthusiastic about
an independent investigation of their systems, the results of which
could embarrass both sides or worse, provide ammunition for
litigants.
These sensitivities are highlighted in a report by the US-based
Rand Corporation, which has carried out a study of the
relationships between suppliers and independent investigators.
In its report for the National Transportation Safety Board,
which investigates aviation accidents, Rand highlights some of the
difficulties that independent investigators face in trying to
identify whether software was to blame for a major incident and, if
so, what lessons can be learned.
Jim Hall, chairman of the Safety Board, which requested the
report, said, "If Rand is correct, and we believe it is, that
accidents will only become more complex, we'll need more - and
better - data to help us determine the cause of such
accidents."
He added, "I read with interest accounts of rocket accident
investigations that found the causes to involve software problems -
in one case a misplaced decimal point in the computer code. The
accidents were attributed to errors in computer code overlooked by
engineers and quality assurance personnel. Fortunately, the reason
the causes could be determined was the existence of sophisticated
data link recording capabilities on the spacecraft.
"Would an investigator be able to discover such a problem in the
computer code of a crashed civil airliner?"
Rand pointed out that manufacturers are placed in a position of
conflicting interests when helping in an investigation into whether
their software was the cause of a major incident.
"The parties most likely to be named to assist in the
investigation are also likely to be named defendants in related
civil litigation," said Rand. "This inherent conflict of interest
may jeopardise, or be perceived to jeopardise, the integrity of the
investigation."
Yet the crash of a Chinook on the Mull of Kintyre and the
subsequent investigation showed the extent to which an independent
investigator is reliant on the manufacturers in identifying whether
their software was in any way involved in an accident.
In his accident report for the RAF, Cable said that the Fadec
system recovered in the wreckage contained only "nuisance" fault
codes. This assessment was one made by the manufacturers.
Cable was asked later, at an official inquiry, what reliance he
had placed on help from the manufacturers.
Question: "Did you have carried out or carry out yourself
any independent checks on the integrity of the Fadec or the
software used in the Fadec?"
Cable: "No, I did not."
Q: "Were such checks carried out?"
C: "Not to the best of my knowledge. Not as part of the
investigation."
Q: "So in checking basically the reliability of the Decu
[part of the Fadec] you went back to the manufacturers and checked
it with them?"
C: Yes, it is a totally normal process.
Q: "Would it have been possible to have some alternative
body from the manufacturers?"
C: "No, I don't believe so in this case."
Q: "Why is that?"
C: "I think it was far too specialised for that"
This reliance on the suppliers, which was highlighted by Cable's
evidence, is not confined to the area of air accident
investigation.
When MPs sought to investigate the causes of the IT disasters at
the Passport Agency, the Immigration Service, and in the NHS, they
interviewed all parties, including the suppliers, and were left
with no clear idea of the causes, who was to blame and what lessons
must be learned.
But if independent investigators are less likely in future to
get to the truth in major incidents in which software is a
suspected factor, it is also less likely that similar incidents and
accidents will be avoided in future.
- Safety in the Skies can be obtained from rand.org
Key comments from the Rand report
- "The magnitude of potential loss can be so high as to call into
question the commitment of private parties to full disclosure and
technical objectivity during investigations."
- "The need to modernise investigative practices and procedures
is particularly acuteÉ techniques are in some respects archaic,
raising doubts that complex accidents will be expeditiously, or
even conclusively, resolved."
- Can traditional relationships with stakeholdersÉ continue to
operate reliably in such a highly litigious environment?"
- "Many stakeholders expressed concern that the Safety Board's
limited staff was no match for the opposition of large commercial
firms facing large potential losses."
- "The growth in complexity is exponential in many areas, with
the most significant trend being the interconnectedness of systems
As complexity grows, hidden design or equipment defects are
problems of increasing concern."