We write to the Public Accounts Committee in relation to its
consideration of the Chinook's FADEC computerised engine control
system
18th April 2000
Mr K Brown
Public Accounts Committee
House of Commons
Westminster
London
Dear Mr Brown
Computer Weekly has made a particular study of the
Chinook's FADEC and aviation accidents in which computers or the
man-machine interface have been a suspected factor.
In the case of the Chinook's FADEC there has never, to
Computer Weekly's knowledge, been a procurement or an
implementation of safety-critical software that has had such a
dense history of significant problems.
Therefore the issue of whether the Chinook's FADEC was poorly
procured, whether it was ready for operational use when it went
into service and whether flaws in the system could have caused the
RAF's worst peacetime accident, are issues that give the Public
Accounts Committee a unique opportunity to examine some of the
wider matters that are of great significance.
The key question, in our view, is whether the pilots of Chinook
ZD576 that crashed on the Mull were at fault, or whether, given the
FADEC's history of causing engine surges, spurious cockpit warnings
and engine run-downs, sometimes without leaving physical evidence
of any software problems, the FADEC's software could have caused
the accident on the Mull of Kintyre by again malfunctioning without
leaving any physical evidence.
The far wider issues that have been raised by the National Audit
Office's research into the Chinook's FADEC, and have not so far
been considered by Parliament include:
- the feasibility of independent testing of safety critical and
mission critical software
- what checks exist if any to stop departments sidelining any
independent advice that goes against the grain
- the conflicts of interest that arise when only manufacturers
understand their software sufficiently to identify any of its flaws
after a major incident
- the accountability of civil servants to Parliament after a
major incident
- the difficulties of establishing physical evidence of software
problems after a major incident, and
- whether, in the light of a major incident, a department will
seek to protect a supplier from criticism rather than allow some of
the opprobrium to spill onto the department's lap.
These are not problems that are confined to aviation. Software
now controls everything from City transportation systems to
missiles, command and control systems, critical telecommunications
equipment and the systems on which the UK's reputation as a major
financial centre depend.
This last point is topical at the present point in time. It is
currently more than a week after a computer failure brought down
the London Stock Exchange for nearly an entire day, and the bug
that brought down the system has not been identified because it has
not proved possible as yet to replicate the exact problem. The
inability to identify the bug is despite the forensic skills some
of the most expert software specialists in the USA and the UK.
Having studied all the available evidence and more - including
hundreds of pages of documents that have never been published by
the Ministry of Defence - we believe that we can show that there is
insufficient evidence to blame the pilots, and good grounds for
believing that the software may have played a critical role.
The evidence we have studied supports the following conclusions
(if the committee so requests, we will supply documentary evidence
for any of the following conclusions):
- the FADEC was procured (without open competition) without
sufficient controls by the prime contractor (Textron Lycoming) on
the subcontractors;
- the RAF was kept at a distance from the development process
rather than working in tandem, which is usually necessary in major
software-intensive projects. Studies into the lessons from IT
problem projects, such as the Stock Exchange's Taurus project and
the "Croeso" system undertaken by South Western Electricity and its
neighbouring utility South Wales Electricity underline the
desirability of joint working to ensure that the final product
fulfils the customer's objectives and also because the developers
can be held accountable for the quality of their work by an
independent organisation (the customer) on an ongoing basis.
- also for reasons of accountability, and in the light of the
failure of the post Office/Benefit Agency "Pathway" project, HM
Treasury have stressed to the DTI select committee the need for
major software developments to be financed by an organisation that
is entirely independent of the developers. In the case of the
Chinook's FADEC, the project was financed by initially by the
subcontractors who were also the developers.
- the original proposal for a Chinook FADEC contained an
undertaking that then project would be low-risk. This low-risk
concept was not carried through to implementation, however. A
change was made. Instead of a mechanical backup the FADEC's main
(primary) and backup (reversionary) lanes were controlled by
software. So pilots were to have no direct mechanical control of
engine acceleration or deceleration. They had to delegate "full
authority" to the software. Two years ago, however, Textron
Lycoming announced a joint project to develop a successor to FADEC
that "unlike FADEC … will feature in independent mechanical backup
subsystems for all critical control functions".
- the UK version of the FADEC software design was inadequately
tested and flawed, as evidenced by the software's ongoing (flight
critical) problems and the number of changes the software underwent
after it was accepted by the MoD and the RAF.
- the system was brought into service to meet operational needs
(the number of Chinooks available for operations in 1994 being at a
record low), despite the fact that the MoD was at the time taking
legal action against the FADEC suppliers over claims that the FADEC
software had not been designed to international military or civil
avionics standards;
- the RAF and the MoD did not take the advice of its
airworthiness assessors at Boscombe Down that the software should
be re-written. This was because of the disruption this would have
caused to the timetable for the Mid-Life Update, and also because
there was a resistance to giving in to Boscombe Down whose concerns
over the FADEC were perceived at the highest level in the RAF as
being exaggerated (memo reference ADD/308/04 dated 6 June
1994).
- the concerns about FADEC that were expressed in 1993 by an
independent contractor EDS-Scicon, which was commissioned to
analyse the software, were not addressed by the time the FADEC came
into operational service. This was partly because the MoD was
assured in a Textron Lycoming "White Paper" that the concerns of
EDS-Scicon and Boscombe Down were misplaced. However the trust that
the MoD placed on Textron's assurances about FADEC were in
contradiction to the MoD's distrust of Textron's assurances that
the Ministry expressed in its arbitration proceedings against
Textron, proceedings which were at that time secret.
The above points suggest strongly that the FADEC was implemented
against the conventions of best practice. The Committee may
therefore wish to consider whether the procurement of the FADEC and
the poor relationship in 1993 and 1994 between the MoD and its
appointed independent arbiter of aircraft software quality at
Boscombe Down is a matter of some concern.
It may be asked how it is possible for a government department
to procure, accept and implement a safety-critical software product
that has not been benchmarked and approved by qualified arbiters,
to the satisfaction of those expert assessors, using methodologies
and tools laid down in the MoD's main standard 00-55 which sets out
procedures for designing, verifying and validating software in
safety-related applications?
Another question the Committee could ask is: how can the Mod
persist with its argument that the software was ready to be put
into operational use when the software's performance and
reliability was questioned by its own assessors and by independent
private contractors?
So far, the MoD response has been to say that Boscombe Down was
using an inappropriate methodology to validate the software. The
Ministry's Permanent Under Secretary of State, Mr Kevin Tebbit, at
the Public Accounts Committee's hearing on 8 March 2000 suggested
that Boscombe Down's chosen methodology, static code analysis, was
more appropriate for the nuclear industry.
"I would not like to comment on Static Code Analysis procedure
on the general side. I do know that it was used in the nuclear
power industry and was applied in this context."
However Martyn Thomas, of the UK's most respected independent
safety-critical software specialists, has written to Computer
Weekly pointing out that a branch of the MoD helped to develop
static code analysis. Indeed it has emerged that static code
analysis was, in 1994, and is today, a recommended methodology in
00-55, the MoD's benchmark standard for the design, verification
and validation of safety-related software.
Thomas says in his letter says that it was the Royal Signals and
Radar Establishment, now part of the Defence Evaluation Research
Agency, an agency of the Ministry of Defence, that helped to
develop static code analysis.
"The RSRE developed the secret technology so that they could
verify security-critical software. Work on static analysis was
declassified as a matter of public policy, precisely so that it
could be used on safety-critical software, such as the Chinook
FADEC".
Other letters to Computer Weekly say that the Lockheed
C130J Hercules aircraft is undergoing static code analysis for the
purposes of UK flight certification; and separately, Bath-based
Praxis Critical Systems, which develops software for the defence,
banking and other industries, says that static code analysis has
been used to validate safety-critical software in aircraft such as
the Tornado F3 and the Eurofighter.
It is also used in safety-critical aircraft support functions,
such as the Sholis system that helps helicopters to land safely on
ships. In addition, it has been used by the Government's
communications centre GCHQ to spot viruses in software.
Praxis said the importance of static code analysis, which
involves testing code without executing it, lies in its ability to
highlight anomalies and faults that could remain undetected by
dynamic testing.
It said dynamic testing, which involves executing the code, can
highlight only a small number of potential problems. This is
because it does not check paths through the software that can be
taken by executable code.
What all this shows is that Boscombe Down's preferred
methodology, that has been much-denigrated by the MoD, was and is
the MoD's own preferred methodology.
The ministry's incorrect statements on static code analysis may
leave it open to the accusation that it has misled the National
Audit Office, and Parliament.
On the basis of its MoD briefings, the National Audit Office
(NAO) reported on the anomalies found by EDS-Scicon, but added that
the contractor had used static code analysis, which it said was an
"internal Boscombe Down policy, not supported by defence
standards".
In 1998, MPs on the Commons' Defence Committee were told by the
Ministry of Defence that, "static code analysis is... a requirement
placed by British Nuclear Fuels on the safety of a nuclear
system".
In July last year, a senior civil servant at the Secretariat
(Air Staff) of the Ministry of Defence wrote in a letter that,
"Static code analysis does not validate the performance of the
software and the department therefore had no requirement for
it".
In August last year, the Ministry wrote to Defence Committee MP
Michael Hancock saying that "Boscombe Down's preferred method of
examination is static code analysis, a system of verification not
widely in use but employed in the nuclear industry."
Also last year, the House of Lords was told by the Ministry
that, "Boscombe Down indicated a wish to assess the design of the
FADEC software using static code analysis - a methodology used by
the nuclear industry."
None of this gives a true impression of the importance to the
MoD of static code analysis. It should be pointed out that static
code analysis is not the only method that should be used to
validate software. Specialists who have written to us say that a
combination of static and dynamic will help to spot flaws and
potential causes of failure.
So, if Boscombe Down was dissatisfied with the software having
used the Ministry's preferred methodology to test the code, why is
the Mod persisting in its claims that the FADEC was not flawed?
Nobody is certain why, but it is evident that the issues
surrounding the crash on the Mull and the FADEC have become mired
in half-truths, doublespeak, and falsehoods. There are a number of
examples of these, which the Public Accounts Committee has
encountered first hand (see later comments relating to the MoD
evidence to the committee).
If the MoD cannot argue rationally and logically on why their
actions were justifiable in putting the FADEC into service despite
expert reservations, can it argue with credibility that the pilots
of Chinook ZD576 were to blame for the accident on the Mull of
Kintyre?
The FADEC is unusually complex piece of equipment, with nearly
two million lines of software code. Indeed we note that when the
Public Accounts Committee asked Mr Tebbit, Sir Robert Walmsley,
Chief of Defence Procurement, or Vice Admiral Sir Jeremy Blackham,
Deputy Chief of Defence Staff (Equipment Capability), about how
FADEC can cause engines surges, none knew sufficient about the way
the FADEC impacts on the helicopter's flying capabilities to give
the Committee an answer.
It took Computer Weekly researchers many months of
studying the design documents, and dozens of conversations with
pilots and technicians, to understand how FADEC interacts with the
controls on a Chinook, what can happen when the system
malfunctions, how pilots should react, and what faults would not
leave any trace.
We were able to conclude that no evidence of technical
malfunction does not mean no technical malfunction.
If, however, the MoD's contention that no evidence of technical
malfunction is analagous to no technical malfunction, this sets a
dangerous precedent.
The "Rand" report for the National Transportation Safety Board
which investigates aviation accidents in the United States points
to the fact that accidents caused by software may not be traceable
to software because it tends to leave no physical trace of its
behaviour in the wreckage.
Indeed it was a conclusion of the RAF Board of Inquiry into the
crash on the Mull of Kintyre that: "an unforeseen technical
malfunction of the type being experienced on the Chinook HC2, which
would not necessarily have left an physical evidence, remained a
possibility and could not be discounted".
This issue, of manufacturers being held accountable for the
software they produce, is of concern to Computer Weekly
readers. If the MoD's view of the crash on the Mull of Kintyre is
accepted - that no evidence of technical malfunction means that
there was no technical malfunction - this in our view provides a
cushion of comfort for manufacturers who supply poor quality
software.
It means that if software causes a fatal accident or fails in a
mission-critical system, and the fault cannot be traced afterwards
because software has left no physical evidence of its behaviour,
then the manufacturers cannot be blamed. This would leave computer
users, Parliament, and safety regulators unable to hold software
manufacturers to account if their products caused critical systems
to fail.
In the case of the crash on the Mull of Kintyre the pilots were
found posthumously to have been grossly negligent in the absence of
any concrete evidence of technical malfunction. If this verdict is
accepted, irrespective of whether this represents an injustice or
not to the families of the pilots, we believe this send the wrong
signals to manufacturers.
Acceptance of the verdict would also, in effect, condone what we
believe is MoD doublespeak. The MoD says it will examine with
compassion any new evidence but it knows no evidence of software
problems can be produced. Therefore the Ministry is relying on its
detractors to produce evidence that it knows cannot physically be
produced. Is this not evidence of the MoD's doublespeak?
This brings us to another of the wider issues. In its anxiety to
defend the FADEC, and therefore the reputation and integrity of the
MoD and RAF, statements have been made to Parliament that have been
incorrect and/or misleading, sometimes patently so. This may raise
questions that go beyond the Chinook and FADEC, and touch on
matters related to accountability of the department to Parliament.
To avoid making this letter too long we have not always given
examples but can do so if requested. The Ministry has:
- withheld relevant information and, when this information has
leaked out, has made incorrect statements about that information.
For example, as attention focused on the FADEC in the light of the
crash on the Mull of Kintyre, the MoD and the RAF made no mention
of its litigation against Textron Lycoming. Therefore the families
of the dead pilots or passengers were unaware that FADEC was
capable of causing a Chinook to crash, and indeed had caused a
serious accident in 1989, after which the Ministry had issued a
writ against the FADEC supplier. When evidence of the arbitration
proceedings leaked out in 1997, the MoD issued incorrect and
contradictory statements about the matter. It made statements, for
example, saying that the RAF Board of Inquiry and the Fatal
Accident Inquiry was aware of the litigation. Then it conceded that
neither inquiry was made aware was the litigation. Then it issued
statements that the arbitration proceedings arose from faulty
testing and had "nothing to do with the software". In fact the
opening page of the government's writ against Textron was that the
1989 accident was "caused by respondent Textron's faulty design of
a computerised engine fuel control device, FADEC." Indeed the legal
papers denied that the accident was due to faulty testing. The
MoD's case was that "Boeing's test procedures were reasonable and
adequate".
- continued to issue incorrect statements even after these have
been shown to be incorrect. For example the Ministry last year
apologised for stating incorrectly to MPs on the Defence Committee
that the FADEC was not safety-critical. However the Ministry last
month repeated the original incorrect statement to the Public
Accounts Committee.
- used selective quotations from the report of accident
investigators. The effect of this has been to give an impression of
certainty when investigators, in sentences immediately before or
after the one selected by the MoD, have expressed uncertainty or a
caveat. For example the MoD, in several letters to MPs and in one
to the British Airline Pilots Association sought to show that the
FADEC's main computer component (called the DECU - Digital
Electronic Control Unit) was not at fault in the accident on the
Mull. All of these letters quoted one particular part of the
accident report which said: "Strip examination of the engines …
revealed no signs of pre-impact failure or malfunction that could
have affected the operation of either engine". However none of the
letters quoted the very next sentence in the accident report which
said: "Fire damage prevented assessment of the functionality of the
No 1 (engine's) DECU and had destroyed its memories of the
operating program and exceedance fault history".
- Omitted facts or parts of official reports that have not been
consistent with the MoD's stated position that FADEC has never
caused a Chinook accident, could not cause a Chinook accident, and
has never, in its production versions, had any serious flaws. For
example the MoD did not mention, in its dozens of letters to MPs
and in Parliamentary Answers that one conclusion of the RAF Board
of Inquiry was that: "… an unforeseen technical malfunction of the
type being experienced on the Chinook HC2, which would not
necessarily have left an physical evidence, remained a possibility
and could not be discounted".
Mixed undisputed facts with disputed MoD speculation, without
making the distinction clear, in such a way as to convey certainty
when none exists. It has also made statements without context that
have had the effect of giving an incorrect impression. For example,
after the fatal crash of a US Army Chinook equipped with FADEC in
1996, the deceased pilots were blamed because no fault was found of
a technical malfunction. Later it was found that the original
verdict was wrong and that, contrary to the initial investigation
report which found no evidence of any electrical problems, there
had in fact been an electrical failure. When the MoD was asked
about possible electrical problems on the aircraft that crashed on
the Mull, the MoD replied that the findings on this were
"unequivocal" and it quoted part of a sentence in the
investigator's report which said that a "major pre-impact loss of
electrical supplies had not occurred". The MoD omitted to mention
the first part of the sentence which said "While none of the direct
indications of electrical system behaviour was conclusive …" To
cite another example, the US Army issued a warning to its Chinook
community in 1999 that hydraulic contamination was a suspected
cause of sudden, unexpected and potentially fatal manœuvres of the
aircraft. When it was put to the MoD that investigators of the
crash on the Mull had found a "considerable quantity" of particles
in hydraulic fluid and had concluded that there was "pre-impact
hydraulic system contamination," the MoD quoted from other parts of
the accident report in which hydraulic components had been found
without abnormalities. The Ministry also, in a letter to the MP
Michael Hancock, dated August 1999, said that hydraulic
contamination had been "ruled out" as a result of an investigation
by the Air Accidents Investigation Branch. The letter added that
the investigators found hydraulic contamination that was
"consistent only with what would have been expected as a result of
normal wear and tear". The MoD omitted to quote the reference in
the investigation report to "pre-impact hydraulic system
contamination and "high" wear rates. Also in its letter, the MoD
omitted to mention that the accident investigators, among their
final conclusion in the accident report had, remarked that there
were "possible utility hydraulic system abnormalities".
- criticised all specialists who have not agreed with the MoD
over the FADEC. The criticism has extended to Malcolm Perks, the
MoD's own expert witness in its arbitration proceedings against
Textron Lycoming, Malcolm Perks, who has expressed the view that
FADEC could have caused the crash on the Mull. The MoD has also
questioned the professionalism of Squadron Leader Robert Burke, a
pilot with more test hours on Chinooks than any other at the time
who expressed the view that that at an engine surge could have
contributed to the crash on the Mull. The Ministry has also
criticised the A&AEE at Boscombe Down for using inappropriate
methods to test the FADEC, when in fact those methods were those
recommended by the MoD. In addition the Ministry belittled the
evidence of its independent contractor EDS-Scicon by saying that it
raised issues on FADEC that were related mainly to documentation.
In fact EDS-Scicon had raised concerns about the safety of the
software. The Ministry has also criticised the media, sometimes by
denigrating statements that the media did not make.
- issued a letter warning a respected aviation magazine that if
it went ahead and published an article that was critical of the
decision to blame the pilots for the crash on the Mull of Kintyre,
it may face an action for defamation.
- defended FADEC by quoting the assurances of the Design
Authority (Textron Lycoming) without drawing attention to the fact
that the Ministry had undermined those same assurances during the
arbitration proceedings, which the Ministry won.
But why, since the crash on the Mull, has the MoD has made so
many misleading statements (many more than have been mentioned
above) with the effect, apparently, of turning attention away from
the FADEC and onto the pilots?
One possible explanation is that the Ministry does not want to
give any ground to those who are critical of its decision to put
the Chinook into operational service before the problems with the
FADEC's software had been eliminated.
It could also be said that the MoD does not wish to lend weight
to the concerns shared by the families of the dead pilots and the
families of the dead VIP passengers on that last flight of Chinook
ZD576, that FADEC may have been involved in the accident.
We are not certain, however, that these are the reasons for the
MoD's position.
What we have seen is that, the more information that comes to
light, from the US Army and elsewhere, which throws doubt on the
original decision to blame the pilots, the more entrenched has
become the MoD's defence of FADEC's implementation on the
Chinook.
We cannot help but conclude that the reason for the MoD's
position is that it has always in public defended the decision to
blame the pilots and that it must, for the sake of consistency,
continue to defend this decision, whatever new evidence or
information arises.
It appears to us that that the matter of whether the FADEC was
fundamentally flawed, or whether it caused the crash on the Mull,
is in some ways subordinate to the need to sustain departmental
pride and not admit that a mistake may have been made.
Indeed there appears to be, within the MoD, an institutional
machismo that will admit no imposition on its decision making by
those whom it regards as outsiders: particularly politicians and
the media.
If this attitude continues to prevail - and we suspect it will -
then no matter what information comes to light, or whatever
political pressure is applied, the Ministry and particularly air
marshals in the RAF will not allow the verdict against the pilots
to be stood down.
Therefore we believe that the matter needs to be investigated by
a body that is independent of the MoD. For if no action is taken to
reinvestigate, or to question the MoD statements in relation to the
FADEC, this will leave unanswered the question of whether the
Ministry has inadvertently misled Parliament or has done so
systematically rather than countenance the possibility that it made
mistakes, firstly by rushing the FADEC into service and then
defending the decision to blame two dead Special Forces pilots for
a crash that has no identifiable cause but which could have
involved FADEC.
CONCLUSION:
a) It is not possible to look at the procurement of the
FADEC and say whether it was safe, procured properly and in a
timely fashion without looking at its performance after it came
into operational service. The greatest possible example of its
malfunctioning could have been the crash on the Mull of Kintyre.
The question of whether the FADEC worked properly could be related
directly to that accident.
b) The MoD is failing its own legal defence team in the
arbitration hearing. The Ministry used hundreds of pages of
evidence to prosecute its case against the FADEC supplier Textron
Lycoming but failed to make any of this material available to any
of the inquiries into the performance of the FADEC including the
Public Accounts Committee. Once details of the MoD's successful
legal case against Textron were put into the public domain by the
media, the MoD attacked its own evidence as irrelevant and
peripheral to the performance of the FADEC.
c) Ultimately there are two issues: Did the FADEC pass
all the tests set for it by the procurement team which rightly
included Boscombe Down and EDS-Scicon, or for a host of operational
reasons, was FADEC rushed into service, amid a hasty dismissal of
expert concerns, with possible tragic consequences?
If the Committee wishes to see any or all of the documents on
which this letter is based, we will be happy to oblige.
Yours sincerely
Karl Schneider
Editor
Executive Editor
Tony Collins
Hooman Bassirian
News Editor