Computer Weekly's Tony Collins uncovers evidence that the MoD has
consistently undermined the authority of its software assessors
since the notorious fatal crash of a Chinook Mk2 in 1994
Computer Weekly's Tony Collins uncovers evidence that
the MoD has consistently undermined the authority of its software
assessors since the notorious fatal crash of a Chinook Mk2 in
1994Software "until it can be verified, it has to be considered
unpredictable" and a possible cause of a Chinook crash on the Mull
of Kintyre in 1994
A leaked draft memo written by one of the RAF's most senior
officers has a voice and tone similar to that of a chief executive
in the private sector who attacks IT staff for delaying a
much-needed new computer system.
The memo criticised the Ministry of Defence's Aeroplane and
Armament Experimental Establishment (A&AEE) at Boscombe Down
that had been asked by the Ministry to evaluate new safety-critical
software for the Chinook Mk2 helicopter.
"I find A&AEE's attitude quite incredible," said the draft
memo which was from one of the RAF's most senior officers to one of
his colleagues.
The reference date on the memo is 6 June 1994 - four days after
the fatal crash of a Chinook Mk2 on the Mull of Kintyre which
killed 29 people.
The resentment expressed in the memo reflects the concerns that
often exist at the highest levels of an organisation, where the
effects of delays in the introduction of a major new system are
most keenly felt.
In the RAF's case, the memo makes it clear that senior officers
were anxious to introduce a modified Chinook helicopter into
service, in order to "maintain a capability". But the IT and
airworthiness specialists at Boscombe Down were refusing to approve
the Chinook Mk2's new Full Authority Digital Engine Control (Fadec)
software and had halted trials flying of the Chinook Mk2 because of
their concerns.
Boscombe Down was describing the Fadec software variously as
"unverifiable", "not fit for purpose" and "unacceptable". It was
always denied that Boscombe Down had declared the software
unsafe.
The memo, however, reveals for the first time that Boscombe Down
had halted trials flying for the Chinook Mk2 because of "concerns
about the safety of the Fadec system". But it went on to attack the
A&AEE's ban on trials flying as contrasting sharply with the
"considerable efforts being made by the front line to bring the
aircraft into service" The A&AEE, said the memo, has an
attitude that "does nothing to engender aircrew confidence in the
aircraft".
The memo points to a far wider and more serious issue than a
simple difference of opinion between the Ministry of Defence and
its own independent software and airworthiness assessors.
At the heart of the dispute is the role played by independent
scrutineers as arbitrators of the fitness for purpose of critical
software.
It raises questions over the value of the A&AEE's advice,
and whether its criticisms should be set to one side if they
contradict the operational objectives of client bodies such as the
MoD.
Malcolm Perks, former IT head of Technology at Rolls-Royce, who
helped to develop the company's Fadec systems, believes that there
was little point in the MoD asking independent assessors for their
views and then discounting any opinions that were not
"on-message".
Computer Weekly has, however, pieced together evidence
that shows that the MoD has, since the crash, consistently
undermined the A&AEE's authority and credibility in letters,
Parliamentary answers and in briefings.
Static code analysis
One of the main criticisms of Boscombe Down was that it tested
the Fadec software using its "preferred method" of "static code
analysis", which was more applicable to the nuclear than the
defence industry. Static code analysis is a process of testing
parts or the whole of the software, during design, development and
final validation, to see that it is free of anomalies and without
executing the code.
Boscombe Down described the Fadec as unverifiable, partly
because it could not validate the software using static code
analysis.
One member of the House of Commons Defence Committee, Mike
Hancock MP, was told in an MoD letter last year that "Boscombe
Down's preferred method of examination is static code analysis, a
system of verification not widely in use but employed in the
British nuclear industry."
The letter added: "The Fadec software is not amenable to static
code analysis and hence Boscombe Down was not able to verify the
software to their [sic] satisfaction... This does not mean that
there was any inherent problem with the software."
A month previously, in July 1999, the MoD wrote in a letter
that: "Static code analysis does not validate the performance of
the software and the Department, therefore, had no requirement for
it."
In 1998, the House of Commons Defence Committee was told that
the Fadec software could be validated but not by certain scientists
within Boscombe Down who insisted on using static code analysis.
The committee was told that static code analysis "is a requirement
placed by British Nuclear Fuels on the safety of a nuclear system",
but Boscombe Down believed it "should be applied to aircraft
software".
The Defence Committee concluded that the "failure" of Boscombe
Down to give approval to the Fadec software was a "management
failure".
And the National Audit Office, based on its MoD briefings, wrote
in a report that was due to be discussed by MPs this week that,
"the requirement for static code analysis was an internal Boscombe
Down policy, not supported by Defence standards."
Computer Weekly has learned, however, that static code
analysis is specified in the MoD's own defence standard 00-55. The
standard is a benchmark for safety-related software supplied to the
MoD.
There are repeated references in 00-55 to the need for static
code analysis. One such reference says, "Static analysis... shall
be performed on the whole of the source code to verify that the
source code is well formed and free of anomalies that would affect
the safety of the system." The standard also specifies that the
code should be capable of being verified independently by
performing dynamic and "static analysis" of the code.
The standard also says that any code not developed to the
requirements of 00-55 should, ideally, be reverse engineered to
meet its specifications. This would mean retrospective activities
that include "static analysis".
Computer Weekly has further learned that the 00-55 standard, as
it applied in 1994, at the time of the crash on the Mull of
Kintyre, specified static analysis.
Value judgement
All this leaves some IT specialists wondering whether there is
an inherent conflict of interest in asking the MoD or the RAF to
act as independent judges of whether Boscombe Down was right or
wrong in its concerns about Fadec, and whether the software was or
was not a contributor to the crash on the Mull of Kintyre.
For if Fadec were ever to be implicated in the crash on the Mull
of Kintyre, it might show that the A&AEE's concern was
justified all along. It might also show that the hierarchies of the
RAF and MoD were wrong to approve the Mk2 despite Boscombe Down's
concerns.
More Chinook news