Complexities of safety critical augmented systems
There are no lessons that can be gleaned from the tragic loss of life following the Ethiopian Airlines Flight 302 crash on March 10, 2019. As has been reported across the web, the crash bears remarkable similarities to Indonesia’s Lion Air Crash of October 29, 2018. Both involved Boeing 737 MAX aircraft. To quote from a statement made by Ethiopian Airlines’ group CEO, Tewolde GebreMariam,: “Until we have answers, putting one more life at risk is too much.”
What is known today is that the crash appears to be a side-effect of a software system known as the Maneuvering Characteristics Augmentation System (MCAS). Boeing says MCAS has been designed and certified for the 737 MAX to enhance the pitch stability of the airplane. Across the web there have been reports of how the system got confused during take-off, forcing the nose down to prevent the aircraft from stalling. The plane continued to dive, despite efforts by the pilots to try to regain control of the aircraft. Reporting the preliminary findings of the investigation into the Ethiopian Airlines Flight 302 crash, the Wall Street Journal noted that a suspect flight-control feature automatically activated before the plane nose-dived into the ground.
Software update
Technically speaking, MCAS is a stall prevention system. According to CNBC, since the crashes of the two 737 Max planes, Boeing has faced fierce criticism for not doing more to tell flight crews about the stall prevention system or alert them when the technology kicks in. It reported that only one Angle of Attack (AOA) sensor for MCAS was fitted as standard. airlines were asked to for additional payment to have a second AOA installed.
Earlier this week Boeing issued a software update. According to Boeing this update has been put through hundreds of hours of analysis, laboratory testing, verification in a simulator and two test flights, including an in-flight certification test with Federal Aviation Administration (FAA) representatives on board as observers.
It said the flight control system will now compare inputs from both AOA sensors. “If the sensors disagree by 5.5 degrees or more with the flaps retracted, MCAS will not activate. An indicator on the flight deck display will alert the pilots.”
Balancing Safety critical automation with human operators
What is clear from these reports is the complex technical and ethical issues that must be addressed in developing safety critical augmented systems that need to coexist with highly trained individuals. Neither entrusting everything to the computer system nor deferring every decision to a human, are the right approach. While the FAA investigation is likely to conclude that the Ethiopian Airlines Flight 302 crash was down to software, could a tragedy like the Germanwings Flight 9525 crash on 24 March 2015 have been avoided if the flight control software actively prevented the co-pilot from flying the aircraft into the Alps?