Inadequate IT contributed to 11 September intelligence failure

Antiquated IT and cultural turf battles between intelligence agencies contributed to the failure to head off the 11 September...

Antiquated IT and cultural turf battles between intelligence agencies contributed to the failure to head off the 11 September 2001 terrorist attacks, according to a Congressional report released yesterday.

The 900-page report of the joint inquiry by the House and Senate Select Committees on Intelligence into the attacks found that despite the collection of a massive amount of intelligence and clues that a major terrorist operation against the US was under way, significant deficiencies in IT and political battles between the CIA and the National Security Agency (NSA) over which agency should control the use and development of certain technologies allowed critical clues to be overlooked. 

The report specifically blames the failure by government agencies, particularly the FBI and the NSA, to ensure that their agents had adequate IT support. It cited, for instance, the absence of a centralised counterterrorism database. 

FBI field agents all cited the FBI's technology problems as among the top three things they would like to see fixed to improve counterterrorism efforts.

In addition, congressional investigators found that because of the limitations of the FBI's Automated Case File (ACS) system, a number of addressees, including the chief of the FBI's Radical Fundamentalist Unit, were unaware of vital pieces of communication before the attacks occurred. 

The FBI deployed the ACS in 1995 to replace a system of written reports and indexes. However, FBI agents told congressional investigators that the system was limited in its search capacity, difficult to use and unreliable. The system was so difficult to use, in fact, that FBI officials informed Congress that as of 26 September 2002, 68,000 counterterrorism leads dating to 1995 remained outstanding and unassigned. 

Concerns about the security of the ACS also hampered its use. Before the 11 September attacks, many FBI field agents did not include sensitive information in the ACS because they believed the system was not secure.

In addition, many agents who did include information in the ACS blocked access to it to limit the number of FBI personnel who could obtain the information. "Given these limitations, ACS does not provide assured retrieval of complete, authoritative information on any subject," the report concluded. 

The chief of the Radical Fundamentalist Unit described the FBI's situation for congressional investigators as "a setup for failure in terms of keeping a strategic picture of what we are up against". 

FBI Director Robert Mueller, however, told members of the Senate Judiciary Committee on July 23 that the bureau is only months away from completing work on a massive upgrade of its global IT infrastructure, including desktop upgrades for all of its field offices around the world and software upgrades based primarily on analyst applications and tools. 

Congress also singled out the NSA, the electronic eavesdropping arm of the Pentagon, for its inability to provide adequate IT tools for its analysts. More important, however, congressional investigators were surprised to learn that many of the problems at the NSA persisted well after the attacks. 

For example, NSA language analysts must still conduct the bulk of their work with pencil and paper, and many develop their own personal databases on index cards which cannot be made readily available to counterterrorism analysts at other agencies.

And, despite the NSA's $282m Trailblazer contract, signed in October 2002 with Science Applications International to help the agency upgrade its data collection and analysis capabilities, the implementation of Trailblazer remains three to five years away and "confusion still exists at NSA as to what will actually be provided by that program". 

Aside from the lack of IT infrastructure and tools, information-sharing and timely collection of intelligence was also significantly hampered by what congressional investigators characterised as a "turf war" between the CIA and the NSA over control of certain technologies. 

Intelligence officials told congressional investigators that while individual relationships and cooperation between the CIA and the NSA at the working level had often been very good, relationships at the mid- and upper-management levels of those agencies were often strained. 

"CIA perceived NSA as wanting to control technology use and development, while NSA was concerned that CIA was engaged in operations that were NSA's responsibility," the investigation concluded. 

NSA director Lt. Gen. Michael Hayden told members of the joint inquiry that "the old divisions of labour are impractical - the new electronic universe requires more and more co-operation."

He added that he "would not be surprised if someday the closeness of this relationship would require organisational changes".

Recommendations from the congressional report on the 11 September 2001 terrorist attacks include:

  • Making better use of future technologies to exploit terrorist communications.

  • Improving and expanding use of data mining technologies and analysis tools.

  • Developing a multilevel security operating system that gives analysts access to different classifications of data on one system.

  • Using existing IT to modernise intelligence reporting and trend analysis.

  • Developing an all-source intelligence fusion center within the Department of Homeland Security.

  • Solving the FBI's "persistent and incapacitating information technology problems".

Dan Verton writes for Computerworld

Read more on IT risk management