GCHQ, the government's electronic spy house, is moving ahead with a £2bn plan to analyse data from internet communications traffic despite Home Office assertions that it is not setting up a centralised Big Brother database.
Reports in weekend papers suggested that the government has already issued contracts, one for £200m to Lockheed Martin, the US defence company, and another to Detica, for hardware and software for the so-called Mastering the Internet (MTI) project.
Lockheed Martin already supplies the IT for the national air traffic control centre at Swanick, and for the 2011 national census. Detica, which has close links to the law enforcement community, has data mining software that can detect links between individuals based on their contacts with sometimes widely separated organisations. Spokesman for the two firms referred queries to GCHQ.
GCHQ declined to comment beyond its statement issued at the weekend. This said accelerating technological change was putting pressure on it to keep up with the people it is watching. "One of our greatest challenges is maintaining our capability in the face of the growth in internet-based communications and voice over internet telephony," it said.
The Sunday Times said GCHQ was looking for companies to develop a "black box probe" that would sit inside communications service providers' premises to intercept messages.
GCHQ said the new technology that GCHQ was developing would work under the existing legal framework, which includes the Intelligence Services Act 1994 and the Regulation of Investigatory Powers Act 2000 (RIPA).
It said, "GCHQ is not developing technology to enable the monitoring of all internet use and phone calls in Britain, or to target everyone in the UK. Similarly, GCHQ has no ambitions, expectations or plans for a database or databases to store centrally all communications data in Britain."
The government has obliged commercial communication services providers to collect message header and other data under the European Data Retention Directive and to keep it for a year, pending requests from law enforcement authorities for access to it.
GCHQ said it does not target anyone indiscriminately. "All our activities are proportionate to the threats against which we seek to guard and are subject to tests on those grounds," it said.
This suggests GCHQ could use the probes to grab subsets of the traffic from different CSPs, combine them in a data warehouse, and then use Detica's connection-finding software to identify or confirm suspects' social networks and transactions.
Meanwhile, in an advertisement in this week's Computer Weekly, another national security agency, HM Government Communications Centre (HMGCC), has called for "technical experts to develop, maintain and safeguard our IT systems".