JRB - Fotolia

Britain to pay billions for monster internet surveillance network

New questions raised about Britain’s snoopers’ charter after Denmark abandons its own UK-style surveillance programme for a second time

Britain’s biggest web companies will be forced to build a national network of massive internet surveillance centres, likely to cost billions of pounds, if MPs approve proposals the Home Office is determined to rush through Parliament after Easter.

The centres will be needed to hold “internet connection records” (ICRs) – databases of all connections made by UK internet users from “devices” to “services” – with records being continuously and compulsorily gathered from all connected devices. Companies will be told to store complete records for one year, and to provide remote automatic unsupervised access for government and intelligence agency search systems.

Questions raised as Denmark abandons its own surveillance programme

Members of the House of Commons scrutiny committee on the new Investigatory Powers Bill, which will meet for the first time this week, on 24 March 2016, will be told that the only country in the world ever to have attempted to build such a system – Denmark – decided last week (17 March 2016) to abandon its plans for the second time.

The first Danish national internet “session-logging” system was installed in 2007, but it was found to have been technically worthless for police and security agencies six years later, despite costs imposed on internet service providers (ISPs) to install and operate it.

The Danish Ministry of Justice launched plans for a modified system at the same time as the British Home Office brought out its revised draft Investigatory Powers Bill, at the start of March. The Danes asked accountants Ernst & Young to challenge Danish industry claims that the system would cost DKK1bn (£104m). Ernst & Young found the industry estimates to be correct.

Denmark drops surveillance plans for second time

On 17 March, Danish justice minister Søren Pind abandoned the session-logging scheme for the second time. “A cost of over DKK1bn is not tenable for the government,” he said, adding that it was important to protect “a vibrant business community” in the technology sector. Danish internet surveillance plans have now been sent back to the drawing board.

After launching the draft Investigatory Powers Bill in November 2015, Home Office sources claimed session logging had been revived in Denmark, using a system more similar to Britain’s. That claim has now rebounded.

The proposed new British law has been introduced partly to rectify technical flaws in the previous Home Office internet data retention law. Regulations passed under the 2015 Counter-Terrorism and Security Act were intended to track multiple use of internet protocol (IP) addresses by customers of mobile phone companies. But the regulations, which are now said not to have provided adequate data specifications, failed after less than nine months’ operation.

What exactly are internet connection records?

The Home Office subsequently produced an operational case for collecting what it referred to as internet connection records (ICRs). What exactly ICRs are, and whether they will have any genuine practical application in fighting crime or terrorism, is a matter of some controversy.

Britain’s internet companies have told Parliament they do not have or hold “anything like an internet connection record”. Addressing a parliamentary review committee in December 2015, Hugh Woolford, director of operations for Virgin Media, said: “This is something completely new for us… there is no need for us to capture this information.”

Mark Hughes, president of BT Security, told the committee: “We will have to deploy new equipment to comply with the legislation. That comes at a cost we have never collected [ICRs] before.”

Cost of UK surveillance programme hugely under-estimated

On a head-per-head comparison basis with Denmark, a similar British system would require an initial investment of £1.2bn, as well as annual running costs. But operational figures provided by the Danish government suggest the proposed British system would be much more expensive.   

The Home Office has estimated to parliament that, although it does not know what internet connection records are, has no specifications and does not know how many will have to be stored every year, the complete cost of the system over 10 years would be £170m to £180m.

“A cost of over DKK1bn [for the session-logging scheme] is not tenable for the government”

Søren Pind, Danish justice minister

In December 2015, service providers told the parliamentary review committee on the bill that that amount, or more, could be needed to set up ICR storage systems by just one of the “big four” internet companies. “It would cover what we [BT] need to do,” BT’s Hughes, told the committee, leaving little or nothing left to pay the other companies or for the central query system.

Vodafone’s security director Mark Hughes told the committee: “Where this figure from the Home Office came from I cannot say. We were not consulted when it was put together. We were consulted only after that figure was put together.”

Rather than create a single central database for all ICRs, as in Denmark, the Home Office plan entails ordering each company to build its own high-security internet surveillance datacentre or centres. The datacentres would then be accessed and managed from central search facilities. The requirement for individual UK internet companies to create, back up and manage at least five new high-security datacentres would add to the cost and complexity of the plan, according to the internet companies.

Why ‘database’ has become a dirty word for ministers

The government has made clear its fury about the previous occasion on which home secretary Theresa May failed to secure a new surveillance law, after Liberal Democrat leader Nick Clegg blocked her previous plan, labelled a “snoopers’ charter”. The use of the word “database”, especially as in “national database”, now appears to have been banned from all Home Office publications and speeches.

“Database” now appears nowhere in two drafts of the Investigatory Powers Bill, nor in nearly 2,000 pages of supporting official documents – except for a single sentence that escaped censorship in a hastily written March 2016 report on the Operational case for bulk powers. The sentence states that “personal datasets [involve] the use of datasets such as travel data or government databases” (emphasis added). The fate of the Home Office civil servant who allowed the word to get printed has not been disclosed.

Because of the same sensitivities, the official Home Office title for a national internet surveillance database management centre is “the request filter”.

UK telecoms companies will need to store trillions of records a year

The Danish government said that for its population of 5.6 million – less than 10% of the UK’s population – 350 billion session-logging and communications data records were created in 2013, the last year of operation. This is an average of 62,000 records person per year, or 170 surveillance records per person per day. In a country the size of the UK, this would mean creating an equivalent annual database of about four trillion (thousand billion) records, spread between different companies.

This is likely to be a low estimate, because the Danish companies were refused reimbursement by their Ministry of Justice, and so generally agreed to sample only one in 500 internet packet connection records. The Home Office said it wants every internet connection session logged, claiming this is the main reason the Danish session-logging system failed to help law enforcement.

Scaling up to full internet tracking capacity could therefore mean British companies storing tens of trillions records a year

Scaling up to full internet tracking capacity could therefore mean British companies storing tens of trillions records a year. According to the Home Office, each record will have to contain, at a minimum: Customer account reference or device identifier; date and time of event; duration; source and destination IP and port number of each session; domain name or linked URL (up to the first slash); volume of data; and name of internet service or server connected to.

Snoopers’ charter will need exabytes of data

Even if such new records could be limited to 100 bytes per record, Britain’s internet companies would have to build datacentres to store many exabytes (many thousands of petabytes) of data annually, and make it all rapidly and centrally searchable. Wikipedia estimates that storing an exabyte is equivalent to 100,000 times the printed material in the Library of Congress.

A single exabyte storage centre would cost at least £100m for equipment alone, provided that a petabyte server rack could be acquired for £100,000. The companies have also pointed out that annual data volumes are continuing to grow almost exponentially.

Read more about the proposed Investigatory Powers Bill

  • Open-Xchange privacy survey shows many Britons are unsure and concerned by controversial bill as it moves closer to becoming law.
  • The Home Office has tweaked the draft Investigatory Powers Bill, taking on committee recommendations – but questions remain.
  • The draft Investigatory Powers Bill could have major implications for telecommunication companies operating in the UK.
  • Facebook, Google, Microsoft, Twitter and Yahoo say they are particularly concerned about six key aspects of the UK’s draft Investigatory Powers Bill.

To these costs would have to be added the costs of construction, cooling and security, as well as the design, installation and nationwide provisioning across at least five major networks of packet inspection equipment that has not yet been specified, to capture data to an as yet unspecified standard.

A single exabyte-sized storage centre would be likely to occupy the space of at least two tennis courts. Finding the space, power and locations to build such datacentres would not be easy, according to Virgin’s Woolford. “It is no mean feat. This bill potentially could look at all of us having almost to mirror our entire network’s traffic to enable us to filter it. It is a huge undertaking,” he said.

Home office does not know how bulk collection will work

In comparing its own plan to the failed Danish system, the Home Office admitted that it has no engineering definition of what it wants to do, or how to do it.

“There is no single set of data that constitutes an ICR – it will depend on the service provider and service concerned,” it said. Plans will be “developed in consultation with individual CSPs [communications service providers] taking account of their network architecture and operational requirements”.   

The way the internet is arranged and operated is not simple. The magnitude of data collected that would be processed would be massively more – a hundredfold more – than we collect today
Adam Kinsley, BskyB

Adam Kinsley, director of policy and public affairs at BSkyB, told the parliamentary review committee: “There will need to be investment in new types of technology for us to be able to get up to the first slash [of a web address]. The way the internet is arranged and operated is not simple. The magnitude of data collected that would be processed would be massively more – a hundredfold more – than we collect today.”   

As matters stood, Kinsley added: “Any capability based on the face of the bill … is pretty close to zero. We absolutely need more detail to be able to deliver."

Virgin warned that even if “necessary discussions and detail were worked through”, and “depending on scale”, “earliest deployments” of Home Office surveillance equipment on its networks could not take place until 2018.

This is the second in a series of articles about the Investigatory Powers Bill by investigative journalist Duncan Campbell.   

  • Read part one: New UK law will criminalise failure to hack on demand.

Read more on Privacy and data protection

Data Center
Data Management