Just how do you store all the UK's comms data?

The government has a difficult job on its hands when it comes to storing communications data in the digital age.

It wants to keep a record of every piece of communication we produce. An enormous list explaining who called who, who emailed who, who messaged who on Facebook and who connected with who on Twitter.

It tried floating the idea of a central database for everything, but was defeated by the cacophony of privacy objections.

But its latest proposals – that internet service providers keep all the data themselves – don’t do much to allay the concerns of privacy campaigners. And on top of that, the ISPs say the technical challenge is far too great.

Not only will they have to store an exponentially growing pile of data, they’ll have to keep in a form that’s easily and quickly accessed if police need it.

The content of the phone calls, emails and messages can’t be stored and can only be accessed with a warrant. So ISPs need to use technology that can search for the communications data and leave the content.

The problems start when you consider how much social networking and the internet has changed the way we communicate. When you instant message someone or @ someone on Twitter, where does communication data stop, and content start? How do you separate them and store one but not the other? Experts say the government hasn’t thought this through enough: the definitions it uses in the proposals don’t fit the challenge at hand.

And the online world moves too quickly for a policy like this. Website protocols change all the time, meaning the technology that searches them needs to as well. Social networks can seem to come from nowhere – both Twitter and Facebook grew quickly, and undoubtedly the next big thing will too. The government apparently cannot comprehend this pace of change, let alone keep up with it.

Instead of lumping the problem onto ISPs, the government needs to think hard about how it is going to achieve what it wants to in a landscape it doesn’t seem to fully understand.