UK open data "failed opportunity" says founding academic

The UK has sent the world’s open data movement up a wrong turn, fear its greatest advocates, by easing restrictions on profits that can be earned from the sale of public data.

Their fears were roused by a reboot of regulations that will make it easier for public bodies to sell data. This was contrary to the campaign promises of the coalition government’s Conservative leadership, who said they would make public bodies publish their databases as open data. Public data was to be put in public hands for the public good.

The new rules, to be drafted in December, will permit those quasi-public bodies who hold the most valuable public data to continue selling it for profit.

Open data advocates linked to prime minister David Cameron’s initiative called an alarm after the National Archives put the new rules out to routine consultation in the summer. One key part of the new rules was how much profit should people be permitted to make from public data. The Archives left this part of its proposals blank.

The government has meanwhile been selling choice public data assets off, eyeing others for long-term lease to entrepreneurs, and putting others at the heart of international expansion plans for those quasi-public data bodies, which the government has subjected to a systematic programme of privatisation since it came to power.

Ellen Broad - policy lead - Open Data Institute.jpgEllen Broad, policy lead at the Open Data Institute (ODI), said in a submission to the UK consultation earlier this month the new rules were a “backwards step” for the government because they went contrary to its open data policy.

“What the UK is proposing would make it easier to charge more,” she told Computer Weekly.

Her submission was backed by some of the key people behind the government’s open data policy: Rufus Pollock, the Cambridge academic and president of the Open Knowledge Foundation whose work on open data was headlined in the Conservative Party’s 2010 election manifesto; Mark Taylor, the open source software pioneer who helped write that manifesto; and Tim Berners-Lee and Nigel Shadbolt, world-wide web pioneers who the prime minister, David Cameron, honoured by establishing the ODI and the Queen honoured with Knighthoods.

Pollock, Berners-Lee and Shadbolt had also formed the Public Sector Transparency Board, inaugurated by Cameron in June 2010 to oversee the transition of public data assets into open data, freely available for public use.


Now five years on, the rebooted open data regulations were a “failed opportunity” to bed that policy down, Pollock told Computer Weekly.

The open data movement’s founders were dismayed because the UK had been an exemplar for open data to the rest of the world. But the new rules had spoiled its visage and obstructed their efforts before their work was done.

The new rules were set by an update to the European Union’s Public Sector Information Directive, an agreement to replicate the UK’s open data policy across the rest of Europe. They were implemented by Neelie Kroes, European Commission vice president who was already pushing the open computing agenda in Brussels when Cameron was using it to campaign for office in 2008. The regulations will be the culmination of all their work.

Rufus Pollock - president - Open Knowledge Foundation - CROP.pngBut Pollock suspected the UK had watered the EU agreement down.

“There was a failed opportunity,” he said. “My sense was the government didn’t take it. The EU wanted a stronger line than was in the directive. That was toned down.”

That stronger line would have been what Cameron’s open data team wanted: a strict prohibition on profit from public data.

The EU agreement contained a provision to accommodate a unique aspect of the UK data landscape: those quasi-public bodies entrusted with looking after the most valuable public data. Called Trading Funds, they include organisations such as Companies House, Ordnance Survey, the Land Registry and the MET Office.

The government permits the Trading Funds look after valuable UK data on condition that they charge access fees to cover their costs. These costs were higher in the pre-digital age, when collecting, managing and distributing public information were manual processes.


But Cameron’s open data policy relied on digitization – the automated collection and distribution of public data sets. This would cut the cost of distributing public data – called its “marginal cost” – to as good as zero.

The government had always allowed the Trading Funds to charge a small profit on top of the cost of distributing their data. Now the marginal cost of public data was almost zero, the open data movement had pushed for all charges to be removed: there was no longer any reason for making a profit on something that didn’t even need to be sold, especially when it was a public asset.

But that may not happen. Pollock suspected the UK was torn between short-termism and open data. Should it generate tax income by selling public data? Or should they stick to their guns and and treat public data like public investment, to be injected into the economy as a long-term growth stimulant – pump-priming for the digital age?

Pollock said governments were failing to see the bigger picture. He suspected HM Treasury was more interested in tax revenue than open data.


The Treasury had however led implementation of the coalition’s open data policy. The Cabinet Office open data team worked from the Treasury after the coalition came to power and the Cabinet Office was staffed by officials put in place by the last, Labour government. Chancellor George Osborne had campaigned with Cameron for open data. The new rules are being implemented by the National Archives.

Howard Davies - Standards Manager - National Archives.jpgHoward Davies, who as standards manager at National Archives helped write the UK consultation, said: “We are not wanting to see a situation where excessive charges are made. That’s just not what government policy is about at all .”

“Where there are going to be charges, they will be limited to marginal cost of reproduction, provision and dissemination of documents. That’s for the vast majority of public sector information,” said Davies.

Profit could be made from public data “only in those exceptional cases, where bodies have to generate revenue to cover part of their own costs”, he said.

Those exceptional cases would be the Trading Funds, he said, and other cases “where a public body is required to cover its costs”.