Copyright compromise keeps UK internet archive offline

When the UK unveils its internet archive, it will be in an incongruous setting: the reading room of a library.

When the UK unveils its internet archive, it will be in an incongruous setting: the reading room of a library.

And that is where UK law says it must stay, never to be accessed by anyone online.

Publishers put that stipulation in the 2013 regulations that authorised the British Library to make the first official copy of every UK website. It allowed publishers to turn old web pages into pay-to-view archives without competition from libraries.

The British Library had been under pressure to compromise with publishers, after negotiations dragged on for 10 years. It wanted to start archiving UK websites before it was too late.

"There was a big black hole in the research material being gathered and with each year that passed that black hole got bigger," said a British Library spokesman.

But nobody appears to have represented the public interest by demanding an internet archive that does what prime minister David Cameron said it would do under his tenure: "democratise information".

"Its not a public archive. That's the absolute key point," said Angela Mills Wade, who as executive director of the European Publishers Council represented the publishing industry during negotiations.

"The [internet archive] legislation has always been constructed for people who go into the Legal Deposit libraries - for the readers, with a capital R. It's an archive for preservation and for research," she said.

Few people are permitted readers' passes to academic libraries because reading space is limited, books are few, and some are precious. The internet promised to change this. But publishers insisted the internet archive must adhere to the old rules.

The US took a different approach. California-based charity The Internet Archive has copied the web on its own initiative since 1996, when it was assumed the internet would keep public domain information in the public domain.

June Besek, executive director of Columbia Law School's Kernochan Center for Law, Media and the Arts, said the US Internet Archive allowed publishers to use the web's automated "robot exclusion" mechanism to remove their pages from the archive. This was the mechanism the Conservative Party used recently to erase its election speeches from internet archives and search engines.

The culture and history of the world needs to be made available for educational purposes

Joy Springer, senior programme specialist for the UNESCO World Memory project

"That's made people more comfortable with The [US] Internet Archive," said Besek. "It will respect this desire of internet publishers not to be scanned if they don’t want to. It keeps the material some period of months before it becomes publicly available. Publishers are less concerned because they figure they get the advantage of the material exclusively for some period. This seems to have been working in the US."

The US system also differed on the principle that when people published things on the internet for free viewing, it was in the public domain.

The US mechanism was enshrined by a meeting of internet archivists at Berkeley University in California on 13 December 2002 - just two days after the last UK government laid before the House of Commons the legislation that became the 2013 regulations.

The United Nations Educational, Scientific and Cultural Organisation (UNESCO) had itself written the principle of public access into guidelines in 2000.

"Information that's published in the public domain we would recommend that it be made available online so that everybody can access it," said Joy Springer, senior programme specialist for the UNESCO World Memory project.

"The culture and history of the world needs to be made available for educational purposes," she said.

UNESCO has however ceded its lead on building an international consensus to the World Intellectual Property Organisation (WIPO).

WIPO meets in Geneva next week to negotiate an international agreement on digital library archives. The International Federation of Library Associations will have an observer. But the agreement - which may lead to a treaty - will be drawn up by country delegations made up mostly of copyright lawyers. The UK representative comes from the Intellectual Property Office.

Richard Gibby, who chaired the decade-long talks as Legal Deposit Libraries’ manager at the British Library, said in a written statement the committee members had considered reopening the question of archive access. But the talks took 10 years because the negotiating panel tried to create a voluntary regime before falling back on its 2003 powers to create regulations.

A spokeswoman for the UK Department for Communities and Local Government, which led the legislative process, said: "The web archive being produced by the British Library will not be available online. This is to ensure the commercial interests and business models of rights owners are not unfairly prejudiced."

Read more on IT for government and public sector