The British Library is on a mission to digitise its
immense collection of books and manuscripts. It is a challenge of
mind-boggling proportions, as Helen Beckett
discovered.
The British Library, whose reading rooms have been frequented by
the likes of Karl Marx, Charles Dickens and Virginia Woolf, is on a
very modern mission. Its aim is to preserve the UK's digital
heritage, as well as continue its stewardship of physical tomes.
But the rapid obsolescence of software publishing formats could
make this task more challenging than preserving the Domesday
Book.
One of the world's great treasure troves, the library houses 13
million books, seven million manuscripts and 4.5 million maps, as
well as 3.5 million sound recordings, eight million stamps and 58
million newspapers in various formats. Although most of its content
is ink on paper, Roger Butcher, programme manager for the
Integrated Library System, says, "The future is without doubt
digital."
The library's digital vision is a strategy of two halves: to
keep in perpetuity the UK's digital publication output, and to
provide digital access to the library's body of historical
material.
The latter encourages a more outward-looking and commercial
perspective as documents are delivered to readers all over the
world. Plus, digitisation improves the quality of access. "It is
possible to view the Golf Book - an illuminated manuscript from
about 1540 showing one of the earliest illustrations of a golf
player - much better on the web than through the glass of an
exhibition case," says Butcher.
Digitisation of existing stock is the most straightforward piece
of the strategy, but it is a huge task. Since the project began 10
years ago just 0.3% of the library's collection has been committed
to code. One inhibitor is cost, as replicating an image in binary
code is just a small part of the process. As well as scanning
documents, other components include building the database,
compiling metadata, plus the project management. All told, the
library calculates that it costs £1 to digitise each page and there
are still more than five billion pages to go.
The bulk of the digital conversion is being performed by the
British Library's sister site in Boston Spa, Yorkshire, with the
aid of a scanning package called Relais. However, the library has
also accepted assistance from Microsoft, which estimates it can
halve the cost. To date, Microsoft has pledged to digitise a
further 25 million pages in the form of 100,000 out-of-copyright
books and deliver search results for this content through the MSN
Book Search service. MSN Search will launch an initial public beta
this year.
However arduous the task of digitisation, it is the other piece
of the strategy that really challenges the library - preserving the
UK's digital publication output. "The bit that no one has the
answer for is digital preservation," says Richard Boulderstone, the
library's director of e-strategy and programmes.
Digital material is ephemeral owing to the changing nature of
software publishing formats and the effort of maintaining sites.
Thus, the original Domesday Book is still readable, yet the laser
disc version produced by the BBC is already technologically out of
fashion. Likewise, the paper archives of elections in the last
century are still intact, but the web content that powered this
year's contest has already disappeared from view.
Another problem that the library has to contend with is the lack
of a standard publishing format. Electronic publishers use a
variety of file formats, including Word, Excel, PDF, HTML, JPeg,
Gif and CSV. And though most are moving to XML, they are using
different document definition types. One hundred years from now,
those file formats will be out of date. "Keeping the bits is not
enough - you need to be able to recreate from them something that
readers in decades and centuries to come will be able to use," says
Boulderstone.
The library has scouted around for answers to this conundrum and
has come up with three solutions that will be familiar to anyone
involved in data migration.
The ideal is to create a universal virtual machine that can view
any file format. File format conversion provides a second and more
practical option, taking the old file and converting it into a
format that is readable by the current version of software. Third
is file emulation, where instead of changing the original file
format, the platform on which it was originally viewed is
emulated.
"The purists seem to think file emulation is the way to go, but
it seems very complicated. We prefer to put our efforts into data
migration - it is something we can do here and now," says
Boulderstone.
Whichever technique is selected, the task of preserving digital
heritage will be a large one because of the library's copyright
status in the UK. The Legal Deposit Libraries Act of 2003 entitles
it to a keep a copy of any publication in the UK, thus paving the
way for a national digital library. Online access has to be
carefully managed in order to safeguard any commercial interests of
owners.
"Certain e-journals may likely only be viewed from the reading
room because anything else might undermine the publisher's business
model, and that is not the purpose of a legal deposit," explains
Boulderstone.
Digital rights management, therefore, promises to be an
important component of the Digital Library Programme. "It is very
important to us, going in to the future, that we have a mechanism
to ensure that readers are approved to access different categories
of material," he says.
But the library does not want to be saddled with the manual
cataloguing of every new digital item it receives, and so the plan
is to automatically generate it from the metadata supplied by
publishers. The library is proposing a standard that would embed
reader privileges as part of the metadata so access could be
filtered accordingly from the library's central catalogue.
Fortunately, there is one part of digital preservation that can
be done here and now: storing data objects. Once an item has been
received and validated as authentic, the object is digitally
signed, sealed and stored. The library uses an algorithm-based
digital signing engine from nCipher to create a sealed package that
is stored four times in multiple modes. "We cannot afford to lose
anything, and the probability of that happening with four copies is
minute," says Boulderstone. As prices for disc and tape media move
closer, discs will play a larger part in storage as the library
moves to store 300Tbytes of data over the next five years.
As the library team mulls over the complexities of a digital
future, it can draw confidence from the fact that the management of
its physical stock is in good shape following the implementation of
a new library system from Ex Libris. The Israeli-developed software
package, called Automated Library Expandable Program (Aleph), was
adopted in 2000 with the purpose of replacing 16 legacy systems
that ran the core functions of managing and locating stock and
maintaining a catalogue.
Reducing cost was a big driver for the upgrade. Legacy
applications were in a gamut of languages from Assembler to Cobol
and were very expensive to maintain as some were over 30 years old.
In some instances, when the packages were retired the people
supporting it took the opportunity to retire too. "In one case we
were the only people in the world using the package," says Butcher
of the Elhill search engine, originally created in the 1960s by the
National Library of Medicine in the US.
As well as cutting costs, there was the desire to be part of a
modern information network. "As a national library, historically we
had felt 'different' and had built a lot of custom packages to
reflect our unique status. However, we took the positive view that
we now wanted to be the same," says Butcher. "A lot of our
functions are not so different from other libraries. Rather than go
the bespoke route of modifying packages, we wanted to have the
benefit of sharing experiences with other users."
The British Library put out an invite to tender and evaluated
five contenders. An end-user requirements group consisting of
library staff and readers sampled products and gave feedback on
aspects including the intuitiveness of the user interface and
search function. A priority was to choose a system with a workflow
system that worked for the library's diverse staff. "With our
legacy systems, we had many people working in silos. With one
integrated system we had the chance to reorganise workflows and a
big issue was how these systems looked," says Butcher.
Although keen to join the mainstream with its system, the
British Library was conscious that its size could complicate
matters. It drew comfort from the fact that two other library
colossi, the Russian State Library and the National Library of
China, were users of the Ex Libris system. Another plus for the
product was the work Harvard University had done with Ex Libris on
streamlining the workflow. The British Library acquires 100,000
books and 300,000 journals each year from legal deposit alone. "The
streamlined workflow means we are able to cope with these sorts of
volumes," says Butcher.
The integrated package was rolled out and the legacy systems
phased out, but work is still in progress. The latest phase is
incorporating the library's 4.5 million manuscripts, and 750,000
are now in the system. However, this phase brings fresh challenges,
as manuscripts are unique and filed just once by their authors. For
this reason they need far more descriptive records, and the normal
short metadata tag provided by the library system is insufficient
for the wordy descriptions that are normally filed.
Whatever technical challenges in building a digital future lie
ahead for the national library, it can face them as part of a
network of peers. Since its decision to use the Ex Libris system,
the British Library has hosted the Aleph annual user conference and
has been pleased with the rewards of community, says Butcher. "We
feel as if we have come in from the cold. Instead of being on our
own we can look to a worldwide pool of expertise to help solve
problems."
Ex Libris library software
The development of Ex Libris’ Automated Library Expandable
Program (Aleph) system started in 1980, when a team of librarians,
systems analysts and programmers took on the challenge of creating
an automated library system that was efficient, user-friendly, and
multilingual.
Prior to Aleph, the ability to search and display texts in
different languages and scripts, such as Latin, Greek or Arabic,
was an immense task, further complicated by the fact that in some
scripts, characters change according to the character that precedes
it.
Four years ago, the emergence of Unicode as a universal standard
to deal with multiple character sets removed Aleph’s unique selling
proposition. Since then, Ex Libris has focused on developing
functionality that enables libraries to better share and deliver
online material.
Initially Ex Libris released SFX, which constructs and embeds
hyperlinks to electronic resources. A year later it launched
Metalib, a generic search interface that functions across a broad
pool of material globally. Digitool, released in 2002, turns
libraries, archives and museums into publishers and enables them to
share collections while protecting content and copyright.