The British Library has one of the digital world's biggest challenges ahead: as technology evolves at breakneck speed, how does it collect, manage and analyse the huge amount of digital content produced by publishers, and how does it keep that information for generations to come?
With its storage requirements quickly approaching petabytes and the output of print material continuing to increase, the library is also starting to consider its role as a research resource of the future. It is not just a case of keeping all this content safe - it has to be easy for researchers to search and analyse.
The question of how it does this is still up for debate. Richard Boulderstone, director of e-strategy and information systems at the library, is part of the team launching the library's Growing Knowledge exhibition, which will investigate the future of research.
The nine-month exhibition, running from 12 October, will feature some of the ideas the library and its partners have had so far. HP, IBM and Microsoft have all worked on possible future research applications and tools, but the exhibition will serve as more than just a showcase of what the library has planned: it wants to start a dialogue with researchers, consult them about the kind of tools they might find useful, and start to figure out its place in the digital world as things continue to evolve.
"The research landscape is changing," says Boulderstone. "New technology is coming out all the time, the internet is being adopted more generally, and we thought it was a really good time to engage with researchers in the deeper debate about the future of research and what role libraries should play."
The British Library might not be the only institution having to deal with the huge shift from physical to digital material, but it is right at the coalface of the change.
Technology now underpins everything the library does, with every main objective of the organisation's 2011 to 2015 strategy containing a technical element. Boulderstone says he is working on his IT strategy at the moment, detailing how technology will help the library achieve its objectives in the coming years.
"One of the main themes of the strategy is guaranteeing access to our content for future generations," he says. "For me, that directly translates into extending the capability of our digital library and working on digital preservation." The challenge is huge but possible, he says. "If we want a book to last a very long time, people have built storage facilities to make that possible. But if you just threw the book on the pavement, it wouldn't last long. With digital it's the same - if you build the right facilities and infrastructure it will last, but if you throw a USB on the street it won't. We'll spend a lot of time developing tools and techniques to ensure the longevity of this digital material and we are well on our way."
The library currently has four data storage sites in the UK, including sites in London, Aberystwyth and Edinburgh. Each site is programmed to respond and recover content if one of the others encounters problems.
"Looking into the future, the things we have to plan for is the large scale-up of that environment," Boulderstone says. "We are going to break into petabytes pretty soon. The current generation of technology for storage has taken us an incredibly long way, but we're still effectively using the same technology as we used in the first PC. Other technologies have the potential for providing much denser storage."
Resources may have to slowly be filtered away from the print side to the digital side, especially with budget cuts expected across all areas of the public sector. "We are having to create a lot of efficiencies in what we do, particularly in handling physical materials, to help fund the digital environment. If the physical side is anything to go by the problem of storage is never-ending. But, as with physical buildings for storage, we will continue to get a better idea of how to do it as time goes by. As digital becomes more mature it will be easier to deal with these large volumes. But we're at the very early stages and a lot of the tools we are using are new."
The Growing Knowledge exhibition is the beginning of the consultation about how the library should move forward, and it is hoping researchers and IT companies will help it determine how it should do so. It is also looking to engage and work with the developer community, who, it is hoped, will have ideas about how to analyse the large amounts of information held by the organisation.HP is a sponsor of 360IT - the IT infrastructure event, taking place at Earl's Court in London on 22-23 September 2010.