Cybrain - Fotolia

IWM digitises vast collection in SpectraLogic tape archive

Imperial War Museum deploys SpectraLogic tape archive systems with LTFS Black Pearl in digitisation project for 21,000 hours of film and millions of images and voice recordings

The Imperial War Museum has deployed a tape archive comprising SpectraLogic tape libraries plus disk storage with a Black Pearl object storage/linear tape file system (LTFS) front end as an archive to hold around half a petabyte of digitised media.

The move has allowed it to replace diverse, ageing and difficult to manage disk storage systems that included Nexsan Satabeast arrays.

The museum is in the process of digitising audio, video and photographic holdings, and has around 21,000 hours of film, one million still images, plus audio interviews and also acts as the archive for the Ministry of Defence.

As film is digitised it is first captured as a digital picture exchange (DPX) file – effectively a series of tag image file format (TIFF) images – and these are archived. Simultaneously, the movie is transcoded into MPEG, Flash etc for use by researchers, on IWM websites and the like.

The key site for its archives are at Duxford in Cambridgeshire. Here, at a distance from each other of about 500m are two SpectraLogic T950 tape libraries. One is linear tape-open-5 (LTO-5) while the other is IBM TS1150. Currently IWM has about 550TB of data across the archive estate but the two tape libraries are expandable to tens of petabytes.

Ian Crawford, IWM CIO, said use of the two tape format standards is to comply with international guidelines.

“We’re following the OIAS [open archival information system] standard for digital architectures that mandates multiple copies of data, with good practice being to have two tape systems,” he said.

More on tape and LTFS

There is also a SpectraLogic ArcticBlue disk-based storage array, which acts as a landing zone for data as it is ingested. DPX files go to tape – for which access times are not an issue as they are accessed rarely – while transcoded files are retained on ArcticBlue for quicker access.

Front-ending it all is a SpectraLogic Black Pearl Storage Gateway that sends data to the LTO and IBM tape format libraries. Black Pearl allows S3 object storage input/output requests from hosts that are then converted to LTFS; a file system-based approach to access data held on tape, which means tape contents can be searched for and access relatively quickly.

Crawford said his team evaluated tape libraries from HP and Overland among others but settled on SpectraLogic for a number of reasons.

“The T950 stood out for its expansion possibilities and that it would be easy to upgrade between LTO tape systems,” he said. “The BlueScale interface that provides health monitoring of tapes was also a big consideration.”

Read more on Storage management and strategy