Long-term archiving needs better media, experts say
The hundred-year archive is coming soon, and industry watchers say a new physical medium for long-term storage should come with it.
With the advent of regulations like the Health Insurance Portability and Accountability Act (HIPAA), which specify that medical establishments must keep data for years after a patient dies, having to keep information -- and even keep it on hand -- for a century is an issue experts said some users will be facing in the very near future.
"Think about the flip side of HIPAA," said Tom Cook, CEO of Permabit Inc., in a presentation at the Storage Networking World (SNW) conference in Orlando, Fla., earlier this month called The Thousand Year Archive. "Right now you have to keep data, but there isn't yet any regulation around [saying] how well you have to be able to access it. That's coming.
"In the near future," Cook continued, "if I'm wheeled into an emergency room on a gurney, there will be a regulation saying that the proper information will have to be available and brought forth to save my life."
Sarbanes-Oxley Act (SOX) and the upcoming updates to the Federal Rules of Civil Procedure, "should start thinking about data in terms of centuries."
"Even if you think in terms of decades," said Steve Remsing, senior systems administrator for research and development at a major Midwestern chemical company, who asked that the company not be named, "Even on the personal level -- people are going to want to be able to access digital baby pictures when the kids are in college."
A two-fold problem
A frequently discussed issue with long-term archiving is software compatibility over long periods of time -- what happens when no one remembers what "Centera" means, but there's still terabyte upon terabyte of disk stored in Centera format? While the debate rages about those issues, the issue of long-lasting physical media is often overlooked. Current digital media formats are far more advanced in the short term, but in terms of readability over vast stretches of time, they've still got nothing on the Rosetta Stone.
One by one, according to Remsing, the different formats can be scratched off the hundred-year archive list for physical reasons. It's difficult to put RAID on tape and difficult to migrate between formats on any form of removable media, whether tape or optical. Disk is flimsy in the long run and requires power and cooling.
Holographic disk a possibility
One company already "thinking in centuries" is Warner Bros. Entertainment Inc. The entertainment giant is currently working toward a concept of archiving its films digitally, a project that would require a capacity approaching 3 exabytes (EB). The film archives are currently stored on film stock, which will easily last a century, but audio tracks for all the films are another matter. Those digital sound files are currently stored on a combination of optical storage -- DVDR jukeboxes --and tape from various vendors, according to Wendy Aylsworth, vice president (VP) of technology and Steven Anastasi, VP of media archiving for Warner Bros.
Right now, according to Anastasi, the software and hardware in the sound archive must be updated every seven to 10 years. Most recently, the archive was converted from the DAT tape it used in the 1980s and 1990s to DVDR and Linear-Tape Open (LTO). Within another decade, the media will need to be replaced again.
"We're pushing very hard with companies that make optical media to figure out what technologies might last at least 50 years and targeting up to 100 years," Aylsworth said. At the very least, "our goal is not to have to replace technology more often than every 25 years."
Right now the best hope, Warner Bros. officials said, is in a form of optical storage known as holographic disk storage, also known as holographic versatile disk (HVD). This uses a multilayered substrate read by two separate lasers, meaning it can pack more bits onto the media. Current optical media has a density of one bit per laser "pulse." Manufacturers of holographic disks are hoping to eventually achieve 600,000 bits per pulse.
"It's just a matter of companies agreeing on a format," Aylsworth said.
"Our hope is that it will soon be recognized that this isn't just an entertainment industry problem," Anastasi said. "It also applies to the medical and high-tech market, anyone who protects high-res images. We're trying to get everyone aware."
"Holographic disk could totally change the equation in this space," Remsing agreed. "It would provide physical write protects, since it's an optical medium. It has good density because it writes and reads from multiple layers of substrate."
Currently, Toshiba, Maxell, InPhase Technologies and Hitachi Ltd. manufacture holographic media. InPhase and Maxell announced their intentions to offer a 300 GB version by the end of this year.
What about MAID?
"I'm not convinced optical is the answer," said Laura DuBois, research director of storage software at IDC. "There's been a lot of movement away from that medium lately."
Instead, DuBois said the future was probably in spinning disk -- but infrequently spinning disk -- a new way of managing disk that would save on power and cooling over long periods, but allow the information to stay accessible. This future medium, DuBois said, would be something like MAID pioneered by Copan Systems, which spins SATA disks down within the array, "waking" them periodically in order to maintain their viability in a process Copan calls "disk aerobics."
The problem with this format is that the disk aerobics still need to be performed too often -- MAID currently is designed around archiving for weeks, months, even a few years but not around decades and centuries. The platters themselves would also need to undergo physical changes for this to work, according to users.
"The lubricant on disks becomes a nice glue if it sits idle for too long," Remsing said. "There's also something called bit rot -- disks today aren't physically built to last hundreds of years."
