"We have yet to see a compelling advantage to disk," said Mark Lemmons, chief technology officer (CTO) of Thought Equity Motion, which licenses production-quality stock clips to movie production houses, cable and Web media companies.
The company, which estimates it serves 5,000 large video files to customers looking for exterior shots or other stock footage to round out programming each week, has 500 terabytes (TB) of video files in multiple formats stored on a mixture of Sun Microsystems Inc./StorageTek 9940 and, more recently, T10000 tape drives in an SL8500 silo, which it manages like a disk system through Sun's SAM-FS storage management software.
More recently, however, when a customer requests a clip, SAM-FS retrieves it through one of the four T10000 drives in the tape library. "Our customers are actually controlling the robot arm in the SL8500," Lemmons said. Once selected, the file is fed directly to Linux Web servers from tape.
Despite advances in clustered NAS and high-I/O storage area network (SAN) systems in recent years, Lemmons said his company still finds tape the best medium for delivering large video files. "An uncompressed high-definition file can require streaming speeds of up to 200 megabytes per second [MBps]," Lemmon said. This requirement put a severe strain on a previous disk-based system from Front Porch Digital meant specifically for video feeds. "Our files are typically very high end -- there was too much stress on that system, which was designed to work with conventional cable television media," Lemmon said.
About a year ago, the company, which also uses Sun/StorageTek Flexline and Apple XSAN arrays along with Advanced Digital Information Corp.'s (ADIC) StoreNext SAN file system for some video tasks, evaluated a wide range of products to replace the Front Porch system. "We considered everything from MAID arrays to clustered NAS," Lemmons said. "We still wanted something with the feel of a disk-based file system."
However, Lemmons said, he discovered that the speed of the tape media from Sun is still greater than that of most disk systems, particularly in newer drives, like the T10000, which advertises a throughput rate of 120 MBps and offers cartridges with a compressed capacity of up to 1 TB.
"We don't have to power and cool that huge archive either, because we're using tape," Lemmons added. "And while our disk systems don't max out our Web pipes, the speeds on the T10000 do."
Not having to power 500 TB of disk comes in especially handy, Lemmons said, since redundancy in the company's archive is a necessity -- the same file must often be stored on multiple formats. "We figured out the cost of ownership per gigabyte for this system is around 25 cents," Noecker said. "For our SAN, it's around $2.50 to $2.75. For an outsourced content delivery network [CDN] it runs anywhere from $1.25 to over $6.00."Thought Equity performs a process called transcoding, which converts files between formats at customer request, on disk, but saves the reformatted files immediately back to the tape repository, Lemmons said. "Then it's at our leisure what we do with it," Lemmons said. "We're not like a Lucas Arts, for example, where we'd need a big pool of live disk for production processing of 3-D sequences."
The T10000 drives have also helped Thought Equity consolidate its primary and offsite storage, according to Lemmon. The SAM-FS system automatically creates and ejects disaster recovery copies of each tape in addition to the production copy for transfer to an offsite facility.
The system is not without its drawbacks. According to Noecker, long-term maintenance of tape media will be more of a headache than with a disk system. "We're already doing periodic audits of our tape archive to make sure the data is there and not corrupted," Noecker said. "As our data grows, this could become more of a management hassle."
Noecker also said that if the company gets to the point where it wants to start "clustering" SL8500s in a high-availability configuration, "the cost there would bridge the gap we have right now between tape and disk." Longer format video files could also still need to be written to DVD and sent out on "mule" drives, he said, depending on their size, though the majority of files Thought Equity serves won't fall into this category, at least in the near future.
For now, he said, "We're a long way from that. Right now our focus is on adding drives to the SL8500 and day-to-day management of the system."
Though the company expects to continue adding capacity to the SL8500, right now it's holding off on a big investment until 1 TB-native cartridges become available. According to Noecker, Thought Equity is also holding off on the encryption feature in the T10000 because of worries about performance overhead until there are more drives in the library.
This was first published in March 2007