DreamWorks draws on storage for animation
DreamWorks' senior technologist says it takes a lot of emerging storage technologies to make an animation movie.
Comparing his shop to a high-performance computing environment, DreamWorks senior technologist Skottie Miller said at a Storage Networking World session that those technologies help maintain storage tiers and keep down costs, while making sure all users have the resources they need.
NetApp's A-SIS deduplication, Ibrix Inc.'s clustered storage on the MSAs and file virtualization from F5 Networks Inc., formerly Acopia Networks Inc. "It takes a lot of technology to make an animation movie," Miller said.
It especially takes a lot of storage. Miller said storage plays a big role in the animation process because every pixel is created from scratch, the value of data varies over time and all data has a personality -- it could be in the foreground, background, etc.
"Our product is data," Miller said. "But in my view, all that data doesn't make a movie, it makes a pile of data."
DreamWorks uses a combination of compression and data deduplication to reduce that data pile. Miller estimates that compression and deduplication reduces the footprint of a data store by about 30%. Storwize appliances compress data on the fly, and A-SIS performs background scans to further reduce the data store in the post process.
Miller said he decided on NetApp's A-SIS over a dedicated deduplication product, such as a Data Domain Inc. appliance, because the dedupe devices aren't cost-effective in his shop. "With Data Domain, if I don't get 10-to-1 compression, I don't get the value," he said. "And I don't get 10-to-1 because I'm using compression first."
DreamWorks uses F5 switches to move files between NetApp and HP systems. "When I put in a new something, I can move something from the old something to the new something for little money," he said.
Ibrix clustered file systems helps handle large files commonly used in animation, video, and gas and oil exploration applications. Pixar Animation Studios also uses Ibrix for its animated movies. But Miller said he also runs a 10-node NetApp OnTap GX cluster aimed at high-performance computing (HPC) shops. "We used it for small files and transient storage," he said of GX. "In Bee Movie, we used it for a crowd scene. But it's still a few years away from general purpose storage."
Storage technologies haven't solved all of Miller's problems. For instance, he hasn't yet found a search engine powerful enough to let animators find a specific frame from an archived movie so they don't have to recreate it. "Google can't do it," he said. "I'm looking for a deep dive index and search for our proprietary files."
