"It required us to install an entirely new system in three to four weeks to accommodate all the changes we were making," Brooks explained. Disney is heavily reliant on 25 million "texture" files -- small but high-resolution images that are grafted onto 3-D models to create colors and textures, like skin, hair or fur on characters.
Disney, Brooks said, figured out it needed to do two things: replace the Panasas cluster and its included storage with a system three times as big, with three times as much throughput, within a month, tops.
"It was a complete panic," Brooks said. Because of that, the company reached out to Ibrix, already battle-tested by its subsidiary.
Pixar has been using the Ibrix SAN filesystem since the production of its latest feature, Cars. Brooks said the recommendation from Pixar was what spurred Disney to work with Ibrix, but that the system Disney put together was newer, bigger and faster.
"They bought their's a year to two years ago," Brooks said, adding that the moviemaking processes of the two studios remain largely separate. "Ours has much more cache on the Dell Inc. NFS heads -- 32 GB each, much more memory and more capacity." Disney also has a bigger "render farm" of 2,000 server nodes that puts all the digital information into each animation frame.
NetApp OnTap GX wasn't a consideration
Behind the Dell servers and the Ibrix file system is the usual suspect: EMC Corp. Disney has two EMC CX3-80 storage area networks (SAN) running at the back end of this system. The studio also added two new Network Appliance Inc. (NetApp) filers for Tier-2 and Tier-3, using a FAS 6070 Fibre Channel-based system and a FAS 6030 with 500 GB SATA disks respectively.
Asked whether NetApp's OnTap GX clustering system had been a possibility, Brooks responded, "No -- at the time, it really wasn't ready to go." (GX had just started shipping last June; Brooks said the purchase process for the Ibrix system took place around July and August last year.) Otherwise, according to Brooks, Disney is a large NetApp shop in general, and he said he will be evaluating OnTap GX again at a later date.
"For right now, though, we're completely happy with the EMC/Ibrix system," he said. Disney did not evaluate newer Panasas products due to time constraints.
The studio is "in transition" between finishing Meet the Robinsons and starting its next animated feature. The next step after completing Meet the Robinsons footage will be to "park" the files on the Tier-3 NetApp filer for awhile before sending them off to archival media, which Brooks declined to describe in detail.
"There isn't a lot of migration with our tiered storage system," he said. "It's designed so that we put the correct data in the correct place only once or twice during the whole process and then just leave it there. It's more about freeing up performance on the most heavily used files than anything else."