EMC, NetApp duke it out in the midrange storage

Last week, NetApp published the results of a test it commissioned from Lionbridge Technologies Inc.'s VeriTest labs, claiming that the newest FAS3000 model, the 3070, out-performed the newest high-end Clariion, the CX3-80. The test measured an OLTP workload using 3.2 terabytes (TB) of 400 GB logical unit numbers (LUN) on 200 Fibre Channel (FC) disks within each array, and found that the total aggregate throughput performance for 8 KB random reads and writes on the 3070 topped out at 31,109 input/output per second (IOPS) vs. 28,352 IOPS on the Clariion.

"NetApp delivers better performance and lower latency [than EMC]," trumpeted NetApp in a press release.

@29074 Hold the phone, was EMC's response. In an email to SearchStorage.com, EMC officials wrote, "The VeriTest performance test was run using a small fraction of the available capacity of the system … out of the 28.8 TB of raw capacity, they are using only 3.2 TB or 11.1%." Furthermore, EMC added, "This sub-15% utilization avoids [a] massive performance degradation [on] … NetApp's SANs."

In a white paper based on its own internal laboratory testing, EMC's own charts showed NetApp's 3050 with higher initial performance than the Clariion CX3-40, which EMC argues is the more comparable product. But, the paper argued, as utilization of the SAN increased beyond 15% of the total system capacity, performance degraded more sharply on the NetApp system than on the Clariion system.

A he-said-she-said battle over product claims is hardly new, analysts said, but it is unusual for EMC to respond in such detail about performance claims. "It shows that the midmarket is a huge open battleground," said Brad O'Neill, senior analyst with the Taneja Group. "That's the clearest thing that can be discerned from this."

IDC numbers released in September showed that NetApp is, in some ways, encroaching on EMC's SAN territory. According to the report, NetApp surpassed EMC in storage terabytes shipped, at 68,898 to EMC's 66,581. EMC, meanwhile, argued that Dell Inc.'s shipments of EMC storage weren't factored in to that report, and EMC remained highest in revenue.

What's the bottom line?

NetApp is making strides, according to users and analysts, but they also said there is a nugget of truth to EMC's report.

The issue centers around NetApp's Write Anywhere File Layout (WAFL), the file system at the heart of NetApp's storage devices, whether SAN or NAS, Fibre Channel or iSCSI. WAFL is a log-structured file system (LFS). WAFL was actually first conceived in order to speed file system performance over traditional CIFS and Network File System (NFS) methods, in which disk heads must seek out disk locations that are optimal for multiblock read operations later on, and then write the block to the platters. Log-structured file systems, on the other hand, append all writes to a consecutive stream of blocks (the log), allowing writes to disk to happen faster.

However, there are two possible inhibitions to performance over a "pure" block-level access system under WAFL: first, blocks must pass through a file system "layer" on their way in and out; and second, as systems grow larger, the log grows continuously, leading to a possible slowdown in performance as systems grow.

But for many users, particularly in the midrange where performance is not as crucial as the high end, WAFL provides a number of distinct advantages over traditional block- and file-level systems. Chief among these advantages is the ability to natively serve any protocol, whether block- or file based, whether Fibre Channel or iSCSI -- something EMC is still catching up on. NetApp is also widely regarded as the first pioneer of snapshots, a feature that's also part and parcel of the WAFL system.

"Although WAFL theoretically presents a performance issue in block-level systems because it is another layer for block-level access to go through", said Brian Garrett, technical director of the Enteprise Storage Group (ESG) Lab, "a unified approach enables Netapp to have a single product line that supports both block and file in the same product line. A unified approach also provides simplified access to performance and capacity optimized WAFL, features like snapshots, writeable snapshots (FlexClones) and thin provisioning (FlexVols)".

Others predict that it's only a matter of time before NetApp comes out with direct block-level access as well.

"Any time there's any extra code in the middle, so to speak, somewhere, you have to pay the piper," said Arun Taneja, founder and analyst with the Taneja Group. "At some point in time, NetApp will find a way to go to the block [level] without even the nanosecond it spends now in WAFL."

NetApp declined to comment on whether or not it had any such plans and also declined requests to specifically discuss the technical aspects of the WAFL system and performance.

Performance doesn't necessarily tell the whole story

According to Michael Israel, senior vice president at Six Flags Theme Parks Inc., his company swears by NetApp and the 3000 series. Each amusement park boots commodity servers from Hewlett-Packard Co. (HP) off of FAS 3020s and 3050s, depending on the size of the operation, and replicates that data to a central data center, where Israel said he is in the process of upgrading to the 3070 for capacity reasons.

"It's just a matter of swapping out the [controller] heads" to upgrade, Israel said. "That's one of the things I like about NetApp."

Other things he likes: the ability to run iSCSI for booting his servers and for some SQL database applications in the same system as file and print shares for corporate headquarters, and snapshots.

Israel admitted that for his purposes, performance wasn't a priority. In a "former life" at a billion-dollar financial institution, (Israel declined to name it) just before he joined Six Flags eight months ago, Israel said he himself had tested EMC's CX700 against a NetApp 3050, as well as boxes from HP and Sun Microsystems Inc. Head to head, Israel said, the boxes performed similarly, but the company still went with the Clariion for 15 TB of Oracle and Sybase databases and Unix servers, and purchased the NetApp to run alongside it for NAS and Windows applications.

"It was more of a reputation thing than anything else -- it was a comfort feeling," Israel said. "At the time, EMC had eons of customer references running Unix systems, and NetApp wasn't quite there."

Nowadays, Israel said, "We don't have that kind of budget leeway at Six Flags." He added that EMC systems would have been much more expensive for his operation than NetApp -- another of the primary reasons Six Flags is a NetApp shop.

"At some point, customers may become impervious to the back and forth over products between vendors," O'Neill said. "In the end, raw performance numbers are nothing more than a useful data point -- every user is different and users are often more or less familiar with one vendor."

In other words, O'Neill said, both EMC and NetApp's performance claims are probably "100% true -- but 5% relevant."