Backup software: BakBone upsets in Quality Awards

This year's Diogenes Labs-Storage magazine Quality Awards for backup software had a come-from-behind winner: BakBone's NetVault.

BakBone NetVault placed first in the enterprise category, besting better known backup software applications such as Symantec Veritas NetBackup, EMC NetWorker and IBM TSM in the latest Diogenes Labs– Storage magazine Quality Awards survey.

More info on Quality Awards
HDS repeats as enterprise arrays champ

HDS wins enterprise NAS

NetApp's midrange quality up for debate

Top tape libraries revealed

Backup and recovery can often seem like eating your vegetables: You know you have to do it, but you don't have to like it. Among all of the product categories surveyed for the Diogenes Labs– Storage magazine Quality Awards, backup and recovery (B/R) has generated the lowest overall scores. User satisfaction apparently remains elusive, as our 2006 survey yielded even lower scores than the 2005 edition. There's also a greater separation between the top and bottom scores, and one out of three respondents wouldn't make the same purchase decision today if they could do it again.

Perhaps the relative antipathy toward B/R is the result of the attention it requires.

"There's no backup solution in the world that doesn't require daily intervention," said Lynn Haueter, senior systems engineer at Alliance Coal LLC, Tulsa, Okla. "It's not a ship that you can put on autopilot."

But not all of the B/R survey results are bleak, as three products significantly increased their scores compared to last year. BakBone Software Inc.'s NetVault, which came in near the bottom in 2005 with a weak 4.60 overall score, was a surprise enterprise-class winner this year with a strong 5.97 (see the "Overall rankings" section of the "Rankings of enterprise backup and recovery software" chart). NetVault's score surpassed last year's winner, CommVault Inc.'s Galaxy, even though Galaxy's score improved from 5.32 to 5.59 (second place). Hewlett-Packard (HP) Co.'s OpenView Storage Data Protector fell from second place to third with a 5.13. IBM Corp.'s Tivoli Storage Manager (TSM), Symantec Corp.'s Veritas NetBackup and EMC Corp.'s NetWorker—which combined control the majority of the B/R software market—all finished in the bottom half of the finalist group. TSM's 5.04, Veritas NetBackup's 4.68 and NetWorker's 4.66 were all slight declines compared to last year's scores. Syncsort Inc.'s Backup Express didn't have enough responses to be statistically valid, but it still wouldn't have been a contender based on the responses it received. CA Inc.'s BrightStor Enterprise Backup was not included in the 2006 survey because the company discontinued new sales of the product a year ago.

In the small and midsized business (SMB) category, Symantec's Veritas Backup Exec placed first (5.14), narrowly edging out last year's winner, EMC Retrospect (5.12) (see the "Overall rankings" section of the "Rankings of SMB backup and recovery software" chart). In the closest competition of any category thus far, Veritas Backup Exec won three categories while Retrospect snared two. Veritas Backup Exec's overall score in this category improved from a 4.99 in 2005, while Retrospect's fell from a 5.42. CA BrightStor ARCserve Backup fell from a slightly positive rating of 4.58 in 2005 to a negative satisfaction rating of 3.93 in 2006. Arkeia Software's Network Backup, Atempo Inc.'s Time Navigator, BridgeHead Software Inc.'s HT Backup and Yosemite Technologies Inc.'s Backup Standard (formerly TapeWare) were included in the survey, but didn't get enough responses to be statistically valid.


Fig. 1: Rankings of enterprise backup and recovery software

How much with how many

For 2006, we also asked how much storage was backed up by respondents and how many administrators they used. Out of the 442 respondents who supplied this information, 131 (30%) use two administrators, while 357 (80.1%) use five or fewer. Of the 25 organizations that use 20 or more administrators, most were in the financial or IT services industries.

We were curious to see which backup software product could lay claim as the "heavy lifter" of the industry. We defined heavy lifting as a workload greater than 20 terabytes (TB) backed up. Based on this criterion, IBM's TSM hoists the most bytes, with 54.6% of respondents using it to back up more than 20 TB. It's followed by Veritas NetBackup (43.3%), EMC NetWorker (31.4%), HP Data Protector (28.8%), CommVault Galaxy (28.5%) and BakBone NetVault (7.7%). However, Veritas NetBackup had the largest number of respondents (21.4%) using it to back up more than 100 TB vs. TSM's 19.6%. NetVault, the enterprise category winner, had no respondents using it to back up more than 50 TB. Among SMB products, more than 80% of respondents use the products to back up less than 10 TB. Two Veritas Backup Exec respondents and one CA BrightStor ARCserve Backup respondent indicated they use these products to back up more than 100 TB; there were no respondents using EMC Retrospect for more than 20 TB.

Recent market hype regarding disk-to-disk backup prompted us to ask which backup software applications are used for this function. It came as no surprise that IBM's TSM had the largest number of such users (84.5%), as disk staging is part of its basic architecture. "It does it really well," said Alliance Coal's Haueter of TSM's disk-based backup prowess. But we were surprised that CommVault Galaxy was almost as high (81%), while products like Veritas NetBackup and EMC NetWorker were relatively low (49.7% and 34.3%, respectively). We were also surprised to see EMC Retrospect used for disk backup by 57.1% of its users, a high percentage for the SMB group.

"We were switching from backup to tape, to disk-to-disk-to-tape backup," said Keith Price, system administrator at Johnson & Wales University in Providence, R.I. "Since we were doing a significant redesign, we went through and did a full investigation into which product would work best for us."

That re-evaluation resulted in replacing Veritas NetBackup with CommVault Galaxy. "They [CommVault] came out better priced and seemed to be a better product," Price said.

BakBone the overall winner, but beaten out by CommVault on product features: Page 2.

Read more on Integration software and middleware