Three best practices for virtual machine backup

Learn about emerging best practice for backing up virtual machines - including user case studies - in this feature.

The VMware backup picture is changing almost weekly as VMware pushes out new documentation and products, and as more vendors scramble to certify their data backup tools with VMware.

There are a number of ways to approach virtualized server backup. Your selection will depend on a variety of factors, including the backup tool vendor, the capabilities of the storage area network (SAN) and the recommendations of the virtualization vendor, which in most cases is VMware.

Three approaches for virtual server backup

Ashley D'Costa, enterprise solutions architect at Mainland Information Systems, Calgary, Alberta, a Canadian systems integration and consulting firm, identifies three main approaches to backing up virtualized servers: conventional server backup with the use of individual backup agents on each virtual machine (VM); use of the data backup management tools provided by the virtualization vendor; and the use of a standalone backup proxy server, such as VMware Consolidated Backup (VCB), or appliances from third-party vendors.

A storage/backup administrator might be tempted to simply back up the virtual machines on the physical host like regular files. In this case, the organization would back up each virtual machine's VMDK, a large disk file containing the virtual machine configuration and data. Nice idea, but it might not work well. Unless the virtual machine is shut down, you might back up data in use, which is likely to result in inconsistent data.

The use of an individual backup agent on each virtual machine can avoid data inconsistency by quiescing the application during backup. However, this approach can result in high backup software licensing fees if you must purchase an instance of the license for each virtual machine. It also creates a potential for resource contention unless you stagger your virtual backups. Still, the advantages of putting a backup agent on each virtual machine are simplicity and familiarity. The backup procedure runs no differently and storage/backup administrators can do all of the usual things, such as file-level recovery, and full or incremental backups.

However, the backup application, unaware of the encapsulated nature of the VMDK files, can't do things it otherwise might do to speed backup, noted D'Costa. This approach also undermines some of the efficiencies organizations hoped for from server virtualization in the first place as each agent must be managed individually.

D'Costa's second approach runs the backup software in the virtualization server itself, such as VMware ESX. This will likely be a Linux backup agent capable of backing up all of the virtual machines. However, this results in image-level backups that, although fast, don't allow for the easy recovery of individual files. It also requires scripting to automate the shutdown, snapshot and restart of the VMs.

The proxy server (particularly VCB), D'Costa's third approach, may become the most popular. The proxy server moves the backup from agents on the VMs or from the virtualization server to a dedicated server. The proxy server is typically a Windows server connected to the same SAN volumes as VMware's ESX server.

VMware Consolidated Backup troubleshooting

But VMware Consolidated Backup is far from a backup panacea. D'Costa points out that it requires a number of pieces to work right: a sync driver on the ESX server to flush the file systems and create a snapshot, a vLUN driver on the proxy server to present VMDKs to the proxy, and command line utilities to assist with scripting automation through the command line interface (CLI). It also will require an integration module provided by VMware or the backup application vendor.

Storage managers often mix different approaches. For example, Health First is a three-hospital healthcare provider network and uses VMware Consolidated Backup through a third-party tool, Vizioncore's vRanger Pro, to back up 1.5 TB of data from approximately 220 virtual machines every night. The data is backed up to disk. However, the hospital also uses IBM Tivoli Storage Manager (TSM) and installs a TSM backup agent on some of the virtual machines. The TSM backups go to disk and then to tape, which is shipped offsite.

Virtualization backup and disaster recovery tools

The major array vendors are tuning their backup, recovery and replication tools to work with VMware, often as part of a two-step process in which companies use VMware Consolidated Backup to protect the virtual machines by creating crash-consistent copies and then deploying array-based replication technology to protect the data.

VMware also introduced Site Recovery Manager to automate disaster recovery management.

Site Recovery Manager works with VMware's VirtualCenter management console and with replication software from various storage partners, including 3PAR Inc., Dell Inc., EMC Corp., FalconStor, Hewlett-Packard Co., Hitachi Data Systems, IBM, LeftHand Networks Inc. and NetApp.

Navicure, a Duluth, provider of revenue-cycle management systems for physicians, relied on extensive manual scripting to back up as many as two dozen virtual servers using the replication capabilities built into its EqualLogic SAN (now owned by Dell). "The scripting was taking us hours," said Donald Wilkins, Navicure's IT director. "We wanted to streamline the process." Just trying to recover a virtual machine and promote it to production involved a cumbersome process of mounting files and changing IP addresses.

"Site Recovery Manager automated all this for us, all the scripting and changing of IP addresses. It also lets us test our DR plan non-intrusively," said Wilkins.

To validate this approach, Wilkins undertook the two-hour challenge -- to bring up 10 virtual machines in two hours. "We started at 7 p.m. on a Friday night and began cloning VMs with EqualLogic. We changed IP addresses, replicated them and defined protection groups in less than two hours. We pressed a button and had all 10 VMs up and running 10 minutes later," he said. By 9 p.m. the team was heading home.

UGL Unicco  turned to STORServer Inc.'s STORServer Appliance for VMware Consolidated Backup. With approximately 100 virtual machines running on five VMware ESX servers and TSM as its primary backup tool, UGL Unicco uses the STORServer Appliance to interface with Tivoli, VMware and VCB. For most of its VMs, it puts a TSM agent on the server and backs up in the conventional way to STORServer disk, which spins it off to LTO-4 tape. It uses VCB with approximately 30 critical VMs to allow for file-level recovery.

"We're an IBM shop and we liked what Tivoli does, but it's very complex, highly scripted and uses a CLI, so we went with STORServer as a GUI front end," said Darrell Stymiest, UGL Unicco's network services manager. VMware backup was similarly challenging, requiring extensive scripting through another CLI. With the STORServer Appliance for VMware Consolidated Backup, "we can take the VCB snapshot and write it to disk on STORServer and then send it to tape," said Stymiest. Recovering files with VMware Consolidated Backup remains a cumbersome multi-step process, but it's much improved over previous VMware backups. "We want to use VCB with 90% of our VMs eventually," he added.

"Backing up virtual servers isn't simple and the tools aren't perfect," said Mainland Information Systems' D'Costa. But the virtualization backup process is still immature. "Backup tool vendors and array vendors are still trying to conform with VMware," which is still evolving its tools and APIs, added D'Costa. Until the virtualization industry matures in a few more years, backing up virtual machines will remain a challenge.

Read more on Data protection, backup and archiving