Spread of Kubernetes spurs backup and disaster recovery products

Container architecture: New challenges

The “traditional” application architecture is well known. Applications run in the datacentre on their server with their operating system (OS), while data resides on the same hardware or in shared storage.

All that – even if, or especially if, virtual servers are used – makes it pretty simple to spin up a fresh deployment in case of a disaster that takes down your primary infrastructure.

At its simplest, you can take backups of virtual machine images and associated data and recover from those, to a known state.

Containers pose new challenges. Although virtual servers are relatively easy to create, use and destroy, the lifespan of containerised applications is mayfly-like in comparison, with most – about 85%, according to a 2019 survey by Sysdig – lasting less than a day.

This reflects the massive advantage of containers. They are easily built components of an application architecture that can be spun up, scaled up and decommissioned easily and quickly – in the cloud as well as on-premise – in most cases today using the Kubernetes orchestration environment.

Why is this good for application development and deployment? In theory, apps can be built and tested on a laptop and reproduced rapidly in the production environment after that.

But although there is simplicity at the core, the operation of containerised apps can also bring huge complexity. They could be distributed globally and, if unplanned downtime strikes, could be in a dizzying variety of states.

Containerised applications are delivered by a container orchestration platform such as Kubernetes.

For each cluster of containerised apps, it runs stateless master and worker nodes, which, in turn, contain the component pods that make up the applications. These can be spun up from the stateful components retained in a central key value database. This is called etcd in Kubernetes, and any changes to the environment are reflected there.

So, the etcd control plane is the first key element that has to be protected. It is like the brain of the containerised environment that preserves the state of the infrastructure and any associated persistent storage.

The second priority will be any persistent volumes used to store data.

If disaster strikes, you will need to restore whole clusters, but it may also be the case that granularity of restore is also needed on occasions.

Core Kubernetes backup and recovery

If you are a small-scale operation, you could, in theory, rebuild your Kubernetes deployments from their git repos. It is totally within the DevOps ethos to rebuild from code.

That, of course, does not take care of any container configuration information held in etcd or any data created by your containers that might need to be recovered.

You could, in theory, back up ectd from within Kubernetes by running and scheduling CronJobs and run restores on those snapshots.

But none of that will be terribly practical in a small and medium-sized enterprise (SME) or enterprise environment, so a number of specialist startups have emerged that supply Kubernetes management with backup and disaster recovery, based on application programming interface (API) connections to the container orchestration platform.

Here is a look at some of those that have emerged.

Specialist Kubernetes management products

Kasten’s K10 data management platform is claimed to be purpose-built for Kubernetes. When deployed, it scans for Kubernetes components that need protection and provides customer-defined, policy-based data protection, mobility, backup and restore, and disaster recovery. It runs in its own namespace in your Kubernetes cluster and its reach extends to state related to persistent storage volumes and databases. Granular restore is possible to point-in-time and application subset. It supports AWS, Azure, GCP and IBM clouds and works on-premise.

Portworx was a pioneer of persistent storage for containers. Late last year, it added its PX-Backup software to the Portworx Enterprise storage management platform. The new functionality protects individual containers, groups of containers or a full Kubernetes namespace with a single command to any S3-compatible object storage. PX-Backup does not require the primary Portworx Enterprise storage management software platform. 

Rancher provides a container management environment that includes access control and allows DevOps engineers to manage application workloads without in-depth knowledge of Kubernetes concepts. The Rancher API server is built on top of the Kubernetes API server and etcd database. In the Rancher UI, etcd backup and recovery for Rancher Kubernetes clusters can be performed. Snapshots of the etcd database are taken and saved locally onto etcd nodes or to a S3 compatible target, which can be remote and used to restore the cluster.

Velero is an open source tool that can back up, recover and migrate Kubernetes clusters and associated persistent storage on-premise and in a public cloud. Velero consists of a server process that runs in the Kubernetes cluster and a command-line interface (CLI) that allows administrators to configure scheduled backups, trigger ad-hoc backups, perform restores, and so on. It uses the Kubernetes API to capture the state of cluster resources down to granular levels.