How Ocado built an online grocery delivery platform business

Manageable service

The company also wanted to develop its platform in a way that would enable it to provide a manageable service, he adds. “This is why we chose a best-of-breed technology stack with Kubernetes, giving us the lowest cost of maintenance and the simplest way to automate.”

Describing how the platform is set up in a typical warehouse, Dabrowski says Ocado operates three availability zones, each using OpenStack for compute and a Ceph cluster for storage. On top of these, the platform runs Docker containers that are used to build out the microservices architecture.

Dabrowski says: “We can run warehouses in the UK and abroad for multiple customers and support multiple warehouses per customer. In the warehouses, we manage the technologies to enable customers to deliver groceries in their geographies.”

The service Ocado provides is delivered as a PaaS, says Dabrowski. The customer provides a warehouse, into which Ocado installs its robotic automation grid. Robot trolleys roam the warehouse on a grid above millions of grocery items, and can assemble a typical 50-item shopping basket order in five minutes. “We operate the grid on behalf of our OSP customers, and we do it at scale,” says Dabrowski. “If we lose an availability zone, the system rebalances.”

One of the areas Ocado has been pioneering is the idea of building a datacentre as a mesh network of computing nodes. The open source project, called Kubermesh, is self-provisioning and self-healing and builds on Kubernetes. Within a few minutes of being powered up and connected, a new node is incorporated into the cluster. It will then be able to present itself, offering extra capacity, so containers can then be scheduled onto the new node as required.

In a blog post introducing Kubermesh, Ocado described how its largest grocery warehouse in Erith, which spans 563,000ft², would need 400 nodes randomly dotted and wired together to create the mesh. The applications deployed on the nodes could then be strategically placed near other applications that they would often communicate with, for optimal speed and performance, it said.

Kubermesh effectively provides a bare metal, self-hosted, self-healing/provisioning, mesh network Kubernetes cluster.

Mike Bryant, platform automation engineer at Ocado, explains: “Kubernetes provides us with the software to deploy and run all our containers for microservices. A key advantage is its ability to self-heal, which makes it incredibly reliable. We run multiple copies across all critical services. As long as we don’t lose more than a third of our compute nodes, we can carry on running.”

Benefits of cloud native

For Dabrowski, cloud native is an IT paradigm shift compared with the way IT was run a decade ago. “People used to buy best-of-breed products like Oracle for their database and NetApps for storage,” he says. “They put too much on the same technology stack. If a product or service in the stack was not available for some reason, all the applications built on top of it were also unavailable, making repairs very difficult and upgrades costly.

“To upgrade, you had a huge monolithic stack to upgrade and the process took 18 months because you had to coordinate downtime. Cloud native offers a seamless migration, and upgrades need no intervention.”

A cloud-native architecture built using microservices running on a computational mesh comprises many on small blocks. If one or more fail or need updating, the overall service is still available, he says. “Maintenance, roll-outs and outages are all handled seamlessly,” adds Dabrowski.