Quobyte apes Google storage model and plans a tilt at NetApp and EMC

Does the world need another software-defined storage/storage virtualisation product? There are plenty to choose from already, such as DataCore, Nexenta, Open-E as well as software storage products from big hitters like VMware with its VSAN.

That’s what I thought before speaking to Germany-based startup Quobyte, but its case is quite compelling. It’s aim is to provide Google-like webscale server-based grids of hugely scalable storage that are manageable by relatively few IT staff.

Quobyte aims to do this – and to attack the customer bases of the likes of NetApp and EMC Isilon – by decoupling hardware from the software intelligence that runs the storage grid, said founder and CEO Bjoern Kolbeck, who is a former Google employee and contributor to the EU-funded Xtreem OS Linux grid environment.

“At Google we saw how large scale infrastructures worked and it inspired us to take Xtreem FS [the file system from Xtreem OS] and build in ideas around operations and automation from Google.”

“Google has datacentres in which a very small team look after storage and don’t need to interact with the software team. It’s a very nice operational model that scales; if you add more data you don’t need to add more people.”

Kolbeck explained that in the Quobyte view its software-defined storage system needs to provide fault tolerance of “split brain” situations, so that, for example traffic can be automatically re-routed if a broken top-of-rack switch where that might cause database writes to differ between instances. The same fault tolerance also comes into play for day-to-day maintenance. “If a server is down that should not be an exception,” said Kolbeck.

Underlying this is the nub of what Quobyte is all about – namely quorum-based replication. IE, there are three copies of replicated data and there must always be a majority, so two out of three copies and it’s OK to shut down one copy.

Quobyte’s quorum-based triple-copy replication is based on the Paxos lease algorithm, which sees replicated files communicate and decide on a master copy and use that master to ensure data integrity between them. “Then, if that master fails the nodes elect a new master,” said Kolbeck.

Currently, Quobyte lacks features such as synchronous replication that can help it break into the enterprise storage market, but that is planned for later this year. Right now, at least three instances of Quobyte have to be deployed in the same datacentre or metro area for latency reasons, said Kolbeck.

At present Quobyte has implementations aimed at HPC, big data (eg, Hadoop) and OpenStack customer deployments but in the long run it will target NetApp filer users and scale-out storage customers of EMC Isilon, said Kolbeck.

Asynchronous geo-replication is on the roadmap for enterprise users but we can already be a good fit for virtual machine storage and are running some proofs-of-concept where we are looking to replace NetApp,” said Kolbeck.

It’ll be interesting to watch the progress of this marrying of software-defined/storage virtualisation and hyperscale storage.