Mesosphere ups automation for ‘on-demand data science’

Multi-cloud automation company Mesosphere has now come forward with Mesosphere Kubernetes Engine (MKE), Mesosphere DC/OS 1.12 and the public beta of Mesosphere Jupyter Service (MJS).

Mesosphere Kubernetes Engine is Kubernetes-as-a-Service on multi-cloud and edge… and (as readers will likely know) Kubernetes is an open source container-orchestration system for automating the deployment, scaling and management of containerised applications.

This provides what is known as ‘high-density resource pooling’ for cloud apps, a type of load balancing that provides cloud applications with the processing, storage, networking ability and analytics (resources) they need in environments where data throughput is extreme (high-density)… and it does all this without the need for virtualisation.

MJS is intended to simplify delivery of Jupyter Notebooks, popular with data scientists, to streamline how they build and deliver AI-enabled services. Readers will also note that Project Jupyter exists to develop open source software, open standards and services for interactive computing across dozens of programming languages.

“Companies need to move fast to stay relevant in today’s competitive landscape. To do this, IT teams are leveraging leading tools such as Kubernetes, Jupyter Notebooks, advanced security and software registries to drive software innovation,” said Florian Leibert, Mesosphere CEO.

Leibert insists that by natively integrating Kubernetes and Jupyter into DC/OS, his team is able to deliver fast deployment and centralised management — but still enabling experimentation and providing developer choice.

Mesosphere DC/OS 1.12 is all about giving cloud developers edge and multi-cloud infrastructure from a single control plane. With Mesosphere Kubernetes Engine (MKE), enterprise IT can centralise scattered Kubernetes clusters on multiple cloud providers managed from a single platform.

Where is all this leading us?

We know that many data sets are too large to fit on laptops or individual workstations. This forces data scientists and engineers working with Jupyter Notebooks to repeatedly work with smaller data sets, constraining progress and increasing the risk of data leaks.

It leads us towards what we could call ‘on-demand data science’ – that is, data scientists get instant access to the Jupyter Notebooks computing environment, preconfigured with a good deal of the tools they need.

The move to compartmentalise, package and automate many of these functions is (arguably) very much ‘where cloud-native application is at’ right now… but it’s complex stuff, we need to move carefully.