DataStax: balancing yin & yang in the holistic dream

Computer Weekly Open Source Insider talks to Patrick McFadin in his role as vice president for developer relations at DataStax.

DataStax is a distributed cloud database built on Apache Cassandra – the firm is a key contributor to the Cassandra project and describes its technology as an always-on data platform.

This interview forms part of a mini-series of posts (also running on the Computer Weekly Developer Network related to the rise of what we have called the ‘holistic’ application.

Not perhaps a formally defined technology industry term, the holistic application is an app that has arisen in the age of cloud with containerisation, microservices and compartmentalisation of discrete components at its core.

CW: Can we put the holistic into legacy?

Patrick McFadin: It is possible to put a wrapper around more traditional applications that provides access to an API. For these legacy applications, this layered approach can be better and safer than trying to completely re-engineer and migrate a whole application from scratch. However, this should not be seen as removing the need to look at legacy applications and how we manage them. Each API call can be eventually replaced with a modern version without the front end code needing to be changed.

CW: Is the ‘cloud developer’ a naturally holistic continuous developer?

Patrick McFadin: Let’s think about cloud – when you implement new services on top of Azure or AWS, you are naturally consuming small amounts of underlying resources, most likely through API calls. So you are naturally going to move over to that more modern approach.

This should open up opportunities to look at tech like serverless and containers, as these help you keep your applications portable. If you want to go all in on public cloud, you can do that; however, this just exchanges one point of lock-in for another. You also have very little control or agency once you are consuming multiple services within one cloud. Large enterprises are looking at this very seriously, as they want to avoid the risk of not having an exit strategy when it comes to moving their data or their application back from a specific provider.

CW: Can you tell us more about cloud lock-in?

Patrick McFadin: Companies did not tend to worry about lock-in so much when it was an internal database sitting in their datacentre. Partly, I think this is because that database company would not be competing with them for customers like an Amazon might; partly, because that internal It felt like it was ‘yours’ rather than being rented from another company. Cloud feels very different, so that exit strategy and need to avoid lock-in has become more important.

Having said that, there will always be certain points of control where you exchange that flexibility for the ability to differentiate what you are doing. If you need specific services in a cloud platform to build a new application or to manage data clusters, then that is acceptable. Enterprise IT teams are going into this environment with their eyes open now, and I think more developers are too.

CW: Do holistic apps naturally promote microservices?

Patrick McFadin: Holistic applications are a great use case for microservices – rather than using one single application to perform a role, you have a set of individual elements that add up to deliver that result instead. There’ no single code path here — and each of those elements are glued together using APIs. This helps companies scale, as each element of an application can have more resources added to meet demand. This can even be automated to make life easier for your developer team, so they can concentrate on software rather than infrastructure.

The end result here is that holistic applications should help you deliver a better service to the customer. By federating all these elements together, you can put together that more personalised experience that includes all the results and data that this customer wants to see.

CW: Is there a downside to holistic architecture?

Patrick McFadin: The flipside of this is that you have a much more complex environment. Each element will be made up of a cluster of nodes running its own service, creating its own data, and having to meet its own SLA. This expansion can be more tricky to manage as it relies on all the elements working together well, and that the data from each of the services is managed effectively

CW: Do we need a different mindset for holistic?

Patrick McFadin: We do… and it’s all about affecting an actual progression in terms of mindset – it’s much harder to call a project ‘done’ as each element can be changed or updated at any time. While the services can be more flexible, it does mean thinking about the role of software developers differently. Companies that have implemented Agile development properly should be equipped to manage this change more effectively – those that namecheck agile or don’t engage in the process fully will struggle with this move.

McFadin All-in public cloud just exchanges one point of lock-in for another