DataStax: Inside the brain of the container

This is a guest post for the Computer Weekly Developer Network written by Patrick McFadin, chief evangelist at DataStax — the company develops and provides support for a commercial edition of the Apache Cassandra database, a NoSQL database.

McFadin writes as follows…

Containerisation makes it easier to deploy new applications that can scale. However, there’s a balance that has to be struck between looking at stateless containers and stateful design, as well whether these apps are in one instance or distributed. Moving over to this new approach of software packaging can have some long-term considerations, particularly when it comes to scaling up over time.

So what steps can you take around software packaging?

Containerising an application stack is the start

Creating apps and scaling them up is easier as containers can be made to perform a specific task as needed or composed into a function. Need more power dedicated to a specific task? Add more containers for that role.

Looking at levels of demand across an app can help here too.

For example, more data streaming instances may be needed to get data into and out of analytics instances. Containerising parts of the application should let you add more of those instances as they are needed, for as long as they are needed. If the level of demand goes down, scrap the unnecessary containers and put the resources into something else. This flexibility is important, as it can cut the amount of hardware or cloud resource required significantly.

Containerisation should be a natural fit for distributed computing architectures too. Distributed computing approaches should involve using components that are built to work around failure. Alongside load balancing work across nodes, this approach should ensure that customers don’t see any failure on IT affecting the service they use.

Containers should help you achieve something more

This is where the fun should really start. Using containers as a way to develop apps can be new and exciting, but how does it help solve a problem more neatly or more effectively than using other methods?

A good example here is in the banking sector. PSD2 (Payment Services Directive version 2) will be coming into force in 2018. Retail banking teams have new challenges to solve around how they make their bank’s most critical data available to other entities on behalf of their customers via APIs.

Solving this does not involve bolting APIs on to existing mainframe apps; instead, there is a whole heap of integration work requires to design the APIs, keep them secure, and then figure out what to do with all the customer data involved internally. Deploying that middle layer using containerised instances could potentially deliver far more flexibility and much lower cost than traditional tiered infrastructure.

But wait, there’s more…

Streaming data into different systems to process the transaction, run analytics against it and write the logging files can all take place at the same time, while the data can be put through to the traditional back-end as part of a batch process. However, this isn’t all that can be achieved here.

Alongside the regulatory demands that PSD2 has, there are internal issues at some banks as well. For many of them, linking up internal silos of data has been a big problem over the years and PSD2 forces them to do something about this.

For the canny developer, containerisation is a way to build that middle layer in a repeatable, faster way, while also providing opportunities to add on more services that can provide internal value too.

Helping banks solve their internal problems around knowledge of customers offers a better return to the developer team, and will involve putting additional services in place. By containerising the apps here, they can be approached in parallel to the regulatory side. If PSD2 has the market impact that it is supposed to have, this work will be invaluable to the bank and it can be scaled up without requiring huge amounts of additional hardware and software infrastructure.

The need to ‘re-compose’ applications

Banks should also be able to ‘re-compose’ their applications to add new services without affecting their compliance. Making changes to the user experience or adding new services should not require new compliance implementations every time.

This approach should link software design choices to the business results that are required. Balancing what is possible now and what might be required in the future is easier with containers, as modules within the app can be redesigned and composed as needs change.

The biggest issue I see here for companies is keeping all these moving parts secure and ready for production; if we are to keep our composure here, containerisation is a necessary element to deliver this at scale.