CI/CD series - Sumo Logic: the pipelines provide continuous data – are you using it?
This is a guest post for the Computer Weekly Developer Network in our Continuous Integration (CI) & Continuous Delivery (CD) series.
This contribution is written by Mark Pidgeon, vice president of technical services at Sumo Logic — the company is known for its cloud-native machine data analytics platform.
Pidgeon reminds us that the model for CI/CD aims to make the process for getting software into production easier, faster and more productive. The pipeline model defines the stages that software has to go through and then makes those stages more efficient through automation.
Pidgeon writes…
The reality of the pipeline works well for getting software out.
However, it’s not the whole story when it comes to the data that this creates over time. Each stage of the development process will create data… and those applications will add data once they are in production.
While it is a by-product of running applications over time, this information acts like a ‘digital exhaust’ that developers can tap into and use over time. This data can – if approached correctly – support a more accurate feedback loop for development, for deployment and for business teams.
By looking at the impact of any changes on application usage, it’s possible to understand how well the application is performing from a technical viewpoint as well as from usage and business demand perspectives too.
Business-wide CI/CD
But truly effective CI/CD should inform the whole business.
This involves thinking about observability from the start. Application logs, metrics and traces can each provide insight into the application and any changes taking place, but the sheer amount of data coming through from these continuous changes has to be understood in context. Putting this data together from different sources – particularly from software containers – can be difficult as it involves a lot of normalisation, after which it can be linked up to data from any CI/CD pipeline.
Linking data sets from application components ahead of release into production provides a strong ‘before and after’ set of data. This can be useful for looking at the impact of changes as part of any move to production.
This data is created continuously during any development, and it provides data that can be used over time for making better decisions across the software process.
However, it does not end there – this data can show where more work is needed in security and compliance, where business decisions affect IT and where IT choices affect business performance too. This continuous stream of data can be made useful for more people over time.
As more companies expand their CI/CD processes to speed up their businesses, the end result should be more efficiency across the whole operation. This approach should help companies improve decisions using metrics and data. However, this is a mindset change. Moving to a continuous intelligence approach can help.