A data driven approach to cloud management

This blog post is part of our Essential Guide: Essential guide to cloud management

A guest blogpost by Matt Davies, head of marketing, EMEA, Splunk

No matter the size, industry, or geographical location of an enterprise, it will have a cloud strategy. But despite this, when it comes to the ‘Everest’ of universal cloud adoption, we have only just left base camp. As companies embark on their cloud journeys, they increasingly find themselves between destinations with some processes still running on-premises and others running flawlessly in the cloud. This is by design and a natural progression along a journey that may take months or years – a progression that has sparked the hybrid age.

This put us in an interesting place when it comes to data. Think about what we have always done with IT. As an industry we’ve tried to manage all the component parts of our IT estate (the network, the hardware, the operating system, the middleware and the applications). We’ve had to secure it, monitor it, measure capacity and troubleshoot problems. This has led to three main challenges:

·         How do I get a holistic view of how my IT is performing and give access to all the people who need that analysis?

·         How do I identify any issues, troubleshoot them, get to the root cause and fix problems?

·         How do I get a real-time view of what’s happening now and predict what’s likely to happen next?

 All of this IT has generated a lot of data, machine data, that we’ve tried to use and master to help us get some form of operational intelligence and real-time analytics. With cloud computing we’ve now got to do this all over again but with “as a Service” based infrastructure, platforms and applications. This raises yet more questions:

·         If you have IaaS, PaaS and SaaS from many different vendors, how do you manage multiple cloud offerings to ensure the right quality of service and ensure business KPIs and SLAs?

·         Are we learning from our experience of managing on-premise IT when it comes to cloud management and are we using our cloud-generated data in the way we use our on-premise data?

·         How are you going to secure an offering that spans multiple cloud systems? Each solution may claim to be secure but how secure are the services you build on top of them. How do you spot an insider threat if the services are cloud based?

 There is hope. UCAS in the UK is using machine data from on-premise and cloud “as one”. It’s managing multiple cloud offerings, including Azure and AWS. They are combining this cloud generated machine data with on-premise information from Windows, Red Hat and application logs. UCAS’ use of machine data from these cloud and on-premise systems supports a pretty intense few weeks around A level results day, with 180 logins per second during peaks. The data is used to give analytics to their CxOs, deliver improved customer experience for students and education institutions, provide a holistic view of their hybrid IT and improve the security of personal data.

Machine data and analytics is emerging as a key part of an organisation’s armoury when it comes to monitoring, troubleshooting, securing and visualising your cloud based IT operations. Machine data is “source neutral” meaning that it doesn’t matter where it comes from, be that cloud or on-premise. This means it acts as a “common currency” between any component of the IT stack – from the network to the application to the mobile device, from on-premise to private to public cloud.  Increasingly we see cloud providers giving customers access to the log files from their services. Think of AWS and CloudTrail data.

To end, let’s go back to where we started – the world is hybrid. So when it comes to cloud computing it is worth remembering that it’s not only your IT that’s hybrid, it’s your data also. Harnessing this gives you the best chance of taking a data driven approach to cloud management.