A benchmarking study by consultancy Hackett Group, titled Beyond world-class service: IT’s digital capability imperative, recently highlighted DevOps as a technique that can help enable digital transformation.
“Agile, DevOps and continuous development models are well-suited to digital business transformation, which demands accelerated development, rapid prototyping and incremental value,” according to the report.
Similarly, a Forrester survey of 4,976 IT professionals found that firms with high-performing business technology “solution delivery organisations” were 1.5 times more likely than their peers to exceed their profitability, market share and productivity goals.
For many IT departments, where roles are delineated between various IT infrastructure groups, software development and operations, merging these functions into a single DevOps team is very much their direction of travel.
A survey of 750 IT decision-makers for Claranet recently found that 29% of European businesses have already adopted a DevOps approach, with a further 54% expecting to do so in the next two years.
The reason so many IT leaders feel DevOps is right for their organisation is that they believe it can deliver better applications (54%), greater operational efficiency (53%) and improved customer satisfaction (45%), according to Claranet’s survey.
Fixing a broken DevOps pipeline
However, getting there can be a challenge for many businesses. “There is an understandable rush to implement DevOps as the promise of delivering technology faster and more efficiently is compelling,” says Tim Russell, chief product officer at Perforce. “The immediacy of action often drives quick-fix or easiest path solutions that, in turn, drive significant, yet unintended downstream costs and efforts, even if it seems as though improvements are visible in parts of the process.”
Chris Hill, software development manager at Jaguar Land Rover (JLR), explained the problem the car maker faces with DevOps in a video interview with Computer Weekly, filmed at the recent DevOps Enterprise Summit (DOES 2017) in London. “A poor software development lifecycle is slower than it needs to be, has roadblocks and a slow path to production,” he said.
If a developer cannot easily see how to get their code into production, or the path that needs to be taken is convoluted, then, for Hill, chances are features are not being released as quickly as they could be.
The situation at JLR, which is unique to certain industries such as automotive, is that there is heavy use of embedded devices. “When we are putting software into vehicles, we do not have the luxury of a web developer,” said Hill.
Clearly, it is not feasible to spin up a fleet of vehicles to run automated test suites. Instead, he says the team has to rely on virtualisation and software-based infrastructure to enable it to build code that is representative of the operating environment of a production vehicle.
People often argue that the cultural change is harder than the technological change, but like JLR’s heavy reliance on embedded systems, some technologies can prove immutable.
Among the big roadblocks for many organisations is when DevOps needs to work with legacy IT. This is the situation Justin Dean, senior vice-president for technical operations at Ticketmaster, described at the Cloud Native Computing Forum earlier this year. Dean believes DevOps principles still apply to those people who manage legacy software stacks.
At Ticketmaster, the team that writes software also has operational duties. “Part of our DevOps transformation has been to support teams that may not have had access to the technical environments needed to deploy their software and let them operate in a DevOps fashion,” said Dean.
One of the approaches Ticketmaster has taken to integrating legacy IT with DevOps teams is the concept of a pipeline of work, where tasks that need to be completed by the legacy team are submitted. Once that work has been done, the DevOps team can complete the deployment.
“When you deliver software in a container through a pipeline, it forces a massive change. This single-handedly had more impact than anything else and dovetailed into DevOps,” said Dean.
Getting the DevOps talent you need
A common knee-jerk reaction to staffing the infrastructure and operations members of an integrated product team is to look outside the firm. However, attracting the right kind of DevOps experience can be difficult.
As more companies adopt modern service delivery methodologies, there is increasing demand for qualified and experienced professionals. For example, in the US, the number of jobs classified as DevOps between December 2014 and December 2015 grew by 15%. And those who can step in and help an integrated team are paid top dollar. From October 2013 to July 2014, salaries in the US classified as DevOps increased by approximately 30%. Compare this salary cost with the cost of selecting and training your current resources.
But even if you are able to find candidates you can afford, you may not be getting the experience you are paying for. To land the higher paid positions, candidates are including DevOps experience on their CVs even if they haven’t worked on an integrated product team. “You may be interviewing a candidate who says they have experience as a DevOps engineer, but instead they have experimented using Puppet or Chef for a short time at their current employment,” says Stephen Thair, co-founder of DevOps Guys.
Source: Organise and staff I&O pros for successful DevOps practices, Forrester
The challenge of scaling a DevOps culture
Culture can be the undoing of many an IT project. DevOps is no exception, and while it may be seen as a software development and IT operations concern, success often requires a shift in altitude across the whole business.
Multinational banking group BBVA has used DevOps at scale across multiple regions as part of a transformation programme. In DevOps, teams take ownership of business products. How does this pan out in a multinational business?
Speaking to Computer Weekly at DOES 2017, Brian Timmeny, head of advanced engineering, DevOps and IT processes at BBVA, said: “When you scale DevOps at more than a team level, some things can become more prescriptive.” His advice to any large enterprise is to start with a simple message: “You are going to achieve speed and you are going to achieve quality.”
With 140,000 employees and 12,000 developers he said the transformation had to involve the business. The bank started by attempting to define the meaning of a BBVA product.
“One of the key tenets of DevOps is for a team to have end-to-end responsibility for the product,” said Timmeny. “You have a team that will build it, deploy it and monitor it – they will own it, cradle to grave.”
What makes this difficult is that the bank is organised by function. While conceptually the product definition may be loosely defined, what is key, according to Timmeny, is to have an internal team in place that is truly self-sufficient.
He said the bank could then start to wrap a business function around the team: “This gives us a place to start – to have a set of core principles that, to begin with, can be done in a minimal viable way. Over time, everything else will evolve.”
This approach has needed top-down support, but one of the great catalysts in banking is that the financial sector knows it has to change. What this has meant at BBVA is that there is full support from the top down to drive through transformational change, according to Timmeny.
Getting the right skills
It may be tempting to hire external experts to fill the gap in skills that needs to be bridged to shift the IT department to DevOps. However, analyst Forrester warns against recruiting from outside to bolster DevOps capabilities. Its Organise and staff I&O pros for successful DevOps practices report found that the number of vacancies for DevOps roles rose by 15% between December 2015 and December 2016. Salaries have also risen, but there is a sense that some applicants are “DevOps washing” their CVs to make them appear to be DevOps experts.
“The success of the integrated product team relies on the ability to pull expertise from all resources required to create a successful digital solution,” says Forrester principal analyst Amy DeMartine, in the report.
“As such, resources from application development and IT and operations professionals need expertise from security professionals, customer experience (CX) professionals, quality assurance (referred to as QA or testing), and enterprise architecture (EA) and business architecture (BA) professionals. The success of these teams is measured by meeting the customer experience goals.”
Recruiting the right set of people can be a big headache, according to Tom Clark, head of common platform at ITV, and a DevOps champion. Speaking to Computer Weekly at DOES 2017, he said: “As technology has moved forward, the skills you require have moved up. But at ITV we have boiled it down to two qualities: you have to be smart and you have to be kind. Anything else is a bonus.”
At ITV, “smart” means the ability to adapt to change, since the tech used today is not likely to be the tech of tomorrow. Clark says “kind” is the ability for someone new to fit into the team. “Don’t be a douche, just be nice,” he added.
Loosely coupled progress
From the experts Computer Weekly has spoken to, a theme seems to be emerging. It can be summed up in the latest State of DevOps report: “In a loosely coupled architecture, it’s easy to modify or replace any individual component or service without having to make corresponding changes to the services or components that depend on it. In an organisational sense, teams can be described as loosely coupled if they don’t depend upon other teams to complete their work.”
So Jaguar Land Rover is working towards developing software-defined infrastructure, which enables it to test software virtually, rather than wait to deploy on the embedded systems running in its vehicles. BBVA’s Timmeny talks about the need for teams to be self-sufficient. Where this is not possible, as in the case of Ticketmaster, there is need for a pipeline to provide a clean handover between DevOps and legacy IT.