The state of agile software development in the UK

Lately it is more surprising to hear that businesses are developing using a methodology other than agile, such is its widespread use

Today it is more surprising to hear that businesses are developing software using a methodology other than agile, such is its widespread use.

With the rapid digitisation of business, agile software development techniques are being used across multiple sectors.

The ability to cater to demand quickly is necessary in the digital world as fierce competition means customers have more easily available options. To this end businesses in sectors such as retail and elsewhere are investing in agile development.

Large corporates are even restructuring IT to accommodate agile techniques. Mariano Albera, CIO at Thomas Cook, told the audience at a recent Computer Weekly CW500 Club event that big companies can learn a lot from startups and their product development  methods: “I am an agile software developer and have been doing it all my life and I deliver software quickly,” he said.

“The customer is the steering committee. It is about the customer telling you what they want by the way they use. Be pragmatic and be very fast. Don’t focus on five-year roadmaps. Focus on small deliverables."

Thomas Cook has two CIOs. Albera focuses on the applications at the front end that engage customers, using agile development techniques, while the other CIO concentrates on IT operations.

The recent UK Agile Awards in London, hosted by YoH, brought together a panel of experts on agile (see winners listed below). The Computer Weekly hosted panel discussed skills, outsourcing and the future of agile in the UK.

Read more on agile development

Agile has been in finance for years

The finance sector has been doing agile for years but for different reasons, according to Mike Burrows, agile expert at management consultancy firm David J Anderson & Associates. Burrows previously worked in the investment banking sector.

“It could be said that the banks - the investment banks especially - have for long periods been busier competing with each other than focusing on their customers' needs. But whatever the driver, they have long understood the value of fast delivery and feedback. Flexible architectures - for example distributed systems connected by middleware - have since the 1990s enabled business-aligned teams to release functionality at a pace that suits their respective markets," he said.

"Behind closed doors, unit testing and other kinds of test automation, continuous integration, automated deployment and so on were well established [at banks] quite some time before the Agile Manifesto was written.”

Because of distribution channels such as the web, mobile, and dedicated connectivity, the slightest mistake is widely visible very quickly, said Burrows: “Now more than ever, banks need architectures and processes that can respond very quickly. In that sense at least, the need for banks of all flavours to pursue agility is unlikely to go away.”

He also said businesses adopting agile development techniques need to get used to failure.

“Agile could be said to mean accepting the truth that we can make progress with incomplete information. That makes the whole process - not just the specification part of it - dedicated to the process of knowledge discovery. Once we understand this, failure is neither shied away from nor pursued recklessly, but accepted as part of the learning process.”

Agile is not just for business

The UK public sector is as conservative an organisation as can be. Spending is closely scrutinised as is the return on investment. But the government is currently pushing the use of agile methodologies for its digital government initiatives.

Jose Casal from the Agile Methods Specialist Group at the BCS said public sector organisations face two main challenges to adopting agile thinking: “One is the heavily hierarchical command-and-control nature of the organisations, while the second one appears to be the big-bang approach to procurement that dominates the public sector.”

He said the key to a successful introduction of agile in the public sector could be to try not to push agile at all: “If we end up in a situation where agile is enforced as a silver bullet, the likelihood is that that agile will fail to deliver any sustained benefit to the organisation.

“Instead, we need to focus on delivering increased business value, arm ourselves with a lot of patience and go as slow as the organisational tolerance for change,” added Casal.

He said there are “exciting” examples of agile working in the UK public sector such as the website, the Government Digital Service (GDS), the work being done by DVLA and NHS Choices.

Carl Bruiners’s top 5 agile recommendations

  • Don’t have remote teams unless you really need to
  • Have teams within the same time zone
  • Implement test automation
  • Implement continuous integration
  • Create a backlog of work you can deliver as independent features

Agile skills in the UK

Agile is a different way of working and a lot needs to be done in the UK to ensure there are enough people with the right skills, according to Casal.

He said if you use LinkedIn as a measure of the agile skills base in the UK there would appear to be no shortage. But he said, “The reality is, perhaps, not so encouraging.”

“I have visited companies where the agile development teams had no opportunity to apply the most basic engineering practices. We have all heard tales of completely prescriptive adoptions of Scrum where self-organisation, inspection and adaption were nothing but a distant concept. Agile cargo cults, where teams follow the practices without being aware of why those practices are there, are easy to find," said Casal. 

He said many organisations are doing incredibly shallow agile adoptions: “Deep agility in the UK mainstream market is probably a decade away and to help bring it forward we need to go back to the fundamentals of agile and ensure that people understand why agile is what it is, rather than a passing fad. This is what BCS is addressing with the introduction of its BCS Agile certifications.”

James Lomas, CIO at, said the company’s reliance on agile development has led it to build specific training programmes for development staff.

“We work with individuals to develop not only their technical skills, but behavioural aspects like how to give and receive feedback, run a stand-up and operate our product team's 'build it – run it' philosophy.  The idea is that in 12-18 months individuals will be contributing and fully paid up members of one of our product teams,” he said.

Lomas said that is working also work with universities to create a syllabus that reflects the needs of an agile workplace. “This means that students are now learning techniques like test-driven development and continuous delivery, event sourcing and NoSQL databases,” he said.

“These sorts of initiatives may never give us any direct payback, but enables us and like-minded companies to bridge the gap in the future."

Where should agile be delivered from?

If you cannot find the skills nor justify an in-house resource, deciding where to outsource is not as simple as looking for the best price. All suppliers in all locations would be committing commercial suicide if they did not at least claim to do agile, but businesses must scrutinise what they will get from different suppliers in different locations.

Many large enterprises have outsourced their application development to offshore locations such as India to cut costs, but Carl Bruiners, agile consultant at home furnishings retailer Dunelm Mill, said working in an agile framework challenges the practice of offshoring.

"Often I have found in various organisations the decision to offshore to low-cost regions is often not the choice of the company's IT function, but the choice of the finance department," he said.

He recommends IT departments engage with the executive to explain the cost of ownership and the value return. 

“It is often narrow sighted not to consider the other factors you inherit when offshoring - language difficulties, communication latency, culture, etc,” he said.

Bruiners said during his time working for GE, an offshore engineer was a third of the cost of an engineer based in Cambridge, UK, but the Cambridge engineer was four times more productive and therefore far cheaper per feature delivered.

He does not see much difference between the locations that services are delivered from. “In practice I have never found there to be a big difference between onshore, nearshore or offshore. While at GE and Lloyds I had teams based in Cambridge / London and ones based in India. At Dunelm I have teams in Leicester and London, yet the communication challenges remain the same as those at GE and Lloyds,” he said. 

“If you have separated your delivery capability you need to ensure that you leverage as much as possible on the various communication / collaboration tools available - instant messaging, video conference, interactive whiteboards - to try to bridge the physical gap between teams.”

He recommends that businesses using agile methods should try not to split the team between two locations but to create discreet teams in each geographic location to deliver features. 

UK agile awards

The 2013 UK Agile Award winners were:

Best agile coach or mentor

  • Amy Thompson, Thebigword

Best agile newcomer

  • Daniel Bass,

Best agile team

  • The Scrumbags, Travis Perkins

Best use of agile in private sector

  • Voyager, SITA

Best use of agile in public sector

  • Amberhill, Metropolitan Police

Most valuable agile innovation

  • Commitment, Olav Maassen, Chris Matts & Chris Geary

Most valuable player

  • Rob Smith, IndigoBlue

“For example, when I arrived at GE an agile team could consist of two developers in Cambridge and two developers in Hyderabad, India. This created numerous problems. Communication latency - even effective stand ups become a challenge - and effective pair programming was virtually impossible.” 

He created discrete agile teams in India and Cambridge so that each team would deliver a feature in its entirety: “This improved our effectiveness as team communication issues were localised and the behaviours of the teams improved as each team felt more inclusive of delivering a great product.”

The future of the agile methodology

With the agile methodology increasingly the norm there are inevitable questions regarding standardisation. But Jose Casal at the BCS said the idea of a more standardised methodology is reminiscent of The Lord of the Rings: "One method to rule them all".

“When considering the possible need to formulate a new standardised methodology, we must remember that agile is firmly based on the empirical process and, as such, any successful implementation of agile will need to be carefully attuned and responsive to the contextual characteristics of complex organisations,” said Casal.

“We could argue that the Agile Manifesto already delivers this and that we have great methods and frameworks at our disposal. XP (extreme programming), Scrum, Kanban, AgilePM, etc can offer what we need to work in agile ways, if they are used - rather than abused - in ways that are sympathetic and align to the existing business context," he said. 

"Essentially, I don't think it is possible, nor advisable to create a ‘one size fits all’ agile method that will work in every business environment.”

Read more on IT outsourcing

Data Center
Data Management