Adopting agile development methodology is a stretch, but the exercise pays off

Organizations including British Airways have learned that agile software development methodologies really do speed up software projects, at the expense of some pain as you stretch out to achieve agility.

As terms like flexibility and agility abound in IT leadership circles today, some CIOs are turning to the agile development methodology for software development. Agile methodologies help organizations respond quickly to market changes and spend their development time on features that will bring the most value to the business.

Companies in that camp include British Airways PLC, which converted to agile software development in 2007, andCatalina Marketing Corporation, (CMC), which adopted a hybrid agile/waterfall methodology for a business transformation effort after a pure agile effort failed.

"One of the key levers you can pull to increase productivity is speed," said Mike Croucher, head of software engineering at British Airways. "The second lever was the business. Seeing the market constantly change, we realized we needed a process to make quicker changes."

The agile development methodology emphasizes iterative software design, where business stakeholders and IT project leaders work together to hash out processes and design in short increments. Using agile, the development team can focus on prioritizing and implementing only the features that bring the highest value to the business -- therefore eliminating waste and increasing productivity. Agile also allows for rapid change with any iteration.

Deciding whether to adopt agile or use the traditional software development methodology, the waterfall model, involves considerations such as organizational culture, costs and current processes. The waterfall model, which progresses in a linear fashion from project conception through maintenance, emphasizes fixed requirements that are determined up front.

"Agile is better because it quickly delivers functionality to business users, maximizing adoption and input from key stakeholders through solution development," said Clay Richardson, a senior analyst at Forrester Research Inc. "In addition, waterfall takes too long and provides less value because of lack of speed to market."

Making the case for the agile development methodology

At British Airways, selling agile to the business wasn't an easy process at first. "BA is quite the committee-driven, consensus environment," said Croucher. "They don't work well with quick decisions. In an agile world, you need business leaders to be engaged on a regular basis."

Croucher and his team initially did an agile pilot on a project involving changes to the company's website. However, because the website group at the time was not ready to actively join the process and be accountable in a timely manner, IT used the experience as a learning process. "The pilot showed us the type of things to focus on for our next stage in the evolution," Croucher said.

The IT organization then moved on to try agile on another project, with the air miles business group. "Finding the right business unit to work with initially became fundamental to the project's success and adoption," said Croucher.

The air miles group, which manages the airline's frequent flier program, came to the IT group with a lot of ideas. Using the agile methodology to prioritize features, together the teams incrementally rolled out a program that would let customers trade miles. This approach let them see how customers were reacting to the new features and adjust for what was working and what wasn't.

Most recently, BA used the agile development methodology for its revenue management/optimization system, which determines at what price to sell a seat in the market.

"We have very sophisticated pricing software that looks at how many seats sold on a plane and, based on supply and demand, if the price should go up or down," said Croucher. "Rather than putting a lot of changes in, doing agile we can quickly see what difference it [an incremental change] makes and go that route."

Since implementing the agile methodology in conjunction with Lean practices, British Airways has seen both productivity gains and faster ROI. Traditionally, it saw ROI on an average project in nine months to a year; now it sees incremental ROI as soon as three months into projects.

Agile challenges

Yet agile can be extremely challenging as well. Catalina Marketing, which ended up using a hybrid agile/waterfall method, started with agile when it undertook a business transformation effort to change the way it worked with clients in its grocery checkout coupon business.

The effort involved changing workflows and building a Web front end for clients and then a services layer to connect various legacy systems. Jeff Mount, vice president of business process management at St. Petersburg, Fla.-based CMC, said the company "didn't have the proper framework for agile." The subject-matter experts involved in the project ran out of time to devote to it, and some didn't know how to think out of the box to contribute to the change effort, he said during a recent presentation at Gartner Symposium/ITxpo.

Plus, "People wouldn't come to meetings," said CIO Eric Williams. "People were worried about their jobs." (Not a single person lost their job, he added.)

As a result, using agile to start was a "major mistake," said Williams. "The business didn't know what it wanted."

So Catalina defaulted to a hybrid model, using waterfall techniques for gathering requirements and producing screen mockups, then moving to agile for development itself.

Read more on Software development tools