How to adopt agile software development

Learn about Agile software development methodologies and how CIOs can lead their teams to this powerful new practice.

  Getting started: How to transition to agile
   Benefits of agile development
   Problems with agile development
   Agile project management methodologies

  Agile tools

If you haven't already moved your team to the agile methodology, now is the time to start thinking about it. We're still in the thick of the recession, and lean economic times call for lean, agile software development. If you are exploring the adoption of agile software development practices and you're prepared to rise to the occasion, this recession and the resulting belt-tightening give you the opportunity to rally your company around a vision that will not just cut costs, but improve morale and help you grow your business in the next economic spring. Agile practices enable teams to build less, but return the same value by focusing on early delivery of the features that have the highest business value and not wasting money on the features that don't.

Interest in agile is high, but our 2008 Agile Trends survey found that many organizations still follow traditional development practices. When asked which development processes their organizations employed, the number of respondents for agile and waterfall was almost the same, with 46% reporting they used Agile and 44% reporting they used waterfall. But more and more teams are switching to agile development -- it's no longer just for innovators and early adopters, nor is it just for small projects. In fact, according to Damon Poole, founder and chief technology officer (CTO) of AccuRev Inc. in Lexington, Mass., asking if agile can scale is the wrong question.

"The question to ask is, 'How does software development scale in general?' Traditional development doesn't scale very well anyway; you hear all the time of large projects that were cancelled or delayed. Scaling is independent of methodology," he said. Suggestions for scaling agile include upfront, lightweight modeling and extra coordination. At the project level, "Scrum masters have to interact and make sure sub-teams are going in the same direction, and product owners of each team have to negotiate priorities to make sure teams are working in the right order," said IBM's Scott Ambler, a practice leader in agile development.

So how do teams transition to agile development methodologies? While the whys of transitioning to an agile methodology are often the same (slipped release dates, scope creep, time to market), the hows of moving to agile can vary among development organizations. Providing agile training is usually the first step. Another consideration is where in a software development cycle to transition to agile practices. Vignette Corp., for example, decided to roll out agile as each team started a new project, phasing agile in over the course of a year rather than doing a full switchover all at once.

In addition, teams employ a variety of agile techniques. Our 2008 survey showed that Scrum was the most popular (41%), followed by Extreme Programming (XP) at 15%. Others used a hybrid of Scrum/XP (14%), and 13% used a custom or other type of hybrid agile methodology. The Crystal and Dynamic Systems Development Method (DSMD) both had a following of 3%. Organizations also need to determine the size and makeup of their agile teams. Another goal of transitioning to agile development is to empower teams with a sense of ownership.

David Christiansen
David Christiansen

Before you switch your team to agile, it's a good idea to start by reading the manifesto and principles behind the agile movement, expert David Christiansen said. They're easy to miss, and you'll struggle to make agile successful if you don't first embrace and apply the principles. Christiansen also recommends reading Lean Software Development by Mary Poppendieck. This book will help you develop an in-depth understanding of the agile approach.

Benefits of agile development

What are the pros and cons of agile development? No software development methodology is without its challenges, but teams who move to agile development usually see a number of process improvements. Topping the list of improvements, according to our survey respondents, are faster time to market and increased productivity. (Blueprint Systems, for example, completed six releases in its first year of using agile methodologies, compared to a previous average of one release every year and a half.) Other benefits of switching to agile include fewer software defects and reduced software development costs.

According to Steve Whatmore, a Java architect at LYNXDev, the benefits of agile have included "not only time to market with our solution -- which is substantially faster than what our clients expected with the size of the team that we assigned to the project -- but also the robustness of the system as a whole." Whatmore added, "Due to the fact that the base architecture has essentially been tested and retested from day one of our development cycle, we have been able to flush out a lot of the defects that would otherwise be found very late in the game with a waterfall approach."

In addition to fewer errors, Danny Allen, director of security research at IBM Rational, said Agile processes can also help improve security. "Applications developed using an Agile process end up being more secure," he said. "You're not only doing functional testing early and often, but you're also doing the automated security testing. You're catching mistakes early."

Agile methods can also benefit software outsourcing providers. Outsourced development projects face many of the same obstacles as in-house projects -- time and cost overruns caused by scope creep, products that don't meet expectations. Who should take the hit for these problems -- the customer or the contractor? Some outsourcers are addressing this issue through agile methodology, resulting in shared risk, more predictability, and products that better meet customer needs through ongoing collaboration.

Watch the below video to learn why agile design is one of Jon Kern's three keys to software development.

Problems with agile development

Agile isn't perfect. While organizations are reaping benefits with agile, practitioners acknowledge there are some challenges to this style of development. Respondents to's 2008 survey cited communication as the top Agile challenge, followed by documentation. Resistance to change and tool integration were also cited as challenges. And agile can be particularly challenging when you're dealing with distributed teams.

In addition, there are a handful of major agile testing perils you'll need to watch out for. Make sure software testers realize they'll need to adjust their mindsets. As an agile tester, you are expected to test without having formal requirement documents, to test in real time, to test changing code, to test on changing requirements, to automate most of your tests and to be a part of a close-knit team. Issues to watch for include waiting for a specific build (in agile, you need to test constantly); trying to test everything manually (using automation is key); and losing sight of the big picture.

Agile project management methodologies

How will you tackle agile software development project management? Can traditional project management and agile development really coexist? Well, it depends, according to Michele Sliger, consultant and co-author of The Software Project Manager's Bridge to Agility.

Michele Sliger
Michele Sliger

"In Scrum there is a Scrum Master -- is that a project manager? In XP there is an XP coach -- is that a PM? Most large organizations I've consulted with have decided on their own version of agile from XP, Scrum [and] Lean and still have PMs on the HR books. The job titles are still 'project manager' but the role might be something different," she said. A traditional project manager's role is to run the project and take the blame if things go wrong. As a result, traditional PMs can be very "command-and-control" in their leadership style. In an agile project manager, however, you're looking for a true coach, guide and leader, not someone to micromanage the development process and dictate what each person on the team should do. The bulk of the job in agile project management is mediating between team members to help them achieve consensus and negotiating with the organization to help them understand how their actions affect the team.

Agile teams frequently use different project management tools. "If you're not colocated, you must have some kind of tooling and an infrastructure to support that tooling. You're working at a very fast clip, so you will need tools like Skype, IM, some shareware like for brainstorming, or XPlanner, which helps with agile project management. All agile PM tools provide these basics: a place where the product backlog is maintained, a place that holds iterations, features being worked on, test results and often the tests themselves," Sliger said. But tools alone won't make you agile. "Don't think you can put down the Gantt chart and pick up a burndown chart, and poof, you'll be agile. Agile is a philosophy, and [it's] value-driven instead of plan-driven. You have to understand that paradigm."

Some companies have successfully combined traditional PM with agile methodologies. A recent Forrester Research report titled "The PMBOK and Agile: Friends or Foes" recommends combining aspects of the Project Management Institute's PMBOK Guide with the Agile Manifesto to get the best of both worlds. PMBOK does bring some strengths that agile lacks, including clear guidance on project initiation and closure; communications management and project integration management; project cost management; and risk management. Conversely, agile's strengths over PMBOK include cross-functional, empowered teams; flexibility and adjustment throughout a project; encouragement of strong working relationships with customers; and just enough rigor and documentation.

Choosing tools for agile development

Respondents to our 2008 agile survey told us they use a variety of agile development tools. Topping the list of essentials were requirements management tools, bug tracking tools, and project management and unit testing tools. Also cited were tools for functional testing, build, collaboration, configuration management and documentation. In addition, with the shorter cycles of agile development, automated testing tools are important, particularly automated regression and unit testing tools.

LYNXDev's Whatmore also considers a daily development meeting a critical tool "to ensure that everyone on the team is up to date with the progress/status of the development team." He said his team holds 15-minute daily development meetings, covering the progress of each person's tasks, problems that a developer may be facing, and any changes or modifications to the application architecture or other functions that could affect the rest of team. "We found that these meetings were invaluable in maintaining communication," he said.

The agile development movement didn't necessarily endorse tools originally. You can be agile and employ no more technology than a command-line interface, a unit tester and some index cards on which to write requirements. But specific tools have evolved in recent years to better support agile efforts. Among these new agile tools are several directly pegged at supporting a new type of project management. Agile project management tool vendors include Rally Software, IBM's Rational software group, TargetProcess, VersionOne and Thoughtworks. Continuous integration build tools and automated testing tools have also become closely associated with agile processes.

Static analysis tools can also be helpful as part of the agile development process. When Mentor Graphics Inc. decided to switch to an agile methodology, the company needed a source code analysis tool that fit its development methodology and its complex code base. Klocwork Insight, a static analysis tool, provided the right services and conformed to agile principles. Mentor Graphics uses the database that Insight creates at each build "to refactor legacy code and to analyze acquired technology details of code, relationships, third-party components and forward architecture," said Kevin Pendleton, the company's director of quality and support systems. Engineers are able to view their own code and correct mistakes on the spot, while the information is still fresh in their minds.

TOMORROW: Testing and other agile issues

Read more on Software development tools