Extreme programming uses revolutionary working practices to improve
the quality of software and the efficiency of its development, but
practicality and traditionalists could delay the popular adoption
of XP by decades.
Listening to Ken Auer talk about collaborative working techniques
or the importance of individual accountability, you would be
forgiven for thinking you had walked in on a human resources
management seminar rather than a conversation about software
development.
But the principal of RoleModel Software is actually at the
forefront of a movement that champions 'extreme programming' - or
XP - a methodology that turns many of the traditional assumptions
about software development on their heads. Gone is the
stereotypical image of the nerdy, lonesome programmer hammering
away on a keyboard into the dead of the night. In its place is a
movement that focuses on limiting the amount of hours people work,
improving team communications and building software specifically to
meet customer demand.
As part of a broader movement emphasising 'agile' development
methodologies, XP is a small but important contribution to the
evolution of software programming. Adopted on a limited scale by a
number of well-known organisations -most famously DaimlerChrysler -
in many ways it reflects the current business landscape. Management
obsession with internal business processes in the early 90s has
generally given way to a more outward-looking customer focus, where
the overriding aim is to deliver what a customer wants to buy
rather than what a vendor wants to sell.
Revolutionary practices
For the software industry, of
course, that concept is something of an eyebrow raiser. For years,
large parts of the development community have got away with
churning out code that's badly designed, riddled with errors,
overbudget and overdue. Release 1.0 of a software application has
become synonymous with advanced beta testing - it may be billed as
a commercial product, but everyone knows it's going to take a
couple more versions before the bugs are fixed and the
functionality is anything near complete. In many sectors,
'maturity' of a product has become a selling point because it
implies stability - an illogical concept for an industry that's
supposed to be at the bleeding edge of innovation.
As well as quality control problems, the software development
sector has also struggled to cope with the practicalities of
meeting customer demand. While service-oriented industries repeat
the mantra that the customer is always right, the reality is that
customers often have no real idea of what they want and, if they
do, their requirements are going to change anyway. Much 21st
century management philosophy is geared towards tackling this
problem and the concept of business agility is a direct response to
the fact that if demand shifts, vendors must shift their ability to
deliver.
For the software development community, however, that isn't always
easy. On a large-scale project, a significant shift in customer
demand can imply major recoding efforts. XP is an attempt to tackle
these issues using a combination of best practices, focusing on the
way software is developed and the working environment.
So its 12 core practices range from pair programming - a
requirement that developers work in twos so that one colleague can
challenge the decisions of another -to a stipulation for reasonable
working hours, designed to reduce the errors that creep in when
programmers are tired. Above all, it aims to tie the customer into
the programming effort, so that its real demands are met, and build
in quality controls.
The methodology has its roots in a troubled development project
undertaken at Chrysler six years ago, commonly referred to as 'C3'.
Kent Beck, acknowledged as the founder of XP, was called in to
salvage the project and, working with colleagues such as Ron
Jeffries, established a series of rules to take the project
forward. These guidelines evolved and were subsequently mapped out
in Beck's book Extreme Programming Explained: Embrace Change.
Team work
At its heart, XP is underwritten by four core
values - communication, simplicity, feedback and courage. Although
the last requires some explanation (it can best be understood as
having the nerve to rise to the challenge and meet changing
requirements), the first three are largely self-explanatory and
would make the cut in any management handbook.
In reality, that reflects the nature of XP. Its component parts are
often simple and critics point out that many aren't particularly
new. What XP is fundamentally about, however, is bringing all of
these elements into play as an umbrella discipline for effective
software development.
Of the 12 XP practices (see box below), the one that tends to evoke
most comment is pair programming, often cited as one of the biggest
barriers to adoption of XP from a management and programmer's
perspective. In a traditional programming environment, individual
developers effectively own their part of the software codeset.
There's very little peer review of the code that's been developed
and the levels of quality and creativity depend to a large extent
on the individual's skill and attentiveness. Pair programming, by
contrast, requires two developers to sit side-by-side at the same
machine. As one codes, the other questions what's being done and
suggests alternatives.
The advantage of this approach is that it brings discipline and
creativity to the programming process. Individuals are less
inclined to take shortcuts with someone breathing down their neck.
At the same time, by challenging and discussing programming
decisions, the combined skills of two developers will often produce
a better result.
From an organisational perspective, pairing ensures that program
knowledge isn't just held in the head of one developer. Since most
XP shops also encourage individuals to switch pairing partners on a
regular basis, knowledge and experience is shared around the entire
team.
Better by design
But for individual programmers, this
is a huge culture shock. In effect, pair programming is a
relatively advanced form of collaborative team working. Being
constantly questioned about the decisions you're making is a hard
discipline, particularly for more experienced developers. It
requires an open, blame-free culture, where there's no expectation
that people will get things right first time round. In essence, the
kind of point-scoring culture that permeates many organisations has
to be eradicated for pair programming to work.
RoleModel Software's Auer, who was briefly involved in the C3
project, acknowledges he encountered problems when he first
attempted pair programming, working with a fellow developer to
rewrite a client's application. "It was very frustrating at the
time, but I had to teach the other guy my approach and this was the
best way to do it. I felt I was being dragged down and wasn't so
productive," he recalls.
At the end of the project, however, he looked back and calculated
that his productivity and the quality of the product had improved.
"After three weeks, I realised there was no way I could have
produced something that good so quickly alone. You're constantly
being challenged. Sometimes, the simplest questions are the most
effective, like: Why are you doing that? You're in programming
mode, not thinking mode - the other guy's thinking," he claims,
adding that people taking this approach should "make sure they know
it's okay to be uncomfortable at first".
But if pair programming presents a challenge to individual
programmers, it's even more of an issue for managers. The idea of
having two programmers working on one keyboard implies lower
productivity and the most common complaint is that programming
resources are being wasted. Advocates of pair programming counter
that the productivity drop-off isn't anything like as severe as you
might expect and that improvements in quality more than make up for
it.
Laurie Williams at the University of Utah, for example, carried out
a controlled experiment in 1999 with undergraduates on a software
engineering course (see box below). In the first experiment, a team
of two collaborators took more than half as long again as an
individual developer to carry out an assignment. But when the pair
had sufficient time to gel, the extra time taken fell to a mere 15
per cent. The code they produced, meanwhile, had 15 per cent fewer
defects, which ultimately represented a huge saving. In addition,
there was a wide range of softer benefits, from improved employee
satisfaction to better design.
Too much too soon?
In addition to pair programming,
several other aspects of XP raise eyebrows, including the concept
of continuous testing. As Ron Jeffries writes in an introductory
guide to XP: "Extreme programming is obsessed with feedback and in
software development, good feedback requires good testing."
Unlike the traditional development environment, XP tests are
designed before programming is carried out and because the
programming effort is broken into short chunks, the testing process
permeates the whole development cycle. It also includes regular
client acceptance tests. Compared with pair programming, this
practice generally goes down well with managers who appreciate the
quality implications - but once again, individual programmers may
find it a hard discipline to swallow.
XP advocates also insist that the customer is represented
throughout the development process, setting out requirements
upfront, getting involved in ongoing planning and ensuring that the
delivered product meets their needs. While this all-inclusive
approach would delight advocates of customer relationship
management, in practice it can prove challenging.
One senior programmer at a major software developer in Silicon
Valley, for example, recalls being impressed by the XP approach
after he attended a training course, but ran into problems because
his customer - an internal department - couldn't clarify its
needs.
These kinds of concerns are likely to hold back widespread adoption
of the methodology for the foreseeable future. While a number of
large organisations have experimented with XP, they have mostly
done so in small pilots rather than as a wholesale corporate
strategy. That will only change slowly. "I think it will happen,
but it will take a long time. The relational database took 20 years
to be adopted by the mainstream and Unix took 20 years. I think XP
is going to take ten to 20 years," Auer predicts.
www.xprogramming.comThe 12 core practices of extreme programming
XP is
commonly broken down into a mixture of 12 techniques and
rules:
- Whole team - all project members sit together as one team,
including programmers, testers, a coach and representatives of the
customer
- The planning game - XP's two-stage planning process reflects
the fact that requirements change as projects develop. In 'release
planning', the customer lays out its requirements, costs are
estimated and a plan is set out, which is regularly revisited. This
is further refined in a more detailed fortnightly 'iteration
planning' process, again involving the customer
- Testing - tests are designed before programming begins and are
run each time a developer sends code to the central repository.
Customers also define acceptance tests for each major feature
- Small releases - the emphasis is on supplying code frequently.
At the end of every iteration, fully tested software is delivered
to the customer
- Coding standard - all code is written to a common standard to
give it the same look
- Simple design - XP guru Ron Jeffries says: "Design in XP is not
a one-time thing or an upfront thing, it is an all the time thing."
The focus is on simplicity
- Pair programming - all coding is carried out with two
programmers on one screen, constantly challenging and reviewing
each other's work
- Design improvement - a process of continuous design improvement
aimed at removing duplication and increasing the cohesion of the
code
- Continuous integration - XP teams focus on frequent integration
(several times a day) as part of the development process
- Collective code ownership - traditionally, code is 'owned' by
the developer, but with XP, where pair programming is the norm and
teams are switched, any pair of programmers can improve any
code
- Metaphor - a simple vision of how the program works
- Sustainable pace - XP emphasises the fact that programmers are
in it for the long term and reasonable working hours prevent
burn-out
Sources: Ron Jeffries/RoleModel Software
Pair programming
In a 1999 experiment, Laurie
Williams at the University of Utah demonstrated that programmers
working in pairs took just 15 per cent longer to build a program
than developers working on their own. At the same time, their code
had 15 per cent fewer defects.
In The Costs and Benefits of Pair Programming, Williams provides an
example to show what the experiment might mean financially. Taking
a program consisting of 50,000 lines of code and working at a
typical rate of 50 lines per hour, individual programmers will take
1,000 hours to complete it. Pair programmers, however, will take 15
per cent longer, or an extra 150 hours.
Quoting other research sources, Williams suggests that programmers
typically make 100 defects per 1,000 lines, of which 70 per cent
are removed during a good development process. That leaves 1,500
defects in the code developed by individuals. Pairs, however, would
have 15 per cent (or 225) fewer defects.
This reduction in defects translates into savings when the code
goes to the quality assurance department. Williams argues that a
systems test takes between four and 16 hours per defect. If you
assume an average of ten hours, the 225 additional defects
generated by individual programmers will take 2,250 hours to fix -
15 times more than the 150 extra hours spent by pair programmers.