Agile development an 'IT fad' that risks iterative failure

Agile promises continuous improvement through an iterative process, but without proper checks, project failure is commonplace

Over half of CIOs have discredited agile development, according to a survey commissioned by London-based technology services firm, 6Point6.

Three-quarters of the 300 UK and US-based CIOs surveyed said they were no longer prepared to defend it. Half said they now think of agile as “an IT fad”.

Chris Porter, CTO of 6Point6 and author of An agile agenda: how CIOs can navigate the post-agile era whitepaper on which the survey results are based, said: “CIOs tell us they expect to undertake six agile projects next year.” According to the survey, CIOs predict two of these fail completely.

One of the key tenets of an agile methodology is that it is an iterative process, where errors can be quickly resolved through continuous improvement.

However, according to 6Point6, while fail fast is an intoxicating prospect, in practice, it can blur the distinction between continuous improvement and genuine failure.

Specifically, Porter believes in agile projects it is hard to know when an agile project is on the road to ruin. He warned the iterative process may lead to iteratively improving, one failure at a time, towards the wrong outcome.

“At no point will this become obvious in the same way it would if you were constrained and measured by a combination of time, budget and scope,” he said.

Read more about agile

“Unless you are paying close attention all the time, your agile project could easily be hurtling towards disaster without you even knowing it,” said Porter. “This may explain why one in three projects fails. It’s a very poor hit rate.”

The study found 44% of failed agile projects did so because of a lack of documentation. According to Porter, this is a symptom of short termism in agile, which does not acknowledge that software development teams do not last forever and handover is inevitable. “That will require comprehensive documentation,” he said.

Projects fail due to lack of planning

Adequate planning is another factor contributing to agile project failures, according to 6Point6.

Its research showed 34% of agile project failures occur as a result of a lack of up-front planning. Porter recommended organisations put in place agile planning, both to reassure CIOs of progress towards strategic goals and to coordinate distributed agile at scale.

Over two-thirds of the CIOs who took part in the survey (68%) believed agile projects would benefit from more IT architects. The architects’ role is to define strategy, champion technical requirements – such as performance and security – and ensure development teams stick with agreed architectural guidelines.

Porter said the role of the architect is sorely missed in the agile space and needs to be reintroduced. The 6Point6 research research found just under a third of projects fail because teams are geographically disbursed. 

Read more on IT project management

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

True. Plus higher cost
This is less about agile and more about teams not having a well defined definition of "done" which includes the important doco, maintenance issues which tend to slide when deliveries are late. Agile does not preclude aspects needed for a quality release. It is up to the org to define the "minimum effort needed for the task".
Agile in my agency has been taken over by non-technical people who ask for things that create impossible contradictions such as displaying a wide table that will not fit on screen without using a smaller font. Fun times.  
Does "geographically disbursed" mean that the check got lost in the mail?

More seriously, this seems like yet another indicator of an obviousl weakness in the Agile dogma: avoidance of the strategic planning inherent in waterfall. A hybrid approach, using waterfall for setting intermediate- and long-term goals and milestones while using Agile methodology scoped within those milestones, can give the  best of both worlds: rapid reaction to failure (and success) while retaining the ability to measure overall progress.

But the real crunch is that it still requires competent management to respond decisively and effectively to the feedback.

As simple as Agile is, many people don't seem to understand what it is as evidenced in this article which claims '44% of failed agile projects did so because of a lack of documentation.' That 44% failure is not due to "Agile", but due to a lack of documentation, as it explicitly says. To state what should be obvious, if the same project was implemented as waterfall, but lacked the documentation, the waterfall projects would also likely have a 44% failure rate. If you believe Agile is a fad, you really do not understand Agile.
Why do you people always attack the "waterfall" paradigm as the only alternative to Agile development.  It never has been the case.

There are in fact approximately 13 different life-cycle paradigms for software engineering for which the "waterfall" paradigm is only one.

The "waterfall" paradigm is still an excellent choice for large-scale, complex development; something that Agile has never been able to accommodate.

The fact that so many people still use this erroneous analogy demonstrates how woefully ignorant current technical personnel are in understanding the nature of true software engineering, which Agile was designed to circumvent with a ridiculous set of claims that were never based in real world analysis.
I have to wonder what percentage of "agile" projects fail simply because their teams cherry-pick those aspects of Agile process that they think will be "enough"?  Couple with the common management attitude that "Agile is a silver-bullet", there is simply no justification for proper iterative refinement: once a "milestone" is passed, code related to such-and-such feature point is "locked", for fear of "breakage".  Alas, clueless management will sink projects no matter what dev methodology is chosen. Sad.
I have published a number of articles over the years that Agile was never a good replacement for sound software engineering standards.  However, no one wanted to listen.  Now the "bugs" are starting top crawl out of the wood-works.

Numerous other articles have appeared over the past 7 years corroborating my own contentions.

There never was any reason for the development of Agile except for those people that wanted to promote a new paradigm that could promise a "silver bullet" for IT organizational woes.

It has failed miserably considering that it was an offshoot of XP Programming that was a disaster in of itself.  However, by the time this disaster was realized it was too late and the IT Community actually believed it was an alternative to sound software engineering practices.

Agile and everything about it has been attempt to literally discredit proven sound software engineering principals that had already been proven to not only work but produce high quality software at levels that Agile has never been able to come close to.

This is what happens to a profession when you get an influx of a new generation of software developers who have no long term experience with what works and what doesn't.  Mix that with a corresponding influx of foreign technical management who think they know everything and you have a recipe for disaster that is now just becoming known...
That's a lot of declarations to make, without offering a shred of evidence. There's a great deal of objective research that unambiguously shows Agile approaches as better on all metrics - risk, productivity, cost, quality. What evidence can you offer to support your opinion?
Can you provide some links to this "objective research"? As evidence to support your opinion, thank you.
Fully agree, thanks for posting!
As I read's not Agile failing, it's people not doing their jobs appropriately. The beauty of methodologies before Agile was that you never knew when you were going to be done, but you pretended too. That false sense of security and accuracy enabled IT orgs to be calm for 10 months and hectic for 2. With Agile, you don't have false progress, you have actual progress. That's far more stressful. It was easier when we all lied and believed the lies. Those were good times.
I cannot believe a CTO is writing this rubbish. Agile is over 20 years old and to call it a 'fad' shows a clear lack of understanding of what Agile is. Agile describes a style of working that is flexible, collaborative, focused on business need and iterative. There are then a number of Agile frameworks and approaches that can be used Scrum, Kanban, Lean, XP, DSDM and SAFe to name a few. I would take a closer look a DSDM, using DSDM by itself or to supplement existing in-house Agile approaches to the full project focus that seems to be the issue. Agile does not mean no documentation, it is about the right level of documentation at the right time. It is how people apply Agile that needs to change.      
Agile is a joke.  How many millions have been spent by corporations following this fad?  Where are the numbers that show comparatively that Agile is better?  You can't just 'say' it's better.  I asked the PM on a big project i was playing Agile on this same question. His response was, "it's expensive" (and nothin else).

This article collects a raft of incorrect conclusions by people who do not understand Agile. A second article is needed that deconstructs all of the false conclusions. It is also important to distinguish between "Agile - the ideas" and "Agile - the common practices", as these two are quite different.