Development methodologies function much like recipe books, laying down guidelines for others to follow or adapt. But, as every cook knows, you can follow all the rules and still make a dog's dinner
Celebrity chefs may be able to knock up a gourmet meal on a whim and a bottle of claret, but for the rest of us, successful cookery means sticking more or less faithfully to the recipe. The same applies in the world of IT, where methodologies have emerged that aim to ensure that software development projects deliver palatable and predictable results, served up on time and within budget.
Development techniques have changed radically over the past 15 years, moving from lengthy 3GL-based projects based around a waterfall lifecycle, to object-based projects using an iterative approach and lasting weeks or months rather than years. Tried and tested development methodologies have gone through a series of metamorphoses to keep pace with these changes, and new ones have appeared to support the changing requirements of 21st century software building.
But software specialists agree that in the end the important issue is not so much which methodology you use, as having some kind of framework for structuring your project. The early evangelism over a methodology-based approach has given way to a more realistic view, which acknowledges that sometimes you need to adapt your recipe to the ingredients you have to hand and to the preferences of the people who are going to consume the results.
In theory at least, a methodology contributes to successful project completion in three ways. It provides a framework for evaluating the problem the project is intended to address. It provides a roadmap for the project, ensuring that no important stages are left out and that everything is proceeding to schedule. Thirdly, it provides a way of representing aspects of the developing system to system engineers, software developers, user representatives, management and other stakeholders in the project, via standard diagrams that generally form part of the methodology. Using industry standard notations also makes it easier to get newly recruited IT staff up and running quickly, or enable staff to move between different projects without a steep learning curve.
There are many different methodologies out there, but the same five names tend to crop up regularly: Prince, Ssadm, DSDM, JAD and UML. Comparing them, however, is rather like comparing apples and pears, because they are designed with different aims in mind.
Prince (projects in controlled environments), for example, is aimed specifically at project organisation, management and control, and can be used equally well in IT and non-IT projects. First developed in 1989 by the UK's Central Computer and Telecommunications Agency, now part of the Office of Government Commerce, it has become the UK's de facto standard for project management in the public sector.
Ssadm (structured systems analysis and design method) is to requirements specification, analysis and design what Prince is to project management.
Influenced by earlier methodologies such as Yourdon, Information Engineering, Merise and DeMarco, it was developed by consultancy LBMS and mandated by the UK government for use in all public sector projects in 1983.
Originally aimed at conventional waterfall development lifecycles, it has been extended and adapted since in an attempt to cater for new requirements such as rapid application development (Rad). Ssadm is widely known and understood, since anyone who wants to work on public sector projects has to learn to live with it, if not love it. However it is generally considered to be over-complex, partly as a result of having been modified extensively over the years.
DSDM (dynamic systems design method) and JAD (joint application development) methodology both have a more specific focus than Ssadm; namely, projects that have to be completed to tight deadlines. JAD was originally developed at IBM in the 1970s as a way of getting user involvement in application development, and was revived in the 1990s as part of the new trend toward Rad and time-boxing. DSDM was developed by a consortium of supplier and customer organisations in response to a perceived need for a structured method to support Rad projects, and first appeared in 1995.
But over the past few years, the growth in object-oriented programming, accelerated by demand for Web-based applications, has pushed UML (unified modelling language) to the fore. The newest of the major development methods, it has swiftly established itself as the de facto standard for object-oriented analysis and design.
As its name suggests, UML is not, strictly speakin,g a methodology at all, but a set of specification and design notations for object-oriented systems, combining features from methods devised by three methodology gurus: Grady Booch, James Rumbaugh and Ivar Jakobsen.
First launched in 1996, it was endorsed by the Object Management Group in 1997, and is backed by major IT companies including Hewlett-Packard, Oracle and even Microsoft.
While meeting one of the key criteria for a methodology - that is, providing a way of representing and communicating the project design to stakeholders - UML has no facilities for managing the project lifecycle; to achieve this it has to be used in conjunction with a project methodology such as Rational's Objectory.
UML's rapid rise can be partly attributed to its early success in gaining official industry sanction, and partly to its comprehensiveness. Most traditional methods provide for some equivalent of dataflow diagrams and entity relationship models. UML goes beyond this with no fewer than nine types of diagram intended to represent many different views of an object-oriented system. One of its particular strengths is its ability to represent system behaviour from an end-user's point of view, in the form of use case diagrams.
This comprehensiveness can, however, be seen as both a strength and a weakness. Some software developers love it. According to Chris Jones, IT manager at e-commerce software company Intentia, the adoption of UML as a standard modelling language has been the best thing to hit the software development world for some time. "The big benefit for developers is the ability to speak a common language," he says. "Before UML there were many different diagrams and schemes for representing models, and precious time could be wasted on learning new modelling languages in different projects."
But the downside of comprehensiveness can sometimes be incomprehensibility, and software specialists are starting to question how helpful UML's maze of class, object, use case, sequence, collaboration, statechart, activity, component and deployment diagrams really is to the average developer and user.
"UML has ended up the way that Ssadm did - too complex, with a whole industry built up around it which relies on its complexity," says Darrel Ince, professor of computing at the Open University. "Though it's true that you can get away with using about 20% of the method, the textbooks go into so much detail that people tend not to notice that."
Steve Saxton, of e-commerce software house Lost Wax, agrees: "Methodologies provide a way for similarly trained individuals and groups to convey information in a way that others should be able to understand," he says. "But what they give with one hand they take away with the other, and generally they place a higher workload burden upon the designer."
In theory, the extra time taken to implement and validate your design using a methodology should pay off further down the line in terms of better understanding of the project by all concerned, and less reworking. In practice, some specialists argue, it does not always work out like that.
"Not everyone understands the methodology to the same level and people seldom stick to the principles they lay down," says Saxton. "It takes considerable buy-in from higher management to make these things work and, when the customer is breathing down your neck, management patience usually goes out the window. In practice the overhead of maintaining the design can quickly become a nightmare."
At their best, methodologies act as a lingua franca to improve intra-organisational communication. At worst, they can become a communication barrier that enables the IT department to blind end-users with science. A system model presented in the form of an object diagram, or even a complex use-case diagram, is not necessarily going to be easy for an untrained user to understand.
Brunel University's Department of Information Systems and Computing carries out consultancy projects with industry partners. According to Robert Macredie, professor of interactive systems at Brunel, "We're working with a partner that has a history of using a very structured approach to development. The IT department feels quite safe because they get to blame the users if anything goes wrong, but the users feel as if IT imposes things on them."
Rather than bend their working practices around the demands of one of the official methodologies, many organisations have adopted a DIY approach. Consultancy RCMS, for example, has developed a methodology called Blueprint to help it build e-business projects. ISIS Technology has evolved a methodology called iRad, which takes features of major methodologies such as UML but applies them in a simplified style that is adapted to the company's way of working.
The alternative is to adopt a pragmatic approach of using methodologies where appropriate but not following them rigidly - and knowing when to abandon them. "Yourdon, Booch, UML, top-down waterfall, flowcharts, they're all just ways of getting information across and trying to make it more convenient for people to understand what you're trying to convey," Macredie says. "But in reality people respond better to Post-It notes than they do to 200-300 pages of formal design which are hard even for technical people to interpret. Sometimes there's just no substitute for plain English."