Open source projects have resulted in some of today's most innovative products. Many of the practices and staffing models used in open source development could be adopted by corporate IT.
In corporate software development, less than a third of projects finish on time, on budget and within scope. Furthermore, the overall percentage of functionality delivered is barely more than half.
So what is being done? Top-down process improvement initiatives have been implemented in many IT organisations, but they have not won developers' hearts and minds.
Agile development processes have gained a considerable following - especially among developers - but sceptics still doubt their scalability.
The open source development model, in contrast, has been proven to work on large-scale projects and has enticed hundreds of thousands of volunteers to work on projects for free.
Learn from open source
Open source software has moved from the margins to the mainstream - 46% of firms surveyed by Forrester currently use open source, and another 14% plan to use it within the next 12 months.
Open source projects - including Apache, Linux, MySQL and Tomcat - have achieved success and won worldwide renown for their development models. Many of their processes are suitable for corporate environments and can improve developers' productivity and quality. For example, open source projects have demonstrated how agile techniques, such as formal user involvement and continuous integration, work for large-scale developments.
There are at least three areas in which open source can benefit corporate IT: getting end-users involved; team communication and staffing models.
Involve users in development
Linux creator Linus Torvalds observed that a large user base can be even more valuable than a large developer base. And Eric Raymond, who has written books on the Linux development model, said, "Given enough eyeballs, all bugs are shallow."
Not only do open source users specify requirements, but they also participate in design reviews, testing and implementation of new systems. IT organisations should look for ways to involve end-users in as many stages of development as possible.
This can be achieved in several ways. IT managers can make source code and other artefacts, such as requirements documents, issues lists and project schedules, available to internal customers - especially those with technical savvy. This will allow customers to identify issues earlier in the project lifecycle and feel a greater sense of ownership.
You can also recruit beta testers. Select from your most involved end-users those who are willing and able to use beta versions of your application. This can work for development organisations of all levels: IBM and Microsoft involve beta testers through the Microsoft Developer Network and IBM's Alphaworks. Corporate IT departments can follow suit on a smaller scale.
In terms of team communication, open source projects rely heavily on tools as modes of communication. For instance, open source projects are more likely to build and use automated communications archives. Mailing list archives are the most commonly used method, although wikis - web-based collaborative writing tools - are growing in popularity.
Corporate IT departments must strive to capture and store various team communication - from e-mail to instant messaging to virtual conferences - to improve team co-ordination. Scalable collaborative development environments such as Collabnet's Sourcecast and VA Software's Sourceforge can also help.
This approach can make documentation less of a chore by automatically documenting parts of the development process, therefore freeing team members for more valuable tasks.
Although team communications systems will not do away with documentation altogether - development teams will still have to create artefacts such as requirements documents, design models and test plans - these tools will cut down on the amount of documentation the team must explicitly create. It can also create a valuable information asset.
Archiving as much of the development process as possible protects a team's knowledge base, which is susceptible to depletion in the event of employee departures. Even sick days are less problematic at firms that use this strategy. Automatically logging development activities on the system is an excellent way to capture and transfer knowledge among team members.
Firms should also increase the visibility of the project information to people who are not team members to improve co-ordination and increase productivity. Increased transparency can also improve the accountability and performance of individual employees by providing a documented workflow and a basis for feedback.
Leading development organisations know that disclosure improves business relationships and will communicate select information about project progress to customers.
Flexibility and functionality
Open source projects typically consist of a few core developers, a larger number of non-core developers (contributors) and a vast pool of users. More than 50% of open source developers participate in two or more projects and another 10% participate in 10 or more. Developers benefit from the broader exposure that results from involvement in multiple projects and they are generally more engaged and satisfied than their peers.
Ten years ago, a Fortune 100 company's IT department launched an innovative staffing process: mentors worked with developers to determine where developers' talents and interests lay, and then negotiated to put them on the most appropriate projects.
The upshot was that the retention rate grew at a time when client/server developers were in great demand and were being wooed by many other firms.
Developers preferred this bottom-up staffing process because it gave them some control over their fate. Managers ended up leading teams of people who were more likely to be excited about their work.
It is important to communicate to programmers that they are the core developers on their primary teams, but that they should function as contributors on other teams. Cross-staffing exposes developers to new methods, and when they are on multiple teams, it is more likely that they will be asked to do the work they can do best, instead of what work is available.
In open source communities, the most dedicated and accomplished members of a project are entrusted with the job of committing new or enhanced code to the code repository. Corporate IT departments should transform this from an administrative role to one of coveted leadership by endowing it with additional responsibilities and clout, similar to those of the open source committer.
The code committer can also be responsible for enforcing peer code review and for maintaining the modularity of the system, even when the pressure is on.
Firms that allow developers to choose which other projects to work on must revise developer goals to reflect this. Managers must reward developers based on how much code they contribute to other projects and how much of that code is accepted.
The success of this strategy depends on developers understanding that they are expected, not just encouraged, to contribute to more than one project. All levels of management must be on board for this strategy to succeed.
Many of the principles of open source development match those of agile development processes, such as Extreme Programming and Scrum. And although adoption of Agile development processes is increasing, there are still many who doubt they are practical for large projects. In fact, open source projects have demonstrated that Agile techniques can work on a large-scale distributed basis.
l Liz Barnett is vice-president and research leader at Forrester. This article was written in conjunction with Carey E Schwaber