Boost productivity with lean software development

Experts say implementing lean software development can lead to productivity gains of between 20% and 40%, and that quality and speed of execution can improve markedly within a matter of months

Lean software development is one of the latest hot topics at strategy consultancies like McKinsey. A report conducted by McKinsey dubbed "Applying Lean To Application Development And Maintenance," says implementing the approach can lead to productivity gains of between 20% and 40%, and that quality and speed of execution can...

improve markedly within a matter of months. Both valuable benefits when the cost of developing and maintaining applications now accounts for about 50% of the average IT budget - a figure that is set to rise in the future.

So, on the surface of it, lean software development sounds like a boon. But what is it, where does it come from and how realistic is it for most organisations not only to go down this route, but also to achieve the promised improvements?

Adrian Wakefield, IT director at sealing products and services company James Walker, which has adopted lean management, describes lean as "an approach for minimising waste by the obsessive reduction of non-value-added processes to speed efficiencies".

The concept was originally developed by Henry Ford in the 1920s as a means of preventing overproduction, while at the same time improving response times to changing customer needs.

Lean manufacturing principles were later incorporated into software development and eventually morphed into the Total Quality Management strategy, with its focus on measurement and documentation. However, the concept of lean product development is a somewhat newer concept.

This branch of lean was developed by Taiichi Ohno, the father of the Toyota Production System, in the 1950s. Rather than focus on manufacturing, his goal was to manage the design process more efficiently, and from the early to mid-1990s his ideas were taken up and applied to development, manifesting in methodologies such as Scrum and Extreme Programming.

But in 2003, the concept was taken a step further by Mary and Tom Poppendieck when they wrote their book "Lean Software Development - An Agile Toolkit For Software Development Managers".

This laid out seven principles behind lean development and provided 22 toolkits or best practice guides to underpin them. These principles are to eliminate waste, amplify learning, decide as late as possible, deliver as fast as possible, empower the team, build integrity in and to see the whole.

But Fred George, vice-president at custom applications supplier ThoughtWorks, says there is still a lack of understanding as to what lean really is. "We are seeing very little awareness of lean, and a lot of people think it is a new concept following on the coat-tails of agile. However, agile is an engineering method by which code is written, and lean is the process by which to do it, and they dovetail into each other. So there is a broad misunderstanding of what they are and how they differ," he says.

Moreover, because lean and traditional waterfall methods of development come from very different philosophical places, they can be badly mismatched, which means that organisations are likely to encounter difficulties if they try to combine the two.

Jon Collins, service director at Freeform Dynamics, explains, "With the waterfall model, the idea is to design, create and implement, so the assumption is that the specification is fixed in time and space and is unchangeable.

"But if you look at areas such as business applications where there is often a lot of change in line with business requirements, there have to be mechanisms to build in change from the outset, and that is what lean aims to do."

The secret behind this lies in eliminating bottlenecks in underlying development processes and tightening up the feedback cycle. So the focus is on optimising how long it takes to turn something around, rather than on finding ways to deliver set products to a set time scale, which is quite different.

"So if someone is waiting to do something, you measure the wait time and ask why they are waiting. When you focus on the cycle time, amazing things can happen," says George.

This means that lean principles can be applied to any process in the IT department and the business as long as it has a clear feedback cycle.

And this was certainly the case at James Walker, which started its lean journey in 2004. "The big question is how to avoid just digitising your business and ending up being locked into processes that are out of date. So we started thinking heavily about how we could apply the lean toolset to our overall project methods," says Wakefield.

This saw the organisation turn to the Manufacturing Institute for best practice advice and subsequently also to Oracle, which set up a Lean Leaders' Circle in November 2006.

These organisations helped Wakefield understand how to apply lean principles. The basic premise is simple, says Wakefield. "You look at existing processes first in a big picture or holistic manner, because if you do not you can end up producing sub-optimised or fragmented solutions that do not address the whole process."

However, he warns that it is necessary to "Analyse weaknesses not from the top down, but using a bottom-up engaged approach by getting the business to understand the issues. Then you look at how you can design processes better and improve those that have been raised as a concern."

This is necessary because systems and processes are organic in nature. As a result they can, over time, end up being more complex than they need to be, which in turn generates waste, so it is important to understand which are important or are not, says Wakefield.

But such activity involves a lot of planning in advance, and also in some instances a reorganisation of the IT department. James Walker moved to a central services model to make it easier to introduce common processes. It also moved from operating profit and loss centres to a matrix system, to promote more synergy and to focus on the customer better, says Wakefield.

Another key factor was to engage stakeholders in the process, which involved obtaining senior management buy-in. When looking at how to initially introduce common processes across its five smallest businesses, Wakefield ran about 10 workshops, each of which lasted between two and four days and included between 10 and 14 people, including IT staff. This equated to about 50 man-days of work.

"I work for the chairman and am the peer of most of my senior operational colleagues, but initially it took a long time to demonstrate to them why we should be doing this and for them to see the benefits. Senior management support is an old message, but you want the business pulling a solution out of IT, not IT pushing a solution on the business or you get a push-pull disconnect," says Wakefield.

It is this need to change behaviour that is by far the most difficult thing to introduce and embed into the organisation, whether on the IT or business side of things. "It is not practical to deliver too much change too quickly. There has to be some pragmatism involved because this is not a one-off process. It has to be part of a continuing philosophy as to how you introduce business systems," Wakefield says.

This means that, while it is useful for an organisation to define a blueprint of where it wants to go over the next five years, this is likely to change and adapt over time, both with circumstances and experience.

Wakefield believes that James Walker is currently about halfway through its lean journey, and the firm is about to embark on the next step of improving what it now has in place.

George, meanwhile, agrees that a radical shift in thinking is required to make lean work, with one of the biggest sticking points often being the attitudes and fears of project and other middle managers. "The question for them is that if they are used to schedules and milestones, how do they manage the team?" he says. This is not least because lean necessitates changes in how personnel operate and also, in some instances, in their roles.

"Traditional methods measure resources and say that while it may take five times as long to get things out, at least staff are all being used. But lean says that is a bad idea, and rather than worry about the local utilisation of resources, you need to train people in multiple skills so there is always something they can do," George says.

The idea of poly-skilled workers means that, for example, although an individual may not be the best tester or designer in the world "they are good enough and can handle the situation no matter what occurs, which is very valuable", says George.

Such a shift involves training and working closely with pockets of resistance, initially by using pilot projects to help staff understand the benefits to themselves and the organisation. Good communications are key to getting this right from start to finish.

David Norton, a research director at Gartner, confirms that adopting lean can require quite a radical organisational change.

"If organisations have rigid prescriptive lifecycles, with fixed dates and restrictive ways of working, trying to look at the value stream moves people out of their comfort zone. So those that could benefit most often find moving to lean the toughest," he says.

To date, he indicates that about 30% of large multinationals are exploring the concept, although active implementation is only taking place in about 5% to 10%, with manufacturing, financial services and petrochemicals companies leading the way.

"I think that within about five years, most CIOs will be aware of lean and will deploy the principles in some fashion. But it is an area that is ripe for consultancy as it is not just a matter of downloading a specification and implementing it.

"Organisations need to understand the principles and interpret and apply them to their own processes, which means that it is up to them to do most of the hard work," he says.

Read more on Software development tools