M&S's new test regime drives up software quality

Retailer works with offshore outsourcers to improve reliability of code

Retailer works with offshore outsourcers to improve reliability of code

Marks & Spencer has embarked on an ambitious programme to ensure its projects are delivered on time, at lower cost, and with fewer coding errors.

The retailer is working with its offshore suppliers to improve its software quality management and testing processes.

It is one of the first UK businesses to introduce Testing Maturity Model (TMM), developed to help IT departments identify and plug gaps in their software quality and testing processes.

Early results show that Marks & Spencer has been able to cut debugging time on typical large projects by 12 days, save a further five days' development time by managing bugs proactively rather than reactively, and eliminate 65 days of effort by removing duplicate testing processes.

Marks &Spencer, which conducts more than 200 development projects a year, expects dramatic improvements in software delivery times, to reduce development costs and to free resources for more strategic IT projects.

"We are bringing in more projects to time and to budget. That's where the business really gets the benefit," said Andrew Goslin, manager for testing methods and standards. "We have far more predictable projects. The quality of the deliverables is higher. There are fewer errors and projects are less likely to overrun."

Analyst group Forrester said that Marks & Spencer had gone further than most companies in systematically improving the quality of its software testing.

"This work is more rigorous than we see most companies doing. But they will benefit from that investment," said analyst Carey Schwaber.

"The cost of fixing a defect after a system is deployed increases exponentially. Not only do you avoid embarrassment before the customer, but it is far more difficult to roll the project back."

The retailer now plans to support development of TMM, which can be scaled to organisations of any size, as an industry standard for improving the quality of software testing.

Marks & Spencer began evaluating a range of process improvement methodolologies last year. It chose TMM because of its potential to allow benchmarking with other firms, and its compatibility with Capability Maturity Model Integration (CMMi), used as a quality standard by its offshore partners.

The firm, which worked with a specialist consultancy, spent 135 man days of effort assessing the quality and testing processes of its own IT department and its three offshore Indian outsourcers. Staff conducted 30 confidential interviews with programmers, developers and managers.

The assessment highlighted a range of areas for improvement, including the need for consistency in application test planning, the need to prioritise testing and fixing work according to risk, and the need to consistently log the results of software tests.

"One of the first surprises we had was that CMMi level 5 organisations did not necessarily have robust testing processes in place. Particularly in the area of testing techniques we found significant weaknesses," said Goslin.

Marks & Spencer and its offshore suppliers have begun rolling out improvements to the software quality processes, which it hopes will become part of the "culture" of the organisation.

"It has paid for itself in less than a year, with the quick wins that we were able to put into place," said Goslin.

The company is backing a non-profit making group, the TMMi Foundation, launched last week, which aims to develop TMMi into an internationally accepted benchmark.

It will enable Marks & Spencer to compare its software quality against competitors, and will provide it with a yardstick for selecting software suppliers.

"What a business wants is to have an IT department that will develop to time and to budget. If you plan the project properly and run the project properly, and use process improvement methodologies, you start to get predictability.

"You get the ability to estimate the timing of project and the cost of projects more accurately," said Goslin.


Testing Maturity Model

The Testing Maturity Model (TMM) aims to help firms assess their software quality processes and identify weak areas. It helps firms identify improvements that can significantly reduce the number of bugs in code, and improve the speed and development of IT projects. The model rates quality processes against five levels. Most firms rate at level 1 or 2, far below the level 5 optimum. TMM was developed in 1996 by the Illinois Institute of Technology to address gaps in the Capability Maturity Model Integration (CMMi), widely used as a quality standard by outsourcers.



Capability Maturity Model

Capability Maturity Model Integration (CMMi) is an independently assessed process improvement standard. It helps organisations integrate traditionally separate functions, set process improvement goals and priorities. Organisations that have adopted it, claim improvements in productivity and quality in IT development work. The technique, developed by Software Engineering Institute at Carnegie Mellon University in Pittsburgh, Pennsylvania, is scalable, and can be used to cover projects or an entire organization. There are five levels of accreditation.



Read more on IT outsourcing