Software usability tests can save you millions

Some of the world's leading corporations are using a new standard developed by the US National Institute of Standards and...

Some of the world's leading corporations are using a new standard developed by the US National Institute of Standards and Technology to make usability a key factor in their choice of software.

Boeing, the US aircraft manufacturer, for example, is making a product's usability - the ease with which end users can be trained to operate the product - a fundamental purchasing criterion.

"We simply can't afford to pay for products that cost us a lot of overhead anymore," said Keith Butler, technical fellow at Boeing's Phantom Works research and development arm.

When thousands of end users are involved, design flaws can cost millions of dollars in lost time and productivity, he said.

Boeing's purchasing approach has been helped by the recent development of a standard for comparing product usability that was spearheaded by the National Institute of Standards and Technology (NIST).

The Common Industry Format for Usability Test Reports outlines a format for reporting test conditions and results and gives user companies enough information about a test to replicate it.

This makes it an effective way of objectively evaluating software, according to its backers.

Next month, NIST intends to seek international recognition for the standard, which has already received American National Standards Institute certification.

The standard's ultimate success as a purchasing tool will depend on whether other companies follow Boeing and make usability a key purchasing criteria alongside traditional requirements of functionality, price and system requirements.

If that happens, users say, the standard could have a far-reaching effect in improving the usability of software.

"The real value of CIF is that if vendors know large software purchasers are expecting it, they will focus their attention on usability and make their products better before they ever come out the door," said an IT usability expert in a US financial services organisation.

Boeing played a lead role in the development of CIF after its experience and internal studies showed that usability played a significant role in total cost of ownership. In one pilot of the CIF standard on a widely deployed productivity application, it estimated that improved product usability had a cost benefit of about $45m (£28.5m).

Butler said was better to have vendors refine an interface design "than to have thousands of end users doing it involuntarily on top of their jobs and then just feeling frustrated".

A key benefit of the CIF approach is that it will allow the IT department to spot problems before a product is rolled out to employees, according to Doug Francisco, director of IS architecture at Boeing's commercial airplane division.

The company has looked at usability in purchasing, "but sometimes we wouldn't discover the inefficiencies of a software product until we brought it in-house", he said.

Microsoft, as a CIF development participant, has incorporated the usability testing it conducted on its Windows XP, Windows ME and Windows 2000 operating systems into the CIF format, said Kent Sullivan, who leads the usability team for Windows.

Sullivan said Microsoft is prepared to use CIF but noted that its adoption will depend on customer demand. Microsoft typically does not receive questions about usability from customers, so when users do ask about it, he said, "it indicates that they are ahead of the curve a little bit".

In the past year, interest in CIF has grown from about 50 firms taking part in the NIST effort to more than 150, including PeopleSoft, Oracle and Eastman Kodak.

The CIF format will also be adapted for hardware testing, said Emile Morse of NIST. Morse believes CIF makes it possible for vendors and users to discuss usability as a science rather than marketing hype. "I think CIF gives a lot of credibility to the practice of usability," she said.

Read more on IT for small and medium-sized enterprises (SME)