Testing time for e-commerce

The majority of Web applications are launched untested. Can companies afford to skimp on trials when the public can so easily...

The majority of Web applications are launched untested. Can companies afford to skimp on trials when the public can so easily vote with their feet? Julia Vowler reports

Let the users do the testing. That is the time-honoured last post of every harassed IT project manager. But in the age of e-commerce it is a luxury that can no longer be afforded.

E-system users are not run-of-the mill workers, as may be the case for the rest of the company. They are fickle and extremely expensively marketed tocustomers whose custom pays the IT department's salaries, and who will take their e-wallets to another e-business site at the drop of slowly downloading screen.

Having a reliable e-business system is an absolute requirement for the e-market, and the only way to be sure of having one is to test it. Yet that imperative all too often runs counter to the actuality of e-business initiatives, which are under huge pressure to launch something - anything - as fast as possible.

If testing is something that gets squeezed out under pressure of time and cost in "ordinary" systems development, how much more likely is it to go the same way when the world has gone e-crazy?

It may also seem to be because users are surprisinglyrelaxed about how critical to their ventures e-business is. A newly published survey of more than 200 companies by Benchmark Research, carried out for testing tools specialist Mercury Interactive, found that only "67% of those were using their Web site for on-line order taking, and only 56% who were producing or delivering a service, felt their Web site to be critical."

Miriam Bromnick, testing consultant and author of the Ovum report on testing, warns, "The opportunities the Web is offering means that people are cutting back on testing and don't realise the risks of doing so."

She says even those in IT need to realise that testing Web development has crucial differences from testing other systems, most notably in the area of load and performance, because of the far higher number of users - potentially millions - compared with the few hundreds or thousands of a corporate-wide client-server system.

"There is no alternative to using automated testing tools which can simulate the hits and then monitor the bottlenecks, including those in the network as well," Bromnick argues.

Web testing is also different in that, unlike ordinary roll-outs, a Web site is inherently dynamic - content must change constantly, and as the online market develops, businesses will do new and different things on the Web. So the whole Web development lifecycle must take into account that regular upgrades are a way of life, and that each one must be properly tested before release to a merciless public, says Bromnick.

These differences have not yet been fully taken onboard, suggests the finding of another survey on testing by software testing tools supplier Original Software.

"Only one third of companies think there are major differences to the way Web applications need to be testing," says the report. "This suggests that two-thirds are missing vital testing areas." And it is not as if the tools needed for Web testing are not out there.

"You do need different tools, for example, that can look at the different languages and protocols the Web uses," says Bromnick. "But over the past two years the major suppliers have been releasing products specially for the Web, even for Wap developmen."

Similarly, testing houses have caught on to the new wave of e-development, and even tools suppliers, such as Mercury, offer a Web testing service for users with little time to do it themselves.

Of course, testing the Web is only part of the task. E-business is a lot more than a good, well-designed Web page, it is the end-to-end business process from the Web site through the back-office production systems and right back to the delivery lorries on the motorway. Failure anywhere could mean customers decamping in droves.

Why testing is tougher on the Web

  • The business risk from failure or inadequate performance is much greater than from an internal system. Your Web site interacts directly with your customers, who will first abandon you and then bad-mouth you (so will the press and the City)

  • Failure is very, very, public

  • There is huge pressure to launch an e-business as soon as possible, making it more tempting to skimp on testing, both in terms of time and money. It is the marketers that are often in control of e-business initiatives

  • E-systems are new technology, and there are precious few industry parameters as yet. Worse, doing business on the Web is new, so predicting user take-up is difficult

  • You cannot do Web testing manually - only tools can simulate the millions of hits a site can receive, and show up where bottlenecks are likely. But automated tools require both a learning curve and an up-front cost

  • E-business is not just your Web site. It is the entire business process, including fulfilment

  • E-business is not static, and nor are Web sites. Content changes constantly, and everyone is still discovering what works on the Web and what doesn't - this means a lot of continuous revamping. Adding to and modifying Web sites means testing all over again, end-to-end, as Halifax Bank found to its cost when it changed the application and went live without retesting and customers found themselves able to access each other's share portfolios

  • If you think Web skills are scarce, Web testing skills are even more so. Expect to pay £800-£1,000 a day. And book them well in advance

  • Web testing tools will cost from £40,000 to £150,000 plus hardware

    The good news

  • First-mover advantage is ceasing to be the mantra of the e-whizzers. "Get it right" is replacing "do it fast"

  • Some usage metrics are beginning to emerge. For example, says Christian Denning at E-net Software, expect half of all hits within two hours at about 4pm

  • Web testing tools are available and testing houses are too

  • There is growing recognition that testing is not optional, but mandatory, and testing Web systems has key differences to testing client-server systems

  • Read more on Web software