Weighing up the options for software testing

We examine the reasons why software testing is increasingly being outsourced

Business-critical applications are increasingly being tested by third-party suppliers as businesses strive to reduce the high costs associated with software failure without the huge investments needed for rigorous in-house testing processes.

The digitisation of business has brought with it an increasing reliance on software for internal and customer-facing activities. This in turn has increased the focus on software testing within the development process. This is increasing costs in terms of time, money and human resources And this in turn is accelerating the rate at which software testing is outsourced.

Software applications are becoming increasingly important because the web offers businesses new ways of servicing customers and linking with third parties. Failures in these systems can be costly. Research from software quality tester Cast found the average big application costs an extra £2.23m to fix problems with the code once software has been released.

Banks are perhaps the organisations that spend the most on software development and, although not necessarily by outsourcing, they are investing in improving their testing processes to cut losses from failures after production.

HBOS has invested in testing to set a benchmark for itself and its suppliers. The bank used the Test Maturity Model integration (TMMi) software testing methodology across three of its four major IT departments in a programme designed to reduce the number of errors in the thousands of applications it builds every year. It will use the knowledge and processes gained as a reference point, either to help improve processes or to get its development partners to meet HBOS standards.

Meanwhile, Nationwide has invested in a virtual testing environment as part of a £1bn IT transformation under which it is replacing core software platforms.

Third-party testing

But not all businesses have the resources to do their own testing. According to a recent report from analyst company Pierre Audoin Consultants (PAC), software testing is one of the most important tasks for 91% of IT departments, and almost all believe it is crucial to outsource this activity. The report also revealed that three-quarters of companies already use service providers with onshore and offshore capabilities to provide testing services.

According to analyst Nelson Hall, the global testing services market totalled $8.4bn in 2011, and although 2012 is expected to be flat, it predicts an average 9% growth every year over the next five years.

The challenge of introducing large numbers of applications to deadline is a challenge that drove energy provider Centrica to outsource testing to dedicated testing firm Software Quality Systems (SQS). Centrica’s software testing requirement increased because it is working on smart metering applications in relation to the government’s Smart Meter Implementation Programme.

Utility companies have to offer smart metering services to 53 million consumers by 2019. Software to collect and analyse huge volumes of data will require rigorous testing before going live because failures will be headline news. Centrica will gain access to testing resources locally and offshore through SQS’s software testing resources in India and South Africa.

Deutsche Bank is also taking advantage of SQS services. SQS will test the bank’s applications until the end of 2013, with integration testing the primary challenge. This will be delivered via the nearshore and onshore test centres in Germany and Egypt, with the target of expanding it to test centres in India and South Africa.

Resource-intensive process

Outsourcing is not always the fastest and cheapest way to test software

Nelson Hall analyst Dominique Raviart says software testing is an increasingly important discipline that requires more focus. As a result, it lends itself to dedicated organisations.
“Cost savings are a major driver, as is the need to do testing in a more professional manner. Testing has become more professional and now requires significant investment in terms of tools, people recruitment and automation,” he says.

Hall points out that there is a shortage of the right skills in the UK: “Finding onshore testing professionals is difficult. There are very few testing classes in universities, and when there are, they are about quality, rather than test execution.”

As a result, software testing is being outsourced and offshored, he says. “Testing is a human resource-intensive activity – in spite of all automation work, it requires people,” he says. “India, therefore, looks like a good alternative for performing testing work.”

Nelson Hall predicts Indian offshore-delivered testing services will grow by 8% in 2012, while onshore-delivered testing services will decline by 5%. As well as offering cost savings though centralisation, the introduction of common processes and a reduction in human resources, 

Raviart says a focus on testing will build up a level of expertise. He says there are about 165,000 career testers in the world, adding that application developers/business analysts perform testing as part of their work.

Testing during development

IDC told Computer Weekly last year that software testers are increasingly in demand. In the past, software testing has been bundled with projects and often done at the end of the software development lifecycle, but businesses are increasingly contracting independent software testers to test throughout software development, according to Jennifer Thomson, software testing researcher at IDC.

“There is a lot more interest in standalone testing across Europe because there is a focus on quality. When we started looking at software testing, it was predominantly a process that was added at the end. It was often a reaction to a business requirement rather than a sound methodology,” she says.

Suppliers are reacting to this trend. Capgemini, for example, integrated its software testing resources to help it compete with pure-play software testers. The French IT services provider integrated its specialist software testing arm Sogeti with its other testing resources. This bolstered the resources and footprint of the Sogeti business to help Capgemini compete with large software testing specialists.

Geoff Thomson, chairman of the UK Testing Board and consultancy director at Experimentus, which runs a TMMi accreditation programme, is a proponent of ingraining testing in the development process. He says more and more suppliers are going through accreditation, and user businesses, such as HBOS, are also doing so to benchmark suppliers.

Thomson says software testing is being outsourced to check it does what it should do and to measure its performance, security and capacity, but there is a shortage of user testing. While outsourcing is increasing, he warns that most successful outsourcing deals happen only after some failed attempts, through which lessons are learnt and better partnership arrangements are reached.

“Overall, I see outsourcing of testing as a positive thing that, with the right focus and experience, can create real value for the customer and supplier,” he says.

Many suppliers are buying into this, says Thomson, but he warns businesses not to lose control when they outsource. “Some suppliers see the value of a real relationship with the customer and spend time nurturing and supporting their customer to achieve a bigger bang for their buck, but there are also a lot which still push the ‘land and expand’ approach regardless of the client’s wishes.

“I also see customers which see their role as policing the contract to the finest detail, creating an adversarial atmosphere which is in no way conducive to long-term success,” he says.
Thomson says outsourcing is not always the fastest and cheapest way to test. “In a lot of instances the failures are still due to a belief that outsourcing will reduce costs – in some instances I have seen increases in the actual resources needed to deliver the outsourced service, some as high as tenfold,” he says. “In some cases I have seen the recreation of the company test team to retest everything that comes back from the outsourcer – in one company, the number of internal testers is larger now than it was before testing was outsourced. How can that be good?”

Getting it right

There are common mistakes when outsourcing testing, says Thomson. These include testing providers not being able to contribute to the software development process, businesses outsourcing accountability and responsibility rather than just the testing itself, and suppliers putting their best resources on projects initially but downgrading them later.

Software runs our lives today. We use hundreds of applications without noticing them. Businesses will only prosper if their internal business, their customers and their partners have software that does what it should when it needs to.

Failure is expensive – but so too is embedding testing within an organisation. If testing processes can be implemented once by a supplier and used by numerous suppliers, the cost savings are obvious, as are the gains in efficiency and effectiveness gleaned through experience by focused testing organisations.

Read more on Software development tools