Crowdsourcing slashes software testing time

US-Israeli firm uTest has pioneered crowdsourcing to help companies test their software. The technique is attracting growing number of converts, including Microsoft and Google.

Most people are familiar with crowdsourcing from reading the online encyclopaedia Wikipedia, which uses volunteers across the world to update its content. But US-Israeli firm uTest has pioneered crowdsourcing to help companies test their software. The technique is attracting growing number of converts, including Microsoft and Google.

Advocates of crowdsourcing claim that it allows them to test their software in fraction of the time taken by traditional software testing - at a significantly lower cost, and often with better results.

It allows companies to recruit tens or hundreds of specialised testers to put their software through its paces via an online portal. The testers, often working at weekends and evenings, collaborate and compete to find and document the bugs. A two-hour online testing session can achieve as much as a week of conventional software testing in the IT department.

uTest has some 18,000 testers on its books. including 2,500 testers in Europe and 950 in the UK. They have experience in testing software ranging from cadcam to mobile phone applications, financial packages and websites.

Why do the testers do it ?


One reason is financial. Testers can expect to earn cash for each bug they find. Rates start at between $2 and $12 when testers first sign up. But participants can earn more as they prove their worth. The top testers earn up to $30 for each bug found.

"Its not just about money, its about reputation. It gives a tester a place he can expand his skills and test his skills. It makes him a better test manager," says UTest CEO and founder Doron Reuveni.

At work, testers focus on a narrow field, but they can use uTest in their spare time to learn new skills. The site provides training to help them develop.

"Testing is an ideal subject for crowdsourcing," says Reuveni. "You need a lot of people. You want them to collaborate, and you want them to work in real time"

Some companies prefer to carry out testing in a real, rather than an online, environment. uTest's UK partner TCL, offers what it calls "Software Slams," where it brings teams of testers together in a pub or a restaurant to put software through its final paces. It calls the process Pub Exploration of Software Testing (Pest).

The result is a geeky version of the pub quiz, where teams compete to find the most bugs, in return for cash prizes, free beer and pizza. The events often raise money for charity.

Tony Prosser, director and co-founder of TCL, says that the key to a successful testing session whether online or offline is to lay down the ground rules before you start.

Testers are pedantic people, and if you question the validity of a bug they have found without good reason, you risk causing offence, he says.

It is important to set out in writing exactly what you want to test and what sort of bugs you are looking before testing begins.

"As someone with 20 years' experience in testing, I understand the way their minds work," he says.

About UTest



uTest was founded in 2007 by Doron Reuveni and Roy Solomon. It went live in April 2008 with 10 beta customers after securing its first funding in October 2007.

Based near Boston in the US, it uses crowdsourcing to offer a range of testing services, including vulnerability testing, combined hardware and software testing, and total performance testing.

The company has 150 customers on its books, including Microsoft, Google, and Thomson Reuters.

uTest employs 20 staff in Boston, including six developers with another 12 developers working from Buenos Aires.

More Computer Weekly articles on software testing:



HBOS turns to TMM for software testing >>

Lack of software testing to blame for Terminal Five fiasco >>

People with autism prove their own in software testing >>

Read more on IT project management