Top 10 reasons for developer crowdtesting

The Computer Weekly Developer Network this week caught up with Mithun Sridharan of Passbrains, a company that describes itself as an on-demand real-world testing specialist for mobile, web and desktop applications.

Sridharan has commented that with Google, Apple and Microsoft “practically giving away” their development tools for free (does he really mean to include Microsoft in that list?), there is a growing developer base creating mobile apps and responsive web sites for Android, iOS and Windows platforms.

But says Sridharan, it is easy to underestimate the costs of building and monetising an App successfully.

Passbrains staunchly advocates that programmers consider crowdtesting.

NOTE: There aren’t many public definitions of crowdtesting available, so we will rely on a public wiki-based one here: Crowdtesting (or crowdsource testing) is an emerging trend in software testing which exploits the benefits, effectiveness and efficiency of crowdsourcing and the cloud platform. It differs from traditional testing methods in that the testing is carried out by a number of different testers from different places and not by hired consultants and professionals. The software is put to test under diverse realistic platforms which makes it more reliable, cost-effective, fast, and bug-free.


Sridharan’s top 10 benefits of crowd testing are as follows:

1. Maximum value and focus

The law of comparative advantage states that maximum value is derived if each party focuses on its core competence. Developers should focus on building products and leave software testing to testing experts to derive maximum value.

2. Improved cash flow

The crowdtesting pay-per-bug pricing model means that you pay only for unique bugs that testers discover – which is better for cash flow compared to an in house team.

3. Sanity check

Developers can quickly test whether their application addresses customers’ pain points and validate the business hypothesis by targeting a select audience, gathering feedback and estimating the chances of adoption.

4. Usability

Crowdtesting helps uncover usability issues and developers’ blind spots much earlier. Once an app is on the Apple or Google Play store, most customers won’t bother to submit crash reports, but simply give the App a low rating, which is incredibly hard to repair.

5. Test coverage

In-the-lab software testing, normally performed alongside development, cannot cover all devices and system configurations, deployment scenarios and usage patterns. Many technical issues and bottlenecks only come to light when the product is ‘in the wild’.

6. Speed

The crowd will help discover most critical bugs in a very short timespan due to much larger numbers (often factor 10-20) of testers engaged in a test. Many case studies show that crowdtesting can discover hundreds of issues within less than two days in applications which have passed internal QA!

7. Comprehensiveness

Crowdtesting eliminates any intrinsic bias, characteristic of a local testing team and adds richness and diversity. As different testers may follow different permutations, a multipath approach leads to more bugs discovered in a shorter timespan.

8. Expertise

Crowdtesters are professionally qualified enthusiasts marked by an intrinsic motivation to discover bugs and improve software quality. These professionals are unsparing in their feedback and make excellent development partners.

9. Flexibility

Crowdtesting is a flexible and smart way of scaling your test workforce at short notice.

10. Well proven

Crowdtesting has already been used by likes of eBay, Amazon, GE, Microsoft, Google and Facebook to develop user-centric products. Google regularly deploys crowd testing for 14 of their major product lines.