“If only we knew then what we know now” is a common lament in business – especially in the supply chain where one small failure upstream, such as a cloud-based application that gets hung up in a particular browser, can have a profoundly negative effect on everything that occurs downstream.
While the quality assurance (QA) team may be diligent in their pre-release testing, there is only so much for which they can account. Supply chain applications, especially those with a mobile component, can end up running on dozens of different platforms and operating system versions in hundreds of languages and dialects while connecting to a multitude of other applications – some of them very niche. It’s impossible for even the largest, most well-funded QA team to account for them all, creating a risk of unhappy customers and low star ratings.
There is a solution, however—crowdtesting. This growing enhancement to traditional QA programs enlists the help of thousands of freelance QA professionals throughout the United States (and around the globe if there is an international component) to put PC and mobile applications, chatbots, web portals and Software-as-a-Service offerings through their paces in all the permutations in which they will operate. In a few hours, days or weeks (depending on the complexity of the application and urgency to complete testing), they will find the bugs that are specific to particular instances that most internal QA teams would not have the time or resources to discover. This thorough approach, which is often referred to as a “mission,” ensures the application works as it should no matter what users throw at it once released.
Following are a few of the reasons every organization should make crowdtesting a component of its overall supply chain software release plan.
Professional testing
A typical question that comes up on crowdtesting is, “Isn’t that what a beta test does?” Not really.
A beta test involves random users, many of whom are already fans of the application. These users like to be on the leading edge of technology, so it’s likely their other applications and operating system have the latest updates. This makes them a relatively homogenous group. When they test the application, they will test it the way they normally use it, which means many features may be left out.
Crowdtesting is performed by QA professionals who are focused on finding the bugs in applications – and know the best ways to do it. By hiring a known variety of testers, companies can ensure applications are tested across a broad range of platforms and interfaces. The result is a diverse, heterogeneous team that digs in deep across nearly every possible usage – including many the QA team would never have considered – before they can affect the real user experience.
Broad range of testing
Internal QA teams and beta testers tend to focus on features and functionality when they test software. Crowdtesters look at everything about the user experience, such as whether language translations sound awkward or have missing words, whether type is mashed together on a particular mobile device, whether cells in charts are aligning properly and all sorts of other issues. With their in-depth experience, they are also able to identify malfunctions they have previously found in similar applications.
Frees up QA team time
If QA teams are testing applications – especially if they are doing thorough pre-release tests – they are not spending time on fixing the bugs that have already been identified. Each person can only do so much work. When organizations use crowdtesting, it frees the QA team to focus on patching software issues and making other corrections, such as improving translations, and avoids the need to bring in members of the development team to supplement QA.
Faster results
The speed with which testing can be completed is often limited by the number of resources available to perform it. In an internal QA department, that number will necessarily be limited by the realities of the business and development cycles, which often puts executives in the position of either releasing applications that have not been thoroughly vetted or delaying release until testing can be completed.
This concern is eliminated by crowdtesting. Organizations can speed the release timeline by increasing the number of testers assigned to the mission. What might have taken several weeks or months can be accomplished in a few days or a couple of weeks (or even a few hours in some cases), ensuring the expected launch occurs on-time – and with fewer (or ideally no) errors.
Cost-effective
Crowdtesting “teams” are selected based on the specific mission the organization has in mind. For example, a company may have a warehouse management system that will be sold in China, Japan and the Philippines as well as the United States and Canada. The team will be assembled based on the devices and operating systems used by businesses in those locations, as well as their understanding of local languages and customs.
The result is the organization only pays for the precise needs and capabilities required to thoroughly test the application. Additionally, the organization that hires the crowdtesters will normally supply a community manager who provides clear instructions to the testers so they can execute the mission more efficiently, relieving the QA team of this additional duty.
Complete bug report
Once the mission is complete, the project manager should inspect the mission report to ensure each bug only appears once and that all reported bugs are properly documented. This report should include a description of the problem, such as an inability to enter a quantity or trace a lot number; where the bug was discovered; a screenshot and/or video of the bug; the criticality of the bug and its priority to be fixed; and any other pertinent details, such as the device and operating system being used at the time of discovery. Internal QA personnel can then use the report to go straight to any issues that need to be remedied. The project manager is also responsible for executing a client agreement to ensure that all sensitive information about products and projects disclosed during the campaign (i.e., project stories) is fully protected.
Team approach
While it is clear that crowdtesting offers many advantages, its true value is realized when it is part of a multifaceted approach that also includes internal QA and external beta testers. Rather than a replacement, it should be seen as an expansion of testing capabilities that ensures no stone is left unturned – and the organization isn’t left wishing it had known then what it knows now.
