Worst beta test ever!
Several years ago, I had the opportunity to participate as a beta tester of a new telephone solution. Even now, the frustration of that experience still haunts me; snapping me out of my restless sleep, drenched in a puddle of sweat. OK, that might be a bit of an exaggeration. Let’s just say the experience wasn’t great. Let me share with you a tester’s perspective with tips to help you with your next beta testing program.
[TESTER]: “Am I the only one here?”
As a coordinator, the use of an online forum is a valuable element in managing user experiences. It is probably the most important scalability tool you can use. Why? Because you can be sure that if one person has had a problem or discovered a workaround, others will also want to benefit from your experience. By establishing a forum to augment your beta tests, you ensure that important information is self-sustaining and self-distributed. As proof, I remember thinking, “I know someone else has run into this problem, but I have no way of finding out how they solved it.”
Tip: Online forums will support your team and allow you to scale.
[TESTER]: “Sorry to bother you with your product, but if you’re not too busy, can I ask you a question?”
Lead by example; be excited about your project. I remember that a beta coordinator often seemed too busy or, worse, annoyed when I called him. Your attitude as coordinator is the cadence of the project.
Hint: Nothing will kill the incentive to participate faster than a beta coordinator not excited about the product.
[TESTER]: “Is this the kind of feedback you wanted?”
Providing clear guidelines, including steps for reporting events, will prevent a lot of confusion during testing. Also, not having a location to accept reported issues may indicate that beta testing is just a cursory exercise. When critical issues are not acted upon, testers will feel that it is not worth taking the time to report an issue.
Tip: Provide written testing guidelines, including a completed incident report form. Make sure the form has enough fields to avoid having to go back and try to get more data later.
[TESTER]: “Do you want to know what else I thought about this?”
While a defect tracking tool is a must for every beta test, extensive survey feedback tools are also important. Remember that your beta test is not just an engineering exercise. This is an opportunity to test the entire launch preparation process. Getting feedback on ease of use, FAQs, and competitor observations are invaluable data. Such information is uniquely secured during field testing. Evaluators will become frustrated if they feel that important information is not being heard.
Tip: Develop a survey to capture specific and generalized feedback. Review this survey with your testers BEFORE the start of the test so they know what to focus on.
[TESTER]: “Hey, is this beta test still going on?”
You probably already have weekly status reports with your management team on beta progress. Why not augment this with a weekly status for your testers? This may include acknowledgments of input provided so far, key findings that may affect the group, and days remaining for testing. I remember being part of a particular beta test for over a year and wondering on several occasions if anyone was still in the “Command Chair”.
Tip: A weekly status report for your raters will keep the group energized and focused; increase the quality and quantity of responses.
I hope you found this article entertaining and educational.
Good luck with your next beta testing program!
-ed