Why beta testing is the new black
Whether it's Google glass, the latest iPhone IOS or NVivo for Mac—beta testing is all the rage. Not so long ago, we left this kind of testing to the IT crowd and the folk with nothing better to do on a Friday night. But now, in a world gone gaga for technology, beta testers are de rigueur and up to their (tired) eyeballs in street cred. So, what is the point of beta testing and what's with the new-found popularity?
What are the goals of a beta test?
Let's start by looking at how beta testing fits into the software development process. Before it's released, software goes through a rigorous in-house 'alpha' testing phase conducted by an expert team of testers. This irons out most of the glitches but it's difficult to replicate every possible computer configuration and mirror all the amazing things users try to do with software.
That's where beta testing comes in.
When a product is fairly robust (but maybe missing a few features) it is ready to go out for beta testing. Developers invite a group of users—with different backgrounds and levels of expertise—to use the product normally. They ask them to report back on any issues including whether the software is fit-for-purpose.
Beta testing is about more than just finding bugs.
By collaborating closely with beta testers, developers can confirm whether the software is headed in the right direction and that it meets user needs. Have we missed something important about the way users work? Have we seen the trees but not the forest? Have we solved the wrong problems? During this stage, developers can forge long-lasting relationships with users that they can draw on as the product evolves. As Adam Long (Chief Technology Officer at QSR) points out:
Close collaboration with beta testers is crucial and helps us to prioritize future development activities
If you're interested in how QSR goes about beta testing, you can find out more in Adam's great post about running a successful beta test program.
Beta testing programs can also create a buzz that gets users (and developers) excited about an impending software release.
What do beta testers get out of the experience?
Developers are aiming for quality software but why do users want to become beta testers? I mean it's hard unpaid work, isn't it?
Allan McDougall is a research coordinator and doctoral student with the Centre for Education Research & Innovation (CERI) in the Schulich School of Medicine & Dentistry in London, Ontario, Canada—he was a beta tester for NVivo 10. I asked him to share his experiences:
What made you want to be a beta tester?
I became a beta tester because I wanted to find out how NVivo 10 would be integrating social media data. I was very impressed by the jump from NVivo 8 to NVivo 9 and I wanted to be a part of NVivo 10. Beta testing is an important role in software development. As it stands, NVivo 10 is the primary software I use for my research work. Being a small part of the latest release was very cool.
What did you expect?
I expected that I would be sent an access link to download the NVivo 10 beta and I would be given a period of time to toy around with the new product. I didn't expect the specialized section in QSR forums, the rigorous participation requirement laid out by the product development team, and the fantastic webinars hosted by Marcus and colleagues outlining the software's new features. I've reviewed the webinar on working with web data at least 6 times since the NVivo 10 beta testing program.
What was actually involved and how much time did you spend testing?
I actively did everything I could to try and break NVivo 10. I imported multiple large project files into test projects. I imported massive sets of PDF documents. I imported social media data, web data, audio and video data. I ran every possible type of test query I could think of on this test project. All in all I think I spent about 6-8 hours fooling around with the NVivo 10 beta software.
What did you get out of it?
I gained the confidence of knowing how all of the new features work and being able to explain those features to my colleagues in research. I've since presented several workshops on using social media data in NVivo to be able to spread the word to my peers. I also had the chance to network and maintain contacts with a number of NVivo power users through the special QSR forum. Further, beta testing left me with a free NVivo 10 license which I deeply appreciated!
Would you recommend the job of ‘beta tester’ to other NVivo users?
Of course! As a researcher, I like to know what's on the horizon when it comes to working with my team's data.
Allan frequently posts about NVivo and other topics on Twitter (@allanmcdougall) and his blog (mrqualitative.com).
Do you have what it takes to be a beta tester?
Beta testing can take time and patience. You have to try things out and make a commitment to providing useful feedback—both positive and negative.
On the flip side, beta testing gives you a voice and helps you to influence the development of the software. But more importantly, it makes you an envied member of the 'beta test brat pack' and gives you license to break things—what's not to like?