After you define your scenario and armed with clear objectives, we are now in a great position to define the participant criteria for our user testing.
Selecting Participants for Usability Testing:
Let’s follow the same example as before and define what participant criteria we’ll need for an online coffee subscription service.
Things to consider:
Normally you’d have an idea of who you’re looking for based on the research you’ve done. For the example that we’ve been working through, let’s assume we have that information on hand to fill out those criteria:
Online coffee subscription criteria:
- Age — 18-50
- Gender — Female / Male
- Behaviour — enjoys making and drinking coffee at least on a daily-weekly basis
- Traits — Familiar with online subscription services
So, the testing participants we are looking for will be:
Anyone who enjoys making and drinking coffee at least on a daily-weekly basis.
It would be good to get a mix of participants, with some who are familiar with online subscription services, and some who are not.
What to avoid when defining your participant criteria:
Keep it broad, stay away from personas
Be careful not to narrow your audience down to exact personas if you have them. Personas use a face to represent a group of people so that you can understand where they might be coming from and where your product is being used.
Narrowing your audience based on personas can be good during the Discovery phase when you need to understand the exact contexts and environments your users are in. On the other hand, when it comes to the Development stage and usability testing, such exact demographics may not be as important, because critical usability issues are generally issues of everyone.
Watch your cost-benefit ratio
Once you have some criteria in mind for testing participants, something we also consider is the cost-benefit ratio:
Will the time, effort and money spent finding the exact right participant be rewarded with the amount of information you can find?
Sometimes it’s good to get someone who’s perfect as long as it doesn’t take a lifetime. We don’t want this to become such an expensive and frustrating process that you end up not wanting to do usability tests all together; just because you can’t find the right people to test with.
Normally we are lenient on superficial requirements like age and gender but have strict rules on the types of behaviour we won’t budge on.
There are times when we are unsure of how beneficial it will be to get ultra-specific with selecting our test participants. Our experience has consistently found we get better results from broadening our participant criteria in favour of making the test happen and as frequently as possible. Making sure to tweak our requirements each time to make sure we’re on the right track.
Our rule of thumb:
Test with broader ranges of people and test more frequently