YES, yes and yes – of course: Usability testing is a method in user research. You are evaluating a system or product you design by observing people who think aloud while they use that system/product and solve pre-defined tasks and so help us to spot problems with the system or product.
Recruiting the right participants is a crucial and often neglected task when planning usability tests. In best-case scenarios, your test participants should also slightly differ within a specified user group. This doesn’t necessarily mean that recruiting has to be expensive and is not feasible for smaller companies or organizations.
If you’re running usability tests with people who do not represent your actual users, the problem is that you might not see incidents or problems that your true users will have. There might be differences between who you run your test with and the people who will be using your system.
Think of expertise for example. Are your actual users technically sophisticated, or do they tend to be relatively new and inexperienced with technology? Or think of cultural backgrounds. These are only some criteria that will influence how people will use the system you are designing.
As userfocus says:
„Screen for behaviours, not demographics.“
You CAN test with people who do not fit your criteria and who are not representative of the usage of the system you want to test – but then you have to be aware of that fact, and write it in your report. You will also have to consider that your findings might or might not apply to the actual people/population who will use your system. You might spot problems or discover ways of thinking, cognitive schemas and mental models that do not match those of your true users.
Even if recruiting the right people may sometimes be challenging, it is worth the effort because it gives you more information about how well your system is designed, for people who will be using it.
Links
