As we all know: UX – U = X . But, unfortunately, as we all also know: sadly it’s often difficult to get a buy in to conduct user research at the beginning of a project or even establishing an ongoing research culture. But, the good news is: There is still hope out there and here’s what you can do while you have to deal with research blockers.
Demand characteristics are one of the so-called experimenter effects in behavioral research. They describe the tendency of interview/usability test participants to give you (the experimenter) what you want based on what the participants think what you might expect from them.
This doesn’t require that you tell them what you want explicitly and give them obvious cues. Participants only need to guess/hypothesize what you want from them based on some subtle cues you send out which are often based on your implicit opinions. So the assumption your participants make regarding what you might want from them is sufficient enough. For example, a general assumption which occurs during usability testing sessions could be that you want them to like the product you’ll be testing.
It’s important because demand characteristics may put the complete validity of your usability test/interview at risk.
Here’s how to weaken/temper the effect:
1) Make clear at the beginning and throughout your test or interview that you want to hear honest feedback. Try to take the fears of non-desireable answers away.
2) Be aware of subtle cues and nonverbal language your participant sends out. Seems the answer forced? Is he/she struggling? Take notes during the test, ask afterward in the debrief interview, eventually play the scenario through again.
3) In addition to an in-person usability test/interview, provide a post-test questionnaire like for example the SUS (System Usability Scale) which is a quantitative tool to measure the perceived usability
4) Last but not least: Demand characteristics may be one of the reasons the designer, who actually designed the system is not always the right person to conduct a usability test or doing debriefing interviews, because he or she may send subtle positive cues about the system he/she designed and so participants might react more positive than they actually feel about the task or question you gave them.
For this reason – even though we also claim research should be not outsourced but should be conducted by the interaction design team itself – it may be worth considering another person to conduct the test or interview – in best case people that are not directly involved in the design or development team. If this is not possible, it’s important to be aware of this bias throughout your test.
Comic image by http://www.markstivers.com/wordpress/?p=67
A few weeks ago Steffi spoke at the Berlin Ladies that UX meetup “An evening about user research” about the different types of usability tests. Here is a brief summary of the talk
YES, yes and yes – of course: Usability testing is a method in user research. You are evaluating a system or product you design by observing people who think aloud while they use that system/product and solve pre-defined tasks and so help us to spot problems with the system or product.
Recruiting the right participants is a critical, very important and often neglected task when planning usability tests. In best case scenarios, your test participants should also slightly differ within a specified user group. This doesn’t necessarily mean that this has to be totally expensive and is not feasible for smaller companies or organizations.
If you’re going to run usability tests with people who don’t represent your actual users, the problem is, that you might not see incidents or problems that your actual users will have – because of the differences between who you run your test with and the people who will actually be using your system: Think only of expertise – are your actual users technically more sophisticated or do they tend to be relatively new and inexperienced with technology? Or think of cultural backgrounds. All those criteria (plus of course more) will have influence on how people will use the system you’re designing.
Or as userfocus says: „Screen for behaviours, not demographics“
Of course you CAN test with people who doesn’t fit in your criteria and who aren’t representative for the usage of the system you want to test – but then you have to be absolutely aware of that fact, and write it in your report – plus: think about the fact that your findings might or might not apply to the actual people/population who will use your system. You might spot problems or discover ways of thinking and cognitive schemas and mental models that don’t match your actual users.
Even if recruiting the right people may sometimes be a challenging task, it’s absolutely worth the effort, because it gives you a lot more information about how well your system is going to be designed for people who will be actually using it.
Links:
Recruiting Better Research Participants
Writing the perfect participant screener
Recruiting Usability Test Participants
Image source: https://flic.kr/p/QHpkCL
In the midst of last summer’s dog days we were following our morning routines containing smalltalk, twitter coffee, dealing with waking up, more coffee, slowly getting into workmode, opening mail, when – bam! – _the_ email appeared in our inbox. Now, we were fully awake.
Attractor theme is extremely customizable, fully responsive and HiDPI ready WordPress theme based on a well structured framework which allows you […]