Demand characteristics are one of the so-called experimenter effects in behavioural research. They describe the tendency of interview/usability test participants to give you (the experimenter) what you want based on what the participants think and what you might expect from them.
This doesn’t require that you tell them what you want explicitly and give them obvious cues. Participants only need to guess/hypothesize what you want from them based on some subtle cues you send out which are often based on your implicit opinions. So the assumption your participants make regarding what you might want from them is sufficient enough. For example, a general assumption which occurs during usability testing sessions could be that you want them to like the product you’ll be testing.
It’s important because demand characteristics may put the complete validity of your usability test/interview at risk.
Here’s how to weaken/temper the effect:
- Make clear at the beginning and throughout your test or interview that you want to hear honest feedback. Try to take the fears of non-desirable answers away.
- Be aware of subtle cues and nonverbal language your participant sends out. Seems the answer is forced. Is he/she struggling? Take notes during the test, ask afterwards in the debrief interview, and eventually play the scenario through again.
- In addition to an in-person usability test/interview, provide a post-test questionnaire like for example the SUS (System Usability Scale) which is a quantitative tool to measure the perceived usability
- Last but not least: Demand characteristics may be one of the reasons the designer, who designed the system is not always the right person to conduct a usability test or do debriefing interviews, because he or she may send subtle positive cues about the system he/she designed and so participants might react more positive than they feel about the task or question you gave them.
For this reason – even though we also claim research should be not outsourced but should be conducted by the interaction design team itself – it may be worth considering another person to conduct the test or interview – in the best case people that are not directly involved in the design or development team. If this is not possible, it’s important to be aware of this bias throughout your test.
Comic image
