Posts in Analysis & Research

How our expectations determine how we process information

May 8th, 2018 Posted by Analysis & Research, User Experience No Comment yet

How do we process information? In cognitive psychology there are two famous processes involved on how we interpret information – and as humans, we constantly use both: Top- down processing and bottom-up processing. (more…)

User research biases to be aware of: Demand characteristics

March 14th, 2018 Posted by Analysis & Research No Comment yet

Demand characteristics are one of the so-called experimenter effects in behavioral research. They describe the tendency of interview/usability test participants to give you (the experimenter) what you want based on what the participants think what you might expect from them.

This doesn’t require that you tell them what you want explicitly and give them obvious cues. Participants only need to guess/hypothesize what you want from them based on some subtle cues you send out which are often based on your implicit opinions. So the assumption your participants make regarding what you might want from them is sufficient enough. For example, a general assumption which occurs during usability testing sessions could be that you want them to like the product you’ll be testing.

It’s important because demand characteristics may put the complete validity of your usability test/interview at risk.

Here’s how to weaken/temper the effect:
1) Make clear at the beginning and throughout your test or interview that you want to hear honest feedback. Try to take the fears of non-desireable answers away.

2) Be aware of subtle cues and nonverbal language your participant sends out. Seems the answer forced? Is he/she struggling? Take notes during the test, ask afterward in the debrief interview, eventually play the scenario through again.

3) In addition to an in-person usability test/interview, provide a post-test questionnaire like for example the SUS (System Usability Scale) which is a quantitative tool to measure the perceived usability

4) Last but not least: Demand characteristics may be one of the reasons the designer, who actually designed the system is not always the right person to conduct a usability test or doing debriefing interviews, because he or she may send subtle positive cues about the system he/she designed and so participants might react more positive than they actually feel about the task or question you gave them.
For this reason – even though we also claim research should be not outsourced but should be conducted by the interaction design team itself – it may be worth considering another person to conduct the test or interview – in best case people that are not directly involved in the design or development team. If this is not possible, it’s important to be aware of this bias throughout your test.


Comic image by


10 questions for voice interfaces

October 26th, 2017 Posted by Analysis & Research, User Experience No Comment yet

For many years now, interface and interaction designers have been designing for graphical user interfaces (GUI). But in the recent years more and more so called natural language user interfaces (NLUI). This means human-machine interaction does not happen by checking radio buttons, clicking buttons, swiping or scrolling but rather through spoken text in natural language.   (more…)

UX Research: Don’t forget the stakeholders

June 7th, 2017 Posted by Analysis & Research, User Experience No Comment yet

When we are hired as a user experience design studio we are often asked to jump straight into production work like sketching and wireframing and to quickly hand over some solutions and deliverables.
We are not very comfortable with this approach and this is why:

Differences in formative and summative evaluations (and why they matter for UX designers)

May 12th, 2017 Posted by Analysis & Research, User Experience No Comment yet

A few weeks ago Steffi spoke at the Berlin Ladies that UX meetup “An evening about user research” about the different types of usability tests. Here is a brief summary of the talk

UX thoughts: Language matters!

April 20th, 2017 Posted by Analysis & Research, User Experience No Comment yet

Have you ever wondered if simply avoiding the term “test” in usability test situations may get you better results? We have and we think replacing the word “test” with the word “evaluation” is an improved description of the situation and will even help us to get less biased and therefore better and more accurate results!

As language determines perception, we think it is better to use the term „evaluation“ when it comes to usability testing situations instead the term „test“ – especially when interacting with our participants, because – simply put – a usability test is nothing else than an evaluation of a product/system with somebody’s help – namely our participants – who are or might be the users of the product/system.

So unlike a heuristic evaluation or cognitive walkthroughs which you will do by analyzing the product by yourself – a usability test is an empirical evaluation of a product/system with the help of people, who use or might use a specific system or product. “Empirical” here meaning: the knowledge or source of the knowledge is acquired by your senses, mostly by observation. In a usability test you will observe (and interview) representative users trying to accomplish certain important tasks as they are using the product.

We prefer the term „usability evaluation (with users)“ over “usability test” or even “user test”, simply because the term “test” is avoided and this little fact may help to reduce stress levels and therefore errors because people will feel less „observed“ and „tested“ during the test situation. “Evaluation” is also a more precise word in this situation because you are not actually testing peoples’ skills while they interact with the product, you are testing/evaluating a product with their help. That is a huge difference, also in the test participants’ perception.  Using the term “test” can result  in so called experimenter effects – like demand characteristics  which describe the tendency of participants to give you (as the experimenter) what you want based on what the participants think/guess what you might expect from them –  here: a test “result”. This may lead to  the participants focusing on how to  solve tasks error-free and to please. Such effects may put the complete validity of your usability test at risk.

In addition, we believe the term „evaluation“ will help people feel much more included because it prevents a „we“ vs „them“ situation, which can help to reduce further stress during the evaluation. We should always keep in mind that our participants help us and cooperate with us in evaluating a product/system and just using slightly different terms may help doing this better.

Image credit: (CC BY 2.0)

Are representative users important for usability testing?

March 24th, 2017 Posted by Analysis & Research, User Experience No Comment yet

YES, yes and yes – of course: Usability testing is a method in user research. You are evaluating a system or product you design by observing people who think aloud while they use that system/product and solve pre-defined tasks and so help us to spot problems with the system or product.

Recruiting the right participants is a critical, very important and often neglected task when planning usability tests. In best case scenarios, your test participants should also slightly differ within a specified user group. This doesn’t necessarily mean that this has to be totally expensive and is not feasible for smaller companies or organizations.

If you’re going to run usability tests with people who don’t represent your actual users, the problem is, that you might not see incidents or problems that your actual users will have – because of the differences between who you run your test with and the people who will actually be using your system: Think only of expertise – are your actual users technically more sophisticated or do they tend to be relatively new and inexperienced with technology? Or think of cultural backgrounds. All those criteria (plus of course more) will have influence on how people will use the system you’re designing.

Or as userfocus says: „Screen for behaviours, not demographics“

Of course you CAN test with people who doesn’t fit in your criteria and who aren’t representative for the usage of the system you want to test – but then you have to be absolutely aware of that fact, and write it in your report – plus: think about the fact that your findings might or might not apply to the actual people/population who will use your system. You might spot problems or discover ways of thinking and cognitive schemas and mental models that don’t match your actual users.

Even if recruiting the right people may sometimes be a challenging task, it’s absolutely worth the effort, because it gives you a lot more information about how well your system is going to be designed for people who will be actually using it.

Recruiting Better Research Participants
Writing the perfect participant screener
Recruiting Usability Test Participants

Image source:

UX Research Methoden – In welchen Fällen sind User Interviews eigentlich angebracht?

November 10th, 2015 Posted by Analysis & Research, User Experience No Comment yet

Im Designprozess kannst du verschiedene Methoden bzw. einen Methodenmix nutzen, um das Produkt, die Webseite oder die App „besser“ im Sinne von „user centered“ zu gestalten – sprich: du bekommt wertvolle Einblicke in die Welt der Nutzer.
Im UX Design bedienen wir uns gerne aus dem Methodenrepertoire, welches sich in den Sozialwissenschaften bewährt hat. Daher dachte ich mir, das wir mal ein paar Einblicke in einige Methoden geben. 🙂
Los gehts im ersten Teil mit einer bekannten und beliebten Methode – dem qualitativen User Interview – und der grundlegenden Frage: Wann macht ein solches überhaupt Sinn?


Intercultural Aspects of Interface Design

December 29th, 2014 Posted by Analysis & Research, User Experience No Comment yet

or: Know who you’re targeting.

In one of our recent projects we were asked to do culture research for a client’s website.
Goal was to optimize the client’s landing pages in terms of  design (& content) as  the page was to be rolled out in different countries with very different cultural backgrounds e.g. USA, China and Russia.

What is culture?

According to social psychologist Geert Hofstede, the term ‚culture’ can be defined as “(..) a shared set of values that influence societal perceptions, attitudes, preferences, and responses”.

In short, culture takes an effect on our perception, cognition and behaviour. When working on websites, interfaces or campaigns this means we need to consider the following things:

We love to write an speak about UX topics.

We have had many nation wide speaker appearances on many different topics. We have spoken at TEDx, UIG spring conference, Usability Congress, World Usability Day, webmondays and the up.front web design community in Berlin. We speak mostly about topics such as user experience and workflows. Have a date? Need speakers? Get in touch and let us know We might be interested in researching a topic