The Geekettez. User Research & Experience Design Studio

Get in touch

Phone Berlin: 0171.12 45 07 3
Phone Mannheim: 0177.71 38 208

taz, die Tageszeitung

UX evaluation and testing: detecting problems before implementation.

Before the technical implementation, we conducted a combination of expert reviews and subsequent usability tests for the new taz app. We identified and improved critical issues during the prototyping phase and before a single line of code was written. This approach resulted in immense time and cost savings for taz.
Services: User Research
Link: iOS App
Link: Android App

“taz, die Tageszeitung” (one of Germany’s most influential alternative-left newspapers) continuously improves its native smartphone apps. We came on board to conduct an expert evaluation during the ongoing design process of the iOS and Android apps to prepare these for subsequent user tests. The results of these tests should steer the further product development.

The goal: Learning before failing

The overarching goal of the taz app is to provide a comprehensive user interface and functionality, supporting a positive digital reading experience and ensuring easy content accessibility. Which newspaper wouldn’t dream of achieving this goal?

As the paper had set a medium-term goal to replace the daily printed newspaper with a digital version, it faced the challenge of ensuring maximum user-friendliness in its apps. The focus was on quickly obtaining concrete results based on the designs to advance functionality and to understand how users cope with the navigation within the app. Ultimately the paper wanted to optimise the interaction and visual design before it went into development.

To achieve this, we conducted an expert review using a heuristic evaluation of the designs. We received insights on which we based a usability test using a click-dummy generated from the visual designs.

Opinionated and independent taz journalism in the taz App.

Opinion-strong and independent taz journalism in the taz app

Project Goals

  1. Identify problems before being coded
  2. Raise awareness among internal stakeholders for the needs of their users.
  3. Introduce user research methodologies within the company in a way that enables them to apply them to future design processes.

Our UX solutions for the taz app team

#1 Issue identification based on established criteria using heuristic evaluation (expert review)

Heuristic evaluation is a method in which one or more experts evaluate a system – in our case, the designs of an app – based on a pre-established checklist of criteria. We used DIN EN ISO 9241-10, Principles of Dialogue Design and Nielsen’s 10 Usability Heuristics as the basis of our evaluation. All well-established tools for evaluating the usability of interactive systems. They include fundamental guidelines of usability such as:

  • Task appropriateness (appropriate functionality, minimisation of unnecessary interactions)
  • Consistency and standards (adaptation to mental user models to the extent known)
  • Error prevention (system reacts tolerantly to errors or enables easy error correction by the user)

Performing a heuristic evaluation before usability testing can help identify and fix critical problems before testing with users. We also included problems causing internal disagreements as hypotheses in this first usability test.

Pop-up usability tests at the taz Cooperative Assembly

For an initial usability test of the taz app prototype, taz aimed to test the iOS prototype with existing readers at the Cooperative Assembly 2019 (annual general meeting), as the thematic orientation of the assembly also included presenting the app prototype.

We recruited five participants who were spontaneously approached and asked if they wanted to test the iOS prototype at the assembly. We recruited digital readers as well as paper newspaper readers. We aimed to determine if existing readers could navigate the app successfully from the start page and find the most important information and features.
We asked each participant to perform tasks while observing their interactions. We requested participants use the “thinking aloud” methodology to gain insights into their thought processes. After task completion, we followed up with a series of questions (post-test interview) to capture the participants’ perceptions and opinions. We also used hypothetical questions such as “If you had magical powers, what would you want from a digital newspaper?”

Our pop-up "test lab" at the taz Cooperative Assembly 2019

Our pop-up “test lab” at the taz Cooperative Assembly 2019

#2 Usability testing with uninitiated participants, but make it continuous to help reduce operational blindness

Next, we recommended a process which allowed continuous usability testing. We suggested that these tests should involve “external” potential readers/users who may not be familiar with the newspaper to gather insights that regular readers may not notice due to their expert status and familiarity with the paper and its concepts, potentially obscuring uncertainties or difficulties.

In the summer of 2020, we conducted numerous remote usability tests with potential readers further expanding our knowledge. We found that some concepts self-evident to loyal fans of the paper and internal staff were difficult to understand for new potential customers and therefore did not appear in the initial tests. The finding demonstrated the importance of including people in tests who belong to the targeted user group but are unfamiliar with the product or service (at least in the B2C scenario).

Testing digital reading in real-life conditions

We also asked where and in which scenarios people used the taz app. We suspected readers use the app in classic “waiting situations” (such as waiting at a bus or train station) and in-far-from-ideal reading conditions (i.e., busy or noisy surroundings with inadequate light, and an unstable surface). We, therefore, recommended “usability stress tests,” which could include testing under unfavourable lighting conditions (i.e., whether functions are visible or if the text is still readable in bright light) or under “shaky conditions,” such as on a subway train. These types of tests evaluate if touch targets can withstand such conditions. Discussing these scenarios can provide ample opportunities for reflection and improvement.

When recruiting participants, don’t forget about inclusivity

We knew from demographic surveys that taz readers and members were often in the middle to later stages of life. As the future of the paper envisioned replacing the daily printed newspaper with a purely digital version, we emphasised the importance of including older users in the tests who usually read their news on paper. We also recommended including readers with visual impairments or tremors.

#3 Enabling the team to run in-house tests with usability testing training

We then provided training on conducting in-house usability tests at taz. The goal of this training was to reduce the perceived barriers associated with user research (“It’s so time-consuming!”), sharpen the awareness for their readers and enable staff to conduct simple tests with (potential) users regularly.

|

>Lena Kaiser, Product Development taz App about the project

“Jenny and Steffi from The Geekettez have done something great for taz during a very important phase: With their help, we were able to place the needs of our readers at the centre of the development of the digital daily edition in the taz app. By combining expert review with subsequent usability tests using a click dummy, we identified critical points in the prototypes before we went into technical implementation. This not only saved time and costs but contributed to greater product satisfaction. In further tests in the live system, we could gather user feedback, making continuous optimisation of user-friendliness possible. As a result, we received a bronze ADC nail in the “newspaper redesign” category for the new taz app.”

Most challenging

When we came on board to conduct the expert review, several design rounds had already taken place without any UX input, meaning that much love and thought had already gone into the design. In such situations, it is usually tricky when an external expert/consultant comes in and questions or challenges the design – simply because of the amount of work and commitment already concluded.

It proved helpful to communicate well and diplomatically, to highlight the positive aspects in the expert reviews, and to involve designers and product managers as observers in the usability tests. This way, the entire team can personally witness the feedback from “real” users, which usually leads to a good reflection round.

Most fun

Conducting usability tests at a lively event such as the taz Cooperative Assembly was new for us. It was an engaging and enjoyable experience! The members of the taz collective were very open and curious about the digital newspaper, despite their age and contrary to stereotypes. Of course, there was some scepticism among the audience when the product developers presented the new taz app on stage, leading to one attendee – perhaps not entirely seriously – proclaiming that “if necessary, we will buy a DIN-A3 printer to print the taz every day ourselves”.

Case Study Image Lena Kaiser, taz, die tageszeitung

Interested in working with us?

We can support you in developing ideas and concepts and advise you on tricky UX challenges or frustrated users. Contact us and tell us more about your plans in a free introductory meeting.

More Case Studies