To identify usability issues and evaluate the accessibility of the dashboard, and establish how people find interaction with the graphs, we conducted 2-weekly usability testing sessions in collaboration with an external research agency. We specifically wanted to know how easy the dashboard was to use and interpret. What are some of the most important lessons we learned about data communication from conducting usability tests on the Dutch COVID-19 dashboard?

A graph is only a part of a story

After the outbreak, there was a lot of public debate on the COVID-19 data. People wanted to know the number of positive cases, hospital admissions, but also what insights they could take from that data.

We conducted usability testing with participants from different backgrounds: workers in medical and non-medical fields, people with different educational backgrounds, and also younger and older people. The results lead to an understanding that a number on itself was not enough to understand the situation. People had many questions arising from looking at the numbers: What does this number actually convey? How is it collected? Is it trustworthy? Answers to these questions were necessary to understand the real data in the graph. Contextual information was the first step to understanding and interpreting the data correctly and also seeing how it fits into the bigger picture.

To solve this, we added explanations of the definitions and included articles introducing the broader topic:

Incorporating articles and useful links to complement the data

Multiple ways to present the same number

During usability testing, common feedback was that displaying data in the format of ‘cases per 100.000’, which is commonly used in science, was challenging to comprehend. While people are more familiar with percentages, these numbers (e.g. infections, hospital admissions) were too low to use this as standard practice. An alternative was to use absolute numbers, yet these didn’t provide the opportunity to compare municipalities, for example. What was the way to solve this? In the beginning, we thought we should stick to one way of showing numbers to prevent clunkiness on the page.

However, in this case, there was no single solution and we took a more flexible approach. On the dashboard, we always showed metrics in multiple ways, to make them accessible to both, scientific and broader audiences.

Showing metrics in different ways

Jumping straight to numbers

To help understand the data, we added introductions to the graphs. However, these were consistently neglected. Users wanted to read data first and later they did not return to read explanations. We then provided more cues and hints throughout the platform that guided users toward understanding the data:

Tooltips for difficult terms

Additional contextual explanations

Summary

Mostly, our journey taught us that the COVID-19 dashboard was more than just a central place to publish numbers on the outbreak. It was essential to add multiple layers to it to make sure it was accessible to a wide audience.

People process numbers in different ways, therefore it helps to show both numbers and percentages, provide introductions, visualize numbers and include images, connect big numbers to the personal context, and add stories that show the overall context of where a number comes from and what it means. Using these methods, we provide needed context in multiple ways and this helps people to interpret data better.

In order to create a truly powerful tool that is data-heavy but also understandable to broad audiences, it’s necessary to observe and listen to how people interpret graphs. This way, dashboards can become fundamental in decision-making and support behavior change.