How to analyze and report usability test results

In this chapter, you'll learn how to analyze and report on usability test results, what to include in your usability testing report to make sense of the data collected, and devise a plan of action.

You probably have multiple data points by now. The first step to understanding them is to organize the issues you recorded during usability testing.

The most straightforward method is to write down every single issue along with a comprehensive description of where in the design it happened, how it occurred, and the task the user was performing, plus all of the details that describe the problem.

Alternatively, you can arrange the data in a table format according to one of the methods presented by Lewis and Sauro in Quantifying the User Experience: Practical Statistics for User Research. On one side, include each task with its respective issue. On the other, include a list of all your usability test participants so you can mark down every user who’s experienced the same issue.

Not all issues discovered during usability testing are equally important. You need to categorize the problems based on the severity of the issues identified. Ranking your findings will make them more actionable and help you classify the issues you need to handle according to their priority.

After this exercise, you will know which ones to prioritize based on criticality and your team’s resources. For instance, a usability issue that makes it hard for users to find the call-to-action on the page might be a more critical issue than a typo.

According to NNGroup, we can categorize usability issues based on five severity levels:

Examples: The user didn’t receive a confirmation message after making a payment or they can’t sign up for an account.

Examples: The user can’t navigate to the next page in their flow or can’t reset their password.

Examples: The user has to scroll a lot to find a category in a dropdown or the text is too small on the Pricing page

Examples: a misspelled word in the heading or a logo that isn’t updated on the About us page.

The fundamental goal of usability testing is to build a product that your customers will be able to use. That’s why involving different team members from the product team in the process of usability testing is important. Every team member will benefit from understanding the issues users experience and help make decisions to improve them.

At the end of the usability testing process, you should create a final report to share your insights and findings with the rest of your team. A usability testing report should be easy-to-understand and actionable, helping you and your team plan future design iterations with clear next steps.

A usability testing report displays the findings in a structured way that helps the team come up with action plans, see what features work well, and most importantly, prioritize the issues that need to be fixed. Here’s what you should include in your usability test report.

First, include an overview of the usability test you ran. Here, describe the design you tested, the location of the tests (in-person vs. remote), the usability testing tools you used, the time and date of the tests, information about the moderator (if any), etc. This provides a concise overlook at the tests to anyone who wasn’t part of the process including other team members and stakeholders.

Next, mention the main goals behind your usability testing in a few sentences. While the general goal of usability testing remains to test and assess how usable, intuitive and easy to use a design is, here are five examples of specific objectives you should opt for:

In this section, mention the different usability testing methods you used to run the tests. This includes information about the type of data collected (qualitative vs. quantitative) and insights on the types of tests that were conducted. For the latter aspect, specify if you turned to unmoderated or moderated usability testing and the location where tests were conducted (in-person vs. remote).

Next, share the methods used to select the participants for your test. Here, focus on sharing screening information on how participants were selected, the number of user testing participants, and why target users were selected.

Here, provide demographic information about each test participant who took part in the tests, including their age, profession, experience with similar technologies, and any other relevant data based on the target audience you defined in the testing plan.

A recommended best practice is to make sure you don’t mention their names to guarantee respondent privacy. Fictional names can be used instead to talk about a scenario or group of people.

This section is the core of your usability testing report as it includes all the findings identified after you’ve analyzed your usability test results. We’ve already learned how to analyze the results based on the severity of the issues above. Once you prioritize the issues into categories, include the most critical findings in the report.

Here, you can complement the main findings with usability metrics such as completion rates, time spent, or bounce rates for individual tasks. This data will help you back-up the findings, and make it easy for stakeholders to understand the problems. You can also highlight user comments or satisfaction rates to provide relevant context around an issue.

If relevant, include recommendations for things that need to be changed. Ideally, those will be discussed and planned with members from your product team and implemented in subsequent design iterations.

Once you finish compiling all findings into a usability test report and discussing the results with your team, planning the next steps is the next important part of any usability testing. No report is useful unless the results are acted upon.

Analyzing the report’s findings and coming up with new solutions to tackle the issues identified is the responsibility of the entire product team—whether that involves discussing the feedback or retesting a design for new results. Try to keep your usability testing report short and the findings actionable, and focus on delivering something your team can act upon.