A Beginner's Guide to Usability Testing

Usability testing is a proven method to evaluate your product with real people. In this complete guide to usability testing, we share everything you need to know to run usability tests and get actionable insights to create better user experiences.

Learn by doing, start usability testing now

Maze generates instant, immediately-actionable reports, including detailed survey and user flow analytics.

Chapter 1

What is usability testing?

Let’s start right away with a definition of usability testing. Usability testing is the process of testing your product with real people by getting them to complete a list of tasks while observing and noting their interactions. The goal of conducting usability testing is to understand if your design is usable and intuitive enough for users to accomplish their goals. 

Good usability testing takes a holistic look at the potential audience that you can serve and the range of intended and unintended uses of a product.

Behzod Sirjani, Founder of Yet Another Studio

However, the benefits of usability testing extend beyond testing functionality and usability. More than that, it helps you really grasp the use cases of your product, understand your audience, and build a good product.

Behzod Sirjani, founder of Yet Another Studio and former Head of Research & Analytics Operations at Slack, explains that "Good usability testing takes a holistic look at the potential audience that you can serve and the ranges of intended and unintended uses of a product to understand how you're enabling people to do what they want with your product while preventing misuse."

A brief history of usability

Before we dive into usability testing specifics, let's define what usability means in the context of the human-centered approach to design. Human-centered design is a creative approach to problem-solving, established and developed by IDEO, which places the user front and center in the design process.

The approach combines quantitative and qualitative UX research methods for developing products, such as ideating, rapid prototyping, the jobs-to-be-done framework, and more. Usability testing is one of those methods and a key principle for building user-centric products.

No matter how much time you spend on ideating and prototyping your design solution, you should always test it with real people. The feedback from the testing session will help you understand what part of your design requires improvements.

Nick Babich, Editor-in-chief of UX Planet

The modern approach to usability testing started in the ’80s as work and personal computers became more prevalent. At that point, most people had very limited exposure to computers, so getting familiar with new technology was usually a very long and frustrating process.

To make sure computers were usable and intuitive, programmers needed to understand users' thought processes for completing a task. In 1981, two researchers–Karl Ericsson and Herbert Simon–published an article called Verbal Reports as Data, which popularized the “think aloud” method in which the participant literally speaks their thought process out loud.

More than a decade later, the ISO 9241 definition of usability—published in 1998—defined usability as “the effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments.”

These developments meant that instead of guessing how a product is used by people, you could observe them complete tasks, learn, and improve usability problems in real-time. That’s where usability testing comes in. It gives you a direct look into what the participant is thinking and how they use your product.

What usability testing is not

Usability testing is not simply gathering information or opinions about a product—that's marketing or user research. It's not even showing users a rough draft of the product and asking them if they know how to use it.

Unlike generative research—which is used to get a deeper understanding of users' goals, motivations, and behaviors—usability testing is the process of testing your product with real people by getting them to complete a list of tasks while observing and noting their interactions.

There are other UX research methods and techniques to test user experience but don't qualify as usability testing:

  • A/B Testing: this is a method of testing design variations to determine which changes improve performance. While usability testing observes and investigates user behavior, A/B testing can't tell you why a version is better than the other.
  • Surveys: you can use surveys to gauge user experience or as part of a usability test, but as they don't allow you to observe users' actions in real-time, they're not actually usability testing.
  • Focus groups: during a focus group, a moderator asks a group of people about their behaviors, preferences, and experiences on a specific topic. With focus groups, the goal is to gather opinions about a product, not to test how people use it.

The benefits of usability testing

Usability testing is important because as much as we like to believe that we are building products for ourselves, each person is unique. We’ve all had unique backgrounds, lived experiences, perspectives, preferences, and abilities. All of these things influence how we understand, approach, and experience a product.

Behzod Sirjani, Founder of Yet Another Studio

Have you ever noticed on cooking shows–when possible–chefs have others taste components of a dish while they’re still cooking? Maybe it’s a sauce or some other ingredient. Their counterpart might say, “needs more salt” or “a little too spicy.” What they’re doing is running a very small-scale user test.

When you’re too close to something–like a chef may be to a dish–it’s very easy to miss things. We tend to overestimate how likely someone else is to share in our opinion. The psychological term for the phenomenon is the false-consensus effect.

The only way to combat false-consensus effects when designing is with usability testing. This allows you to get another point of view, and most importantly, a perspective from the actual users who’ll be using your product in real life.

Usability testing is also relevant for long-term customer success. A broader perspective means you can design better for your audience, and make sure more people can be successful with your product. Forbes reports that companies investing in UX have a lower customer acquisition cost, lower support cost, and higher customer retention rates.

Usability testing is one of the most fundamental ways to test the success of a design. Ensuring customers can find, understand, and use a solution contributes to the overall success of a product and business.

Taylor Palmer, Product Design Lead at Range

To give a more concrete example, Shopify reports that one of the most common reasons a customer abandons a shopping cart is due to a complicated check-out process. When you employ usability testing during the design process, you can find and fix usability issues and complex flows before you launch your product, and ensure the user experience is up to standard.

Product tip ✨

Maze allows you to test your prototype with users and get actionable data that helps you make informed design decisions. Learn more in the complete guide to using Maze.

When should you do usability testing?

There are multiple times when you and your team should run usability tests during the design process. In most cases, it’s something you should do more than once, and early and often in the process to inform your design decisions.

Usability testing often happens towards the end of the product development process and is seen as a way to “catch bugs” in the product. That’s not usability testing, that’s QA. Usability testing needs to happen at a point where you can not just learn how usable your product is, but when you can take those learnings and make improvements so that your product is more usable.

Behzod Sirjani, Founder of Yet Another Studio

Here are some general guidelines you can follow when deciding when to do usability tests and creating your test plan.

Before you start designing

If you want to start a redesign, chances are there’s something about your current site that’s not working. You can run a usability test before you start any redesign to discover users’ pain points and bottlenecks with the current design. If you have that data as a jumping-off point, you will be able to benchmark your new design against the existing one to make sure it’s a better version.

Additionally, one of the best times to start testing a product is in the product-discovery phase. If you’re designing a new product from scratch, you can do usability testing with competitor products to observe how people use similar tools and services in real life and understand their mental models when using products like yours. The test results can be a benchmark for your UX design when the time comes to test it.

Once you have a wireframe or prototype

With the initial research phase completed, you can start in on your new design. It’s important to test your designs as you go through the iterative design process. You can start testing as soon as you have your initial ideas sketched in a low-fidelity prototype or even a paper mock-up.

I think usability testing is applicable at every stage of the design process. I try to test at an early concept phase when I have a low fidelity wireframe to get some early input from real users. Once we get the first results, it’s important to use those results throughout other design phases and re-test with users.

Vaida Pakulyte, UX researcher and Designer at Electrolux

This type of early testing is called formative testing, and it helps you test and validate ideas as you move through the process, so you don’t get to the end without any user feedback.

For this stage, you might want to do card sorting or tree testing to test your website or app's information architecture. Generally, if you’re starting from scratch, card sorting is recommended first to understand how your target audience groups and associates information. That, followed by a round of tree testing, ensures that the content on your website or app is easy to find, and the web pages follow an intuitive flow.

Product tip ✨

If you have a Figma, InVision, Sketch, or Marvel prototype ready, you can import it into Maze and start testing your design with users immediately. Get started for free.

Prior to the launch of the product

Once you’re at the end of the development process, you’ll probably have a fully-sketched high-fidelity prototype. That’s where summative testing comes in. Summative testing is a usability evaluation you do at the final stage to prevent any major issues and test if the end-users can complete tasks with the user interface.

When running summative testing, you should aim for a large sample size to get statistically significant results from a large pool of test participants. Usually, the test results can be benchmarked against any data you collected throughout the testing process, so ease of use and user satisfaction can also be measured before launch.

At regular intervals after launch

Just as products aren’t static, neither are user bases. Since they’ll also continue to change and evolve, it’s good to run test sessions every so often to make sure everything is still working as intended. Consider regularly checking after launch to make sure your product stays in tip-top shape, and your actual users aren't experiencing any issues.

5 best practices for usability testing

This section was written by Behzod Sirjani, founder of Yet Another Studio and former Head of Research & Analytics Operations at Slack.

There are lots of things to keep in mind when you are usability testing. That said, here are some of the most important ones:

1. Get consent

You should get consent at two separate points during usability testing from your test participants. First, at the beginning of the test and then at the end. Here's why. At the beginning of the test, you should ask for their consent to record the test and the results. Then at the end of the test, you should ask for their consent to keep the test results and use the data. It's important to ask for consent twice because a participant only has a loose idea of what they are consenting to in the beginning.

Being intentional and inclusive in your testing audience will help you build better products because you will learn from people who are different than you and who see and do things that you do not.

Behzod Sirjani, Founder of Yet Another Studio

2. Be inclusive

You should be intentional about the people you want to participate in your usability test, and you should make sure to include people who may have different perspectives on your product. This includes people of different abilities, demographics, usage behaviors, market segments, etc. Being intentional and inclusive in your testing audience will help you build better products because you will learn from people who are different than you and who see and do things that you do not.

3. Run a pilot test

Before you ask for any of your customers or potential customers to spend their time giving you feedback, make sure to pilot your usability test with someone inside your organization, ideally someone not on your team. Among other things, this allows you to identify aspects of the study or product that are obvious to you, but that may be missing from your study experience.

4. Establish evaluation criteria

You should know what success looks like for your product testing and have clear ways to identify if and when people were (not) successful. Bonus points if you have easy ways to keep track of that in your notes.

5. Be mindful of length

Your usability test should be only as long as it needs to be for you to have confidence in your results. People do not exist to give you product feedback, and very few people want to spend all day evaluating the usability of your product. If you worry that your test is too long, pilot it and gather feedback. It’s much better to run multiple separate tests than to gather feedback from participants who are cognitively exhausted because you’re asking too much of them.

In the next chapters, we'll learn more about the different usability testing methods and about the different types of usability testing, such as in-person user testing versus remote usability testing, or unmoderated versus moderated sessions.

Start usability testing now

Learn how to create a user testing project in Maze to test your prototype with users and get actionable insights fast. Read more.

Frequently asked questions about usability testing

How do you perform a usability test?

If you want to run a successful usability test, you should create a usability testing script. Start by clearly outlining the goal of your research, the type and number of participants, and the specific tasks you want them to complete. If you are looking for more information or examples, check out "How to write an effective usability testing script."

What are the main characteristics of usability testing?

Five main characteristics define usability testing: the primary goal is to improve the usability of a product, the participants represent real users, the participants do real tasks, the researchers observe and record the participant's activities, the researchers analyze the findings, diagnose problems, and recommend changes.

How do you write a usability testing report?

At the end of the usability testing process, you should create a final report to share your results with the rest of the team. The report should be easy-to-understand and well structured, helping you and your team to see what works well, prioritize what needs to be fixed, and plan the next steps. Learn more about how to analyze and report usability test results in the final chapter of this guide.