The magic number for user testing: How many users do you need?

Conducting UX research with the correct number of testers ensures your UX insights are valid, reliable, and generalizable.

However, finding that sweet spot—the ideal number of participants—is often challenging. Too few participants may not reveal the full picture, while too many can lead to overwhelming amounts of data, wasted budget, and energy.

In this guide, we'll look at different UX research methods and what experts have to say about the number of participants needed for each test, including some well-known rules and myths about the number of user testers you need.

The 5-user rule was first proposed by Jakob Nielsen, researcher and founder of NN Group, in his article ‘Why You Only Need to Test with 5 Users’ (2000), where he argued that five user testers will uncover 85% of usability problems. The article was based on his studies with Thomas K. Landauer in 1993.

For the Nielsen-Landauer formula to be true, the problem discovery rate—the chance that any individual user will find a specific usability issue—should be 31% (this is the L value in the formula).

This means that at least one user will discover a third of the usability problems. According to Nielsen, the 31% in the formula has been “averaged across many projects they studied”.

The curve demonstrating the effectiveness of additional testers shows a sharp increase in insights from zero to the first user, with each subsequent user adding progressively less new information.

The steepest part of the curve is at the beginning, where the first user will bring nearly a third of the usability insights. By the time you test with the fifth user, you've typically uncovered most usability issues, with additional users offer diminishing returns.

Simple, right? Five users and you’re good to go! Not quite. The rest of Nielsen’s article highlights some crucial caveats which are often forgotten.

It's worth noting that the 5-user rule only applies to qualitative usability testing when you're observing participants completing tasks with the product, while recent guidance from NN Group suggests 40 users as recommendation for quantitative user testing. The 5-user rule is also only valid when the product has a relatively homogeneous target user group.

In the final section of the article titled ‘When to test more users’, a part overlooked by many adopters of the 5-user rule, Nielsen writes:

Laura Faulkner, Head of AWS Computer UX Research at Amazon Web Services (AWS) led a study that revealed the 31% problem discovery rate is harder to predict than first thought. The study tested a group of 60 users and sampled random sets of five users or more to determine the percentage of usability problems found by each group.

The study showed that, as predicted by the Nielsen formula, the average of usability problems found was 85% in 100 tests of five users. However, that percentage ranged significantly across the spectrum.

Faulkner said: “Groups of 10 found 95% of the problems. Groups of five found as few as 55% of the problems, whereas no group of 20 found fewer than 95%.”

Five users can be a good starting point—and sometimes enough to provide valuable insights (depending on your method and test type)—but it shouldn't be thought of as a 'one size fits all' number that applies to every user test, every single time.

In practice, the decision to test with more than five users should be driven by the requirements of the project and the insights you need. Here’s when you should consider using more testers to uncover key insights:

Recognizing when to increase your user testing pool is just the beginning. The next step is determining exactly how many participants you need to ensure your user testing delivers actionable, reliable results.

Discover this by using our handy calculator below 👇

The type of user testing you run often dictates the number of users you’ll need in your testers' panel.

We asked Matthieu Dixte, Product Researcher at Maze, for his thoughts on recruiting participants for each research method, and what the ‘magic number’ is.

Matthieu advised that participant numbers hinge a lot on your goal. “Firstly, consider whether you’re running a tactical study or a strategic study.”

Here’s a quick overview of the number of participants you should aim to recruit for each type of study:

When it comes to usability testing, it's not about volume; it's about value. Matthieu's emphasis on iterative testing and continuous discovery aligns with the agile development philosophy—iterating quickly based on user feedback to refine the product continuously.

For moderated usability tests, where the researcher can interact with the participants, a smaller group is often sufficient to identify key issues. This way, you can delve deeper into individual user thought processes and behaviors.

For unmoderated tests, there is no direct interaction with users, so larger participant numbers are recommended to ensure a comprehensive data set. Here, the volume compensates for the lack of interaction, providing broader statistical insight into usability trends.

Matthieu recommends the ideal number of participants for moderated and unmoderated usability testing below:

User surveys can help you gather qualitative insights from users like their needs, motivations, and behaviors. The number of participants needed for a survey depends on project complexity, which reflects the potential impact of UX design decisions on the user experience and business outcomes.

Complex projects—characterized by multiple user types, high user engagement, and significant business implications—require a larger sample size to capture diverse perspectives and mitigate risk. Simpler projects with a narrower scope can rely on fewer participants, as the implications of potential oversights are less severe.

Matthieu recommends the following:

Prototype testing lets you explore user reactions to your product’s early versions before full product development. The right time to test your prototypes is before launching new features.

If you’re using Maze, your team can import your clickable designs easily with the AdobeXD, Figma, Marvel, InVision, and Sketch integrations.

Matthieu recommends:

Card sorting is a generative UX research method used to develop or evaluate the information architecture of a site. Unlike usability testing, card sorting requires a larger number of users to achieve reliable results, due to the variability in how people categorize information.

Opting for a midsize group of testers helps provide a detailed understanding of user categorization behaviors and vocabulary while remaining cost-effective.

Matthieu recommends:

Unlike methods focused on user behavior and preferences, A/B testing relies on quantitative data to inform decisions. With A/B testing, your design team presents two variations of a design or prototype—to measure which one resonates better with users. There are also several testing methods to choose from for A/B tests.

This approach is versatile; you can apply it to early-stage concept validation or fine-tune a live feature. It’s particularly useful when deciding between two design paths, introducing new elements, or overhauling existing features.

Matthieu recommends the following:

With Maze, you can run design comparison and create custom expected paths for each version using testing templates. You can also directly test a prototype for multiple variables, then simply share the test with recruited users via a link, collect their answers, and get an automated report with success rates and alternative paths.

Often done after card sorting, tree testing helps evaluate the navigability of a system structure.

The NN Group suggests at least 50 participants, but ideally, a sample size between 50 and 150 participants is recommended for tree testing. When we spoke with Matthew, he agreed:

“This range provides a reliable basis for understanding how users navigate through a site's hierarchy, allowing for a detailed analysis of the site or product structure's effectiveness​.”

While we've explored the need for more testers in specific user testing scenarios, let's take a look at some general factors that can help you decide how many testers to recruit for well-rounded, inclusive UX research:

Conducting user research with just one user significantly narrows the scope of your insights and can lead to a limited understanding of your product's user experience.

Consider your research type alongside the 5-user rule and its limitations to determine exactly how many users you need for your user testing.

It’ll take a few tries to really nail the right number of participants for different user tests you’ll want to run. The right user testing tool with the right number of participants can provide clear metrics on user behavior, streamlined participant management, and richer data analysis to transform raw data into actionable UX insights more efficiently.

If you’re looking for a research companion to make participant recruitment easier and user testing more efficient, consider Maze. It’s designed for agile teams, and balances efficiency and time-to-insights with quality of testers.