user testing how many users blog cover

Conducting UX research with the correct number of testers ensures your UX insights are valid, reliable, and generalizable.

However, finding that sweet spot—the ideal number of participants—is often challenging. Too few participants may not reveal the full picture, while too many can lead to overwhelming amounts of data, wasted budget, and energy.

In this guide, we'll look at different UX research methods and what experts have to say about the number of participants needed for each test, including some well-known rules and myths about the number of user testers you need.

TL;DR 💡

The number of user testers you need depends on numerous factors, particularly your testing method and research goals. We suggest you test with at least 20 users to uncover most issues in your design.

The 5-user rule for user testing

The 5-user rule was first proposed by Jakob Nielsen, researcher and founder of NN Group, in his article ‘Why You Only Need to Test with 5 Users’ (2000), where he argued that five user testers will uncover 85% of usability problems. The article was based on his studies with Thomas K. Landauer in 1993.

For the Nielsen-Landauer formula to be true, the problem discovery rate—the chance that any individual user will find a specific usability issue—should be 31% (this is the L value in the formula).

This means that at least one user will discover a third of the usability problems. According to Nielsen, the 31% in the formula has been “averaged across many projects they studied”.

user testing diminishing returns chart

The curve demonstrating the effectiveness of additional testers shows a sharp increase in insights from zero to the first user, with each subsequent user adding progressively less new information.

The steepest part of the curve is at the beginning, where the first user will bring nearly a third of the usability insights. By the time you test with the fifth user, you've typically uncovered most usability issues, with additional users offer diminishing returns.

Why five users aren't always enough

Simple, right? Five users and you’re good to go! Not quite. The rest of Nielsen’s article highlights some crucial caveats which are often forgotten.

It's worth noting that the 5-user rule only applies to qualitative usability testing when you're observing participants completing tasks with the product, while recent guidance from NN Group suggests 40 users as recommendation for quantitative user testing. The 5-user rule is also only valid when the product has a relatively homogeneous target user group.

In the final section of the article titled ‘When to test more users’, a part overlooked by many adopters of the 5-user rule, Nielsen writes:

You need to test additional users when a website has several highly distinct groups of users. The formula only holds for comparable users who will be using the site in fairly similar ways.

Jakob Nielson, Founder of NN Group

Laura Faulkner, Head of AWS Computer UX Research at Amazon Web Services (AWS) led a study that revealed the 31% problem discovery rate is harder to predict than first thought. The study tested a group of 60 users and sampled random sets of five users or more to determine the percentage of usability problems found by each group.

The study showed that, as predicted by the Nielsen formula, the average of usability problems found was 85% in 100 tests of five users. However, that percentage ranged significantly across the spectrum.

Faulkner said: “Groups of 10 found 95% of the problems. Groups of five found as few as 55% of the problems, whereas no group of 20 found fewer than 95%.”

Five users can be a good starting point—and sometimes enough to provide valuable insights (depending on your method and test type)—but it shouldn't be thought of as a 'one size fits all' number that applies to every user test, every single time.

User testing: when to test with more than five users?

In practice, the decision to test with more than five users should be driven by the requirements of the project and the insights you need. Here’s when you should consider using more testers to uncover key insights:

  • Complex or diverse user bases: More users can provide a broader range of perspectives. If your product caters to a varied audience with differing needs and behaviors, testing with more users is crucial.
  • High-risk applications: In some products, errors can have serious consequences. Medical devices or financial services, for example, need more comprehensive testing for mitigating risks.
  • Iterative discoveries: Subsequent rounds of testing continue to reveal significant new issues? Expanding the user group can be beneficial for uncovering all insights.
  • Specific usability concerns: Some products have particular aspects of the design that are critical to the user experience—like navigation ease, accessibility features, checkout processes, or mobile responsiveness. If that's the case, testing with more users can ensure these elements function optimally for a diverse user base.
  • Post-release feedback: After launch, gathering data from a larger sample can help in refining the product based on real-world usage, like feature adoption rates, user engagement metrics, customer service inquiries, app store reviews, and website analytics. This diverse range of data points can inform continuous improvement efforts.

Recognizing when to increase your user testing pool is just the beginning. The next step is determining exactly how many participants you need to ensure your user testing delivers actionable, reliable results.

Build a robust tester database for rapid insights

Like a CRM for product research, Maze Reach helps you efficiently target diverse users and capture engagement insights

question-bank-3

How many users do you need for each research method?

The type of user testing you run often dictates the number of users you’ll need in your testers' panel.

We asked Matthieu Dixte, Product Researcher at Maze, for his thoughts on recruiting participants for each research method, and what the ‘magic number’ is.

Matthieu advised that participant numbers hinge a lot on your goal. “Firstly, consider whether you’re running a tactical study or a strategic study.”

  • A tactical study is one you conduct to make quick fixes and immediate improvements, meaning a smaller group will likely suffice
  • A strategic study aims to shape a product’s long-term direction, and may require a larger and more diverse group to provide insights

Here’s a quick overview of the number of participants you should aim to recruit for each type of study:

Research method Tactical study Strategic study
User interviews 3 (low-risk projects) 5 (medium-risk projects) 10-20
Moderated usability test 5 (uncover 80% of friction areas) 6-10
Unmoderated usability test 20+ 40
Early prototype concept validation 2-5 5-10
Surveys 20+ (low-risk projects) 50 (medium-risk projects) 100+
Card sorting 30 30+
Tree testing 50 50+
Focus group (A/B test) 5-8 5-8
Eye-tracking (A/B test) 40 50+

Usability testing

When it comes to usability testing, it's not about volume; it's about value. Matthieu's emphasis on iterative testing and continuous discovery aligns with the agile development philosophy—iterating quickly based on user feedback to refine the product continuously.

For moderated usability tests, where the researcher can interact with the participants, a smaller group is often sufficient to identify key issues. This way, you can delve deeper into individual user thought processes and behaviors.

For unmoderated tests, there is no direct interaction with users, so larger participant numbers are recommended to ensure a comprehensive data set. Here, the volume compensates for the lack of interaction, providing broader statistical insight into usability trends.

Matthieu recommends the ideal number of participants for moderated and unmoderated usability testing below:

Research method Tactical study Strategic study
Moderated usability test 5 (uncover 80% of friction areas, so it still needs to be completed with further users, moderated or unmoderated) 6-10
Unmoderated usability test 20+ 40

Surveys

User surveys can help you gather qualitative insights from users like their needs, motivations, and behaviors. The number of participants needed for a survey depends on project complexity, which reflects the potential impact of UX design decisions on the user experience and business outcomes.

Complex projects—characterized by multiple user types, high user engagement, and significant business implications—require a larger sample size to capture diverse perspectives and mitigate risk. Simpler projects with a narrower scope can rely on fewer participants, as the implications of potential oversights are less severe.

Matthieu recommends the following:

Research method Tactical study Strategic study
Surveys 20+ (low-risk projects) 50 (medium-risk projects) 100+

Prototype testing

Prototype testing lets you explore user reactions to your product’s early versions before full product development. The right time to test your prototypes is before launching new features.

If you’re using Maze, your team can import your clickable designs easily with the AdobeXD, Figma, Marvel, InVision, and Sketch integrations.

Matthieu recommends:

Research method Tactical study Strategic study
Prototype testing 2-5 participants 5-10 participants

Card sorting

Card sorting is a generative UX research method used to develop or evaluate the information architecture of a site. Unlike usability testing, card sorting requires a larger number of users to achieve reliable results, due to the variability in how people categorize information.

Opting for a midsize group of testers helps provide a detailed understanding of user categorization behaviors and vocabulary while remaining cost-effective.

Matthieu recommends:

Research method Tactical study Strategic study
Card sorting 30 participants 30+ participants

A/B testing

Unlike methods focused on user behavior and preferences, A/B testing relies on quantitative data to inform decisions. With A/B testing, your design team presents two variations of a design or prototype—to measure which one resonates better with users. There are also several testing methods to choose from for A/B tests.

This approach is versatile; you can apply it to early-stage concept validation or fine-tune a live feature. It’s particularly useful when deciding between two design paths, introducing new elements, or overhauling existing features.

Matthieu recommends the following:

A/B Testing Tactical study Strategic study
Focus group 5-8 5-8
Eye-tracking 40 50+

With Maze, you can run design comparison and create custom expected paths for each version using testing templates. You can also directly test a prototype for multiple variables, then simply share the test with recruited users via a link, collect their answers, and get an automated report with success rates and alternative paths.

Tree testing

Often done after card sorting, tree testing helps evaluate the navigability of a system structure.

The NN Group suggests at least 50 participants, but ideally, a sample size between 50 and 150 participants is recommended for tree testing. When we spoke with Matthew, he agreed:

“This range provides a reliable basis for understanding how users navigate through a site's hierarchy, allowing for a detailed analysis of the site or product structure's effectiveness​.”

Tree testing 50+

Other variables to consider when determining your sample size

While we've explored the need for more testers in specific user testing scenarios, let's take a look at some general factors that can help you decide how many testers to recruit for well-rounded, inclusive UX research:

  • Confidence level: This is the likelihood that your results accurately reflect the true experience of your entire user base. Essentially, it measures how certain you can be that the outcomes aren't down to chance. The industry standard for confidence level is between 90-95%. Aiming for a higher confidence level, closer to 95%, means you'll need a larger number of testers to ensure the data is robust. More testers increase the reliability of your results, but also impact the cost and time of testing.
  • Criticality of your project: Highly critical projects—like medical devices, security services, or key business platforms—demand more rigorous testing to prevent costly or harmful errors. For these, you may need to expand your pool of testers beyond standard practices to ensure the design meets the strictest criteria for usability, safety and security.
  • Type of data you're collecting: The type of data you're collecting—quantitative or qualitative—also dictates the number of users needed for testing. For example, a user interview session might involve five to 10 participants, providing rich, narrative data. On the other hand, card sorting to understand user mental models could require about 15 participants to reach a high correlation with a larger population's categorization patterns.

Why you shouldn't conduct user research with a single user

Conducting user research with just one user significantly narrows the scope of your insights and can lead to a limited understanding of your product's user experience.

  • You risk missing the diversity of user interactions and behaviors
  • You can't uncover patterns or commonalities in user experience issues
  • You receive limited feedback, potentially leading to biased improvements
  • You fail to represent the full spectrum of your user base
  • You neglect the possibility of contrasting opinions that can enrich the design process

Consider your research type alongside the 5-user rule and its limitations to determine exactly how many users you need for your user testing.

The secret to effective user testing

It’ll take a few tries to really nail the right number of participants for different user tests you’ll want to run. The right user testing tool with the right number of participants can provide clear metrics on user behavior, streamlined participant management, and richer data analysis to transform raw data into actionable UX insights more efficiently.

The golden rule is to stop when you no longer learn anything new.

Matthieu Dixte, Product Researcher at Maze

If you’re looking for a research companion to make participant recruitment easier and user testing more efficient, consider Maze. It’s designed for agile teams, and balances efficiency and time-to-insights with quality of testers.

Get insights in hours, not days

Maze Panel helps you recruit the right participants for uncovering decision-driving insights

Frequently asked questions about how many users you need for user testing

How many users do you need for user testing?

The number of users you need for user testing depends on the research method and the complexity of your product. For example, low-risk user interviews can involve only three participants, but tree testing requires around 50.

Can I test with only one user?

No, it’s not recommended to test with only one user. A single user can’t represent the diverse experiences and issues different users may encounter. For meaningful insights, consider involving a broader range of participants.

Build better products with powerful user research