Chapter 6

Quantitative vs. qualitative UX research: An overview of UX research methods

UX research is a multi-dimensional process that includes different user research methods and techniques. In this chapter, we provide an overview of the quantitative and qualitative research methods and explain why the best solution uses a mix of both methods.

What are quantitative UX research methods?

Quantitative research is used to collect and analyze numerical data, identify patterns, make predictions, and generalize findings about a target audience or topic. The data is collected indirectly, either through a UX research tool that automatically records it, such as Google Analytics or Maze, or manually by measuring and analyzing UX metrics.

Qualitative UX research made easy

Explore the powers of both quantitative and qualitative research to discover new insights and test final solutions.

Here are some of the most popular quantitative research methods you can use to collect valuable quantitative data:

  • Quantitative usability testing collects usability metrics like time on task, error rate, or success rate. You can use this information to keep an eye on your product's UX and make sure it improves over time.
  • Web analytics (or app analytics) provides insights into what people actually do in your product. Analytics data can help you monitor your product's performance and identify problems.
  • Card sorting is used to discover how people understand and categorize information. Analyzing the percentage of participants who grouped cards in a similar way can help you determine which categories would be understandable to most users.
  • Surveys are a great way to gather information about your users' attitudes and behaviors. You can get qualitative data through open-ended feedback or quantitative data by tapping into a larger volume of responses.

Importance of quantitative research methods

Quantitative data provides a foundation for benchmarking and ROI calculations and can help you decide the best performing version of a design or product.

Quantitative UX researchers collect information by measuring actions, thoughts, or attitudes in different ways, such as conducting voluntary surveys and online polls or analyzing log data.

Duyen Mary Nguyen, Quantitative UX Researcher

Quantitative data aims to answer research questions such as ‘what,’ ‘where,’ or ‘when.’ For example, when collecting usability metrics such as task success rates, time on task, completion rates, clicks, conversion rates, and heatmaps, you can measure how well a design performs and spot issues on a page or in the user flow.

One of the advantages of quantitative research is the ability to run studies with large sample sizes and collect statistically relevant data. As opposed to qualitative feedback, which is interpretable by the researcher and subjective, quantitative research is more objective and representative of a broader audience.

I choose quantitative methods if I need to prioritize one solution over the possible alternatives or to validate an idea, wireframe, prototype or even MVP.

Yuliya Martinavichene, User Experience Researcher at Zinio

What are qualitative UX research methods?

Qualitative user research includes research methods like user interviews and field studies and helps you collect qualitative data through the direct observation and study of participants. Qualitative data yields an understanding of the motivations, thoughts, and attitudes of people. This type of research is key to uncovering the ‘why’ behind actions and develop a deep understanding of a topic or problem.

Yuliya Martinavichene, User Experience Researcher at Zinio, highlights: “Since researchers are curious folks, we prefer not only to observe what people are doing by looking at analytics but also to understand the “why” behind the user behavior.”

She compares the process of running a qualitative study to casting a wide lens to identify user behavioral patterns:

Qualitative research methods come into play when you need to discover, understand and empathize with users, and are not conducted only in the exploratory research phase, but iteratively, throughout the whole development process.

Yuliya Martinavichene, User Experience Researcher at Zinio

There are different qualitative research methods you can employ for your studies, such as user interviews, diary studies, focus groups, card sorting, usability testing, and more. We explore the most common UX research methods in the next chapter.

Choosing the right user research techniques depends on the project and your research goals. Yuliya explains:

In real-life, there is no “Oscar-winning” scenario and the best answer for the eternal question “What user experience research method should you use? is simply an unsatisfactory “It depends!” Different research pain points call for specific methods and approaches.

Yuliya Martinavichene, User Experience Researcher at Zinio

Yuliya collects qualitative feedback through different methods depending on the goals of the projects. For example, she might conduct walk-throughs with users and asks them to show her around the software she is researching to understand how they currently use the product. Or she may ask research participants to perform everyday tasks to observe their behavior in real-time, such as logging in or out of the platform.

To gather more qualitative insights, Yuliya also checks social media mentions, analyzes blog posts, and reads app store reviews to collect information about the experience users have with the product.

Qualitative research gives you rich insights about the people, product, and the problem you’re researching, and helps you inform decision-making throughout the design and product development process.

Quantitative vs. qualitative research methods

The key differences between quantitative and qualitative research are in the data they deal with and the questions they answer–where quantitative research focuses on numbers and statistics to answer ‘what’, ‘where’ and ‘when’, qualitative research broadly looks to words and meaning for the ‘why’.

Both methods have their merits, and likewise their drawbacks. As we go on to explore, for the most robust and meaningful research, it’s best to use a combination of quantitative and qualitative, but in certain situations, such as challenges due to time or resource constraints, you may decide to use one or the other.

Quantitative research:

  • Answers the questions ‘what’, ‘where’ and ‘when’
  • Provides a foundation for benchmarking and ROI calculations
  • Allows for large sample sizes
  • Analyzes numerical data, identifies patterns, makes predictions
  • Collected indirectly through UX research tools or metrics

Qualitative research:

  • Answers the question ‘why’
  • Provides rich insights about the people, the product and the problem
  • Allows tight focus on small sample sizes
  • Develops a deep understanding of the topic or problem
  • Collected through direct observation or study

Balancing qualitative and quantitative UX research

Tip ✨

Employ qualitative research to explore ideas and discover new insights, and then tap into quantitative research methods to test a hypothesis or final solution.

While qualitative and quantitative research yields different data types, they are both essential for conducting effective research and getting actionable insights. Not one method can give you a complete picture, so using both in combination is often the best way to ensure you’re making the right product decisions that fit with your business goals.

Qualitative and quantitative research reinforce each other and help to triangulate the research results. You can be surer of the validity of your findings if both qualitative and quantitative approaches produce convergent results.

Yuliya Martinavichene, User Experience Researcher at Zinio

Usually, the best solution is built using a combination of insight sources. For example, you can kick-off the discovery phase of a project with qualitative research, and run user interviews to understand people’s needs, preferences, and opinions.

After this initial batch of research studies, the product and design team can start building an incipient solution, usually in the form of a low-fidelity prototype or mockups. The initial solution is then tested through interviews and surveys, and the feedback gathered can help you iterate on the solution until final.

Sometimes you want to start with a round of qualitative methods such as interviews, fly-on-the-wall observations, and diary studies to explore the field and follow up with a quantitative study on a larger sample to generalize the results.

Yuliya Martinavichene, User Experience Researcher at Zinio

Lastly, when you’ve arrived at a final product, doing user testing quantitatively will help you ensure your solution is easy to use, usable, and intuitive for the end-users—and there are no significant issues with the design before going into the development phase. This mix and match of methods is the best way to research and test during the entire design process until arriving at a solution.

Very often, the solution is built on mixed methods–less quantitative versus qualitative–and more somewhere in-between the two.

Yuliya Martinavichene, User Experience Researcher at Zinio

In the next chapter, we will dive deeper into common types of research you can use such as tree testing, card sorting, and usability studies, and help you choose the right one for you.

Frequently asked questions

What is quantitative UX research?

Quantitative research is a research methodology used to collect and analyze numerical data, identify patterns, make predictions, and generalize findings about a target audience.

What is qualitative UX research?

Qualitative UX research is a research methodology used to answer questions and understand the motivations, thoughts, and attitudes of a target audience.

What are examples of quantitative research methods?

Quantitative user research methods include usability testing, web analytics (or app analytics), card sorting, and surveys.

What are examples of qualitative research methods?

Qualitative user research methods include user interviews, diary studies, focus groups, card sorting, and usability testing.