Quantitative vs. Qualitative UX Research Methods
UX research is a multi-dimensional process that includes different user research methods and techniques. In this chapter, we provide an overview of the quantitative and qualitative research methods and explain why the best solution uses a mix of both methods.
Quantitative UX research methods
Quantitative research is used to collect and analyze numerical data, identify patterns, make predictions, and generalize findings about a target audience or topic. The data is collected indirectly, either through a UX research tool that automatically records it, such as Google Analytics or Maze, or manually by measuring and analyzing UX metrics.
Here are some of the most popular quantitative research methods you can use to collect valuable quantitative data:
- Quantitative usability testing collects usability metrics like time on task, error rate, or success rate. You can use this information to keep an eye on your product's UX and make sure it improves over time.
- Web analytics (or app analytics) provides insights into what people actually do in your product. Analytics data can help you monitor your product's performance and identify problems.
- Card sorting is used to discover how people understand and categorize information. Analyzing the percentage of participants who grouped cards in a similar way can help you determine which categories would be understandable to most users.
- Surveys are a great way to gather information about your users' attitudes and behaviors. You can get qualitative data through open-ended feedback or quantitative data by tapping into a larger volume of responses.
Importance of quantitative research methods
Quantitative data provides a foundation for benchmarking and ROI calculations and can help you decide the best performing version of a design or product.
Quantitative UX researchers collect information by measuring actions, thoughts, or attitudes in different ways, such as conducting voluntary surveys and online polls or analyzing log data.
Duyen Mary Nguyen, Quantitative UX Researcher
Quantitative data aims to answer research questions such as ‘what,’ ‘where,’ or ‘when.’ For example, when collecting usability metrics such as task success rates, time on task, completion rates, clicks, conversion rates, and heatmaps, you can measure how well a design performs and spot issues on a page or in the user flow.
One of the advantages of quantitative research is the ability to run studies with large sample sizes and collect statistically relevant data. As opposed to qualitative feedback, which is interpretable by the researcher and subjective, quantitative research is more objective and representative of a broader audience.
I choose quantitative methods if I need to prioritize one solution over the possible alternatives or to validate an idea, wireframe, prototype or even MVP.
Qualitative UX research methods
Qualitative user research includes research methods like user interviews and field studies and helps you collect qualitative data through the direct observation and study of participants. Qualitative data yields an understanding of the motivations, thoughts, and attitudes of people. This type of research is key to uncovering the ‘why’ behind actions and develop a deep understanding of a topic or problem.
Yuliya Martinavichene, User Experience Researcher at Zinio, highlights: “Since researchers are curious folks, we prefer not only to observe what people are doing by looking at analytics but also to understand the “why” behind the user behavior.”
She compares the process of running a qualitative study to casting a wide lens to identify user behavioral patterns:
Qualitative research methods come into play when you need to discover, understand and empathize with users, and are not conducted only in the exploratory research phase, but iteratively, throughout the whole development process.
There are different qualitative research methods you can employ for your studies, such as user interviews, diary studies, focus groups, card sorting, usability testing, and more. We explore the most common UX research methods in the next chapter.
Choosing the right user research techniques depends on the project and your research goals. Yuliya explains:
In real-life, there is no “Oscar-winning” scenario and the best answer for the eternal question “What user experience research method should you use? is simply an unsatisfactory “It depends!” Different research pain points call for specific methods and approaches.
Yuliya collects qualitative feedback through different methods depending on the goals of the projects. For example, she might conduct walk-throughs with users and asks them to show her around the software she is researching to understand how they currently use the product. Or she may ask research participants to perform everyday tasks to observe their behavior in real-time, such as logging in or out of the platform.
To gather more qualitative insights, Yuliya also checks social media mentions, analyzes blog posts, and reads app store reviews to collect information about the experience users have with the product.
Qualitative research gives you rich insights about the people, product, and the problem you’re researching, and helps you inform decision-making throughout the design and product development process.
Balancing qualitative and quantitative UX research
Employ qualitative research to explore ideas and discover new insights, and then tap into quantitative research methods to test a hypothesis or final solution.
While qualitative and quantitative research yields different data types, they are both essential for conducting effective research and getting actionable insights. Not one method can give you a complete picture, so using both in combination is often the best way to ensure you’re making the right product decisions that fit with your business goals.
Qualitative and quantitative research reinforce each other and help to triangulate the research results. You can be surer of the validity of your findings if both qualitative and quantitative approaches produce convergent results.
Usually, the best solution is built using a combination of insight sources. For example, you can kick-off the discovery phase of a project with qualitative research, and run user interviews to understand people’s needs, preferences, and opinions.
After this initial batch of research studies, the product and design team can start building an incipient solution, usually in the form of a low-fidelity prototype or mockups. The initial solution is then tested through interviews and surveys, and the feedback gathered can help you iterate on the solution until final.
Sometimes you want to start with a round of qualitative methods such as interviews, fly-on-the-wall observations, and diary studies to explore the field and follow up with a quantitative study on a larger sample to generalize the results.
Lastly, when you’ve arrived at a final product, doing user testing quantitatively will help you ensure your solution is easy to use, usable, and intuitive for the end-users—and there are no significant issues with the design before going into the development phase. This mix and match of methods is the best way to research and test during the entire design process until arriving at a solution.
Very often, the solution is built on mixed methods–less quantitative versus qualitative–and more somewhere in-between the two.
In the next chapter, we will dive deeper into the most common types of research you can use such as tree testing, usability studies, or card sorting, and help you choose the right one for you.
Frequently asked questions
What is quantitative UX research?
What is quantitative UX research?
Quantitative research is a research methodology used to collect and analyze numerical data, identify patterns, make predictions, and generalize findings about a target audience.