Mastering user research analysis: A comprehensive guide to qualitative and quantitative methods

User research analysis is the process of distilling data into actionable insights. It’s the crucial last step in the research process that uncovers meaning from your results.

In this deep dive, we explore when to analyze your user research data, which methods to use depending on your study, and best practices to keep your results reliable.

🤔 Before we begin
Keep in mind that while user research and UX research are often used interchangeably, they’re two different concepts.

TL;DR: User research looks at who users are broadly, while UX research drills down into users from a product and user experience perspective—it’s about analyzing user behavior in relation to your product. All UX research is a form of user research. However, not all user research is UX research.

In this article, we’ll be focusing on UX research analysis methods for product development. Many of these techniques also apply to general user research, but we’re keeping our focus on their application regarding UX research data.

If we’re being literal, UX research analysis begins after you collect data. But from a strategic perspective, UX research analysis starts when your project starts.

Here’s when you should be thinking about user research analysis:

Keeping the analysis in mind throughout the UX research process ensures you choose the right UX research methods, and can pivot and adapt where necessary as the project evolves, helping you gather results that align with your goals.

Both quantitative and qualitative UX research methods provide valuable user insights, but each offer different angles of understanding your user. Mixed methods research combines both types, bridging the gap between hard data and user sentiment.

Here’s an overview at the main differences when analyzing quantitative vs. qualitative data in UX research:

Now, let’s explore different methods to synthesize and evaluate quantitative and qualitative data.

At this point, you should have already conducted UX quantitative research, such as live website usability testing, click tracking, or UX surveys. From here, you’ll have a large dataset of raw results.

There are different approaches to quantitative statistical analysis—descriptive and inferential.

Inferential statistics help identify whether what you’ve found is unique to the group you studied, or if it can also be considered true for all users. This is crucial when using insights to inform product development.

For the purposes of this guide, we’ll be focusing on inferential statistics within quantitative user research analysis. But before jumping into the methods of evaluating quantitative data, here are key terms you’ll need to know:

Now, let’s look at four types of quantitative user research analysis:

A t-test allows you to assess if there’s statistically significant differences between groups of users. It helps identify whether the means, standard deviations, and skewness of groups differ significantly.

In UX research, a t-test helps you infer if your sample-based assumptions apply to the entire population. For example, you might want to see if there's a significant difference in usability scores between new users (< six months) and experienced users (> six months).

While comparing means can show initial differences, analyzing skewness and standard deviations lets you determine if these differences are significant or just a coincidence.

There are multiple kinds of t-tests types, each with a different purpose:

When to use T-tests: Use t-tests whenever you want to identify statistical differences between two groups.

This stands for “Analysis of Variance” and also helps compare the means of multiple groups. It’s similar to a t-test, but focuses on analyzing more than two groups at once. For example, if you want to explore whether there's a significant difference between customer experience metrics, like NPS, of three different user personas.

When using ANOVA, you also want to look at data skewness and statistical deviation to determine whether your results are valid. For instance, if your NPS is +70 but the data has a high deviation, you may infer that your results aren’t statistically significant as there aren’t any patterns within your groups.

Use ANOVA to:

When to use ANOVA: Use ANOVA analysis when you need to compare the statistical relationships between more than one group.

This method measures and describes the strength and direction of the relationship between two variables. It helps understand whether, and how strongly, pairs of variables are related. For example, ‘If the CSAT scores (Variable 1) go down, does the retention rate (Variable 2) go down as well?’

You’ll be presented with a correlation coefficient that ranges from +1 to -1 and refers to positive or negative correlations. A positive correlation indicates both variables move together in the same direction, while a negative one means they move in opposite directions.

Correlation analysis is best for:

When to use correlation analysis: Opt for correlation analysis if you’re looking to assess the relationship between two variables.

Regression analysis could be seen as an extension of correlation analysis, as it explores the relationship between variables—but to identify if these have a cause-effect link. This method seeks to answer to ‘When Variable X moves, is it because of Variable Y?’

Where correlation analysis identifies the existence of a relationship between two pieces of data, regression analysis identifies if one causes an effect on the other.

Use this method to identify which variables cause others to change. For example, identifying that whenever a page takes more than 1.5 seconds to load, users switch tabs.

Use this analysis to identify any cause-effect relationships in design and user experience variables, such as what happens to your conversions if you move the ‘check-out’ button.

All of these analysis methods give you a list of insights and results, but it’s your job to turn them into a digestible report to share with stakeholders, and drive data-informed product decisions from those insights. For maximum insight, you’ll want to merge your quantitative analysis results with qualitative insight to make better product iterations.

When to use regression analysis: Go with regression analysis if you’re looking to identify the specifics of cause and effect relationships between variables.

Qualitative UX research methods provide a lot of data, so whether you’ve conducted user interviews, surveys with open-ended questions, focus groups, or contextual inquiries, step one is compiling the data into a central location.

Your results will likely look like interview transcripts, long-form text-based answers, or time-stamped user behaviors, so you’ll need a more robust location than a spreadsheet—here’s when your UX research repository can really shine!

From here, you can then begin to synthesize and analyze the data using one of these techniques.

Thematic analysis involves identifying recurring themes and patterns within your qualitative data and assigning codes (tags) to organize the research results. This allows you to summarize huge text-based databases into a few key shared insights and results. It’s also particularly effective for uncovering nuanced insights and capturing diverse perspectives from participants.

By reading through the themes, you can make assumptions, see clear patterns, and determine the why behind your sample’s behavior. You can also turn this analysis into affinity diagrams to manage information groups and their relationships.

Imagine you’re analyzing the user experience of different demographics engaging with a language-learning app. With this method, you could combine feedback about ‘online learning’, ‘remote learning’, and ‘learning app’ under the code: ‘E-learning’.

When to use thematic analysis: Use thematic analysis as a starting point for broadly grouping and organizing large amounts of qualitative data.

This qualitative research method focuses on interpreting users’ motivations and experiences by analyzing their discourse and stories. It involves reviewing the structure, content, and context of all story-based research data, e.g., testimonials, user interviews, case studies, or focus groups.

A narrative analysis captures the complexity and richness of human experiences and provides deep insights into users' motivations, emotions, and thought processes. This helps you identify themes and patterns that may not be clear in quantitative results.

For example, you might spot that only a handful of users followed the intended sign-up path during usability testing. This is a fact, but it lacks an explanation. However, by analyzing the text-based data, you notice that users express confusion aloud, saying the red color of the registration button threw them off.

When to use narrative analysis: Go with this method for exploring user experiences and stories, understanding the context of their behaviors, and developing personas and journey maps.

Grounded theory analysis invites you to generate new theories based on data collected in the field. These new theories and frameworks need to be grounded on empirical data.

In UX research, you can use this method to develop new theories about user interactions and behaviors. For example, while coding users’ interview data, you might observe a correlation between positive CSAT and a highly-responsive support team.

This could lead you to develop a theory stating that offering timely responses to users' questions plays a big role in their overall satisfaction. You would then test and refine this theory over time to ensure it remains valid and grounded in empirical data.

When to use grounded theory analysis: Grounded theory analysis is best for exploring new or under-researched areas and continuously developing theoretical models that explain user behavior.

This method allows you to synthesize the text and all qualitative data to make generalizations, find patterns, and draw conclusions.

Here, you should also turn content into codes, like in thematic analysis. In content analysis, however, you want to identify the frequency and repetition of words and topics, getting to more granular detail, rather than just grouping recurring themes.

For example, if you’re analyzing unmoderated usability tests, you can code every comment that references the sign-up page as ‘SU’. Whereas, in thematic analysis, you’d group it into a bigger theme, such as ‘conversions’.

You can also aggregate and interpret the data by writing comprehensive summaries of the findings. You can then take these insights and compare them with findings from quantitative data analysis.

When to use content analysis: Content analysis is ideal for viewing data in snippets and digging into the weeds of specific insights.

Analyzing UX research can be daunting: there’s lots of data, conclusions are subject to cognitive biases, and it can be hard to know which sources to focus on first.

Before you get started, here’s our four top tips for tackling the the challenges of user research analysis:

1. Triangulate UX data from multiple sources: Data quality and integrity is crucial. Corroborate findings using different UX research methods, secondary research, and public studies to maximize your field of view and generate unbiased, conclusive results.
2. Invite collaboration: Involve stakeholders into the user research analysis process—not only does this help minimize bias, but it adds transparency to the UX research process, and aligns everyone within a customer-centric, data-informed culture.
3. Take notes as you go: While you want to avoid jumping to conclusions before finishing your research study, it’s never too early to take notes or consider takeaways. Start tagging and identifying themes during your research sessions to jumpstart analysis and simplify the final task.
4. Use the right tools: Invest in a UX research tool that simplifies research analysis and data quantification. Having the right tool by your side during testing makes it significantly easier to test, analyze, and share UX research reports with stakeholders.

Keeping these tips in mind enables you to produce more reliable, actionable insights with less effort.

The analysis stage of user research projects is when all your hard work starts to pay off. It’s when all the UX stars align to shed light on your user’s wants and needs.

However, user research analysis can be hard. It’s time-consuming, precise, and easy to get wrong if you don’t have the right tools to help.

To get started simplifying and streamlining your user research analysis, try Maze.

Maze is a leading user research platform that provides UX researchers and product teams with countless research methods, detailed reporting, and intuitive data analysis—so you can focus on applying the insights.