Chapter 5
Evaluative Research: Definition, Methods, and Types
In the last chapter, we learned what generative research means and how it prepares you to build an informed solution for users. Here, we look at evaluative research and the benefits of this type of research.
What is evaluative research?
Evaluative research is a type of research used to evaluate a product or concept, and collect data to help improve your solution. Evaluative research has many benefits, including identifying whether a product works as intended, and uncovering areas for improvement within your solution.
Evaluation research should definitely start early–before anyone gets too attached to a specific design or implementation of a concept.
Mithila Fox, Senior UX Researcher at Stack Overflow
Also known as evaluation research or program evaluation, evaluation research is typically introduced in the early phases of the design process to test existing or new solutions and continue to be employed in an iterative way until the product becomes ‘final’. According to Nannearl LeKesia Brown, Product Researcher at Figma, “With evaluation research, we’re making sure the value is there so that effort and resources aren’t wasted.”
However, evaluation research doesn’t stop when a new product is launched. For the best user experience, solutions need to be monitored after release and improved based on customer feedback. For Nannearl, this ensures the solutions continue to add value and meet user needs:
There may be times when I'd want to do research on a solution that has already been made available. For example, if my team created a website, after a certain amount of time, it’d be necessary to make sure that this site is still achieving the team’s goal or to find an even better way for it to achieve that goal than it already does.
Nannearl LeKesia Brown, Product Researcher at Figma
According to Mithila Fox, Senior UX Researcher at Stack Overflow, the research process evaluation includes various activities, such as testing for accessibility, testing the content, assessing desirability, and more. Mithila also mentions that during UX research, evaluation research can also be conducted on competitor products to understand what solutions work well in the current market, before you start building your own. She explains:
Evaluation research can start as soon as you understand your user’s needs. Even before you have your own mockups, you can start by testing competitors or similar products. There is a lot we can learn from what is and isn't working about other products in the market.
Mithila Fox, Senior UX Researcher at Stack Overflow
Why is evaluative research important?
Evaluative research allows you to test a solution and collect valuable feedback that can help you refine and improve the user experience. Findings from evaluative research are key to assessing what works and what doesn't and identifying areas of improvement.
Evaluative research should always be a part of the design and product development process. By allowing users to "evaluate" your product early and often, you'll know whether you're building the right solution for your audience.
Also, the insights that you get from this type of research can demonstrate the effectiveness and impact of your project. You can show this information to stakeholders to get buy-in for future projects and achieve your product goals efficiently.
Evaluative vs. Generative research
Generative research helps you understand your users' motivations, pain points, and behaviors. The goal is to define the problem you'd like to solve and design the best possible solution for it.
On the other hand, evaluative research is used to assess an existing product solution to ensure it's easy to use, works as intended, and meets users' needs and expectations.
Tip ✨
Use generative research to bring forth new ideas during the discovery phase. And use evaluation research to test and monitor the product before and after launch.
Generative and evaluation research are both valuable decision-making tools in the arsenal of a researcher. They should be similarly employed throughout the product development process as they both help you get the evidence you need.
When creating the research plan, study the competitive landscape, the target audience, the needs of the people you’re building for, and any existing solutions.
Depending on what you need to find out, you’ll be able to determine if you need to run generative or evaluation research. However, both methods will be useful throughout the research process to get different types of evidence.
Mithila from Stack Overflow explains the benefits of using both research methodologies:
Generative research helps us deeply understand our users and learn their needs, wants, and challenges. On the other hand, evaluation research helps us test whether the solutions we've come up with address those needs, wants, and challenges.
Mithila Fox, Senior UX Researcher at Stack Overflow
Types of evaluative research studies
Tip ✨
Run formative research to test and evaluate solutions during the design process, and conduct a summative evaluation at the end to evaluate the final product.
There are two types of evaluative studies you can tap into: summative and formative research. Although summative evaluations are often quantitative, they can also be part of qualitative research.
A summative evaluation helps you understand how a design performs overall. It’s usually done at the end of the design process to evaluate its usability or detect overlooked issues. You can also use a summative evaluation to benchmark your new solution against a prior one or that of a competitor’s and understand if the final product needs assessment.
On the other hand, formative research is conducted early and often during the design process to test and improve a solution before arriving at a final one. Running a formative evaluation allows you to test and identify issues in the solutions as you’re creating them, and improve them based on user feedback.
Looking to conduct UX research? Check out our list of the top UX research tools to run an effective research study.
Choosing an evaluative research method
Part of the research design process is choosing the type of evaluation that best suits your project, and then determining which research method you will use. These can be anything from usability testing and tree testing to focus groups, but it's essential first to identify what fits the project.
Mithila and Nannearl rely on evaluation methods such as usability testing, surveys, A/B testing, card sorting, and more to test the features that the product teams at Figma and Stack Overflow build.
Surveys help with data collection and gathering useful feedback from users and tap into a large volume of responses for detailed data analysis. Mithila, Senior UX Researcher at Stack Overflow, explains the value of survey research as part of the evaluation process:
Surveys can help test your users’ specific feelings on a certain feature, and if you continue to run them continuously, you can establish a baseline of user satisfaction. By understanding this baseline, you will be able to see how the changes directly impact your users.
Mithila Fox, Senior UX Researcher at Stack Overflow
Other types of evaluation include usability testing, which allows you to test your product with real people by having them complete a list of tasks. Check out this complete guide to usability testing to learn more.
For Nannearl, Product Researcher at Figma, selecting the correct method goes back to coming up with the right research questions and hypotheses at the beginning of the research project. She explains:
This is the value of having a UX research plan before diving into the research approach itself. If we were able to answer the evaluative questions we had, in addition to figuring out if our hypotheses were valid (or not), I’d count that as a successful evaluation study. Ultimately, research is about learning in order to make more informed decisions—if we learned, we were successful.
Nannearl LeKesia Brown, Product Researcher at Figma
No matter the method used, the important thing is to test with people who are part of your target audience:
Regardless of the type of evaluation, my goal is to help my team build confidence in what we decide to make available to our users. If the update isn’t understandable or useful to those we’re striving to help, then it’s really not worth the time or effort at the end of the day.
Nannearl LeKesia Brown, Product Researcher at Figma
Evaluative research question examples
To gather valuable data and make better design decisions, you need to ask the right research questions. Here are some examples of evaluative research questions:
Usability questions
- How would you go about performing [task]?
- How was your experience completing [task]?
- How did you find navigating to [X] page?
- Based on the previous task, how would you prefer to do this action instead?
Get inspired by real-life usability test examples and discover more usability testing questions in our guide to usability testing.
Product survey questions
- How often do you use the product/feature?
- How satisfied are you with the product/feature?
- Does the product/feature help you achieve your goals?
- How easy is the product/feature to use?
Discover more examples of product survey questions in our article on product surveys.
Closed card sorting questions
- Were there any categories you were unsure about?
- Which categories were you unsure about?
- Why were you unsure about the [X] category?
Find out more in our complete card sorting guide.
In the next chapters, we'll learn more about quantitative and qualitative research, as well as the most common research approaches, and share some practical applications of how UX researchers use them to conduct effective research.
Frequently asked questions
What is evaluative research?
What is evaluative research?
Evaluative research, also known as evaluation research or program evaluation, is a type of research you can use to evaluate a product or concept and collect data that helps improve your solution.