Concept Testing in UX: Moving Forward with the Right Ideas

Concept Testing in UX: Moving Forward with the Right Ideas

Concept testing allows you to validate product, design, and marketing ideas early on and save time and resources by moving forward with the right concepts. Over the course of this guide, we’ll look at the benefits of concept testing, how to run a concept test, how to craft a survey, and real-life examples of concept tests. With advice from expert product managers, you’ll be ready to test, learn, and iterate with your customers—fast.

Move forward with the right ideas

Know the value of your solution and validate ideas and concepts before wasting valuable time and resources.

Chapter 1

What is concept testing?

Concept testing is a type of research that evaluates the feasibility, appeal, and potential success of a new product before it’s built. It puts the user front and center in the ideation process, using research methods like A/B testing, surveys, and customer interviews.

The main difference between a good and bad product concept is whether it actually appeals to the people it’s made for. Thorough testing is critical to gauge your audience’s interest, understanding, and likelihood-to-purchase, before committing time and resources to a concept.

When to use concept testing

Concept testing can be done at multiple stages in the product discovery process—during ideation, when prototyping, when defining marketing messaging, or just before the launch of the product.

“Concept testing is the fine line between the phases of scoping the problem, and defining the solution, as a part of wider product discovery,” says Netali Jakubovitz, Director of Product Management at Maze.

Here are some specific points in the product development process when concept tests can be run:

  • At the discovery and ideation stage: After idea generation and idea screening, narrow down the solution to move forward with during the product discovery, and identify how best to solve user problems
  • During the design stage: Test the particulars of different design concepts, from low-fidelity (e.g. the layout of a web page) to high-fidelity prototypes (e.g. user flow, graphics, branding)
  • Before the product launch: Catch any last-minute changes that need to be made—a pre-launch concept test can also help identify which marketing ideas to move forward with for your launch campaign

There’s no one way to conduct a concept test, either. Some teams opt for concept testing surveys, while interviews and focus groups are also useful for gathering qualitative information. The method you go for will depend on your goals and resources—more on this in Chapter 2.

The same problem could be solved in many different ways and from many different angles, but not all would actually work or deliver significant value to your customer.

Netali Jakubovitz, Director of Product Management at Maze

Netali Jakubovitz, Director of Product Management at Maze

The benefits of concept testing

Concept testing proves valuable because it validates that your customers actually want the product. Idea validation is an essential part of UX research and a necessary step in creating successful products.

We could go on for days about the advantages of concept testing, but here are just a few of the reasons why this technique is an essential part of product success:

Save time and money

Approximately 95% of new products fail, according to research from Harvard Business School. A whopping amount of resources go down the drain when a product flops, from the money spent to create it, to the effort exerted by the product development team.

It's important to validate the concept we're going after before starting to build it due to the costly charge of engineering time, the complexity entailed in changing direction if development already started, or deploying a solution that solves a symptom but not the problem. This would send the team back to the concept drawing board and would be a waste of precious time, resources, and motivation.

Netali Jakubovitz, Director of Product Management at Maze

Netali Jakubovitz, Director of Product Management at Maze

While concept testing itself requires a slice of your budget, in the long term that investment is far less costly than launching an idea that fails. For example, once a software product is launched, correcting an error can cost 100 times as much as it would have if it were fixed in the development phase. Studies also reveal that development teams waste 40 to 50% of their time on avoidable reworks, a reality that can be avoided with concept testing.

It provides actual value for early validation, contributes to prioritization, and eventually increases the overall quality of our product at a low cost. There’s simply no reason not to do it!

Shelly Shmurack, Product Manager at Walmart Global Tech

Shelly Shmurack, Product Manager at Walmart Global Tech

Validate and launch with confidence

Concept testing also provides data to back up your idea and get buy-in from stakeholders or other members of your team. Bringing a new concept to life can be daunting, and management may understandably be cautious about taking that risk. However, once you have the research from concept testing on your side, even your most cautious team members will see the potential in your product concepts.

The data [from concept testing] allows product managers and decision-makers to gain confidence in favor or against an idea or a solution.

Andrea Ruggeri, Product Manager at Doodle

Andrea Ruggeri, Product Manager at Doodle

Andrea Ruggeri, Product Manager at Doodle, says that he finds concept testing useful for decreasing uncertainty. “The data allows product managers and decision-makers to gain confidence in favor or against an idea or a solution,” Andrea explains.

Discover new business opportunities

Concept testing can expose a new path for your company to take, whether that means optimizing the existing features of your product or pivoting entirely.

Concept testing helps explore various potential solutions to the problem and uncover the sweet spot of customer delight; so then you can know where to invest costly resources and energy. This translates to business revenue and growth.

Netali Jakubovitz, Director of Product Management at Maze

Netali Jakubovitz, Director of Product Management at Maze

Your assumptions about what makes your concept great are just that—assumptions. So testing shows what aspects of the idea actually appeal to your customers, which you can then capitalize on when developing or highlighting the concept's best attributes in your marketing campaigns.

Even a concept test that appears to fail still has value. "Early failures are not only desirable but also necessary since experimenters can quickly eliminate unfavorable options and refocus their efforts on more promising alternatives (often building on ideas that were initially unsuccessful)," writes Stefan H. Thomke, Author of “Experimentation Works”.

Martín Burgener, Product Director at eDreams, explains how failure is a key element of building products at eDreams: “If we think about the products eDreams have been working on, roughly 70-85% of all tests we do result in a failure. And that’s for a company that is product-led and has been making digital products for 12 years.”

Once you evaluate what matters to your target audience through concept testing, you can use those learning opportunities to tweak or completely change your product to maximize its success.

Best practices for effective concept testing

To ensure that your concept testing efforts are effective and yield actionable insights, you need to follow some best practices that align with your research objectives and the development stage of your product idea.

Tip 💡

For an in-depth guide to conducting a concept test, check out chapter 2 of this guide.

1. Implement concept testing as part of your continuous product discovery

Continuous product discovery and concept testing work effectively in tandem. As part of your continuous discovery habits, every time you make a change to your product, the new concept should be tested with your audience. Conducting iterative rounds of concept testing, even after your product launch, can help identify new opportunities for innovation and improve your product-market fit.

2. Set specific goals and benchmarks for your test

Just trying to gauge if customers ‘like’ your product concept is a bit too vague. You’ll get more value from your test if you have specific goals, for example: “We want to validate that most of our users (80% or more) find this feature useful.”

You can measure different aspects of the product concept, like perceived value, messaging, price sensitivity, and overall appeal. Do this by using a variety of testing formats to gather feedback on the different aspects of your product concept (more on this later).

3. Ask your audience the right questions

The right research questions will reveal the insights necessary to validate your concept or ascertain what it’s lacking. For example, asking participants if they "like" your product concept may not provide enough detail to guide product development decisions.

On the other hand, targeted questions like "What do you like about Product A?" or "What features do you think Product B is missing?" can help you gain deeper insights into customer needs and preferences.

Questions should be open-ended rather than leading, allowing participants to express their opinions and feedback in their own words.

For example, if you ask, "How satisfied are you with our product?" and give options such as "very satisfied," "somewhat satisfied," or "not satisfied," participants may be more likely to choose the options that are provided, even if they don't accurately reflect their level of satisfaction.

4. Tap into your target audience

There’s no point in conducting concept testing with participants who don't fit the demographics of your customer base, as different demographic groups may have different needs, preferences, and behaviors.

For example, if you’re testing a product concept designed for women aged 35-50, it wouldn’t be useful to gather feedback from male participants or participants aged 60+, as you’ll gather irrelevant insights.

Recruiting the right participants is key. With their feedback as guidance, you can develop and launch a successful product that meets customers’ needs. One way to help is by using a user testing platform with a recruitment panel that allows you to filter participants.

Product tip ✨

If you're using Maze, you can access 127k+ participants and filter them based on your demographics, language, device, and country. Or, if you handle your own recruitment, try Reach, our participant management platform—set up campaigns with personalized messaging and streamline workflows according to your specific audience segments.

5. Use different testing formats for different ideas

Different goals require different testing formats, each suited to unique objectives and gathering various research metrics.

  • Monadic approach: This format is ideal for evaluating the performance and feedback of a single product concept or design variant in isolation. You can focus on specific strengths, weaknesses, and user feedback of a concept without the influence of other options.
  • Comparative testing approach: When you present different ideas to participants simultaneously, you can gather insights into the relative appeal, preferences, and distinguishing features of each concept.
  • Exploratory interviews: This approach gives you valuable qualitative insights through in-depth interviews or group discussions with participants. You can explore new product ideas, gather rich insights, and identify potential opportunities or challenges.

Moving forward

The beauty of concept testing is that it's beneficial for everyone involved. Your customers get a product that's actually relevant to them, designers are reassured they're on the right track, and product teams save time and money moving forward with the right ideas.

In the following chapters, we’ll walk you through how to run a product concept test, the creation of a concept testing survey, and some real-life examples of concept tests. With these tools in your belt, you can move forward with confidence and help ensure your users find value in your products.

Here’s to making life easier for product owners and users 🎇

Frequently asked questions about concept testing

What is the purpose of concept testing?

The purpose of concept testing is to validate product, design and marketing ideas early on in the design stage.

What is the main benefit of concept testing?

Concept testing’s main benefit is its ability to provide proof of concept and substantiate product, design, and marketing ideas before launching the product. By testing these ideas, you are able to save time and resources by moving forward with the right concepts.

What are the 6 steps of concept testing?

The 6 stages of concept testing are:

  1. Set a goal for your test
  2. Craft your script and questions
  3. Recruit the right participants
  4. Determine the flow of the concept test
  5. Integrate quantitative measurements
  6. Review and interpret your results

What types of content testing are there?

The four main methods of concept testing are:

  • Monadic testing (testing a single concept or design variant with multiple focus groups)
  • Sequential monadic testing (testing several concepts or variants with multiple focus groups)
  • Comparative testing (asking individual testers to compare multiple concepts)
  • Protomonadic testing (monadic testing with an additional comparative and evaluation stage)

What’s a well-known example of concept testing?

Here are 2 well-known examples of concept testing:

  1. Tesla: In 2015, Tesla announced its Tesla Model 3, which hadn’t been made. Within a month, around 400,000 customers invested in the Model 3. Not only did Tesla determine whether their target customers would purchase the new car, but they also had a massive $400 million investment in the concept.
  2. Yamaha: Instrument manufacturing company Yamaha was deciding between a knob or a sliding fader for their new electric keyboard. They enlisted their customers’ help in choosing the optimal feature to ensure the keyboard met their target audience’s preferences.