Concept testing: 4 methods to validate your product ideas + real-life examples

Masooma Memon

Jan 21, 2021

Not all ideas are good ideas. There. I said it. But running ideas through a concept testing method to see if they’re well worth your time and resources is always a good idea. Why? Because a concept test validates your idea by exposing it to your target audience and finding out if it deserves seeing the light of day.

Sounds like what you need to do right now? That’s more like it!

Let’s give you details of what concept testing is and the methods available to conduct a test. We’ve also included real-life examples to show you how teams are doin’ it.

Let’s get on with it.

What is concept testing?

Concept testing involves evaluating an idea or concept – this could be a new product, landing page design, ad copy, or anything else – by talking to your (potential) customers about it.

This way, you can learn whether your idea resonates with your target audience and whether they like it enough for you to take it to the finish line.

Think of it like this: you have a new product idea in mind that you think will make a great addition to your customers’ lives. But, will it?

Since you can’t read customers’ minds and seeing the future is also still not a thing, it’s best you directly question your customers about the new product idea. Such a product concept test helps you access this new product’s likeability and decide based on real data whether it’s worth the investment.

When to conduct a concept test?

Here’s the beauty of concept testing: it’s not limited to the early stages of product development.

If anything, you can pick your customers’ brains at any stage including early on when you’re validating a product idea or in the final stages when working on granular details such as choosing between font types.

And to top it up: concept testing isn’t for product launches only. You can test any new concept including website design, a rebrand, logo testing, name testing, landing page copy, pricing updates, newsletter design, and so on.

Benefits of concept testing

Since you get a peek into your audience’s mind with concept testing, you can reap the following benefits: 

  • Concept tests help save time and resources by giving you a data-backed scoop of your target audience’s thoughts on the idea you test
  • These tests can also give you customer insights that, in turn, can help you change course or iron out particular details of what you’re working on based on customer preferences
  • Quizzing your potential customers also ensures launch success. In the case of testing logos, rebranding, web design, and ads, this would mean your work speaks to your target audience. As for new products and features, concept testing helps ensure you launch what’s actually needed.

Pro benefit: You can guarantee a successful feature/product launch by pairing concept testing with usability testing. How? By learning your target audience’s interest in the feature/product with a concept test. Then, running usability tests to get real users to test drive the feature/product for its usability and intuitiveness.

Concept testing methods

With the benefits of concept tests established, let’s walk you through the four concept testing methods that you can work with.  

But before that, follow this checklist for any concept testing method you go with:

👉  Set a specific goal for your test

Such a goal is one that outlines exactly what you’re looking to test. So, for instance, if you’re testing ads for their click-through rates, writing: ‘I need to learn if the respondents like my ad,’ won’t cut it. Instead, go with a clearer goal that reads, ‘I want to ask my audience if they find my ad trustworthy enough to click on it.’

👉  Know what to ask your audience

For these tests to deliver actionable insights, you need to ask the right user research questions in the concept test surveys you prepare for the respondents.

Let’s say you are concept testing your ad’s CTA. In this case, questions like these can help:

  • Is the CTA box easy enough for you to spot at first glance?
  • Do you think the CTA copy is easy to read?
  • Which CTA box color do you prefer the most? (use images to show them all variants)

👉  Know who to target

Equally important here is knowing who to recruit for your tests. Do you need to survey your customers? Or a target audience reflective of your customers will do? The answer is simple: the right research demographic depends on your objective.

If you’re testing a new feature, you’ll need insights from customers who already have your existing product’s know-how. If you’re testing a new design instead of an update, a reflective audience will do.

With that out of the way, let’s talk about concept test methods. The methodology is nothing complicated—just surveys but each one takes a slightly different approach. You’ll learn how below.

1. Monadic testing

A monadic test involves dividing test participants into focus groups and showing each group one concept to answer an in-depth survey on the concept. So, for instance, if you’re testing logo designs, give each group only one logo variant to review and share insights on.  

Since each focus group reviews one variant, consumer responses tend to be unbiased, detailed, and highly targeted. The response rate also tends to be high since these concept testing surveys are short.

But here’s the thing: the more variants you have at hand, the larger the audience you’d need for the test as you’d be dividing them into focus groups for evaluating one concept each. This counts as a downside for those with limited resources as a larger participant pool increases research costs.

2. Sequential monadic testing

As with monadic testing, sequential monadic testing requires you to divide test participants into groups. The chief difference, however, is that all groups are shown all the concepts at once.

 This way, the audience size doesn’t have to be large. And, to reduce the odds of bias, the order of variants shown to participants is randomized.

Sequential monadic testing is great for teams with budget constraints and availability of a smaller pool of test participants.

But, be prepared, handing over a lengthy survey to your test respondents comes with its pitfall of reduced completion rate.

3. Comparative testing

This one’s the simplest of all the concept test methods since you share two or more concepts with survey respondents and ask them which they like the best.

You’ve two options to get your audience’s insights. One: use rating questions with a common scale to ask respondents to compare test variants.

A rating question will read along the following lines for an example ad design test:

Please rate the trustworthiness of the following ad designs on a scale of 1-10 where 1 is not trustworthy at all and 10 is the most trustworthy.

And, two: use ranking questions. These survey questions ask respondents to compare variants by arranging them in order of their preference. So, for the same ad design example, a ranking question will read:

Please rank the trustworthiness of the following ad designs with the first one in your list being the most trustworthy and the last one being the least trustworthy. 

Alternatively, use a Likert scale that asks respondents to rate (usually on a point scale of one to seven) how much they agree or disagree with a statement you give them. 

With comparative tests, you don’t need a wide respondents sample size, which makes it a budget-friendly concept test method. The only kicker? Results from this test lack context as they don’t explain why respondents choose one concept over another. This also means that you don’t get sufficient details to improve your prototype.

4. Protomonadic testing

This involves asking respondents to evaluate different concepts, followed by doing a comparative test to summarize which variant they prefer.

Put simply, a protomonadic test is a combination of sequential monadic and comparative tests.

In a sense, this test makes up for the shortcomings of comparative testing by quizzing respondents about the concepts. 

So, in a concept test on a web page redesign, for example, participants will be first divided into focus groups with each seeing all the designs and sharing their observations by answering a lengthy questionnaire. This will then be followed with another survey – one that asks them to either rank or rate their preferred design.

Naturally, these concept tests are lengthier than other tests as they pack two tests into one. This also translates into more time and work for both the researchers and respondents who have to prepare more test surveys and answer more questions, respectively.

 Not to mention, all the time that this user research method takes can increase the risk of nonresponse bias and reduced completion rate.

Product tip: Maze is a rapid testing platform that allows you to do concept testing and get actionable data you can act on. Learn more about how concept testing with Maze works.

Concept test examples

Since no theory is complete without examples, we went on to source real-life examples for you. Read on to find out how companies are concept testing their way through new ideas:

 Example 1: Todoist rolled out a new feature called Boards

 Business: Todoist, a to-do list app

The team at Todoist conducted several concept tests for a new, highly-requested feature Boards. Owing to this demand, their Product Designer, Alex Muench, notes, “We knew it was important to get the first version right so that [it] fits our product and user’s needs.”

Consequently, the team conducted concept tests throughout the development process. The challenge? “Figuring out how to focus on the key features that build a solid foundation for the future and make our users happy,” in Alex’s words.

Tests conducted

The team “contacted people who have requested the feature in the past and interviewed 17 of them from our worldwide user base in 30-minute 1:1 sessions,” Alex writes. 

They asked the following: 

  • “We asked questions about understanding and adapting to the board view concept inside Todoist and how it enhances their workflows.
  • We also tried to get insights into the transition to this view and if we designed the interface in an intuitive way.
  • Lastly, we tried to find out if this new feature meets people’s expectations. We kept the interview script structured but also open at the end to allow participants to share any ideas and wishes they might have.”

Results gathered

With the test conducted, the Todoist team was able to “identify the most relevant usage patterns and were then able to prioritize specific requests regarding sections,” Alex elaborates.

In short, concept testing throughout the process enabled the team to “ship a beta feature that was perceived as very mature and at the end led to a successful public launch that met our user’s needs.” 

Example 2: Better Proposals tested their blog strategy

 Business: Better Proposals, an online proposal making software

The team managing Better Proposals blog was unsure of which type of blog content to create. Petra Odak, the Chief Marketing Officer, says, “we were doing a mix of SEO-focused keywords and content based purely on our own experience with sales, marketing, and product development. 

While the first type of content brought in new visitors from organic search, the second type of content kept our existing readers coming back to our blog.”

The challenge? Finding out which content the team should continue creating for their blog.

Tests conducted

The Better Proposals’ team went ahead and “asked our subscriber base what they thought.”

Some of the questions they asked included: 

  • “How would you rate the quality of our content on a scale from 1-5?
  • Where have you found our blog? (recommendation, newsletter, Google search result, social media)
  • Which posts do you read the most? (Listicles and marketing type of content vs. Content describing our own experience with sales and marketing)” 

Results gathered

From the survey, Petra and her team learned that the readers’ interests showed “a 50/50 split between the two types of content.”

 For them, this meant they continued creating both content types. Or, as Odak puts it, “we now write content both focused around specific SEO keywords and based on our own experience so we can get the best of both worlds.”

Summing up

The benefits that concept testing delivers are too good to deny. In a nutshell, this research method lets you, “test your assumptions [which] reassures you that you are on the right track,” Alex summarizes.

Moving forward: pick one of the concept test methods that suits your objective, resources, and available respondents, then get to work. Back your ideas with data so you can work on what’s valuable to your audience or what speaks to them the most. 

Here’s to doing more for your users. 🎉