Chapter 4
3 Real-life concept testing examples
Learn how companies Todoist, NewsWhip, and Better Proposal used concept testing to test and validate the right ideas to move forward with.
Reading about concept testing is all well and good, but putting it into practice is another story. We ’ve asked our community for three examples of concept testing to help you figure out what concept testing goals, methods, and questions may be best for you and your product.
Example 1: Todoist validated their new Boards feature
Company: Todoist, a to-do list software
Todoist were aware that lots of their customers were clamoring for the Boards feature. With this in mind, their Product Designer Alex Muench says, “We knew it was important to get the first version right so that [it] fits our product and user’s needs.” That’s where concept testing came in.
The product team decided that the aim of their concept testing was to “figur[e] out how to focus on the key features that build a solid foundation for the future and make our users happy.”
Conducting the test
In order to obtain the necessary insights, Alex and his team “contacted people who have requested the feature in the past and interviewed 17 of them from our worldwide user base in 30-minute 1:1 sessions.”
The questions they asked
- “We asked questions about understanding and adapting to the board view concept inside Todoist and how it enhances their workflows.”
- “We also tried to get insights into the transition to this view and if we designed the interface in an intuitive way.”
- “Lastly, we tried to find out if this new feature meets people’s expectations. We kept the interview script structured but also open at the end to allow participants to share any ideas and wishes they might have.”
Using the data
The concept test helped Alex and his colleagues “identify the most relevant usage patterns and were then able to prioritize specific requests regarding sections.”
The results from the test allowed them to roll out the Boards feature to a favorable reception from their customers. As Alex puts it, their research helped them “ship a beta feature that was perceived as very mature and at the end led to a successful public launch that met our user’s needs.”
Example 2: NewsWhip rolled out a new Top Authors Leaderboard feature
Company: NewsWhip, a social media engagement analytics firm
NewsWhip was looking to launch a Top Authors Leaderboard feature with the expectation that other Leaderboard features could follow.
As NewsWhip Product Manager Grainne Conefrey explains, “We wanted to work very closely with users to get the first version right because we were aware that what we were introducing is the foundation of other Leaderboard types that we wanted to work on in the future.”
After their ideation phase, Grainne and her team whittled down the concepts to a few that they felt would best meet their users’ needs. They started concept testing in order to ensure the Top Authors Leaderboard’s success and pave the way for future Leaderboard iterations.
Conducting the test
The team set up 30-minute one-on-one calls with users “who had requested this feature, who had been using this data in our tool (because it was available in other parts of the tool), and who'd been showing an interest in this data and understanding it more.” Each user was introduced to one of the concepts Grainne and her team were considering on the call.
The questions they asked
- “We asked them more about the problem area and more about what their needs were, so we could capture that upfront.”
- “We asked them questions around their first impressions and if they could describe certain elements of the feature to us to make sure our language was industry standard and easy to understand.”
- “We asked questions about usability and the navigation around the feature as well.”
- “Since we were introducing a new way of ranking the authors into our product, we also asked them to explain the ranking back to us so we could see if they understood the ranking system.”
- “We asked questions around our visual elements to make sure it was beneficial there and it was providing an easier way to interpret the data.”
- “To get a better understanding of the value of the feature as we went, we asked how users would include it in their workflow.”
Using the data
Grainne says through these one-on-one interviews, her team was able to solidify their core values and eliminate friction:
“Concept testing really helped us to narrow down core values and narrow down on areas where we could make things a little easier for users. We also identified a couple of opportunities as well to come back to at a later stage.”- Grainne Conefrey, Product Manager at NewsWhip
With the feedback from the interviews, Grainne and her team set about eliminating areas of friction and identifying opportunities they could iterate on in the future. “These were just kind of small delighters that were identified by users during those calls. They were bigger elements that we could iterate on, but they weren't part of our core value add there,” she says.
Once they refined their design, the product development team used concept testing again, “making sure that we interpreted everything they said in their feedback and that we captured everything important to them.”
In the end, Grainne says that concept testing was useful because it helped them to remove any early friction in the products before the launch, making the Top Authors Leaderboard feature more intuitive. Since then, Grainne and her team have released two additional Leaderboards using the same structure.
Example 3: Better Proposals tested their blog strategy
Company: Better Proposals, online proposal making software
The team managing Better Proposals blog was unsure of which type of blog content to create. Petra Odak, the Chief Marketing Officer, says, “we were doing a mix of SEO-focused keywords and content based purely on our own experience with sales, marketing, and product development.
While the first type of content brought in new visitors from organic search, the second type of content kept our existing readers coming back to our blog.”
The challenge? Finding out which content the team should continue creating for their blog.
Tests conducted
The Better Proposals team went ahead and “asked our subscriber base what they thought.”
The questions they asked
- “How would you rate the quality of our content on a scale from 1-5?"
- "Where have you found our blog? (recommendation, newsletter, Google search result, social media)"
- "Which posts do you read the most? (Listicles and marketing type of content vs. Content describing our own experience with sales and marketing)”
Results gathered
From the concept test survey, Petra and her team learned that the readers’ interests showed “a 50/50 split between the two types of content.”
For them, this meant they continued creating both content types. Or, as Odak puts it, “we now write content both focused around specific SEO keywords and based on our own experience so we can get the best of both worlds.”
Final words
The benefits of concept testing are undeniable, and their usefulness persists throughout the product development process. As Grainne puts it, concept testing in its many forms “provides insight into how users perceive your idea, if it solves a problem for them and if they are willing to pay for it.”
You have the tools now to pick the right concept testing methods for you and your team. With those at the ready, you can craft solutions that are data-informed and user-centred.