How to conduct content testing like a pro: A step-by-step guide

Content is everywhere. It’s very often the first way your users will engage with your product.

You want your content to stand out for the right reasons, and for users to find (and understand) your content first-time round. If they’re re-reading, you’re doing it wrong.

So, what’s the key to ensuring the content on your site meets user needs? Content testing.

In this article, we’ll cover the what, why, and how of content testing—complete with a step-by-step guide to running your own content tests. Plus, we'll share some insights from industry experts: Vaida Pakulyte, UX Lead at Technigo, Steve Howe, Content Designer, and Nicole Michaelis, Content Design Lead at Wolt for their expert insights throughout.

What is content testing?

Content testing is a research method that helps understand if users can find and understand your content. With content testing, you can check if the content you create resonates with users, speaks directly to their pain points, and provides enough context to help users complete tasks successfully.

Content testing is widely used during user research to test product, app, or website content, and can be conducted at any stage in product development.

Validate content with real users

Test your content with Maze and guarantee your content is clear, intuitive, and consistent.

Why should you test content?

Content testing is important as a preemptive tool to prevent bad content. Bad content can completely disrupt a user’s experience of a product and leave them confused or unengaged, which may drive them away from using your product altogether.

Content testing helps you:

  • Identify which content works, which doesn’t, and why
  • Create accessible products and experiences
  • Build an intuitive user experience (UX) for your audience
  • Choose effective keywords and language
  • Prioritize content development efforts
  • Prevent users from becoming confused or disrupted

Put simply, content testing helps ensure good content makes it into the product, and bad content never makes it out of the testing process.

You need to make time for testing content as it touches all other aspects of your product. And who knows, you may discover something you never considered in the first place.

Nicole Michaelis, Content Design Lead at Wolt

What content should you test?

Wherever there is content in your product, it’s worth doing content testing. We’re yet to see a product that uses no content at all (as unique as that would be).

Think about every piece of content on your product—your homepage, error messages, menu buttons, filters, product pages, and more. At first glance you may not think of this as content, but it is—and it all needs testing.

To decide which content to test, prioritize based on the urgency and importance the task holds for users. If it’s a critical message, such as an error message in payment flows, then it’s vital that the content is easily understood and helps the user achieve their goals.

Vaida Pakulyte, UX Lead at Technigo, emphasizes that when planning to test content, it’s important to give a reference for users to evaluate the content. For example, you can provide context such as a design mockup or a prototype.

Include visual elements, buttons, and everything else that will be on the final design. Your users will be seeing your content in context, so the way you test it should be the same (whenever possible).

5 Ways to evaluate content

So, what are you testing for? There are five main ways to evaluate content, each looking at a different key aspect for in-product content success:

  • Usability: Usability refers to testing how easy your product’s content is to engage with. Product content doesn’t afford the same character flexibility as other content, so brevity is everything. However, brevity gives you fewer words to communicate with—ensuring users understand your product content is key.
  • Readability: You want your content to be readable by your users from the get-go, not after numerous readthroughs. Remember that your users will expect to be able to scan the text rather than reading every word.
  • Accessibility: Accessibility refers to your product’s ability to provide a great experience to all users. Consider how screen-readers or other accessibility aids will interact with your content. Not only does this widen your audience, but it creates an inclusive product that everyone can use.
  • Searchability: This is to do with search engine optimization (SEO) and dictates whether people are able to find your website or product easily online. Getting it right can hugely increase the numbers of people who find and read your content.
  • Tone and voice: Finally, you should consider the tone and voice of your content. Your product, like your brand, has a voice it uses to speak to users, so making sure each piece of content resonates with your audience is important.

Looking out for these five areas when testing your content will help ensure it’s exactly what your audience wants.

When should you perform content testing?

Determining the right stage of development to test your content is tricky. On the one hand, you want your content to reflect the real content your users will encounter. On the other, you don’t want to spend lots of time creating fully-formed and finalized content only to discover it’s not what your users need.

Ultimately, like most types of research and user testing, content testing works best when used iteratively at multiple stages in the design process. You can use content testing:

  • During the conceptualization phase, you can use content testing to understand the broad framework of content you’ll need (what information they need, the language they use etc.) and get an overview of the conversation you’ll be having.
  • Midway through design, you can run content tests to validate your current content and understand where you may want to change direction, before you invest too much time in the wrong area. Content testing at this stage can be invaluable in understanding whether you’re meeting your goals; are you confusing or communicating?
  • Close to and after launch, content testing is invaluable to check that the content—after your iterations—is clear to all potential users and achieves what you set out to do. How does it make users feel after they read it? Does it work with accompanying design or imagery? Here, you can also make any last minute changes.

8 Content testing methods

There are a number of methods for testing content, and each is particularly useful depending on your goals and use case.

Task-based usability testing

You can run a usability test to test content, with slight modifications to the usual way the method works. In a typical usability test, users complete pre-scripted tasks that are closed-ended and shorter in form.

When testing content, keep the tasks open-ended and allow users to explore the product in their own time. The goal is to notice how users engage with a website or app, how they find the information they need, and whether they run into any confusion. You want to assess if the content is clear and enables people to use the product successfully.

Testing this way will provide you with qualitative information about what works and what doesn’t and give you insight into which content may need to be changed or improved. Apart from paying attention to what users say during the test, also notice their behavior: sometimes, the two may contradict each other.

Always keep the behavior in mind: what users do during the test might contradict or confirm what the user says.

Vaida Pakulyte, UX Lead at Technigo

Usability testing provides the perfect opportunity to make sure you and your users are on the same page in regards to the language used.

Cloze test

A Cloze test measures how well your readers can understand a piece of text through context and prior knowledge. It works by taking a 125- to 250-word piece of text and removing one every few words, typically every fifth or sixth.

You present this text to a test participant and see how many of the missing words they can guess correctly. An overall score of 60% or above is deemed comprehensible enough to meet user needs.

Here’s a short example of what a Cloze test looks like:

When Mr. Bilbo Baggins __________ Bag End announced that __________ would shortly be celebrating __________ eleventy-first birthday with a __________ of special magnificence, there __________ much talk and excitement __________ Hobbiton.

Card sorting

Card sorting enables you to learn how users understand and categorize information. It consists of having users group ideas or information in a way that makes sense to them using physical or virtual cards—revealing where users expect certain content to be found.

There are three different types of card sorting, all of which help at different stages of the content testing process. For example:

  • Open card sorting helps generate ideas and define new information architecture, and should be used during the initial content development process
  • Closed card sorting helps evaluate current information architecture and naming conventions, and should be used to test existing content
  • Hybrid card sorting combines the above two card sorting processes, and should be used throughout the card sorting process—particularly during the first iterations of your content development
Table showing differences between three card sorting types: open, closed, hybrid

Card sorting type comparison table

For more information on card sorting and it can help you better structure content, take a look at our guide to running a card sort.

Highlight test

A highlight test is particularly useful for seeing how people feel about your content, rather than testing for comprehension. As we pointed out before, your product’s voice is important, and this test is perfect for seeing how your copy resonates emotionally with users.

To do a highlight test, you’ll need a sample of your text and a red and green highlighter. You give participants the text and ask them to highlight in green what made them feel confident about the product, and in red what made them feel less confident.

If you’re testing for reactions and emotions other than confidence, you can just switch out “confident” for any other feeling.

Five-second testing

In a five-second test, you present your users with a design and give them five seconds to look at it, then ask them questions. This method can be used to test landing page copy, UI content, and more. You’re testing to see if the information on your page can be quickly absorbed, perceived, or understood.

The questions you ask can range from broad, like “What do you think of the page?” or “What do you remember seeing?” to more specific ones, like “Which of the items are on sale?”.

It’s best to start the test with broad questions to get general feedback and thoughts from your participants, then asking more specific questions to see what information stuck with them.

Preference testing

Preference testing consists of showing research participants two to three versions of your design—in this case, with variations in the content—to get an idea of which they prefer and why. It can help you understand the user’s perception of your content, how it makes them think and feel, in order to optimize content in line with those user preferences.

Preference testing is best used during your initial testing process—not right when you begin, but early enough to understand what works and what doesn’t before you commit to a certain content or communication style. It’s similar to A/B testing—which we’re covering next—with the primary difference being when you should use it. Preference testing should be used on content drafts and wireframes, whereas A/B testing is better suited for live content. Let’s take a look.

Testing content helps me understand the emotion my designs need to convey: more formal, informal, chill, educational, and so on.

Vaida Pakulyte, UX Lead at Technigo

A/B testing

An A/B test involves creating two variations of content and testing them with users on live websites. A/B tests can give you quantitative data on which content performs better.

This type of testing can provide useful metrics, such as the click-through rate (CTR) that allow you to determine the winning variation easily. However, to do well, you need to have a large sample size for statistical significance.

For example, Netflix is well-known for A/B testing artwork for titles to identify which images perform best at getting users to watch videos.

Readability testing

Readability formulas are a computer-run testing tool that provides an analysis of your content. They’re a simple way of testing how easy your copy is to read by prioritizing small words and short sentences.

However, if you’re truly aiming to test your content with real users, then readability formulas might not be the best option. While they may be an attractive solution, especially if you’re on a budget, they can often produce inconsistent results, and they don’t give any actionable feedback.

For better results, use readability tests in combination with the other methods described above.

How to test content in 6 steps

So now you know why you should test content, and what you’re testing for, let’s cover how you can go about testing content.

We’ll break this process into six steps:

  1. Identify goals
  2. Choose your method and create the test
  3. Test the test
  4. Gather test participants
  5. Conduct the test
  6. Analyze results

1. Identify goals

Before you even start planning your test, you need to consider what your objective is. In short, who and what are you testing, and what are your testing goals?

You should know by this point who your intended users are, and what their needs and expectations from your product are. If your product is used by people from a range of demographics, then the product should be tested with users from each group. If your test underrepresents a demographic, then you risk creating content that doesn’t speak to all your personas.

So choose the content that you want to test, create a concrete hypothesis, and you’re ready to start outlining the test.

2. Choose your method and create the test

With the who and why decided, you can then choose the most suitable method for testing and create the test.

How you design a test heavily impacts its result, so constantly questioning why and how you’re testing something is crucial.

Nicole Michaelis, Content Design Lead at Wolt

For example:

  • We want to test if our users can understand how to make in-app purchases. We will use a task-based usability test to see if they can successfully complete tasks with the app.
  • We want to test if our homepage speaks to our target demographic. We will use a highlight test to gauge their feelings about it.
  • We want to see how users perceive our product homepage at first glance. We will use a five-second test to measure this initial impression.

Tip 💡
In your test, don’t ask the users if they 'like' the content. Ask if they could navigate easily, if they understand a paragraph, or if the tone makes them feel comfortable. What matters is whether it’s relevant and comprehensible.

3. Test the test

Once you have created your initial test, you want to see if it makes sense. Before you start finding test participants, grab some colleagues or other stakeholders, and run the test with them first.

Test the test! A mistake I’ve made in the past is making the test and then launching it straight to users. It was then that I discovered issues in the test as I was running it.

Steve Howe, Content Designer

Testing the test will ensure the instructions make sense and that your test participants can successfully understand what is asked of them. This allows them to focus on your content, rather than trying to decipher the test itself.

4. Gather test participants

With your test created, it’s time to find some test users.

If you’re running qualitative sessions, you need to find at least five participants. This number will give you a consensus to inform your content decisions. If you want to collect quantitative data, then it’s best to run the test with at least twenty people, more if possible. Ultimately, how many people you need to recruit depends on the type of test you’re doing, the method, and the criticality of the project.

The big focus here is to find representative users for the real test. Proxy users, such as the people you work with, may not be the people who use your product. You run the risk of getting biased feedback since they may already know information about the product or content, so the data isn’t reliable.

In our guide to finding and recruiting research participants, we break down seven ways to source participants for your user research. For example, you can find candidates online, use research participant recruitment tools, or even reach out to professional networks for feedback.

5. Conduct the test

You now need to decide to run a moderated or unmoderated test. There are advantages to each method.

With a moderated test, you have greater control during the test. Having a facilitator present can provide an extra layer of feedback from the study. Moderating these studies allows you to ask follow-up questions specific to the user and the situation they find themselves in.

If you’re moderating the test, allow enough time for participants to read the content and give you feedback. Unlike traditional moderated usability testing where users complete tasks and talk through their thoughts, content testing may require significant stretches of time when both you and the participant are silent. Make sure to explain to the participants at the start that this is okay.

With unmoderated testing, you don’t need to find someone to act as a moderator for the test. This type of testing is usually done remotely and it allows you to run more tests at a faster rate, even simultaneously, which will give a greater amount of quantitative data for you to analyze. What’s more, it allows you to find users from different parts of the world, which can unlock insights about how people from different countries interact with your content.

6. Analyze results

Once this round of testing is done, you’re now able to analyze the test results. You should have an overall idea of how your piece of content works, as well as specifics to improve on—be it readability, voice, or any other important factor.

Note how we said ‘this round of testing’. That’s because these tests shouldn’t be a one-off—as Nicole puts it: “testing is never complete”. As your product and user base grow and evolve, you need to keep testing to see if your content works and speaks to them.

Example questions for your content test

Several of the above content testing methods involve asking your participants follow-up or mid-test questions, but knowing the right questions to ask can be tricky. Let’s take a look at some example questions, and when to use them.

Goal: Help participants open up about any problems they encountered with the content or suggested changes

Questions to ask:

  • If you could change anything about that information, what would you change?
  • What was easy to understand in the information, and what was more difficult—can you explain why?
  • What did you think about that information?

Goal: Learn whether participants fully understood the content

Questions to ask:

  • Could you please explain the information in your own words?
  • If you had to tell someone what you gathered from the information, what would you say?

Goal: Reveal whether any terminology or jargon used in the content is confusing

Questions to ask:

  • Were there any words throughout the information that you were unsure of?
  • If you were retelling the information to someone else, are there any words you think you would have to explain?

Goal: Understand how participants interpreted the tone of voice of the content

Questions to ask:

  • What are three words you would use to describe how the information made you feel?
  • If this information came from a person, what do you think that person may be like—e.g. their appearance, their job, their demeanor.

Tip 💡 Check out our research question bank of 350+ questions for more ideas

Make content testing a habit in your UX research

Your content is how you communicate with customers. It’s how you inform and engage users throughout their interactions with your product and business. Great content is crucial for intuitive and delightful user experiences, and content testing is how you get there.

Whether you’re working with a first product launch, or a new feature, it’s an imperative step for any of your in-product or on-site content—and key for making sure you meet user wants and needs.

Validate content with real users

Test your content with Maze and guarantee your content is clear, intuitive, and consistent.

Frequently asked questions about content testing

What is content testing?

Content testing is a research method used to evaluate the effectiveness of your content in terms of usability, readability, accessability, and more. Using content testing, you can check if the content created makes sense to users, resonates with their challenges, and provides enough context to help them.

How to test content?

There are six steps to testing content:
1. Identify goals
2. Choose your method and create the test
3. Test the test
4. Gather test participants
5. Conduct the test
6. Analyze results

How do you find content testing participants?

Finding participants for content testing is the same as gathering testers for any kind of research. The easiest and most successful way to find content testing participants is through a participant recruitment tool, like Reach by Maze.

Continue Reading