maze logo

Share

Chapter 4

Tree testing vs. card sorting: Which is best for your product research strategy?

If you’re embarking on a user research project, a frequent stumbling block is deciding between tree testing or card sorting. In this chapter, we’ll get into the nitty-gritty of both research methods, their similarities and differences, and when to use which.

So you’ve got your product concept, your problem statement, perhaps even a basic sketch—now’s the time to kick-off the user research. But should you go with tree testing or card sorting? On paper, they seem similar—don’t they?

What is tree testing?

A tree test—also known as a reverse card sort—is a UX research method used to assess the findability of content on a website or app, and evaluate a proposed site structure.

In a tree test, users are given a simplified text version of the site’s information architecture (IA) in the form of a hierarchical category structure called a tree, then asked to complete a set of tasks with that information.

Their ability to complete the tasks, and the ease at which they succeed, in turn determines how effective the site’s organization and terminology is.

Tree testing should be the starting point for designing better digital applications.

Mario Tedde, Senior UX Researcher at FedEx Express

What is card sorting?

Card sorting is a UX research method where participants group ideas or information into different categories, based on what feels natural to them. There are many types of card sorting, depending on whether the study takes place in person with paper cards, or virtually using a digital card sorting tool.

The purpose of a card sorting session is to understand how your target audience thinks, feels, and understands the information presented to them. Using the results, UX researchers can make informed decisions about website IA, content grouping, and labeling decisions.

Card sorting is a great exercise to understand your users’ mental models and a helpful way to question your biases and validate the information with real people.

Vaida Pakulyte, UX Researcher and Designer at Electrolux

What’s the difference between tree testing and card sorting?

While card sorting shares the same goal of content categorization as tree testing, each method approaches this from a different perspective and is used at various stages during the research process.

Both tree testing and card sorting have a place in UX research, but it’s important to know the right time and context to use each method.

When conducting thorough user research, card sorting and tree testing work efficiently in unison; for example, first gathering ideas on how to structure your IA via an open card sort, then validating that proposed IA through a tree test or closed card sorting.

Similar to closed card sorting, tree testing can be used to evaluate existing IA, so they’re typically used in tandem. Open card sorting is used to generate ideas, followed by a tree test to test the ideas.

Guillermo Gineste, Senior Product Designer at Maze

The key difference between tree testing and card sorting is that tree testing suits itself to a more specific goal. Unlike card sorting, which can generate ideas, assess user mental models, or validate concepts or IA, tree testing’s strength is in its tight focus on evaluating site hierarchy.

The benefits of tree testing benefits in UX research

Tree testing has many benefits, some that align with card sorting and some that are unique. It’s a useful way to review ideas before expending too many resources into designs, and is also an ideal follow-up to open card sorting, as a way to validate the insights you came away with.

Guillermo Gineste, Senior Product Designer at Maze, points out that tree testing allows for more detailed structures to be tested:

Tree testing allows for more than one level of nesting. In closed card sorting, you can only have one level of categorization but in tree tests you can have multiple levels of navigation within themselves.

Guillermo Gineste, Senior Product Designer at Maze

The most common use case for tree testing is to test how IA performs—you can identify any UX issues during a tree test, collect data to benchmark later tests, and uncover where the next design iteration needs to focus.

Some of the key benefits of tree testing are that you can:

  • Test the usability of your navigation
  • Evaluate the effectiveness of your navigation
  • Assess the labeling and language used in your website or app
  • Gather insight into user mental models
  • Validate ideas before designing
  • Prioritize issues for iteration

Improve navigation before it becomes a problem

Don’t just go with the flow—improve it. Increase findability and develop your information architecture with feedback from users, through tree testing with Maze.

How to conduct tree testing

Step 1: Create your research plan

Start by ensuring your site structure (the tree) is ready. Participants should see a text-only version of the site, so double check everything for clarity and decide what results metrics you’ll be measuring.

Tip: We recommend keeping the tree testing sessions between 15-20 minutes with <10 tasks.

Step 2: Define the tree structure

Next, outline the categories, subcategories, and pages you want to present in your test (including wider context of where those areas sit, even if not being tested). This is where it’s crucial to ensure you use the same language and terminology you will on the site—even something as small as a missed apostrophe can change the meaning, so review your labels closely.

Step 3: Plan your tasks and questions

Now’s the time to write your questions and determine the tasks you want participants to complete. Tasks should be focused on locating a certain area within the tree, for example requesting users find a way to upgrade their account or navigate to a certain help page.

It can be challenging to put together a set of effective research questions that are clear, concise and unbiased, but it’s worth taking the time to carefully curate the perfect set of Qs.

For everything you need to know about writing questions, we’d suggest heading to our Survey Question guide. In the meantime, some of our top tips for writing research questions are:

  • Begin generic: It’s helpful to lead with context, so beginning with generic questions then moving to more specific ones helps respondents keep track of information.
  • One question at a time: Avoid confusing participants by over-explaining, giving too many details or asking multiple questions at once.
  • Be clear and concise: Don’t leave room for interpretation. Strike a balance between clarity, precision and consistency—good questions can be understood by anyone.
  • Validate your assumptions: Consider asking questions you think you know the answer to. This can bring up answers you might not have considered otherwise, and you may be surprised at the responses!
  • Use first person: Word your questions from the respondent’s perspective, not your own or a collective’s. This helps participants to feel focused on their own answers.
  • Avoid leading questions: Keep questions neutral. You’re an unbiased party collecting information, so ensure your questions reflect that.

Finding the right question is hard 💡

That's why we created a Question Bank with 350+ questions for you to use, ready to go. Check out the Maze Question Bank here.

Step 4: Recruit participants

Once your test and tasks are prepped, it’s time to find your participants. Ensure you’ve got solid analysis of your target audience, then set out to recruit test users from that base. The number of users you test will depend on several factors, but generally most tree tests should use at least 50 users, so you can identify clear patterns.

Some online tree testing tools, like Maze, allow you to directly recruit participants through the same tool you use to run the tree test.

Step 5: Select the tree testing method

The tree test itself can be conducted in-person or virtually with remote testing tools, like Maze. While in-person testing has the advantage of moderation, remote testing allows for easier and quicker testing.

Get to the root of the problem with tree testing 🌲

Hone in on any navigational challenges before it goes live. Use Tree Testing with Maze to ensure users can find what they need, when—and where—they need it.

Step 6: Conduct a pilot run

The penultimate stage is to conduct a practice run before initiating the real thing. There’s nothing worse than getting halfway through your tree test and realizing one of the tasks was worded unclearly.

Step 7: Run the tree test

Now’s the time to hit ‘go’ on the main test! You may have follow-up questions you’d like to ask via survey, or if you want to test multiple versions of a tree you can use split testing to later compare results.

For an in-depth walkthrough, check out our guide to running a tree test.

Step 8: Analyze the results

After your test is complete, you can start analyzing the results. At this stage, a remote testing tool is invaluable, as it will analyze and organize the results for you. The typical results you look for include: success rate, time to complete, and path taken.

What tree testing reveals about IA

Tree testing reveals multiple useful insights on the findability, navigability and usability of your website or app’s IA. Tree testing can answer questions such as:

  • Is the content grouped logically and in a way users expect?
  • Does the language used and labeling make sense?
  • Can users find the information they want without help?
  • Could areas of navigation be streamlined?

The benefits of card sorting in UX research

Card sorting is a varied research method that can significantly guide design stages by generating new ideas, and development by validating early concepts.

When it comes to tree testing vs. card sorting, they both have their uses.
Card sorting particularly comes into its own with regards to qualitative data, as it allows you to gain insight into why users may group information together or not. Card sorting excels at understanding users, whereas tree testing focuses on validating information.

Some of the key benefits of card sorting include being able to:

  • Reveal your user’s mental models
  • Generate new ideas for organizing content
  • Understand how users approach navigation
  • Learn how people feel about different concepts
  • Uncover how well people use your product

How to conduct card sorting

Step 1: Document your assumptions

Before heading in a card sorting session, you need to define your research goals, including any assumptions you may have about the results. Keep these aside, and approach sessions with an open mind.

Step 2: Choose a topic

Selecting the focus of your card sort should be done based on research objectives and any design priorities. Pick a topic you want to gain a deeper level of insight into.

Take a look at chapter two for a more in-depth walkthrough of how to run a card sort. 👈

Step 3: Prepare the card sort

Time to write your cards—whether you’re working on paper or digitally, you need to write down the content, labels, or ideas onto your cards. Remember to keep an eye out for bias by ensuring each card looks the same, and any words aren’t repeated unless necessary to their meaning.

This is also the opportunity to prepare instructions for participants. Ensure these are clear, such as ‘Look at the list of items on the left. Sort them into groups that make sense to you, there is no right or wrong answer’.

Step 4: Choose your type of test

Before you can begin your card sort, you need to determine what type of card sorting study you need to conduct. This will depend on a few factors, namely your research objectives.

Type What it is Good for
Open card sorting Participants sort the cards into groups then name the groups themselves. • Creating new information architectures or improving existing ones
Finding patterns in how users expect to find content
Generating ideas for how to structure and label your website or app information
Closed card sorting A moderator gives participants predefined categories to sort the cards into. Finding out if users understand your categorizations
Pinpointing misleading categories and make informed decisions on how to improve
Ensuring information is organized in a way that makes sense to your audience.
Learning how users prioritize and rank information within existing categories
Hybrid card sorting Some categories have already been defined for participants to sort the cards into, but participants also can create their own categories. Testing existing IA and website structures to see if they work as expected
When you have some information categorized, but need input for organizing the remainder
If flexibility is a priority: you can give category examples to participants, while allowing them to add their own

Step 5: Conduct a run-through

The next stage is to run an initial test before the main event. Practice the session with a colleague to double check everything makes sense and—if you’re using an online tool—works as it should.

Step 6: Launch the test

Now’s the exciting part! Whether you choose to go with an open, closed, or hybrid card sort, users will start sorting cards into the categories that make the most natural sense to them.

Step 7: Hold follow-up interviews

Once the session is over, most researchers choose to run follow-up interviews. If you conducted a moderated card sort, this may feel less necessary, but additional qualitative information is always useful when later analyzing the quantitative data.

Step 8: Analyze the results

Finally, it’s time to get into the depths of the data. Your results will vary based on the type of study, but results to consider include: outlier cards or categories, groups that were most popular, and similar categories with different names.

Take a look at chapter two for more guidance on analyzing card sorting results.

What card sorting reveals about IA

Card sorting offers a wide breadth and depth of understanding into the way users think. Some of the questions card sorting can answer include:

  • How do users organize content in a way that makes sense to them?
  • What names should be given to certain categories?
  • What categories should go where in a site hierarchy?

Which is best for your UX research strategy: tree testing or card sorting?

Card sorting helps you organize information, while tree testing helps you understand if the way the information is organized makes sense to the user. Tree testing comes in when you already know how the information architecture should be, card sorting will help define said architecture.

Anca Croitoru, UX Researcher at New Relic

As Anca says, it’s helpful to use both UX research methods in tangent; using card sorting as a way to understand your user and define the IA, followed by tree testing to validate your changes based on card sorting feedback.

Deciding whether you want to use tree testing or card sorting depends on a variety of factors, primarily your research goals and desired outcome.

Goal Card sorting Tree testing
Test the usability of current or proposed IA ✔️ ✔️
Assess labeling and language ✔️ ✔️
Learn user mental models ✔️ ✔️
Validate and prioritize ideas ✔️
Generate categories and structure ideas ✔️
Evaluate findability of content ✔️
Understand how users feel about different concepts ✔️
Reveal how intuitive your product is ✔️

Regardless of your chosen research method, the key steps remain the same: studying your objectives, preparing carefully, and running a thoughtful session with the user at the center.

That’s it for this chapter; next up we’ll be running through our top-rated card sorting tools to get you started.

Get started with our card sorting template

Gain a better understanding of your user and assess the effectiveness of your product with this free closed card sorting template from Maze.

Frequently asked questions about card sorting vs. tree testing

What is UX card sorting?

UX card sorting is the practice of running a card sorting session as part of UX research. In a card sorting session, an individual or group of people are given information on cards—physical or digital—and must group them into different categories that feel right for them.

Is card sorting good for UX research?

Card sorting can be a valuable tool for UX research, as it allows researchers and product designers to understand their users’ mental models, generate ideas, and evaluate information architecture.

What are the disadvantages of card sorting?

Card sorting is a great UX research method for many reasons, but like all research methods, it has some disadvantages.

If you want to understand how usable your product is or whether the categories set help users achieve their goals, then tree testing is the better research method.

The other drawback of card sorting is that it has less focus on context than other research methods. While card sorting is great for understanding how users group content, people may group things differently depending on the context of product type and situation, e.g. while in a card sort people may group ‘tomato’ in the ‘fruit’ group, when shopping in a grocery store, they may group ‘tomato’ with ‘vegetables’.