Tree testing vs. card sorting: Which is best for your product research strategy?

So you’ve got your product concept, your problem statement, perhaps even a basic sketch—now’s the time to kick-off the user research. But should you go with tree testing or card sorting? On paper, they seem similar—don’t they?

A tree test—also known as a reverse card sort—is a UX research method used to assess the findability of content on a website or app, and evaluate a proposed site structure.

In a tree test, users are given a simplified text version of the site’s information architecture (IA) in the form of a hierarchical category structure called a tree, then asked to complete a set of tasks with that information.

Their ability to complete the tasks, and the ease at which they succeed, in turn determines how effective the site’s organization and terminology is.

Card sorting is a UX research method where participants group ideas or information into different categories, based on what feels natural to them. There are many types of card sorting, depending on whether the study takes place in person with paper cards, or virtually using a digital card sorting tool.

The purpose of a card sorting session is to understand how your target audience thinks, feels, and understands the information presented to them. Using the results, UX researchers can make informed decisions about website IA, content grouping, and labeling decisions.

While card sorting shares the same goal of content categorization as tree testing, each method approaches this from a different perspective and is used at various stages during the research process.

Both tree testing and card sorting have a place in UX research, but it’s important to know the right time and context to use each method.

When conducting thorough user research, card sorting and tree testing work efficiently in unison; for example, first gathering ideas on how to structure your IA via an open card sort, then validating that proposed IA through a tree test or closed card sorting.

The key difference between tree testing and card sorting is that tree testing suits itself to a more specific goal. Unlike card sorting, which can generate ideas, assess user mental models, or validate concepts or IA, tree testing’s strength is in its tight focus on evaluating site hierarchy.

Tree testing has many benefits, some that align with card sorting and some that are unique. It’s a useful way to review ideas before expending too many resources into designs, and is also an ideal follow-up to open card sorting, as a way to validate the insights you came away with.

Guillermo Gineste, Senior Product Designer at Maze, points out that tree testing allows for more detailed structures to be tested:

The most common use case for tree testing is to test how IA performs—you can identify any UX issues during a tree test, collect data to benchmark later tests, and uncover where the next design iteration needs to focus.

Some of the key benefits of tree testing are that you can:

Start by ensuring your site structure (the tree) is ready. Participants should see a text-only version of the site, so double check everything for clarity and decide what results metrics you’ll be measuring.

Tip: We recommend keeping the tree testing sessions between 15-20 minutes with <10 tasks.

Next, outline the categories, subcategories, and pages you want to present in your test (including wider context of where those areas sit, even if not being tested). This is where it’s crucial to ensure you use the same language and terminology you will on the site—even something as small as a missed apostrophe can change the meaning, so review your labels closely.

Now’s the time to write your questions and determine the tasks you want participants to complete. Tasks should be focused on locating a certain area within the tree, for example requesting users find a way to upgrade their account or navigate to a certain help page.

It can be challenging to put together a set of effective research questions that are clear, concise and unbiased, but it’s worth taking the time to carefully curate the perfect set of Qs.

For everything you need to know about writing questions, we’d suggest heading to our Survey Question guide. In the meantime, some of our top tips for writing research questions are:

Once your test and tasks are prepped, it’s time to find your participants. Ensure you’ve got solid analysis of your target audience, then set out to recruit test users from that base. The number of users you test will depend on several factors, but generally most tree tests should use at least 50 users, so you can identify clear patterns.

Some online tree testing tools, like Maze, allow you to directly recruit participants through the same tool you use to run the tree test.

The tree test itself can be conducted in-person or virtually with remote testing tools, like Maze. While in-person testing has the advantage of moderation, remote testing allows for easier and quicker testing.

The penultimate stage is to conduct a practice run before initiating the real thing. There’s nothing worse than getting halfway through your tree test and realizing one of the tasks was worded unclearly.

Now’s the time to hit ‘go’ on the main test! You may have follow-up questions you’d like to ask via survey, or if you want to test multiple versions of a tree you can use split testing to later compare results.

After your test is complete, you can start analyzing the results. At this stage, a remote testing tool is invaluable, as it will analyze and organize the results for you. The typical results you look for include: success rate, time to complete, and path taken.

Tree testing reveals multiple useful insights on the findability, navigability and usability of your website or app’s IA. Tree testing can answer questions such as:

Card sorting is a varied research method that can significantly guide design stages by generating new ideas, and development by validating early concepts.

When it comes to tree testing vs. card sorting, they both have their uses.
Card sorting particularly comes into its own with regards to qualitative data, as it allows you to gain insight into why users may group information together or not. Card sorting excels at understanding users, whereas tree testing focuses on validating information.

Some of the key benefits of card sorting include being able to:

Before heading in a card sorting session, you need to define your research goals, including any assumptions you may have about the results. Keep these aside, and approach sessions with an open mind.

Selecting the focus of your card sort should be done based on research objectives and any design priorities. Pick a topic you want to gain a deeper level of insight into.

Time to write your cards—whether you’re working on paper or digitally, you need to write down the content, labels, or ideas onto your cards. Remember to keep an eye out for bias by ensuring each card looks the same, and any words aren’t repeated unless necessary to their meaning.

This is also the opportunity to prepare instructions for participants. Ensure these are clear, such as ‘Look at the list of items on the left. Sort them into groups that make sense to you, there is no right or wrong answer’.

Before you can begin your card sort, you need to determine what type of card sorting study you need to conduct. This will depend on a few factors, namely your research objectives.

The next stage is to run an initial test before the main event. Practice the session with a colleague to double check everything makes sense and—if you’re using an online tool—works as it should.

Now’s the exciting part! Whether you choose to go with an open, closed, or hybrid card sort, users will start sorting cards into the categories that make the most natural sense to them.

Once the session is over, most researchers choose to run follow-up interviews. If you conducted a moderated card sort, this may feel less necessary, but additional qualitative information is always useful when later analyzing the quantitative data.

Finally, it’s time to get into the depths of the data. Your results will vary based on the type of study, but results to consider include: outlier cards or categories, groups that were most popular, and similar categories with different names.

Take a look at chapter two for more guidance on analyzing card sorting results.

Card sorting offers a wide breadth and depth of understanding into the way users think. Some of the questions card sorting can answer include:

As Anca says, it’s helpful to use both UX research methods in tangent; using card sorting as a way to understand your user and define the IA, followed by tree testing to validate your changes based on card sorting feedback.

Deciding whether you want to use tree testing or card sorting depends on a variety of factors, primarily your research goals and desired outcome.

Regardless of your chosen research method, the key steps remain the same: studying your objectives, preparing carefully, and running a thoughtful session with the user at the center.

That’s it for this chapter; next up we’ll be running through our top-rated card sorting tools to get you started.