Chapter 8
Tree testing: The guide to an effortless user experience
In the previous chapter, we talked about how you can conduct a card sorting session to solve issues related to your information architecture (IA). Card sorting is typically complemented by tree testing to validate the identified categories. In this chapter, we delve into this research method to learn how it can help you create a better user experience.
What is tree testing?
Tree testing is a UX research method that allows you to evaluate the hierarchy and findability of topics in a website or app. It's an important step in your research plan. In a tree test, participants are presented with a text-only version of the site's hierarchy and asked to complete a series of tasks. The ultimate goal boils down to answering the question, “Can users find what they are looking for?”
Tree testing is a highly valuable exercise to get a clear view of what real users expect as topics in the navigation of a website and how these topics are clustered from primary to secondary. Tree testing should be the starting point for designing better digital applications.
Mario Tedde, Senior UX Researcher at FedEx Express
Sometimes described as "reverse card sorting," tree testing evaluates a hierarchical category structure called the tree. By getting participants to find the locations in the tree and asking them where they would click, you can find insights about the order of the topics and the effort to accomplish a goal.
The benefits of tree testing in UX
Tree testing is an excellent method to evaluate the effectiveness of your website or app’s navigation hierarchy early on in the design process. The data collected from a tree test helps you understand how users navigate your site so you can better organize your content.
Some of the key benefits of tree testing are that you can:
- Test the usability of your navigation
- Evaluate the effectiveness of your navigation
- Assess the labeling and language used in your website or app
- Gather insight into user mental models
- Validate ideas before designing
- Prioritize issues for iteration
Tree testing helps you gather insight into people’s mental models of a product and how they would naturally think about exploring it.
Melanie Buset, Senior UX Researcher at Shopify
Pioneered by design leader Donna Spencer, tree testing has been performed initially on paper using index cards. In that respect, tree testing is very similar to paper prototyping. Both methods are easy to do and give you actionable insights to improve the UX of your product.
Tree testing examples
Before we get into how to conduct your own tree test, let’s take a look at some examples of how you can use tree testing:
- A TV streaming service may task participants with locating where they can upgrade their account plan to add more devices.
- An online grocery store could ask users to find how to change their delivery time slot or add a new payment method.
- A bus service might set participants the goal of finding where they can see the next bus stopping at their local station.
- A tech company may request users navigate to the settings page of their laptop and find where to change their screensaver.
I think many designers underestimate the complexity of navigation design. Even if you already have a standard pattern that you’re leveraging, the information design can make or break your navigation.
Sinead Davis Cochrane, UX Manager at Workday
Now that you know why tree testing is important, let's explore how you can use this method to create better user experiences.
What is tree testing used for?
Tree testing is used to assess the findability, labeling, and information architecture of a website or app. With tree testing, you can identify navigation issues early on and make improvements to ensure your users can quickly find the information they need.
A tree test can help you validate ideas before designing and can be an excellent follow-up to card sorting. While card sorting gives you insights into how people group and label ideas, tree testing allows you to test and validate the information architecture.
In particular, tree testing can help you answer questions like:
- Do users understand how to navigate your site based on the labels provided?
- Is your content grouped logically?
- Can users find the information they want easily and quickly?
- If not, what's stopping them from finding the information they need?
In the next sections, we run you through a step-by-step process of how to prepare and conduct a tree testing session.
How to conduct a tree test
Step 1: Create a user research plan and prepare your tree testing questions
As with any UX research method, the first step to running a tree test is to create a research plan and align with stakeholders on the objectives of the research. Plus, defining the research questions and communicating the timeline to the team are also key.
Make sure that everyone is on board and understands what the test implies. For example, if the results come back and show that the current IA is not working, you should be able to allot enough time to make the appropriate changes and, maybe even test again.
Melanie Buset, Senior UX Researcher at Shopify
To conduct a tree test, start by getting your site structure ready, create tasks for your participants, and define the key metrics you’ll record to analyze the data gathered.
Keep in mind that during the tree testing session, only the text version of the site is given to your participants, who are asked to complete tasks to locate particular items on the site. It’s recommended that you keep these sessions short ranging from 15 to 20 minutes and ask participants to complete no more than 10 tasks.
Step 2: Define the tree structure
Begin by outlining the tree structure with the categories, subcategories, and pages in your site or app. Being specific about your subcategories is important because it will prompt realistic user behavior. For instance, a category in the navigation could be called Resources. Respectively, the menu structure for subcategories can be Blog, Help Center, Guides.
Even if you want to test a particular area in your product, make sure your target audience understands how that area relates to the product as a whole. This will enable you to get actionable information you can act on when reviewing your results.
Step 3: Come up with a set of goal-based tasks
Next, create tasks for participants to find a page or location in a tree using a top-down approach. Just like in usability testing, writing good tasks is key when doing a tree test study.
For example, if you want to test the discoverability of the upgrade page in your product, you can create a task that asks participants to find the best way to upgrade the product. Here’s an example of a tree test task:
“You’ve signed up to a 7-day free trial for a budgeting app. You’ve enjoyed using the app and want to upgrade your account. Find how to do that.”
NNgroup recommends that for each task you write, you should also define the right answers that correspond to where the information is located within the tree, so you can automatically calculate success rates for each task.
Other best practices for writing great tasks include making tasks actionable, setting a scenario, and avoiding giving precise instructions to avoid bias.
Tip ✨
When phrasing a task, it's best to avoid matching keywords in the tree.
Typically, tree testing shouldn’t take longer than 20 minutes and it should include up to 10 tasks. For example, Melanie from Shopify shares that for the product she works on, her team had to test the taxonomy structure of a list of services so they wanted to understand what path people would take when navigating different categories of services.
Here’s the task they created:
“Imagine you wanted to hire someone to help set up your business on Shopify; please find which service you’d have to hire for.”
Step 4: Recruit participants
When you’re preparing a tree test, an important element to take into consideration is the participants you will work with. The number of participants depends on a variety of factors like the type of testing you are conducting, your product’s target group, the confidence level you need, and the goal of your project.
Melanie Buset, Senior UX Researcher at Shopify, recommends using at least 50 users when running a tree test so you can identify user behavior trends and clear patterns. She says:
Usually, a good rule of thumb is, once you start seeing pretty clear patterns emerge, then you’ve got enough participants. In my experience, having about 50 participants for tree testing is when you start to see these patterns clearly. It really doesn’t hurt to have more than 50, but I’d say aim for 50 minimum if possible. This also depends on the complexity of the problem you’re dealing with and what needs to be tested.
Melanie Buset, Senior UX Researcher at Shopify
The key thing for selecting the right participants is to spend time understanding your target audience and identifying who will be the most impacted if you were to make changes to your design.
For example, Mario Tedde, Senior UX Researcher at FedEx Express explains: “Let’s assume that you want to design a website that is going to be used by different types of personas such as psychiatrists, tutors, parents, and young adults that need mental care. If you decide to make one website that will fit all their needs, you need to consider performing tree testing with participants that represent all these personas.”
You can use tree testing with a high volume of participants since only a short amount of time is required from each user to complete the test. This is especially easy when running remote, unmoderated testing.
Step 5: Select the tree testing method
Tree testing can be run in-person or remotely using remote online tools such as Maze.
With in-person, moderated testing, the advantage is that you can ask participants why they made certain choices. Mario says:
Moderated tree tests give you the opportunity to figure out the why behind a participant’s actions and identify the rationale behind their decisions. To avoid biased answers, I let the participant do the exercise in silence and only start asking questions when they have completed the task.
Mario Tedde, Senior UX Researcher at FedEx Express
However, remote testing is advantageous because of its ease of use and speed. You will only need a web browser for testers to participate and they can do so anywhere and any time, without you being present.
Step 6: Conduct a pilot run
An important step in the process is organizing a pilot test before your official tree test session to see if the test makes sense and works as expected.
Do a pilot test and practice with your team. Once you set up the actual tree test, ask someone to go through the tasks as a real participant would to ensure things will run smoothly.
Melanie Buset, Senior UX Researcher at Shopify
Atlassian recommends doing this by opening up the study to a small portion of participants from your panel. This approach will help you mitigate the risk of missing important details, adjust your instructions, and get more valuable insights for future sessions.
Pilot runs are useful as they bring new perspectives to the study. You will be able to find out what’s missing or what’s confusing and be prepared for the actual session.
Step 7: Running a tree test
This step is fairly straightforward. If you choose to conduct tree testing as a remote, unmoderated study, the testing tool will give you a link to the test that you can send to participants.
You can also follow-up the tree test with survey questions that participants will answer after or before they complete the tasks. The questions can help supplement the user research data you need to know about the participants, such as demographic information or their familiarity with the product.
Once all the participants have completed the test, you can start analyzing the results and making informed design decisions. If you want to test different versions of a tree—say a new version of a tree with the existing one, you can run split testing and compare the results of the new tree to the old version.
Step 8: Understanding tree testing results
After participants complete a tree test, the results will be recorded in the tool you are using, allowing you to start analyzing them (see how to analyze results in Maze). Typically, the metrics you can analyze for a tree test include success rate, directness, average time to complete a task, and the path taken by users.
- Success rate: the percentage of users that completed the task
- Directness: the percentage of participants who completed the task without hesitation and got the correct answer the first time
- Time: the time it took participants to finish a task
- Path: the routes participants took up and down the tree before selecting an answer
Once you ask your participants to go through the test, the tool will highlight which entries went down the correct path(s) and which didn’t. Seeing where people went off the ‘ideal path’ will help you identify where the navigation issues are within your product.
Melanie Buset, Senior UX Researcher at Shopify
These results often tie back to questions like:
- “Where did participants think they’d find your content?”
- “Did they find the navigation or wording confusing?”
- “What paths did they use first?”
- “Which was the place where they backed up to try a different path?”
- “How long did it take them?”
By analyzing the accumulated data, you can validate or invalidate your hypotheses and design a navigation that makes sense to users.
The disadvantages of tree testing
Just as with all UX research methods, tree testing has some limitations. One of the benefits of tree testing is its tight focus on evaluating IA and navigation, but this can also be a disadvantage.
Tree testing tests top-down organization of a site and labeling of features, but this means you won’t get any feedback on other areas of navigation or overall usability of the site, as you may with other research methods.
However, this can be easily mitigated by supplementing tree testing with additional types of research, such as card sorting or basic usability testing, to build a wider picture of any issues with your site’s design and structure.
The other main downfall of tree testing is that it often works best as a remote, unmoderated study. This does, however, mean that you miss out on qualitative data such as comments users may make while conducting the test, or wider context for how users approach the task at hand.
Ideally, a tree test would be one of many research methods used in conjunction to get a broad overview of all areas of your design. If you aren’t able to conduct any further research, we recommend compensating for this by including follow-up questions after your tree test or simply asking users to share further comments and feedback.
In the next chapter, we go over the five-second testing research method, which you can use to measure how well a design communicates a message.