How to identify and avoid leading questions in UX research

If you’ve ever been asked, “How satisfied were you with [product]?” then you’ve experienced a leading question. This question assumes that you had a satisfactory experience (“how satisfied”) and primes you to think positively about it, which can end up influencing your reply.

That is the essence of leading questions–they can strongly influence how participants answer them based on their structure and words. As a result, they impact your results and affect what your team decides to prioritize in product development.

Whenever we do any research, whether it's user interviews, customer satisfaction surveys, or field research, we should keep our assumptions in check and ask better questions.

In this article, we’ll look at what leading questions are, how they influence UX research, identify four kinds of leading questions you should know about, and share actionable advice on how to avoid asking them.

Leading questions are questions or statements that contain assumptions or affirmations and encourage respondents toward a specific answer or outcome. When using leading questions, we make inferences about people's feelings or experiences, and as a result, collect biased results.

We might not notice them but leading questions are quite present in our daily lives. For example, when we have a nice dinner out, we'll ask our dining partner 'Wasn't that a good meal?'

To be completely neutral and not influence their response, we should instead ask‘What did you think of the meal?’. In these kinds of situations, there's not a significant negative effect to using leading questions—it's completely normal to frame the question that way.

Unfortunately, in other instances, leading questions are more harmful. For example, detectives or police officers may ask suggestive or assumptive questions to prompt witnesses and receive false testimonies in a cross-examination. This is also known as suggestive interrogation. If you've seen the Netflix series Making a Murderer, you'll be familiar with the dangers that using leading questions contains.

In UX research, leading questions impact the accuracy of results and what a team prioritizes to build. If you’re running user interviews or feedback surveys with leading questions, you can get false feedback—either too positive or negative feedback unrepresentative of people’s actual lived experiences.

Leading questions are often caused by UX cognitive biases, like the framing effect, where the way a question is presented (either positively or negatively) impacts how someone responds.

A frequently used example of a positively-framed, leading question is ‘How easy was this product to use?’. This question instantly assumes the product is easy to use.

Positively-framed questions over-index on delightful experiences, leaving no room for people to share what didn't go well or what they disliked. When research questions are framed incorrectly, it leads to missed opportunities to learn how to improve your product, which is the inherent intent of user research.

As UX practitioners, we should make our participants feel like they can share all types of feedback, whether positive or negative. We must allow them to do so by asking open-ended, non-biased questions that focus on actual experiences, not assumptions.

The ease of use question above makes an assumption based on how someone experienced a product. Other types of leading questions make assumptions about people’s feelings or emotions, how they compare to others, and the cause-and-effect of their future actions. Let’s look at four examples of the different types of leading questions.

These are questions that make assumptions about how others feel, what they’ve done or will do. These questions are usually framed in a positive light and achieve predetermined responses.

Examples of assumptive questions:

How to reframe assumption-based leading questions:

The reframed questions remove the assumptions that they enjoyed, liked, or disliked the product and instead prompt respondents to share more about their experience. When asking people to recall and walk you through their experiences, you will naturally learn what went well (or what they liked) and what didn’t (or what they disliked) from their responses.

The order in which you ask questions is also critical to avoiding leading questions. First, confirm that respondents had the experiences you’re asking them to speak about, then ask a neutral question about their experiences.

These questions share an assumptive statement and then ask the respondent for feedback on the assumed experience. These questions can easily result in completely false answers due to response bias and sometimes even FOMO (fear of missing out).

Examples of statement-based leading questions:

How to reframe statement-based leading questions:

The reframed questions remove the leading statements to help avoid agreeable answers. The first example promotes breakfast as the most preferred meal and makes it easy for the respondent to say ‘yes.’ The second question shares a belief that is apparently held by the group's majority, which may make it difficult for the respondent to disagree because people want to appear socially desirable to others.

These questions begin with a seemingly factual statement and then ask you to confirm if it's the truth. These are the types of questions you will often see in legal interrogations. For example, questions related to whereabouts, such as ‘You were at the grocery store at 8 PM that evening, so you most likely saw the robber, right?’.

Other examples of coercive questions:

How to reframe coercive questions:

Similar to statement-based questions, coercive questions are loaded with assumptions. The use of affirmative language such as “yes?” “right?” nudges the respondent to agree.

The more particular and personal the assumption is, the more difficult it can be to disagree and doubt its validity. These leading questions produce predetermined responses, leaving no opportunity to collect data about the actual context of respondents' experiences.

These questions ask people to predict their behavior and future events. Questions that require users to make predictions about their future behavior are ineffective because we’re not good at predicting our behavior reliably.

Examples of consequential leading questions:

How to reframe consequential questions:

Framing questions based on current or existing experiences that people have had will produce more accurate results. For example, the first leading question above can be redesigned into a multi-select question:

Based on your previous experience(s) shopping with us, please share what contributes to you visiting our store:

This allows the person to share why they visited the store, which yields more useful information than just asking if they will return or not.

Remove adjectives related to feelings, such as how much people like or dislike something. Instead, ask them to share their experience and naturally uncover what works and what is challenging.

Be mindful of where and when you ask questions, so you don’t assume people have had experiences. A good rule of thumb is to start broad and then get specific. For example, start by uncovering if someone uses a particular feature and then ask specific questions related to that feature—don’t make assumptions right off the bat.

Create space for any type of feedback—positive and negative—and make participants feel safe to share their opinions. Remind people that there are no right or wrong answers and that you are not testing them—you’re testing the product.

Asking your questions out loud with someone not part of your company or team is a great way to identify if you’ve accidentally created a leading question. Call on a friend, family member, or even someone at the company who is unfamiliar with the product area to do a mock interview or run through your survey.

Leading questions are pretty common in everyday life—whether we're the respondent or the interviewer. But while asking someone to confirm they enjoyed their meal is no big deal—using leading questions in product research leads you to a path of assumptions and misinformed product roadmaps.

The next time you prepare for your next user interview, keep these best practices in mind and practice framing questions in a neutral, unbiased way. The more experience you have with crafting effective research questions, the easier it will become to spot leading ones.

That was pretty helpful, wasn't it? 😉