Feb 15, 2022

How to identify and avoid leading questions in UX research

Understand what leading questions are, how they influence UX research, and learn about the four types of leading questions you should know about.

If you’ve ever been asked, “How satisfied were you with [product]?” then you’ve experienced a leading question. This question assumes that you had a satisfactory experience (“how satisfied”) and primes you to think positively about it, which can end up influencing your reply.

That is the essence of leading questions–they can strongly influence how participants answer them based on their structure and words. As a result, they impact your results and affect what your team decides to prioritize in product development.

Whenever we do any research, whether it's user interviews, customer satisfaction surveys, or field research, we should keep our assumptions in check and ask better questions.

In this article, we’ll look at what leading questions are, how they influence UX research, identify four kinds of leading questions you should know about, and share actionable advice on how to avoid asking them.

What are leading questions?

Leading questions are questions or statements that contain assumptions or affirmations and encourage respondents toward a specific answer or outcome. When using leading questions, we make inferences about people's feelings or experiences, and as a result, collect biased results.

We might not notice them but leading questions are quite present in our daily lives. For example, when we have a nice dinner out, we'll ask our dining partner 'Wasn't that a good meal?'

To be completely neutral and not influence their response, we should instead ask‘What did you think of the meal?’. In these kinds of situations, there's not a significant negative effect to using leading questions—it's completely normal to frame the question that way.

Unfortunately, in other instances, leading questions are more harmful. For example, detectives or police officers may ask suggestive or assumptive questions to prompt witnesses and receive false testimonies in a cross-examination. This is also known as suggestive interrogation. If you've seen the Netflix series Making a Murderer, you'll be familiar with the dangers that using leading questions contains.

How leading questions affect UX research

In UX research, leading questions impact the accuracy of results and what a team prioritizes to build. If you’re running user interviews or feedback surveys with leading questions, you can get false feedback—either too positive or negative feedback unrepresentative of people’s actual lived experiences.

Leading questions are often caused by UX cognitive biases, like the framing effect, where the way a question is presented (either positively or negatively) impacts how someone responds.

A frequently used example of a positively-framed, leading question is ‘How easy was this product to use?’. This question instantly assumes the product is easy to use.

When research questions are framed incorrectly, it leads to missed opportunities to learn how to improve your product, which is the inherent intent of user research.

Positively-framed questions over-index on delightful experiences, leaving no room for people to share what didn't go well or what they disliked. When research questions are framed incorrectly, it leads to missed opportunities to learn how to improve your product, which is the inherent intent of user research.

As UX practitioners, we should make our participants feel like they can share all types of feedback, whether positive or negative. We must allow them to do so by asking open-ended, non-biased questions that focus on actual experiences, not assumptions.

The ease of use question above makes an assumption based on how someone experienced a product. Other types of leading questions make assumptions about people’s feelings or emotions, how they compare to others, and the cause-and-effect of their future actions. Let’s look at four examples of the different types of leading questions.

4 Main types of leading questions

1. Assumptive leading questions

These are questions that make assumptions about how others feel, what they’ve done or will do. These questions are usually framed in a positive light and achieve predetermined responses.

Examples of assumptive questions:

  • How much do you enjoy [product]?
  • What did you like and dislike about our product?
  • Why do you prefer our product over [competitor product]?
  • How often do you run?

How to reframe assumption-based leading questions:

  • Please walk us through your experience when you did [x] with our product
  • Recall a time when you used our product for [x] and share that experience
  • First, ask: Do you partake in any physical exercise? Then, ask: On average, how many times per week do you engage in physical exercise?

The reframed questions remove the assumptions that they enjoyed, liked, or disliked the product and instead prompt respondents to share more about their experience. When asking people to recall and walk you through their experiences, you will naturally learn what went well (or what they liked) and what didn’t (or what they disliked) from their responses.

The order in which you ask questions is also critical to avoiding leading questions. First, confirm that respondents had the experiences you’re asking them to speak about, then ask a neutral question about their experiences.

2. Statement-based leading questions

These questions share an assumptive statement and then ask the respondent for feedback on the assumed experience. These questions can easily result in completely false answers due to response bias and sometimes even FOMO (fear of missing out).

Examples of statement-based leading questions:

  • Our previous feedback survey showed that most people prefer breakfast as their favorite meal. Do you agree?
  • Finance organizations report that many employees work overtime. Do you work overtime?

How to reframe statement-based leading questions:

  • If you had to choose just one, which meal do you most prefer: breakfast, lunch, or dinner?
  • What are your thoughts on working overtime?

The reframed questions remove the leading statements to help avoid agreeable answers. The first example promotes breakfast as the most preferred meal and makes it easy for the respondent to say ‘yes.’ The second question shares a belief that is apparently held by the group's majority, which may make it difficult for the respondent to disagree because people want to appear socially desirable to others.

3. Coercive leading questions

These questions begin with a seemingly factual statement and then ask you to confirm if it's the truth. These are the types of questions you will often see in legal interrogations. For example, questions related to whereabouts, such as ‘You were at the grocery store at 8 PM that evening, so you most likely saw the robber, right?’.

Other examples of coercive questions:

  • We can see that the last time you logged into your profile was on March 7th. That’s correct, right?
  • You’ll provide me with a 5-star rating once I drop you off, yes?
  • Our services met your needs, correct?

How to reframe coercive questions:

  • Can you recall the last time you logged into your profile?
  • After I drop you off, could you please rate your experience today?
  • Please share if our services met your needs or not.

Similar to statement-based questions, coercive questions are loaded with assumptions. The use of affirmative language such as “yes?” “right?” nudges the respondent to agree.

The more particular and personal the assumption is, the more difficult it can be to disagree and doubt its validity. These leading questions produce predetermined responses, leaving no opportunity to collect data about the actual context of respondents' experiences.

4. Consequential leading questions

These questions ask people to predict their behavior and future events. Questions that require users to make predictions about their future behavior are ineffective because we’re not good at predicting our behavior reliably.

Examples of consequential leading questions:

  • If you found what you were looking for today, will you come back and shop with us again?
  • Imagine our product helped you save more money. Would you open another account with us?

How to reframe consequential questions:

Framing questions based on current or existing experiences that people have had will produce more accurate results. For example, the first leading question above can be redesigned into a multi-select question:

Based on your previous experience(s) shopping with us, please share what contributes to you visiting our store:

  • The staff
  • Cost of goods
  • Selection of goods
  • Quality of goods
  • Location of the shop
  • Other (please specify)

This allows the person to share why they visited the store, which yields more useful information than just asking if they will return or not.

How to avoid leading questions

Access 350+ ready-to-use research questions 🙌

Get your research up and running with our Question Bank of hundreds of research questions ready for you to use.

Avoid words related to feelings and/or sentiment

Remove adjectives related to feelings, such as how much people like or dislike something. Instead, ask them to share their experience and naturally uncover what works and what is challenging.

Be mindful of the order of your questions

Be mindful of where and when you ask questions, so you don’t assume people have had experiences. A good rule of thumb is to start broad and then get specific. For example, start by uncovering if someone uses a particular feature and then ask specific questions related to that feature—don’t make assumptions right off the bat.

Create an open environment

Create space for any type of feedback—positive and negative—and make participants feel safe to share their opinions. Remind people that there are no right or wrong answers and that you are not testing them—you’re testing the product.

Run through your questions with someone

Asking your questions out loud with someone not part of your company or team is a great way to identify if you’ve accidentally created a leading question. Call on a friend, family member, or even someone at the company who is unfamiliar with the product area to do a mock interview or run through your survey.

Conclusion

Leading questions are pretty common in everyday life—whether we're the respondent or the interviewer. But while asking someone to confirm they enjoyed their meal is no big deal—using leading questions in product research leads you to a path of assumptions and misinformed product roadmaps.

The next time you prepare for your next user interview, keep these best practices in mind and practice framing questions in a neutral, unbiased way. The more experience you have with crafting effective research questions, the easier it will become to spot leading ones.

That was pretty helpful, wasn't it? 😉

Frequently asked questions

How can I identify a leading question?

To identify leading questions, review the questions you prepared and ask yourself if they focus on a particular sentiment (e.g., like or dislike), or if they are balanced and neutral. Indicators of leading questions include affirmative language such as ‘right?’ ‘wouldn’t you?’ ‘isn’t that true?’ and so on; sweeping statements that assume an experience actually happened are usually followed by leading questions; and questions that ask participants to predict future behavior.

What are some examples of leading questions?

There are four types of leading questions you should be aware of, from assumptive questions to statement-based, coercive, and consequential questions. Examples of leading questions are: ‘How much do you enjoy using [product]?’ ‘What did you like and dislike about [x]?’ ‘Our services met your needs, right?’

How can I avoid asking a leading question?

To avoid asking leading questions, don’t use words related to feelings and/or sentiment in your questions. Be mindful of the order of your questions, and create an open environment, encouraging your UX research participants to share their thoughts. A best practice is to run through the questions with someone before using them in real-life research.

Stay in the loop

Sign up for our monthly newsletter to get notified of new resources on research and testing.