Survey design principles to keep in mind

Let’s run you through the most important principles to keep in mind when creating surveys to ensure you keep participants engaged and get the information you need.

To keep participants engaged in your survey, it’s a good idea to keep in mind some of the key principles of survey design, like mixing up the survey question types, keeping the language consistent, and keeping your survey concise. Let’s run through the most important principles one by one.

Always start with a research (survey) plan to align on the goals of the survey and what research questions you hope to answer with it.

Surveys are great for gathering thoughts and opinions but not so much for understanding behaviors and sometimes make it difficult to dig deeper into insights. If your team is interested in collecting behavioral data, methods such as usability tests or quantitative tests are a better fit.

Like any other research method, always test your plan with someone before launch to ensure there are no confusing parts.

Avoid bombarding your participants with surveys; make sure someone keeps track of when they get sent and how often. For example, if you run large customer satisfaction surveys, you should limit this to once a quarter at the most. For feature-specific surveys, try to send these to different customer segments at intervals to avoid bombarding the same people over and over again.

Highlight the intention and goals of the survey so the respondents feel like there's something in it for them to give you their time.

If you plan to send a similar survey in the future, reuse the same questions and track the responses over time to see if anything has improved. This is a great way to gauge if any product changes have been improving or declining user satisfaction.

Similarly, keep the tone and branding consistent. For example, if you’re using the first person in one survey question, use the verb form in all questions.

Don’t ask for demographic or very personal information unless it’s absolutely necessary. Depending on the questions being asked, some answers may make people uncomfortable or feel as though there is now a power imbalance between you and them. You never want people to feel inferior to you when you’re trying to make their lives easier.

If you work for a product company that requires some kind of sign-up, you can usually ask your data team to pull this information from the database rather than asking the participant to tell you in a survey.

Make sure your surveys are inclusive. For example, if you have to ask for personal information such as gender, leave this an open-ended option, so people can fill in how they identify, or include an ‘other’ option in a pre-populated multiple choice selection.

Only ask questions that you’ll use to inform decisions. This seems obvious, but as you share your survey plan with others on the team, your colleagues might want to add more questions, leading to a very long survey.

Remember that people’s most valuable asset is their time, so keep your survey as concise as possible. Similarly, try not to include too many open-ended questions because they take the most time from your customers. Lastly, remember to ask only one question at a time.

Mix up the types of questions you have in your survey to keep the participants engaged. For example, you could include a selection of multiple-choice questions, multi-select, open-ended, Likert-scale, drop-down, and matrix questions.

For example, asking “What did you like about this?” implies there was something they liked and primes them to think positively when they could have not actually liked anything.

Provide an option for customers to add any additional thoughts. The questions you posed may have jogged their memories for a moment or experience they want to share that pre-populated options don’t allow. You can get around this by adding an ‘Anything else you’d like to add?’ question at the end of the survey.

Let participants know where they are in the survey and how much more time they have until they’re done.

Randomly order the options to your question to account for order effects in participant responses. It’s easy for people to go through a survey and select the same option for each question (such as the first or last option). To get around this, you can re-order the options for each participant to help mitigate bias in the answers you receive.

People report negative feedback more often because negative events naturally stick in our memories. Survey feedback can often feel overwhelmingly negative, so remember this when analyzing the data and take it with a bit of a grain of salt.

Avoid asking questions that ask your participant to recall something late in the past or predict their future behavior. If you have to ask a question like this, timebox the period. For example, you could ask, “Within the past week, roughly how many times did you log in to your email account”? Rather than “How often do you log in to your email account”? When questions use language like ‘often,’ it leaves a lot of room for subjective interpretation about what often actually means.

Ask for an email or way to contact if you do want to follow up or need a way to identify who provided the feedback.

Even if you run a pilot test, there still may be some questions that participants don’t understand or simply aren’t yielding the quality of answers you were hoping to receive. For this reason, it’s an excellent idea to track the progress of the first few responses to get a sense of the quality and decide if you need to make any adjustments.

Compensation is a way to thank your participants for taking time out of their day to help you out. Filling out a survey or engaging in any feedback helps you shape the product or experience you’re working on, so it’s important to show your customers that you appreciate their input.

The way you compensate will depend on your budget, but a few ways you can do this is by compensating every participant or setting up a draw whereby you decide how many people can win whatever the compensation is. You can compensate with money, a free subscription to your service, or other ways that are available to you and your company.