A detailed guide that'll show you everything you need to know about questionnaires.
A well-designed and considered questionnaire can be the difference between success and failure.
Customers have wants and needs that are constantly changing and evolving. It’s no longer enough to be reactive when situations arise. Now, your customers expect you to solve problems before they become problems.
Questionnaires make it possible to better understand the wants and needs of your customers so you’re in a position to meet them.
This article walks you through what a questionnaire is, the pros & cons, and how to properly design them so you can unlock deep insights.
What is a questionnaire?
A questionnaire is a research device or instrument that is made up of a series of questions which are closed-ended or open-ended. The goal is to collect relevant information from respondents which can then be used for a variety of purposes. When you give the respondent the ability to give a longer answer, it can yield more insights because they can elaborate on their thoughts.
It was first developed by the Statistical Society of London in 1838 and has been in continuous use ever since.
Questionnaires, though versatile, can’t be used in every situation. It’s not advisable to use a questionnaire to ask specific questions about a product or service you’re still considering. This may lead to bias and false positives about the feasibility of the product.
Instead, questionnaires should be used to collect more general information qualitative or quantitative data regarding features and preferences. For example, instead of asking if they’d buy a new pink button down shirt with a unique collar, ask if they like to wear the color pink or if they like the type of collar you’re considering.
When you're actively collecting data from your surveys and even other methods, data governance becomes even more important. What is data governance? It's a set of rules and principles that'll ensure you maintain data quality throughout its lifecycle, prevent accidental tampering, and ecourage transparency.
Surveys vs questionnaires
Over time, surveys and questionnaires have gotten mixed up and are used interchangeably. They’re not the same thing. The difference is subtle but important.
A questionnaire is a list of questions used to collect data about someone or something. It’s not used to do statistical analysis or find trends and patterns. An example, would be when you sign up for a gym or go for a checkup and have to answer a series of questions about your current physical condition.
The answers you provide are used to understand your overall health, assess risk, and in some cases help find or diagnose issues. It’s not being used as part of a larger data set to clarify the bigger picture or find trends in a population.
A survey is a bit different. Instead of looking at individual questionnaires, it’s used to understand trends, do detailed analysis, and reveal deep insights. The key with a survey is that it’s collecting data with the express purpose of analysis.
An example would be customer feedback surveys, demographic surveys, market research surveys, NPS surveys, etc. If only one person were to respond to these types of surveys, it would severely limit its usefulness. The more respondents, the easier it is to spot patterns and make informed decisions.
Why do they get mixed up?
Previously, researchers and professional marketers were the main groups who used surveys and questionnaires regularly. They made a clear distinction about what they are and when they were to be used.
With the advent of easy to access survey software, more and more businesses have started to handle their own research. The terms became interchangeable.
A questionnaire is when you ask someone a series of questions and don’t use it for data analysis.
A survey is when you ask someone a series of questions and you use it for data analysis.
For example, if you send an employee a series of questions about the working environment, it’s a questionnaire. When you send out that same questionnaire to 500 employees then compile the data to find trends, it’s a survey.
Let’s dive into the types of questionnaires.
Types of questionnaires
There are two main types of questionnaires and the one you’ll use depends on what kind of information you want and purpose of that information.
Exploratory questionnaire (qualitative)
These are also known as unstructured questionnaires. They’re used to collect qualitative data which is information that can be observed and recorded but isn’t numerical in nature. It’s used to approximate and characterize.
An example of qualitative data would be someone giving your feedback about your writing. They may mention things about the tone, clarity, word choice, etc. it helps you categorize your writing but you can’t attach a number to the feedback.
Exploratory questionnaires are ideal when you’re in the early stages and want to learn more about a topic before designing a solution or hypothesis. For example, if you’re in the early stages of product development and don’t know enough about the market then exploratory questionnaires are ideal.
Formal standardized questionnaire (quantitative)
They’re also known as structured questionnaires. These ones are used to collect quantitative data which is information recorded as a count or numerical value.
The data is quantifiable which means it can be used for mathematical calculations or statistical analysis. In essence, it answers the question of how much, how many, or how often.
An example of quantitative data would be the answer to the following question, “how old are you?” which requires a numerical reply.
Standardized questionnaires are best used when you’ve already formed an initial hypothesis or built out a prototype for a product. You’ll use it to stress test your assumptions, designs, use cases, etc. before going further with product development. Because of its clear focus, the questions you ask are narrow in scope and solicit specific information.
Just as important as the questionnaire type are the question types you choose.
Questionnaire question types
Not all question types are ideal in every situation. That’s why it’s important to understand the type of questionnaire you’re creating first. With that information, it becomes easier to choose the right question types.
Open ended questions
As the name implies, these questions are open for the respondent to answer with more freedom. Instead of presenting a series of answers choices, the respondent writes as much are as little as they want. This is ideal for exploratory questionnaires which collect qualitative data.
Multiple choice questions
This question presents the respondent with a list of answer options and they can select one or more. The challenge with multiple-choice questions is providing incomplete answer options.
For example, you may ask what industry do you work in and list out 5 of the most common industries. There are more than 5 industries in the world so some people won’t be represented in this situation. A simple solution to this problem is adding an “other” option.
This is a question with only two possible answers. It tends to be a yes or no question but it can also be something like agree/disagree or true/false. Use this when all you need is basic validation without going too deeply into the motivations.
Scaled questions are common in questionnaires and are often used to judge the degree of a feeling. This can be used in both exploratory and standardized questionnaires because there are many different types of scaled questions such as:
The final type of question used in questionnaires substitutes text for images. Respondents are asked a question and shown pictures to choose from. It usually has a higher response rate than other question types.
Questions to avoid in a questionnaire
While you can ask almost anything in your questionnaire, it may not be a good idea to do so. Some questions may give you poor data while others may stop people from completing the questionnaire.
Here are a few question types to avoid.
A hypothetical question asks a respondent what they would do, think, or feel about a situation that may happen in the future. It’s asking people to talk about their future actions and behavior which we’re notoriously bad at. This kind of question may give you data that can’t be used or will skew your overall understanding of the topic.
Embarrassing or offensive
Even though questionnaires can be anonymous, it’s not a good idea to embarrass or offend the respondent. It may lead to them dropping the questionnaire without completing it or giving you poor answers on purpose. Neither one is a good scenario.
You don’t want to bias your respondents before they’ve had a chance to form their own opinion on a topic. If a question is presented as extremely positive or negative then it may create a bias that should always be avoided. In the end, your data will be skewed.
Designing your own questionnaire
There are quite a few factors to consider when you’re designing a questionnaire that gives you the exact information you’re looking for.
At the very least, think about the goal, audience, distribution method, etc. Let’s look at the factors to consider while creating a well thought out questionnaire.
1. What’s the goal of the questionnaire?
This may be the most important aspect of the questionnaire creation process. The goal of your questionnaire will determine both the type and questions to ask your respondents.
As mentioned earlier, if you’re in the beginning stages and are still trying to form a hypothesis, it’s an exploratory questionnaire with open-ended questions. If you’re trying to prove or disprove an already formulated solution or hypothesis then a standardized questionnaire with closed-ended questions would be used.
A clear goal also makes it easier to determine if a specific question is necessary or not. For example, if you’re doing initial product research for a dog toy, a question about the kinds of toys they’ve purchased in the past may be useful. When you have an initial prototype dog toy and want to gauge market response, that question wouldn’t be as useful to you.
2. Who is the target group?
Whether or not it’s obvious, every market has multiple groups within it. Let’s take an average SaaS company for example. It usually has pricing tiers that are mapped to different personas. The customers on each subscription plan have different wants and needs.
The questionnaire you create and send out should reflect that. If you have the resources, create more than one so you can cater to the specific needs of different groups in your customer base.
In a situation where you’ve not seen different customer groups, it may be worth it for you to identify and segment your customers. Not only will your messaging become more effective, any time you send out a questionnaire or a survey, but it’ll also be more targeted and get a higher response rate. On average, you can expect only 12.5% of an external audience (nonemployees) to respond to your survey.
3. How will you reach the target group?
This is often overlooked until the last minute but it’s an important consideration. If you have an email list full of past and present customers then this may not be an issue for you.
What about when you’re trying to enter a new market with a new type of product and don’t have customers there? How will you be able to reach them? Can you even reach them online?
This can have major implications on the design of your questionnaire. For example, if it’s a paper-based questionnaire, the design will necessarily be different and the questions won’t be as dynamic. If you’re using ads to get people to take your questionnaire, you may need to provide an incentive and make it shorter.
4. Do you have a clear question progression?
The way your questions are ordered sets the tone for the entire questionnaire. You don’t want to start with a deep philosophical question that challenges the meaning of life. That’s too heavy. Almost everyone will bounce.
Instead, you want to start with simple questions that almost anyone can answer without too much thought. These are questions like age, sex, and geography – demographic information. These answers can also be used to further segment your respondents.
After you’ve built up some momentum, move into the core questions you want an answer to. The questions you ask here will depend on your goals but it should relate to your products and services. These questions help you flesh out your product development initiatives as well as create better and more focused marketing messages.
Finally, tie up any loose ends with your final questions. A common but subpar question is “is there anything else you think we should know?” try to avoid this one. Instead, ask things like how they found you, their experience with buying another similar product, how they’d describe a specific problem, etc.
5. What kind of questions will you use?
Do you want well thought out answers that give you deep insights into the inner workings of the respondent’s mind? Or, do you want a narrow but easily analyzed response? The type of questions you use will determine the type of data you get.
As a rule of thumb, open-ended questions are often used earlier in the research process. Closed-ended questions tend to be used to prove or disprove hypothesis or solutions. Of course, you can use both of them but be sure to pay close attention to question progression so respondents aren’t put off or confused.
6. Length of questionnaire
There are no hard and fast rules about how long your questionnaire should be. Some of them are hundreds of questions while others are less than five questions. The more questions, the lower your completion rate.
On average, it takes 5 minutes to answer 10 questions. Depending on whether the answers are open-ended or close-ended, the time could be considerably more.
Your customers are busy and most of them won’t sit through a long questionnaire without some form of incentive or compensation. If you’re able to provide that then fine but most customer surveys shouldn’t require it.
Instead, be considerate of the time of others. Keep your questionnaires less than 15 questions and ideally under 10 questions. It makes it easier for respondents to complete the survey and easier for you and your team to analyze the information.
Contrary to popular belief, you don’t need a thousand bells and whistles to get people to take your questionnaire. A simple design that emphasizes the questions is more valuable than a flashy one. Of course, you can go flashy if you like. The thing is, most people just don’t care.
Select a font that’s easy for people to read and make sure the size is large enough to be legible on all devices. Apart from that, keep the number of pages to a minimum. 2 pages is much better than 30 pages when it comes to a questionnaire.
8. Choose language carefully
If you ask a question that creates bias or confuses your respondents then you may accidentally contaminate your data. Use clear terms, be concise, and avoid industry jargon.
For example, “We’ve been told we make great eggs, would you agree or disagree?” this question causes bias before the customer can answer. An unbiased question would be “how would you rate our eggs on a scale of 1 – 5?”
Also, avoid combining multiple questions into one. An example of a combination question would be “how did you enjoy your stay and would you recommend us to a friend?” These are two distinct questions bundled into one.
Advantages & disadvantages of questionnaires
It’s important to understand both the pros and cons of questionnaires and put proper safeguards in place before you start using them to make important business decisions.
Let’s start with a few of the good things.
Sending out an online questionnaire is one of the cheapest customer research strategies available. Unless you’re offering some type of incentive or are using ads to get in front of respondents, there are few costs associated with it.
Self-administered questionnaires avoid the need for hiring people to administer it, remove the cost of in-person interviews, and have versatile distribution methods.
Results come in quickly & can reach a large audience
Business moves fast so one of the most powerful advantages of a questionnaire is the ability to get it in the hands of a large group of people quickly. You don’t need to start mailing it out and waiting days for it to get to the intended recipient.
Instead, you can send an email, post it on your website, or share it on social media and start getting responses you can use almost instantly. Also, there’s no real upper limit to the number of people who can respond to the questionnaire.
Easy to analyze the results
The majority of questionnaires are quantitative in nature which allows for quick analysis of the answers. This is even more important when you have a larger pool of respondents.
With a survey tool like KyLeads, you can easily spot trends and derive insights from your questionnaire with our easy to use & understand reporting features.
Respondents can remain anonymous
If respondents are unable to remain anonymous, they may not answer some of the questions truthfully. As long as you’ve done proper targeting and they’re not answering for an incentive, it’s ideal to leave the respondents anonymous. They’ll be more comfortable and answer honestly and thoroughly.
Can cover all aspects of a topic
This is an overlooked aspect of questionnaires. With them, it’s possible to ask 100 questions. Of course, we don’t advise this because almost no one will finish an online questionnaire of that length.
With that being said, you can ask as many questions and solicit as much detail as you want. Play around with the number of questions you ask but try not to overdo it.
There are a few disadvantages to questionnaires which you should be aware of.
Sometimes, people will just skip answers or drop off halfway. Since the questions are online and no one is there to prompt the respondent, this happens fairly often.
There is any number of reasons for this like unclear or confusing questions, irrelevant questions, incomplete answer options, etc. Making the answer required can help with this but it also increases the chances of someone abandoning the questionnaire altogether.
Fatigue with your survey as well as the other surveys being sent out by other companies. More and more companies are using surveys and customers can’t answer all of them. This results in a lower overall response rate to surveys or questionnaires as a whole.
Conversely, someone may start your survey but drop off because there are too many questions or the questions seem to be irrelevant. You can’t get rid of the fatigue 100% but you can reduce it by creating shorter questionnaires and making your questions easy to answer.
Everyone who takes the questionnaire gets, for the most part, the same series of questions presented in the same way. Now, technology is making this better with features like logic branching and answer piping so the experience can be personalized a bit more.
In the end, it’s still limited because there’s a predetermined series of questions and the questionnaire can’t react to open ended statements.
Improper interpretation of questions
This is why it’s so important to choose your question language so carefully. It’s easy to misinterpret a written question and give a wrong answer or skip the question entirely. Another thing to consider is that certain words have multiple meanings and, without context, a different meaning may be applied.
Prevent this by using simple direct language in your questions and avoiding jargon.
Difficult to analyze certain types of questions
Multiple choice questions and dichotomous questions are simple to analyze. Open ended questions can’t be analyzed so easily.
They’ll require a human touch to ensure you’ve understood what the person is trying to tell you. This isn’t a bad thing but it can get tedious when there are a lot of answers to sift through.
Examples of questionnaires
There are countless types of questionnaires and surveys you can use to get deep insights about your customers and business. In this section, you’ll learn 6 common types that’ll help you improve your business immediately.
This questionnaire example is ideal when you’re actively focusing on building awareness and doing demand generation. It helps you gauge whether or not your efforts are yielding fruit.
It’s one thing for people to end up on your website through a search on Google or a random post on social media. It’s another thing for there to be brand recall or positive associations about your business.
The brand awareness questionnaire will give you a better understanding of whether people looking for solutions you provide think of your brand, the kind of associations your name creates, and if you’re considered a leader in your field.
The NPS questionnaire has become popular over the last few years and it helps you measure customer loyalty and satisfaction.
It’s important to note that in its original form, it’s measuring loyalty and satisfaction that pertains to your entire business as opposed to specific products.
It uses a scale to measure customer loyalty. You calculate the score by subtracting the percentage of detractors from promoters and it’s expressed as an absolute number.
The customer satisfaction (CSAT) questionnaire example we’re sharing is just one of many. CSATs are incredibly varied. Even the NPS questionnaire is a type of CSAT. In general, it’s used to understand how satisfied a customer is with specific products and services or your business as a whole.
Use the basic outline below then tweak the questions to apply to your business or specific product lines. For example, if you were a shoe company, you could ask how often they wear shoes purchased from you.
If you were a hair extensions company, you could ask how satisfied they were with the product or the shopping experience as a whole.
Demographic questionnaires are often used to identify and segment the groups you have in your audience. This type of questionnaire is ideal if you’re entering a new market and want to start building up a profile of the people who will be your customers.
At the same time, you may want to use this to understand your current customer base so you can create better messaging or product pricing.
Oftentimes, they’re a small part of larger questionnaires used to understand who’s giving what kind of answer.
For example, if you serve a customer group that varies in age and income, you’d like to know what kind of customers are giving answers so you can make decisions properly.
Psychographic segmentation has a firm place in modern business because everyone has demographic data (or can get it).
Demographic segmentation pales in comparison to knowing why a group of people do what they do.
Look at it this way, demographic data helps you understand the characteristics and buying power of your customers.
Psychographics helps you understand the why behind their actions and their attitudes behind certain stances. It can be a goldmine if gathered and used properly.
Ah, events. If you’re like most of us mere mortals then there’s a love-hate relationship with them. On the one hand, if they go off well then it can power your business to the next level.
On the other hand, everything that can go wrong probably will. As the organizer of the event or someone who had a key role, it may seem like you know exactly what went right and what went wrong.
If you don’t get feedback from as many people as possible then those are just assumptions which may or may not be correct.
Use post-event surveys to talk to as many people as humanly possible to get a clear picture of how you can improve.
This guide has covered a lot of ground so don’t expect to cram everything in one sitting.
Questionnaires are the backbone of surveys. Without them, there’s nothing to analyze. Before you dive in and start designing your questionnaire to collect all that juicy customer data, there are a number of things to do.
Decide on the type of questionnaire and your goals, focus on the right questions, figure out who the target group is, and so much more. Be sure to revisit this guide whenever you’re in doubt.