The Ultimate Guide to Effective Customer Surveys
How to avoid insufficient data, poor insights, and wrong decisions in product management surveys.
Hey, Paweł here. Welcome to the premium edition of The Product Compass.
Every week, I share actionable tactics and resources for PMs to help you boost your career.
If you are not a premium subscriber, here’s what you might have recently missed:
The Ultimate Guide to Effective Surveys for Product Teams
Surveys are a great tool product teams can use to understand new markets, collect customer feedback, and validate ideas.
But many struggle with:
Low response rates
Short, meaningless, or random answers
Results that are easy to misinterpret
This leads to insufficient data, poor insights, and wrong decisions.
Honestly, almost all product surveys make me sick. Irrelevant, boring, and difficult to complete.
So, in today’s post, I combine my work experience with what matters the most from:
Surveys That Work by Caroline Jarrett
People Aren't Robots by F. Annie Pettit (my favorite)
Let’s discuss:
Types of Product Management Surveys
How to Prepare Before a Survey
How to Increase The Response Rate
🔒 How to Write Good Survey Questions
🔒 Common Pitfalls
🔒 Conclusions
1. Types of Product Management Surveys
For practical purposes, we can identify:
Qualitative surveys, where we ask only open-ended questions. They are rare.
Quantitative surveys, where we ask only closed-ended questions.
Mixed surveys, where two types of questions are combined. I’d argue that’s the most common survey type.
A more important classification considers the goals of the surveys:
Market research surveys: These surveys focus on gathering data from the market, not necessarily only from your current customers. They typically follow a set of customer interviews to refine the focus and relevance of the survey questions. An example might be a Van Westendorp survey.
Product analytics surveys: Standardized surveys that collect metrics like Customer Satisfaction Score (CSAT), Customer Effort Score (CES), and the controversial Net Promoter Score (NPS). I described those metrics in The Ultimate List of Product Metrics.
Product validation surveys: These surveys often leverage tools like Maze or Optimal Workshop to automate usability testing. I described those experiments involving User Prototypes in Testing Business Ideas: The Ultimate Validation Experiments Library.
See how they relate to customer interviews:
Regardless of the survey type, you can beneft from many tactics from this post.
2. How to Prepare Before a Survey
Recommended steps before performing a survey:
Step 1: Define your research goals
The research goal is what you are trying to achieve. The “Why” behind the survey.
While sometimes it might be obvious (e.g., “validate an idea” or “figure out pricing preferences”), it’s worth stating the goals directly, particularly for market research surveys, to align your questions properly.
Step 2: Define participants based on your goals
For an in-app survey, the list of participants may be dynamic and based on their behaviors. For others, you might leverage your knowledge about your existing customers.
When performing a market research survey, it’s often a good idea to start by interviewing selected customers to refine your goals, target audience, and questions.
Step 3: Prepare research questions and interview questions
Product teams should plan two types of questions:
Research questions are the questions you want to answer, aligned with your research goals.
Interview questions are the questions you will ask. More in the following points.
Step 4: Design and test your survey
It’s essential to test your survey on a smaller population before sending it to everyone. This way, you can mitigate three risks:
Engagement: How likely are people to complete this survey?
Usability: Will people understand the questions?
Relevance: Is it the right audience? What will be the quality of the answers?
Some surveys, like CSAT with an optional comment, might be a no-brainer, especially if you use an existing, proven template from another product area. Others, like a pricing survey, require careful consideration.
3. How to Increase a Survey Response Rate
To ensure higher response rates, take care of the eight elements:
Perceived effort: The shorter and simpler the survey, the more likely people will complete it. Many product surveys have 1-3 simple questions. Even a longer survey should take no more than 5-10 minutes to finish.
Perceived reward: Incentives, such as gift cards or discount codes, can motivate people to participate. At the same time, I've found that simply being able to share personal expertise and influence the product you care about can also work as an incentive.
Relationship and trust: People are more likely to respond to surveys from people and brands they trust. I noticed that email surveys sent from a personal account with a photo in the footer work much better than from a generic email address.
Relevance: Surveys relevant to the target audience's interests, needs, or experiences tend to have higher response rates. The key is to adjust the content to your specific audience. Make them feel special by providing an unique, highly customized context in the intro. You can also leverage the techniques I described in The Secrets of Copywriting: Proven Ways to Transform Your Writing as a PM, particularly hooks.
Audience: Despite best efforts, some people (e.g., CXO) may be less likely to respond to surveys due to busy schedules, limited access to email, etc.
Channel: The way you distribute your survey can largely impact response rates. For example, email surveys often have higher response rates than web banners. Consider testing multiple channels. Also, remember that people are busy. When using an email survey, send 1-2 friendly reminders. This can easily increase the response rate by 2 times.
Design: A visually appealing, user-friendly survey design can make the experience more enjoyable and reduce the perceived effort. A good UX also builds trust. Hence, the Product Designer and/or Content Designer should participate in preparing the survey.
Timing: The timing of your survey can significantly impact response rates. A good example is displaying a product analytics survey in the middle of the job that distracts the participant.
Finally, do not obsess over the response count. It’s more important to get high-quality data. I loved this quote:
“Asking one person the right question gets better results than asking 10,000 people the wrong question” - Carolyn Jarrett, Surveys That Work
4. How to Write Good Survey Questions
Here, I selected 12 non-obvious tactics that are not addressed by common survey tools:
Tactic 1: Start with broad, general questions
Start with broad questions and gradually become more specific. This allows people to share their opinions without being influenced by the survey.
For example, you could start by asking about various product categories, then narrow down to your specific product category, and finally ask about particular aspects of your product.
Tactic 2: Simplify everything
The more complex the question, the more effort it requires to answer. As explained in The Secrets of Copywriting: Proven Ways to Transform Your Writing as a PM, you should use short, simple sentences, words, and (if needed) paragraphs.
Anything more than 2-3 lines of text is a no-go. Also, get rid of the lengthy introductions and instructions nobody reads.
Tactic 3: Use common, spoken language
As F. Annie Pettit noticed in People Aren't Robots, questionnaires are not the place to enforce perfect grammar.
I liked her examples:
Please don’t ask:
In which of the following groups do you belong?
In which country do you live?
For whom do you buy cold cereal?
In which vehicles are you interested?
Instead, ask:
Which of these groups do you belong to?
Where do you live?
Whom do you buy cold cereal from?
What types of cars are you interested in?
Occasionally, it’s ok to use sentence fragments. For most audiences, you can also write “Thanks a bunch” or “Sharing is caring.” It makes the experience more personalized.
Participants are not robots. They breathe, eat, smile, and even shit like everyone else. So, avoid being monotonous.
Tactic 4: Don’t ask about opinions
It’s easy to take opinions at face value, but people are biased. We should prioritize facts and actual behaviors over abstract ideas.
Abstract, biased question: How often do you post on LinkedIn?
Regularly
Sometimes
Rarely
Never
Better question: How often do you post on LinkedIn?
5 times a week or more
3 to 5 times a week
1 to 2 times a week
Less often than 1 time a week
Never
Best question that asks about specific situations people can remember: How many times did you post on LinkedIn in the last 7 days?
5 times or more
3 to 5 times
1 to 2 times
0
Tactic 5: Avoid leading questions
Keep your questions open and unbiased. Don’t put words in your customers’ mouths, for example:
Bad question: How helpful was this article in solving your problem?
Good question: Did this article help you to solve your problem?
Keep reading with a 7-day free trial
Subscribe to The Product Compass to keep reading this post and get 7 days of free access to the full post archives.