Surveys to test your idea: YES or NO?
Create User Surveys that don't stink
04 April 2022
Don’t just rely on surveys (or for that matter focus groups) to test your idea. We have seen many startups and corporates deciding to invest just based on survey results.
For testing any new idea – always start with Problem Interviews — conversations with your potential customers about the problem you want to solve for them.
And once you’ve conducted about 10-20 Problem Interviews, you might be able to draw some insights on:
- Does the problem really exist?
- Is it the right customer segment you are targeting?
- Are they frustrated with the problem?
- Why are they unable to solve the problem today? (Are they even actively looking for a solution or created some workarounds?)
Now survey is a next logical step – in order to re-confirm your Problem Interview insights with a much bigger audience – 50-200 people.
What are surveys though?
They are simple questionnaires used to collect information from a sample of your target customers.
What’s so special about surveys anyway?
They are quick and have a wider reach. Speaking individually to potential customers takes a lot of time and resources. Surveys can help you target 100 or more customers in a short space of time.
What’s the catch then?
Surveys will only tell you what customers SAY they will do or what they think they WILL DO? (their opinion) And usually there is almost always a pretty big gap between what they SAY they will do vs what they ACTUALLY DO (their actual behaviour).
Healthy Food vs Junk Food
Sleeping at 22:00 vs watching netflix till 02:00
Meditating everyday vs going out for a drink
Exercising vs Watching Tv instead
So you have to be EXTRA careful while using insights from surveys. Their success or failure are both LOW EVIDENCE.
Insights from surveys are a good indication towards what your customers are thinking, but you will need to run a couple of more experiments to learn what they will actually DO.
Combining the insights from Problem interviews and surveys – usually gives us a good starting point for next experiments. We use insights like what users are thinking, how they explain the problem, which problems they perceive as big and sometimes how they visualize the solution to be?
All these insights become input to your next experiments that could generate HIGHER EVIDENCE.
Here’s how you can prepare your surveys:
- Define your goal for the survey and what you are trying to learn
- Identify your target audience for the survey
- Assuming a 10-20% response rate, calculate how many people should receive the survey
- Set a start and stop date for the survey
- Create your survey
EXECUTION
Keep the survey short and focused on one topic. A good survey should have more closed ended questions and a few open-ended questions.
For example, let’s say you want to understand how users clean their formal expensive clothes today.
You might ask:
How do you clean your formal clothes today?
☐ Washing machine at home
☐ Wash by hand at home
☐ Wash at Laundromat
☐ Dry cleaners
☐ Others (Type In)
If they chose, dry cleaners, like customers in Problem Interviews did, you could ask them the next question:
What is your top worry about going to the dry cleaners?
☐ I don’t like queuing up at the dry cleaners
☐ I worry they would use unknown chemicals on my clothes
☐ I don’t like the smell of the place
☐ I don’t like the experience of carrying my clothes, drive and drop and go back to pick up my clothes
☐ I have some other problem
If your user chooses “the have some other problem”, you could ask an open ended question: What is your big problem? And let the user type in the answer.
Overall, it’s good to keep the survey length anything between 30 seconds to 2 minutes — short enough to finish quickly and long enough to engage with your potential customers.
Skin in the game: In the end, it’s important you ask your potential customers for their emails, saying something like: We are working on a new app to make dry cleaning a pleasant experience. Please leave your email – if you want to get one free wash.
Set up time: Surveys should ideally take a few hours to a day at most for your team to set up.
Cost: Surveys can be free to low-cost, depending on what services you use.
We use Typeform for running our surveys. They provide beautiful templates and provide lots of integrations.
Run Time: Don’t spend more than a few days or at most a week to gather results from surveys.
How to REACH your potential CUSTOMERS?
- Run ads to bring traffic to your survey.
- If you have an existing site with lots of traffic, then leverage that to get to your potential customers there.
- Find out where your customers hangout online and try to convince them to complete the survey. Online platforms such as instagram, snapchat, facebook, twitter are some of the best platforms to conduct online surveys.
- If your potential customers are available offline in co-working spaces, coffee shops or supermarkets, print physical copies of your surveys and have them fill them in person.
Iterate your Questions
Ideally, you should run the survey for 1-week and collect raw reactions. If you see low response rate, anything less than 10-20%, try to analyze the reason for it:
- Are your target customers struggling with the language?
- Are they put off by the length of the survey?
- Are you pushing it to the right customer segment?
Find your goof-ups and quickly make small tweaks and test again.
Remember, surveys can be very fast and useful, but only when they are paired with Problem Interviews, or another validation experiment, to gather data. Otherwise, there is a high risk of collecting false positives or false negatives.
