Effective Survey Techniques
Simple questionnaires
Sending out a questionnaire is a good way to gather volumes of data to help with business decisions. It is also a great way to waste valuable time.
The problems with questionnaires starts with a lack of awareness of what the questions are for, what will be done with the data and then of course there are the questions themselves.
If you have paid any attention to politics you will have seen many examples of questions that almost guarantee one answer, politicians then use these solid results to justify the things they were going to do anyway.
Most companies do the same thing either accidentally or otherwise.
Writing unbiased questions is a mixture of art and science, like the rest of marketing really. It is also an iterative process where your final question may be quite different from the one you start with.
Developing effective questions
Rough flow for question creation
|
|---|
1. Identify the problem you need additional data to resolve
|
2. Rough out the answers you expect to get
|
3. Decide what use you would put those answers to
|
4. Look for gaps that would be filled by additional information
|
Who to ask
So many companies assemble the final results of their survey only to sit back thinking, 'if only we had asked that question as well.'
Once you have worked out what you need to know and then refined it with ideas of how you will actually use the data you need to think about who to ask. Do you have a specific segment of business to ask? Is it homeowners in a specific area or people doing a specific job? Maybe you need as wide a spread of the population as possible.
Who you ask is every bit as important as what you ask, it will skew the results if you ask too tightly defined a group. Then again, you may want results from that kind of group.
Once you have defined your target audience you should feed that back into your analysis of what you are going to do with the data. Does the group you have defined restrict the uses of the final data in any way that can be mitigated by modifying the target?
Now you have got to this stage it is time to consider the questions you will ask. To do that you need to consider how big your survey will be and how you will enter the data into your database.
Controlling the answers
If you allow people to give freeform answers it will be very difficult to do anything meaningful with the data and it often leads to interpretation by the reviewer which can lead to anomalous results.
The best way is to ask you question and then offer a choice of set answers that cover the possible answers. The drawback of this is that you often have to give people an 'other' answer and if you didn't get your planning right there will be too many others and not enough of the answers you expected.
The only effective way to generate unbiased questions is to trial them. Once you have devised a set of questions you should get people you work with to answer them and see if they all give the same answers. If they do, either you have a biased question or the outcome is so obvious that it may not be worth asking, use the time to ask something that does not give such an obvious answer.
Once you have tested your questions internally, try a small sample group of your target audience. Once you have done this you will have a fairly good idea as to whether your questions are leading people to one answer and if you will generate too many 'other' answers.
Refine your questions, test them again and then distribute.
Don't feed them the answer you want
How to distribute a questionnaire
Distribution and collection are another subject in themselves. You may have chosen a representative target audience for your survey but unless you can get all of them to respond you may end up with a skewed result. Even if you allow your responders to be anonymous you should gather some demographic information if you can so that you can see how good a spread of people you obtained answers from.
Several of the surveys I have carried out for larger organisations have been done in stages.
* Internal testing and refinement
* 30 surveys brought back by sales staff who have taken them out to customers and walked them through the questions.
* Test the sample results, 30 samples gives statistical significance so trends can be noted at this time
* Mass survey published on web site and mailed to targets
When you are putting together your results, be sure to include your methodologies in the report so people can see that a rigorous process was used to ensure valid, unbiased data was collected.
