A Guide to Survey Design

Be clear on the purpose of the survey

• Why are you conducting the survey?
• What should be the scope of the survey?
• Who do you want to respond to the survey?

If you are able to answer these three questions, you have established an ideal starting point for developing a good survey. Regardless of the type of survey you wish to conduct, it is essential to be crystal clear on the purpose of the survey and who you want to respond i.e. the target group, before you begin creating the questions.

How many Questions should you Ask?

In answering this question, you should keep in mind the following rule of thumb: the stronger the relationship between the scope of the survey and the respondent group you have sent it to, the higher the response rate you will enjoy and the greater the acceptance will be for answering longer surveys. That said, respondents should be informed up front how much time is expected to complete the survey. If it is estimated that it will take eight minutes, then write eight minutes, not four minutes!

Another important guideline is that rather than creating long, extensive surveys, which you send out infrequently, you should aim for short, precise surveys that are sent out on a more frequent basis. It is tempting to add questions which are nice to have since you already have the respondent’s attention. Don’t! If you’re not concise, you run the risk of not receiving any answers at all.

The Order of Your Questions is Important

Per incrementare la probabilità che i vostri partecipanti completino effettivamente l'indagine e vi forniscano le informazioni necessarie, dovreste prendere attentamente in esame il modo in cui approntate l'ordine delle domande nell'indagine.

Iniziate l'indagine con una fase di "riscaldamento", in cui ponete domande semplici ed elementari che portino il partecipante ad acquisire dimestichezza con la struttura dell'indagine e a sentirsi a proprio agio nella situazione in corso. Spostatevi quindi sulle domande più importanti ed impegnative. Sono queste le domande che costituiscono il nucleo centrale dell'indagine. Verso la fine passate alla fase di "raffreddamento", in cui chiedete al partecipante i dati demografici, ulteriori commenti, ecc.

Seguendo la metodologia sopra descritta, eviterete fin dall'inizio dell'indagine che i partecipanti si sentano sotto pressione nel rispondere a domande delicate ed impegnative. I partecipanti entreranno quindi nell'indagine in modo garbato e ne usciranno con totale naturalezza.

Semantics are More Important than Statistics

It is a known fact that semantics are often more important than statistics when it comes to market research. Statistics are obviously of vital importance, but one of the key foundations for all good research, can be found in the language.

It is therefore extremely important that the questions that are to be included in the survey fulfil certain criteria:

• The questions need to be relevant, to such an extent that they have an obvious connection with the overall purpose of the survey.
• The questions should be easily understood. Always consider the weakest link in the chain. There should be no room for misunderstanding or misinterpretation.
• The questions should be of interest for the majority of people in the target group – not just a select few.
• The questions should not be offensive, or provocative.

Read on to learn more about how to create relevant, crystal clear, concise and neutral questions!

Be as Neutral as Possible

As far as possible, questions should be written in such a way that they appear to be neutral. To ask leading questions that guide the person being surveyed to draw a specific conclusion or provide a certain answer is probably the most common error committed in market research.

Here are a couple of examples of questions that are obviously skewed:

• “Don’t you agree that our new product meets all your needs and expectations?”
• “Most people find our level of service to be excellent. Do you?”

Avoid Ambiguity

A question that can mean different things for different people is an ambiguous question. Using, for example, generic wording or words with double meanings can often lead to ambiguity. Using ambiguous phrasing may mean that the person answering is really not sure what the question that they are trying to answer means.

Watch Your Language!

It can be tempting to ”spice up” the questions you ask, but you should avoid phrasing that could result in stereotypical answers or that provoke the respondents. This may include words such as “radical”, “old fashion”, “progressive”, or “extremist”. Clich‚s and slang should also be avoided, in addition to any foreign words and abbreviations that may cause confusion.

Taking Things for Granted

Respondents may not have knowledge or strong opinions about issues that do not affect them on a day to day basis. It is therefore important to make sure that you do not take things for granted when asking questions. Words and definitions may have different meanings for different individuals and groups. In addition, a questionnaire is not a test meant to reveal the extent of knowledge that the respondent has on a specific subject. Stick to the obvious facts!

Mutually Exclusive Answers

A common mistake is using categories that do not exclude each other.

For example:

”How old are you?”

• Below 20 years
• 20-30 years
• 30-40 years
• 40-50 years
• 50 years or older

In this case the respondent is asked to provide only one answer, and if the respondent is 30 years old, he or she can choose two of the offered alternatives.

Below is an example on how to mutually exclude age categories:

”How old are you?”

• Below 20 years
• 20-29 years
• 30-39 years
• 40-49 years
• 50 years or older

Another element that may create confusion is if the scale does not have clear definitions, leaving it up to the respondent to define the border between the different alternatives:

“How often do you eat pizza?”

• Twice a week
• Sometimes
• On occasion
• Seldom

The alternative “Twice a week” is more clearly linked to a specific number, than the other alternatives.

Cover all Possible Alternatives

It is important to provide alternatives that cover all likely (and to some extent unlikely) responses to a given question. If there are not enough relevant alternatives, this will provide a potential source of error when interpreting the results. For example, “Which of the following elements is the most important for you when choosing a petrol station?”

• Opening Hours
• Brand
• Price

This question is missing several alternatives, such as: “other”, “don’t know” etc. If respondents are not able to find any suitable answers among the alternatives given, they will not be able to provide reliable answers. If we had kept the first three categories and added a “don’t know” alternative, this would have provided formally exhaustive alternatives, but we might still not have included the most important element for the respondent: location. And “don’t know” may in this case either mean that the respondent doesn’t know enough in order to give a qualified answer or is not (due to the lack of alternatives) able to provide a decisive answer.

Another example of vague alternatives is when the question relates to money, time or other measurable units, but where there is no alternative for zero or “none”. In this case, some respondents may leave the question unanswered because they have nothing to report. But it might also be left unanswered if people don’t know or don’t want to answer the question. The quality of the results will be reduced if we don’t know how to interpret unanswered questions in a survey.