How often have you struggled to respond to a questionnaire which just isn’t quite right for your answers? No doubt you became annoyed, frustrated and maybe even cynical about market research. These problems arise because of poor questionnaire design.
The most frequent errors in questionnaires are:
- Questions which don’t quite mean what the researcher intended
- Questions which don’t probe to find out what the respondent really meant
- Long and complicated questions
- Questions which repeat what has already been asked
- Questions which don’t allow the respondent to answer in a way which is relevant
- Questions which are inappropriate to the research method
- Bad routing which leaves the interviewer wondering which question to ask next, or worse, routing to the wrong one
- Too many questions
- Poorly laid out questions which are badly grouped in the questionnaire
- Questions which have been missed out completely
- Pages of the questionnaire which are missing or out of order.
These errors can be eliminated or at least minimised if the researcher keeps two things clearly in mind: who will be answering the questions and what the research is aiming to achieve.
The researcher must understand four fundamentals of questionnaire design:
- What types of question can be asked
- What types of response should be built in
- How to lay out the questionnaire
- How to test it.
What Types Of Question?
The researcher’s tool box contains three different types of question: behavioural, attitudinal and classificatory.
Answers to behavioural questions tell the researcher where a brand is now and where it could be in the future. They show who is doing the buying and the pattern of purchasing amongst companies using the product. They allow the researcher to position a company or brand in terms of its market share. Here are some examples of the subjects which behavioural questions can address:
- Market size. Questions which need asking to assess the size of the market are: “Do you ever buy?”, “How often do you buy?”, “How much do you buy?” To make it easier for the respondent a time period may be built into the question: “Over the last month how much have you spent on… ?”
- Ownership patterns. Linked to market size questions are ownership questions. “Do you own a certain product?”, “How long have you owned it?”, “How many do you own?”
- Purchasing patterns. The researcher needs to understand the pattern of purchasing to work out the best way to meet the demand. This understanding is acquired from questions such as “How often do you buy?”, “When did you last buy?”, “How much do you usually buy?”
- Behavioural patterns. Knowing the behaviour of a buyer helps the researcher provide the product at the right time at the right place. Questions which provide an understanding of purchasing behaviour include “How often do you buy/use/visit.. ? ” and “Who is it at your company who has the final choice on the selection of a supplier?”
- Market shares. The shares which companies or brands hold can be assessed by asking buyers, “Which brands do you use?”, “How much of this particular brand do you buy per month?”
- Future trends. Being able to predict or simply obtain a feel for future trends is one of the most important yet most difficult of the researcher’s tasks. Pointers to the future can be obtained from questions such as “Do you intend to buy/visit/use… over the next six months?”, “if your purchases this year were 100, what would you say they could be in five years’ time?”, “What are your reasons for saying that you will buy less in the future?”
Attitudinal questions establish respondents’ opinions and their image or perception of a subject. Their perceptions may, of course, be totally at variance with reality, but that may make them all the more valuable as marketing data. Attitudinal research can help the researcher plan a campaign to win sales by:
- Showing why a company buys a certain brand
- Showing whether or not a demand is being satisfied
- Establishing the views which people hold on certain products, services, prices, standards of delivery, etc.
The third type of question in the researcher’s armoury is that which classifies the respondent. Such questions allow answers from different companies to be grouped and compared. Classification questions enable the researcher to segment the market and decide where marketing effort will be most rewarded. Common classifications used in industrial market research are:
- Size of company (measured by number of employees or turnover)
- The nature of the company’s business (usually determined by its Standard Industrial Classification)
- Geographical location.
There may well be classifications which are special to a particular survey. Most industrial surveys, for example, classify according to the size of the respondent company’s purchases, so that the researcher can see the difference in attitudes or purchasing patterns between large and small companies.
What Types Of Response Should Be Built In?
Having selected the type of question (attitudinal, behavioural or classificatory), the researcher must decide on the response option. The questions can be either open ended or closed.
Open ended questions leave the respondent free to give any answer, and verbatim responses are recorded. This type of question and answer is extremely flexible, as the researcher has not committed to a list of pre-coded responses.
However, open ended questions are sometimes an easy option by researchers too lazy to think of possible pre-codes. Open enders should only be asked where the response is truly unknown.
A closed question is one that has pre-coded answers. The simplest is the dichotomous question to which the respondent must answer yes or no.
Closed response questions save the respondent having to think of possible replies. They also make life easier for the interviewer who simply has to tick a box or circle a number. Moreover, they spare the coding staff difficult judgements which, if wrong, can skew the findings.
A special type of pre-coded question is a scale. Scales can be of different types.
Verbal rating scales. Typical of these is the five point scale running from very good through to very poor. Verbal rating scales can be applied to almost any adjective – good/poor, happy/unhappy, satisfied/ dissatisfied, pleased/displeased, suitable/unsuitable, and so on.
Instead of words, the researcher can use a numerical weighting scale. This usually runs up to five or 10. The respondent needs to know the direction of the scale, viz:
“I would like to ask your opinion on THE XYZ COMPANY. What score would you give it out of 10 where one is totally dissatisfied and ten is totally satisfied?”
Problems can occur if the researcher wants a rating on too many questions. Answers deteriorate as the questions grind on and the respondent repeats the same score in an effort to speed the completion of the interview.
Diagrammatic rating scales. These, as the name suggests, ask the respondent to mark an answer on some continuum. This could be a line, a balance, or boxes. Diagrammatic scales are self completed by the respondent and so they need to be clearly marked at each end of the scale. They can be more fun and faster to complete than the verbal or numerical scale.
How To Lay Out The Questionnaire
The questions must now be organised logically and attractively. This is for the benefit of the interviewers, the respondents and the coders. The layout depends very much on who will be administering the questionnaire, who will be answering it, and the type of questions which are being asked.
A well laid out questionnaire must have a sensible sequence of questions. The smooth flow of the interview depends largely on the order in which questions are asked.
Questions on image, purchasing behaviour and classification should all be arranged together. They should lead from one to another with the general subjects at the beginning and the more particular later on. This allows the respondent to relax and loosen up.
For the same reasons, easy questions should precede the difficult ones. Sensitive questions should be placed towards the end, so that the respondent is “warmed up” and in autonomic response mode.
The questions and responses should be clear and easy to read. If the boxes to tick (or numbers to circle) are out of line or placed too far from the pre-coded responses, the interviewer could mis-cue an answer. There should be ample space for the interviewer to record verbatim responses.
A clear and attractive layout is even more important in a self-completion questionnaire, where it has a significant effect on the response rate.
How To Test The Questionnaire
Questionnaires may have to be designed with only a modicum of knowledge about the subject. In these circumstances mistakes are bound to occur unless the questionnaire is tested. The test should be carried out using the interviewing medium – ie telephone, face-to-face, postal – for which the questionnaire has been designed. The number of interviews required to test a questionnaire could be as few as five to 10, though the more the better. Cost and time are invariably the constraints. In industrial market research, the project may use semi-structured questionnaires with a very small number of interviews. Piloting here is a continuous process, and modifications are made from the moment the study gets under way.
