Making sure your client feedback is meaningful in your quality management system

​To ensure that questions asked are valid and meaningful, it is sound practice to test a questionnaire on a number of respondents, so that potential problems can be solved before the cost of a full survey is incurred. 

If a computer is being used for analysis, this also needs to be part of the test, so that it is known whether the questions asked will yield the clarity of information required. Clear questions are amply rewarded by clear results. Next we will look at the two main types of question used in surveys. 

Survey question types 

Open questions. The respondent answers in their own words. The questions are asked in the same way each time, but there is no preconceived set of expected answers. However, to be statistically useful the answers then have to be categorised in a meaningful way. This imposes extra costs, and requires that the person analysing the results understands what cryptic comments by respondents mean. 

A simple open question is ‘Why did you buy Brand A?’ The respondent can give any answer they like, such as ‘Because it was the only one available’ or ‘I have tried all alternatives and found this to be the one that suits me best.’ 

Projective techniques such as the three below can generate answers which are more consistent in style for analysis purposes: 

Sentence completion. “When I chose Brand A, the most important thought on my mind was ... (complete the sentence)”. 

Word association tests. The interviewer reads a word to the respondent, and asks them to say the first thing that comes to mind. Word association tests are used to select brand names, slogans and advertising campaigns. 

Third-person techniques. Rather than asking someone about themselves, questions are asked about, for instance, ‘most people’. For example, a researcher might ask, ‘Why don't most people get as much exercise as they should?’ This is useful if they want to avoid embarrassing a respondent, or if a respondent might not want to answer a question. 

Closed questions 
Most questionnaires are based on closed questions, in which the respondent is asked to choose between a number of answers. Alternatively, the interviewer is asked to assign the respondent's answer to an apparently open question to one of a number of answers which have been decided in advance. 

Answers to closed questions are easier to analyse and less ambiguous. However, the respondents are precluded from giving an answer outside these parameters, unless an ‘other’ response category is included. Typical approaches to closed questioning are: 

Numbers. Questions are in the form ‘How frequently do you give donations to Organisation A?’ or ‘How much did you pay for Brand X?’ These are dependent on the accuracy of respondents' memories, unless the ‘process’ being investigated is very regular or easily remembered. 

Yes / No statements. The respondent is simply asked to agree or disagree: for example, ‘I have donated money to the International Red Cross (Yes/No).’ 

Multiple choice questions. The basic question is expanded so that the respondent chooses an answer (or sometimes more) from various alternatives. For instance: ‘Which of these organisations have you ever supported, by becoming a member or by giving money?’

Often interviewers simply ask questions such as 'Which organisations have you ever supported?' and code the answer on a preprinted list. This is a ‘spontaneous’ or ‘free-form’ answer. 

Semantic differential scale. The respondent is asked to choose their position on a scale between two contrasting words (or a range of words or numbers representing different viewpoints). For example: Excellent; Good; Adequate; Poor; Inadequate.

Some scales deliberately omit a middle answer, so that the respondents have to choose which side of the scale they are on. Sometimes a question may combine numbers and phrases: ‘In the last year, have you contacted the Citizens Advice Bureau —five or more times; between two and four times; once; not at all?’