Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5...

20
Consultation in Nottinghamshire How to design and use questionnaires guide 6

Transcript of Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5...

Page 1: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

Consultation in NottinghamshireHow to design and use

questionnaires

guide 6

Page 2: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

6How to

design and usequestionnairesand surveys

12Evaluation

3Targeting –selecting youraudience

9Analysis andwriting up theconsultation

5Methods –capturingpeople’sviews

4Consulting

with people wefind hard to

reach

8Events

7Focusgroups

1Introductionto public

consultation

11Influencingdecisionmakers

2Design –planningeffective

consultation

10Feedback –keepingconsulteesinformed

PublicConsultation

Page 3: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires 3

Contents

1. Introduction 42. Defining the questions 43. Audience 44. Accessibility 45. Questionnaire types 5

Postal 5On-line 5Telephone 6Face-to-face 6

6. Combining survey techniques 67. Questionnaire format 7

Question sequence and layout 7Answers to closed questions 7Question wording tips 8

8. Questionnaire presentation 99. Sampling 910. Selecting a sample 911. Piloting and revising 1012. Increasing responses from self-completion surveys 1013. Analysis 11

Presentation 1114. The 12-step survey process 11

Appendices1. Tips on conducting research projects 122. Sample questionnaire 1 15

Page 4: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires4

Specifying exactly what information is to be collectedis key to devising a successful questionnaire.

Think about the target audiences, as questions mayneed to deal with the concerns of those directlyaffected by the issue as well as those indirectlyaffected. A whole population questionnaire will haveto address this variety of viewpoints on the samequestionnaire. When a targeted sample is beingsurveyed, questions can be more tailored.There are three factors to consider when devising thecontent of a questionnaire:

� relevance – does the subject of the questionprovide information related to the consultation?

� clarity – can the question be misinterpreted or isit ambiguous?

� appropriateness – does the respondent have thenecessary information to answer the question?How articulate is the respondent likely to be?Does the question rely on memory – ‘do yourecall?’

Whenever you are unsure of a question refer to theconsultation aims and objectives. Avoid thetemptation to ask questions because it would be‘interesting to know’.

Questionnaires can be for a whole population (acensus) or a sample.

Whole population – this is where an entire audienceis surveyed and is most common. On projectspecific consultations – for example on a trafficmanagement scheme – everyone living within acertain distance from it would usually be consulted.

Sample – this is where just a part of the potentialconsultation audience is surveyed. This may typicallybe used for county-wide consultations where it is tooimpractical or expensive to send a questionnaire andconsultation document to all residents. There mayalso be occasions where time constraints mean thata sample has to be used. On project specificconsultations, the views of a particular group mightalso be of relevance – for example in a consultationon changes to a library, the views of users might alsobe sought through an in-library or outside librarysurvey. It is usually important that the sampleconsulted is generally representative of thepopulation as a whole.

Bear in mind that some members of the targetaudience may be visually impaired, deaf, havelearning disabilities, or do not have English as theirfirst language.

Introduction

Questionnaires are the workhorses of consultation. They allow local peopleto tell the County Council their views in a structured and direct way. Mostconsultations will use some form of questionnaire – they can be as simple asone or two questions on a letter or a more in depth survey dealing with arange of issues including attitudes, behaviour, lifestyle, decisions, needs andwants.

The use of questionnaires in consultation is different to market research – ingeneral responses in consultations will be attributable and feedback given.Although in market research similar techniques are used, its objectives areinherently different. Market researchers are under no obligation to surveyeveryone, to tell anyone what they are doing or why, or to provide any publicfeedback on their results.

However the techniques used by professional market researchers provide afirm basis for best practice when drawing up consultation questionnaires.

Defining the questions Audience

Accessiblity

Page 5: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires 5

There are four main methods of deliveringquestionnaires:

1. postal2. online3. telephone4. face-to-face

Questionnaires can be interviewer-led or available forself-completion. Self completion questionnairesreduce the control that can be exercised over whoresponds and this can make the views receivedunrepresentative of those of the audience as awhole.

Self-completion questionnaires often give a strongerindication of the views of high-interest participants –over-representing people who have strong views(both for and against). This doesn’t necessarilyinvalidate the results but care must be taken to avoidassuming that any self-completion questionnairecaptures the views of a whole audience rather thanthe respondents.

Interviewer led questionnaires offer tighter controlover who is selected to respond, can ask morecomplicated questions and can therefore lead tomore accurate results and deeper insight. However,they are more costly and time consuming than self-completion surveys and the reliability of the resultsdepends on the quality and consistency of theinterviewers.

Table 1: Questionnaire types at a glance

PostalPostal surveys are relatively straightforward and costeffective as they deliver the questions required directto the intended respondents.

Response rates between 5% and 20% are typical, butcan be increased by offering incentives or throughreminders.

The three most commonly cited reasons for the poorresponse to postal surveys are: their length, overuseand the ‘fatigue’ that this generates. Targetedpostal surveys may reduce the general impact and‘fatigue’ and are relatively inexpensive to organise.The Council should be able to organise the mail outof postal surveys using the electoral register, butthere is still a major cost of the postage. However,hand delivery by Council or other staff can reducethis cost and is particularly suitable for small localconsultations.

On-lineWebsites are a passive medium and while potentiallyaccessible to the majority of any population in realitythey tend to attract younger and more economicallyactive people. Unless the consultation is on a topicof particular interest, the numbers completing on-linesurveys tends to be small. However, these groupsalso tend to be less easily reached by other surveymethods, so an online survey can still produce usefulresults.

On-line surveys are also particularly useful forobtaining the views of council staff. Many haveaccess to a computer and are a ‘captive audience’at work.

Unless the Council has large numbers of e-mailaddresses on-line questionnaires are poor forsampling, but responses may still reveal informationthat is useful and the cost of such surveys is low.Response rates can be boosted through advertisingon-line questionnaires through traditional mediasuch as newsletters.

Questionnaire types

Postal � � � �

Online � � � �

Telephone � � � �

Face-to-face � � � �

Whole

population

Sample

Interviewer

led

Self

completion

Page 6: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires6

Simple electronic versions of printed surveys – usuallya pdf or word file – should be avoided. But thismethod is useful for making available large print orminority language versions where they can beprinted off and sent back to the consultor. Theavailability of on-line large print versions (and whenused alternate community language versions) maybe advertised on the paper version.

On-line questionnaires are most effective when theyare in interactive format, allowing the respondent tocomplete it on-line and where inapplicable questionscan be filtered out as a respondent completes thequestionnaire.

Access to email addresses with which to contactpotential respondents is usually the limiting factor ingenerating a high level of response. On-line surveysneed to be given publicity through other media.

On-line questionnaires are more vulnerable to beingskewed by interest group campaigns and this is oftencited as a reason to not carry out consultation on-line. However, well managed relationships with suchgroups (including ‘the opposition’ to proposedinitiatives) should help to minimise any campaign.The existence of an organised on-line campaign is initself one indication of the strength of public feelingon the issue under discussion.

TelephoneTelephone interviews allow questions to be askedspeedily and under close supervision and can use atightly controlled sample that is drawn from thetarget audience. Market researchers often usecomputer aided telephone interviewing (CATI) toensure consistency and accuracy of the data records.Telephone surveys offer a cost effective means ofsampling a large population in a controlled andscientific manner1.

If there is to be voice recording of an interview,respondents must be informed about this at thebeginning of the interview.

Face-to-faceFace-to-face interviews usually involve selecting acarefully controlled sample of the target audienceand visiting them in their own homes. Face-to-faceinterviews allow more complex issues to be exploredand the use of visual aids.

Face-to-face interviews can also be used for smallerconsultations – for example changes to a localfacility – where there is a clearly defined localpopulation in the surrounding area and it is feasibleto visit them all. They can also be conducted on-street with interviewers selecting a sample or invitingpassers-by to respond.

If there is to be any recording, monitoring orobservation during an interview, respondents mustbe informed about this both at recruitment and atthe beginning of the interview.

Each survey method has its own merits and marketresearch best practice emphasises that in anyresearch strategy it is desirable to use severalmethods to gain a broader perspective of a targetaudience.

The advantages and disadvantages of each responsemethod can be balanced by using complementaryapproaches. Because of the different strengths andweaknesses and their different audience appeal, theresponses to different methods should never becombined. They should be reported separately.

Combining survey techniques

1 Telephone surveys cost around £2,500 for 500completed calls.

Page 7: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires 7

Give your questionnaire a title that is short andmeaningful to the audience. A questionnaire with atitle and clear purpose is generally perceived to bemore credible than one without and is more likely tobe completed.

Another way to make a questionnaire interesting isto provide variety in the type of questions used.Varying the question format will also preventrespondents from falling into a pattern of answers(such as answering every question with option b).

There are two main types of questions:

� open – where space is given for the respondentsto answer in their own way

� closed – where questions are followed bystructured responses (for example simple YES orNO tick boxes or multiple choice answers).

Open questions reveal more information than closed,but are more difficult to interpret and tabulate.

Both open and closed questions can be used inconjunction with each other to provide additionalinformation, to pinpoint particular difficulties and toaid the interpretation of responses.

The Market Research Society has identified fourmajor issues that are known to have a negativeimpact on the quality of data collected andrespondents’ attitudes towards events. These are:

1. excessive length of questions2. repetitive questions3. insufficient opportunity for respondents to

comment4. excessive personal information sections.

Question sequence and layoutWherever possible, a questionnaire should follow anatural flow, reflecting a train of thought, a logicalconversation or a chronology of events, dependingon the subject matter.

The order of questions will have an effect on theanswers that are given. This is easier to control in aface-to-face setting but with self completionquestionnaires it is not possible to change the orderof questions, so extra care needs to be taken.People tend to look at the first few questions beforedeciding whether to complete the questionnaire. Sointeresting questions should come first, along withthe most important issues, so even partiallycompleted questionnaires may still contain importantinformation.

Personal and intrusive questions should be avoided(and this has implications for equal opportunitiesmonitoring).

Wherever possible, space should be provided toallow respondents to enter their comments on anyother issues not covered by the questionnaire. Thisencourages respondents to feel that their views arevalued and it provides a useful guide to aspects ofthe topic not adequately covered in thequestionnaire.

Answers to closed questionsThe order in which closed answers are presented willaffect how people respond. Just as it is important tovary the format of questions to avoid a pattern ofanswers, multiple-choice answers should also bevaried (for example high to low followed by low tohigh) to avoid people falling into a pattern ofanswers.

The possible responses to closed questions shouldnot overlap – for example the three answers 0 – 5,5 – 10 and 10 or more will confuse. It is far cleareras 0 – 4, 5 –10 and 11 or more.

Questionnaire format

Page 8: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires8

Closed questions should provide a broad range ofanswers to avoid accusations of bias or leading theresponse.

Most closed questions are in the form of an oppositescale – for example the question ‘how helpful do youfind the following…’ might have five answers:

� very helpful

� helpful

� neither helpful nor unhelpful

� unhelpful

� very unhelpful

Alternatively the answers can ‘force’ respondents toone or other side of the debate by removing the‘neither option:

� very helpful

� helpful

� unhelpful

� very unhelpful

Using a scale with an even number of options alsohas the effect of shifting the results away from themiddle, which can be helpful in a consultation thathas no ‘no change’ option.

Closed question answers should include a ‘don’tknow/no opinion’ option and depending on thenature of the question a ‘not applicable’ option toavoid forcing people to answer randomly. This isparticularly important when the question onlyinvolves a yes/no answer. Avoid ‘no response’options as abstainers may simply elect not to answerthe question offering little useful information.

Question wording tips� Ensure the instructions are clear.

� Keep instructions with the questions they applyto.

� Use simple and direct language.

� Do not use uncommon words or long sentences.

� Avoid using biased or potentially loaded words.People will feel that the questionnaire is forcingthem to answer in a particular way and this islikely to reduce the response (and adverselyaffect the reputation of the Council).

� Avoid asking for estimates unless you wantrespondents to make generalisations.

� Ask short, specific, clear, questions about asingle issue.

� Questions that contain multiple concepts shouldbe avoided, for example ‘are staff friendly andefficient?’ Questions like these are moreeffective if split into two. Break down complexissues.

� Avoid double negatives – for example ‘would younot use a non-smoking area’?

� The length of a questionnaire will determine thenumber of responses – so try to keep thenumber of questions down. Avoid duplicationand unnecessary questions.

� On written questionnaires use a legible typeface(14 point san serif is recommended by the RoyalNational Institute of the Blind and is thereforegood practice. Anything less than 12 pointshould be avoided).

� Space questions out. Leaving white spacemakes a written questionnaire easier on the eyeand this increases the response.

� Ensure plenty of space for answers to open-ended questions.

� Print the return address on a writtenquestionnaire itself (since questionnaires oftenget separated from the reply envelopes).

� Don’t make assumptions, for example some ofour customers are in single sex relationships,wheelchair users don’t always considerthemselves to be disabled, so phrase questionssensitively.

Page 9: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires 9

It is not sufficient just to produce and design aquestionnaire – it will need to be part of a packageincluding an envelope (and any reply paid envelope),covering letter and other consultation material.

Provide a well-written covering letter. It offers thebest chance to persuade the respondent to completethe survey and it should contain clear instructions forcompleting and returning the questionnaire as awhole.

Letters should be addressed to either namedrecipients or unnamed recipients in the singular(‘Dear resident’ will gain a greater response than‘Dear residents’ and letters should be wordedaccordingly).

The reliability of a survey is determined by thestrength of the sampling frame – the selection ofsubjects from the total population, the sample size,the margin of error, confidence level, overallpopulation size and the percentage of the studypreferring one option rather than another.

A sample of 1,000 would normally enable results tobe predicted within roughly three percentage points.

Table 2: Sample size and margin of errorfor a study of 6,033 service users.

The margin of error takes into account anyinaccuracies in the characteristics of the sample andthe population as a whole. It is expressed as thenumber of percentage points above or below thestated result that the actual result lies. For exampleif 45% of 547 randomly selected respondents saythey prefer a particular service then in 95 cases outof 100 (the confidence level), the actual value of thetotal population would be between 41% and 49%(45 +/- 4%).

Smaller samples can be used so long as the relevantlimits on the reliability of the data are made clear.As table 2 shows, a sample of 547 can give a resultwithin a margin of error of 4 percentage points.

Cost and the degree of accuracy required are keydeterminants of the sample required.

There are three different methods of selecting asample.

Random sampling – whereby each member of thestudy population has an equal and random chanceof being selected.

Systematic sampling – where every Nth record isselected from a list containing the whole studypopulation. It is a simple technique and is frequentlyused to select a specified number of records fromtelephone list or electoral role.

Questionnaire presentation

SamplingSelecting a sample

Margin of error 1 2 3 4 5 6 7 8 9 10

Sample required 3,748 1,727 909 547 362 256 190 147 116 95

Page 10: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires10

Stratified sampling – a sample is created by taking aspecific number of individuals from each stratum (orgroup) in proportion. A stratum is a subset of thepopulation that shares at least one commoncharacteristic – for example ethnicity. So a stratifiedsample is designed to reflect the make up of theoverall population, so would include, for example,similar proportions of black and minority ethnicpeople, disabled people and reflect the age profile ofthe county.

A small number of pilot interviews should be carriedout to test the appropriateness of a questionnaire. Itmay be necessary to test whether the questionnaireis compatible with the rights of respondents,particularly where sensitive issues or subjects areinvolved.

Questions should be piloted on a test sample to testfor length, clarity and good sense. The sampleshould preferably have the same characteristics asthe target audience and have a range ofbackgrounds and abilities to ensure that thequestionnaire really is accessible to all. As aminimum, if a sample of the target audience cannotbe used, you could try out an internal pilot, with halfa dozen colleagues to see if there are any majorproblems.

Certain circumstances demand particularly rigorouspiloting. For example:

1. when dealing with unfamiliar concepts –observation (watching pilot interviews) orqualitative pilots can help in framing questions

2. if there are areas in the questionnaire where awide range of responses to open questions isexpected, pilots can help generate codes forthese responses

3. where it is obvious the interviews are likely to bevery complex or lengthy

4. where questions are potentially contentious

5. where it is necessary to test and reviewalternative question strategies.

A pilot will show if the questionnaire generates theinformation you want, whether any of the questionsneed to be rephrased and it will also give anestimate of how long it takes to complete.

It is usually useful to be present when pilotrespondents are completing the questionnaire. Italso allows a direct exchange of any concerns aboutthe appropriateness of the questionnaire.

There are two common but not essential methods ofboosting responses:

1. use direct follow-ups (face-to-face or phone) orwritten reminders including another copy of thequestionnaire

2. use incentives such as entry into a prize draw orcharity donation. Incentives need to be handledcarefully and for some issues would beinappropriate

Piloting and revising

Increasing responses fromself-completion surveys

Page 11: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires 11

Although there are no limits on the answers that canbe given to open questions, the vast majority ofanswers will fall into a number of broad themes.Coding is the process whereby similar answers aregrouped into these themes, allowing a large numberof ‘freeform’ responses to be analysed and clear,simple, results obtained.

Once coded, data from both closed and open-endedquestions can be placed into a common table andthe relationships between categories investigated.For example the views of service users and non-userscan be separated and compared; similarly theresponse from different parts of the county can beexamined.

Processing answers to questionnaires is timeconsuming and often requires specialist analysis.There are a number of software packages that canhelp and market research agencies can provide adata analysis service.

Computer programmes can offer a very sophisticatedway of analysing survey data and producingstatistics. But for most purposes ‘frequency counts’(how many people answered each question eachway) and a simple summary of views will suffice.

Sometimes, having an independent third party toundertake the coding and reporting of responses cangive greater confidence that coding has not beenused to support a particular view. Verification(checking by an external party that coding has beendone accurately) can often be as costly, so if a thirdparty is likely to be asked to provide assurance in acontroversial consultation, it is usually better to getthem to draft the analysis report.

PresentationGraphical displays - histograms, line and scattergraphs, pie charts are usually easier to understandthan tables of numbers and should be used tosupport analysis and conclusions.

The 12-step survey process

1. Scoping.2. Selecting the sample.3. Choice and combination of questionnaire

types.4. Devising the questionnaire

• wording• format• sequence• layout• presentation

5. Conduct pilot.6. Revise questionnaire (if necessary).7. Printing/production.8. Distribution/fieldwork.9. Pursuing non-responses.10. Closing date for responses.11. Data analysis

• coding open-ended questions• data entry and verification• analysis.

12. Preparation of the report.

Analysis

Page 12: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires12

1. Recruitment

1.1 Tell respondents:

• the name of the interviewer(an interviewer’s identity card should beshown if face-to-face)

• the general subject of the interview• the purpose of the interview• the likely length of the interview

1.2 Allow respondents to check the identity andbona fides of any individual and/or theiremployer conducting a research project(including any contractors).

1.3 Respondents should be openly asked forconsent to take part in research, and tosuggest any attributed use of their commentsand concerns.

1.4 Respondents (including employees) must not beunduly pressurised to participate.

1.5 If there is to be any recording, monitoring orobservation during an interview, respondentsmust be informed about this both atrecruitment and at the beginning of theinterview.

1.6 Respect the right of respondents to withdrawfrom a research project at any stage.

1.7 For telephone and face to face interviews, callsmust not be made to households before 9amweekdays and Saturdays, 10am Sundays orafter 9pm any day, unless by appointment.

2. Children

2.1 The consent of a parent or responsible adult(acting in loco parentis) should be obtainedbefore interviewing a person under 16including during:

• in home/at home (face-to-face andtelephone interviewing)

• group discussions/depth interviews

• postal questionnaires

• internet questionnaires

• email

• where interviewer and child are alonetogether

• in public places such as in-street/in-store/central locations.

2.2 Personal information relating to other peoplemust not be collected from children unless forthe purposes of gaining consent from a parentor a responsible adult.

2.3 Where the consent of a parent or responsibleadult is required, ensure that the adult is givensufficient information about the nature of theresearch to enable them to provide informedconsent.

2.4 Record the name, relationship or role of theparent or responsible adult giving consent.

2.5 When it is known (or ought reasonably to beknown) that all or a majority of respondents arelikely to be under 16, self-completion postalquestionnaires should be addressed to theparent or responsible adult and should includea note explaining that consent is required forchildren to participate.

2.6 For research administered electronically overthe internet, when it is known (or oughtreasonably to be known) that all or a majorityof respondents are likely to be under 16,respondents should be asked to give their agebefore any other personal information such asname, address or other contact details isrequested. If the age given is under 16, thechild must be excluded from giving furtherpersonal information until the appropriateconsent from a parent or responsible adult hasbeen obtained.

2.7 Always ensure that a child has an opportunityto decline to take part, even though a parent ora responsible adult has given consent on theirbehalf. This remains the case even if theresearch takes place in school.

2 These tips are derived from the MRS code of conduct.

Appendix 1Tips on conducting research projects2

Page 13: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires 13

3. Anonymity

3.1 The anonymity of respondents should bepreserved unless they have given consent fortheir details to be revealed or for attributablecomments to be passed on.

3.2 Be particularly careful if sample sizes are verysmall (for example in business and employeeresearch). This can sometimes inadvertentlyreveal the views of individual respondents.

3.3 Do not reveal to respondents the answers oridentity of previous respondents.

3.4 Respondent details should not be passed on tothird parties for research or any other purposeswithout the prior consent of the respondent.

3.5 If respondents have given consent for data tobe passed on in a form which allows them tobe personally identified:

• take all reasonable steps to ensure that itwill only be used for the purpose for which itwas collected; and

• inform respondents as to what will berevealed, to whom and for what purpose.

3.6 Delete any responses given by the respondent,if requested, and if this is reasonable andpracticable.

4. Qualitative research

4.1 In qualitative research, explain

• the location of the discussion and if it is totake place in a viewing facility

• whether observers are likely to be present

• when and how the discussion is to berecorded

• the likely length of the discussion includingthe start and finish time

• the name of the moderator and/or researchagency that will be conducting the research.

4.2 Ensure that observers are fully introducedbefore the group/interview begins and makesure respondents are given a chance towithdraw.

4.3 Make clear to respondents the capacity inwhich any observers are present.

4.4 In instances where observers may knowrespondents, respondents should be informedbefore the start that their interviews are to beobserved, with a warning that the observer mayinclude people who already know them.

5. Mystery shopping

5.1 The objective of a mystery customer researchproject is to provide management informationon processes and/or quality of service in orderto aid training and retraining plans,improvements in service and increase customersatisfaction.

5.2 For mystery shopping exercises the ‘respondent’will be the person delivering the service, whichis being researched by the mystery shopper. Assuch, there are different levels of allowabledisclosure and data usage.

5.3 Employees should be advised that their servicedelivery might be checked through mysteryshopping.

5.4 The objectives and intended uses of the resultsshould have been made clear to staff (includingthe level of reporting if at departmental/locationor individual level) particularly if mysteryshopping is to be used in relation to anyemployment terms and conditions.

5.5 Audio, photographic or video equipment shouldnot be used in mystery shopping unless allpotential respondents have been asked for theirpermission.

Page 14: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires14

6. Observation

6.1 Clear and legible signs must be placed in areaswhere surveillance is taking place.

6.2 Signs must state the individual/organisationresponsible for the surveillance, includingcontact information and the purpose of theobservation.

6.3 Cameras must be sited so that they monitoronly the areas intended for surveillance.

6.4 Attitudes to the presence of surveillanceequipment can change during the course of aninterview and it may suddenly become anunwarranted intrusion. The ability to end theobservation quickly must be built into anyresearch situation and the moderator/facilitatormust be sensitive to the possibility that theremay be a need to end recording and beprepared to do so.

7. Analysis and reporting

7.1 The conclusions of research should be clearlyand adequately supported by the data.

7.2 Provide sufficient technical details to enablepeople to assess the validity of results ofresearch projects.

7.3 Ensure that data tables include sufficienttechnical information to enable reasonableinterpretation of the validity of the results.

7.4 Comply with any reasonable additional requeststo make available further information necessaryto assess the validity of any published findings.

7.5 When interpreting data make clear which datais being used to support the interpretation.

7.6 Distinguish between facts and interpretation inall reports and presentations.

7.7 If the findings from a research project areincorrectly or misleadingly reported in themedia, provide the relevant technical detailsand results necessary to correct any incorrector misleading report at the earliest opportunity.

8. Data storage

8.1 Take reasonable steps to ensure that all hardcopy and electronic lists containing personaldata are held securely in accordance with therelevant data retention policies and/orcontractual obligations.

8.2 Take reasonable steps to ensure that thedestruction of data is adequate for theconfidentiality of the data being destroyed. Forexample, any personal data must be destroyedin a manner that safeguards confidentiality.

Page 15: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

Date: Code No:Office Use OnlyQuestionnaire No:

1. Which of the following best describes where youset out from today? READ LIST

Your own homeA friend’s or relative’s homeHoliday accommodationOther (please specify)

2. What was the main method of transport used toget here today? ONE RESPONSE

Car / vanBus/CoachMotorcycleBicycleOn footOther (please specify)

Appendix 2Sample face to face questionnaire 1

No. 6 • How to design and use questionnaires 15

NOTTINGHAMSHIRE COUNTRY PARK- VISITOR RESEARCH AUTUMN 2006

Good afternoon. My name is and I work for .We are undertaking some research into people visiting Sherwood Forest /Rufford Country Park. It should only take a few minutes. Would you be willingto take part?

Interviewer to complete:Sample point

Sherwood Forest RuffordSherwood Forest Visitor Centre CourtyardForest table restaurant The Mill

Time interview completed: 10.30 – 13.0013.01 – 15.0015.01 – 17.00

Day of the week: Monday FridayTuesday SaturdayWednesday SundayThursday

1

2

1

2

3

4

1

2

3

3

4

1

2

3

4

5

6

7

1

2

3

4

5

6

Page 16: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires16

3. Have you visited THIS country park before?

YesNo → GO TO Q5

4. ASK ONLY IF RESPONDENT HAS VISITEDTHIS COUNTRY PARK Approximately howmany times have you visited in the past 12months? READ LIST

Daily2 or 3 times a weekOnce a weekLess than 3 times a monthOnce a month3 times a yearOnce every 6 monthsOnceNot visited in past 12 months

5. ASK ALL Which of the following prompted yourvisit today?

Previous visit/local knowledgeFriend/relative’s recommendationGuidebookLeafletNewspaper/magazineRadio or TVWebsite (Internet)Signpost/when driving pastSaw it on a mapChild’s visit with schoolTourist Information CentreOther

→ Please specify:

Special event

6. If you visited the shops how would your ratethem, on the following scale (SHOWCARD)for….

Quality of goodsChoice of goodsQuality of serviceEase of gettingaround the shopLength of queuesValue for money

7. Do you or anybody in your party have anyphysical, sensory or learning impairment thatcould limit your use of this country park and itsfacilities?

YesNo

8. What measures do you feel could be put inplace to make the country park and its facilitiesmore accessible to people with physical, sensoryor learning impairments?

Verygood Good Ok Poor

Verypoor

Don’tknow

1

2

3

4

5

6

7

8

9

10

11

12

1

2

1

2

1

2

3

4

5

6

7

8

9

1

1

1

1

1

1

13

2

2

2

2

2

2

3

3

3

3

3

3

4

4

4

4

4

4

5

5

5

5

5

5

6

6

6

6

6

6

Page 17: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires 17

9. ASK ALL Do you feel this country park is(READ LIST)…

Very well maintainedWell maintainedAveragely maintained / OKPoorly maintainedVery poorly maintained

Finally can I ask you a few questions aboutyourself?

10. Including yourself, how many in your immediateparty are male/female and looking at the cardprovided, in what age groups are they?(SHOWCARD)

11. What is your full home postcode?

12. Are you, or anyone in your party registereddisabled?

YesNo

13. Finally, could I please have your name and acontact telephone number? I should emphasisethat this will be treated in the strictest confidenceand will only be used for the purpose of back-checking the questionnaire.

Name of respondent:

Home / Business / Mobile Tel No:

THANK AND CLOSE

Declaration: I declare that the respondent wasunknown to me until the interview took place, andthat this questionnaire has been conductedaccording to instructions and has been checked.

Interviewer name:

Interviewer no.

Signed:

Date:

1

2

3

4

5

1

2

Nottinghamshire holds an annual consultation on itsbudget – consultation material and the questionnairecan be obtained from the CorporateCommunications Team.

Page 18: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

No. 6 • How to design and use questionnaires18

Notes

Page 19: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

Copyright:

All County Council copyrights are reserved. The supply ofCouncil information does not grant the applicant the right to re-use the information in any way that would infringe the copyrighton that information.

For any wider use, the County Council may choose to allow there-use under licence. This enables the County Council to ensureit is not used in a manner which is inconsistent with the CountyCouncil’s copyright.

Page 20: Guide 6 - How to design and use questionnaires...audience 9 Analysisand writingupthe consultation 5 Methods– capturing people’s views 4 Consulting withpeoplewe findhardto reach

Nottinghamshire County Council’s consultation standards

1. A consultation mandate should be completed and approved by your departmental co-ordinator/champion before the start of any consultation. (Guides 2 and 11)

2. The consultation should be logged on the County Council’s consultation database.(Guides 2, 10 and 11)

3. For key decisions a public engagement plan should be published with the forward plan.(Guides 2 and 11)

4. In planning a consultation, the specific communication needs of groups and individuals who areoften otherwise excluded should be considered. (Guides 3 and 4)

5. A minimum of 12 weeks should be allowed for consultation on major decisions. (Guide 2)

6. The name, address and contact number of the person responsible for the consultation should bepublished on all consultation materials. (Guides 2 and 10)

7. The consultation material should make clear by what date responses are required, in whatformat and to whom they should be sent. (Guides 2 and 6)

8. The consultation should include a face-to-face element where consultees are able to meet,question and put their views to the decision-maker(s). (Guides 4, 5, 7 and 8)

9. Any venue selected for a consultation event should meet the Council’s accessibility code.(Guides 4 and 8)

10. Any complaints about the consultation, questions asked, materials or time allowed should benoted in the consultation report. (Guides 9, 10 and 11)

11. A notice of decision should be published for each consultation. (Guides 10 and 11)

12. Feedback regarding the responses, the Council’s decision and how the consultation influenced itshould be given to consultees. (Guides 4, 7, 9, 10, 11 and 12)

This publication can be made availablein alternative formats and languagesupon request.

Contacting usemail [email protected] 0115 977 2937fax 0115 977 3886post Chief Executive’s Department, County Hall,

West Bridgford, Nottingham NG2 7QPinternet www.nottinghamshire.gov.ukpublished May 2007

DP&P4.07/CE/5237