Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.
-
Upload
lee-thomas -
Category
Documents
-
view
243 -
download
3
Transcript of Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.
![Page 1: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/1.jpg)
Chapter 7: Appendix - 1
Questionnaire Surveys: Typology, Design and Coding
![Page 2: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/2.jpg)
Introduction
• Definitions and terminology• Roles• Merits of questionnaire surveys• Limitations• Interviewer-completion or respondent-completion?• Types of questionnaire survey
![Page 3: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/3.jpg)
Definitions
• Questionnaire or ‘interview schedule’:– A printed or on-line list of questions
• Survey– Whole process of conducting an investigation which
involves a number of ‘subjects’
• Questionnaire survey– A survey involving the use of a questionnaire
• i.e. a ‘survey is not a ‘questionnaire’
![Page 4: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/4.jpg)
Roles of questionnaire surveys
• Used when a specified range of information required
• Typically involve just a sample of the population being studied – For implications see - Sampling
• But, the aim is to make statements about the population as a whole
![Page 5: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/5.jpg)
Merits of questionnaire surveys
• An ideal method of providing policy-related data
• Transparent methodology
• Quantification easily communicated/understood
• Repeat surveys can study change over time
• Can cover a wide range of (leisure/tourism) activities
• Can study attitudes, meanings, perceptions of population as a whole
![Page 6: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/6.jpg)
Limitations of questionnaire surveys
• Samples
• Self-reported data
– Exaggeration/under-reporting
– Accuracy of recall
– Sensitivity to some questions
![Page 7: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/7.jpg)
Interviewer-completion Respondent-completion
Advantages • More accuracy• Higher response rates• Fuller & more
complete answers• Design can be less
‘user-friendly’
• Cheaper• Quicker• Relatively anonymous
Disadvantages • Higher cost• Less anonymity
• Patchy response• Incomplete response• Risk of frivolous responses
• More care needed in design
Figure - Interviewer-completion or respondent-completion?
![Page 8: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/8.jpg)
Type Int. or resp.completion
Cost Sample drawn from:
Response rate
Household
StandardEither Expensive Whole popn* High
Time-useRespondent Expensive Whole popn* High
OmnibusEither Medium Whole popn* High
* refers to population of subjects to be studied
Figure - Types of questionnaire survey
![Page 9: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/9.jpg)
Type Int. or resp.completion
Cost Sample drawn from
Response rate
Street Interviewer Medium Most of population
Medium
Telephone Interviewer Medium People withland-line phone
High but falling
Mail Respondent Cheap General or Special
Low
Figure - Types of questionnaire survey (Continued)
E-survey Respondent Cheap People access-ible via email/ internet
Medium
On-site Either Medium Site users only High
Captive Respondent Cheap Captive group only
High
![Page 10: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/10.jpg)
Household survey
• Nature – Can cover all adult age-groups– Representative of whole community – Common for government, market research, etc.
• Conduct– Typically interviewer-completed, but also ‘drop-off and collect’ with
respondent-completion sometimes used – or combination– Sampling
• Omnibus surveys– One questionnaire includes questions on a number of topics for
multiple clients• Time-budget studies
– Respondents complete a 1 or 2-day diary of activities• National surveys
– Often seen as secondary data sources
![Page 11: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/11.jpg)
Street surveys
• Nature– Conducted in:
• Shopping street/mall• Tourism areas (but may be seen as ‘site survey’)• Transport nodes (bus-stations, airports) - tourism
• Conduct– Interviewers typically given quotas related to known
demographics of the community
– Still a problems of representativeness re people who• are housebound • do not visit shopping streets • do not visit particular tourist locations (e.g. VFR, business
tourists)
![Page 12: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/12.jpg)
Telephone survey
• Nature– Common for political polling– Growing response and representativeness problem
because of • excluding those without land-line telephones• growing resistance/non-response in some countries
• Conduct– Fast, using computer-aided telephone interviewing(CATI)
• Numbers selected/dialled automatically• Data keyed directly into computer
– Not possible to show lists to respondents– Anonymity may help honesty of response
![Page 13: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/13.jpg)
Mail/postal survey
• Nature– Ideal for surveying widely dispersed lists
• E.g. Members of organisations
• Conduct– Low response rates – e.g. 30 % often quoted as
‘acceptable’ …. but is it?
– Factors affecting response rates (next slide…)
• Mail and user/site survey combos– In some on-site user/visitor surveys a brief face-to-face
interview is combined with a hand-out questionnaire which respondents complete and mail back after their visit
![Page 14: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/14.jpg)
1. Interest of the respondent in the survey topic
2. Length of the questionnaire
3. Questionnaire design/presentation/complexity
4. Style, content, authorship of accompanying letter
5. Provision of a postage-paid reply envelope
6. Rewards for responding
7. Number and timing of reminders/follow-ups
Factors affecting mail survey response rates
![Page 15: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/15.jpg)
0
10
20
30
40
50
60
70
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50Days
ReminderCard sent Follow-up
Letter sent
Start
0
10
20
30
40
50
60
70
80
Daily returns
Cumulative%
Mail survey response pattern
![Page 16: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/16.jpg)
E-surveys
• Nature– Conducted via email/Internet– Dependent on availability of e-mail list or other
means of inviting sample to participate• Conduct
– Partially or Fully electronic – Commercial organisations offer online service,
including on-line questionnaire design, and analysis
– Problem of response rate due to growth of ‘junk mail’
![Page 17: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/17.jpg)
TypeRequest Questionnaire Completion Return
Hybrid email/mailEmail Attached text file Manual on hard
copyMail
Hybrid emailEmail Attached text file Word-process-
or/spreadsheetEmail + text file
Fully electronic:ad
hoc
Email On-line, interactive Online On-line submission
Fully electronic: panel
Panel email
On-line, interactive Online On-line submission
Types of e-survey
![Page 18: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/18.jpg)
User/on-site/visitor surveys• Nature
– Survey conducted at a leisure facility/site– Alternative terms:
• On-site survey, site survey, user survey, visitor survey, audience survey
– Commonly used by facility/site managers
• Conduct– Interviewer completion preferred for quality/ response rate– Respondent-completion, if not very closely supervised, can result
in:• Low response rates• Unrepresentative sample• Poor quality responses (incomplete etc.)
• On-site/mail combo (mix)– face-to-face interview can be followed by handout of a mail-back
questionnaire – particularly in long-stay sites, such as a nationalpark or theme park
![Page 19: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/19.jpg)
Uses of User/on-site/visitor surveys
• Establish catchment area (place of residence)• User profile (age, gender, socio-economic
group)• User opinions• Non-users :
– Use census data to assess non-users within the catchment area
![Page 20: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/20.jpg)
Captive (ready) group surveys
• Nature– Respondents in organised group– May have little choice but to participate– Ethically, must be given the option
• Conduct– Typically respondent-completed under supervision– Quick and cheap
![Page 21: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/21.jpg)
Example questionnaires:Case study
A. Questionnaire for a site/street survey:respondent-completed: Campus Life Survey
B. Questionnaire for a household survey: interviewer-completed: Short-stay Holiday Survey
C. Questionnaire for a site survey: interviewer-completed: Ramsey Street Park Survey
![Page 22: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/22.jpg)
Principle Bad example Improved version
Use simple language
What is your frequency of utilisation of retail travel outlets?
How often do you use travel agents?
Avoid ambiguity Do you play sport very often?
Have played any of the following sports within the last four weeks? (show list)
Avoid leading questions
Are you against the extension of the airport?
What is your opinion on the extension of the airport? Are you for it, against it or not concerned?
Ask just one question at a time
Do you use the local arts centre, and if so what do you think of its facilities?
1. Do you use the local arts centre? Yes/No2. What do you think of the facilities in the local arts centre?
General design issues: wording of questions
![Page 23: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/23.jpg)
• Open-ended:– What is the main constraint on your ability to study?
________________________________________
• Pre-coded/closed:– A. My job ☐1
– B. Timetabling ☐2
– C. Child care ☐3
– D. Spouse/partner ☐4
– E. Money ☐5
– F. Energy ☐6
– G. Other __________ ☐7
• In interview situation card shown to interviewee
Pre-coded vs open-ended questions
![Page 24: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/24.jpg)
Open-ended questions produce large numbers of answers
• Question: Do you have any complaints about this (beach/picnic) area? (Site survey in a beachside National Park with boating and camping. Number of responses in brackets)– Sand bars (22) - Uncontrolled boats (23)– Parking (5) - Jet skis (39)– Wild car driving (1) - Surveys (1)– Lack of beach area (1) - Should be kept for locals (1)– Too few shops (1) - Seaweed (3)– Too few picnic tables (4) - Need showers (1)– No timber for barbecue (2) - Administration of National Park (1)– Need more picnic space (3) - Maintenance & policing of Park (1)– Need boat hire facilities (1) - Trucks on beach (2)– Need active recn facilities (1) - Anglers (1)– Litter/pollution (74) - Crowds/tourists (26)– Urban sprawl (1) - Having to pay entry fee (6)– Need wharf fishing access (1) - Houses along waterfront (2)– Lack of info. on walking trails (1) - Unpleasant smell (drain) (2)– Not enough facilities (3) - Sales people (1)– Slow barbecues (2) - Need electric barbecues (1)– Etc. - Etc.
Example of range of replies resulting from an open-ended question
![Page 25: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/25.jpg)
Types of information
1.Activities/events/places What?
2.Respondent characteristics Who?
3.Attitudes/motivations Why?
![Page 26: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/26.jpg)
% of persons aged 16+ attending in last:
12 months 4 weeks
Film: cinema/other venue 59 22
Play or drama 25 4
Carnival, street arts, circus 26 4
Art/photography /sculpture exhib. 22 6
Craft exhibition 19 4
Attendance at arts events, England, 2003
![Page 27: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/27.jpg)
Respondent characteristics: range of data
• Gender• Age• Economic status• Occupation/social class
(own or ‘head of household’)
• Previous job history• Income (own or household)
• Education/qualifications
• Marital/family status • Household type/family size• Life-cycle• Ethnic group/country of
birth• Residential location• Mobility - driving licence,
access to private transport• Party/group size/type
(site/visitor surveys)
![Page 28: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/28.jpg)
Attitude/opinion questions
• Formatsa.Open-ended or direct questions
b.Checklist
c.Ranking
d.Likert scales
e.Attitude statements
f. Semantic differential
g.Repertory grid
![Page 29: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/29.jpg)
a. Open-ended/direct: What attracted you to apply for this course?
____________________________________
b. Checklist: Of the items on the card, which was the most important to you in applying for this course?A. Good reputation
B. Easy access
C. Curriculum
D. Level of fees
E. Easy parking
Attitude/opinion questions
![Page 30: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/30.jpg)
c. Ranking: Please rank the items on the card in terms of their importance to you in choosing a course. Please rank them 1 for the most important to 5 for the least important.
RankA. Good reputation ___B. Easy access ___C. Curriculum ___D. Level of fees ___E. Easy parking ___
Attitude/opinion questions (Continued)
![Page 31: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/31.jpg)
d. Likert scales: Looking at the items on the card, please say how important each was to you in choosing this course; was it: Very important, Quite important, Not very important or Not at all important?
Very Quite Not very Not at allimportant important important important
Good reputation □1 □2 □3 □4
Easy access □1 □2 □3 □4
Curriculum □1 □2 □3 □4
Level of fees □1 □2 □3 □4
Easy parking □1 □2 □3 □4
Attitude/opinion questions (Continued)
![Page 32: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/32.jpg)
e. Attitude Statements: Please read the statements below and indicate your level of agreement or disagreement with them by ticking the appropriate box.
Agree Agree No Disagree Disagree
Strongly opinion stronglyThe learning experience is more important than the
qualification in education □1 □2 □3 □4 □5
Graduate course fees are
too high □1 □2 □3 □4 □5
Attitude/opinion questions (Continued)
![Page 33: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/33.jpg)
f. Semantic differential: Please look at the list below and tick the line to indicate where you think this course falls in relation to each factor listed.
Difficult |_______|________|________|________| Easy
Irrelevant |_______|________|________|________| Relevant
Professional |_______|________|________|________| Unprofessional
Dull |_______|________|________|________| Interesting
Attitude/opinion questions (Continued)
![Page 34: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/34.jpg)
Introductory remarks, Ordering of questions
• Consider content/length of opening/introductory remarks
• Question order:– Start with easy questions– Start with 'relevant' questions– Leave sensitive questions until later
• Layout:• Be aware of the needs of the reader/user – interviewer or
respondent?• Special care with mail survey questionnaires• Compactness (e.g. single page) = ease of handling• Two-column layout often helps
![Page 35: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/35.jpg)
• General:• Non-response• Questionnaire design: lack of clarity• Accuracy of recall• Desire to impress• Privacy concerns/sensitivity• Language/accent• Interviewee patience/fatigue• Physical context• Interviewer-administered• Interviewer-respondent rapport• Interviewer consistency• Respondent-completed• Literacy• Non-completion
Validity/reliability of questionnaire-based data
![Page 36: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/36.jpg)
Checking validity
• Dummy questions or answer categories• Semi-disguised(hidden) duplication of
questions• Comparing participation time periods• Use of/comparing alternative data sources
![Page 37: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/37.jpg)
Planning fieldwork: tasksa. Seek permissions – to visit sites, obtain records, etc.
b. Obtain lists for sampling – e.g. voters lists
c. Arrange printing – of questionnaires etc.
d. Check insurance issues
e. Prepare written instructions for interviewers
f. Prepare identity badges/letters for interviewers
g. Recruit interviewers and supervisors
h. Train interviewers and supervisors
i. Obtain quotations for any fieldwork to be conducted by other organisations
j. Appoint and train data coders/processor
Conducting questionnaire surveys
![Page 38: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/38.jpg)
Pilot survey purposesa. Test questionnaire wording
b. Test question sequencing
c. Test questionnaire layout
d. Code open-ended questions
e. Gain familiarity with respondents
f. Test fieldwork arrangements
g. Train and test fieldworkers
h. Estimate response rate
i. Estimate interview etc. time
j. Test analysis procedures
Conducting a pilot survey
![Page 39: Chapter 7: Appendix - 1 Questionnaire Surveys: Typology, Design and Coding.](https://reader031.fdocuments.us/reader031/viewer/2022031807/56649d0b5503460f949dea75/html5/thumbnails/39.jpg)
End of Appx 1