Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... ·...
Transcript of Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... ·...
![Page 1: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/1.jpg)
Dillman on Survey Instrument Design
Professor Ron FrickerNaval Postgraduate School
Monterey, California
Excerpted from Mail and Internet Surveys: The Tailored Design Method, second edition, by Don A. Dillman, 2000. 1
![Page 2: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/2.jpg)
Goals for this Lecture
• Define and consider Dillman’s principles for survey instrument design– Make surveys easy for respondents to follow and
answer• Location of instructions• Formatting, including use of white space• Navigational cues• Answer placement, etc.
Advance our understanding of what makes a good survey instrument
2
![Page 3: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/3.jpg)
Visual Consistency and Navigational Clarity Very Important
• Dillman:– “Define a desired navigational path for reading all
information presented on each page of the questionnaire.”
– “Create visual navigational guides and use them in a consistent way to get respondents to follow the prescribed navigational path and correctly interpret the visual information.”
• Make it easy to follow in order to minimize respondent errors and drop-outs
3
![Page 4: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/4.jpg)
For Paper-based Surveys…
• …Dillman recommends:– Print on 8½” x 14” paper and fold into 8½” x 7” booklets– Print on11” x 17” paper and fold into 8½” x 11” booklets– Print on single side of 8½” x 11” paper and staple in upper
left corner
• Last option lacks a professional look but is often cheaper to produce
• Of course, these paper-size issues don’t apply to web-based surveys– But clarity and consistency of the instrument design are just
as important– Principles that follow are just as applicable to web surveys
4
![Page 5: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/5.jpg)
Survey Format Example
55
![Page 6: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/6.jpg)
Survey Format Example
6
Double column format using 8½” x 11” paper
![Page 7: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/7.jpg)
Make the Survey Easy to Follow
7
![Page 8: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/8.jpg)
A Survey that is Hard to Follow
8
![Page 9: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/9.jpg)
Put Instructions Where Needed(1 of 2)
9
![Page 10: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/10.jpg)
Put Instructions Where Needed(2 of 2)
10
![Page 11: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/11.jpg)
Good First Questions are Critical (1 of 2)
1111
![Page 12: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/12.jpg)
Good First Questions are Critical (2 of 2)
12
![Page 13: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/13.jpg)
Only Ask One Question at a Time (1 of 2)
13
![Page 14: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/14.jpg)
Only Ask One Question at a Time (2 of 2)
14
![Page 15: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/15.jpg)
For Qs with Common Responses, Use Item-in-Series Format (1 of 2)
15
![Page 16: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/16.jpg)
Item-in-Series Format (2 of 2)
16
![Page 17: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/17.jpg)
Minimize the Use of Matrices
17
![Page 18: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/18.jpg)
Use Visual Elements to Improve Readability
18
![Page 19: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/19.jpg)
Use Spacing and White Space to Appropriately Group Elements
19
![Page 20: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/20.jpg)
Keep the Format Consistent (1 of 2)
20
![Page 21: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/21.jpg)
Keep the Format Consistent (2 of 2)
2121
![Page 22: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/22.jpg)
Include Instructions as Part of Question When Possible (1 of 2)
2222
![Page 23: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/23.jpg)
Include Instructions as Part of Question When Possible (2 of 2)
23
![Page 24: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/24.jpg)
Answer Category Placement (1 of 3)
24
![Page 25: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/25.jpg)
Answer Category Placement (2 of 3)
25
![Page 26: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/26.jpg)
Answer Category Placement (3 of 3)
26
![Page 27: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/27.jpg)
Put Answers in One Column
27
![Page 28: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/28.jpg)
Be Consistent in Scale Ordering
28
![Page 29: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/29.jpg)
Use Visual Cues to Guide Respondents (1 of 2)
29
![Page 30: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/30.jpg)
Use Visual Cues to Guide Respondents (2 of 2)
30
![Page 31: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/31.jpg)
Emphasize Words Sparingly
31
![Page 32: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/32.jpg)
Which Survey Would You Prefer to Take?
32
?
![Page 33: Dillman on Survey Instrument Design - Facultyfaculty.nps.edu/rdfricke/MCOTEA_Docs/Lecture 2... · Visual Consistency and Navigational Clarity Very Important • Dillman: – “Define](https://reader034.fdocuments.us/reader034/viewer/2022052611/5f08706d7e708231d4220498/html5/thumbnails/33.jpg)
What We Have Covered
• Discussed Dillman’s principles for survey instrument design– Goal: Make surveys easy for respondents to follow
and answer• Location of instructions• Formatting, including use of white space• Navigational cues• Answer placement, etc.
Advanced our understanding of what makes a good survey instrument
33