Post on 09-Apr-2018
8/8/2019 Jac Methods Ch14
1/27
Chapter 14Questionnaire
Development
Winston Jackson and Norine Verberg
Methods: Doing Social Research, 4e
8/8/2019 Jac Methods Ch14
2/27
14-22007 Pearson Education Canada
Questionnaire Design
Questionnaires have many applications and
are widely used
The textbook covers:
General guidelines
Types of questions illustrated
Steps and rules for questionnaire design
Rules for ordering, formatting, and presenting
8/8/2019 Jac Methods Ch14
3/27
14-32007 Pearson Education Canada
General Guidelines
Consult the respondent
Able to express opinion freely, opinions valued
Keep it short
Ask only pertinent questions
Achieve precise measurement
Collect data in its rawest form (income in
dollars, precise occupation, age to the year,not age category)
highest level of measurement possible
8/8/2019 Jac Methods Ch14
4/27
14-42007 Pearson Education Canada
Types of Questions Illustrated
1. Pre-coded, single-choice questions
2. Open-ended questions
3. Presence-absence questions
4. Rank-ordering questions
5. Likert-type questions
6. Index development
8/8/2019 Jac Methods Ch14
5/27
14-52007 Pearson Education Canada
Types: Pre-Coded, Single-Choice
Pre-coded, single-choice questions askrespondents to indicate which one categoryapplies
Answers are pre-coded for easy data entry If not all options can be listed, include a
category entitled Other with a space toindicate what the Other category implies
Please specify ______________ Categories should be mutually-exclusive
i.e., no categories overlap with one another
8/8/2019 Jac Methods Ch14
6/27
14-62007 Pearson Education Canada
Pre-Coded, Single-Choice: Example
4. What year are you in?
Freshman 1
Sophomore 2
Junior 3
Senior 4
See Box 14.1 (p.383) for more examples
8/8/2019 Jac Methods Ch14
7/27
14-72007 Pearson Education Canada
Types: Open-Ended Questions
Open-ended questions do not have pre-setanswers. Excellent way to explore new areas
Use open-ended questions when:
Too many response categories (year of birth) You dont wish to impose categories on
respondents
Really consulting respondents
Qualitative source of quotations Determining appropriate categorization
You want a change in pace for respondents
8/8/2019 Jac Methods Ch14
8/27
14-82007 Pearson Education Canada
Open-Ended: Example
21. What is (or was) your fathers occupation (e.g.,
supervisor, railway machine shop supervises work of
about 25 people)?
Job ________________________________________
Brief Job Description _____________________________________________________________________
23. What is the one thing you would like to see changed
at the university?
________________________________________________________________________________________
More examples in Box 14.3 on page 385
8/8/2019 Jac Methods Ch14
9/27
14-92007 Pearson Education Canada
Open-Ended: Caution
Minimize the number of open-ended questionsbecause they:
Are time-consuming to code
May generate responses that are inconsistentAre more likely to be left blank
May decrease response rate
Takes more time to complete the questionnaire
Use opinion-seeking questions sparingly
Place open-ended questions at 2/3 mark
8/8/2019 Jac Methods Ch14
10/27
14-102007 Pearson Education Canada
Types: Presence-Absence Questions
Presence-absence questions ask
respondents to check off which items in a list
do or do not apply to them
Less commonly used than other types ofquestions
8/8/2019 Jac Methods Ch14
11/27
14-112007 Pearson Education Canada
Presence-Absence: Example
23. Have you ever had contact with a physically
handicapped person in any of these groups? (Circle
to indicate yes or no for each group)
Yes NoCommunity ----------- 1 0
Family ----------------- 1 0
Relatives -------------- 1 0
Elementary school -- 1 0
High school ----------- 1 0
University class ------ 1 0
As a co-worker------- 1 0
8/8/2019 Jac Methods Ch14
12/27
14-122007 Pearson Education Canada
Types: Rank-Ordering Questions
Rank-order questions ask respondents to
indicate an ordering of response items,
usually from most preferred to least preferred
Must be done with great care
Ask for only three most important items
Must make instructions explicit
These types of questions should be avoidedor minimized because they take time
8/8/2019 Jac Methods Ch14
13/27
14-132007 Pearson Education Canada
Rank-Ordering Questions: Example
31. Rank-order the three most important characteristics you
want in the job you make your lifes work. (Place a 1
beside the most important one; a 2 beside the second
most important one; and a 3 beside the next most
important one.)
High salary.. ____
Security... ____
Continued interest..____
Power.. ____Prestige ____
Excitement..____
8/8/2019 Jac Methods Ch14
14/27
14-142007 Pearson Education Canada
Types: Likert-Type Questions
Likert-type questions ask respondents toindicate how much they agree or disagreewith a statement
Response options originally included: stronglydisagree, disagree, are undecided or neutral,agree, strongly agree
Today often used with numbered response
options (see example, next slide) Used in the development of indexes (see
Chapter13)
8/8/2019 Jac Methods Ch14
15/27
14-152007 Pearson Education Canada
Likert-Type Questions: Example
In the following, circle a number to indicate the extent to
which you agree or disagree with each statement:
52.I believe capital punishment represents the most effective
deterrent to murder.
Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree
53.I believe a murderer can be rehabilitated to become a
resonsible, functioning member of society.
Strongly disagree1
2
3
45 6
7
8
9Strongly agree
54.I believe a life sentence is a satisfactory penalty for
murder.
Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree
8/8/2019 Jac Methods Ch14
16/27
14-162007 Pearson Education Canada
Tips for Likert-Style Questions
1. Avoid the word and Makes the questions multidimensional
2. Always place strongly agree on right side,
with 9 indicating strong agreement Response set, a situation in which the respondent
answers similarly to all answers, is best avoidedby wording some statements positively, othersnegatively
3. Avoid negatives that can confuserespondents Use direct negative statements
8/8/2019 Jac Methods Ch14
17/27
14-172007 Pearson Education Canada
Tips for Likert-Style Questions
(contd)4. Vary strength of wording to produce
variation in response
1. The nursing care I received was good.
2. The nursing care I received was perfect in
every instance
5. Before the first Likert-type item, provide a
brief explanation of how respondents are to
to indicate their answers
8/8/2019 Jac Methods Ch14
18/27
14-182007 Pearson Education Canada
Steps in Questionnaire Development
1. Make a list of variables. Usually includes:1. Background variables
2. Dependent variable(s)
3. Independent variables4. Others: intervening, antecedent, spurious
2. Anticipate how data will be analyzed Procedures depend on level of measurement
3. Write the proposed questions on index cards Facilitates editing and re-arranging order
4. Double check: do you have all thevariables?
8/8/2019 Jac Methods Ch14
19/27
14-192007 Pearson Education Canada
Steps (contd)
5. Review the conceptual definitions of
variables
Done in anticipation of step 6 - developing
the wording
Example: how to measure socioeconomic
status?
As reflecting various levels of respect and
prestige: use Pineo and Porters scale ofoccupational prestige to measure SES
As reflecting variation in access to scarce
resources: use total income to measure SES
8/8/2019 Jac Methods Ch14
20/27
14-202007 Pearson Education Canada
Steps (contd)
6. Develop wordings for questions
After considering the conceptual definition,also:
Use simple words: Grade 7 or8 level Avoid and: make sure only one question
Vary wording to produce variation in replies
Avoid complexity make it simple
Use existing wording if comparative study
Take the edge off sensitive questions
Be precise, highly specific when choosingwordings
8/8/2019 Jac Methods Ch14
21/27
14-212007 Pearson Education Canada
Steps (contd)
7. Pre-test the questionnaire Start by filling out the survey yourself, then ask
individuals to do so
Ask for feedback on the questionnaire Is it too long? Any problem with wording?
8. Pilot study Send questionnaire to a small sample of
respondents
Use data to determine which items will be used forindexes, modify unclear questions, create pre-coded, single-choice questions based on responsesto open-ended questions
8/8/2019 Jac Methods Ch14
22/27
14-222007 Pearson Education Canada
Ordering the Questions
Use a brief statement to introduce the surveyto the respondent re: legitimacy of study
Say who is conducting/sponsoring the study
Assure confidentiality and anonymity Ease them into it
Place easy to answer/salient questions at thebeginning
Place key and repeated questions at 1/3 point Group questions by type
E.g., put Likert-type questions together
8/8/2019 Jac Methods Ch14
23/27
14-232007 Pearson Education Canada
Formatting the Questions
Begin conditioning respondents
E.g., have consistent formatting so respondents can get
used to how to answer (format types shown in next two
slides)
Anticipate computer data entry
Pre-code questions before data collection
Vary placement of response categories
Response options are appealing and easy to understand
Clearly indicate branching (shown on slide 14-26)
Clear respondents away from irrelevant questions
Make it easy to fill out; will increase response rate
8/8/2019 Jac Methods Ch14
24/27
14-242007 Pearson Education Canada
FourFormat Styles Illustrated
1. Babbie Format Style for Fixed-Choice Responses:
23. Have you ever smoked marijuana?
[ ] Yes
[ ] No
2. Dillman Format Style for Fixed-Choice Responses:
Q-22 Your Sex (Circle number of your answer)
1 MALE
2 FEMALE
Sources:Earl Babbie (1992) The Practice of Social Research, 6th ed. Belmont, CA: Wadsworth Publishing
Company. p.155.
Don A. Dillman (1978).Mail andTelephone Surveys: the Total Design Method. New York: John Wileyand Sons. p.134.
8/8/2019 Jac Methods Ch14
25/27
14-252007 Pearson Education Canada
FourFormat Styles Illustrated
3. Krahn Format Style for Fixed-Choice Responses:
54. In the past year, has any member of your
immediate family (not counting yourself) been
unemployed (out of work and not looking for work)?
No 1
Yes 2
4. Jackson Format Style for Fixed-Choice Responses:
4. What year are you in?
Freshman ------------- 1
Sophomore ----------- 2
Junior------------------ 3
Senior----------------- 4Sources:
Harvey Krahn (1991). Sociological Methods of Research. In Lorne Tepperman and R. JackRichardson, eds., The Social World: An Introduction to Sociology. Toronto: McGraw-Hill Ryerson.p. 51.
Winston Jackson (1995).Methods: Doing Social Research. Toronto: Prentice Hall. p.388.
8/8/2019 Jac Methods Ch14
26/27
14-262007 Pearson Education Canada
Branching Question Illustrated
8/8/2019 Jac Methods Ch14
27/27
14-272007 Pearson Education Canada
Presentation of Questionnaire
Have a distinctive look
E.g., coloured paper, graphics for questions is
appealing
Can increase the appeal of doing thequestionnaire
Dont squeeze too much onto one page
Crowding questions on a page can make the
questionnaire appear too long