Survey Design: Designing Questionnaires Ronda L. Cochran, MPH Office of Prevention Research and...
-
Upload
nyla-morriss -
Category
Documents
-
view
215 -
download
1
Transcript of Survey Design: Designing Questionnaires Ronda L. Cochran, MPH Office of Prevention Research and...
Survey Design: Designing
Questionnaires
Ronda L. Cochran, MPHOffice of Prevention Research and
EvaluationDivision of Healthcare Quality Promotion
Centers for Disease Control and Prevention
Why Survey? - To know or not to know… that is
the question To find out the characteristics,
behavior, or opinions of a particular population (i.e., healthcare personnel in long term care centers)
To measure the degree to which an individual exhibits the characteristic(s) of interest (i.e., use of alcohol-based handrub)
A Simple Example Do you eat candy?
How often do you eat candy?
□Yes □No
□Never…□Rarely…□Sometimes…□Often…□All the time
Survey Research Most frequent mode of
observation/measurement Exploratory, descriptive, and
explanatory Unit of analysis usually individuals
(respondents) Used most often for:
describing a population too large to observe (practices/behaviors) directly
measuring attitudes Common terms: Survey, Poll,
Evaluation, Assessment, Questionnaire…
Modes of Survey Administration
Face-to-Face
Computer-assisted
Telephone
Interactive Voice Response
Surveys
Internet
Types of Survey Research
Self Administered Questionnaires: Respondents complete the
questionnaire themselves (e.g., mail survey)
Interview: researchers ask the questions and
record respondents answers (e.g., face-to-face interview and telephone interviews)
Self-Administered Questionnaire
Mail Survey: Cover letter of explanation (why
is this important to complete???) Self-addressed stamped return
envelope
But… Why not return?
Self-Administered Questionnaire
Response Rate: how many respondedhow many mailed
Representativeness of the sample
Higher response rate, less response bias
Depends on target audience How to increase???
Follow-up mailings Non respondents All respondents
Interview Surveys
Usually have higher response rate than mail surveys
Less “do not knows” or blanks Less items misunderstood Additional observations possible
(e.g., ability to speak/understand English, general reactions)
Interview Surveys
Guidelines for Interviewing Appearance Attitude/Demeanor Familiar with survey Follow wording exactly Record responses exactly Probe (request elaboration)
appropriately Trained and supervised
Survey Research and Technology
CATI - computer assisted telephone
interviewing CAPI - computer assisted personal
interviewing
CASI - computer assisted self-interviewing
CSAQ - computerized self-administered questionnaire
TDE - touchtone data entry
VR - voice recognition
Define the purpose
Write rules (e.g., for administration)
Determine survey format
Write itemsIdentify
respondents
Write objectives
General Steps in Survey Development
Constructing a Survey First & foremost, decide what
information you need and why you need it Define the purpose of the survey –
be specific (i.e., assess extent to which occupational exposure management policies and practices are available and consistent
with CDC guidelines)
Think in terms of results – prepare hypothetical tables – what you put in… is what you get out…
Constructing a Survey: Determine Data to be
Collected Demographics (e.g., age, gender, years in practice)Consider the respondents:
Who are they?What are their expected levels of
language and reading skills?How familiar/aware are they with
the topic ?How will they be recruited?How much time will they have to
complete the survey?
Constructing a Survey: Determine Data to be
Collected Variables important to testing
hypothesis/research questions (e.g., independent variables, outcomes, time in study [if applicable]) – date of admission/discharge)
Research questions/objectives should guide the selection of items
Write theoretical definition
Write items
Identify formatIdentify
sources of items
Determinenumber of items
Createblueprint
Identify conceptdimensions
Steps in Item Writing
Item Sources….Generating
Items Experts
Literature (review – can often find templates for item phrasing)
Interviews (information from members of the targeted population)
Other Instruments (designed to measure the concept/area of study)
Types of Questions Open-ended: respondent
provides his or her own answer to the question What is the most common problem
associated with survey construction? Close-ended: respondent is asked
to select an answer from among a list provided (response choices) The most common problem
associated with survey construction is:A. Inadequate instructionsB. Use of vague or unfamiliar termsC. Overall format of the instrument D. All of the above
Responses Response choices/categories
should be EXHAUSTIVE Other (Please
specify):________________ Response choices/categories
should be MUTUALLY EXCLUSIVE How many times have you visited the
doctor over the past 6 months?A. 0-1 A. 0-1B. 1-2 vs. B. 2-3C. 2 or more C. 4 or more
Question Development Writing Good Questions
Clear and unambiguous Spell out acronyms and define unusual
terms Use simple words (reading level) Give specific time frame/period (for recall) –
i.e., “past 12 months”
Relevant to the respondent (Do not know, no opinion, undecided, not applicable… however, use selectively)
Short items are best
Question Development Writing Bad Questions
Avoid double-barreled questions (questions with multiple parts)
Patient safety applies to patients and healthcare personnel.
Patient safety initiatives should be funded on a national level without any tort reform.
Avoid negative items (negation leads to misinterpretation)
Patient safety should not be a national priority.
Avoid biased items and terms (noting social desirability of questions and answers)
Questionnaire Construction
Format - spread out, uncluttered, not squeezed on to one page, response choices should be clear
Contingency questions: survey question intended for only some respondents, determined by their responses to some other question (skip patterns) – i.e., “Does your facility provide Hepatitis B vaccine to healthcare personnel? If NO, Go to #10” (skipping questions on Hepatitis B vaccine administration practices)
Questionnaire Construction
Order and group items and answers to use space efficiently and effectively Be careful of -- Response sets:
respondents develop a pattern of responses (e.g., agreeing with all statements) -- Alternate statements/questions
INSTRUCTIONS!!! (and transitions too)
Pretesting the questionnaire (pilot): writing rules for administration and scoring
Survey Data Entry and Analysis
Clear measurement rules for administration and scoring:Standardized - process applied the same way each time (e.g., duplicate data entry/verification)Explicit - all users able to follow the same process Unambiguous - clear rules that cover all possibilities (e.g., missing data)Subject to evaluation - modify based on experience
Measurement Rules
For example… Survey Construction – one must consider: How a correct response is scored? How incorrect response is scored? What to do about skips? What to do with >1 answer to an
item?
Cognitive design principles & formatting
Simple design – reduce noise, include only the information you need
Use alternatives to grid lines – dot leaders and response boxes De-emphasize irrelevant info – use footers for copyright and
other related info Use response boxes Emphasize selected terms – use judiciously and consider
underlining Avoid repeated instructions Use a consistent design throughout the questionnaire Order scale options consistently – use negative and positive
item wording to control acquiescence Place instructions in the context of the questions Group concepts Create a natural reading flow (left to right) Positioning question screens and skip instructions – place to
the right at the appropriate location, consider arrows Use left justification Use graphical guides to enhance tracking (upside down
Triangle) Use spacing to help respondent distinguish between sections Enhance interest with a title page – may include graphics
What you put in…is what you get out…
Be specific about what new information you need and why
Choose the survey method that works best for you and the situation
Determine sampling method(s) Write good questions that will provide useful,
accurate information Design and test an instrument that is easy
and interesting to answer Put together the necessary mix of people,
equipment, and supplies in the necessary time frame (logistics)
Analyze the data Share and present your results
Find the problem…
Do you exercise regularly?□ Yes □ NoVague wording (regularly); what is considered exercise?
Find the problem…
How many times have you been admitted to a hospital?_____Reference period not included
Find the problem…
Do you exercise and eat a low-fat diet? □ Yes □ NoDouble-barreled; what is low-fat?
Find the problem…
What percentage of your weekly grocery bill is spent on dairy products?_____ PERCENTRequires a calculation, what counts as dairy? Consider listing items and asking for a total.
‘The ability to ask the right question is more than half the battle of finding the answer.’ -Thomas J. Watson (1874– 1956, president of IBM)
*The finding and conclusions in this presentation are those of the author and do not necessarilyrepresent the official position of the Centers for Disease Control and Prevention.