8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 1/35

Questionnaire & FormDesign

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 2/35

Questionnaire Design – lookseasy….but….. 

• We all think we are experts at asking questions

• Mistakes often do not lead to obvious errors

 – Interviewers and responders do their best

• Consequences of mistakes are not immediate

• Disparate interests involved

 – Client, interviewer, respondent, data entry persons

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 3/35

About 26 steps involved!

• Mistakes could occur in any one of these

 – Questionnaires are part of a lengthycommunication process

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 4/35

“Don’t shoot me, I’m just the statistical

analyst” 

• Extreme phrasing of question or combination ofquestions is the most frequent source ofdistortion in survey results

• So let’s see how we can design goodquestionnaires… 

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 5/35

Questionnaire Definition

• A questionnaire is a formalized set of questions forobtaining information from respondents.

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 6/35

Types of Questions

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 7/35

Choosing Question Structure Unstructured Questions

• Unstructured questions are open-ended questions thatrespondents answer in their own words.

How do you feel?

 __________________________________ 

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 8/35

Choosing Question Structure Structured Questions

• Structured questions specify the set of responsealternatives and the response format. A structuredquestion may be multiple-choice, dichotomous, or ascale.

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 9/35

Choosing Question Structure Dichotomous Questions

• A dichotomous question has only two responsealternatives: yes or no, agree or disagree, and so on.

• Often, the two alternatives of interest are supplemented

by a neutral alternative, such as “no opinion,” “don'tknow,” “both,” or “none.”

Do you intend to buy a new car within the next sixmonths?

 _____ Yes _____ No _____ Don't know

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 10/35

Layout Considerations

Make your questionnaire… 

• Short

• Simple

• Sweet

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 11/35

For interviewers

 –Make it short • 15 minutes by phone

• 30 to 40 minutes in-home

 – use show cards and shuffle packs

 – Make instructions explicit

• Put them in capital letters

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 12/35

For Respondents

• Make it simple

• Be respectful

 – Address concerns up front

 – “Thank you” at the end 

• Make it flow – general > interesting > difficult or tedious > interesting

> sensitive > classification

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 13/35

For data entry persons

• Make it sweet (looking) – Use pre-codes

 – Lay-out matters

• Questions on left, responses to right

• Include a space for survey/project number

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 14/35

Questionnaire

Design

Process

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 15/35

Questionnaire design issues

• Factual data

• Problem of understanding

• Problem of willingness to answer

• Problem of ability to answer

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 16/35

Are Several Questions Needed Insteadof One?

• Sometimes, several questions are needed to obtain therequired information in a clear manner. Consider thequestion,

“Do you think Coca-Cola is a tasty and refreshing soft drink?” (Incorrect)

• Such a question is called a double-barreled question,because two or more questions are combined into one.

To obtain the required information, two distinctquestions should be asked:

“Do you think Coca-Cola is a tasty soft drink?” and

“Do you think Coca-Cola is a refreshing soft drink?” (Correct)

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 17/35

Overcoming Inability To Answer Is the Respondent Informed?

• In situations where not all respondents are likely to beinformed about the topic of interest, filter questions thatmeasure familiarity and past experience should be asked

before questions about the topics themselves.

• A “don't know” option appears to reduce responses

without reducing the response rate.

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 18/35

Overcoming Unwillingness To Answer Effort Required of the Respondents

• Most respondents are unwilling to devote a lot of effort toprovide information.

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 19/35

Overcoming Unwillingness To Answer

Please list all the departments from which you purchased

merchandise on your most recent shopping trip to a department

store. (Incorrect)

In the list that follows, please check all the departments from

which you purchased merchandise on your most recent shopping

trip to a department store.

1. Women's dresses ____ 2. Men's apparel ____ 3. Children's apparel ____ 

4. Cosmetics ____ ...16. Jewelry ____ 17. Other (please specify) ____ (Correct)

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 20/35

Choosing Question Wording Define the Issue

• Define the issue in terms of who, what, when, where,why, how?

• Who, what, when, and where are particularly important.

Which brand of shampoo do you use? (Incorrect)

Which brand or brands of shampoo have you

personally used at home during the last month?In case of more than one brand, please

list all the brands that apply. (Correct)

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 21/35

Choosing Question Wording Use Unambiguous Words

In a typical month, how often do you shop in departmentstores?

 _____ Never _____ Occasionally

 _____ Sometimes _____ Often _____ Regularly (Incorrect)

In a typical month, how often do you shop in departmentstores?

 _____ Less than once _____ 1 or 2 times _____ 3 or 4 times

 _____ More than 4 times (Correct)

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 22/35

Choosing Question Wording Avoid Leading or Biasing Questions

• A leading question is one that clues the respondent to what theanswer should be, as in the following:

Do you think that patriotic Americans should buy importedautomobiles when that would put American labor out of work?

 _____ Yes _____ No _____ Don't know

(Incorrect)

Do you think that Americans should buy imported automobiles? _____ Yes _____ No _____ Don't know

(Correct)

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 23/35

Choosing Question Wording Avoid Implicit Assumptions

• Questions should not be worded so that the answer isdependent upon implicit assumptions about what willhappen as a consequence.

1. Are you in favor of a balanced budget?(Incorrect)

2. Are you in favor of a balanced budget if it wouldresult in an increase in the personal income tax?

(Correct)

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 24/35

Determining the Order ofQuestions

Opening Questions 

• The opening questions should be interesting, simple,and non-threatening.

Type of Information • As a general guideline, basic information should be

obtained first, followed by classification, and, finally,identification information.

Difficult Questions 

• Difficult questions or questions which are sensitive,embarrassing, complex, or dull, should be placed late inthe sequence.

D t i i th O d

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 25/35

Determining the Order

of Questions

Effect on Subsequent Questions • General questions should precede the

specific questions (funnel approach).

Q1: “What considerations are important toyou in selecting a department store?” 

Q2: “In selecting a department store, how

important is convenience of location?” 

(Correct)

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 26/35

Determining the Order ofQuestionsLogical Order  

The following guidelines should be followed for branchingquestions:

• The question being branched (the one to which therespondent is being directed) should be placed as closeas possible to the question causing the branching.

• The branching questions should be ordered so that therespondents cannot anticipate what additional informationwill be required.

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 27/35

Some thumb-rules for Questionnaire Design

• 1. Don’t crowd the questions

 – makes the questionnaire look difficult

• 2. Don’t split questions across pages. 

• 3. Put special instructions on the questionnaire• use a different typeface (bold, CAPITALS, or italics).

• put “Don’t read” above the question. 

• put “Skip” immediately after trigger answers. 

• 4. Precode all closed questions.

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 28/35

• 5. Use vertical formats for closed questions

 – NO! What is your favorite brand of cereal?

• A ___ B___ C____ Other_____ 

 – Yes! What is your favorite brand of cereal?

• 1……….BRAND A 

• 2……...BRAND B • 3……...BRAND C 

• 4……...OTHER 

Some thumb-rules for Questionnaire Design

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 29/35

• 6. Record the interview start and stop times.

• 7. Provide a beginning (a greeting ) and an endto the interview (A thank you ). – “Hello, my name is (your name), and I’m calling from

Harmon Research, a marketing research company. I needto speak with the person who is most responsible for healthcare decisions in your household. Is that you?” 

Some thumb-rules for Questionnaire Design

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 30/35

Less than 30 minutes....................-1

30 to 59 minutes............................-2

1 hour or more……………………..-3

The American LawyerA Confidential Survey of Our Subscribers

(Please ignore the numbers alongside the answers. They are only to help us

in data processing.)

1. Considering all the times you pick it up, about how much time, in total, do

you spend reading or looking through a typical issue of THE AMERICAN

LAWYER?

Example of a Precoded Questionnaire 

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 31/35

Pretesting

Pretesting refers to the testing of the questionnaire on asmall sample of respondents to identify and eliminatepotential problems.

• A questionnaire should not be used in the field surveywithout adequate pretesting.

• The pretest sample size varies from 15 to 30

respondents for each wave.

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 32/35

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 33/35

Pre-test, pre-test, pre-test!

• Discuss with client, data entry persons, and respondents

• The career you save may be your own!

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 34/35

Types of errors

Random sampling errors: • sample too small• sample not representative• inappropriate sampling method used

Research design errors: • bias introduced• measurement error• data analysis error• population definition error• scaling error• question construction error

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 35/35

Interviewer errors: • recording errors• cheating errors

• questioning errors• respondent selection errorRespondent errors: • non-response error

• inability error• falsification errorHypothesis errors: • type I error (also called alpha error)

 – the study results lead to the rejection of the null hypothesis even

though it is actually true• type II error (also called beta error)

 – the study results lead to the acceptance (non-rejection) of thenull hypothesis even though it is actually false