11© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Chapter 9Chapter 9

Surveys, Questionnaires, and PollsSurveys, Questionnaires, and Polls

Most commonly used quantitative methodMost commonly used quantitative method

– Used for obtaining information about what Used for obtaining information about what people do, and respondents’ attitudes or people do, and respondents’ attitudes or characteristicscharacteristics

– In experimental, quasi-experimental, and In experimental, quasi-experimental, and descriptive research designsdescriptive research designs

22© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

What is a Survey?What is a Survey?

System for collecting comparable System for collecting comparable information across many peopleinformation across many people

– Paper-and-pencil self-administered or self-Paper-and-pencil self-administered or self-reportsreports

– Face-to-face Face-to-face

– Telephone Telephone

– Mail Mail

– Computer-assistedComputer-assisted

33© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Designing a SurveyDesigning a Survey

Develop the research designDevelop the research design

Evaluate existing questionnaires/surveysEvaluate existing questionnaires/surveys

– Recommended over creating your ownRecommended over creating your own

– Has undergone extensive testing and revisionHas undergone extensive testing and revision

– Minor changes are okay; substantial changes Minor changes are okay; substantial changes will require that you pretest or pilot test the will require that you pretest or pilot test the questionnairequestionnaire

44© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Writing Your Own QuestionnaireWriting Your Own Questionnaire

Start with literature reviewStart with literature review

Designing survey itemsDesigning survey items– StraightforwardStraightforward– One complete thought written in sentence or One complete thought written in sentence or

question formatquestion format– Respondent should know how to answerRespondent should know how to answer– Avoid abbreviations and slang expressionsAvoid abbreviations and slang expressions– Shorter items are better than long onesShorter items are better than long ones

55© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Open QuestionsOpen Questions

Respondents use their own words to respond Respondents use their own words to respond – Makes data less comparable and more difficult to Makes data less comparable and more difficult to

interpretinterpret

Consider what constitutes an adequate answerConsider what constitutes an adequate answer– Build that request into the questionBuild that request into the question

Use a recall cue to draw participants’ attention to Use a recall cue to draw participants’ attention to issue, topic, or timeframeissue, topic, or timeframeRecord everything participant saysRecord everything participant saysCode responses after all data are collectedCode responses after all data are collected

66© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Closed QuestionsClosed Questions

Respondents given a question or Respondents given a question or statement statement and and given a set of responses to given a set of responses to select fromselect from– All responses must be known in advanceAll responses must be known in advance

– Creates easily comparable responsesCreates easily comparable responses

Use a recall cue or stimulus statement to Use a recall cue or stimulus statement to draw participants’ attention to issue, topic, draw participants’ attention to issue, topic, or timeframeor timeframe

77© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Response Sets Response Sets for Closed Questionsfor Closed Questions

Nominal or categorical responsesNominal or categorical responses– ExhaustiveExhaustive– Mutually exclusiveMutually exclusive– EquivalentEquivalent

Likert-type scalesLikert-type scales– 5- or 7-point scale 5- or 7-point scale – Includes middle or neutral responseIncludes middle or neutral response

Semantic differential scalesSemantic differential scales– Bipolar adjectives anchor 7-point scaleBipolar adjectives anchor 7-point scale

88© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Examples of Response Sets Examples of Response Sets for Likert-Type Scalesfor Likert-Type Scales

Very oftenVery often Fairly oftenFairly often OccasionallyOccasionally RarelyRarely NeverNever

Very Very

positivepositiveGenerally Generally positivepositive MixedMixed Generally Generally

negativenegativeVery Very

negativenegative

Completely Completely agreeagree

Generally Generally agreeagree UnsureUnsure Generally Generally

disagreedisagreeCompletely Completely

disagreedisagree

Strongly Strongly agreeagree AgreeAgree UnsureUnsure DisagreeDisagree Strongly Strongly

disagreedisagree

99© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Pretesting the Pretesting the Survey or QuestionnaireSurvey or Questionnaire

Also called pilot testingAlso called pilot testing

Should be done if you developed a Should be done if you developed a questionnaire or modified an existing onequestionnaire or modified an existing one

Four approaches Four approaches – Cognitive approachCognitive approach

– Conventional Conventional

– Behavior codingBehavior coding

– Expert panelExpert panel

1010© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Sampling Issues for SurveysSampling Issues for Surveys

Sample size and response rate are not the sameSample size and response rate are not the same– Response rate or return rate = number of people who Response rate or return rate = number of people who

provide usable responses after being asked to provide usable responses after being asked to participateparticipate

Acceptable response rates vary by survey Acceptable response rates vary by survey techniquetechnique– May be differences in those who choose to respond May be differences in those who choose to respond

and those that don’t respondand those that don’t respond

– Unusable responsesUnusable responses

1111© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Survey Reliability and ValiditySurvey Reliability and Validity

Internal reliability must be computed for Internal reliability must be computed for multiple item questionnairesmultiple item questionnaires– Cronbach’s alphaCronbach’s alpha

Varies from 0 to 1.00Varies from 0 to 1.00Generally .70 considered acceptableGenerally .70 considered acceptable

Three types of validity should be Three types of validity should be consideredconsidered– Content validityContent validity– Face validityFace validity– Construct validityConstruct validity

1212© 2006 by The McGraw-Hill Companies, Inc. All rights

reserved.

Analyzing Survey DataAnalyzing Survey Data

Data of all participants combined to create Data of all participants combined to create a picture of the wholea picture of the whole

Often limited to descriptive purposesOften limited to descriptive purposes

Data can test existing modelsData can test existing models

Data from closed questions Data from closed questions – Descriptive statisticsDescriptive statistics

Data from open questionsData from open questions– Categorized or content analyzedCategorized or content analyzed