JAMM 444: Public Opinion

Survey Methodology& Question Wording

Feb. 12, 2008

Tuesday, Feb. 12

Quiz 2 Schedule update Research paper topic memo Democratic, Republican contests Survey methods & question wording End of class: Return news clipping

assignment

Quiz 2

1. What term is given to governors, senators, and other “party bigwigs” who have voting privileges at the Democratic national convention?

2. Which Republican candidate pulled out of the race after Super Tuesday?

3. What might explain why Barack Obama received more support in primaries in Alabama and Georgia than pre-election polls indicated?

Quiz 2

4. Why might a potential candidate for public office commission a poll before deciding whether to run? (Give one reason.)

5. In Travis Ridout’s opinion, which of the two remaining Democratic candidates has the best chance of winning the nomination? Why?

BONUS: What color tie did Travis wear during his guest lecture on Feb. 7?

Schedule update

TODAY: Chapter 3, Question Wording Turn in extra-credit papers on Idaho

caucus Thursday: Chapter 4: Sampling

Guest speaker: Chris Williams, Department of Statistics

Feb. 19: Interviewing & Data Collection Research topic memo due (new date)

Research Topic Memo

Due Tuesday, Feb. 19 See Web site:

http://www.class.uidaho.edu/jamm444/assignments.htm

Possible Research Topics

1. Practice (methodology)

2. Issue or controversy

3. Person (historical figure)

4. Presidential candidate

5. Primary or caucus (one state)

Elements of memo

1. Topic, why you have chosen it, and its importance to public opinion.

2. At least three research questions (things you hope to discover).

3. At least five sources:1. One must be from a scholarly journal,

such as Public Opinion Quarterly

2. No more than one Web site

Superdelegates

Delegates drawn from the Democratic National Committee, members of Congress, governors and distinguished party leaders, such as former presidents, vice presidents, and congressional leaders. Former House Speaker Tom Foley is a

superdelegate from Washington state.

Why superdelegates matter

Current tally of superdelegates (CNN) Clinton – 224 Obama – 135 Neutral/undecided/no public preference – 437*

TOTAL: 796* *This number may change over the next several

months as people die, leave office or leave the Democratic Party.

The race for the nomination

Democrats Clinton 1,148 Obama 1,121 Edwards 26 Needed for nomination: 2,025

The race for the nomination

Republicans McCain 783 Romney 286 Huckabee 217 Paul 16 Needed for nomination 1,191

Brokered convention

A situation in U.S. politics in which no candidate obtains a majority of delegates during the primary and caucus process. Because no candidate will receive enough votes on the first ballot to win the nomination, the convention is brokered through political horse-trading and/or multiple ballots.

Last brokered conventions: 1948: Republicans (Thomas Dewey) 1952: Democrats (Adlai Stevenson)

Survey methodology

Why do a survey? Measure opinion of a population too large

to contact individually Save time, save money

“Surveys can be used in a scientific way to realize the great benefits of interviewing a representative sample instead of the whole population.” Salant & Dillman, 1994

Surveys work, if...

Sample is large enough to yield desired precision.

Everyone in population has equal chance of being selected.

Questions enable respondents to give accurate answers.

Sampled respondents have similar characteristics to non-respondents.

5 key elements of a survey

1. Identify population

2. Choose type of sample

3. Choose method(s) for gathering data

4. Write (and test) questionnaire

5. Analyze the results

Choosing a sample

Identify population of interest: University of Idaho students Moscow residents Idaho voters ‘The American public’

Choosing a sample

Non-probability sampling 1936, Literary Digest

Probability sampling Simple random sampling (rarely used) Systematic random samples (i.e., every 30th

name in student directory) Stratified sample: divided into groups (strata) Cluster sampling (spread over a wide area)

Cluster sampling

Used for many national surveys: National Election Survey (NES) General Social Survey (GSS) Gallup Organization

350 geographical segments Sample to reflect U.S. as a whole

(geographic area, size of community) Random telephone numbers in each area

How big is a typical sample?

UI or Moscow: 200 to 500 Idaho: 500 to 800 National: 1,000 to 1,500

For Thursday

Read: P&P 4: Sampling Techniques Non-probability vs. probability samples Sample size Sampling error

Chapter 3: Question wording

1. How can a sponsor with an ax to grind manipulate wording of questions? What types of problems with question

wording are more subtle but just as harmful to the accuracy of responses?

Chapter 3: Question wording

2. Describe the potential problems with the following types of questions:

a) Compound (multi-topic) questions

b) Factual questions

c) Double-negative questions

d) Leading (argumentative) questions

e) Branching (follow-up questions)

f) Use of multiple items or indexes

Chapter 3: Question wording

3. The book gives several questions of how alternative wording of questions influenced responses. Discuss the examples presented in these cases:a) Impeachment of Bill Clinton

b) Use of government wiretaps to fight terrorism

c) The National Security Agency’s collection of telephone records of U.S. citizens

 

Impeachment of Bill Clinton

 Q: If the full House votes to send impeachment articles to the Senate for a trial, then do you think it would be better for the country if Bill Clinton resigned from office, or not?

Q: If the full House votes to impeach Bill Clinton, then do you think it would be better for the country if Bill Clinton resigned, or not?

Use of government wiretaps

 Q: After 9/11, President Bush authorized government wiretaps on some phone calls in the U.S. without getting court warrants, saying this was necessary to reduce the threat of terrorism. Do you approve or disapprove?

Q: After 9/11, President Bush authorized government wiretaps on some phone calls in the U.S. without getting court warrants. Do you approve or disapprove of George W. Bush doing this?

NSA collection of phone records

 Q: It’s been reported that the NSA has been collecting phone records of millions of Americans. It then analyzes calling patterns in an effort to identify possible terrorist suspects… Would you consider this an acceptable way for the federal government to investigate terrorism?

Q: Based on what you have heard about the program to collect phone records, would you say you approve or disapprove of this government program?

Chapter 3: Question wording

4. How can the order in which questions are asked affect responses? Give an example. Context effect: When the preceding question

influences a response Q: Would you say that traffic contributes more or

less to air pollution than industry? Q: Would you say that industry contributes more

or less to air pollution than traffic?

Tips for better question wording

1. Include filter questions to exclude some respondents based on interest or knowledge of issue

2. Use a neutral tone Unbiased phrasing of controversial topics Avoid words that lead respondent to

preferred answer (pseudo-surveys) Examples: Bill Sali constituent surveys

Question wording

3. Provide middle alternatives Offer a position between extremes Likert scale: 5 options, from ‘strongly

agree’ to ‘neutral’ to ‘strongly disagree’ 4. Be aware of context effect:

preceding questions affect responses hard knowledge vs. attitude general vs. specific

Question wording

5. Note the order of alternatives (especially in phone surveys) Primacy (first asked) Recency (last asked) Solution: rotate the alternatives.

Reading for Thursday

P&P 4: Sampling Techniques Non-probability vs. probability samples Sample size Sampling error

Pick up news clipping assignments