SURVEY RESEARCH

Topics Appropriate to Survey Research

• Descriptive

• Exploratory

• Explanatory

Guidelines for Asking Questions

• Choose appropriate question forms.

• Make items clear.

• Avoid double-barreled questions.

• Respondents must be competent to answer.

Guidelines for Asking Questions

• Respondents must be willing to answer.

• Questions should be relevant.

• Short items are best.

• Avoid negative items.

• Avoid biased items and terms.

Guidelines for Questionnaire Construction

• One question per line.

• Use contingency questions when necessary.

• Format matrix questions so they are easily answered.

Guidelines for Questionnaire Construction

• Be aware of issues with ordering items.

• Include instructions for the questionnaire.

• Pretest all or part of the questionnaire.

Question/Response Wording

• Likert Scales

• Thermometer ratings

• Agree/Disagree statements.

• Ordered responses.

• Open responses.

• Check all that apply.

Acceptable Response Rates

• 50% - adequate for analysis and reporting

• 60% - good

• 70% - very good

Guidelines for Survey Interviewing

• Dress in a similar manner to the people who will be interviewed.

• Study and become familiar with the questionnaire.

• Follow question wording exactly.• Record responses exactly.• Probe for responses when necessary.

Telephone Surveys

Advantages:

• Money and time.

• Control over data collection.

Disadvantages:

• Surveys that are really ad campaigns.

• Answering machines.

Strengths of Survey Research

• Useful in describing the characteristics of a large population.

• Make large samples feasible.

• Flexible - many questions can be asked on a given topic.

Weaknesses of Survey Research

• Can seldom deal with the context of social life.

• Inflexible in some ways.

• Subject to artificiality.

• Weak on validity.

Survey Problems

• Reader’s Digest presidential survey of 1936; Alf Landon vs. FDR

• Hite Sexuality Survey – 70% of women married 5 years or more are having sex outside of marriage (4,500 completed surveys out of 100,000).

• Recorded phone surveys.• Push & Entertainment polling: political

telemarketing masquerading as a poll

Bush campaign in South Carolina asked the following:

"John McCain calls the campaign finance system corrupt, but as chairman of the Senate Commerce Committee, he raises money and travels on the private jets of corporations with legislative proposals before his committee. In view of this, are you much more likely to vote for him, somewhat more likely to vote for him, somewhat more likely to vote against him or much more likely to vote against him?"

Independent Sample T-test Formula

21

21

21

222

211

221 NN

NN

NN

sNsNs xx

t =

21

21

XXs

XX

Independent-samples t-tests

• But what if you have more than two groups?

• One suggestion: pairwise comparisons (t-tests)

Multiple independent-samples t-tests

• That’s a lot of tests!

# groups # tests2 groups = 1 t-test3 groups = 3 t-tests4 groups = 6 t-tests5 groups = 10 t-tests...10 groups = 45 t-tests

Inflation of familywise error rate

• Familywise error rate – the probability of making at least one Type I error (rejecting the Null Hypothesis when the null is true)

• Every hypothesis test has a probability of making a Type I error ().

• For example, if two t-tests are each conducted using = .05, there is a .0975 probability of committing at least one Type I error.

• The formula for familywise error rate:

# groups # tests nominal alpha familywise alpha

2 groups 1 t-test .05

3 groups 3 t-tests .05

4 groups 6 t-tests .05

5 groups 10 t-tests .05

...

10 groups 45 t-tests .05

Inflation of familywise error rate

1 1c

11 1 1 .95 .05

c

31 1 1 .95 .14

c

61 1 1 .95 .26

c

101 1 1 .95 .40

c

451 1 1 .95 .90

c

Analysis of Variance: Purpose

• Are there differences in the central tendency (mean) of groups?

• Inferential: Could the observed differences be due to chance?

Assumptions of ANOVA

• Normality – scores should be normally distributed within each group.

• Homogeneity of variance – scores should have the same variance within each group.

• Independence of observations – observations are randomly selected.

Logic of Analysis of Variance

• Null hypothesis (Ho): Population means from different conditions are equal– m1 = m2 = m3 = m4

• Alternative hypothesis: H1

– Not all population means equal.

Logic of Analysis of Variance

• Null hypothesis (Ho): Population means from different conditions are equal– m1 = m2 = m3 = m4

• Alternative hypothesis: H1

– Not all population means equal.

Lets visualize total amount of variance in an experiment

Between Group Differences(Mean Square Group)

Error Variance (Individual Differences + Random Variance) Mean Square Error

Total Variance = Mean Square Total

F ratio is a proportion of the MS group/MS Error.The larger the group differences, the bigger the FThe larger the error variance, the smaller the F

Logic--cont.

• Create a measure of variability among group means– MSgroup

• Create a measure of variability within groups– MSerror

Loves Statistics Hates Statistics Indifferent

9 4 8

7 7 4

6 5 5

11 6 6

12 3 7

Example: Test Scores and Attitudes on Statistics

• Find the sum of squares between groups

• Find the sum of squares within groups

• Total sum of squares = sum of between group and within group sums of squares.

22totaltotalgroupgroupbetween XNXNSS

22groupgrouptotalwithin XNXSS

22totaltotaltotaltotal XNXSS

• To find the mean squares: divide each sum of squares by the degrees of freedom (2 different dfs)

• Degrees of freedom between groups =

• k-1, where k = # of groups

• Degrees of freedom within groups = n-k

• MSbetween= SSbetween/dfbetween

• MSwithin= SSwithin/dfwithin

• F = MSbetween / MSwithin

• Compare your F with the F in Table D