Session 11: Survey Research Methods
Transcript of Session 11: Survey Research Methods
![Page 1: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/1.jpg)
Copyright 2007, The Johns Hopkins University and Lynda Burton. All rights reserved. Use of these materials permitted only in accordance with license rights granted. Materials provided “AS IS”; no representations or warranties provided. User assumes all responsibility for use, and all liability related thereto, and must independently review all materials for accuracy and efficacy. May contain materials owned by others. User is responsible for obtaining permissions for use from third parties as needed.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this site.
![Page 2: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/2.jpg)
Survey Research: Choice of Instrument, Sample
Lynda Burton, ScDJohns Hopkins University
![Page 3: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/3.jpg)
Section A
Sample
![Page 4: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/4.jpg)
4
Survey Research Methods
What makes a survey a survey?−
Scientific methodology
−
Data collection from an individual−
Usually samples from a large population
−
Conducted for the purpose of . . .DescriptionExplorationExplanation
![Page 5: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/5.jpg)
5
Characteristics of Good Survey Research
QuantitativeSelf-monitoringContemporaryReplicable
SystematicImpartialRepresentativeTheory-based
![Page 6: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/6.jpg)
6
Types of Surveys
Cross sectionalLongitudinalTrendTime cohortPanel
![Page 7: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/7.jpg)
7
General Sampling Issues
Basic rule—all individuals must have equal chance of being selectedMay be more accurate data than a censusIf all members of a population were identical, sampling would not be necessaryAim for a sample that is generalizable to total population of interest
![Page 8: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/8.jpg)
8
Some Sampling Definitions
Element—unit from which data are collectedUniverse—aggregation of all elementsSurvey population—population from which you are drawing sampleSampling frame—actual list or “hat” from which you select elements
![Page 9: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/9.jpg)
9
Probability Sampling Approaches
Simple random sample−
Random number generator
−
Draw number out of a “hat”
![Page 10: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/10.jpg)
10
Probability Sampling Approaches
Systematic sample−
Every element of list
−
Select every “kth”
element
−
Make sure list isn’t “periodic”
![Page 11: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/11.jpg)
11
Probability Sampling Approaches
Stratified sample−
Same as for simple random sample
−
However, select from within specific pre-determined groupings
−
Insures heterogeneity
![Page 12: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/12.jpg)
12
Probability Sampling Approaches
Multi-stage sample−
Select cluster of elements first, for example, physicians
−
Then select elements, for example, patients
![Page 13: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/13.jpg)
13
Non-Probability Sampling Approaches
Purposive or judgmental−
Educated guess of representative unit
![Page 14: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/14.jpg)
14
Non-Probability Sampling Approaches
Purposive or judgmental−
Educated guess of representative unit
Quota sampling−
Select any way you want following a pre-set quota pattern
Available subject sampling−
Use whomever is available
![Page 15: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/15.jpg)
Section B: Sample Size Considerations
Based on chapter seven of Designing and Conducting Health Surveys
(2nd. ed.); Aday, LuAnn. San Francisco: Jossey-Bass
Publishers
![Page 16: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/16.jpg)
16
Key Concepts in Determining Sample Size
From a sampling point of view, the key concern is having a large enough number of cases to minimize the variable sampling (standard) error in the estimatesSampling error can be reduced by increasing the sample size (denominator) or minimizing the random errors in the data collection process
![Page 17: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/17.jpg)
17
Normal Sampling Distribution
![Page 18: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/18.jpg)
18
What Differences Do You Expect?
In experimental designs, there is a particular interest in determining the effect (or difference) between experimental and control groupsThe effect size, which essentially reflects the hypothesized difference between groups, provides a basis for calculating the sample size for these types of designs
![Page 19: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/19.jpg)
19
Type I and II Errors
A Type I error results from falsely rejecting the null hypothesis when the hypothesis is actually true (alpha)A Type II error refers to the reverse error—failing to reject the null hypothesis when it is actually falseThe probability of Type I and Type II errors decreases as the sample size increases, primarily because the estimates obtained from larger samples are more reliable (have less random sampling variation)
![Page 20: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/20.jpg)
20
Objective—to test a hypothesisFramework—power analysisSteps1.
Identify the major study hypotheses
2.
Determine the statistical tests for the study hypotheses, such as a t-test, F-test, or chi-square
Criteria for Estimating Sample Size Based on the Study Design: Aday
![Page 21: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/21.jpg)
21
Steps3.
Select the population or subgroups of interest (based on study hypotheses and design)
4.
Step 4a.
Indicate what you expect the hypothesized difference to be
b.
Estimate the standard deviation of the difference
Criteria for Estimating Sample Size Based on the Study Design: Aday
![Page 22: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/22.jpg)
22
Steps4.
Step 4
c.
Compute the effect size
5.
Decide on a tolerable level of error in rejecting the null hypothesis when it is true (alpha) (this is usually set at .05.)
Criteria for Estimating Sample Size Based on the Study Design: Aday
![Page 23: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/23.jpg)
23
Steps6.
Decide on a desired level of power for rejecting the null hypothesis when it is false (power) (this is usually set at .80.)
7.
Compute sample size, based on study assumption
Criteria for Estimating Sample Size Based on the Study Design: Aday
![Page 24: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/24.jpg)
24
Example of a Major Study Hypothesis
Proportion of patients (improving in health [by a global health index measure] and who received the treatment) will not differ from the proportion who improve and did not receive the treatment (null hypothesis)−
Ho P1 -
P2 = 0
Alternative hypothesis−
Ha P1 -
P2 ne 0
![Page 25: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/25.jpg)
25
Determine Statistical Test
Calculation of sample size depends on the study design and what statistical test you will be using to test the hypothesesIt is possible that you will have several outcomes, each of which will be determined using a different statistical testDifferent sample sizes will be required, depending on the outcome
![Page 26: Session 11: Survey Research Methods](https://reader031.fdocuments.us/reader031/viewer/2022020213/5866826d1a28ab3b408b5e56/html5/thumbnails/26.jpg)
26
Weighting the Sample Data to Reflect the Population
You may want to sample some sub groups in the population at different rates to ensure that there will be enough of these individuals without having to increase the overall size of the sampleThe sample can be weighted so that it resembles the population from which it was drawnWeighting literally involves a process of statistically assigning more, or less, weight to some groups than others so that their distribution in the sample corresponds more closely to their actual distributions in the population as a whole