Sampling Decisions in Your TC Program Evaluation TCEC.
-
Upload
tracy-basil-alexander -
Category
Documents
-
view
215 -
download
0
Transcript of Sampling Decisions in Your TC Program Evaluation TCEC.
![Page 1: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/1.jpg)
Sampling Decisionsin Your TC Program Evaluation
TCEC
![Page 2: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/2.jpg)
Overview Why sampling? Terms Sampling methods Sample size Sample size and analysis Answers to program questions Resources Announcements
![Page 3: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/3.jpg)
Why sampling?
Evidence from data drive the program
Census (collecting data from each entity such as stores, tenants, population) may not be possible
A representative sample can do the trick
![Page 4: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/4.jpg)
Terms
Population = Entire group from which to collect data (e.g. county population; stores in a certain area, all city parks, etc.)
Census = all members of the population
Sample = a portion of the population
![Page 5: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/5.jpg)
More terms
Sampling unit = the objects, people, timeslots, etc. that are being sampled (stores, residents, casino patrons, etc)
Sampling frame = How you derive your sample (phone book, street corner traffic…)
![Page 6: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/6.jpg)
![Page 7: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/7.jpg)
Sampling Methods
Simple Random Stratified Random Cluster Convenience Purposive
![Page 8: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/8.jpg)
Simple Random
Each member of the population has the same chance of being in the sample
![Page 9: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/9.jpg)
Stratified Random
A sample of equal size is drawn fromdifferent sub groups of the population
Use example:TRL: different neighborhoods that have different foreseeable consequences if TRL has passed – you want equal representation of all groups
![Page 10: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/10.jpg)
Cluster
B
C
A
A = ruralB = urbanC = college town
Census ofAll members ofthe population in a group, e.g. all tobacco retailers
![Page 11: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/11.jpg)
Purposive
Who or what you sample depends on the purpose of the use for the data
Examples KIIs generally POP: only those who smoke at the fair Observation: only parks with tot lots
![Page 12: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/12.jpg)
Convenience Reason: limited time and capacity (give
reason)
Examples: Intercept public opinion surveys YTPS only when youth are available to do
stings Only housing residents who come to a
housing association meeting
![Page 13: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/13.jpg)
Sample Size Sample size = The number of
population members you will use to collect data
Confidence level = The level of
certainty (usually set at 95%) that your sample represents the whole population
Confidence interval = The percentage of error you expect in your results
![Page 14: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/14.jpg)
Population Confidence Confidence Sample
size level interval size 150 95% 10 59 150 95% 6 96 150 95% 4 120
![Page 15: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/15.jpg)
Question
What is the minimum POS sample size to make statistically valid decisions for a population of 1.8 million?
Go to http://surveysystem.com Research Aids Sample Size Calculator
![Page 16: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/16.jpg)
Sample size calculator
Determine Sample Size Confidence Level: _x_ 95% __99%Confidence Interval: 5 (2)Population: 1,800,000
Sample size needed: 384 (2398)
![Page 17: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/17.jpg)
What it means for analysisExampleSurvey Question:
Please say if you agree or disagree with the following statement:
“Smoking can shorten a person’s life.”__ Agree __ Disagree
Let’s say 75% of those asked said “agree.”
Analysis at confidence interval 5: We can say with 95% certainty that between 70 and 80 % (75 plus/minus 5) of the population in the county agree that smoking can shorten a person’s life.
Analysis at confidence interval 2: We can say with 95% certainty that between 73 and 77 % (75 plus/minus 2) of the population in the county agree that smoking can shorten a person’s life.
![Page 18: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/18.jpg)
How to reportIn evaluation plan
A public opinion survey will be conducted with a sample size of 384 county residents randomly selected from the phone book (or “through a convenience sample”), for a 95% confidence level and a + 5 % confidence interval for a total of 1.8 million county residents.
(Note 1: you have to start out with a bigger sample size since a large percentage will decline to participate)
(Note 2: if you analyze sub-groups of your survey, let’s say an ethnic groups’ collective responses, your sample size is much smaller)
![Page 19: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/19.jpg)
Determining statistical power afterwards
Resource:
http://www.greatbrook.com/survey_statistical_confidence.htm
![Page 20: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/20.jpg)
Requirements
Q: Do I need to do sample size calculations?
Reviewers like to see it but will not require it. An estimated sample size is acceptable if it is at an “acceptable” rate, with acceptable referring to reasonableness with regards to available resources (in this case “200” will usually satisfy reviewers).
![Page 21: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/21.jpg)
Other sample sizes (ss)
KII policy makers: ............(5-6) POS w/MUH residents: ……use calculator/ estimate ss Observation people at event:……………………………estimate total event
participants/ calculate or estimate ss
Tobacco litter …………………See “Sampling Plan” in Tips and Tools # 8,
Observation
![Page 22: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/22.jpg)
Representativeness
Sampling method determines representativeness
Less likely inConvenience sample (e.g. intercept survey)
More likely inRandom samples
![Page 23: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/23.jpg)
Question
How do you deal with small sample sizes in a survey?
If the survey is not yet completed – add to the sample
If survey is completed – report results, add limitations explanation; acceptance of results is still quite possible
![Page 24: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/24.jpg)
Planning versus Analysis
What to focus on duringPlanning
“population” size, sample size, sampling method,
AnalysisRepresentativeness, limitations
![Page 25: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/25.jpg)
Resources
TCEC website: Tips and Tools # 8: Observation Tips and Tools # 9: Sampling
Decisions OTIS project plans and reportsIndividual assistance: Call TCEC at 530-752-9951 E-mail at [email protected]
![Page 26: Sampling Decisions in Your TC Program Evaluation TCEC.](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649d005503460f949d3107/html5/thumbnails/26.jpg)
Announcements
New Evaluation Associate TCEC will be on facebook A webinar on Cost-effective TC
Evaluation this Thursday at 10 (same phone number and website)