How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian...
Transcript of How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian...
![Page 1: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/1.jpg)
How to Randomize?
East Asia Regional Impact Evaluation Workshop Seoul, South Korea
Christian Borja-Vega, World Bank
![Page 2: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/2.jpg)
Overview
1. Opportunities to Randomize 2. Define the target group 3. Select program features to evaluate 4. Level of randomization 5. Should you evaluate?
![Page 3: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/3.jpg)
Video: Opportunities to Randomize
![Page 4: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/4.jpg)
Basic setup of a randomized evaluation Randomization methods Individuals/Clusters Multiple/Cross-cutting treatments
Random Assignment
Treatment group
Control group
Balance Check
Endline Survey
Measure Impact
Internal Validity
Target Population
Not in evaluation
Evaluation Sample
Random Sampling
External Validity
Baseline Survey
![Page 5: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/5.jpg)
Randomly sample from area of interest
Random sampling and random assignment
![Page 6: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/6.jpg)
Randomly sample from area of interest
Randomly assign to treatment and control
Random sampling and random assignment
Randomly sample from both treatment and control
![Page 7: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/7.jpg)
Random sampling and random assignment
2006
Income per person, per month, shillings
1000
500
0 Treat Compare
1457 1442
![Page 8: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/8.jpg)
Opportunities to Randomize when Resources are Limited
![Page 9: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/9.jpg)
Lottery when Oversubscription More are eligible than can be served by the program
![Page 10: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/10.jpg)
Lottery when oversubscription: Youth Group Support Uganda
![Page 11: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/11.jpg)
Targeted Lottery Exclude some from the lottery
Include others
![Page 12: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/12.jpg)
Targeted Lottery
Randomize the rest of the
eligible population
![Page 13: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/13.jpg)
Randomly vary when to receive program
Time During Before
evaluation period
After
![Page 14: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/14.jpg)
Randomly vary when to receive the program: Preschools in Cambodia
![Page 15: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/15.jpg)
Randomly vary program strength
![Page 16: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/16.jpg)
Summary: opportunities to randomize
Consider 1. Targeting rules: Who is eligible for the program? Is
program participation restricted, widely open, some combination?
2. Resources: Sufficient program resources to reach all eligible beneficiaries?
3. Timing: How are potential beneficiaries enrolled in the program – all at once or in phases over time?
![Page 17: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/17.jpg)
Overview
1. Opportunities to Randomize 2. Define the target group 3. Select program features to evaluate 4. Level of randomization 5. Should you evaluate?
![Page 18: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/18.jpg)
Whom does this program hope to benefit?
Whole Population Target Population
Not Eligible
![Page 19: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/19.jpg)
Overview
1. Opportunities to Randomize. 2. Define the target group. 3. Select program features to evaluate 4. Level of randomization 5. Should you evaluate?
![Page 20: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/20.jpg)
Testing multiple program features Basic Setup Randomization methods Individuals/Clusters
Target Population
Not in evaluation
Evaluation Sample
Total Population
Random Assignment
Treatment Group 3
Control Group
Treatment Group 2
Treatment Group 1
![Page 21: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/21.jpg)
Cross-cutting program features Randomization methods Individuals/Clusters Multiple/Cross-cutting treatments
Target Population
Not in evaluation
Evaluation Sample
Random Assignment
T 1 C 1
T 2 T1*T2 T2
C 2 T1 C
![Page 22: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/22.jpg)
Overview
1. Opportunities to Randomize. 2. Define the target group. 3. Select program features to evaluate 4. Level of randomization 5. Should you evaluate?
![Page 23: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/23.jpg)
Which level of randomization?
INDIVIDUAL unit:
CLUSTER unit: e.g. pupil, mother, father, household
e.g. class, health clinic, village
![Page 24: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/24.jpg)
Which level of randomization? Individual or Cluster?
![Page 25: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/25.jpg)
Which level of randomization? Individual
![Page 26: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/26.jpg)
Which level of randomization? Cluster (group)
![Page 27: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/27.jpg)
Which level of randomization? For example, school
![Page 28: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/28.jpg)
Which level of randomization? Factors that matter
1. Unit of Implementation 2. Spillovers 3. Sampling unit and statistical power 4. Ethics and Fairness
![Page 29: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/29.jpg)
Which level of randomization?
Pupil
School
Community
Class
Unit of randomization should be at least at the level of the unit of program intervention: Example: IF program targets classes -> Randomization may be at class, school, community level
![Page 30: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/30.jpg)
Which Level of Randomization?
A. Unit of Implementation B. Spillovers: individual in the control also
benefits from the treatment
C. Sampling and Cost D. Ethics and Fairness
Individual tutoring
Spillover: program pupils talk with class mates
![Page 31: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/31.jpg)
With spillovers over/under-estimate impact
Control
True Impact Spillover
Measured Impact
Treatment
Impact
![Page 32: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/32.jpg)
Higher unit of randomization reduces risk of spillovers
Example: randomize at school level to reduce interaction between treatment and control pupils
School
Pupil
![Page 33: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/33.jpg)
Question: Is there risk of spillovers in the following programs?
• Example 1: Job training program
• Job seekers in the control are less likely to get jobs as a
result of the program, if the available jobs are given to those who received training.
• Measured program impact = jobs in treatment – jobs in comparison
• As a result, measured program impact is higher than the real program impact
![Page 34: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/34.jpg)
Question: Is there risk of spillovers in the following programs?
• Example 2: HIV-AIDS education program
• People who attend education program will tell their friends
• Measured program impact = HIV-aids awareness in
treatment – HIV-aids awareness in comparison
• As a result, measured impact is lower than the real impact
![Page 35: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/35.jpg)
Which Level of Randomization?
A. Unit of Implementation B. Spillovers C. Sampling unit and statistical power D. Ethics and Fairness
![Page 36: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/36.jpg)
Higher unit of randomization requires larger sample
Same statistical power:
School randomization: • 800 in treatment and
800 in control ….in • 40 treatment and 40
control schools, 20 pupils per school
Pupil randomization:
• 393 in treatment and 393 in control
Assume intra-cluster correlation of 0.05
![Page 37: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/37.jpg)
Need to increase number of clusters, not number of individuals per cluster
For example, the following two studies have the same power: • 80 clusters, 20 individuals per cluster • 40 clusters, 1067 individuals per cluster That’s 1,600 individuals compared to 42,680! Assume intra-cluster correlation of 0.05
![Page 38: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/38.jpg)
Which Level of Randomization?
A. Unit of Implementation B. Spillovers C. Sampling unit and statistical power D. Ethics and Fairness
![Page 39: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/39.jpg)
Sometimes perception of unfairness among control if randomizing peers, community
members etc. at individual level
![Page 40: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/40.jpg)
Less so if all in same group or village are in the control
![Page 41: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/41.jpg)
Which Level of Randomization?
A. Unit of Implementation B. Spillovers C. Sampling unit and statistical power D. Ethics and Fairness
Randomize at the smallest feasible level
![Page 42: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/42.jpg)
Some Examples
![Page 43: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/43.jpg)
At what level should we randomize for a school deworming program?
A. School B. Teacher C. Classroom D. Clinic E. Community F. Pupil
![Page 44: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/44.jpg)
At what level should we randomize for a conditional cash transfer program?
A. Individual B. Household C. Village/Community
![Page 45: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/45.jpg)
At what level should we randomize for a teacher training program?
A. School B. Teacher C. Classroom D. Pupil
![Page 46: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/46.jpg)
At what level should we randomize for a small-business start-up grant?
A. Individual B. Household C. Clinic D. School E. Village/Community
![Page 47: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/47.jpg)
Overview
1. Opportunities to Randomize. 2. Define the target group. 3. Select program features to evaluate 4. Level of randomization 5. Should you evaluate?
![Page 48: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/48.jpg)
Does this need an Impact Evaluation?
Maximize learning
Consider 1. Do we know the answer already? 2. Is it policy relevant?
a) Is this a program that government could scale up if it is successful?
b) Is it strategically relevant for reducing poverty? 3. Does it add to knowledge? 4. Is the program ready for an impact evaluation? 5. Are there program design options that needs to be
compared? 6. Is there an resource constraint that provides an
opportunity for randomization?
![Page 49: How to Randomize?pubdocs.worldbank.org/pubdocs/publicdoc/2016/5/...Seoul, South Korea . Christian Borja-Vega, World Bank . Overview . 1. Opportunities to Randomize . 2. Define the](https://reader035.fdocuments.us/reader035/viewer/2022070805/5f0397ef7e708231d409d1b7/html5/thumbnails/49.jpg)
Now is your chance to integrate an impact evaluation in your own program