8/11/2019 Sarah Baird CCT vs UCT Education
1/30
Relative Effectiveness of Conditional vsUnconditional Cash Transfers forSchooling in Developing Countries: asystematic review
Sarah Baird (George Washington University)Francisco Ferreira (World Bank)Berk zler (University of Otago/World Bank)Michael Woolcock (World Bank)
8/11/2019 Sarah Baird CCT vs UCT Education
2/30
2
Outline Background & objectives
Search strategy & selection criteria
Data collection & analysis
Main Results
Authors Conclusions
Acknowledgements & Funding
8/11/2019 Sarah Baird CCT vs UCT Education
3/30
BACKGROUND ANDOBJECTIVES
3
8/11/2019 Sarah Baird CCT vs UCT Education
4/30
4
Background
Increasing educational attainment around the world is one of thekey aims of the Millennium Development Goals There are many social protection programs in developingcountries that aim to improve education
Conditional Cash Transfers (CCTs) are targeted to the poorand made conditional on certain behaviors of recipienthouseholds.
As of 2007, 29 countries around the world had some type of a ConditionalCash Transfer program (CCT) in place, with many others planning orpiloting one (World Bank, 2009)
Unconditional Cash Transfer programs (UCT) are also commonand have also been shown to change behaviors on which CCTsare typically conditioned.
8/11/2019 Sarah Baird CCT vs UCT Education
5/30
8/11/2019 Sarah Baird CCT vs UCT Education
6/30
Background
The debate over whether conditions should be tied to cashtransfers has been at the forefront of recent global policydiscussions.
The main argument for UCTs is that the key constraint forpoor people is simply lack of money (e.g. because of creditconstraints), and thus they are best equipped to decide whatto do with the cash (Hanlon, Barrientos and Hulme 2010). Three main arguments for CCTs: market failure that causessuboptimal levels of education; investments in educationbelow socially optimal level; political economy .
6
8/11/2019 Sarah Baird CCT vs UCT Education
7/30
Objectives
This systematic review aims to complement the existing evidenceon the effectiveness of these programs and help inform thedebate surrounding the design of cash transfer programs.Our main objective was to assess the relative effectiveness ofconditional and unconditional cash transfers in improvingenrollment/dropout, attendance and test scores in developingcountries.Our secondary objective was to understand the role of differentdimensions of the cash transfer programs including:
Role of the intensity of conditions Transfer sizeBaseline enrollment
7
8/11/2019 Sarah Baird CCT vs UCT Education
8/30
SEARCH STRATEGY ANDSELECTION CRITERIA
8
8/11/2019 Sarah Baird CCT vs UCT Education
9/30
Search Strategy
Five main strategies were used to identify relevant reports(1) Electronic searches of 37 international databases (concludedon April, 18 2012)
(2) contacted researchers working in the area(3) hand searched key journals(4) reviewed websites of relevant organizations(5) given the year delay between the original search and the final
edits of the review we updated our references with all neweligible references the study team was aware of as of April 30,2013.
9
8/11/2019 Sarah Baird CCT vs UCT Education
10/30
Eligible Reports
Report had to either assess the impact of a conditional cashtransfer program (CCT), with at least one condition explicitlyrelated to schooling, or evaluate an unconditional cash transferprogram (UCT). The report had to include at least one quantifiable measure ofenrollment, attendance or test scores. The report had to be published after 1997 The report utilize a randomized control trial or a quasi-experimental design. The report had to take place in a developing country.
10
8/11/2019 Sarah Baird CCT vs UCT Education
11/30
11
8/11/2019 Sarah Baird CCT vs UCT Education
12/30
DATA COLLECTION AND ANALYSIS
12
8/11/2019 Sarah Baird CCT vs UCT Education
13/30
Calculating Effect Sizes
Measures of treatment effects come from three different types ofstudies: CCT vs. control, UCT vs. control, and, for fourexperimental studies, CCT vs. UCT. For these latter set ofstudies, a separate effect size for CCT and UCT (each compared with the control group of no intervention) is constructed. We construct odds ratios for effect size measures of enrollmentand attendance, and report test score results in standarddeviations.
Economists typically do not report the ideal level of information,almost exclusively use cluster designs, and there are multiplereports per study, as well as multiple measures per report.
13
8/11/2019 Sarah Baird CCT vs UCT Education
14/30
Calculating Effect Sizes
We define anintervention to be a UCT or a CCT. We define astudy to be a different version of a UCT or a CCT(or in a few experiments a UCTand a CCT) implemented indifferent placesFor many of these studies, there are multiple publications(journal articles, working papers, technical reports, etc.). We referto these asreports .In our meta-analysis, the unit of observation is thestudy . Thismeans that we would like to construct one effect size per studyfor the overall effect on any of our three outcome variables andfor each subgroup (if reported).
14
8/11/2019 Sarah Baird CCT vs UCT Education
15/30
Calculating Effect Sizes
For each subgroup, we construct one effect size by synthesizingand summarizing multiple effect sizes within each report, thenagain synthesizing and summarizing those combined effect sizesfrom different reports within a study.
We createsynthetic effects when the effect sizes are notindependent of eachother. This is the case when there are multiple effects reported for thesame sample of participants. These effects are combined using a simpleaverage of each effect size (ES) and the variance is calculated as the variance of that mean with the correlation coefficientr assumed to beequal to 1 When two or more ES are independent of each other, we createsummaryeffects . To combine these estimates into an overall estimate (or an estimatefor a pre-defined subgroup), we utilize a random effects (RE) model.
15
8/11/2019 Sarah Baird CCT vs UCT Education
16/30
MAIN RESULTS
16
8/11/2019 Sarah Baird CCT vs UCT Education
17/30
Results of the search
17
75 reports were included in our review.
Table 4: Characteristics of analysis sample
Panel A: Reference level characteristics: (N=75) Number %
Publication type:Journal article 33 44.00%Working paper 27 36.00%Technical Reports 10 13.33%Dissertation 4 5.33%Unpublished 1 1.33%
Reports effects on:Enrollment/Dropout 67 89.33%Attendance 17 22.67%Test Score 12 16.00%
8/11/2019 Sarah Baird CCT vs UCT Education
18/30
Results of the search
18
Panel B: Study level characteristics, binary (N=35)
Number %UCT 5 14.29%CCT 26 74.29%UCT/CCT 4 11.43%
Regional DistributionLatin America and the Caribbean 19 54.29%Asia 8 22.86%
Africa 8 22.86%Female recipient 16 45.71%Pilot Program 9 25.71%Random Assignment 12 34.29%
Panel C: Study level characteristics, continuous (N=35)Mean Std
Control Follow-up Enrollment Rate 0.785 0.146
# of Reports per Study 2.17 2.360Transfers per Year 8.24 4.020Transfer amount (% of HH Income) 5.66 7.890Annual per Person Cost (USD) 351 414
8/11/2019 Sarah Baird CCT vs UCT Education
19/30
19
.
.
Overall (I-squared = 84.5%, p = 0.000)
Familias en Accion
Bolsa Escola
Old Age Pension Program
CCT
Red de Opportunidades
Japan Fund for Poverty Reduction
UCT
SIHR
CT-OVC
Bolsa Familia
Old Age Pension
PROGRESA
TekoporaNahouri Cash Transfers Pilot Project
Chile Solidario
Red de Proteccion Social
PRAF II
Subtotal (I-squared = 86.5%, p = 0.000)
Female Secondary Stipend Program
China Pilot
Pantawid Pamilyang Pilipino Program
Bono de Desarrollo
Social Risk Mitigation Project
OportunidadesIngreso Ciudadano
Name
Tayssir
Comunidades Solidarias Rurales
Child Support Grant
Conditional Subsidies for School Attendance
Jaring Pengamanan Sosial (JPS)
CESSP Scholarship Program
SIHR
Tayssir Nahouri Cash Transfers Pilot Project
Bono Juancito Pinto
Subtotal (I-squared = 52.2%, p = 0.041)
Program Keluarga Harapan (KPH)
Juntos
Social Cash Transfer Scheme
Program
Colombia
Brazil
South Africa
Panama
Cambodia
Malawi
Kenya
Brazil
Brazil
Mexico
ParaguayBurkino Faso
Chile
Nicaragua
Honduras
Bangladesh
China
Philipines
Ecuador
Turkey
MexicoUruguay
Country
Morocco
El Salvador
South Africa
Colombia
Indonesia
Cambodia
Malawi
MoroccoBurkino Faso
BoliviaIndonesia
Peru
Malawi
1.36 (1.24, 1.48)
1.29 (1.06, 1.56)
1.90 (1.01, 3.58)
1.15 (0.82, 1.62)
1.85 (1.23, 2.80)
1.34 (0.95, 1.88)
1.98 (1.53, 2.57)
1.11 (0.84, 1.47)
1.96 (0.82, 4.66)
1.15 (0.96, 1.38)
1.48 (1.27, 1.72)
1.53 (0.72, 3.24)1.50 (1.03, 2.17)
1.22 (1.00, 1.50)
4.36 (2.08, 9.11)
1.45 (1.20, 1.75)
1.41 (1.27, 1.56)
1.74 (1.10, 2.77)
2.74 (1.18, 6.37)
1.48 (0.80, 2.73)
1.30 (1.07, 1.57)
0.72 (0.47, 1.11)
1.25 (1.09, 1.43)1.25 (0.87, 1.79)
Ratio (95% CI)
1.40 (1.20, 1.64)
3.78 (1.62, 8.82)
1.04 (0.53, 2.04)
1.05 (0.96, 1.16)
1.42 (1.19, 1.70)
2.72 (1.92, 3.87)
1.30 (0.96, 1.75)
1.59 (1.38, 1.85)1.31 (0.94, 1.83)
1.02 (0.92, 1.14)
1.23 (1.08, 1.41)
0.98 (0.95, 1.02)
1.33 (1.16, 1.53)
1.04 (0.82, 1.31)
Odds
1.36 (1.24, 1.48)
1.29 (1.06, 1.56)
1.90 (1.01, 3.58)
1.15 (0.82, 1.62)
1.85 (1.23, 2.80)
1.34 (0.95, 1.88)
1.98 (1.53, 2.57)
1.11 (0.84, 1.47)
1.96 (0.82, 4.66)
1.15 (0.96, 1.38)
1.48 (1.27, 1.72)
1.53 (0.72, 3.24)1.50 (1.03, 2.17)
1.22 (1.00, 1.50)
4.36 (2.08, 9.11)
1.45 (1.20, 1.75)
1.41 (1.27, 1.56)
1.74 (1.10, 2.77)
2.74 (1.18, 6.37)
1.48 (0.80, 2.73)
1.30 (1.07, 1.57)
0.72 (0.47, 1.11)
1.25 (1.09, 1.43)1.25 (0.87, 1.79)
Ratio (95% CI)
1.40 (1.20, 1.64)
3.78 (1.62, 8.82)
1.04 (0.53, 2.04)
1.05 (0.96, 1.16)
1.42 (1.19, 1.70)
2.72 (1.92, 3.87)
1.30 (0.96, 1.75)
1.59 (1.38, 1.85)1.31 (0.94, 1.83)
1.02 (0.92, 1.14)
1.23 (1.08, 1.41)
0.98 (0.95, 1.02)
1.33 (1.16, 1.53)
1.04 (0.82, 1.31)
Odds
intervention reduces enrollment intervention increases enrollment
1.5 1 1.5 2 3 4
8/11/2019 Sarah Baird CCT vs UCT Education
20/30
20
Table 10: Summary of Findings (Enrollment)Odds of Child
Being Enrolled inSchool:
StatisticallySignificant?*
#EffectSizes*
Comments
CCT vs. UCT Our analysis of enrollment includes 35effect sizes from 32 studies. Both CCTsand UCTs significantly increase the oddsof a child being enrolled in school, withno significant difference between thetwo groups. This binary distinctionmasks considerable heterogeneity in theintensity of the monitoring andenforcement of the condition. When wefurther categorize the studies, we find asignificant increase in the odds of achild being enrolled in school as theintensity of the condition increases. Inaddition, studies with explicit conditionshave significantly larger effects thanstudies with some or no conditions.
Overall (vs. Control) 36% higher Yes 35
UCT (vs. Control) 23% higher Yes 8
CCT (vs. Control) 41% higher Yes 27
CCT (vs. UCT) 15% higher No 35
Condition Enforcement No Schooling Condition (vs. Control) 18% higher Yes 6Some Schooling Condition (vs.Control) 25% higher Yes 14
Explicit Conditions (vs. Control) 60% higher Yes 15
Intensity of Condition Increases by 7%for each unitincrease in
intensity ofcondition.
Yes 35
Notes: We consider a study to be statistically significant if it is significant at the 90% level or higher. I use the term effect sizehere instead of study since the studies that directly compare CCTs and UCTs have two effect sizes in the analysis. All otherstudies have one.
8/11/2019 Sarah Baird CCT vs UCT Education
21/30
CCT vs. UCT too simplistic?
Could we categorize all programs, and not just the CCTs, inorder of the intensity of schooling conditionalities imposed bythe administrators?
0. UCT programs unrelated to children oreducation such as Old Age Pension Programs (2)
1. UCT programs targeted at children with an aim of improving schoolingoutcomes such as Kenyas CT - OVC or South Africas Child Support Grant 2. UCTs that are conducted within a rubric of education such as MalawisSIHR UCT arm or Burkina Fasos Nahouri Cash Transfers Pilot Project UCTarm (3)
3. Explicitconditions on paper and/or encouragement of childrensschooling, but no monitoring or enforcement such as Ecuadors BDH or
Malawis SCTS (8)
21
http://blogs.worldbank.org/impactevaluations/comment/1055http://blogs.worldbank.org/impactevaluations/comment/10558/11/2019 Sarah Baird CCT vs UCT Education
22/30
CCT vs. UCT too simplistic?
4. Explicit conditions, (imperfectly) monitored, with minimal enforcement such as Brazils Bolsa Familia or Mexicos PROGRESA (8)5. Explicit conditions with monitoring and enforcementof enrollment condition such as Honduras PRAF - II or Cambodias CESSPScholarship Program(6)6. Explicit conditions with monitoring and enforcementof attendance condition such as Malawi's SIHR CCT arm or Chinas PilotCCT program(10)
22
8/11/2019 Sarah Baird CCT vs UCT Education
23/30
23
- . 5
0
. 5
1
1 . 5
2
0 2 4 6Condition Enforced
8/11/2019 Sarah Baird CCT vs UCT Education
24/30
24
.
.
.
Overall (I-squared = 84.5%, p = 0.000)
Bono de Desarrollo
Jaring Pengamanan Sosial (JPS)
Child Support Grant
Some Schooling Conditions with No Monitoring or Enforcement
No Schooling Conditions
Red de Proteccion Social
Nahouri Cash Transfers Pilot Project
PROGRESA
Old Age Pension Program
Conditional Subsidies for School Attendance
Program Keluarga Harapan (KPH)
Red de Opportunidades
Name
Explicit Conditions Monitored and Enforced
CESSP Scholarship Program
Chile SolidarioOportunidades
Nahouri Cash Transfers Pilot Project
CT-OVC
China Pilot
SIHR
Comunidades Solidarias Rurales
Ingreso Ciudadano
Social Cash Transfer Scheme
Japan Fund for Poverty Reduction
Subtotal (I-squared = 87.2%, p = 0.000)
SIHR
Female Secondary Stipend Program
Familias en Accion
Subtotal (I-squared = 80.6%, p = 0.000)
Bono Juancito Pinto
Social Risk Mitigation Project
Tekopora
Pantawid Pamilyang Pilipino Program
Subtotal (I-squared = 0.0%, p = 0.950)
Juntos
Bolsa Familia
Old Age Pension
PRAF II
Bolsa Escola
Tayssir
Tayssir
Program
Ecuador
Indonesia
South Africa
Nicaragua
Burkino Faso
Mexico
South Africa
Colombia
Indonesia
Panama
Country
Cambodia
ChileMexico
Burkino Faso
Kenya
China
Malawi
El Salvador
Uruguay
Malawi
Cambodia
Malawi
Bangladesh
Colombia
Bolivia
Turkey
Paraguay
Philipines
Peru
Brazil
Brazil
Honduras
Brazil
Morocco
Morocco
1.36 (1.24, 1.48)
1.30 (1.07, 1.57)
1.42 (1.19, 1.70)
1.04 (0.53, 2.04)
4.36 (2.08, 9.11)
1.50 (1.03, 2.17)
1.48 (1.27, 1.72)
1.15 (0.82, 1.62)
1.05 (0.96, 1.16)
0.98 (0.95, 1.02)
1.85 (1.23, 2.80)
Ratio (95% CI)
2.72 (1.92, 3.87)
1.22 (1.00, 1.50)1.25 (1.09, 1.43)
1.31 (0.94, 1.83)
1.11 (0.84, 1.47)
2.74 (1.18, 6.37)
1.30 (0.96, 1.75)
3.78 (1.62, 8.82)
1.25 (0.87, 1.79)
1.04 (0.82, 1.31)
1.34 (0.95, 1.88)
1.25 (1.10, 1.42)
1.98 (1.53, 2.57)
1.74 (1.10, 2.77)
1.29 (1.06, 1.56)
1.60 (1.37, 1.88)
1.02 (0.92, 1.14)
0.72 (0.47, 1.11)
1.53 (0.72, 3.24)
1.48 (0.80, 2.73)
1.18 (1.05, 1.33)
1.33 (1.16, 1.53)
1.96 (0.82, 4.66)
1.15 (0.96, 1.38)
1.45 (1.20, 1.75)
1.90 (1.01, 3.58)
1.40 (1.20, 1.64)
1.59 (1.38, 1.85)
Odds
1.36 (1.24, 1.48)
1.30 (1.07, 1.57)
1.42 (1.19, 1.70)
1.04 (0.53, 2.04)
4.36 (2.08, 9.11)
1.50 (1.03, 2.17)
1.48 (1.27, 1.72)
1.15 (0.82, 1.62)
1.05 (0.96, 1.16)
0.98 (0.95, 1.02)
1.85 (1.23, 2.80)
Ratio (95% CI)
2.72 (1.92, 3.87)
1.22 (1.00, 1.50)1.25 (1.09, 1.43)
1.31 (0.94, 1.83)
1.11 (0.84, 1.47)
2.74 (1.18, 6.37)
1.30 (0.96, 1.75)
3.78 (1.62, 8.82)
1.25 (0.87, 1.79)
1.04 (0.82, 1.31)
1.34 (0.95, 1.88)
1.25 (1.10, 1.42)
1.98 (1.53, 2.57)
1.74 (1.10, 2.77)
1.29 (1.06, 1.56)
1.60 (1.37, 1.88)
1.02 (0.92, 1.14)
0.72 (0.47, 1.11)
1.53 (0.72, 3.24)
1.48 (0.80, 2.73)
1.18 (1.05, 1.33)
1.33 (1.16, 1.53)
1.96 (0.82, 4.66)
1.15 (0.96, 1.38)
1.45 (1.20, 1.75)
1.90 (1.01, 3.58)
1.40 (1.20, 1.64)
1.59 (1.38, 1.85)
Odds
intervention reduces enrollment intervention increases enrollment
1.5 1 1.5 2 3 6
8/11/2019 Sarah Baird CCT vs UCT Education
25/30
25
Table 10: Summary of Findings (Enrollment)Odds of Child
Being Enrolled inSchool:
StatisticallySignificant?*
#EffectSizes*
Comments
CCT vs. UCT Our analysis of enrollment includes 35effect sizes from 32 studies. Both CCTsand UCTs significantly increase the oddsof a child being enrolled in school, withno significant difference between thetwo groups. This binary distinctionmasks considerable heterogeneity in theintensity of the monitoring andenforcement of the condition. When wefurther categorize the studies, we find asignificant increase in the odds of achild being enrolled in school as theintensity of the condition increases. Inaddition, studies with explicit conditionshave significantly larger effects thanstudies with some or no conditions.
Overall (vs. Control) 36% higher Yes 35
UCT (vs. Control) 23% higher Yes 8
CCT (vs. Control) 41% higher Yes 27
CCT (vs. UCT) 15% higher No 35
Condition Enforcement No Schooling Condition (vs. Control) 18% higher Yes 6Some Schooling Condition (vs.Control) 25% higher Yes 14
Explicit Conditions (vs. Control) 60% higher Yes 15
Intensity of Condition Increases by 7%for each unitincrease in
intensity ofcondition.
Yes 35
Notes: We consider a study to be statistically significant if it is significant at the 90% level or higher. I use the term effect sizehere instead of study since the studies that directly compare CCTs and UCTs have two effect sizes in the analysis. All otherstudies have one.
8/11/2019 Sarah Baird CCT vs UCT Education
26/30
26
Table 11: Summary of Findings (attendance and test scores)
Panel A: Attendance Odds of ChildBeing Enrolled in
School:StatisticallySignificant?*
#EffectSizes*
Comments
Overall (vs. Control) 59% higher Yes 20 A smaller number of studies assess the affect of CCTs
and UCTs on attendance compared to enrollment. BothCCTs and UCTs have a significant affect on attendance.While the effect size is always positive, we do not detectsignificant differences between CCTs and UCTs onattendance.
UCT (vs. Control 42% higher Yes 5CCT (vs. Control) 64% higher Yes 15CCT vs. UCT (regression) 17% higher No 20Intensity of Conditionality
(regression)Increases by 8% foreach unit increasein intensity ofcondition.
No 20
Panel B: Test Scores Standard DeviationIncrease in Test
ScoresStatisticallySignificant?*
#EffectSizes*
Comments
Overall (vs. Control) 0.06 Yes 8 There are very few studies that analyze test scores. Wehave a total of 8 effect sizes measured from 5 studies.CCTs significantly increase test scores, though the size isvery small at 0.08 standard deviations. We find noimpact of UCTs on test scores. Additional research onthe impact of CCTs and UCTs on test scores is needed.
In order to include these results in meta-analysis testsshould be conducted with the entire sample, and results presented in terms of standard deviations.
UCT (vs. Control 0.04 No 3
CCT (vs. Control) 0.08 Yes 5
CCT vs. UCT (regression) 0.05 No 8Intensity of Conditionality
(regression)
Increase of 0.02
standard deviationsfor each unitincrease inintensity ofconditions
No 8
Notes: We consider a study to be statistically significant if it is significant at the 90% level or higher. I use the term effect size here instead ofstudy since the studies that directly compare CCTs and UCTs have two effect sizes in the analysis. All other studies have one.
8/11/2019 Sarah Baird CCT vs UCT Education
27/30
AUTHORS CONCLUSIONS
27
8/11/2019 Sarah Baird CCT vs UCT Education
28/30
Authors Conclusions (1) Our main finding is that both CCTs and UCTs improve the oddsof being enrolled in and attending school compared to no cashtransfer program.
The pooled effect sizes for enrollment and attendance are always larger for CCTprograms compared to UCT programs but the difference is not significant. The findings of relative effectiveness on enrollment in this systematic review arealso consistent with experiments that contrast CCT and UCT treatments directly.
When programs are categorized as having no schoolingconditions, having some conditions with minimal monitoringand enforcement, and having explicit conditions that are
monitored and enforced, a much clearer pattern emerges. While interventions with no conditions or some conditions that are not monitoredhave some effect on enrollment rates (18-25% improvement in odds of beingenrolled in school), programs that are explicitly conditional, monitor complianceand penalize non-compliance have substantively larger effects (60% improvementin odds of enrollment).
28
8/11/2019 Sarah Baird CCT vs UCT Education
29/30
Authors Conclusions (2) The effectiveness of cash transfer programs on testscores is small at best .
It seems likely that without complementing interventions, cash transfersare unlikely to improve learning substantively.
Limitations:Very few rigorous evaluations of UCTs need more research!Study limited to education outcomesMost of the heterogeneity in effect sizes remains unexplainedNot much information on cost
Researchers:Report relevant data to calculate effect size (i.e. control means atbaseline and follow up)Self reports vs. more objective measures.
29
8/11/2019 Sarah Baird CCT vs UCT Education
30/30
Acknowledgements and Funding
Thank you!!International Development Coordinating Group of the Campbell Collaboration for theirassistance in development of the protocol and draft report. John Eyers and Emily Tanner-Smith as well as anonymous referees for detailed commentsthat greatly improved the protocol. David Wilson for help with the effect size calculations.
Josefine Durazo, Reem Ghoneim, and Pierre Pratley for research assistance.Funding
This research has been funded by the Australian Agencyfor International Development (AusAID).
The views expressed in the publication are those of the authors and not necessarily those
of the Commonwealth of Australia. The Commonwealth of Australia accepts noresponsibility for any loss, damage or injury resulting from reliance on any of theinformation or views contained in this publication
The Institute for International and Economic Policy (IIEP) at GeorgeWashington University also assisted with funding for a researchassistant.
30
Top Related