Porter 2 a_education2

Smarter social protection? Impacts of Ethiopia’sProductive Safety Net on Child Cognitive Outcomes

Marta Favara, Catherine Porter, Tassew Woldehanna

Young Lives Conference

[email protected]

September 8th, 2016

(Porter, YL Oxford) Smarter social Protection September 8th, 2016 1 / 22

Overview

1 Introduction

2 Empirical strategy

3 Data

4 Results

5 Tentative conclusions and next steps


Motivation

Productive Safety Net Program in Ethiopia (PSNP) is the largestsocial protection scheme in SSA outside South Africa

Previous evaluations have shown improvements in food security,consumption (IFPRI evaluation)

Porter and Goyal find +ve effect on child nutrition (height-for-age) in2009

Does this extend through to educational achievements in 2013?


Framework: Cognitive development of children

Households invest in children subject to their preferences andconstraints- there is a “production function” for child cognitivedevelopment, which depends on inputs (nutrition, learning activitiesinside school, home).

Todd and Wolpin (2003, 2007) note that under the assumption thateffect of inputs (both observed and unobserved) as well as that ofinitial ability decline geometrically over time, then a “lag valueadded” model can be specified, using only the immediate lag ofachievement serving as a proxy for all previous inputs, and ability.

Vector of controls includes PSNP participation, plus other exogenoushousehold controls (household size, shocks, prices).


Context and previous literature

Evaluations of (Conditional) cash transfers have shown improvementsin nutrition outcomes (e.g.Behrman and Hoddinott, 2005)

Evidence on improvement in enrollment, grade, school attendance(Fiszbein and Schady 2009 review, Glewwe and Kassouf, 2012 BolsaEscola)

Baird et al (2012, Malawi) CCT outperforms UCT at schoolenrolment (2013, systematic review): CCT and UCT both havesignificant effect on enrolment and attendance

Less evidence for any impact on cognitive achievement

Akresh et al (2016, Burkina Faso) CCT outperforms UCT onenrolment, neither have impact on cognitive.

Barham et al (2013, Nicaragua) cognitive impacts for boys of CCT


Public Works and Child Outcomes

Positive impacts of public works on nutrition in Bangladesh(Maskie-Taylor et al 2010) and Ethiopia (Porter and Goyal, 2016)

Mixed results for child cognitive outcomes of India’s NREGA, the largestworkfare programme in the world:

Mani et al (2014) find strong positive effects on grade progression anda number of cognitive skills tests for the Young Lives cohorts in 2013

Shah and Steinberg (2015) find the opposite - NREGs hampersprogression and cognition due to time demands - though this ismainly for children over 12


PSNP Background

2005 Productive Safety Net Scheme introduced after long history ofresponsive food aid and drought

Workfare programme for 80% of participants, UCT for others(labour-poor hh)

Cash-first principle (tho in practice 60-40)

Co-ordination between donors, and with Government innovation

In 2013 7.2 million beneficiaries (roughly 10% of the nationalpopulation) in 8 of 10 regions.

Phase 3 from 2010-2015 attempted to improve timeliness andpredictability of transfers, strengthen public works and accountability,


Empirical Strategy (1)

Estimate reduced-form: cognitive development in 2013, as a functionof child and household characteristics

Equivalent to difference-in-differences strategy (test parallel trendsusing 2002-2006 HAZ and wealth index)

YC (OC) child is aged 12 (19) years in 2013. Lagged ability ismeasured at age 8 (12).

PSNP enters directly into equation (1) with exogenous controls -dummy variable if a HH member participated in either public works ordirect support

Include lag (pre-prog) child achievement as per conceptual model, tocontrol for ability and child-hh unobservables

Cluster fixed effects to control for village unobservables (schoolquality etc)

Remaining challenge is non-random targeting


Empirical Strategy (2): Selection of Comparison Group

Selection of comparison group: never treated may be systematicallydifferent from beneficiaries

We create a propensity score for treatment based on pre-programcharacteristics, and keep those with a PS in the common support andparallel trend to the treated.

Table: PSNP groups defined by years of treatment

PSNP Groups Obs Percentage

Comparison 728 51.3%2009 PSNP only 266 18.8%2009 & 2013 PSNP 424 29.9%

Total 1,418 100%


Young Lives dataset

Younger Cohort 2,000 children born in 2000, Older cohort 1,000children born in 1994

20 sentinel sites (random selection within sites)

Multi-purpose survey including cognitive skills, nutrition, education,non-cognitive skills, household background, program participation

4 rounds (2002, 2006, 2009, 2013)

Attrition rate around 2.2 percent (very low)


Figure 3. PSNP and survey timing

Figure: PSNP and survey timing


Cognitive Tests

Peabody Picture Vocabulary Test: administered to YC but not theOC in round 4

Maths test in R1-4 for OC and YC

Difficulty of using raw scores across rounds/ages - we calculatez-score across cohort for math and IRT score for PPVT (controllingfor language)

Therefore results interpreted slightly differently


Table: Comparison and Treatment groups: baseline characteristics

Comparison PSNP beneficiaries t-testMean Std.Dev Mean Std.Dev

In 2002 (age 1 or 8)HAZ -1.64 0.065 -1.64 0.074 0.981In 2006 (age 5 or 12)HAZ -1.61 0.042 -1.71 0.041 0.076Food expenditure 79.71 2.003 77.41 2.034 0.421Non-food expenditure 41.49 2.197 26.43 0.785 0.000Changes between 2002-06HAZ 0.04 0.061 -0.08 0.062 0.168No. of shocks 1.25 0.059 0.83 0.050 0.000

Observations 728 690

Note: Average values measured in round 1 and 2 at the age of 1 and 5 (8 and 12); standard deviation reported in paren-theses. Food, non-food expenditure are per month/adult and in current Birr. The p-value for a t-test for differences inmeans between Control group and PSNP beneficiaries is reported in the last column.


Headline results: some effects on cognitive outcomes,gender story is complex...

No effect of PSNP overall on maths scores using value added model(both cohorts)

We split PSNP into a) continuing beneficiaries b) graduates

Math score significant effect for graduates (only), this driven by boys(only)

In all (math) models, no apparent difference in effect by cohort


Table: Math score: value added approach

All Boys Girls

PSNP until 2012 0.273* 0.443** 0.134(0.145) (0.215) (0.205)

PSNP-continuing 0.096 0.158 0.025(0.144) (0.212) (0.213)

YC*PSNP until 2012 -0.091 -0.272 0.073(0.174) (0.250) (0.250)

YC*PSNP-continuing -0.144 -0.226 -0.070(0.162) (0.237) (0.239)

Younger Cohort 0.571 1.087 -0.114(0.708) (0.933) (1.130)

Math z-score (round 2) 0.181*** 0.197*** 0.140***(0.031) (0.040) (0.050)

Male 0.077 . .(0.075) . .

Age (in months) 0.008 0.013 -0.000(0.009) (0.011) (0.014)

Obs. 938 507 431R-squared 0.221 0.249 0.234

Note: * p<0.1 ** p<0.05 ***p<0.1. Controls for all models includewealth index; shocks between 2006 and 2013; height for age at age5 (z-score); food and non-food expenditure; time use; maternal edu-cation; agency and pride index and parental educational aspirations.Robust standard errors.


Mechanisms of impact?

From previous work we know that there is a nutritional gain in 2009of beneficiaries

We examine school enrolment, and child work (recall timecommitment of PW programme)

Again no significant results with PSNP (any) dummy

We again split into a) continuing beneficiaries b) graduates

Headline results show reduction in enrolment and increase in hoursworked, for those in graduate householdsHowever, breakdown by cohort shows that enrolment is only lower forOC (for graduates)And puzzlingly, hours worked by OC lower for continuing householdsBreakdown by gender - shows that OC girls in graduate households areworking more, and OC boys in continuing households are working lessNo overall effects for YC

A caveat on these results around enrolment and work hours in R2...


Table: Enrollment and hours of work

Enrol Enrol Enrol Work Work Work

(1) (2) (3) (4) (5) (6)PSNP until 2012 -0.064** -0.058** -0.053* 0.428* 0.443* 0.417*

(0.027) (0.028) (0.028) (0.231) (0.237) (0.236)PSNP-continuing -0.009 -0.004 -0.002 -0.152 -0.085 -0.096

(0.022) (0.023) (0.023) (0.176) (0.184) (0.185)Younger Cohort 0.136 0.116 0.106 -1.966 -1.514 -1.464

(0.204) (0.206) (0.206) (1.646) (1.646) (1.648)Male -0.037** -0.037* -0.037* 0.102 0.089 0.087

(0.019) (0.019) (0.019) (0.157) (0.157) (0.157)Age (in months) -0.003 -0.004 -0.004 0.000 0.006 0.007

(0.002) (0.002) (0.002) (0.020) (0.020) (0.020)

Obs. 1,417 1,417 1,417 1,413 1,413 1,413R-squared 0.240 0.247 0.249 0.104 0.120 0.121

Note: * p<0.1 ** p<0.05 ***p<0.1. Controls for all models include wealth index. Columns 2-6 include shocks between 2006 and2013. Columns 3 and 6 include maternal education. Robust standard errors.


Table: Enrollment and hours of work: heterogeneity across cohorts & gender

Enrol WorkAll Boys Girls All Boys Girls

PSNP until 2012 -0.160*** -0.140* -0.175** 0.802 0.210 1.458**(0.058) (0.079) (0.087) (0.523) (0.762) (0.706)

PSNP-continuing 0.030 0.030 0.024 -1.009** -1.414** -0.474(0.057) (0.078) (0.085) (0.451) (0.568) (0.718)

YC*PSNP until 2012 0.175*** 0.142* 0.219** -0.675 -0.020 -1.484**(0.062) (0.085) (0.089) (0.549) (0.798) (0.740)

YC*PSNP-continuing -0.041 -0.049 -0.015 1.273*** 1.739*** 0.646(0.060) (0.083) (0.088) (0.476) (0.618) (0.741)

Younger Cohort 0.029 0.208 -0.177 -1.281 -2.108 -0.042(0.206) (0.300) (0.278) (1.654) (2.361) (2.288)

Male -0.038** . . 0.097 . .(0.019) . . (0.157) . .

Age (in months) -0.004* -0.002 -0.006** 0.012 -0.000 0.028(0.002) (0.004) (0.003) (0.020) (0.028) (0.027)

Obs. 1,417 777 640 1,413 775 638R-squared 0.257 0.250 0.287 0.132 0.105 0.199

Note: * p<0.1 ** p<0.05 ***p<0.1. Controls for all models include wealth index. Columns 2-6 includeshocks between 2006 and 2013. Columns 3 and 6 include maternal education. Robust standard errors.


Summary

A complex story on the effect of PSNP participation on cognitiveoutcomes of boys and girls aged 12 and 19

Effects seem to be greater for those who have graduated by 2013 (in2012, mainly) - selection effect?

Order of magnitude around 0.15 SD on maths test score(standardised) - greater for boys

Relatively similar to CCT findings in the literature on younger children- fairly surprising given no conditionality, work requirement

Enrollment and time use effects for OC only, OC girls in graduatehouseholds working more, OC boys in continuing households workingless

Comments most welcome...


Table: Year of graduation for “2009 only” PSNP beneficiaries

Year of graduation Frequency Percent

2007 1 1.922008 4 7.692009 1 1.922010 6 11.542011 9 17.312012 26 50.002013 5 9.62

Total 52 100.00

Note: Much missing data, total 195 participants who reported receivingPSNP in 2009, but not in 2013.


Table: Descriptive Statistics

Total Older Cohort Younger CohortVariables Mean Std. Error Mean Std. Error Mean Std. Error

Younger Cohort 0.678 0.012 . . . .Individual characteristicsMale 0.549 0.013 0.561 0.023 0.543 0.016Age (in months) 172.57 1.042 229.26 0.183 145.69 0.128Height-for-age (z-score) in R2 -1.657 0.029 -1.659 0.054 -1.656 0.035Household characteristicsMother’s education: primary and above 0.394 0.013 0.368 0.023 0.406 0.016Wealth Index: bottom tercile 0.532 0.013 0.456 0.023 0.568 0.016Wealth Index: mid tercile 0.351 0.013 0.417 0.023 0.320 0.015Wealth Index: top tercile 0.117 0.009 0.127 0.016 0.112 0.010Food expenditure 271.77 5.373 260.93 13.138 276.91 4.886Non-food expenditure 142.90 5.007 146.40 8.803 141.24 6.090% expenditure on education 0.009 0.000 0.013 0.000Number of shocks since 2009 0.904 0.029 0.901 0.050 0.905 0.036Number of shocks between 2006 &2009 1.916 0.037 1.884 0.065 1.931 0.044Time use (Hours/day)On household chores 2.045 0.045 2.248 0.108 1.949 0.042In domestic tasks 2.277 0.070 2.281 0.153 2.275 0.074In paid activities 0.687 0.065 1.864 0.181 0.126 0.029In school 4.652 0.073 3.090 0.155 5.400 0.067Studying (outside school) 1.266 0.033 1.357 0.082 1.223 0.028

Observations 1418 456 962


Porter 2 a_education2

Government & Nonprofit

Transcript of Porter 2 a_education2