David Evans Katrina Kosec Joachim De Weerdt World Bank ... · Health and nutrition: Regular check...
Transcript of David Evans Katrina Kosec Joachim De Weerdt World Bank ... · Health and nutrition: Regular check...
David Evans Katrina Kosec Joachim De Weerdt World Bank IFPRI EDI [email protected]
Workshop on evaluating the impact of cash transfer programs in sub Saharan Africa
The Transfer Project February 5-8, 2012
Community Based-Conditional Cash Transfer Program Overview A pilot CCT program implemented in 40 villages of three
districts. 5,000 households and 13,000 beneficiaries Conditionalities:
Education: Primary school enrollment and 80% attendance for children 7-15
Health and nutrition: Regular check ups for children younger than 7 (three times a year) and for elderly (once a year)
Selection of beneficiaries: Community targeting and proxy means test
Benefits: US$3 per month per child US$6 per month per elderly Maximum benefit per month: $18
Bi monthly payments since January 2010. Ten payment cycles
Institutional arrangements Community-based managed by a Social Fund
Central. Overall management and monitoring, support to subnational authorities, disbursement of funds.
District. Technical support and guidance including training and follow-up of implementation in the villages.
Community level. Selection of beneficiaries, day to day implementation, collection of information to register beneficiaries and to verify compliance with conditions, direct payments
Program overview – survey results
Median household has received 8 payments
received TZS 22,500 (US$13) last payment
Delivery 94% pick up from village office
3% CMC member delivers at home
At mid-line, asked people about conditionalities 15% mentioned all 3 conditions
27% mentioned 2 conditions
51% mentioned 1 condition (To be fair, not all conditions apply to each household.)
How did you spend the last payment?
Education • Uniforms • Books • Pens • Facilities
Health • Treatment • Medication • Fare (transport) • Health insurance (13%)
Investment • Poultry • Farming • Livestock
Other • Clothes • Kerosene • Various household goods
20%
30%
40%
50%
Education Health Investment Other
% of households that mentioned…
Overall program timeline Process Evaluation
July-September 2011
Targeting Assessment
April-July 2011
Impact Evaluation
Baseline: February 2009
First payments: January 2010
Follow up household survey: July-September 2011
End-line household survey: October 2012
Beneficiary perception: Focus groups
August 2011
December 2012
Impact evaluation – Key questions What is the impact of CCTs on various outcomes for vulnerable
children and the elderly? Conditionalities
Education Health
Cash transfers Consumption Savings Transfers within community
Community administration Trust in community
Underlying question: Is the community-based model as effective at
achieving health, education, and consumption gains as has been the case of more centrally administered models used elsewhere?
Impact evaluation model Randomized-control trial
Among 80 villages with long history of TASAF assistance
Household selection process conducted in all 80 villages
Randomly select 40 to enter the program
Other 40 serve as control
Project will be rolled out in control villages after 2012 end line survey.
Baseline Characteristics Treatment vs Comparison Villages Variable Comparison Difference
for Treatment
Significant difference?
Household size 3.9 0.08 No
Can read & write (adults) 40% -0.02 No
Enrolled in school (age 6-18) 75% -5% No
Missed school in last week (enrolled) 42% +9% 90%
Sick or injured in last month 27% +2% No
Seek care (if sick) 86% -4% No
Improved roof (iron sheets, cement) 8% -5% 95%
Improved floor (tiles, concrete) 37% -3% No
Any toilet / latrine facilities 75% -6% No
Evaluation Strategy Treatment and comparison communities are roughly
balanced
If anything, treatment communities are slightly worse off – highlights importance of baseline
Differences-in-differences
Treatment & Control, Before & After
Specification
(Treatment dummy is collinear with households fixed effects and so is dropped.)
Impact of interest is β3
Caveat: Baseline survey was carried out before final selection of households, so that 9% of households in treatment villages ultimately not treated
FEsHouseholdAfterTreatmentAftery *321
Treated hh Untreated hh in Treated village Diff
# children (<18) 1.72 1.45 -0.27
# elderly (60+) 1.21 1.07 -0.14
Improved roof? 0.33 0.30 -0.03
Two specifications Intent-to-Treat: Treatment = Assignment to Treatment
Village
What is the impact of being initially assigned to treatment (whether you received it or not)?
Treatment on the Treated: Treatment = Self-reported Treatment, using Assignment to Treatment Village as Instrumental Variable
What is the impact of actually receiving treatment?
(If we didn’t use the instrument, this would be biased. The instrument is randomized and solves this problem.)
Depending on variable, first stage is between 0.9 and 0.95
Being assigned to treatment increases your odds of treatment by 90 to 95%: great instrument
Follow-up results
Health outcomes
No systematic impacts on illness. Potential reduction in sick days.
Sick in past 4 weeks? Sick days missed in past year?
HH ave HH yng child ave
HH elderly ave
HH child ave HH elderly ave
TOT 0.02 (0.03)
0.03 (0.07)
0.05 (0.05)
-2.90* (1.76)
-2.29 (4.23)
ITT 0.02 (0.04)
0.03 (0.11)
0.04 (0.06)
-2.70 (2.88)
-2.17 (6.38)
Average for Baseline Comparison Villages
0.30*** (0.01)
0.28*** (0.03)
0.39*** (0.02)
12.30*** (0.67)
39.16*** (1.57)
# households 1,611 347 1,368 495 984
Follow-up results
Health seeking behavior Visits to clinic in past year Visited
dispensary / hospital for main health problem (per capita)
Medication taken for main health problem (per capita)
Per capita Per elderly Per child
TOT 1.26*** (0.31)
1.14*** (0.35)
1.33** (0.60)
0.17*** (0.06)
0.11** (0.04)
ITT 1.18*** (0.43)
1.08** (0.50)
1.20 (1.12)
0.16* (0.09)
0.10 (0.07)
Average for Baseline Comparison Villages
2.6*** (0.1)
2.7*** (0.1)
3.3*** (0.1)
0.5*** (0.0)
0.9*** (0.0)
# households 1,492 1,368 361 673 673
Systematic increase in health-seeking behavior.
Follow-up results
Education Ever attend school?
(Age 2-17) Currently in school? (Age 6-17)
Absent last week (Age 6-17)
TOT 0.07*** (0.03)
0.04 (0.03)
0.03 (0.04)
ITT 0.07* (0.04)
0.04 (0.04)
0.03 (0.07)
Average for Baseline Comparison Villages
0.80*** (0.01)
0.90*** (0.01)
0.22*** (0.02)
# households 843 739 664
No impact on school dropouts or attendance for 6-17, but “ever attend” probably picking up earlier enrollments.
Follow-up results
Annual Expenditures (in TSH)
Significant increase in non-food expenditures, particularly for children’s clothing and women’s clothing (surprising).
Also tested for health service expenditures and expenditures on ceremonies: No significant impact.
Food expenditures (and total expenditures) analysis to come.
Cigarette Children’s clothing Adult clothing
Full sample With children Men Women
TOT -80.70 (73.07)
168.70*** (43.49)
254.18*** (73.54)
-22.16 (78.9)
219.8** (93.01)
ITT -73.00 (105.36)
152.59* (77.39)
238.22* (139.5)
-20.77 (139.63)
206.0 (173.0)
Average for Baseline Comparison Villages
264.29*** (26.30)
266.05*** (19.32)
427.30*** (34.86)
350.3*** (33.92)
516.9*** (42.90)
Follow-up results
Children’s goods & activities
No systematic results on initial analysis of children’s activities around the household.
Have shoes? (Per child average)
Have slippers? (Per child average)
Work on income-generating activity?
TOT 0.16** (0.07)
0.11 (0.07)
-0.01 (0.01)
ITT 0.15 (0.12)
0.10 (0.14)
0.001 (0.050)
Average for Baseline Comparison Villages
0.41 0.63 0.69
Follow-up results
Savings & Credit
Only small proportion of households have savings, but this increases slightly.
Bank account? Non-bank savings? Took loan in last 12 months?
TOT -0.01 (0.01)
0.03* (0.02)
0.04 (0.03)
ITT -0.01 (0.01)
0.026 (0.02)
0.04 (0.05)
Average for Baseline Comparison Villages
0.02*** (0.003)
0.01** (0.005)
0.19*** (0.01)
# households 1,611 1,609 1,611
Follow-up results
Trust Trust most people Trust your
community Trust leaders in your community
TOT -0.07* (0.04)
0.04 (0.04)
0.06* (0.03)
ITT -0.07 (0.05)
-0.04 (0.04)
0.05 (0.04)
Average for Baseline Comparison Villages
0.24*** (0.01)
0.55*** (0.01)
0.81*** (0.01)
# households 1,599 1,595 1,602
Summary of results Strong impacts
Health-seeking behavior Expenditures (especially children’s)
Trust Increase in trust of leaders: Seen in action Decrease in trust on people in general: Friction over
targeting?
Limited impacts in other areas Fewer sick days for children Early enrollment to school (but not enrollment or attendance
for 6-17 year olds) Increase in non-bank savings (from 1% of households to 3% of
households)
Discussion of impact results Dissolution of impacts
Households in treatment villages received 8,313 TZS (US$4.90) less in private transfers over last 12 months
6% reduction in benefits
Education condition
High baseline levels of enrollment
Not binding in most cases
How well did the program actually function?
Complementary evaluation of process Focus groups
Open-ended questions within household survey
Principal complaints
What do you most dislike about the program? Nothing
Insufficient money
Timing issues
Deductions
CMC
Dropped
Other
Irregularities 2% reported having been asked for a contribution
Contribution = 9% of transfer
7% reported receiving less than usual at the last payment but were unable to explain why
Some complains of dishonest staff in quantitative survey
Issues to look into, but relatively small proportion of program
Focus Group Findings Range of groups (in 6 TV)
Health & education providers, village councils, beneficiaries, non-beneficiaries
Findings Beneficiaries: Program is great, but more money
General openness to conditionalities: Some complaints on the need for elderly health visits [but note big improvements in elderly clinic visits]
Non-Beneficiaries: Insufficient transparency in selection process
Overall positive – highly participatory
Some cases of people with means included or truly needy excluded
No suggestion of reduced traditional solidarity systems
Policy Implications: the way forward
Government decision to scale up the CCT as part of a safety net (0.2% of the GDP per year)
First phase: 250,000 households nationwide
Combined with a cash-for-work to same households to smooth consumption in lean seasons
Required institutional adjustments at central and subnational levels
Strengthen implementation arrangements and adjust design parameters:
Eligible groups, conditions, benefit structure, targeting system and process, decentralized operation, information systems