A Presentation of the 2015 National ECCD

Centre Program Impact Evaluation

Outline of presentation

Research questions

Study design

Results

• Access

• Learning

• Quality

• Equity

Conclusion

Recommendations

Research Questions

1. What are the added gains of children in ECCD versus children who are

not in ECCD over the course of one academic year?

2. Which ECCD type (community, CSO, private, NFE parenting) produces

the greatest gains in children's early learning and development in one

academic year?

3. How does children's learning and development vary as the ECCD center

quality improves?

4. How does children's learning and development vary as the quality of the

home environment improves?

5. Which children are most disadvantaged at the start of the academic year

and how does the ECCD program (through centers or parents) address

inequity?

Timeline of evidence collection

March 2015

• Child skills at start of school year

• Caregiver survey

May-September 2015

• ECCD program quality observation (National Quality Monitoring)

November 2015

• Child skills at end of school year

• Caregiver survey

Study Sample Overview

• 3 districts from each region selected (9 total)

• Districts selected based on prevalence and diversity of ECCD programming

• ECCD/NFE Centers selected randomly (120 total)

• 10-15 children ages 3, 4 and 5 who are new to ECCD program within each center selected randomly

• Total sample: 1,200 – 1,800 children and parents/guardians

Stratified Random Sample

Study Sites

Study Sample: Centers

CSO Community Private Corporate

NFE

Parenting

No

ECCD Total Region

Chukha 0 2 2 5 0 5 14 1

Samtse 5 5 1 0 0 5 16 1

Thimphu 0 2 13 2 0 3 20 1

Dagana 6 5 0 0 5 0 16 2

Wangdi 0 2 3 1 3 0 9 2

Zhemgang 4 2 0 0 1 0 7 2

Mongar 5 7 1 1 9 0 23 3

SJ 0 2 0 0 2 0 4 3

Trashigang 0 4 0 0 0 7 11 3

Total 20 31 20 9 20 20 120

Final Study Sample: Children

Present at

endline

# Missing % Missing

Community 393 62 14%

Corporate 87 14 14%

Private 206 24 10%

CSO 125 27 18%

NFE parenting 169 18 10%

No ECCD 209 43 17%

Total 1,189 188 14%

TO O L S

International Development and Early Learning

Assessment (IDELA)

Motor Development

• Fine and Gross motor skills: Hopping; Copying shape; Folding paper; Drawing

Emergent Language and Literacy

• Print awareness; Expressive language; Letters; Phonological awareness; Listening comprehension

Emergent Math/ Numeracy

• Number sense; Shapes; Simple operations Sorting; Problem Solving; Measurement & comparison

Social-Emotional Development

• Empathy; Emotional awareness; Self awareness; Conflict resolution; Peer relationships

Learning Approaches

Execu

tive F

unctio

n

Bhutan: Spiritual,

Moral and Cultural

Development

Aligning IDELA with Bhutan’s ELDS

• 4 items

Physical Well-Being, Health, and Motor Development

• 5 items

Social and Emotional Development

• 6 items

Language, Literacy and Communication

• 13 items

Approaches to Learning

• 7 items

Cognition and General Knowledge

• 3 items

Spiritual, Moral and Cultural Development

IDELA Caregiver Questionnaire

Section Description

General family information Sex of child, child age, parental literacy, parental

education, languages spoken at home

ECCD experience and

educational expectations

Child participation in ECCD program, details of

participation, parental expectation of child’s educational

attainment

Home learning environment

and parenting practices

Types of reading materials at home, types of toys at

home, child-parent interactions

Parent self-efficacy Parental attitudes about their role in child’s development

Socio-economic status Roof and wall of home materials, objects/appliances

owed, land/animals owned, child work status

Quality Monitoring Tool for ECCD Centers (QMTEC):

4 Guiding Principles/Areas

ECCD programs are safe and protective of children’s health and wellbeing: Five Indicators

ECCD programs meet the physical, social-emotional, linguistic and cognitive developmental needs of young children: Ten Indicators

ECCD programs encourage and support active engagement for children, child centered teaching, and improved learning outcomes of all children.: Five Indicators

ECCD programs, parents and local communities are actively involved in planning, decision-making and action to improve early education: Three Indicators

Early Childhood Environment Rating Scale

(ECERS)

Language and Literacy Environment

• Print environment

• Book and literacy access

• Adult reading with children

• Words and sounds

• Emergent writing

• Talking and listening

• Books and pictures

Emergent Mathematics Environment

• Math and numbers

• Counting

• Reading simple numbers

• Math activities

Interactions in Classroom

• Discipline

• Teacher-child interactions

• Child-child interactions

Overview of results framework

Home learning environment

Family background

Educational expectations

Home learning environment and parenting practices

Parent self-efficacy

Socio-economic status

ECCD program quality

4 principles of QMTEC

ECCD facilitator training and program enrollment information (child/teacher ratio)

Quality of literacy and numeracy environment in classroom; interactions within classroom (ECERS)

Preparedness for primary school

Knowledge and skills gained by children between beginning and end of school year (IDELA)

R E S U LT S :

AC C E S S

Access Results: Conclusions

Children who attend ECCD centers are better prepared for primary school than those who do not

Child who attend ECCD are better prepared

for primary school

Figure 1. Learning and Development at Endline for 5 & 6 year olds

50%

29%

39%

35%

70%

45%

71%

48%

53%

48%

78%

60%

0% 20% 40% 60% 80% 100%

Motor

Literacy

Numeracy

Social-emotional

Culture

IDELA

% Correct

ECCD No ECCD

Access Results: Recommendations

Children who attend ECCD are better prepared for primary school than those who do not

• Recommendation: Increase access to ECCD centers

• Recommendation: Undertake longitudinal study of the effects of ECCD in primary school

Child who attend ECCD are better prepared

for primary school

Figure 1. Learning and Development at Endline for 5 & 6 year olds

50%

29%

39%

35%

70%

45%

71%

48%

53%

48%

78%

60%

0% 20% 40% 60% 80% 100%

Motor

Literacy

Numeracy

Social-emotional

Culture

IDELA

% Correct

ECCD No ECCD

Access Results: Recommendations

Children who attend ECCD are better prepared for primary school than those who do not

• Recommendation: Increase access to ECCD centers

• Recommendation: Undertake longitudinal study of the effects of ECCD in primary school

R E S U LT S :

L E A R N I N G &

D E V E L O P M E N T

Learning & Development Results: Conclusions

Strongest learning gains occur in Community, CSO and Private centers

Children’s weakest skills areas are literacy, social-emotional and math

Child Learning & Development:

Emergent Literacy

Figure 2. IDELA Emergent Literacy Gains, by ECCD Type

9% 14% 15% 16%24%

12%

11%14%

21% 24%

26%

26%

0%

20%

40%

60%

80%

NFE

Parenting

No ECCD Corporate Community Private CSO

% C

orr

ect

Baseline Gain

Child Learning & Development:

Emergent Math

Figure 3. IDELA Emergent Math Gains, by ECCD Type

24% 21% 23% 28%35%

21%

12% 14%17%

20%20%

23%

0%

20%

40%

60%

80%

No ECCD NFE

Parenting

Corporate Community Private CSO

% C

orr

ect

Baseline Gain

Child Learning & Development:

Social-Emotional Development

Figure 4. IDELA Social-Emotional Development Gains, by ECCD Type

14% 21%30%

19% 16%23%

15%16%

16%

18% 20%23%

0%

20%

40%

60%

80%

NFE

Parenting

Corporate Private No ECCD CSO Community

% C

orr

ect

Baseline Gain

Child Learning & Development:

Total IDELA Score

Figure 5. Total IDELA Score Gains, by ECCD Type

30%22%

30%42%

32%25%

14%16%

17%

19%23%

26%

0%

20%

40%

60%

80%

No ECCD NFE

Parenting

Corporate Private Community CSO

% C

orr

ect

Baseline Gain

Learning & Development Results:

Recommendations

Strongest learning gains occur in Community, CSO and Private centers

• Recommendation: Provide further support to Corporate centers

• Recommendation: Review and improve NFE parenting program

Children’s weakest skills areas are literacy, social-emotional and math

• Recommendation: Improve instruction in literacy, social-emotional development and math for all ECCD centers

R E S U LT S :

Q UA L I T Y

Quality Results: Conclusions

More learning happens in higher quality classrooms

Classroom quality is driven by the interactions between facilitators and children

Lower social-emotional gains in larger classes

More learning in higher quality classrooms

Figure 6. Quality (ECERS) Scores & IDELA Learning Gains, Community

Centers

11%

17%

22%

28%

0%

5%

10%

15%

20%

25%

30%

1 2 3 4

% ID

ELA

Gai

n

Overall Quality (ECERS) Rating

Classroom quality driven by facilitator-child

and child-child interactions

Figure 7. Quality (ECERS) Classroom Interactions & IDELA Learning

Gains, Community Centers

15%

19%

23%

27%

0%

5%

10%

15%

20%

25%

30%

1 2 3 4

% ID

ELA

Sco

re G

ain

Quality (ECERS) Rating – Classroom Interactions

av

Lower social-emotional gains in larger

classrooms

Figure 8. Relationship between social-emotional gains and child-teacher

ratio, All Centers

24%22%

20%18%

15%

0%

5%

10%

15%

20%

25%

30%

5:1 10:1 15:1 20:1 25:1

% S

oci

al-e

motional

Gai

n

Child-Teacher Ratio

Quality Results: Recommendations

More learning happens in higher quality classrooms

• Recommendation: Focus on ensuring quality in all ECCD programs

Classroom quality is driven by interactions between facilitators and children

• Recommendation: Invest in quality pre-service training and in-service support for facilitators

Lower social-emotional gains in larger classes

• Recommendation: Maintain class size of 15:1 as recommended by Bhutan’s Ministry of Education

R E S U LT S : E Q U I T Y

Equity Results: Conclusions

Children with more learning and play at home have stronger baseline skills and make stronger gains

Children in rural areas have the weakest home learning environments – both materials and activities

Children experiencing more negative discipline at home make smaller learning gains

No gender differences in learning gains

Children in rural areas gained as much as children in urban areas despite many disadvantages

av

Children with more learning and play at home

have stronger baseline skills

Figure 9. Home Learning Activities & IDELA Baseline

13%

17%

22%

13%

22%

30%

13%

26%

38%

0% 10% 20% 30% 40% 50%

Age 3

Age 4

Age 5

IDELA % Correct

Child

age

A lot of learning/play at home Some learning/play at home

No learning/play at home

And make stronger gains, across all types of

ECCD centers

Figure 10. Home Learning Activities & IDELA Gains

15%

20%

24%

0%

10%

20%

30%

40%

No learning/play at

home

Some learning/play at

home

Lots of learning/play at

home

% C

orr

ect

Children in Private and Corporate centers

experience more learning activities at home

Figure 11. Home learning activities, by center type

0%

20%

40%

60%

80%

100%

Read to

child

Tell stories Sings Plays with

child

Draws with

child

Teaches

new things

Teaches

letters

Teaches

numbers

% p

arents

do

ing

activi

ty in p

ast

week

CSO NFE Parenting No ECCD Community Corporate Private

Children in Private and Corporate centers

have more reading materials at home

Figure 12. Home Learning Materials, by ECCD Type

10%

85%

30%

74%

49%

32%

51%

25%

55%

35%

57%

27%

0%

20%

40%

60%

80%

100%

No storybooks 4+ reading material types

Private Corporate Community CSO No ECCD NFE Parenting

Children experiencing more negative

discipline at home make smaller learning gains

Figure 13. Negative Discipline & IDELA Learning Gains

28%

20%

0%

10%

20%

30%

40%

No negative discipline Hitting, spanking & yelling

All parents report using negative discipline

Figure 14. Negative discipline, by center type

48% 46%50%

46%

53% 50%

0%

20%

40%

60%

80%

100%

CSO NFE

Parenting

No ECCD Community Corporate Private

% n

ega

tive

dis

ciplin

e a

ctiv

itie

s in

pas

t

week (

out

of 3)

av

No gender differences in early learning

Figure 15. IDELA Learning & Development Gains, by Gender

26% 22%36%

22% 19%25%

27%

19%

21%

29%

17%15%

29% 26%35%

19% 19% 25%

23%20%

23%

26%16%

16%

0%

20%

40%

60%

80%

Girls Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls Boys

Community Corporate Private CSO NFE

parenting

No ECCD

% C

orr

ect

Girl Baseline Girl Gain Boy Baseline Boy Gain

Children in rural areas make gains as strong

as those in urban areas

Figure 16. Total IDELA Score Gains, by Urban/Rural

21%

24%

13%

19%

49%

23%

33%

18%

20%

20%

23%

22%

36%

33%

23%

28%

62%

35%

29%

18%

23%

15%

13%

20%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Rural

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Moto

rM

ath

Litera

cySE

LC

ult

ure

IDELA

Rural Baseline Rural Gain Urban Baseline Urban Gain

Equity Results: Recommendations

Children with more learning and play at home have stronger baseline skills and make stronger gains

• Recommendation: Invest in parenting programs that emphasize learning and play at home, as well as creation of child-friendly learning materials

Children in rural areas have weakest home learning environments – both materials and activities

• Recommendation: Increase provision of parenting programs, especially in disadvantaged communities

Children experiencing more negative discipline at home make smaller learning gains

• Recommendation: Incorporate positive parenting messages into all parenting curricula

No gender differences in learning gains

Children in rural areas gained as much as children in urban areas despite many disadvantages

• Recommendation: Continue to increase opportunities for rural children to enroll in ECCD centers

Conclusions

Access

• Children who attend ECCD centers are better prepared for primary school

Learning

• Strongest learning gains come from Private, Community and CSO centers

• Weakest skill in areas are literacy, social-emotional and math

Quality

• More learning in higher quality classrooms

• Classroom quality driven by interactions between facilitators and children

• Lower social-emotional gains in larger classes

Equity

• Children with more learning and play at home have stronger baseline skills and make stronger gains

• Children in rural areas have weakest home learning environments – both materials and activities

• Children experiencing more negative discipline at home make smaller learning gains

• No gender differences in learning gains

• Children in rural areas gained as much as children in urban areas despite many disadvantages

Recommendations

Access

• Increase access to ECCD center programs

• Undertake longitudinal study of the effects of ECCD in primary school

Learning

• Provide more support to Corporate ECCD centers

• Review and improve NFE parenting program

• Improve instruction in literacy, numeracy and social-emotional development for all ECCD centers

Quality

• Focus on improving quality of all ECCD programs

• Invest in quality pre-service facilitator training and in-service support

• Maintain class size of 15:1 as recommended by Bhutan’s Ministry of Education

Equity

• Invest in parenting programs that emphasize learning and play at home, as well as creation of child-friendly learning materials

• Increase provision of parenting programs, especially in disadvantaged communities

• Incorporate positive parenting messages into all parenting curricula

• Continue to increase opportunities for rural children to enroll in ECCD centers

Discussion

www.savethechildren.org

lpisani@savechildren.org