UNICEF’s Data Must Speak Projectfor increased accountability and community engagement

Overview of Data Must Speak

• A technical assistance program using data for increased community engagement, improved system management and accountability.

• Project implemented by UNICEF HQ, ROs, COs and UNESCO-IIEP Pole de Dakar, supported financially by the Global Partnership for Education, Hewlett foundation and UNICEF Education Thematic Fund.

• Implementation: since August 2014.

• Participating countries: Madagascar (2014), Togo (2014), Zambia (2015), Nepal (2015), Peru (2015, financial support through EduTrac project), the Philippines (2016) requests from Indonesia, Tanzania, DRC, Côte d’Ivoire, Chad, Timor-Leste etc.

Rationale

Issues:• Lack of equity in resource distribution• Lack of value for moneyFrom traditional accountability route -> direct accountability

Community empowerment

Providing user-friendly relevant information to stakeholders at all levels

Theory of change

Central Level

Budgeting and planning

Region

Some resource allocation decision

District / Inspectorate

Pedagogic supervision

School / Community

School Based Management

Working at several interrelated levels

Benefits

Improved data quality

Informed resource

allocation

Targeted pedagogic

support

School/District

Profile CardCode:

Region

Number of schools

Total number of students:

Schools with water 25

CP1

CP2

CE1

CE2

CM1

CM2

Total

Teachers Textbooks

Current Need Current Need Math Reading

Grade 1

Grade 2

Grade 3

Grade 4

Inspectors and advisors Grade 5

Grade 5

Inspectors Total

Advisors

Total

A. Results C. Indices of context & resources and results

NB: Efficiency = Results/Context and resources

B. Resources

Reading textbook to pupil ratio

Math textbook to pupil ratio

% of seats per pupil

% of temporary classrooms

Pupil to classroom ratio

Pupil to teacher ratio

% of volunteer teachers

R2

School to inspector ratio

School to advisor ratio

Teacher to inspector ratio

Teacher to advisor ratio

R2= Consistency in distribution of teachers

Ministry of Primary and Secondary

Education PLATEAUX

KPELEDistrict

Secretariat

-----------

-----------

Department of Education Planning and

Evaluation

47.3

47.6

47.9

49.5

47.2

32.5

29.7

28.0

27.2

Enrolled

124

121 3209

3712

5022

3612

3215

Survival Rate 47.1 51.1

Parity index of CEPD pass rate

42.4

43.8

42.3

0.4

31

68.0 62.6

46

22.9

77.0

1.1

1.0

1 1 2

32.1

30.1

32.8

121

5233

207 416 335

60 66 68

0 0 0

1 1 2

63 47

0.3

27.3

Assistant 71 12 83

Parity index of repeaters

Parity index of survival rate 0.80 0.85

0.80 0.71

0.85

0.82

0.74 0.76% of Repeaters 28.9

0.98

Results

0.46 0.41

0.90 0.93

Context & resources 0.47

District Region TOGOCEPD Pass rate 77.1 80.6

Efficiency

3706 0 3659 0

4146

0 3319Need

24.5

Total 372 42 414

Private 88 7 95

4. Comparaison of District at Regional and National Levels

75.1

District Region TOGO

Government

50.8

71.1

44

32.5

74.5

0.3

0.5

0.76

1.62 1.861.63

45.6

Level

1.01

1.17 1.16

1.17

92 0.992436

Female TotalMale 0.76

2495

0

20724 142 19265 879

2131 0 1378 436

1.15

0.96

3333

33

1.08

1.14

21.1

26.0

19.5

119

119

121 2853

28.730.6

1814

46.7

46.4

43.8 44.5

% girl Enrolled % girl

48.8

49.3

49.5

725

2687

2564

1868

1917919138

2528

28.9

213

Reading

47.6 29.5

Pupil book Ratio

4913

47.9

Mathematics

109 0.98 0.93

3 Resources

Male Female Total

23 236

4671 351

1.020 1.123802 0

2. Bottleneck Analysis

Schools accesible year round ? 90Number of double shift

schools99 Number of schools with a sportsfield 58

Number of urban schools? 12

313

Level

551 Number of schools with electricity

Section % repeaters % repeaters

120

19,138School Year: 2012-2013 District Profile

1. General Statistics

Community: Schools with latrines 25

12 Number of classrooms

Government 96 3 Private: 21

2012-2013 2011-2012

71.9

34.7

55.8

38.7

#DIV/0! 395 231

#DIV/0!

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Context & resources Results

District Region TOGO

Improved community

Involvement

Informed school

based management

Major activities

• Community level:

“Simplified school profiles”: comparative information that can be understood by communities (Madagascar, Togo, etc.).

Real time monitoring by communities (Peru etc.).

School based management using the profiles.

• System level e.g. linking EMIS and learning data, district profiles for targeted support & resource allocation, equity index (Nepal).

• Global level: knowledge sharing – what works in which context, global public good.

Lessons learnt / principles

• Lessons learnt: Impact of information feedback on absenteeism/repetition

(Madagascar), learning (Pakistan), grant reception rate (Uganda), EMIS data (Niger).

But… Sustainability issues. Tools often too technical / not comprehensive.

• Hence principles: National ownership (at all levels) / flexibility. Cost-effective systems, integrated in national sector plans. Synergies with other initiatives. Learning by doing. Knowledge sharing. Global public good.

Some results

Togo school profile(extract)

Using EMIS & exams (central) data

Provide comparative (with district) information to communities

Peru school profile

Community discussion around profile

Issue: student absenteeism

Reason: fear of snakes

Decision: cutting the grass around the school to ensure the school is a snake-free zone.

Results at other levels

Zambia: school profiles developed, extensive consultation (400 headteachers), EMIS-generated, distributed to all schools alongside the EMIS questionnaire.

Results at other levels

• Togo, Madagascar: composite indices to contrast resource, context, and performance for typology of districts/schools.

• Equity Index in Nepal on access, completion, learning (dimensions of gender, caste, location, disability, etc.), jointly supported by UNICEF and the WB endorsed and used by government for financial allocations to districts.

13

Thank you!

More information:

Gabrielle Bonnet: gbonnet@unicef.org

Mathieu Brossard: mbrossard@unicef.org

Short / long accountability route

Source: World Development Report 2004, Making Services Work for Poor People, World Bank, 2003

Examples of activities supported

Madagascar simplified school profile (extract)

Ministry of Basic & Secondary Education School Scorecard

Directorate of Planning 2009/10

Based on the information collected from the school by the CMs

Nb of Students

Nb of classrooms ##1.0

0.5

0.8

0.3

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

## ##

## ##

District

0.3

M

0.8

0.8

0.5

School

15

NAT G3

Maths

NAT G5

Englis h

Region

Dro p-o ut

ra te G1-6

Student

teacher rat io

% o f qualified

teachers

Nb o f

s tudents per

Eng lish

textbook

663

Albion

Banjul

Region1

Nb o f

s tudents

p er Maths

textbook

NAT G3

Englis h

NAT G5

Maths

####

##

##

## ## ##

Performance

Resources

Simplified School Profile in

Gambia

Fewer indicators

Use of icons for concepts and comparisons

School Profile in Madagascar

Highlight on notably high or low indicators

Automatically generatedanalysis text

School Profile: Togo

Includes school indices (context/resources and

results) used in the calculation of school grants with an equity perspective

Code:

DRE:

Inspection:

Statut: PUBLIC

Directeur: BIMIZI TOCTAWONGOU

Canton Localité

CP1

CP2

CE1

CE2

CM1

CM2

Total

Enseignants tenant classe Equipements et manuels scolaires

Niveau

CP1

CP2

CE1

CE2

Montant reçu (en FCFA) CM1

CM2

Total

A. RESULTATS C. INDICES DE CONTEXTE/MOYENS ET DE RESULTATS

NB: Efficience = Résultat/Contexte et moyens

B. MOYENS

Ratio Elèves/Maître

% d'Enseignants Volontaires

% de places assises /élève

Ratio Manuel de lecture/élève

Ratio Manuel de calcul/élève

% de salles en matériaux provi

Rapport élèves/salle de classe 44

0.87

39

75,000

44

0.84

0.66

0.75

35

54

68

0.54

0.71

33.25

0.48

0.28

37

39

44

36

70

0.40

42.9

0 0 - -

0

0

24

Indice de parité de % de redoublants

Indice de parité de rétention 0.91

0.81 0.67

0.71

0

75,000

73

14

27

0.67

0 0 -

78.8

0 3

Manuels de calcul

Existant Besoin

0.31

Ecole

ONG 0

35

0

1

30

% de redoublants

Indice de parité de réussite au CEPD

79

54

24

0.92

23

0.95

87

Total

0

0.4

0.23

0.0

0.0

0.0

0.0

0.5

0.3

0.14

0

0

- -

47.7 30.7

0.3

0.2

0.0

0.0

0.5

- 0

4. Moyens

54

Ratio manuel/élève

4 160 51.9 37.5 153

0

3

404P004PMinistère des Enseignements Primaire

et Secondaire

-----------

Secrétariat Général

-----------

Direction de la Planification de

l'Education et de l'Evaluation

37 51.4

2011-2012

Inscrits % fi l les %redoublant

47 40.4 51.1

2. Contexte de l'école

Accès à l'école toute l'année ?

RURAL

40.6

Année scolaire: 2012-2013 FICHE ECOLE

1. Localisation de l'école

OUI

Electricité ?

Manuels de lecture

15

8

32

Niveau2012-2013

Homme Femme

Sections Inscrits % fi l les %redoublant

1 54 53.7 35.2

1 38 50.0 34.2

1 27.0

43.6 33.3

0Terrain de sport?

3. Effectifs d'élèves

39

43.8

0

Besoin

Volontaires

Cours jumelés?

37.5

0.92

5. Comparaison des résultats et des moyens de l'école à ceux de l'IEPP, de la DRE et du Pays

Tx de rétention 14.6 61

IEPP DRE TOGO

Besoin CalculLecture

Tx de réussite au CEPD en 2012

1

0

0Auxiliaires

0

1

0

159

0.30

0.67

2.2

Résultat

40

53

0

69

0.23

0.14

100.0

Efficience

32

47

82

0.5

0.8

1.6

DRE

0.4

0.8

1.7

TOGO

Contexte/moyens 0.40

0.74

1.9

IEPPEcole

0

Autre

Total

0

Dotation budgétaire de l'Etat

Subvention du PERI

0

1 0

Cotisation des parents 0

Fonctionnaires

22 0

136

0

1

Existant

0

38

32

35

0

CENTRALE

EPP ADJENGRE CENTRE/C

SOTOUBOUA_NORD

Ecole:

0 - -

ADJENGRE ADJENGRE

Total

Nb salles occupéesZone urbaine?

NB: 1 = oui / 0 = non

4

1

0

0

1 36 61.1 38.9 30 60.0

Eau?

Latrines ?

0.0

32

0.00

0.50

1.00

1.50

2.00

2.50

Contexte/moyens Résultat Efficience

Ecole IEPP DRE TOGO

School Profile: Zambia (draft, abstract 1)

Distribution of scores by level

Level 1 Level 2 Level 3 Level 4 Level 1

Pre school Gr 1 Gr 2 Gr 3 Gr 4 Gr 5 Gr 6 Gr 7 2013196 196 189 181 173 170 168 151 1595

98 98 95 91 87 85 84 76 850

98 98 94 90 86 85 84 75 745

5

2%

5

5%

5

1%

(%)

Children with Special Education Needs

(%)

Readmited after pregnancy 17

SCIENCE: 25% of boys and 23% of girls achieved Division 1 and 18% of boys and 40% of girls failed.SDS: 23% of boys and 25% of girls achieved Division 1 and 4% of boys and 19% of girls failed.

Refugees

Total (2014)1510616

612

ENGLISH: 6% of boys and 31% of girls achieved Division 1 and 21% of boys and 17% of girls failed.

Enrolment

Learning outcomes: Grade 7 Pass Rates

The overall results in 2014 have increased from 2013 (but

are lower than in 2012)

There is little difference in 2014 between the results of boys

and girls.

Learning outcomes: Grade 7 results by division

EnrolmentBoys

Girls

Orphans and Vulnerable Children

(%)

MATHEMATICS: 21% of boys and 24% of girls achieved Division 1 and 25% of boys and 18% of girls failed.ZAMBIAN LANGUAGES: 5% of boys and 2% of girls achieved Division 1 and 45% of boys and 41% of girls failed.

70%

89%

75%

87%

0%

20%

40%

60%

80%

100%

2012 2013 2014 District Province Zambia

school 2014Boys Girls

21% 17%25%

18%

45% 41%

18%

40%

4%19%

6%

31%21% 24%

5% 2%

25% 23% 23% 25%

0%

20%

40%

60%

80%

100%

Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls

English Maths Zambian languages Science Social and DevelopmentStudies

Fail Div. 4 Div. 3 Div. 2 Div. 1

5%

41%

39%

15%Age of new entrants in Grade 1

Under 7

Age 7

Age 8

Over 8

Automated

Text

School Profile: Zambia (draft, abstract 2)

Gr 1 Gr 2 Gr 3 Gr 4 Gr 5 Gr 6 Gr 7 Last year District1 2 1 2 3 3 1 1 23 3 3 2 2 3 3 3 21 2 3 3 3 1 1 1 32 2 2 2 2 2 2 3 2

Gr 1 Gr 2 Gr 3 Gr 4 Gr 5 Gr 6 Gr 70 0 0 9 27 28 0

23 29 22 0 3 24 180 0 24 21 28 0 0

10 11 0 4 16 0 0

34 40 46 34 75 52 18

School environment: Your school environment is more favourable than the district average.

Resources: Your school resources are above the district average, and are approximately equal to the provincial and national averages.

Performance: Your school performance is below the average district, provincial and national performance.

Science 42

# of textbooks needed TotalEnglish 65

Learning Materials

2

Pupil- Textbk. RatioEnglish

MathematicsZambian languages

Science

Total23

Rating of School environment, Resources and Performance

2

Total 299

Mathematics 120Zambian languages 73

Pupil-Textbook Ratios: More children have to share a single textbook in Grades 5 and 6 in English, and in Grade 2 in Mathematics.

Needs: The greatest textbooks needs are in Grade 2 for mathematic and Grade 5 for Zambian languages.

School environment Resources Performance

District average

School is well above average

School is about average

School is well below average

Automated

Text

District Profile in

Madagascar

Performance and Resources

of District -compared to the

regional averages

10-01-00

DISTRICT: ANDOHARANOFOTSY

CP1 Enrolled in CM2

CP2 Sat

CE Passed

CM1 Pass Rate

CM2 Passed/Enrolled in CM2

Average

Nb of schools

As a % of total number of primary schools

Nb of students Nb of students per school

Nb of teachers Nb of students per teacher

In a classroom % community teachers

Civil servants and Contracts % of teachers not in classroom

Community

Substitutes and others

Not in a classroom

Total

Drop-out Rate A

Drop-out Rate

Repetition Rate

Repetition Rate A Exam Pass Rate

Student-Teacher Ratio

Exam Pass Rate A

41

Your District

Worse than your

Department Average

Ratsy raha oharina amin'ny

salan'isa hita eo anivon'ny

CISCO

In Difficulty

Mahakivy

(Tsy ampy ny enti-manana

sady ratsy ny voka-

panadinana)

Better than your

Department Average

Tsara kokoa raha oharina

amin'ny salan'isa hita eo

anivon'ny CISCO

HOW DOES YOUR DISTRICT FARE IN TERMS OF ACADEMIC PERFORMANCE AND STUDENT-TEACHER RATIO?

Aiza ho aiza no misy ny District raha ny lafiny fahombiazana sy ny hamaroan'ny mpampianatra miasa ao aminy no jerena?

Better than your

Department Average

Tsara kokoa raha oharina

amin'ny salan'isan'ny hita eo

anivon'ny CISCO

Worse than your

Department Average

Ratsy raha oharina amin'ny

salan'isa hita eo anivon'ny

CISCO

OutstandingLafatra

(Tsy ampy aza ny enti-

manana: mpampianatra,

boky, sns. nefa tsara ny voka-

panadinana)

24.5

71.8

Your

Department

Your District Your Department

281

41

28.4

4.7

Type of District

Fisokajiana

Academic Performance

Vokatra azo

Student:Teacher RatioIsan'ny mpianatra tazonin'ny

mpampianatra iray

Worse than your

Department Average

Ratsy raha oharina amin'ny

salan'isa hita eo anivon'ny

CISCO

Better than your

Department Average

Tsara kokoa raha oharina

amin'ny salan'isa hita eo

anivon'ny CISCO

Disappointing

Mandiso fanantenana

(Ampy ny enti-manana

nefa ratsy ny voka-

panadinana)

Better than your

Department Average

Tsara kokoa raha oharina

amin'ny salan'isa hita eo

anivon'ny CISCO

Good

Mahafapo

(Ampy ny enti-manana

sady tsara ny voka-

panadinana)

Worse than your

Department Average

Ratsy raha oharina amin'ny

salan'isa hita eo anivon'ny

CISCO

Your District

30.4

19.0

CE

CM1

Your

Department

CP1

CP2

Your DistrictYour

Department

Public and Community Primary Schools

Exam Pass RateDrop-Out Rate Repetition Rate

PERFORMANCE

Your DistrictYour

Department

ANTANANARIVOANTANANARIVO ATSIMONDRANO

(According to data collected in the schools of your district in school years 2004 and 2005)

MENRSREGION:DPT:

DISTRICT PROFILE: YEAR 2005

4.1

12.1

83.5

54

4.8

65536

2.8

3.6

1.6

27.2

7.5

1.4

3.5

3.8

3.6

1.7

5.8

4.0

4.5

4.8

CM2

Average

Your District vs Your

Department

4.1

5.9 442

410

369

90.0

12.1

18.1

28.421.2

4

79.2

71.8

5

17.9

24.8

24.5

83.5

SCHOOLS, STUDENTS AND TEACHERS

Your District

56

516

54

5.8

7.1

2831

52

47

3

2

Madagascar: Classification of schools for district education officers

Exam Pass RateDrop-out Rate Repetition Rate

HOW DO YOUR SCHOOLS FARE IN TERMS OF ACADEMIC PERFORMANCE AND STUDENT-TEACHER RATIO?

Good schools EF1 AMBALAVAO

EFPC AMBOHIBARY

EPP MORATSIAZO

EPP LOHAMANDRY

EPP D'AMPANGABE

EF1 AMBALAVAO

EFPC AMBOHIBARY

EPP AMBONIANDREFANA

EPP MORATSIAZO

EPP LOHAMANDRY

EPP D'AMPANGABE

EPP AMBOHIDAHY

EFPC AMBOHIBARY

EPP AMBONIANDREFANA

EPP MORATSIAZO

EPP LOHAMANDRY

EPP D'AMPANGABE

EPP AMBOHIDAHY

Outstanding schools EPP AMBOHIBARIKELY

EPP AMBOHITSILAZAINA AMBALAVAO

ANTAMBOHO

EPP AMBOHIBARIKELY

EPP AMBOHITSILAZAINA AMBALAVAO

ANTAMBOHO

EPP AMBOHITSILAZAINA AMBALAVAO

Disappointing schools EPP AMBONIANDREFANA

EPP AMBOHIDAHY EF1 AMBALAVAO

Schools in difficulty EPP AMBOHIBARIKELY

ANTAMBOHO

District Profile in Togo

Bottleneck analysis integrated into the profile

(cf. Annex for details)

Code:

Region

Number of schools

Total number of students:

Schools with water 25

CP1

CP2

CE1

CE2

CM1

CM2

Total

Teachers Textbooks

Current Need Current Need Math Reading

Grade 1

Grade 2

Grade 3

Grade 4

Inspectors and advisors Grade 5

Grade 5

Inspectors Total

Advisors

Total

A. Results C. Indices of context & resources and results

NB: Efficiency = Results/Context and resources

B. Resources

Reading textbook to pupil ratio

Math textbook to pupil ratio

% of seats per pupil

% of temporary classrooms

Pupil to classroom ratio

Pupil to teacher ratio

% of volunteer teachers

R2

School to inspector ratio

School to advisor ratio

Teacher to inspector ratio

Teacher to advisor ratio

R2= Consistency in distribution of teachers

Ministry of Primary and Secondary

Education PLATEAUX

KPELEDistrict

Secretariat

-----------

-----------

Department of Education Planning and

Evaluation

47.3

47.6

47.9

49.5

47.2

32.5

29.7

28.0

27.2

Enrolled

124

121 3209

3712

5022

3612

3215

Survival Rate 47.1 51.1

Parity index of CEPD pass rate

42.4

43.8

42.3

0.4

31

68.0 62.6

46

22.9

77.0

1.1

1.0

1 1 2

32.1

30.1

32.8

121

5233

207 416 335

60 66 68

0 0 0

1 1 2

63 47

0.3

27.3

Assistant 71 12 83

Parity index of repeaters

Parity index of survival rate 0.80 0.85

0.80 0.71

0.85

0.82

0.74 0.76% of Repeaters 28.9

0.98

Results

0.46 0.41

0.90 0.93

Context & resources 0.47

District Region TOGOCEPD Pass rate 77.1 80.6

Efficiency

3706 0 3659 0

4146

0 3319Need

24.5

Total 372 42 414

Private 88 7 95

4. Comparaison of District at Regional and National Levels

75.1

District Region TOGO

Government

50.8

71.1

44

32.5

74.5

0.3

0.5

0.76

1.62 1.861.63

45.6

Level

1.01

1.17 1.16

1.17

92 0.992436

Female TotalMale 0.76

2495

0

20724 142 19265 879

2131 0 1378 436

1.15

0.96

3333

33

1.08

1.14

21.1

26.0

19.5

119

119

121 2853

28.730.6

1814

46.7

46.4

43.8 44.5

% girl Enrolled % girl

48.8

49.3

49.5

725

2687

2564

1868

1917919138

2528

28.9

213

Reading

47.6 29.5

Pupil book Ratio

4913

47.9

Mathematics

109 0.98 0.93

3 Resources

Male Female Total

23 236

4671 351

1.020 1.123802 0

2. Bottleneck Analysis

Schools accesible year round ? 90Number of double shift

schools99 Number of schools with a sportsfield 58

Number of urban schools? 12

313

Level

551 Number of schools with electricity

Section % repeaters % repeaters

120

19,138School Year: 2012-2013 District Profile

1. General Statistics

Community: Schools with latrines 25

12 Number of classrooms

Government 96 3 Private: 21

2012-2013 2011-2012

71.9

34.7

55.8

38.7

#DIV/0! 395 231

#DIV/0!

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Context & resources Results

District Region TOGO