RECURSO PERU

RECURSO-Perú

Popularizing and instrumentalizing reading

attainment standards: a modest Odyssey.

Ian Walker, World Bank

EGRA Workshop

Washington DC, 13th March 2008

RECURSO PERU

Outline

• “60 words a minute”: a standard and a video to unbalance the low quality equilibrium

• An excited response: politicians, NGOs, development professionals

• Institutionalizing the measurement of attainment in the REACT DPL series: a national census; an alternative metric

• Comparing the fluency test with a written comprehension test: preliminary findings from Junin

• Next steps

RECURSO PERU

“60 words a minute”

• The video was shown at breakfast• Uncovering the scandal of schools poor

performance: “shocking the system”• Showing that outcomes can be different even

in poor, isolated, indigenous communities• Promoting a metric which links to an

appropriate pedagogical strategy • A standard parents can understand: how can

you know if your child is learning well at school?

• Operationalizing accountability at local level: making schools responsible for what they achieve (school and teacher performance as critical variables determining outcomes)

RECURSO PERU

An excited response

• Politicians: demand for improved performance of schools; link to results based management

• Media: broadcasting the video for free• NGOs: adopting the standard to guide

performance of their schools – eg Solaris, Peru

• Development professionals: a breath of fresh air. Pirated copies!

RECURSO PERU

Institutionalizing a policy

• REACT DPL: strengthening standards and accountability

• Use of censal and administrative data - making service producing units accountable

• Peru decided on census of second grade reading attainment. Now covering almost 80% of schools/kids

• Disagreements on how & what to measure: fluency test versus written reading comprehension test

RECURSO PERU

Junin regional study: comparing the two metrics

• Bank funded in-depth study of education conditions in Junin

• Fluency tests and reading comprehension tests applied to almost 500 children

• Result: strong correlation between two metrics: correlation coefficient of 0.54 (99% confidence level). Rises to 0.59 in rural schools and 0.62 in multigrade.

• 70% of kids are ranked in same decile or within a quintile (up or down) on both tests.

RECURSO PERU

Figure 1. Histogram of Reading Fluency

0

5

10

15

20

25

30

Per

cent

age

0 20 40 60 80 100 120 140Fluency (maximum number of words read per minute)

Figure 2. Histogram of Written Comprehension

0

5

10

15

20

25

30

Per

cent

age

0 2 4 6 8 10 12 14 16 18 20 22Written comprehension

RECURSO PERU

Figure 1. Histogram of Reading Fluency

0

5

10

15

20

25

30

Per

cent

age

0 20 40 60 80 100 120 140Fluency (maximum number of words read per minute)

Figure 2. Histogram of Written Comprehension

0

5

10

15

20

25

30

Per

cent

age

0 2 4 6 8 10 12 14 16 18 20 22Written comprehension

RECURSO PERU

Conclusions

• Two tests gave very similar results– Children coached on fluency will do well on the

comprehension test – Children with poor fluency likely to do badly on

comprehension

• Difficult to argue for using fluency test in universal evaluation; comprehension test correlates well and is easier to administer

• Obfuscating transformations of test scores into categories with no common-sense meaning is not intrinsic to either test

• In either case, one can use score of 100% as an implicit (absolute) standard against which each child is measured

RECURSO PERU

Next steps

• Institutionalizing and broadening the censal evaluation

• Feedback from census into school improvement plans / school level targets

• Implementing effective strategies to improve attainment – Early reading specialization– Reading materials– Support and supervision– Tracking individual children’s attainment

RECURSO PERU

Thank you