Building Better Teachers in the Caribbean - World...

Building Better Teachers

in the Caribbean

Barbara Bruns, Lead Education Economist

World Bank

Regional Learning EventImproving Teaching and Learning Outcomes in the English-speaking Caribbean Countries with ICT

Bridgetown, BARBADOS13-16 April 2011

Overview

1. Context: Raising teacher quality is the central education challenge

– Student learning is key to economic gains from education

– Teacher quality is key to student learning

– Teacher effectiveness is highly variable

2. Key challenges for teacher policy

3. Understanding what makes teachers effective in the classroom– New methods for looking “inside the black box”

– Evidence from Brazil: Six conclusions

– Reactions from education policymakers

4. Learning from innovations in the Caribbean and Latin America

1. New evidence on the payoffs to teacher quality

Student learning Is the prime determinant of individual returns and economy-wide gains from education (Hanushek and Woessman, 2007)

Figure 3: Test Scores and Growth in Selected Developing Countries

Source: Hanushek and Woessmann: Education Quality and Economic Growth (2007)

BRAZIL

Chile

China

Colombia

Cyprus

Egypt

Ghana

Hong Kong

IndonesiaIndia Iran

Jordan

Korea, Rep. of

Morocco

Malaysia

Peru Philippines

Portugal

Romania

Singapore

ThailandTunisia

Turkey

Taiwan

Zimbabwe

-4

-3

-2

-1

0

1

2

3

4

-1.5 -1.0 -0.5 0.0 0.5 1.0

Con

diti

onal

Gro

wth

Conditional Test Score

Teacher quality is the prime determinant

of student learning

Best teachers can raise student learning by 2-3 grade levels in a single yearWorst teachers raise learning 0.5 grade (Hanushek and Rivkin, 2005)

Cumulative effects of these variations are huge, esp. given evidence that lower quality teachers are clustered in low-income schools

Highly variable teacher effectiveness found across all countries and contexts to date

Teacher effectiveness varies tremendously (measured as ability to generate year on year learning gains)

…and raising learning is a challenge for the Caribbean and LA

PISA Math 2009

Lower middleincome

Upper middle income High income:NonOECD High income: OECD

300

350

400

450

500

550

Indon

esia

Tunis

iaJo

rdan

Thail

and

Avera

ge: L

ower

midd

le inc

ome

Pana

ma Peru

Alban

iaCo

lombia

Braz

ilAr

genti

naMo

ntene

groKa

zakh

stan

(*) M

exico

(*) C

hile

Roma

niaUr

ugua

yBu

lgaria

Azerb

aijan

Serbi

a(*)

Turke

yRu

ssian

Fede

ration

Lithu

ania

Avera

ge: U

pper

midd

le inc

ome

Qatar

Trinid

ad an

d Tob

ago

Israe

lDu

bai (U

AE)

Croa

tiaLa

tvia

Eston

iaMa

cao-C

hina

Liech

tenste

inHo

ng Ko

ng-C

hina

Singa

pore

Avera

ge: H

igh in

come

: non

OECD

Gree

ce Italy

Spain

Portu

gal

Irelan

dUn

ited S

tates

Luxe

mbou

rgHu

ngary

Unite

d King

dom

Czec

h Rep

ublic

Swed

enPo

land

Austr

iaSlo

vak R

epub

licFra

nce

Norw

aySlo

venia

Denm

arkIce

land

Germ

any

Austr

alia

Belgi

umNe

w Ze

aland

Nethe

rland

sCa

nada

Japa

nSw

itzerl

and

Finlan

dKo

reaAv

erage

: High

inco

me: O

ECD

2. Elements of teacher qualityRecruiting, Grooming and Motivating Talent

Recruiting Failure to recruit talent Recruiting talent

Teachers recruited from bottom third of students, with low cognitive skills

Higher standards for content mastery. Brazil, Chile, Colombia, Jamaica, Guyana, Guatemala

Shortages of math and science teachers: 85% of OECS science teachers lack science background

Special incentives for scarce skills

Low entry requirements to teacher training programs: private schools w/o accreditation

Competitive entry into teacher training:only top 10-30% of students hired as teachers in Finland, Singapore and Korea

Few graduates from top universities pursue teaching as a career

Alternative certification and “public service” appeal (Teach For All model)Chile, Peru, Argentina, Brazil, Colombia, Mexico

GroomingFailure to groom Grooming Talent

Little supervision and support for new teachers despite documented low effectiveness in first year

Induction programs (like medical residency): Effective support, encouragement and performance evaluation in first years

No “de-selection” of low-performing teachers once hired…but also high attrition in some countries

Reforms that allow de-selection. Chile – 2011 reform: school directors can fire 5% of worst performing teachers each yearSingapore – bottom 10% of directors fired each year

Theoretical and ineffective in-service training

In-service training based on observed classroom practice

Teachers not convinced that “every child can learn”

Screen and train teachers for values: perseverance and belief in kids

MotivatingFailure to motivate Motivating high performance

Flat salary scalesAverage salary after 15 years only 30% above salary at entry

More attractive career laddersChile AVDI, Sao Paulo Prova de Promocão,Mexico CM, Peru CPM offer large pay increases with promotions based on competence

Salaries and promotions de-linked from performance

Pay for performance (school bonus pay)

No strategic staffing rewards Incentives to get best teachers to most challenging schools

Parents typically lack ability to pressure teachers to perform

Parents empowered to demand quality services and informed about school performance (Brazil example)

Alibis for failure: child poverty, lack of parent support, lack of school resources

“If the child is not learning, it’s our fault”Sao Paulo state Secretary of Education

Evidence on Pay for Performance

Motivating

Rigorous global evidence limited and conflicting

Positive impact Rural India, Rural Kenya, Israeli high schools and Chile’s SNED (but effect sizes vary a lot)

No impact US (Nashville), NYC (first year)

Emerging evidence from Brazil

Design features matterSize of bonusCriteria for bonus Coverage or predictability Possible perverse impacts over time (teaching to test, cheating)

What changes in the classroom when incentives are introduced?

Ongoing evaluations in 3 different systems in Brazil, possible scope to launch more

3. Understanding what makes teachers effective in the classroom

having materials in the classroom is not

enough

having the teacher in the classroom is not

enough

education is the transformation of inputsinto learning outcomes

ACTIVITIES

construction curriculum

design trainingtesting

supervision

INPUTS

Funding salaries

buildings books

materials

OUTPUTS

Teacher practice in

the classroom

Inputs

Activities

Results

OUTCOMES

Student learning

Completion rates

…and it happens in the classroom

Tool for analyzing“the black box” of the Classroom

• “Stallings method” classroom observations

• Evidence on teacher practice and classroom dynamics from LAC

• First large-scale, systematic classroom observations in LAC

Ongoing in: Brazil (Pernambuco, MG, Rio municipality)

Starting in: Colombia, Mexico, Jamaica, Guatemala

Planned in: Guyana, El Salvador, Chile

What does it measure?

• Teacher’s use of instructional time• Use of materials, including ICT• Core pedagogical practices• Ability to keep students engaged

Stallings “classroom snapshot” instrument:

How does it work?• 10 observations of each class taken at regular

intervals and coded using a standardized grid


What are some key advantages?

• High reliability - across observers/countries• Benchmarking - Results can be compared to

US/OECD good practice indicators• Simple - Relatively short training needed to produce

qualified observers• Analytical power - Stallings variables are:

• statistically representative of the education system•highly correlated with learning results


OBSERVATION CODING GRID

MATERIAL

ACTIVITY

NOMATERIAL

TEXTBOOK NOTEBOOKBLACKBOARD

LEARNING AIDES ICT COOPERATIVE

1. READING ALOUDT 1 S L

E1 S L E

1 S L E

1 S L E 1 S L E 1 S L

E S L E

I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L

CHECK IF CHORAL READING 2.DEMONSTRATION/ LECTURE

T 1 S L E

1 S L E

1 S L E

1 S L E 1 S L E 1 S L

E S L E


3. DISCUSSION/QUESTIONS AND ANSWERS

T 1 S L E

1 S L E

1 S L E

1 S L E 1 S L E 1 S L

E S L E

I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L4. PRACTICE& DRILL

T 1 S L E

1 S L E

1 S L E

1 S L E 1 S L E 1 S L

E S L E

I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L5. ASSIGNMENT/ CLASS WORK

T 1 S L E

1 S L E

1 S L E

1 S L E 1 S L E 1 S L

E S L E


6. COPYINGT 1 S L

E1 S L E

1 S L E

1 S L E 1 S L E 1 S L

E S L E


7. VERBALINSTRUCTION

T 1 S L E

1 S L E

1 S L E 1 S L E 1 S L

E1 S L E S L E

I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L8. SOCIALINTERACTION

T 1 S L E

I 1 S L9. STUDENT(S)UNINVOLVED

I 1 S L

10. DISCIPLINET 1 S L

E11. CLASSROOMMANAGEMENT

T 1 S L E

I 1 S L

12. CLASSROOM MANAGEMENT ALONE T

13. TEACHER SOCIAL INTERACTION OR TEACHER UNINVOLVED T

14. TEACHER OUT OF THE ROOM T

School No.:Classroom No. Exact time of observation:

Coding grid

FIGURE 1: EXCERPT FROM CLASSROOM SNAPSHOT (APPENDIX 2)

CLASSROOM OBSERVATION SNAPSHOT

MATERIAL

ACTIVITY

NO MATERIAL TEXTBOOK NOTEBOOK BLACK

BOARD LEARNING AIDES ICT COOPERATIVE

1. READING ALOUD T 1 S L E 1 S L E 1 S L E

1 S L E 1 S L E 1 S L E S L E


CHECK IF CHORAL READING

T line: Indicates activities that involve the teacher

I line: Indicates activities that involve the student & not the teacher

1, S, L, E: Indicate one individual, a small, large group and entire class respectively

School and Classroom

Demographic information

OECD Good Practice Benchmarks

Results: Evidence from Brazilian classroomsSix conclusions

#1 Brazil is far from OECD benchmarks

Teachers’ Use of Instructional Time

Rio deJaneiro

City (100 schools)

PernambucoState (300 schools)

Minas Gerais State

OECD Good Practice

Benchmarks

Schools that improved 2005-2007

Schools that declined

2005-2007

Teaching activities 62% 61% 66% 63% 85%

Classroom organization 31% 27% 27% 25% 15%

Non-academicactivities

7% 12% 8% 12% 0%

Teacher out of the classroom

3% 8% 3% 6% -

#2 Improving and Declining Schools Look Different

Teachers’ Use of Time All schools (average) Improving/ Declining Schools

Teacher out of the classroom

4% 3%

6%

Teacher not engagedwith students

3% 2%

3%

Teacher using Q and A 12% 15%

10%

Small or large group of students not engaged

28% 19%

37%

#3 More time on learning is highly correlated with learning outcomes

5th grade

Teacher Use of Time

1. Learning Activities 2. Classroom management 3. Teacher off-task 3.1 Teacherout of room

Total Sample – Rio 58% 37% 6% 1%

10% highest IDEB schools 70% 27% 3% 0%

10% lowest IDEB schools 54% 39% 7% 3%

Difference 0.16 -0.13 -0.03 -0.03

[0,09]* [0,09]* [0,02] [0,01]**

- Erros robustos em parênteses: *Estatisticamente significativo a 10%, **Estatisticamente significativo a 5%, ***Estatisticamente significativo a 1%

- Dados no nível do ano/escola

#4 More time on learning predicted schools’ learning gains and attainment of bonus (PE)

Bônus 2009

Teacher use of Time

1. Learning activities2. Classroommanagement

3. Teacher off-task 3.1 Teacher out of the classroom

Pernambuco State sample 61% 28% 11% 8%

Achieved bonus 2009 62% 27% 10% 8%

Didn’t achieve bonus 2009 53% 30% 17% 12%

Difference 0.09 -0.03 -0.06 -0.04

[0,04]** [0,03] [0,02]*** [0,02]***

Sample of 300 State schools, with 1,800 classrooms observed – math and portuguese classes in 5th, 9th and 12th grades

Standard errors in parenthesis. * Statistically significant at the 10% level, **Statistically significant at the 5% level, ***Statistically significant at the 1% level

# 5 Tool for identifying schools with highly positive (and negative) performance

Extreme situations

% of classrooms observed

Rio de Janeiro

PE

MG pilot

Improvingschools

Decliningschools

Teacher absent from the classroom more than 50% of total class time 0% 2% 0% 0%

Teacher absent from classroom more than 25% of total class time 1% 10% 3% 3%

Teacher spent over 50% of class time using no materials 4% 10% 17% 31%

Teacher spent more than 50% of class time writing on black board 24% 32% 5% 9%

Large group of students not engaged during more than 50% of class time 9% 7% 9% 19%

#6 Teacher practice varies a lot…with huge implications for learning

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 0.5 1 1.5 2 2.5 3 3.5

70 % total schools

TIME USE IN THE CLASSROOMRio de Janeiro schools, June 2010

Learning activities

Classroom management

Teacher off task

On average, only 61% of class time is used for instruction

But some schools average 81%

Others average 37%

88 more days of instruction per school year

TIME USE IN THE CLASSROOMRio de Janeiro schools, June 2010

Learning activities

Classroom management

Teacher off task

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 0.5 1 1.5 2 2.5 3 3.5

Across different classrooms in the same school

•70% of time on learning in best class• 52% in worst class

36 more days of instruction per year in the most efficient classrooms

#6 Teacher practice varies a lot…even within schools

Reactions in Brazil From Education secretaries, district supervisors and school pedagogical coordinators…

It allowed me to observe better the dynamics of class, because in my work environment, observations do happen, but not too closely or with such details

This type of classroom observation is directly relevant to my daily work...(PE)

We know this intuitively, but this is the first time we have hard data we can take back to our regional offices and schools to discuss and work on. Secretary of Education (MG)

I was surpised to see...very traditional

teacher practice

Even in different schools and municipalities the teacher practices are the same

Teachers work in a very traditional way: exercises and corrections

Despite the intense technological progress, we still give priority to old ways of teaching-learning

The textbooks that come via MEC are rarely used in the classroom; the teachers prefer to write on the blackboard

the opportunities for Caribbean countries and Latin America

Conclusions• Virtually all Caribbean and LA countries trying to

raise teacher quality• Innovative approaches, universal issues • Little hard evidence on impact or cost-

effectiveness – in OECS, OECD or LAC • Great opportunity to generate knowledge that

will benefit OECS and others• Combining test data with classroom observations

creates a platform for solid evidence on how to “Build Better Teachers”

Thank you! Barbara Bruns

[email protected]

Building Better Teachers in the Caribbean - World...

Documents

Transcript of Building Better Teachers in the Caribbean - World...