What Are We Learning About Evaluating Teachers More Effectively? A District’s

Perspective

2012 Southern Regional Education Board Meetingand 61st Legislative Work Conference

 June 24, 2012

Williamsburg, Virginia

Andy Baxter Director, Human Capital Strategies

Charlotte-Mecklenburg Schools

2012 Southern Regional Education Board Meeting

2

Start with students.

Section 1

6/24/2012

From the first day,

we chart our children’sgrowth.

<picture: first day of kindergarten>

We can’t stop on the

first day of school.

Honey, how was school?

Fine.

I better call the teacher.

?What will my child learn in

this class?

How will you know if my child has mastered

the material?

Is my child keeping up with other children?

Is my child being

challenged to their

potential?

What can I do to help?

Each year our students enroll in close to 2,000

classes.We can measure what they’ve learned in 10% of

them.

We measure what we care about.

We must develop multiple ways to

assess what our students

learn.

We can track every child’s development.

K 1 2 3 4 5 6 7 8 9 10 11 12

Math

Critica

l

Thinkin

g

The

Arts

Reading

Problem-

Solving

Advanced

Beginning

Every grade. Every subject.

2012 Southern Regional Education Board Meeting

11

Support their teachers.

Section 2

6/24/2012

2012 Southern Regional Education Board Meeting

126/24/2012

2012 Southern Regional Education Board Meeting

136/24/2012

2012 Southern Regional Education Board Meeting

146/24/2012

2012 Southern Regional Education Board Meeting

15

But teachers differ in impact on students.

6/24/2012

2007 2008 2009 2010-100

-80

-60

-40

-20

0

20

40

60

80

-1

40

6067

-34

-60-69

-75

Average Days of Instruction in Math (Grades 4-8) for Students who had either Top or Bottom Third Teachers

Three Years in a Row

Top Third TeachersBottom Third Teachers

2012 Southern Regional Education Board Meeting

16

This difference matters for students later in life.

For one year, difference for a student between having a 50th and 84th percentile teacher.

Difference in lifetime earnings is equivalent to giving each student $4,600. Raises probability of attending college at age 20 by 0.5 percentage points.Decreases probability of teenage pregnancy.

http://obs.rc.fas.harvard.edu/chetty/value_added.html

6/24/2012

2012 Southern Regional Education Board Meeting

17

We have not differentiated feedback to teachers.

6/24/2012

6%

Current Teacher Evaluation

Satisfactory

94%

Hey Boss,Can you give me any feed-back?

You’re fine.

2012 Southern Regional Education Board Meeting

18

A new paradigm for human capital

Section 3

6/24/2012

2012 Southern Regional Education Board Meeting

19

Targeted recruiting

6/24/2012

Value-Added of Teachers by Undergraduate Institution First 5 Years of Teaching

Notes: 4th-8th grade math and reading teachers with five or fewer years of experience between 1998-99 to 2008-09

-.1

-.05

0.0

5R

eadi

ng V

AM

-.1 -.05 0 .05 .1 .15Math VAM

Institutions marked with a red dot have VA in math or reading significantly different from zero.

2012 Southern Regional Education Board Meeting

20

Improving access to top teachers

6/24/2012

2012 Southern Regional Education Board Meeting

21

Development0

510

1520

Val

ue-A

dded

(Day

s of In

stru

ctio

n)

0 1 2 3 4 5 6 7 8 9-25 26+Years of Experience

Math Value-Added 95% Confidence Intervalcreated 20100526 at 15:30:48 using on

MathReturns to Experience

6/24/2012

22

Feedback

6/24/20122012 Southern Regional Education Board Meeting

How am I doing over time?

How am I doing with different

types of students?

23

In the past, all year-end appraisal.

46%3%

1% 50%

-90

-45

0

45

90

135

Tea

cher

Eff

ective

ness

(Day

s Val

ue-A

dded

+/- 1

80)

0 1 2 3 4

Evaluation Score

Correlation of value-added estimate and evaluation rating is .24. Scores of 3 or higher on evaluation indicate that the teacher performed at or above standard.Value-Added estimates above 0 indicate above-average effectiveness.

CMS Teachers (2010), MathCorrelation of Evaluations to Value-Added Teacher Estimates

6/24/20122012 Southern Regional Education Board Meeting

24

Classroom Observatio

nsValue-Added

(Individual,

Team,School) Contributions

to Professional

Learning Communities

Student Feedback

Content Pedagogy

Student Work

Now, continuous feedback.

25

Purposeful retention

51 6580

30 19 31

31 3265

-80-60-40-20

020406080

100

-80-60-40-20

020406080

100

2008 2009 2010

2008 2009 2010

Teacher 1 Teacher 2

Teacher 3

Tea

cher

Eff

ect

(+/- 1

80 D

ays In

stru

ctio

n b

y Ave

rage

Tea

cher

)

Year (Spring)District percentile across all subjects taught by teacher label each point .

Middle School CTrajectory of Effectiveness for Teachers Just Granted Career Status

6/24/20122012 Southern Regional Education Board Meeting

26

Actual

CMS DataRandom

Data

Percentile of Teacher Value-added Score

Compensation

6/24/20122012 Southern Regional Education Board Meeting

2012 Southern Regional Education Board Meeting

27

We have learned the hard way.

Section 4

6/24/2012

28

They are measuring my performance.

6/24/20122012 Southern Regional Education Board Meeting

2012 Southern Regional Education Board Meeting

29

I don’t understand how they get this number.

6/24/2012

30

What am I aiming for?

6/24/20122012 Southern Regional Education Board Meeting

Top 1/3Middle 1/3Bottom 1/3

2012 Southern Regional Education Board Meeting

31

Everyone seems to hate tests, at least part of the time.

6/24/2012

2012 Southern Regional Education Board Meeting

32

Dancing with the State to the beat of Race to the Top

6/24/2012

2012 Southern Regional Education Board Meeting

33

Remember the status quo

1. In what ways, and to what extent, might principal observations be biased?

2. How reliable are principal observations? (i.e., inter-rater reliability?)

3. How connected are observation results to student achievement results?

6/24/2012

2012 Southern Regional Education Board Meeting

34

Teachers want to be a part of the change

1.

6/24/2012

We asked 9 teams of teachers:1.Does this measure of teaching

effectiveness matter?2.If so, how do we measure it?3.Who should measure it?

They answered with:• 3,912 person-hours of volunteer

engagement over 18 months• Thoughtful policy recommendations based

on research and their experience

2012 Southern Regional Education Board Meeting

35

Conclusion

• The work is will help students and teachers.

• It is scary.• It is not easy.• We will make mistakes.• And, we can do it.

6/24/2012

2012 Southern Regional Education Board Meeting

36

The graduation slides with the chairs

6/24/2012

2012 Southern Regional Education Board Meeting

376/24/2012

38

Contact Information

Andy BaxterDirector, Human Capital StrategiesCharlotte-Mecklenburg Schools980-343-1603andrew.baxter@cms.k12.nc.us

6/24/20122012 Southern Regional Education Board Meeting