Teacher Effectiveness Initiative Value-Added Training

TEACHER EFFECTIVENESS INITIATIVEVALUE-ADDED TRAINING

Value-Added Research Center (VARC)October 2012

MinneapolisMilwaukee

Racine

Chicago

Madison

Tulsa

Atlanta

New York City

Los Angeles

Hillsborough County

NORTH DAKOTA

SOUTH DAKOTA

MINNESOTA

WISCONSIN

ILLINOIS

Districts and States Working with VARC

Collier County

NEW YORK

The Power of Two

&A more

complete picture of student learning

Achievement Value-AddedCompares students’

performance to a standard

Does not factor in students’ background characteristics

Measures students’ performance at a single

point in time

Critical to students’ post-secondary opportunities

Measures students’ individual academic growth longitudinally

Factors in students’ background characteristics

outside of the school’s control

Critical to ensuring students’ future academic success

Measures the impact of teachers and schools on

academic growth

Adapted from materials created by Battelle for Kids

How does Value-Added fit into the Teacher Effectiveness Initiative?

Overview

Where does VARC get data?

VARC

State Department

s of Education

(Student Test Scores,

Demographics, Enrollment;

Teacher Licensure and Assignments)

Institutions of Higher

Education(Graduate

Information)Pilot

Districts(student-teacher linkages, non-NCLB

test scores)

Where does VARC usually send School-Level and Grade-Level output?

District

Schools

IHE?If

District Shares

Teachers

VARC

Where will VARC send Teacher-Level output?

VARC

Teacher

IHE(Graduates Only)

Principals?

Districts?

How will this inform your IHE’s Program Improvement?

Teacher Education Program

Improvement

Graduate Value-Added

IHE-Level Aggregate

Value-Added

Surveys

Observational

Data

varc.wceruw.org/welcome

The Oak Tree Analogy

The Oak Tree Analogy Review

This method is analogous to using an Achievement Model.

Gardener A Gardener B

61 in.72 in.

Method 1: Measure the Height of the Trees Today (One Year After the Gardeners Began) Using this method, Gardener B is the more effective gardener.

This is analogous to a Simple Growth Model, also called Gain.

61 in.

72 in.Gardener A Gardener B

Oak AAge 4

(Today)

Oak BAge 4

(Today)

Oak AAge 3

(1 year ago)

Oak BAge 3

(1 year ago)

47 in. 52 in.+14 in. +20 in.

Method 2: Compare Starting Height to Ending Height Oak B had more growth this year, so Gardener B is the more

effective gardener.

+20 Average+20 Average

+ 3 for Rainfall

- 3 for Soil + 2 for Soil

- 8 for Temp + 5 for Temp_________+12 inchesDuring the year

_________+22 inches During the year

59 in.

74 in.Gardener A Gardener B

47 in. 52 in.

- 5 for Rainfall

Controlling for Non-Gardener Factors The predicted height for trees in Oak A’s conditions is 59 inches. The predicted height for trees in Oak B’s conditions is 74 inches.

PredictedOak A

PredictedOak B

This is analogous to a Value-Added measure.

Above Average

Value-Added

Below Average

Value-Added

PredictedOak A

PredictedOak B

ActualOak A

ActualOak B

59 in.

74 in.Gardener A Gardener B61 in.

72 in.+2-2

Method 3: Compare the Predicted Height to the Actual Height By accounting for last year’s height and environmental conditions of the trees during

this year, we found the “value” each gardener “added” to the growth of the trees.

Sample Report Review

Color Coding

Explanation

Table of Contents

Reporting Period and

Context

School-Level Value-Added

Estimates

Grade-Level Value-Added

Estimates

TopSchool-Level Value-Added

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)

VALUE-ADDED ESTIMATES VALUE-ADDED ESTIMATES

Past Academic Year 2010-2011 Up-To-3-Year Average

1 2 3 4 5 1 2 3 4 5

READING

MATH


182.9

182.9

559.4

559.4Subject

Level of Analysis

Value-Added Estimate• Point Estimate (number in

color-coded bubble)• 95% Confidence Interval (black

line)

Number of students included

in the analysis

2.5 2.4

1.6 1.7

Past Academic Year

Up-To-3-Year Average

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

READING

MATH


58.7

68.3

171.9

187.5

55.9 200.1


58.7

68.3

171.9

187.5

55.9 200.1

Page 2 BottomGrade-Level Value-Added

Grade 3

Grade 4

Grade 5

Grade 3

Grade 4

Grade 5

FAQ 1:Which school year is

this?

2.1 1.9

3.3 4.3

2.6 2.1

0.71.1

1.6 1.8

3.8 4.1

Value-Added on the NDSA

Grade 3 Summer Grade 4 Summ

er Grade 5 Summer Grade 6

Nov Nov Nov Nov

3rd GradeValue-Added

4th GradeValue-Added

5th GradeValue-Added

4th grade example: “Starting knowledge” is the November 2010 4th grade

test. “Ending knowledge” is the November 2011 5th grade

test. This aligns to growth in the 2010-2011 4th grade school

year. Why don’t we have 8th grade Value-Added in

North Dakota?

BottomGrade-Level Value-Added

FAQ 2:How do I interpret the

“Up-To-3-Year Average”?

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

READING

MATH


58.7

68.3

171.9

187.5

55.9 200.1


58.7

68.3

171.9

187.5

55.9 200.1

Grade 3

Grade 4

Grade 5

Grade 3

Grade 4

Grade 5

2.1 1.9

3.3 4.3

2.6 2.1

0.71.1

1.6 1.8

3.8 4.1

NOT Jimmy as he goes through three consecutive school years 3rd grade to 4th

grade 4th grade to 5th

grade 5th grade to 6th

grade

3rd grade team with 2008-2009 cohort

(3rd grade to 4th grade) 2009-2010 cohort

(3rd grade to 4th grade) 2010-2011 cohort

(3rd grade to 4th grade) Keep teacher mobility

in mind

Does not follow individual students for 3 years

Represents the 3rd grade teaching team over three cohorts of students

What Does “Up-To-3-Year Average” Mean for the 3rd Grade?

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

READING Grade-Level Value-Added

20

20

60

60

20 60

Grade 3

Grade 4

Grade 5

What Does “Up-To-3-Year Average” Mean?

2010-2011 3rd Graders

The “Past Academic Year” represents longitudinal growth over a single school year.

10-11 3rd Gr.

09-10 3rd Gr.

08-09 3rd Gr.

2010-2011 4th Graders

2010-2011 5th Graders

10-11 4th Gr.10-11 5th Gr.

09-10 4th Gr.09-10 5th Gr.

08-09 4th Gr.08-09 5th Gr.

The “Up-To-3-Year Average” represents average longitudinal growth of three different groups of students at each grade level.

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

READING Grade-Level Value-Added

What Does “Up-To-3-Year Average” Mean?

Which grade-level teaching team… Was most effective in the 2010-2011 school year? Was most effective over the past three school years? Was more effective in 2010-2011 than in the past?

48.5

44.5

146.0

141.1

46.0 147.8

Grade 3

Grade 4

Grade 5

3.4

4.1

4.4

3.5

0.9

2.8

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

READING

MATH


58.7

68.3

171.9

187.5

55.9 200.1


58.7

68.3

171.9

187.5

55.9 200.1

Grade 3

Grade 4

Grade 5

Grade 3

Grade 4

Grade 5

2.1 1.9

3.3 4.3

2.6 2.1

0.71.1

1.6 1.8

3.8 4.1

FAQ 3:Does this show student growth

to go from red to yellow to green over time?

Page 2 BottomGrade-Level Value-Added

Value-Added, Not Achievement

Grade 3 61

READING

Grade 4 63

Grade 5 60

3

Grade 3 61

MATH

Grade 4 63

Grade 5 60

3 In your groups: Describe this

school’s math performance

Describe this school’s reading performance

3.8

3.9

3.9

4.8

3.0

1.1

BottomGrade-Level Value-Added

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

READING

MATH


58.7

68.3

171.9

187.5

55.9 200.1


58.7

68.3

171.9

187.5

55.9 200.1

Grade 3

Grade 4

Grade 5

Grade 3

Grade 4

Grade 5

2.1 1.9

3.3 4.3

2.6 2.1

0.71.1

1.6 1.8

3.8 4.1

FAQ 4:Why are there non-integer numbers of

students?

Mobile Students If a student is enrolled in more than one school

between the November NDSA administration and the end of the school year, each school gets credit for a portion of the student’s growth.

Grade 3

Nov NDSA

School A School B

45% Attributed to B

55% Attributed to A

End of School Year

Student Group Interpretation

Pages 3 & 4

School-Level Value-Added Estimates by

Student Groups:

Special EdLow Income

GenderLEP

Example Student Group Interpretation

At this school which student group is growing faster than their similar peers from across the state?

Does that mean the “Special Ed” group grew more scale score points on the test than “Not Special Ed” group?

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

READING By Special Ed

22.0

160.9

72.5

486.9

Special Ed

Not Special Ed

3.9

2.3

3.6

2.2

Student Group InterpretationWhat Would You Do?

What do these results mean? If this was your school, how could you use these

results to monitor instructional improvement?

NUMBER OFSTUDENTS

(WEIGHTED)

NUMBER OFSTUDENTS

(WEIGHTED)



1 2 3 4 5 1 2 3 4 5

By LEP

52.0

130.9

153.0

406.9

LEP

Not LEP

0.8

2.2

0.9

2.4

MATH


and Achievement

Scatter Plot Interpretation

Scatter Plots

How to Read the Scatter Plots

1 2 3 540

20

40

60

80

100

Value-Added (2010-2011)

Perc

ent P

rof/A

dv (2

010)

These scatter plots are a way to represent Achievement and Value-Added together

Achi

evem

ent

Value-Added

How to Read the Scatter Plots

1 2 3 540

20

40

60

80

100

Value-Added (2010-2011)

Perc

ent P

rof/A

dv (2

010)

Schools in your district

A

A. Students know a lot and are growing faster than predicted

B

B. Students are behind, but are growing faster than predicted

C

C. Students know a lot, but are growing slower than predicted

D

D. Students are behind, and are growing slower than predicted

E

E. Students are about average in how much they know and how fast they are growing

(or 6 & 7 for large grade span schools)


and Achievement

Example (Grade 4)

Last Pages (Part 1)

1-5 Value-Added

Scale

Note Regarding

Comparison of Student

Groups

Number of Students

(Weighted)

Last Pages (Part 2)List of Control

Variables

Note Regarding the Bush

Foundation

List of Reasons for

Result Suppression

Teacher Effectiveness Initiative Value-Added Training

Documents

Transcript of Teacher Effectiveness Initiative Value-Added Training