CLASSROOM-OBSERVATION EVIDENCE IN TEACHER EVALUATION: PLAYING THE PONIES

W. James Popham University of California, Los Angeles

National Conference on Student Assessment

Council of Chief State School Officers

New Orleans, Louisiana June 25-27, 2014

THE PRESENTING QUESTION:

For purposes of summative teacher evaluation, how much weight should be given to evidence obtained through classroom observations?

WHY USE OBSERVATION EVIDENCE AT ALL?

We now know, with considerable confidence, if particular events transpire in classrooms, odds are that students will learn better.

WHY USE OBSERVATION EVIDENCE AT ALL?

We now know, with considerable confidence, if particular events transpire in classrooms, odds are that students will learn better.

A CORRELATIONAL CERTAINTY:

Hefty correlations between teachers’ classroom acts (or teacher-directed acts of students) and students’ assessed achievement are not perfect correlations. (And besides, whoever said an r of even .60 is hefty?)

A PROFOUND TEACHER-EVALUATION PRECEPT:

When engaged in teacher evaluation, we are not focused on “teachers in general” but, rather, on determining the quality of an individual teacher whose professional success and job security are hanging in the balance.

THE CATALYSTS FOR TODAY’S TEACHER-EVALUATION CONCERNS:

Two incentive-linked federal initiatives calling for state teacher-evaluation programs to use multiple sources of evaluative evidence, one of which must be student growth. Moreover, this student-growth evidence must play a “significant” role in informing “personnel decisions.”

(A reminder for those of scant recollective powers:)

THE PRESENTING QUESTION:

For purposes of summative teacher evaluation, how much weight should be given to evidence obtained through classroom observations?

WEIGHTING CLASSROOM-OBSERVATION EVIDENCE IN SUMMATIVE TEACHER EVALUATION

• Factor 1: The quality of the observation evidence itself.

• Factor 2: The quality of the test(s) being used to ascertain student growth.

CLASSROOM-OBSERVATION EVIDENCE QUALITY

• Adequate numbers of observations made. • Typical, not atypical classroom events observed. • Usage-refined, research-based forms employed. • A manageable number of variables observed. • Observers have been trained so both their accuracy

and their inter-observer agreement are adequate.

QUALITY OF TESTS GAUGING STUDENT GROWTH

• Quality-Determining Criterion One: The degree to which the test(s) being used are accompanied by evidence attesting to their instructional sensitivity

• Quality-Determining Criterion Two: The quality of the curricular outcomes being assessed by the tests.

A WRAP-UP QUERY FOR YOUR CONSIDERATION:

Because classroom-observation evidence of a teacher’s skill is, at best, a probability-swathed predictor of students’ actual learning, shouldn’t we be trying harder to measure students’ achievement properly rather than refining what is only a surrogate for student learning?