Post on 13-Jan-2016
Project on Educator Effectiveness & Quality
Chancellor SummitSeptember 27, 2011
Cynthia Osborne, Ph.D.
Accountability System for Educator Preparation Programs (SB 174)
Standard 1: Certification Exam Passing Rates
Standard 2: Principal Survey of 1st Year Teacher; Graduates’ Survey of Preparedness
Standard 3: Teachers’ Influence on Growth in Student Achievement in First Three Years Post Certification
Standard 4: Quality of the Field Supervision
September 27, 2011 2
Purpose of PEEQ
Tasked by TEA to design a metric to determine EPP’s graduates’ influences on student achievement
Metric is required due to:o SB 174 student achievement standard (#3)o State longitudinal data system grant (SLDS II)o School improvement grant (SIG)o State fiscal sustainability fund (SFSF)
Will focus only on 1st through 3rd year teachers Objective is to provide feedback to EPPs to improve teaching and
student performance – not just for accountability
September 27, 2011 3
SB 174 and PEEQ
September 27, 2011 4
SB 174 Standard III
Student Performance Growth (VAM)
Teacher Observation
Principal Assessment
Teacher's Impact
School or Grade-Level Performance
Standard IV
Standard III
Standard II
Standard I
Please note that the weights of SB174 and the weights of Standard III have not been determined.
What Predicts Effective Teaching?
Years of experience: teachers show 2/3 of all improvement in first 5 years
General intelligence: selectivity of the college, college entrance exams, teacher licensing exams, and IQ tests
Certification in subject area (especially math): masters degree is not predictive
Rigorous observations of teachers in classroom Most variation in teacher effectiveness is not explained by
“teacher inputs” – focus has shifted to “student outcomes”
September 27, 2011 5
Measuring Teacher Effectiveness in Texas
Our current system, PDAS, does not provide information to the state or EPPs on effective teachers
Over 98% of teachers are rated proficient, thus EPPs have no indicators to improve programming
NCLB standards (highly effective) are inconsistent with PDAS findings; students are not all proficient
Goal is to improve the quality of the pipeline of new teachers in the state by providing EPPs with information on how their graduates are influencing student achievement
September 27, 2011
Measuring Teacher Effectiveness Across the Nation
States and districts are increasingly using student performance to measure teacher effectiveness
Have moved from a system of measuring “teacher inputs” to measuring “student outcomes”
31 states have initiated plans
States are moving quickly from plan to implementation; LEAs move more slowly
Wide variety exists in components measured and instruments used
Few states use teacher effectiveness for EPP accreditation – although momentum is gaining
September 27, 2011
An Effective Metric Should Be:
Comprehensive: Includes multiple sources; not just test scores Useful: Leads to improvement in EPPs and teaching Valid & Reliable: Reflects student achievement and considers
variations in populations of students; Problems with measurement error, test scaling, and data quality must be minimized
Integrated: EPPs held accountable for training teachers to do what teachers will be held accountable for doing
Transparent: Clear how components are measured and weighted
September 27, 2011 8
Decisions PEEQ Must Make
What components should be included in a comprehensive metric to determine a teacher’s effect on student achievement? How should we weight the components?
What is the most valid and reliable method to measure student performance using test scores?
September 27, 2011 9
A Comprehensive Metric Should Include:
Growth in Student Performance on TAKS (VAM) Observations of Teachers in Classroom
o Classroom Environmento Curriculum and Instruction
Principals’ Assessment of Influence on Achievement Teacher’s Impact on Student Achievement School or Grade Level Growth in Performance
September 27, 2011 10
Weighting of Components Will Be Determined Empirically
September 27, 2011 11
30%
20%
50%
TAP Model
Student Performance (Value-Added Measure)
School Performance (Value-Added Measure)
Skills, Knowledge, & Responsibilities
50%
40%
5% 5%
IMPACT - DCStudent Performance (Value-Added Measure)
Observations
School Value-Added
Commitment to Community
50%50%
LouisianaStudent Per-formance (Value-Added Measure)
TBD
35%
35%
15%
15%
MemphisStudent Performance (Value-Added Measure)
Observations
Stakeholder Perceptions
Content Knowledge
40%
30%
30%
Hillsborough
Student Performance (Value-Added Measure)
Principal Observation
Mentor/Peer Teacher Observation
PEEQ’s Metric
September 27, 2011 12
Student Performance Growth (VAM)
Teacher Observation
Principal Assessment
Teacher's Impact
School or Grade-Level Performance
Please note that the weights have not been determined.
Measuring Student Performance of EPPs’ Graduates
Common approacheso Percent of students passing standardized testo Percent increase in students passing test
Strengthso Straight forward and transparento Common standard and expectation of knowledge
Limitationso Does not account for a student’s prior knowledge or growtho May lead to focus on teaching to test
September 27, 2011 13
Measuring Student Performance of EPPs’ Graduates
Value Added Modelso Goal is to predict what a student should score and compare to
actual performance – “teacher effectiveness” is the differenceo Various methods used to calculateo Require a large enough sample to create comparisonso Need sufficient data to accurately predict student performanceo Attribution of students to teachers is crucialo Tests must be vertically aligned to show growtho PEEQ will use a VAM, but not at individual teacher level, yet
September 27, 2011 14
Measuring Student Performance of EPPs’ Graduates
Value Added Models (Continued) Strengths
o Account for past performance of students and student - and school-level characteristics that may affect teaching and student performance
o Emphasis is on growth in student performanceo Teachers’ past effectiveness is highly predictive of future
Limitationso Difficult to understand calculations (less transparent)o Measurement error may bias estimates
September 27, 2011 15
Observations of Teachers
Must be mindful of the scale:o 75,000 beginning teachers (in first 3 years)o 1,400 school districts
Principals are required to observe beginning teachers and fill out a survey on each teacher (ASEP)
This survey provides the platform to gather information on teachers’ classroom environment and instruction (elements linked to student achievement)
September 27, 2011 16
Classroom EnvironmentTo what extent is this beginning teacher able to . . . (4 pt scale)
Effectively implement discipline-management procedures? Communicate clear expectations for achievement and behavior
that promote and encourage self-discipline and self-directed learning?
Provide support to achieve a positive, equitable, and engaging learning environment?
Build and maintain positive rapport with students? Build and maintain positive rapport and two-way communication
with students’ families?
September 27, 2011 17
InstructionTo what extent is this beginning teacher able to . . . (4 pt scale)
implement varied instruction that integrates critical thinking, inquiry, and problem solving?
respond to the needs of students by being flexible in instructional approach and differentiating instruction?
use the results of formative assessment data to guide instruction? engage and motivate students through learner-centered instruction? integrate effective modeling, questioning, and self-reflection (self-assessment)
strategies into instruction? assume various roles in the instructional process (e.g. instructor, facilitator,
audience)? set clear learning goals and align instruction with standards-based content? provide quality and timely feedback to students?
September 27, 2011 18
Principal Assessment
“How would you rate this teacher’s influence on student achievement?”
10 point scale: unacceptable to exceptional
September 27, 2011 19
Information to EPPs for Standard #3
EPPs will get a total score, and disaggregated information on graduates
Information to EPPs will include:o Overall effectivenesso Aggregate scores by each metric componento Scores by:• Certification program• Type/location of school• Grade level/subject matter
September 27, 2011 20
Feedback from Districts/Teachers
Want better indicators of effectiveness; have concerns about the uses of the metric
Claim that quality of field supervision distinguishes EPPs
Encourage a stronger dialog and more interaction between districts and EPPs to align teaching and practice
September 27, 2011 21
Feedback from EPPs
Deans want better indicators of effectiveness of graduates
Concerned about variation in support teachers receive across districts
Concerned that information has not previously been available; would like information for two years prior to accountability
No consensus on what new teachers should know relative to experienced teachers
September 27, 2011 22
We want your input!
We have a statistical advisory group and invite each EPP to designate a member.
We conduct quarterly stakeholder meetings and invite all stakeholders to participate.
We update our website regularly with new research briefs: www.utexas.edu/lbj/peeq
September 27, 2011 23
Expected Timeline
Principal Survey (Standard #2) administered in summer 2011
Data linking teachers and students available fall 2011
Pilot Metric available spring 2012
Pilot Metric presented to SBEC June 2012
September 27, 2011 24