Ceppe march 2011 final (Susanna Loeb)
-
Upload
ceppe-chile -
Category
Documents
-
view
675 -
download
3
Transcript of Ceppe march 2011 final (Susanna Loeb)
Teacher Trajectories and Teacher Effectiveness: the role of policy and leadership
Susanna LoebCEPPEMarch 11, 2011
A Collaborative Effort
• Don Boyd, Hamp Lankford & Jim Wyckoff – Teacher workforce
• Pam Grossman– Teaching
• Jason Grissom, Eileen Horng, Demetra Kalogrides– School Leadership
Problems and Contexts
Generally low student
outcomes
Systematic inequality in
student outcomes
Importance of teachers for
students’ opportunities
Great differences in
teacher effectiveness
Systematic differences in
teachers across communities
Little knowledge of how to improve
effectiveness
Knowledge and Abilities of Teachers in the Classroom
is a Function of…
Recruitment and Selection
Development and Supports
Assignment RetentionHow can
we improve each of these?
In the US, Differences and Similarities
Recruitment and Selection
Development and Supports
Assignment Retention
Similarly, little direct recruitment or selection until recently
Similarly, substantially differences in teachers across schools (also within)
Similarly, little differentiation of the teaching career
Preparation more similar to the 4+1 model for high schools – state policy
More hours for PD, with recent focus on learning communities
Much bigger problem. (e.g. in NYC about 18% of teachers leave their school each year).
Low agreement / knowledge on how to select
Low agreement / knowledge on support
hypothesis: improvement rests on…
WHO MAKES DECISIONS?
central government
local government
schools
teachers
families / communities
for example…
WHO CAN TEACH? many
options
Central government
selects
Central government sets requirements, local governments
select
Central government set requirements and schools
select
Schools select
Central government sets requirements,
teacher prep programs select, then schools
select
what is
best ?
In order to make good decisions, the decision
makers need…
TO MAKE GOOD DECISIONS
incentives
ability
information
So considering our decision makers…
WHO MAKE
S DECISIONS?
central government
local government
schoolsteachers
families / communities
Often little information to
make good decisions
Often better information (still imperfect) but little
ability and often unaligned incentives
Key is to improve incentives and abilities
locally so they make better decisions
But also to improve overall knowledge since the knowledge base in teaching is weak
May be missing different information and abilities
rest of today: how incentives, information and
abilities can improve..
Recruitment and Selection
Development and Supports
Assignment Retention
3 Examples Incentives
influenced by regulations
Information
Ability of local decision makers
1. Federal Regulations Affecting Recruitment and Selection Incentives
considering the past decade in NYC
THE STARTING PLACE: YEAR 2000, NY
Unequal Distributionteacher attributes by student test scores
(New York State, in 2000)
Little Enforcement of State Policy
Why did we see this distribution?
• Teacher Preferences• High achieving students• White students (at least for white teachers, majority)• Location - Close to home
• Little State Enforcement
• Little Direct Recruitment• Teacher education programs(little outreach, rarely selective)• Difficult-to-staff districts recruited certified teachers, little school-
level control, late hiring (July and August), often relied on waivers
• Leadership labor market subject to same forces, which in turn affect teachers
A BIT OF CHANGE
Innovation in Direct Recruitment (prior to certification)
• Teach for America - 1990
• The New Teacher Project – 1997
• UTeach
Change in Accountability
• State requirements no longer based solely on compliance (inputs)
• Standardized tests used for accountability instead of just to inform parents or for student opportunities (e.g NY regents)
• Started in the states but moved to Federal with NCLB – increased emphasis on subgroups (and teacher quality).
Highly Qualified Teacher Provision
• Intent - all students have a good teachers and equity in teacher quality
• State flexibility
• Weak research base for defining HQ – use professional consensus & State policy process
• defines HQT as: (1) fully State-certified; (2) holding a bachelor’s degree; (3) demonstrating competency in the core academic subject or subjects he or she teaches – multiple options
Highly Qualified Teacher• All teachers have the option of passing a
state exam.
• Middle and secondary teachers may complete an undergraduate or graduate degree in their field or advanced certification or credential.
• All veteran teachers also have the option of completing a High Objective Uniform State Standard of Evaluation (HOUSSE)
IMPLICATIONS FOR TEACHERSa dramatic change in the workforce
10%
15%
20%
25%
30%
35%
2000 2001 2002 2003 2004 2005
% o
f n
ew
te
ac
he
rs
Lowest quartile 2nd quartile 3rd quartile Highest quartile
Certification Exam Failure Rate of New Elementary
Teachers, 2000-’05 High-poverty schools
Low-poverty schools
10%
15%
20%
25%
30%
35%
2000 2001 2002 2003 2004 2005
% o
f n
ew
te
ac
he
rs
Lowest quartile 2nd quartile 3rd quartile Highest quartile
Certification Exam Failure Rate of New Elementary
Teachers, 2000-’05 High-poverty schools
Low-poverty schools
Certification Exam Failure Rate of New Elementary Teachers, 2000-’05
10%
15%
20%
25%
30%
35%
40%
2000 2001 2002 2003 2004 2005
Per
cen
t o
f te
ach
ers
Low est quartile 2nd quartile 3rd quartile Highest quartile
High-poverty schools
Low-poverty schools
Effect of changes in teacher characteristics on student math
scores
25% decrease in the predicted gap
Conclusion 1
• A small change in regulation can create a large change in incentives
• Incentives can inspire meaningful change in schools and for students
2. Improvements to a Weak Knowledge Base
linking teacher preparation to effectiveness to improve information
Current Knowledge Base
• Virtually no research linking preparation experiences to teachers’ classroom performance and, particularly to student outcomes
• Lack of information hinders simple decisions about certification policy at the state level as well as program design locally
Approach to Understanding Prep:
first explore
Program FeaturesTeachers’ Reported
ExperiencesStudent test Performance
a first try
program features spring/summer 2004
• 18 institutions: 26 univ. programs + 4 TF + 1 TFA
• Document collection, interviews with director and director of field experiences, surveys of math and ELA methods faculty
• Many many measures but here focus mainly on link to practice. Program data are not ideal for this.– whether or not the program required a capstone project – a composite measure of the extent that the program maintains
oversight over student teaching experiences• requires a minimum number of years of teaching experience for its
cooperating teachers (32%)• program picks the cooperating teacher (42%) • program supervisor observes their participants a minimum of five
times during student teaching (27%)– for comparison math and English content course requirements and
the percent of tenure-line.
reported experiences: survey spring 2005• All first year teachers in NYC
• Practice Variables– extent to which preparation included links to practice;
• opportunities to listen to an individual child read aloud for the purpose of assessing his/her reading achievement;
• …Plan a guided reading lesson, and • …Study or analyze student math work (each 5-point scale).
– opportunities to study curriculum used in New York City – whether or not the teacher had student teaching
experiences, not as the teacher-of-record in the classroom;
– the congruence between student teaching placement and their current job assignment in terms of subject matter or grade level;
• Additional measures of preparation as controls and comparisons– opportunities to learn about teaching math and ELA; learning;
handling student misbehavior; and teaching English language learners.
opportunities to learn math methods
• alpha = 0.97• In your teacher preparation program, prior to September
2004, how much opportunity did you have to do the following (5-point scale)? – learn typical difficulties students have with place value; – learn typical difficulties students have with fractions; – use representations (e.g., geometric representation, graphs, number
lines) to show explicitly why a procedure works; – prove that a solution is valid or that a method works for all similar
cases; – study, critique, or adapt math curriculum materials; – study or analyze student math work; – design math lessons; – learn how to facilitate math learning for students in small groups; – adapt math lessons for students with diverse needs and learning
styles; – And practice what you learned about teaching math in your teacher
preparation program in your field experience.
first year results: program features
* *
1st year results: reported experiences
Conclusion 2• Evidence the aspects of preparation can
matter• Well-supervised practice• Content course (effects in the second year)• Preparation for teaching English Learners
• Surprisingly little research on preparation (little systematic identification of best practices)
• Stems, at least in part, from little codification of practice
Similar work linking teaching practice to student learning
• Find systematic differences in the practices of more effective teachers
• Similarly still early on in developing the knowledge base.
• Useful because it can be used in– decision making in general– and in developing supports
Not today…
3. Understanding and Improving the Abilities of Local Decision Makers
variation in the abilities and behaviors of school principals
Motivation: Why Teachers Leave
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
Close
r Hom
e
Plac
e
SALA
RY
Pres
tige
Benefi
ts
Job
Secu
rity
Preg
nanc
y/Chi
ld
Health
Fam
ily/P
erso
nal
Scho
ol A
ction
Oppor
tuni
ty
Dissat
isfac
tion
College Recommended (N=156)Teaching Fellows (N=36)
Other (N=114)
when we look within dissatisfaction…
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
Assin
gmen
t
Facilit
ies
Safe
ty
Adm
in
Behav
ior
Colle
ague
s
Teac
hing
Phi
loso
phy
Auton
omy
Test
ing
Distric
t Policy
Resp
ect
Own
Abilit
y
College Recommended (N=150)Teaching Fellows (N=35)
Other (N=112)
Motivation, more generally• Principals linked to teacher satisfaction and career
choices
• Principals central actors in most recent school reforms (accountability, school-based budgeting, charter schools)– Better local information– Improved incentives from accountability system– Concern about abilities
• Increased policy attention on attracting and preparing effective school leaders, but– Lack understanding of principal abilities to look for when hiring
or to target development – Lack of organized systems for recruiting and developing
leaders (in most places)
our approach: again, exploratory
What do Principals do?• How do these
behaviors vary across schools?
• Do principals in high performing behave differently
What abilities do principals need to do these tasks?• How do
these abilities vary across schools?
Given these findings,
explore in more detail
the behaviors and abilities
identified
How are schools affected by leadership labor market dynamics• Do principals, like
teachers, demonstrate preferences for working in some schools and not in others?
What do principals do? Developed list of 47+ tasks that principals might do
based on: Research literature Discussions with principals and Piloting and shadowing in local California schools
Collected observational time use data Observed each principal for one full day Recorded time use on 47 (later more) tasks every five minutes
Sample All high school principals in Miami-Dade County Public schools
(plus 6 elementary and 6 middle school principals) All schools serving 6th graders and above in Milwaukee Public
Schools All schools in San Francisco and Kansas City Today focus only on Miami-Dade County schools
How Do Principals and Teachers Perceive Principal
Effectiveness on These Tasks? Surveyed all principals, assistant principals and teachers Principals in M-DCPS (n = 314; 89% response) Assistant principals (n = 585; 85%) Teachers (n = 15,842; 83%) -- satisfaction
Asked principals how effective they felt at each of the tasks
Asked assistant principals how effective their principals were at each task
Identify groupings of self-assessed task effectiveness reflecting underlying skills
Link responses and observations to administrative data, other survey data, and interviews to assess the relationship between skills and school outcomes
5 Primary DimensionsInstruction
Management
Skills for promoting
and improving
the implementin
g of curricular
programs in classrooms
Using assessment results, providing
instructional feedback,
implementing PD
Internal Relations Skills for building strong
relationships within the
school
Handling staff conflicts,
counseling students and
teachers
Organization
Management
Skills employed to maintain a
highly functioning
organization
Maintaining facilities,
budgeting, hiring personnel
Admin-istration
Skills related to
compliance and
regulatory tasks
Maintaining records, fulfilling
special ed requirements,
managing attendance
External Relations Skills for
working with outside stake-
holders
Communicating with the
district office, fundraising, working with
the community
Split into program and day-to-day for observations
Across Districts: Most time on Administrative Tasks
Org
anizatio
n Mana...
Admin
istra
tion
Day-to
-Day
Inst
ruc.
..
Inst
ruct
ional P
rogra
m
Inte
rnal R
elatio
ns
Extern
al Rela
tions
-5%
0%
5%
10%
15%
20%
25%
30%
High School Principal Time-Use by District
SFUSD (n=17) MPS (n=30) M-DCPS (n=40)
Most Time Spent On:• Disciplining students• Supervising students• Observing classrooms• Internal relationships• Compliance requirements• Managing budgets
Least Time Spent On:• External relationships• Coaching teachers• Using data and
assessments• PD for teachers• PD for themselves• Teaching students
Findings: principal time-use
MPS, M-DCPS
The Importance of Organization Management
Time-Use
Organization Management
Student Performance
Gains
Instructional Program
Teacher Satisfaction at Current
School
Day-to-Day Instruction
Teacher Assessmen
ts of School
Parent Assessmen
ts of School
positive relationship negative relationship
Internal Relationships
Principals’ Organization
Management Efficacy
Student Performance Gains
ParentAssessments
TeacherSatisfaction
Other domains NOT related to outcomes.
The Importance of Organization Management
Efficacy
Corroboration Using Assistant Principal
Assessments• Principal self-assessments may not be the best
way to measure task effectiveness– Principals can’t be objective about themselves
• Administer the same inventory to their APs (multiple per school) and look for the same patterns
• Uncover 3 factors from AP responses: Instruction, Internal Relations, Operations
Assistant Principals’ Assessment of
Principals’ Organization
Management Efficacy
Gains in Student Achievement
ParentSatisfaction
TeacherSatisfaction
Note: Other domains of efficacy NOT related to outcomes.
Assistant Principal Assessment of Principal Task Efficacy and Outcomes
M-DCPS
Conclusion 3• Evidence of variation in the behaviors and abilities of
local decision makers
• Given current behaviors and abilities – improvements in Organization Management appear beneficial
• Current focus on instruction likely useful but poorly focused• Classroom visits good for gaining information• But, only useful if information is used for improvement – rare.
• Now looking at how to select leaders for organizational leadership skills as well as develop skills in principals
• Principals subject to many of the same market forces as teachers
In summary: US faces many of the same
ChallengesGenerally low
student outcomes
Systematic inequality in
student outcomes
Importance of teachers for
students’ opportunities
Great differences in teacher
effectiveness
Systematic differences in
teachers across communities
Little knowledge of how to improve
effectiveness
A range of factors affect and can improve the teaching workforce
Recruitment and Selection
Development and Supports
Assignment Retention
solutions are complex and changing
• Needs vary– For example, best professional development depends
on the skills of the teachers and context of the teaching
• Best decisions vary and change over time
• Useful to consider who makes decisions and their abilities, information and incentives
• Our aim is to understand these dimensions and how policies can influence them
Some findings to date• Simple regulations (such as higher test requirements)
can incentivize local recruitment and improve the teaching workforce.
• Systematic knowledge generation in teacher development and teaching practice is both useful and doable
• Local variation in needs and options lead to benefits of local decision making but local decision makers (e.g. principals) vary in their abilities – a similar policies and information are needed for the leadership
labor markets.
Teacher Trajectories and Teacher Effectiveness: the role of policy and leadership
Susanna LoebCEPPEMarch 11, 2011