Don Boyd, Pam Grossman, Karen Hammerness, Hamp Lankford, Susanna Loeb, Matt Ronfeldt & Jim Wyckoff

www.teacherpolicyresearch.org

This work is supported by IES Grant R305E6025. The views expressed may not reflect those of the funder.

Recruiting Effective Math Teachers, How Do Math Immersion Teachers

Compare?: Evidence from New York City

New Math Certified Teachers Hired in New York City, by Pathway, 2002-2008

0

100

200

300

400

500

2002 2003 2004 2005 2006 2007 2008

CR

NYCTF-MI

NYCTF

TFA

Research Questions How does the preparation of Math Immersion

teachers compare to math teachers entering through other pathways?

How do the achievement gains of the students taught by Math Immersion teachers compare to those of students taught by math teachers entering through other pathways?

How does the retention of Math Immersion candidates compare to math teachers entering through other pathways?

Achievement effects of alternate route teachers comparable to traditional preparation programs on average (Decker et al., 2004 (RCT); Boyd et al., 2006; Kane et al., 2007; Harris and Sass, 2008; Constantine et al., 2009 (RCT))

TFA in NC high schools exceeds other paths (Xu et al., 2007)

More limited work on aspects of preparation that may make a difference (Constantine et al., 2009; Boyd et al., 2009 and Harris and Sass, 2007)

Prior Research

Data Collection Program analysis

– State documents, program documents, accreditation reports, interviews, surveys, course syllabi;

– 5 Math Immersion programs,18 institutions, and TFA that prepare most traditional route teachers for NYC schools

Surveys – 603 new NYC middle and high school math teachers (2005)– Questions about their preparation in math– e.g, opportunities to

learn math content, math methods, etc.

Administrative data – All NYC teachers 2004-2008; rich measures of teacher qualifications,

including certification exams and areas, teacher retention. – Student achievement 2004-2008; value-added scores in math and

ELA, grades 6-8 linked to teachers. – Data on schools and students

Effect of Preparation PathwaysGeneral specificationAigcst=0+1Aig-1cst-1+Xigcst2+Cgcst3+Tgcst4+ Pgcst5 + s + igcst

Achievement as a function of: • prior achievement, • student characteristics• classroom characteristics • teacher characteristics (sometimes)• Preparation pathway (e.g., math immersion)• Student or school fixed-effects• random error

Attributes of Students Taught by First Year Grade 8 Math Teachers by Pathway, 2006

Student Attributes CR NYCTF NYCTF-MI

TFA other

Lagged Math Achievement

0.238 -0.125 -0.051 -0.139 -0.061

Proportion Black 0.292 0.277 0.322 0.442 0.403

Proportion Hispanic 0.358 0.496 0.493 0.527 0.372

Proportion Free Lunch

0.547 0.664 0.635 0.619 0.66

Classsize 27.6 27.8 26.9 26.3 26.1

Lagged Student Absences

12.3 13.4 13.1 14.8 13.5

Lagged Suspensions

0.037 0.064 0.062 0.023 0.042

Attributes of Entering Math Certified New York City Teachers by Pathway, 2004-2008

  CR NYCTF-MI TFA

Teacher Attributes High School

Middle School

High School

Middle School

High School

Middle School

Female 0.648 0.732 0.479 0.546 0.492 0.551Black 0.073 0.105 0.142 0.200 0.082 0.141Hispanic 0.065 0.046 0.085 0.074 0.066 0.043Age 29.7 28.9 31.1 30 23.6 23.5

Last Score 255 251 274 271 279 279CST Math Score 262 251 257 251 268 269SAT Math 600 556 616 589 710 648SAT Verbal 506 483 577 564 627 623

             Number of Teachers 478 157 1098 542 61 98

Effect of Preparation Paths Relative to NYCTF-MI

  1 3 5 7 9

Pathways Level Level Level Level LevelCollege Recommend 0.016 0.005 0.017 0.004 0.006  [1.86] [0.47] [2.60]** [0.40] [0.55]

NYC Teaching Fellows 0.021 0.022 0.023 0.015 0.012  [1.87] [1.68] [2.74]** [1.38] [0.85]

Teacher for America 0.055 0.018 0.068 0.032 0.046**  [3.71]** [0.86] [5.74]** [1.88] [2.77]

Other -0.011 -0.003 -0.004 0.002 -0.02  [1.27] [0.28] [0.66] [0.27] [-1.74]NYCTF-MI Below   -0.044    [-1.52]NYCTF-MI NA   -0.014

    [-1.04]Teacher controls   School fixed effects Student fixed effects      

Distribution of Teacher Value Added by Pathway, with Empirical Bayes Shrinkage, 2004-2008

Teachers

Effect of Pathways and Experience Relative to Math Immersion of Same Experience, Grades 6-8, 2004-08*

  No Teacher Controls

  Experience

Pathway 1 2 3 4+College Recommending 0.018 0.024 0.010 0.028

  [1.60] [1.90] [0.65] [1.53]

NYCTF 0.011 0.010 0.005 0.065

  [0.74] [0.58] [0.24] [2.76]**

TFA 0.054 0.056 0.041 0.048

  [3.13]** [2.64]** [1.09] [1.29]

Other -0.028 -0.032 -0.018 0.009

  [2.22]* [2.61]** [1.29] [0.50]* Same variables as model 1 above.

Effect of Pathways and Math Immersion Programs 2004-08, Relative to NYCTF-MI Program Z*

Pathway and Program Level LevelCollege Recommend 0.057 0.033  [3.94]** [1.89]NYC Teaching Fellows 0.062 0.047  [3.81]** [2.54]*Teacher for America 0.096 0.031  [4.96]** [1.21]Other 0.030 0.027  [2.12]* [1.55]Institution A 0.034 0.018  [1.50] [0.71]Institution B 0.051 0.029  [2.66]** [1.28]Institution C 0.048 0.035  [1.71] [1.16]Institution D 0.055 0.037  [2.99]** [1.72]Teacher controls School fixed effects

* Same variables as earlier model specification.

Teacher Retention by Pathway, Math Certified Teachers, 2004-2008

  NYCTF-MI   CRExperience Transfer Leave   Transfer Leave

1 12.2 12.4 9.6 13.42 18.7 26.5 12.3 19.13 23.6 36.4 16.0 27.74 26.5 42.1   18.0 31.4

  NYCTF   TFAExperience Transfer Leave   Transfer Leave

1 8.9 15.7 5.0 8.22 16.2 29.6 9.9 58.83 19.2 42.3 12.1 75.6

4 24.4 47.5   13.2 78.7

Simulation of Average Value Added by Pathway and Experience Accounting for Attrition

Simulation Average Value Added

Year NYCTF-MI CR NYCTF TFA

1 0.000 0.018 0.011 0.054

2 0.045 0.068 0.053 0.103

3 0.066 0.086 0.072 0.086

4 0.052 0.081 0.088 0.088         

  Value Added by Pathway and Experience

Experience NYCTF-MI CR NYCTF TFA

1st year 0.000 0.018 0.011 0.054

2nd year 0.051 0.075 0.061 0.107

3rd year 0.085 0.095 0.090 0.126

4th year 0.063 0.091 0.128 0.111

Conclusions MI teachers have about the same value-added as

College Recommended teachers Driven largely by selection, TFA performs much

better than either College Rec or Math Immersion Some evidence that both selection and preparation

make a difference

Hypothesis: selective post BA program with tailored coursework that includes content and high quality field experience can meaningfully improve student achievement

For papers and surveys:

www.teacherpolicyresearch.org

Pathways to Teaching in NYC, New Teachers, 2002-08

0

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

2002 2003 2004 2005 2006 2007 2008

CR

NYCTF

TFA

TL

Outline Research questions Data and methods Math preparation in Math Immersion and

College Recommending programs Achievement gains by pathway Retention by pathway Summing up

The Teacher Workforce and Student Outcomes

Student Outcomes

School

teacher quality

leaders

student ability &motivation

facilities

class size

District policies

Teacher preparationpathway

Student and Environment

family

Staterequirements

learningenvironment

neighborhood

teacher certification

peers

Prospectiveteachers

other teachingpolicies

teacher education

Teacher Workforce

salary

hiring

program structure

subject specific

field experiences

preparation fordiverse learners

professionaldevelopment

academic abilityprior experience

teaching

mentoringinduction

retention

Campus D Campus ZTeacher Attributes High

SchoolMiddle School

High School

Middle School

Female 0.468 0.534 0.509 0.538Black 0.108 0.165 0.125 0.127Hispanic 0.088 0.046 0.069 0.093Age 31.7 31.9 29.9 29.7

Last Score 272 268 277 276CST Math Score 255 249 257 257SAT Math 618 586 625 616SAT Verbal 573 542 589 586 Number of Teachers 154 116 322 119

Attributes of Entering Math Certified NYCTF-MI Teachers by Preparing Campus, 2004-2008

Required Credit Hours for Key Courses by Pathway, 2004-2008

College Recommending

Math Courses

Math Methods

Classroom Manage.

Learning

Graduate programs        

Mean 4.93 5.79 0.86 3.75Standard deviation 5.34 3.29 1.83 2.16

Undergrad prog. 

Mean 11.00 4.71 1.75 4.50Standard deviation 11.29 1.38 2.26 2.70

   

Math Immersion

Math Courses

Math Methods

Classroom Manage.

Learning

Mean 12.60 8.40 0.60 2.40Standard deviation 5.77 2.51 1.34 1.34

In your preparation to become a teacher, prior to September 2004, how much opportunity did you have to the following:

learn different ways that students solve particular problems

learn theoretical concepts underlying mathematical applications

explore how to apply mathematical materials to real world problems

learn specific techniques for teaching Algebra (Geometry, Number Theory, Probability and Statistics, Calculus)

learn about typical difficulties students have with Algebra (Geometry, Calculus)

study or analyze student math work

study examples o secondary mathematics teaching in the form of videotapes, written cases, etc.

Practice what you learned about teaching math in your field experience

etc.

Survey of 1st year NYC Teachers—Middle and High School Math

Teachers' Perceptions of Preparation by Pathways Relative to NYCTF-MI, (2005 Survey of 1st Year Teachers)

Pathway

Preparation in Specific

Strategies

General Opps to Learn Teaching

Math

Subject Matter Prep in

MathCollege Recommending 0.331 0.386 0.038

  [2.99]*** [3.54]*** [0.33]

Teaching Fellows 0.274 -0.350 -0.462

  [2.50]** [-3.32]*** [-4.12]***

Teach For America 0.604 -0.007 -0.561

  [2.74]*** [-0.03] [-2.48]**

Other Path 0.004 0.371 0.320

  [0.04] [3.31]*** [2.74]***

 

N 558 543 541

Estimated Value Added Model*Student Measures   Class Average Measures Experience  Lag score 0.593 Hispanic -0.161 2nd year 0.050  [269.33]**  [6.81]** [8.92]**Lag score sqrd -0.005 Black -0.152 3rd year 0.082  [3.70]**   [6.11]** [12.70]**Female 0.010 Asian 0.099 4th year 0.091  [6.58]**   [3.71]** [12.22]**Asian 0.126 Class size 0.000 5th year 0.100  [35.45]**   [0.85] [12.64]**Hispanic -0.059 English home -0.026 6th year 0.096  [19.07]**   [1.48] [11.01]**Black -0.060 Free lunch 0.014  

  [18.21]**   [1.57] Pathways  

Change school -0.078 Lagged absent -0.007 Coll. Recomm. 0.016  [16.22]**   [13.30]** [1.86]English home -0.060 Lag suspended -0.002 NYCTF 0.021  [31.51]**   [0.15] [1.87]Free Lunch -0.017 Lag ELA score 0.194 TFA 0.055  [10.46]**   [24.73]** [3.71]**Lagged absent -0.005 Lag Math score 0.076 Other -0.011  [64.92]**   [9.16]**   [1.27]Lag suspended -0.024 Std Dev ELA score 0.043    [12.20]**   [4.78]** N 651191

* Also includes student and class ELL status, std dev class math score, indicators for experience through 21 years, year and grade effects

Challenges of this type of analysis

Conceptualizing relationships Research designs Collecting appropriate data

Achievement tests, tested grades, subjects Strong controls from administrative data Other data about teachers

Legal/political Privacy Concerns about misuse

Technical/modeling Models that isolate contribution of teacher attributes

Effect of Preparation Paths Relative to NYCTF-MI*

  1 2 3 4

Pathways Level Level Level Level

College Recommend 0.016 0.017 0.005 0.004

  [1.86] [2.60]** [0.47] [0.40]

NYC Teaching Fellows 0.021 0.023 0.022 0.015

  [1.87] [2.74]** [1.68] [1.38]

Teacher for America 0.055 0.068 0.018 0.032

  [3.71]** [5.74]** [0.86] [1.88]

Other -0.011 -0.004 -0.003 0.002

  [1.27] [0.66] [0.28] [0.27]

     

Teacher controls   School fixed effects  

Student fixed effects    

* Same variables as earlier model specification.