Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008 spicciad@hsd401
description
Transcript of Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008 spicciad@hsd401
Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008
The difference between teachers one SD above and below the mean is one year’s worth of achievement (Hanushek, 1992)
Teacher effects are cumulative; three years with top vs. bottom quintile teachers opens a 54 percentile gap (Sanders & Rivers, 1996)
Rowan, Correnti, & Miller’s (2002) comprehensive study of teacher measurement methodology concluded 52%-72% of student mathematics variance lies between classrooms, with the rest between students and between schools.
A one SD increase in teacher effectiveness is equal to a reduction in class size from 25 to 15 (Nye, Konstantopoulos, & Hedges, 2004)
Experience Experience has a curvilinear relationship
with achievement. Achievement rises with experience for
between 2 and 5 years, with “on-the-job training”, then levels off (Ferguson, 1991; Darling-Hammond, 2000; Rockoff, 2004; Rivkin, Hanushek, Kane, 2005).
Advanced Degrees Master’s degrees are important in mathematics
and science in secondary (Goldhaber & Brewer, 1997; Wenglinsky, 2000).
Findings on advanced degrees are split for elementary.◦ Many studies find that advanced degrees do not relate
to elementary mathematics achievement...(Hanushek, 1986; Rivkin, Hanushek, Kain, 2005; Clotfelter, Ladd, & Vigdor, 2007).
◦ However, some reputable studies find a positive, significant relationship (Ferguson & Ladd, 1996; Greenwald, Hedges, Laine, 1996; Nye, Konstantopolous, & Hedges, 2004).
College Selectivity A teacher’s academic ability, particularly
verbal ability, is among the most established teacher variables in relation to student achievement (Hanushek, 1986; Rice, 2003).
College selectivity, often measured by Barron’s rankings, is a proxy for academic ability that is moderately related to student achievement (Wayne & Youngs, 2003).
Mathematics Courses Mathematics content knowledge, as measured
by tests of teachers, relates to achievement (Harbison & Hanushek, 1992; Hill, Rowan, & Ball, 2005).
Mathematics courses relate to math achievement in secondary (Monk & King, 1994).
However, Hill, Rowan, & Ball (2005) found there is little empirical evidence examining math courses and achievement at the elementary level, and their findings were not significant.
Teacher effectiveness. The present study is focused on “teachers,” as opposed to “teaching.” In this context, “teacher effectiveness” is defined by the mathematics achievement of a teacher’s students, as measured by growth on the Measures of Academic Progress (MAP) test, compared to expected growth. While teacher effectiveness is a term used in the literature, this will be a correlational study and will not imply effects.
1. In terms of descriptive statistics, what is the distribution of achievement growth at the classroom level?
2. Is there a significant relationship between advanced degrees, experience, college selectivity, or total mathematics courses taken at the university level and growth in mathematics achievement?
3. What combinations of the above teacher variables best explain the variance in student growth?
4. Since poor and minority communities generally attract and retain less qualified and experienced teachers than other communities, would the achievement of diverse classes be significantly higher if they had equal or even equitable access to teachers with experience and advanced degrees?
3,558 students◦ 70.7% of all students in grades 3-6◦ 84.2% of all students with complete scores,
excluding self-contained classes 156 teachers
◦ 68.7% of all teachers in grades 3-6◦ 89.7% of all eligible teachers
Required teacher variable data was located for all teachers
Measures of Academic Progress (MAP)◦ Published by Northwest Evaluation Association (NWEA)◦ Computer adaptive; item response theory◦ Multiple choice; typically 40 items◦ Measures the content strands found on the math WASL◦ Administered fall, winter, and spring
Reliability and Validity◦ Test-retest reliability: r = .88 to r = .93 ◦ Marginal reliability: r = .94◦ Concurrent validity (with state tests): r = .79 to r = .89
Permission granted by superintendent and SPU Institutional Review Board
Gathered existing data◦ MAP scores accessed in “raw” format from district
database◦ Teacher data accessed from Human Resources
Degree database contained universities and degrees Highly Qualified Teacher database contained record
of course taking Samples double checked against actual transcripts
Independent Variables Demographic
◦ Class Percent Non White (CPNW)◦ School Free and Reduced Lunch Percentage (SFRL)◦ Class Percent of English Language Learners (CPEL)
Teacher◦ Experience (EXP)◦ Experience Dichotomized (EXPDI)◦ Degree (DEGR)◦ College Selectivity (COLL)◦ Number of Mathematics Courses, Content and Pedagogy (MC)◦ Math Courses Dichotomized (MCDI)
Dependent Variable Class Percent of Expected Growth (CPEG)
◦ Fall to spring student level MAP growth, divided by NWEA expected (normal) growth, aggregated to class level
Descriptive Statistics◦ Overall◦ Disaggregated by quartile level of diversity
Correlation Multiple Regression
◦ Identification of best model for this dataset◦ Regression equation used to estimate results with
various staffing scenarios
Means and standard deviations for each variable
N Mean SD
1. Class Percentage of Expected Growth (CPEG) 156 98.9 28.3
2. Class Percentage Non-White (CPNW) 156 62.3 21.1
3. School Free and Reduced Lunch Percentage (SFRL) 156 59.5 18.7
4. Class Percentage of English Language Learners (CPEL) 156 11.8 11.5
4. Experience (EXP) 156 9.3 8.3
5. Exp. Dichotomized (EXPDI) (0 = 0-5 years; 1 = 6+ years) 156 .57 -
6. Degree (DEGR) (0 = BA; 1 = MA) 156 .58 -
7. College Selectivity (COLL) (0 = low; 1 = high) 156 .51 -
8. Num. of Math Courses, Content, and Pedagogy (MC) 156 3.4 2.1
9. Math Courses Dichotomized (MCDI) (0 = 0-3; 1 = 3.5+) 156 .45 -
Performance Least diverse
quartile grew most
Demographics Poverty and ELL
highly related to diversity
Teachers Low diversity
classes taught by more experienced teachers
Other variables have weaker relationships
1st Quartile
CPNW 5-48
M (SD)
2nd Quartile
CPNW 48-68
M (SD)
3rd Quartile
CPNW 68-78
M (SD)
4th Quartile
CPNW 79-100
M (SD)
1. CPEG 111.0 (32.0) 96.8 (28.0) 96.0 (27.0) 91.7 (22.7)
2. CPNW 32.9 (10.1) 57.1 (6.4) 73.3 (3.2) 86.2 (5.6)
3. SFRL 35.1 (13.4) 58.4 (12.3) 69.1 (8.2) 75.4 (7.7)
4. CPEL 1.7 (4.3) 9.7 (9.7) 14.8 (10.2) 21.0 (10.9)
5. EXP 13.0 (8.5) 10.6 (8.5) 6.3 (6.8) 9.2 (7.7)
6. EXPDI .77 .64 .44 .44
7. DEGR .69 .49 .51 .64
8. COLL .51 .44 .59 .49
9. MC 3.1 (2.2) 3.9 (2.4) 3.2 (1.7) 3.3 (2.0)
10. MCDI .38 .51 .41 .49
Overall n = 156; each quartile n = 39
Diversity level only explains 9% of growth
Large range of growth at every level of diversity
Many highly diverse classes outperform expected growth
Explanation Each color
represents a diversity quartile
Each bar represents 9 or 10 classrooms, grouped by growth, with average CPEG shown
Interpretation Top classrooms
in every diversity quartile outperform the average non diverse class
Variable 1 2 3 4 5 6 7 8 9 10
1. CPEG ___ -.30** -.26** -.25** .05 .17* .32** .05 -.03 .07
2. CPNW -.30** ___ .85** .63** -.35** -.30** -.02 -.01 .02 .05
3. SFRL -.26** .85** ___ .61** -.29** -.24** -.02 -.16* .07 .10
4. CPEL -.25** .63** .61** ___ -.30** -26** -.06 -.15 -.08 -.08
5. EXP .05 -.35** -.29** -.30** ___ .75** .06 -.04 .08 .09
6. EXPDI .17* -.30** -.24** -.26** .75** ___ .11 -.11 .19* .13
7. DEGR .32** -.02 -.02 -.06 .06 .11 ___ .13 -.10 -.02
8. COLL .05 -.01 -.16* -.15 -.04 -.11 .13 ___ -.10 -.01
9. MC -.03 .02 .07 -.08 .08 .19* -.10 -.10 ___ .78**
10. MCDI .07 .05 .10 -.08 .09 .13 -.02 -.01 .78** ___
** Correlation is significant at the 0.01 level (2-tailed).
* Correlation is significant at the 0.05 level (2-tailed).
Diversity explains 9% of CPEG scores
Advanced degrees and experience explain an additional approximately 9%
Experience does not significantly explain CPEG scores beyond advanced degrees
R R2 Adj. R2 R2
Change
B Sig.
Constant 110.206 .000
Set One .299 .090 .084 .090
CPNW -.372 .000
Set Two .435 .189 .173 .100
DEGR 17.575 .000
EXPDI 2.923 .507
The reduced model includes only diversity and advanced degrees
Advanced degrees explain more than 9% of variance in CPEG scores beyond what diversity explains
The model as a whole explains about 18% of the variance in CPEG scores
R R2 Adj. R2 R2
Change
B Sig.
Constant 112.998 .000
Set One .299 .090 .084 .090
CPNW -.393 .000
Set Two .433 .187 .176 .097
DEGR 17.870 .000
Explanation Using Beta weights
from the multiple regression equation, achievement levels are simulated using different allocation methods
Interpretation An equitable approach
could close achievement gap between Q1 and Q4 from 21% (in the status quo model) to 7%◦ See limitations
This approach would be more powerful with a stronger measure of teacher quality or a characteristic that varies more greatly across schools
Scenario 1st Qtile. 2nd Qtile. 3rd Qtile. 4th Qtile. Overall
State Average CPEG 111.1 101.6 95.2 86.2 99.5
DEGR .62 .62 .62 .62 .62
Equality (District Avg.) CPEG 110.4 100.9 94.6 89.5 98.9
DEGR .58 .58 .58 .58 .58
Equitable Distribution CPEG 104.2 98.4 96.2 97.0 98.9
DEGR .23 .44 .67 1.00 .58
Status Quo* CPEG 112.4 99.3 93.3 90.6 98.9
DEGR .69 .49 .51 .64 .58
* “Status Quo” is based on actual CPNW and DEGR, but computed CPEG.
Q1: Distribution of Achievement by Classroom◦ 1 SD of classroom effectiveness = nearly 3 months of growth
Q2: Teacher Characteristics and Math Achievement Growth◦ Advanced degrees
Different findings may be attributable to small “n” of colleges, local bargaining context, or methodology that cannot link individual teachers with their characteristics.
◦ Experience Findings herein consistent with research.
◦ College selectivity Data lacks variability to show results.
◦ Mathematics courses Content knowledge appears to matter, based on Hill, Rowan, & Ball
(2005), but coursework is a poor proxy. Q3: Combinations of Variables
◦ No significant interactions. Q4: Equal or Equitable Distribution of Teachers
◦ Simulations of this nature may be needed to encourage policy.
Knowing and acting on data◦ Disaggregating achievement data by classroom◦ Using responsible and ethical assessment and HR
practices◦ Engaging in courageous conversations and
leadership actions Teacher distribution and assignment
◦ Monitor teacher characteristics data to prevent neediest schools from having disproportionately inexperienced/less qualified teachers
◦ Referee student assignment to avoid repeated exposure to low performing classrooms (Sanders)
Methodology◦ Gain scores◦ Small student “n” size per teacher◦ Multiple regression vs. HLM
Internal Validity◦ Does measuring classrooms = measuring teachers?◦ Unidentified covariates
External Validity◦ Ability to generalize◦ Assumptions that teachers would perform similarly
in different situations
1. A multi-state study. 2. A study of teachers that lasted more than
one year. 3. A study of other forms of mathematics
content acquisition. 4. A study that includes variables for teachers
who took a remedial mathematics course or who failed a mathematics course.
5. A qualitative study of teachers whose students significantly outperform.
Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008