Racial Concentration and School Effectiveness in SFUSD

Racial Concentration and School Effectiveness in SFUSD

SFUSD Board Meeting, 12/14/09

Stephen NewtonProfessor Linda Darling-HammondSchool Redesign Network at Stanford University

1

Background SFUSD’s Strategic Plan sets a goal to “disrupt the

predictive power of demographics.” This problem has also been called the “achievement gap” between historically underserved populations of students and others.

SFUSD schools with high concentrations of African-American, Latino, and Samoan (AA/L/S) students generally have lower achievement levels than other schools. Findings presented to Ad Hoc Committee on 2/28/09.

Key question: do these lower outcomes reflect a relationship between racial concentration and school effectiveness?

2

Lens for this study

These analyses focus on whether school composition, specifically racial concentration of AA/L/S students, plays a role in increasing the achievement gap for historically underserved populations.

3

On average, schools with greater proportions of AA/L/S students generally have lower API scores, but there were also exceptions to this trend

School AA, Latino & Samoan Enrollment vs API Growth 2008Correlation Coefficient = -0.69993

400

600

800

1000

0 20 40 60 80 100 120

School AA, Latino & Samoan Enrollment %

Sch

oo

l AP

I Gro

wth

200

8

Source: Presented to Board, February 2009:4

-0.74 **API Score

AA/L/Samoan

Enrollment

-0.25 *Staff Satisfaction

0.33 **Suspension Rate

0.48 **Teacher Turnover

-0.43 **Attendance (seat time)

-0.54 **Teacher Average Years of Services

Correlations between racial concentration and other school factors

* Correlation is significant at the 0.05 level (2-tailed).

** Correlation is significant at the 0.01 level (2-tailed).

Concentration of AA/L/S students is strongly correlated with a range of measures related to school quality

Source: Ad Hoc Committee Presentation, March 2009:

5

District ELA achievement gap has grown (Gap in percent proficient has grown by 5% for Latinos and 7% for African-Americans)

0

10

20

30

40

50

60

% A

t o

r A

bo

ve

Pro

fic

ien

t

AA 12 13 14 15 19 21 22 23

L 15 16 19 20 24 27 28 28

SFUSD 33 35 39 40 45 48 49 51

2001 2002 2003 2004 2005 2006 2007 2008

CST English Language Arts:8 Year Trends for Proficient and Above (Grade 2 to 11)

18%

21%

23%

28%

Legend

African-American Achievement

Latino Achievement

SFUSD AchievementSource: Ad Hoc Committee Presentation, March 2009:

6

District math achievement gap has also grown (Gap in percent proficient has grown by 3% for Latinos and 6% for African-Americans)

0

10

20

30

40

50

60

70

80

% A

t o

r A

bo

ve

Pro

fic

ien

t

AA 10 14 14 21 24 25 26

L 16 22 23 31 34 35 35

SFUSD 37 44 46 53 57 58 59

2002 2003 2004 2005 2006 2007 2008

CST Mathematics:7 Year Trends for Proficient and Above (Grade 2 to 7)

21%

27%

24%

33%

Legend

African-American Achievement

Latino Achievement

SFUSD AchievementSource: Ad Hoc Committee Presentation, March 2009:

7

Review of findings

Three conclusions from these background slides:1. Racial concentration of AA/L/S students is related to lower

average school performance.2. The achievement gap for historically underserved students is

widening.3. Racial concentration is also related to other school quality

factors. This analysis delves deeper into the role of racial

concentration by using inferential statistics to control for other factors that can influence student outcomes.

8

Our charge – The impact of school composition on school effectiveness

Key Question: On average, are schools with concentrations of African-American, Latino, and Samoan (AA/L/S)* students as effective as other schools in SFUSD? Effectiveness/value-added/productivity compares whether, on

average, students gain more or less than similar students in other SFUSD schools. In other words, it focuses on fair peer-to-peer comparisons.

Positive effectiveness means students gain at a faster rate than average, and negative effectiveness means students gain at a slower than average rate.

If you want to know a given school’s effectiveness, it is better to look at its matrix gap, a value-added estimate, than to consider its racial concentration.

9

Analyses conducted to answer the key question1. Did students in AA/L/S concentrated schools have different academic

outcomes compared with similar students in other SFUSD schools?• School-level outcomes - Matrix gaps.• Student-level outcomes – Productivity, Propensity score matching.

2. When a school’s proportion of AA/L/S students changed, did its achievement also change?

3. Was concentration of AA/L/S students related to non-academic outcomes?

• Graduation rates, Mobility.

4. Was concentration of AA/L/S students related to teacher experience and stability?

• Years experience.• Percent first- and second-year teachers.• Teacher retention.

10

Question 1: Did students in AA/L/S concentrated schools have different academic outcomes compared with similar students in other SFUSD schools?

Methods Compare outcomes while controlling for prior year

achievement in ELA and math and student demographic characteristics.

Value-Added/Productivity - Use statistical models to estimate future achievement and then compare actual with estimated achievement (matrix, productivity analysis).

Propensity Score Matching - Find similar students and compare outcomes.

These methods provide an estimate of school effectiveness.

11

Question 1 (school-level): School value-added in ELA is lower, on average, in concentrated AA/L/S schools (2007-2008)

-1-.5

0.5

0 20 40 60 80 100AALASa08_pct

95% CI Fitted valuesEnglish

t= -5.65, p<.001

School ELA Matrix Gap (sds)

Percent AA/L/S Students in School

12

Question 1 (school-level): School value-added in math is also lower, on average, in concentrated AA/L/S schools (2007-2008)

-.4

-.2

0.2

.4

0 20 40 60 80 100AALASa08_pct

95% CI Fitted valuesMath

t= -8.11, p<.001


School Math Matrix Gap (sds)

13

Question 1 (student-level): Racial concentration is not related to lower elementary school productivity for all students, on average, except for highly concentrated schools (2003-04 to 2008-09)

Student Value-Added Gains (Grades 3-5, All Students)

-0.1

-0.05

0

0.05

0.1

0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%

School Percent AA/L/SA

Ave

rag

e V

alu

e-A

dd

ed (

sds)

ELA

Math

Question 1 (student-level): Elementary schools do show lower productivity for AA/L/S students, on average, with increasing racial concentration (2003-04 to 2008-09)

Student Value-Added Gains (Grades 3-5, AA/L/S Only)

-0.1

-0.05

0

0.05

0.1

0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%


Ave

rag

e V

alu

e-A

dd

ed (

sds)

ELA

Math

Question 1 (student-level): Middle schools show a mixed productivity picture for all students, with lower gains at schools with 80% -100% AA/L/S students (2003-04 to 2008-09)


-0.1

-0.05

0

0.05

0.1

0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%


Ave

rag

e V

alu

e-A

dd

ed (

sds)

ELA

Math

Question 1 (student-level): Middle schools also show a mixed productivity picture for AA/L/S students (2003-04 to 2008-09)

Student Value-Added Gains (Grades 6-8, AA/L/S Only)

-0.1

-0.05

0

0.05

0.1

0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%


Ave

rag

e V

alu

e-A

dd

ed (

sds)

ELA

Math

Question 1 (student-level): High schools show lower productivity for all students, on average, with increasing racial concentration (2003-04 to 2008-09)


-0.1

-0.05

0

0.05

0.1

0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%


Ave

rag

e V

alu

e-A

dd

ed (

sds)

ELA

Question 1 (student-level): High schools also show lower productivity for AA/L/S students, on average, with increasing racial concentration (2003-04 to 2008-09)

Student Value-Added Gains (Grades 9-11, AA/L/S)

-0.1

-0.05

0

0.05

0.1

0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%


Ave

rag

e V

alu

e-A

dd

ed (

sds)

ELA

Question1 (student-level matched*): Students in concentrated (60%+) AA/L/S schools do less well than matched students at other SFUSD schools Students show smaller gains in ELA and math in AA/L/S

concentrated schools than similar students in other SFUSD schools (2003-04 to 2008-09). This comparison combines grades 3-11 in ELA, and all math courses above

grade 2. Students were matched using a statistical technique called propensity score

matching. All Students

ELA: -.04 sds (t=-8.56, p <.001) Math: -.02 sds (t=-3.93, p <.001)

AAL Students ELA: -.04 sds (t=-6.81, p <.001) Math: -.01 sds (t=-1.98, p <.05)

*Students matched on prior achievement in ELA and math, gender, race/ethnicity, parent education, EL status, retained in grade, special education, and poverty

20

Differences in effectiveness add up over time (hypothetical)

A student starting at 50th percentile in 2nd grade in a school with -.04 sd effectiveness would decline to 36th percentile, on average, by 11th grade.

Similarly, with -.02 effectiveness, the student would decline to 43rd percentile by 11th grade.

0

10

20

30

40

50

60

70

80

90

100

2nd 3rd 4th 5th 6th 7th 8th 9th 10th llth

Grade

Per

cen

tile

ELA

Math

Example based on effectiveness difference found for concentrated AALS schools

21

Question 2: Changing demographics and achievement When a school’s demographics changed, what

happened to its average achievement? We focused on the change between 1999 and 2008 in SFUSD schools.

This has important implications for accountability because the state accountability system requires schools to demonstrate adequate yearly progress (AYP) regardless of their demographic composition.

22

-1-.

50

.51

-40 -20 0 20 40AALSchange_pct

95% CI Fitted valuesZELA_change

Schools that increased in percent AALS students tended to decrease in average ELA achievement (1999 to 2008)

Change in ELA Achievement for All Students (z-scores)

Change in %AA/L/S Students in School

t=-5.87, p<.001

23

Schools that increased in percent AALS students also tended to decrease in average math achievement (1999 to 2008)

-1-.

50

.51

-40 -20 0 20 40AALSchange_pct

95% CI Fitted valuesZMath_change

Change in Math Achievement for All Students (z-scores)

Change in %AA/L/S Students in School

t=-6.96, p<.001

24

Question 3 (non-academic outcomes): Graduation and student mobility

Concentrated AA/L/S schools had an 11% lower graduation rate than other SFUSD schools in 2007-08 (controlling for poverty).

Concentrated AA/L/S schools had a 3.8% higher student mobility rate (students entering or leaving a school) than other SFUSD schools in 2007-08 (controlling for poverty).

25

How can we explain differences in school effectiveness? Research has suggested segregated schools are often less

effective: In Florida, segregation mattered in predicting school-level performance on

Florida’s state tests (Borman, 2004). In Texas, high racial concentrations of African American students in schools

reduced achievement for African American students, and racial composition of a school explains a meaningful portion of the racial achievement gap (Hanushek, et al., 2007).

The reasons for lower effectiveness are not fully-understood but appear to be complex.

These were average effects, and not all schools fit these trends.

We decided to explore differences in teacher experience and stability in AA/L/S concentrated schools.

26

Question 4 (teachers): Schools with more AA/L/S students, on average, had less experienced teachers and more teacher turnover

Teacher experience and stability (2004-05 to 2007-08) AALS concentrated schools had teachers with significantly less

experience than other SFUSD schools (average 10.3 years exp. vs. 13.4 years exp.) (t= -11.26, p<.001).

AALS concentrated schools had significantly more first and second year teachers (3.7%) than other SFUSD schools (2.1%) (t= 4.23, p<.001).

AALS concentrated schools had a significantly lower rate of teacher retention than other SFUSD schools (73.4% vs. 83.9%, t=-7.61, p<.001).

27

020

4060

8010

0

0 20 40 60 80 100AALS_pct

95% CI Fitted valuesteacher_return_pct

Schools with higher concentrations of AA/L/S students had lower average rates of teacher retention (2003-04 to 2007-08)

t=-8.7, p<.001


Percent of Teachers Retained in School

28

Schools with concentrated poverty – Checking an alternative possibility

Schools with concentrated poverty also had lower effectiveness than other schools. Results were similar to racial concentration when using a

propensity score matching model.

Racial concentration and poverty contribute separate effects When put into the same model, both factors independently

contribute to lower effectiveness.

29

Conclusions (1 of 3) – Racial concentration reduces school effectiveness for historically underserved students

SFUSD schools with concentrations of AA/L/S students have been less effective, on average, in raising student achievement overall. This difference does not depend on differences in the individual

students, because students who are demographically similar still show smaller gains, on average, at concentrated AA/L/S schools.

The clearest patterns were seen in high school for all students and AA/L/S students specifically, and in elementary schools for AA/L/S students.

This effect is larger in ELA than math. Concentrated schools also had lower effectiveness for AA/L/S students.

Lower effectiveness increases the achievement gap. Because effectiveness measures rate of academic progress, the racial

achievement gap is increased if AA/L/S students attend less effective schools.

30


On average, teachers at racially concentrated schools are less experienced and have higher mobility.

Racial concentration is one factor influencing a school’s effectiveness, but there are others. Some schools with high concentrations of AA/L/S students had

strong effectiveness, whereas other schools with low concentrations of AA/L/S students had weak effectiveness.

It is important for the district to study why schools are effective or ineffective. We have found some initial clues that could be further pursued:

Disproportionate allocations of inexperienced teachers, plus high teacher turnover in concentrated AA/L/S schools – both found in other research to reduce student achievement – may be contributors to lower productivity in these schools.

31


SRN case studies of SFUSD schools which effectively serve low-income students of color have identified other factors that may play a role, including:

quality of school leadership; coherent curriculum providing rich literacy and learning

experiences; and extensive professional development focused on

equitable instruction. It may be especially useful to further study effective

schools with high concentrations of AA/L/S students, as well as those that are currently struggling.

Thank you.

32

Citations

Borman, et al. (2004). Accountability in a postdesegregation era: The continuing significance of racial segregation in Florida’s schools,” American Educational Research Journal, v41, n3, p. 605.

Hanushek, E.A., Kain, J.F., Rivkin, S. G. (June 2007) “New Evidence about Brown v. Board of Education: The Complex Effects of School Racial Composition on Achievement.”

33

Racial Concentration and School Effectiveness in SFUSD

Documents

Transcript of Racial Concentration and School Effectiveness in SFUSD