When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term...

27
109 American Economic Journal: Applied Economics 2012, 4(1): 109–135 http://dx.doi.org/10.1257/app.4.1.109 I think the best thing that happened to the education system in New Orleans was Hurricane Katrina. — Arne Duncan, US Secretary of Education, January 2010. 1 O ne of the many effects of Hurricane Katrina was to close temporarily and later to force the reconstitution of schools in one of the nation’s worst performing urban school districts. Many, including US Secretary of Education Arne Duncan, have speculated that there may be a silver lining to the disaster in that students were forced to exit a poorly performing school system or that adults were forced to exit a labor market with limited opportunities. 2 Many New Orleans evacuees migrated permanently to different and, in most cases, better performing schools, while other evacuees returned to the newly formed Recovery School District in New Orleans. This paper is intended to be a thorough examination of long-run test score and col- lege going outcomes for the evacuees. Hurricane Katrina was one of the worst natural disasters in United States his- tory. Roughly 1,900 deaths have been blamed on Katrina and estimates of the 1 Anderson (2010). “Education Secretary Duncan Calls Hurricane Katrina Good For New Orleans Schools.” Washington Post, January 30. 2 Vigdor (2007) finds little support for the latter hypothesis. * Department of Economics, Dartmouth College, 6106 Rockefeller, Hanover, NH 03755 (e-mail: bruce.sacerdote@ dartmouth.edu). I thank Joshua Angrist, Amitabh Chandra, Alan Gustman, Doug Staiger, and seminar participants at NBER Summer Institute, RAND, Louisiana State University, Harvard University, Cornell University, and University of California-Santa Barbara for helpful suggestions. I thank Sam Farnham and Celia Kujala for outstanding research assis- tance. I thank Fen Chou (Louisiana Department of Education), Ann Payne and Breanna Squire (Data Recognition Corp), Lizabeth Frischhertz (East Baton Rouge Parish School System), and Richard Reeves (National Student Clearinghouse) for considerable help in assembling the data. The National Science Foundation and the US Department of Educations’ Institute for Education Sciences (IES R305A080132) provided generous funding. Data are provided by the Louisiana Department of Education, the East Baton Rouge School District, and the National Student Clearinghouse. To comment on this article in the online discussion forum, or to view additional materials, visit the article page at http://dx.doi.org/10.1257/app.4.1.109. When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita By Bruce Sacerdote* I examine long-term academic performance and college going for students affected by Hurricanes Katrina and Rita. Students who are forced to switch schools due to the hurricanes experience sharp declines in test scores in the first year following the hurricanes. However, by the third and fourth years after the disaster, evacuees displaced from Orleans Parish see a 0.18 standard deviation improve- ment in scores. Gains are concentrated among students initially in the lowest quintiles of the test score distribution. Katrina evacuees do not show gains in college going relative to earlier cohorts from their same pre-hurricane high schools. (JEL I20, Q54, R23).

Transcript of When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term...

Page 1: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

109

American Economic Journal: Applied Economics 2012, 4(1): 109–135 http://dx.doi.org/10.1257/app.4.1.109

I think the best thing that happened to the education system in New Orleans was Hurricane Katrina.

— Arne Duncan, US Secretary of Education, January 2010.1

One of the many effects of Hurricane Katrina was to close temporarily and later to force the reconstitution of schools in one of the nation’s worst performing

urban school districts. Many, including US Secretary of Education Arne Duncan, have speculated that there may be a silver lining to the disaster in that students were forced to exit a poorly performing school system or that adults were forced to exit a labor market with limited opportunities.2 Many New Orleans evacuees migrated permanently to different and, in most cases, better performing schools, while other evacuees returned to the newly formed Recovery School District in New Orleans. This paper is intended to be a thorough examination of long-run test score and col-lege going outcomes for the evacuees.

Hurricane Katrina was one of the worst natural disasters in United States his-tory. Roughly 1,900 deaths have been blamed on Katrina and estimates of the

1 Anderson (2010). “Education Secretary Duncan Calls Hurricane Katrina Good For New Orleans Schools.” Washington Post, January 30.

2 Vigdor (2007) finds little support for the latter hypothesis.

* Department of Economics, Dartmouth College, 6106 Rockefeller, Hanover, NH 03755 (e-mail: [email protected]). I thank Joshua Angrist, Amitabh Chandra, Alan Gustman, Doug Staiger, and seminar participants at NBER Summer Institute, RAND, Louisiana State University, Harvard University, Cornell University, and University of California-Santa Barbara for helpful suggestions. I thank Sam Farnham and Celia Kujala for outstanding research assis-tance. I thank Fen Chou (Louisiana Department of Education), Ann Payne and Breanna Squire (Data Recognition Corp), Lizabeth Frischhertz (East Baton Rouge Parish School System), and Richard Reeves (National Student Clearinghouse) for considerable help in assembling the data. The National Science Foundation and the US Department of Educations’ Institute for Education Sciences (IES R305A080132) provided generous funding. Data are provided by the Louisiana Department of Education, the East Baton Rouge School District, and the National Student Clearinghouse.

† To comment on this article in the online discussion forum, or to view additional materials, visit the article page at http://dx.doi.org/10.1257/app.4.1.109.

When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita†

By Bruce Sacerdote*

I examine long-term academic performance and college going for students affected by Hurricanes Katrina and Rita. Students who are forced to switch schools due to the hurricanes experience sharp declines in test scores in the first year following the hurricanes. However, by the third and fourth years after the disaster, evacuees displaced from Orleans Parish see a 0.18 standard deviation improve-ment in scores. Gains are concentrated among students initially in the lowest quintiles of the test score distribution. Katrina evacuees do not show gains in college going relative to earlier cohorts from their same pre-hurricane high schools. (JEL I20, Q54, R23).

ContentsWhen the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109

I. Existing Literature on Hurricane Katrina 111II. Empirical Framework 112A. Analysis of College-Going 115III. Data Description 116IV. Results 121V. Discussion and Conclusion 130Appendix 132References 133

Page 2: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

110 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

damage to homes and infrastructure was approximately $80 billion. As of 2008, $60 billion in Federal money had been allocated for disaster relief and recov-ery.3 Because Katrina destroyed so many homes and flooded 80 percent of New Orleans, nearly 1 million people were displaced from their homes. Thirty-five thousand Katrina evacuees relocated to Houston, Texas, while 24,000 relocated to Mobile, Alabama, and 15,000 people moved to Baton Rouge, Lousiana. Rand Corporation estimates that of Louisiana’s 760,000 public school students (pre-Katrina and Rita), 196,000 were temporarily or permanently displaced from their schools (Pane et al. 2006).

This paper is among the first attempts at analyzing the long-term effects of dis-location from Hurricanes Katrina and Rita on student achievement and college going.4 Test score data are provided by the Louisiana Department of Education and include reading and math test scores, basic student demographics, school and school district for each student in each year, and whether or not the student was displaced by Hurricanes Katrina or Rita. In addition, data from the National Student Clearinghouse are used to track five complete cohorts of high school students who reached high school graduation age pre-and post-Katrina.

The existing literature suggests at least two different effects that may be at work. The literature on the disruptive effects of moving schools (e.g. Hanushek, Kain and Rivkin 2004; Alexander, Dauber Norc, and Entwistle 1996) would sug-gest modest sized negative effects from switching schools. Having one’s family displaced by a hurricane is likely far more disruptive than a conventional move between schools. Vigdor (2007) estimates that adult evacuees lost, on average, three weeks of work, and that adult evacuees who did not return home lost closer to ten weeks of work. Pane et al (2006)finds that the median student evacuee missed five weeks of school.

Second, the literature on school and teacher quality (for example Rivkin, Hanushek, and Kain 2005; Kane, Staiger, and Rockoff 2008; Hoxby and Rockoff 2004; Hoxby and Murarka 2009; Betts, Zau, and Rice 2003; Clotfelter, Ladd, and Vigdor 2006) suggests that some New Orleans natives could actually benefit from being forced to move from low-performing schools. Interestingly, the test score gains I observe for Orleans evacuees are quantitatively similar to the gains that other researchers observe for students who win a lottery to attend a high value added char-ter school (e.g. Abdulkadiroglu et al. 2009).5

On average, Katrina evacuees from Orleans Parish experienced many sources of disruption and a large increase in school quality as measured by average test scores. Following the hurricanes, the Orleans Parish evacuees exited schools that had math test scores 0.45 standard deviations below the state average and enrolled in schools that were only 0.08 standard deviations below the state average. Roughly one-third of the Orleans Parish evacuees relocated permanently to a school district within

3 Department of Homeland Security. http://www.dhs.gov/xprepresp/programs/gc_1157649340100.shtm.4 Imberman, Kugler, and Sacerdote (2009) examine peer effects on students in schools that receive evacuees.5 And one might compare and contrast the effects here to those observed in the Moving to Opportunity experi-

ment (Katz, Kling, and Liebman 2001; and Ludwig, Duncan, and Hirschfield 2001).

Page 3: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 111SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

Louisiana but outside the New Orleans MSA, while another third relocated tempo-rarily and then return to the Recovery School District. Rita evacuees and Katrina evacuees from New Orleans suburbs (Jefferson, Saint Bernard, Saint Tammany, Plaquemines Parishes) experience similar schooling disruptions in the first weeks following the hurricanes, but the majority of Rita and suburban Katrina evacuees eventually return to their original school district.

In the spring of 2006, following the hurricanes, I find reasonably large (0.07 to 0.20 standard deviation) declines in test scores for all students who are displaced by the hurricanes.6 These results are for students who remain in Louisiana pub-lic schools and hence remain in my test score dataset. Through 2009, and in the hardest hit areas, including New Orleans, about 40 percent of the students exited the Louisiana school system. For students from suburban New Orleans (Jefferson Parish) and Lake Charles (in Calcasieu Parish), the negative effect disappeared gradually during 2007 and 2008.

Students from Orleans Parish saw the negative effect of Hurricane Katrina mod-erate by 2007. By 2008, these students were experiencing gains of 0.05 to 0.07 stan-dard deviations relative to their pre-hurricane test score. And by 2009, these gains widened to 0.18. This suggests that long-run academic achievement for Orleans students may have been improved by the forced move. I conducted several exercises to ask whether these results could be explained by attrition from the sample, and I found no evidence that sample selection was driving the results.

Orleans Parish students do not see a corresponding gain in college going. There is a modest negative effect on the likelihood that these students attend four-year colleges and a 1–4 percentage point drop in the rate of attending any college. The latter effect may be explained by relocation away from the established relation-ships between high schools and two-year colleges in New Orleans. The dichotomy between test score and college-going results may stem from the fact that the students who gain the most in test scores from moving schools were not the ones on the mar-gin of attending college.

I. Existing Literature on Hurricane Katrina

Several papers have examined labor market effects from Hurricane Katrina. One of the most in depth is Vigdor (2007), which asks whether evacuees benefit in the medium run from being forced out of New Orleans, which was high in poverty and unemployment, and had lower income than other cities in the South. This hypothesis is similar in spirit to the investigations of the Moving to Opportunity Program (F. Katz, Kling, and Liebman 2001) and the demolitions of public housing in Chicago (Jacob 2004). Vigdor (2007) finds that evacuees are hurt by the dislocation, both in terms of income and weeks worked. Zissimopoulos and Karoly (2007) find that one year after the storms, adult evacuees who relocate permanently have lower rates of employment than those who return. And all evacuees have higher rates of

6 These declines are relative to all other Louisiana students in the same grade. This effect is roughly one to two times the size of being assigned a teacher with test score value added that is one standard deviation below the mean of all teachers in a state. (Kane, Rockoff, and Staiger 2008; and Hanushek et al. 2005).

Page 4: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

112 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

self-employment relative to nonevacuees. McIntosh (2008) finds that the in-migra-tion of evacuees hurt native wages and employment in Houston, Texas.

Several studies, including Paxson and Rouse (2008), Groen and Polivka (2008), and Sastry (2007), investigate patterns of population movement caused by Katrina. Katrina has reduced the size of New Orleans proper from roughly 480,000 to about 258,000 (as of the 2008 American Community Survey). Paxson and Rouse (2008) find that whites and homeowners were the most likely to return. Those whose homes faced the worst flooding were the least likely to return.

Most relevant for this paper are two Rand Corporation studies by Pane et al. (2006, 2008) that document the number of displaced students, where they went, and how many days of school were lost. The first study finds that 196,000 public school students in Louisiana were displaced. This represents roughly one quarter of Louisiana’s total enrollment. About 81 percent of the evacuees came from just three parishes (Orleans, Jefferson, and Calcasieu). Orleans is coterminous with the city of New Orleans. Jefferson contains much of the suburban portion of the New Orleans metro area and includes 21 cities, towns, and unincorporated areas. Calcasieu con-tains the city of Lake Charles and is in the southwest corner of New Orleans, which was devastated by Hurricane Rita.

Pane et al. (2006) show that the median evacuee missed five weeks of school. Thirty-eight percent of evacuees were out of school and then returned to their origi-nal school. Thirty-one percent relocated to another Louisiana school, while another 31 percent disappeared from the dataset. Pane et al. (2008) study one-year effects on attendance, mental health, and test scores. They find one-year effects from disrup-tion on test scores similar to the effects I find.

II. Empirical Framework

My goal is to estimate the effect of the hurricanes on the academic performance and college-going of the evacuees. I ask how this effect varied for different types of evacuees and by their new location. Large numbers of students left the state as a result of the hurricanes. My main analysis of test score changes is focused on stu-dents remaining in Louisiana, but I do have specifications in which I include data for evacuees who arrive in Houston. My analysis for college-going is for all students regardless of post-hurricane location.

The hurricanes did not hit a random set of students but instead Katrina affected a group of students who were disproportionately poor and low scoring while Rita affected a group of students who were disproportionately richer and higher scoring. I use three related strategies to investigate the impacts of sample selec-tion on my results. First, I reweight my regressions by the estimated propensity to leave the sample. Second, I show that attrition from the sample is unrelated to pre-hurricane test score growth (even though for some groups attrition is related to past test score levels). Within Orleans Parish, attrition is not strongly correlated with levels of test scores. Finally and perhaps most usefully, I directly remove about half of the attrition problem by adding back in evacuees who arrive in Houston. Adding these evacuees to the sample strengthens slightly, rather than weakens, my results.

Page 5: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 113SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

My simplest OLS specification asks how the test scores of the eventual evacuees vary over time. Define Si and yi to be, respectively, the test score and a dummy for college-going for student i. S i ′ is student i’s pre-hurricane test score. I standardize test scores at the level of the state × year × grade. Xi is a vector of student-level characteristics and includes dummies for black, Asian, Hispanic, male, and free lunch status and a constant term. Gi is a set of grade dummies. Dt is a set of year dummies, and εi is the error term. Oi is an indicator variable for ever being an evac-uee from Orleans Parish, Ri is an indicator for ever being a Rita evacuee, and Ni is an indicator for being a Katrina evacuee from suburban New Orleans (which includes Jefferson, St. Tammany, St. Bernard, and Plaquemines Parishes). For the purposes of the college-going analysis, I further define for each school j, O j ′ , R j ′ , and N j ′ as indicator variables for a student’s school being evacuated in 2005 for Katrina in Orleans Parish, for Rita, or for Katrina in suburban New Orleans.

I run the following regression for each year t in the dataset:

(1A) Si = β1Oi + β2 Ni + β3 Ri + Xiγ + Gi λ + εi.

This is for student i in year t. Standard errors are clustered at the level of the cur-rent school, but my results are robust to clustering at the level of the pre-hurricane school.

β1, β2, and β3 tell me the relative position of three different types of evacuees within the test score distribution in a given year. I then look across the six regres-sions for each year in the period 2002–2009 and examine the pattern in coefficients.

I also run a difference-in-differences regression in which I stack the data and estimate the effects of the hurricane as the interaction between each post-hurricane year and eventual evacuee status:

(1B) Sit = ∑ m=2006

2009

H t m ( β 7

m Oi + β 8 m Ni + β 9

m Ri ) + β4Oi

+ β5 Ni + β6 Ri + Xi γ + Git λ + Dt τ + εi.

Here, t indexes years and H t m is a dummy variable equal to one when m = t. Dt are

a full set of year effects. β4, β5 , and β6 measure the baseline effects of being an eventual Orleans, non-Orleans, or Rita evacuee, while β 7

m , β 8 m , and β 9

m capture the interactions between evacuee status and each post-hurricane year. In short, I am ask-ing how the test scores for evacuees change (pre- versus post-hurricane) relative to the change in test scores for all other students.

The advantage of the above approach is that I combine observations from sev-eral cohorts and use every observation for which I know evacuee status, rather than limiting myself to observations with complete sets of pre- and post-hurricane test scores. The disadvantage of the cross-sectional approach above is that I am allow-ing students to enter and exit the sample, and the mix of grades and cohorts varies

Page 6: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

114 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

in each year. For example, pre-hurricane, only students in grades 4, 8, and 10 were tested, whereas post-hurricane, all grades, 3–10, were tested.

A more sophisticated version of equation (1A) is to limit the sample to specific cohorts of evacuees that I observe both pre-Katrina and post-Katrina. I also limit the sample to students who are observed each year post-Katrina in order to eliminate the change in sample as a possible explanation for variation in the coefficients. I regress the growth in test scores (from baseline to each year t for t = 2006 to 2009) on student characteristics and dummies for evacuee status. Specifically, I run the following regression separately for each t = 2006 to 2009:7

(2) ΔSi = β10 Oi + β11 Ni + β12 Ri + Xi γ + Gi λ + εi.

A myriad of other related specifications are possible. For example, I have tried controlling for baseline score on the right-hand side as opposed to imposing a coef-ficient of one (i.e. subtracting as above) on the baseline score. I have tried pooling all of the years and interacting evacuee status with each post-Katrina year. I have also tried pooling all years and using student fixed effects. All these variations yield results that are quite similar to the results from equation (2). For details and results see Sacerdote (2008).

I run equation (2) for fixed cohorts of students over time. My two cohorts are the students who were fourth graders in Spring 2004 or Spring 2005. I have also run equation (2) for the 2003 cohort of fourth graders and the eighth grade of 2005 cohort, and the results are quite similar. Unfortunately, I can only follow the eighth graders for two years post hurricane, whereas I can follow the fourth graders for four post-hurricane years.

As mentioned above a large fraction of evacuees from Orleans Parish migrated to Houston, Texas and show up in the dataset provided by the Houston Independent School District.8 I do not have student identifiers in either dataset, so I simply add the Houston evacuees into the sample used for the difference-in-differences speci-fication in equation (1B). While Texas and Louisiana are using different standard-ized tests, both states provide a (normed) National Percentile Rank (NPR) for each student. For the evacuees in Houston, I take their NPR and normalize their scores by the mean and standard deviation of the NPR in Louisiana for each grade and year. This tells me where the evacuees in Houston place within the Louisiana test score distribution.

The Houston data contain 4,450 student × year observations for evacuees during 2006 and 2007. In my baseline specification, I assume that all evacuees observed in Houston are from Orleans but my results are robust to other assumptions. The Louisiana only sample contains 11,958 student × year observations for Orleans evacuees during 2006–2007. My calculations suggest that with zero attrition, the

7 The inclusion of grade effects is not essential since the data are already demeaned at this level. But I demeaned at the level of the whole dataset before imposing sample limitations.

8 These data are described extensively in Imberman, Kugler, and Sacerdote (2009).

Page 7: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 115SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

sample of Orleans evacuees would contain an additional 7,900 observations.9 Therefore, adding back in the Houston evacuees adds back 56 percent of the miss-ing observations.

A separate question that I explore is whether the effects on test scores differ by an evacuee’s location within Louisiana after the hurricane. Specifically, I allow the effects for Orleans evacuees to vary by whether they return to the City of New Orleans, reside in suburban New Orleans, or exit the MSA. Of course choice of location is endogenous. In an attempt to overcome this endogeneity problem, I instrument for exiting the New Orleans MSA with the severity of hurricane and flood damage sustained by an evacuee’s school. This proxies for the damage level in an evacuee’s immediate home area. The instrument is from Federal Emergency Management Agency assessments and is available on a very localized level. Because I have, at most, one instrument, I only instrument for being an Orleans evacuee who is out of the MSA, rather than trying to instrument for multiple pos-sible locations.

A. Analysis of college-Going

In addition to test score data, I have college-going outcomes for all students in five cohorts of tenth graders and five cohorts of eighth graders. One major advantage of these data is that they track students across state lines and school districts so that (unlike the test score data) evacuees who left Louisiana are included.

For this analysis, I classify students as evacuees based on which high school they attended and when. I estimate the effect of the hurricanes as a standard difference-in-differences in which I compare the within school difference (pre and post) for evacuated schools to the within school difference (pre and post) for nonevacuated schools. Specifically, I regress a dummy for college going status on cohort effects, high school fixed effects, and the interaction between being from a cohort that evac-uated and a high school that evacuated.

I begin with the tenth grade cohorts of 2001–2005. The first three cohorts were scheduled to graduate pre-hurricane, and the second two cohorts were schedule to graduate post-hurricane. All cohorts are observed in tenth grade pre-hurricane in their pre-hurricane school. Using all students in these five cohorts, I run the follow-ing regression:

(3) yijk = Ak (β13 O j ′ + β14 N j ′ + β15 R j ′ )

+ Xijk γ + S ijk ′ χ + ηj + ρk + εijk.

Here, i indexes students, j indexes schools, and k indexes cohorts. The set of school effects is represented by ηj , and the set of cohort effects is represented by ρk . Ak is an indicator variable which takes the value one, if the cohort is the tenth grade cohort

9 This is based on a long-run attrition rate of 40 percent. The long-run attrition rate shown in Table 2 is 37 percent.

Page 8: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

116 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

of 2004 or 2005, meaning that the cohort graduated after the hurricanes. Ak interacts with the dummies for a school being evacuated as part of Rita, Orleans, or suburban New Orleans evacuations (i.e., R j ′ O j ′ , and N j ′ ). I control for tenth grade test score S i ′ using a cubic polynomial in test score.

Robust standard errors are clustered at the school × year level, but results are robust to clustering at the school × treatment level or school level. I report results for linear probability model, but results from probit specifications are similar.

I also run the analogous regression for the eighth grade cohorts. The eighth grade cohort of 2001 is the only pre-hurricane cohort, while the remaining eighth grade cohorts graduate post-hurricane.

III. Data Description

My dataset contains student level test scores, demographics, and college going outcomes for Louisiana public school students during the period 2000–2009. Pre-hurricane, I observe students in grades 4, 8, and 10; and post-hurricane I observe students in all of grades, 3–10. Under Louisiana’s accountability program, students in grades 4, 8, and 10 are tested in March of each year. These tests are known as the LEAP, or Louisiana Educational Assessment Program (grades 4 and 8), and the GEE, or Graduation Exit Examination. The subjects tested include math and English Language Arts (ELA) for grades 4, 8, and 10. Science and social studies are tested in grades 4, 8, and 11. For the sake of brevity, I do not report results for sci-ence and social studies tests. In spring 2006, tests known as the ILEAP (Integrated Louisiana Educational Assessment Program) were added for grades 3, 5, 6, 7, and 9. (The Iowa Test of Basic Skills was previously used for these students. I do not have the Iowa test scores.) Students in these five grades are tested in both math and English language arts. Students in grades 3, 5, 6, and 7 are tested in science and social studies. The tests in the ILEAP grades do not have a high-stakes compo-nent at the student level.

I have a randomly generated ID number which allows me to link a given student across years in the dataset. For the spring of 2006, I have a field that tells me which students are evacuees and whether they were displaced from a public school or pri-vate school, and whether they were displaced by Katrina or Rita. This was collected by teachers and principals and then reported to the state at the time the exams were taken. For each year, I know a student’s school and district, race, gender, and free lunch status.

Students are coded as displaced if the hurricane forced the closing of their school during September 2005 or the hurricane forced the family to move, or both. Nearly all students displaced from Orleans Parish are in a different school and district in the spring of 2006. In contrast, most of the students displaced by Rita (in Cameron and Calcasieu Parishes) and many of those displaced from a suburban New Orleans parish (Jefferson, St. Tammany, St. Bernard, Plaquemines) returned to their original school district by Spring 2006. A student is counted as an evacuee (equivalently “displaced”) even if they returned to their original school within a few weeks after the hurricane.

Page 9: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 117SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

Figure 1 shows the location of the evacuee students in 2006. The evacuees are sprinkled throughout Louisiana with concentrations in suburban New Orleans, Baton Rouge, and Shreveport. The parishes most affected by Hurricane Katrina are Orleans, Jefferson, Plaquemines, Saint Tammany, and Saint Bernard. These parishes comprise the New Orleans Metropolitan Statistical Area. Post-hurricane, the count of evacuees in grades 4, 8, and 10 in Jefferson Parish grew by about 1,200 evacuees, while East Baton Rouge School District gained about 1,000 of these evacuees. This implies that East Baton Rouge gained roughly 3,300 student evacuees in all grades. The remaining school districts in the state each gained 0–150 evacuees. The number of evacuees in Orleans itself shrunk dramatically post-Katrina. The Recovery School District (RSD) was set up to administer most of the schools in the former Orleans Parish School District. The RSD had roughly 1,100 fourth, eighth, and tenth graders by 2007.

My main analysis sample is constructed by taking all students in the test score database (2003–2009) and limiting the data to students observed in 2006 since that is the year during which the Louisiana Department of Education required schools to provide information on a student’s evacuee status. To run equation (2), I then further limit the sample to students also observed in grade 4 during 2004 or 2005 and each of the post-Katrina years (2006–2009). In all cases, in the tables and text, when I refer to a single year, I mean March of that year. Hence, 2005 refers to March 2005, which is the spring of the 2004 –2005 school year. All references to 2005 test score data are pre-hurricane.

In Table 1, I show summary statistics at the student level for both fourth grade cohorts during 2006. There are 60,860 students originating from these two cohorts.

Figure 1. Number of Evacuees by School in 2006

0 – 5

6 – 17

18 – 35

36 – 60

61 – 115

116 – 191

192 – 257

258 – 512

Page 10: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

118 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

I also show summary statistics separately for the Orleans, suburban Katrina, and Rita evacuees. About 8 percent of the students in the fourth grade sample are Katrina evacuees, with about one-third of those originating from Orleans Parish. Roughly 3,600, or 5.9 percent, of the former fourth graders are Rita evacuees. Forty-two per-cent of all students are black, while 90 percent of the Orleans evacuees are black and 28.5 percent of the Rita evacuees are black.

I standardize math and ELA scores to be mean zero, standard deviation one within each year and grade. The standardization is performed using the entire state and before limiting the sample. Overall, the Orleans evacuees have math scores that are 0.49 standard deviations below the state average and ELA scores that are 0.46 standard deviations below the state average. The Rita evacuees have math scores that are 0.23 standard deviations above the state average and ELA scores that are 0.28 standard deviations above the state average. Sacerdote (2008) has more detail about how average test scores vary by school district over time. Pre-hurricane, Jefferson Parish had test scores that were 0.18 standard deviations below the state average, while Plaquemines and St. Tammany were substantially above the state averages.

Table 1—Student Level Summary Statistics for 2006

Entirestate

Orleansevacuees

Suburban Katrina evacuees

Ritaevacuees

Variable Mean SD Mean SD Mean SD Mean SD

Free lunch eligible 0.583 0.493 0.579 0.494 0.749 0.434 0.541 0.498Male 0.488 0.500 0.472 0.499 0.491 0.500 0.500 0.500Student is black 0.422 0.494 0.904 0.295 0.394 0.489 0.285 0.451Student is hispanic 0.017 0.128 0.022 0.147 0.084 0.277 0.007 0.083Student is asian 0.012 0.109 0.025 0.156 0.052 0.222 0.005 0.069Math score (standardized) 0.150 0.944 −0.486 0.976 0.019 0.932 0.231 0.875English language arts score (standardized)

0.154 0.924 −0.457 0.961 0.090 0.936 0.282 0.855

Math score 03–05 0.310 0.858 −0.136 0.936 0.286 0.861 0.447 0.768ELA score 03–05 0.307 0.852 −0.094 0.958 0.270 0.867 0.479 0.755Raw math score 36.632 10.131 30.282 10.843 35.062 9.949 37.220 9.332Raw ELA score 61.885 15.641 52.142 15.995 61.140 15.443 64.226 14.163School average ’05 math score (in 2005)

0.076 0.398 −0.448 0.455 0.004 0.367 0.204 0.301

School average ’05 math score (in 2006)

0.139 0.332 −0.078 0.356 0.049 0.298 0.250 0.279

Percent college grads (near 2005 school)

0.108 0.064 0.132 0.067 0.123 0.062 0.118 0.050

Percent college grads (near 2006 school)

0.107 0.062 0.143 0.055 0.117 0.059 0.115 0.050

Percent black (near 2005 school)

0.285 0.235 0.683 0.203 0.231 0.167 0.190 0.208

Percent black (near 2006 school)

0.272 0.220 0.403 0.239 0.233 0.163 0.171 0.194

Med family income (near 2005 school)

39,625 11,120 33,376 12,068 44,641 8,997 42,758 9,909

Med family income (near 2006 school)

40,179 11,225 40,747 12,850 44,877 9,657 43,484 9,888

Notes: This table shows the means for student-level observations in the analysis sample for 2006. These are for the two cohorts of fourth graders (spring 2004–2005) who remain in Louisiana public schools for each spring 2006–2009. Means are shown for all students (across the state), Orleans evacuees, suburban (non-Orleans parish) Katrina evacuees, and Rita evacuees. Sample sizes are 60,860, 1,446, 3,459, and 3,600, respectively. School average math scores are measured for each school in 2005. I show the school average 2005 score for a student’s 2005 school and a student’s 2006 school.

Page 11: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 119SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

I report school average scores for a student’s pre-hurricane and post-hurricane schools. The school averages are all computed using 2005 data. Pre-Katrina Orleans students were attending schools in which the average math score was 0.45 standard deviations below the state average. After the hurricane, these same evacuees attended schools with math scores only 0.08 standard deviations below the state average. In contrast, the Rita evacuees and suburban Katrina evacuees saw much more modest changes in average school test scores. I also report the median family income for the zip code of a student’s pre-hurricane and post-hurricane school. Both figures use census 2000 data. Pre-hurricane, Orleans evacuees were in areas with a median family income of $33,000, which rises to $40,000 after the hurricane.

Katrina evacuees are more likely to disappear from the Louisiana public school sample relative to nonevacuees. Roughly 64 percent of the evacuees appear at some point in the post-Katrina sample, versus roughly 80 percent for all other students. In Table 2, I show mean characteristics for Orleans fourth graders in 2004 and 2005, and I split the sample by whether or not the student disappears from the test score data entirely or ever returns by 2009. Gender, race, and free lunch status are mod-estly related to exiting the dataset. For example, 94.4 percent of the Orleans students who remain are black, versus 95.3 percent of students who exit. However, the rela-tionship between test score levels and attrition is a bit weaker. The Orleans students who remain have baseline math scores 0.50 standard deviations below the state aver-age, while the Orleans students who exit have baseline scores 0.52 standard devia-tions below the state average. The t-statistic for this difference is 1.22.

In results not reported, I ask whether exiting the dataset is related to past growth in test scores for Orleans students. Given the limited pre-hurricane data (grades 4, 8, and 10 are tested), I can best ask this question for students who are in fourth grade in 2000 or 2001. Most of these students take the eighth grade exam before the hurricanes. I find that pre-hurricane growth in test scores is unrelated to attrition. When I regress growth in math scores on a dummy variable for remaining in the

Table 2—Means for Orleans Students Who Leave versus Stay Post Katrina

Return Exit

Variable Mean SD Mean SD t-test for difference

Free lunch eligible 0.892 0.310 0.879 0.326 −2.33Male 0.512 0.500 0.530 0.499 1.94Student is black 0.944 0.230 0.953 0.212 2.17Student is hispanic 0.012 0.108 0.011 0.103 −0.56Student is asian 0.014 0.116 0.011 0.103 −1.43Math score (standardized) −0.497 1.030 −0.521 1.054 −1.22English language arts score (standardized)

−0.462 1.046 −0.472 1.061 −0.52

Average math score at school

−0.544 0.439 −0.539 0.433 0.65

Notes: I identify all fourth graders in Orleans schools in the 2004 and 2005 cohorts. I then look at means for those who ever reappear post-hurricane versus those who exit the dataset permanently. This corresponds to the sample selection rule used for equation (1A) and Figures 2 and 3. There are 8,257 Orleans students in these cohorts who return to Louisiana Public Schools and 4,644 who exit.

Page 12: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

120 AmericAn economic JournAl: Applied economics JAnuAry 2012

sample, the coefficient on remaining in the sample is a modest 0.014 with a stan-dard error of 0.034. For growth in English Language Arts scores, the coefficient is 0.012 with a standard error of 0.032.

As mentioned above, I also have data on college enrollments for all tenth and all eighth graders in Louisiana Public Schools during the period 2001–2005.10 The data are from the National Student Loan Clearinghouse database and were created in a collaboration between me, the Louisiana Board of Regents, the Louisiana’s Department of Education’s Office of Assessment and Accountability, and Data Recognition Corp, which organizes and warehouses certain portions of the student level data. The Clearinghouse data have a useful feature, which is that the data track students across school districts and state lines. Hence, all evacuees are included in my analysis whether they return to their initial district, switch to a private school, or move to Houston, Texas or any other part of the country.

My two outcomes are enrollment in any college and enrollment in a four-year col-lege. For the college-going analysis, I infer evacuee status from whether or not a stu-dent’s high school was closed following the hurricanes. Note that for many students I have their actual evacuee status as reported by their receiving school. However, I do not have this information for students who left the state or public school system, and I do not want to limit the analysis to students who remain. Summary statistics for a single cohort, the tenth grade of 2005, are reported in Table 3. The cohort

10 I have data on later cohorts too, but I limit the analysis here to pre-hurricane cohorts, some of which were affected by the hurricanes.

Table 3—Summary Statistics College Going for 2005 (Tenth Grade) Cohort

Variable Observations Mean SD Min Max

Enrolled in four year college 40,074 0.396 0.489 0.000 1.000Enrolled in any year college 40,074 0.551 0.497 0.000 1.000Katrina evacuee, not New Orleans 40,074 0.093 0.291 0.000 1.000New Orleans Katrina evacuee 40,074 0.089 0.285 0.000 1.000Rita evacuee 40,074 0.052 0.222 0.000 1.000Orleans evacuee moved out of Orleans parish

40,074 0.022 0.147 0.000 1.000

Orleans evacuee who returned 40,074 0.015 0.120 0.000 1.000Orleans evacuee who left Louisiana 40,074 0.053 0.223 0.000 1.000Katrina evacuee (not Orleans) moved districts

40,074 0.011 0.103 0.000 1.000

Katrina evacuee (not Orleans) who returned

40,074 0.051 0.220 0.000 1.000

Katrina evacuee (not Orleans) who left louisiana

40,074 0.032 0.175 0.000 1.000

Student is black 40,074 0.410 0.492 0.000 1.000Student is hispanic 40,074 0.018 0.131 0.000 1.000Student is asian 40,074 0.018 0.133 0.000 1.000Standardized value of math score (LEAP or ILEAP)

40,074 0.050 0.994 −3.193 4.376

Standardized value of ELA score (LEAP or ILEAP)

40,074 0.054 0.971 −4.368 3.255

notes: The data are the census of all tenth graders in Louisiana public schools. College-going outcomes are obtained by merging the Louisiana DOE data with the National Student Clearinghouse Database. College-going outcomes refer to “ever enrolled” as of January 2009. Evacuee status is inferred from cohort and high school.

Page 13: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 121SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

contains about 40,000 students and 39.6 percent of these students enroll in a four-year college by January 2010. Fifty-five percent enroll in any college covered by the Clearinghouse. And, 8.9 percent of students in the cohort are Katrina evacuees from Orleans Parish schools.

IV. Results

Figure 2 displays for math scores the results from estimating equation (1A), i.e. the repeated cross sections with Orleans and Rita evacuee status on the right-hand side. I include the dummy for suburban New Orleans in the regression but, for clarity, do not show this third line in the graph. In 2004 and 2005, control-ling for demographics, eventual Orleans evacuees have math test scores that are 0.19–0.16 standard deviations below the math scores of other Louisiana students. After the hurricanes, this gap widens to −0.30 standard deviations in 2006. The gap then narrows dramatically to −0.13 by 2007 and to −0.10 by 2008. The gap is actu-ally statistically insignificant by 2009. This suggests that the Orleans evacuees saw an initial decline of 0.10 standard deviations immediately following the hurricane. However, these students then made enough gains to eliminate their pre-hurricane disadvantage. The elimination of the test score gap is only true after controlling for race and free lunch status.

The Rita evacuees show a different pattern. Pre-hurricane, the Rita evacuees are about 0.04–0.05 standard deviations ahead of the rest of the state. They lose this

Figure 2. Repeated Cross-sectional “Effects” on Math Scores New Orleans versus Non-New Orleans Evacuees

Notes: I regress math scores (all grades) on dummies for eventual Katrina Evacuee Status. The latter is split by evacuees who are in Orleans Parish in 2004 or 2005 versus all others. The 2006 and 2007 scores are post hurri-cane. Students are tested in March of each year. Regressions include dummies for race, gender, free lunch status, and grade. Suburban Katrina evacuees are also included in each regression but the line is not shown for simplicity.

0.200

0.100

Rita evacuees

−0.500

−0.400

−0.300

−0.200

−0.100

0.0002002 2003 2004 2005 2006 2007 2008 2009

Orleans Parish evacuees

Page 14: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

122 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

advantage in the year immediately following the hurricane, with their test scores fall-ing by 0.08 standard deviations.11 The Rita evacuees then have slight gains in 2007 and 2008.

Figure 3 repeats this exercise using the English Language Arts scores. The pattern is quite similar to that observed for math scores. Before the hurricane, the Orleans evacuees are about −0.25 standard deviations below the state average and experi-ence a drop of 0.10 standard deviations in 2006. By 2007, the Orleans evacuees are ahead of where they started, and they make further gains in 2008. The Orleans evacuees end the period near the state average, meaning that they have eliminated the pre-hurricane gap. The Rita evacuees again start the period above the state aver-age and lose a portion of their advantage.

Certainly one possible interpretation of these facts is that the new schools for the New Orleans evacuees have such higher value added relative to the old schools that within two years the evacuees have more than made up for the large costs of the dislocation imposed by the hurricane. The Rita students do not see an increase pos-sibly because they receive only costs and no benefits from the disruption.

A concern with this interpretation is that test scores for students in Orleans may have been trending upward prior to the hurricane. Appendix Table 1 shows that this was not the case. I use the 2006 data to identify which schools will be closed due

11 The size of the 2006 decline for all three groups of evacuees is consistent with Pane et al.’s (2008) finding of a decline of −0.09 to −0.20 standard deviations.

Figure 3. Repeated Cross Sectional “Effects” on ELA Scores New Orleans versus Rita Evacuees

Notes: I regress English language arts scores (all grades) on dummies for eventual Orleans and Rita evacuee Status, race dummies, male, and free lunch status. The latter is split by evacuees who are in Orleans Parish in 2004 or 2005 versus all others. The 2006–2009 scores are post-hurricane. Students are tested in March of each year. Suburban Katrina evacuees are also included in each regression but the line is not shown for simplicity.

−0.600

−0.500

−0.400

−0.200

−0.100

0.000

0.200

2002 2003 2004 2005 2006 2007 2008 2009

0.100

Rita evacuees

Orleans Parish evacuees

−0.300

Page 15: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 123SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

to Katrina or Rita. I limit the sample to successive cohorts of fourth graders. I then examine test scores in these schools in each year from 2000 to 2005. This sample is based upon a student’s current school and that school’s future evacuation status. This differs from the main analysis sample since, in Appendix I, I use all students rather

Table 4—Baseline Results With and Without Houston Data

Math score (no Houston

evacuees)

Math score (with Houston

evacuees)

ELA score (no Houston

evacuees)

ELA score (with Houston

evacuees)(1) (2) (3) (4)

Orleans evacuee × year is 2006 −0.049 −0.046 −0.009 −0.024(0.057) (0.056) (0.063) (0.060)

Orleans evacuee × year is 2007 0.072 0.102 0.064 0.079(0.054) (0.052)* (0.063) (0.058)

Orleans evacuee × year is 2008 0.063 0.063 0.063 0.063(0.068) (0.068) (0.072) (0.072)

Orleans evacuee × year is 2009 0.149 0.149 0.159 0.159(0.063)* (0.063)* (0.063)* (0.063)*

New Orleans evacuee −0.200 −0.200 −0.291 −0.291 (0.063)** (0.063)** (0.066)** (0.066)**

Non Orleans Katrina evacuee × −0.099 −0.099 −0.065 −0.065 year is 2006 (0.026)** (0.026)** (0.026)* (0.026)*Non Orleans Katrina evacuee × −0.071 −0.071 −0.077 −0.077 year is 2007 (0.040)+ (0.039)+ (0.037)* (0.037)*Non Orleans Katrina evacuee × −0.003 −0.003 −0.032 −0.032 year is 2008 (0.049) (0.049) (0.047) (0.047)Non Orleans Katrina evacuee × 0.080 0.081 0.045 0.045 year is 2009 (0.045)+ (0.045)+ (0.044) (0.044)Non New Orleans evacuee −0.092 −0.092 −0.060 −0.060

(0.025)** (0.025)** (0.024)* (0.024)*Rita evacuee × year is 2006 −0.117 −0.117 −0.056 −0.056

(0.031)** (0.031)** (0.023)* (0.023)*Rita evacuee × year is 2007 −0.064 −0.064 −0.068 −0.068

(0.032)* (0.032)* (0.031)* (0.031)*Rita evacuee × year is 2008 −0.043 −0.043 −0.057 −0.057

(0.034) (0.034) (0.039) (0.039)Rita evacuee × year is 2009 −0.063 −0.063 −0.048 −0.048

(0.041) (0.041) (0.032) (0.032)Rita evacuee 0.088 0.088 0.071 0.071 public school due to Rita (0.033)** (0.033)** (0.029)* (0.029)*

Observations 763,887 768,337 755,737 760,187R2 0.1551 0.1555 0.1410 0.1420

Notes: I run equation (1B) using all students in the sample. The regressions include year dummies, dummies for eventual evacuee status (Orleans, Non-Orleans, Rita), and controls for race, gender, grade, and free lunch status. This equation estimates the effect of evacuee status as a difference-in-differences; the effect is measured as the change in test scores for evacuees from pre- to post-hurricanes relative to the same change for the nonevacuees, i.e. the rest of the state. Columns 1 and 3 are only for students in Louisiana. Columns 2 and 4 add in test scores for evac-uees who are living and attending school in Houston. This provides an additional 4,450 student-year observations for evacuees. Both Houston, TX and Louisiana provide National Percentile Rankings for their students’ scores, and the Houston test scores are standardized using the Louisiana means and standard deviations. In this table, students are allowed to enter and exit the sample across years. The tables that follow this one hold the sample constant to the group of students observed in every single year in two specific cohorts.

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level.

Page 16: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

124 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

than those observed before and after the hurricane. Controlling for race and free lunch status, math scores in the Orleans schools are consistently 0.23–0.29 standard deviations below the state average. The only anomalous year is 2002 when Orleans schools scored particularly poorly on both math and reading.12 If I stack these data and test for a differential trend in the Orleans schools, I find only a small one for math scores. The coefficient on the time trend (in years) for math scores for Orleans is 0.018 with a standard error of 0.009. The time trend for ELA scores is 0.025. That means that from 2005 to 2009, we might have expected to see an improvement of 0.07 in math scores, versus the 0.20 that we observed. More importantly this estimated “pre-trend” of 0.018 per year appears to be driven largely by a recovery from the anomalous dip in 2002, as opposed to being a long-term trend.

Table 4 contains the difference-in-differences estimates of the effects of hurri-canes via equation (1B). The results in Table 4, which pool the years, are generally similar to those already shown in Figures 2 and 3. Column 1 is for math scores and for the Louisiana sample. Pre-hurricane, the Orleans evacuees have math scores that are 0.20 standard deviations below the state average. In 2006, the first year after the hurricanes, the Orleans evacuees fall an additional 0.05 standard deviations in the math score distribution. By 2007 and 2008, the Orleans evacuees have statisti-cally insignificant gains relative to their initial position. By 2009, the evacuees are significantly ahead (by 0.15 standard deviations) of their baseline scores.

Column 2 adds to the sample an additional 4,450 student-year observations for evacuees located in Houston. This addition has little effect on the results. The math test score gain for 2007 rises a bit. Columns 3 and 4 are the difference-in-differences estimates for ELA scores with and without the evacuees in Houston. Adding the evacuees in Houston slightly lowers the Orleans evacuees’ scores for 2006 and slightly raises the evacuees’ scores for 2007, but neither change is statisti-cally significant.

Table 5 proceeds to the estimation of equation (2) in which I limit the sample to specific cohorts and identify the effects of evacuee status on test score growth for each of the four post-hurricane years. Table 5 includes results for math and ELA scores for cohorts that are in fourth grade in 2004 or 2005. All regressions include grade effects and controls for race, gender, and free lunch status. Each column is a regression for a different post-hurricane year. The upper panel of the table is for math scores. In 2006, immediately following the hurricane, the Orleans evacuees are 0.17 standard deviations behind where they started. By 2008, the Orleans evacuees are slightly (but statistically insignificantly) ahead of their original position in the test score distribution. By 2009, the evacuees, most of whom are now eighth and ninth graders, are 0.19 standard deviations ahead of where they started.

The lower panel is for ELA scores, and the pattern for Orleans evacuees is much the same. In the year immediately following the hurricanes, the Orleans evacuees fall by 0.17 standard deviations in the test score distribution. These students have a mod-est gain from baseline by 2008 and a large and statistically significant gain by 2009.

12 As of this writing, I have not yet found out what was unusual about 2002.

Page 17: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 125SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

Table 5 also reports separate coefficients for each post-hurricane year for the Rita evacuees and the non-Orleans (suburban) Katrina evacuees within the two cohorts. Consider first the effects on math scores. The fourth grade non-Orleans evacuees experience a 0.10 decline in scores in 2006. They eliminate most of the decline by 2008 to end the period with a statistically insignificant gain of 0.027 standard devia-tions. The fourth grade cohorts of Rita evacuees have an initial decline of 0.07 stan-dard deviations in 2006, and this gap is eliminated by 2008.

The patterns in test score growth for suburban Katrina and Rita evacuees for ELA scores are somewhat different than the pattern for math scores. Suburban Katrina evacuees experience only a small (0.024) test score decline in 2006, followed by a larger (0.07) decline in 2007. This effect disappears by 2008. The Rita evacuees also see a modest decline in ELA scores in 2006, which worsens in 2007 and is not eliminated by 2009.

Table 5—Effects of Katrina or Rita Displacement on Math and ELA scores Two Fourth Grade Cohorts Before and After Katrina

Math growth baseline to 2006

Math growth baseline to 2007

Math growth baseline to 2008

Math growth baseline to 2009

(1) (2) (3) (4)New Orleans evacuee −0.173 −0.047 0.052 0.185 in public school (0.042)** (0.046) (0.043) (0.039)**

Non New Orleans −0.104 −0.102 −0.032 0.027 in public school (0.020)** (0.025)** (0.026) (0.025)Rita evacuee −0.068 −0.019 0.006 −0.023 in public school (0.024)** (0.027) (0.026) (0.026)Constant −0.157 1.246 −0.558 0.457

(798.101) (0.266)** (0.048)** (0.054)**

Observations 59,211 59,211 59,211 59,211R2 0.1445 0.0679 0.0568 0.0420

English growth baseline to 2006

English growth baseline to 2007

English growth baseline to 2008

English growth baseline to 2009

(1) (2) (3) (4)New Orleans evacuee −0.174 −0.062 0.067 0.173 in public school (0.048)** (0.049) (0.049) (0.044)**

Non New Orleans −0.024 −0.077 −0.022 0.040 in public school (0.019) (0.018)** (0.016) (0.019)*Rita evacuee −0.060 −0.122 −0.103 −0.091 in public school (0.023)** (0.022)** (0.023)** (0.024)**

Constant −0.415 0.110 −0.417 0.378(0.047)** (107.463) (0.050)** (0.051)**

Observations 59,220 59,217 59,209 59,188R2 0.1112 0.0674 0.0503 0.0366

Notes: These are the estimates from equation (2) in the text. I follow two fixed cohorts of fourth graders over time from 2004 or 2005 through 2009. I distinguish between Katrina evacuees from Orleans Parish School District, Katrina evacuees from suburban districts, and Rita evacuees. All other Louisiana students are also included and are the omitted category. The dependent variable is the growth in standardized score from baseline (2004 or 2005) to the current year. Regressions include dummies for race, gender, year, grade, and free lunch. Test scores are stan-dardized within year and grade at the state level. Standard errors are clustered at the school level.

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level.

Page 18: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

126 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

One robustness check I perform is weighting the observations in my sample by the propensity to attrit. I have tried numerous specifications to estimate the propen-sity to exit the sample but have focused on using all the Orleans students (not just the fourth graders). I regress a dummy for exiting the sample on baseline test scores and demographics.

I have re-run my regressions weighted by the propensity to exit the sample. The results are quite similar to the unweighted results shown in Table 5. For example, in 2006, Orleans evacuees experience an initial drop in math test scores from a base-line of 0.17 standard deviations. However, by 2009 they have a gain of 0.20 standard deviations from baseline. The Orleans evacuees show a drop of 0.17 in ELA scores in 2006, which becomes a gain of 0.18 by 2009. If anything, my results strengthen slightly when I account for attrition.

Table 6—Test Score Gains For Evacuees By Initial Quintile of Performance

Math growth baseline to 2006

(quintiles 1 and 2)

Math growth baseline to 2009

(quintiles 1 and 2)

Math growth baseline to 2006

(quintiles 3 and 4)

Math growth baseline to 2009

(quintiles 3 and 4)(1) (2) (3) (4)

New Orleans evacuee −0.182 0.207 −0.233 0.118(0.036)** (0.035)** (0.070)** (0.050)*

Suburban Katrina evacuee −0.096 0.037 −0.130 0.026(0.021)** (0.027) (0.028)** (0.029)

Rita evacuee −0.059 −0.032 −0.064 −0.024(0.024)* (0.028) (0.028)* (0.026)

Constant −1.116 0.497 −0.754 −0.986(0.391)** (0.057)** (0.330)* (0.091)**

Observations 20,167 20,250 25,236 25,027R2 0.2376 0.0566 0.0575 0.0615

ELA growth baseline to 2006

(quintiles 1 and 2)

ELA growth baseline to 2009

(quintiles 1 and 2)

ELA growth baseline to 2006

(quintiles 3 and 4)

ELA growth baseline to 2009

(quintiles 3 and 4)(1) (2) (3) (4)

New Orleans evacuee −0.114 0.241 −0.280 0.062(0.030)** (0.036)** (0.071)** (0.061)

Suburban Katrina evacuee −0.021 0.056 −0.028 0.038(0.026) (0.026)* (0.025) (0.023)

Rita evacuee −0.041 −0.103 −0.029 −0.058(0.026) (0.029)** (0.028) (0.026)*

Constant −0.561 −0.324 −0.702 −0.775(0.199)** (0.080)** (0.080)** (0.103)**

Observations 19,979 20,100 25,493 25,299R2 0.1653 0.0502 0.0624 0.0623

Notes: These are estimates from equation (2) in the text. I split the sample by initial quintile of test score perfor-mance. I follow two fixed cohorts of students over time from 2004 or 2005 through 2009. I distinguish between Katrina evacuees from Orleans Parish School District, Katrina evacuees from suburban districts, and Rita evacuees. All other Louisiana students are also included and are the omitted category. The dependent variable is the growth in standardized score from baseline (2004 or 2005) to the current year. Regressions include dummies for race, gender, year, grade, and free lunch. Test scores are standardized within year and grade at the state level. Standard errors are clustered at the school level.

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level

Page 19: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 127SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

In Tables 6 and 7, I provide more detailed results on which Orleans evacuees showed the largest effects from the hurricanes. Table 6 divides the sample by quintiles of baseline (pre-hurricane) test scores and runs specification (2) for 2006 and 2009. Quintiles are defined using the statewide distribution of test scores, and Orleans evacuees are more likely to score in the bottom two quintiles. Column 1 examines math scores for fourth graders who were in quintiles 1 and 2 pre-Hurricane. The Orleans evacuees from these bottom two quintiles show an initial drop of 0.18 stan-dard deviations immediately after the hurricane. This changes to a gain of 0.21 standard deviations by 2009, however.

Column 3 limits the sample to fourth graders who were in the top two quintiles pre-hurricane. The Orleans evacuees from this group see only half as much gain in relative test scores by 2009 (0.12 instead of 0.21). When I look at ELA scores in the bottom panel, this pattern is even more pronounced. Orleans evacuees in the bottom two quin-tiles see a gain of 0.24 standard deviations by 2009. But Orleans evacuees in the top two quintiles have only a small (0.06) and statistically insignificant gain in test scores. The ELA gains are concentrated among evacuees who had the lowest baseline performance.

Table 7—Test Score Gains for Evacuees by Current Location

Math growth from baseline

to 2008 or 2009

ELA growth from baseline

to 2008 or 2009

Math growth from baseline (IV for leave

Orleans MSA)

ELA growth from baseline (IV for leave

Orleans MSA)

Math growth from baseline (reduced form)

ELA growth from baseline (reduced form)

(1) (2) (3) (4) (5) (6)Orleans evacuee currently 0.206 0.148 0.473 0.460 outside N.O. MSA (0.032)** (0.032)** (0.094)** (0.104)**

Orleans evacuee currently 0.155 0.255 in Orleans Parish district (0.062)* (0.040)**

Orleans evacuee currently 0.045 0.005 in recovery district (0.087) (0.070)Orleans evacuee currently 0.034 0.082 in N.O. suburban districts (0.071) (0.068)Suburban Katrina evacuee −0.000 0.005 0.001 0.006 −0.007 −0.002

(0.022) (0.014) (0.022) (0.013) (0.021) (0.013)Rita evacuee −0.014 −0.105 −0.014 −0.105 −0.105 −0.106

(0.021) (0.020)** (0.021) (0.020)** (0.021)** (0.020)**

FEMA damage assessment 0.026 0.026 to tract (0.005)** (0.005)**

Constant −0.783 0.030 0.687 −0.650(0.075)** (1,224.243) (0.099)** (0.073)**

Observations 125,202 125,222 125,202 125,222R2 0.040 0.037 0.039 0.035

Notes: I look at test score growth for two cohorts of fourth graders from (2004 or 2005) to (2008 or 2009). I divide the Orleans evacuees into those that left the MSA, those that are in the original (now very small) Orleans Parish School District, those that are in the Recovery School District, and those in suburban Orleans districts. Because location in 2008 or 2009 is endogenous, I instrument for leaving the MSA with the amount of hurricane damage experienced by the evacuee’s census tract. See the text for a description of the instrument. Columns 1 and 2 are OLS. Columns 3 and 4 are two stage least squares. Columns 5 and 6 are reduced form regressions of test score growth on the instrument. Regressions include dummies for race, gender, year, grade, and free lunch. Test scores are standardized within year and grade at the state level. Standard errors are clustered at the school level. Robust standard errors in parentheses.

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level

Page 20: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

128 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

In Table 7, I ask how the effects vary by an evacuee’s current (2008 or 2009) location. I again combine two cohorts of fourth graders. I run an expanded ver-sion of equation (2) in which I allow separate effects for evacuees who went to the Recovery District in Orleans, those that relocated in suburban New Orleans, and those that left the New Orleans MSA altogether. I regress growth from baseline in 2008 or 2009 test scores on demographic indicators, grade dummies, and dummies for the above categories for current location.

Large gains accrue to evacuees who left the New Orleans MSA. These evacuees have a gain of 0.21 standard deviations in math scores and 0.15 standard devia-tions in ELA scores. These students make up 35 percent of the 3,048 fourth grade evacuees in the sample. Evacuees who move to suburban New Orleans also show gains, but these are not statistically significant. These students are 15 percent of the sample. The evacuees who return to the Recovery District (32 percent of the sample) show the smallest test score gains by 2009.13

A major concern with the regressions in columns 1 and 2 of Table 7 is that fami-lies self-select into their 2008 or 2009 location. Those students destined to have better test score growth may have relocated permanently outside New Orleans. My partial solution to this endogeneity problem is to instrument for remaining outside the New Orleans MSA using the severity of hurricane damage at the location of the student’s initial school. Clearly this identification strategy may be flawed because a student’s initial location and damage experience may have a direct impact on test score growth that does not work through post-hurricane location.

13 In results not reported, I show that the weighted average of these separate effects by location equals the overall effect for the Orleans evacuees.

Table 7 Panel 2—First Stage Regressions of Exiting Orleans MSA on FEMA Damage Assessment

First stage Orleans evacuee currently outside N.O. MSA (whole sample)

First stage Orleansevacuee currently outside N.O. MSA

(Orleans evacuees only)(1) (2)

FEMA damage assessment 0.056 0.019 to tract (0.007)** (0.006)**

Suburban Katrina evacuee −0.018(0.003)**

Rita evacuee −0.002(0.001)*

Constant −0.043 −0.114(0.023)+ (0.099)

Observations 125,222 3,259R2 0.141 0.027

Notes: Column 1 is the first stage run for the sample and column 2 is the first stage limiting the sample to only Orleans evacuees. Regressions include dummies for race, gender, year, grade, and free lunch. Test scores are standardized within year and grade at the state level. Standard errors are clustered at the school level.

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level

Page 21: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 129SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

I use localized FEMA damage assessments, which are available on Google Earth. I use the following coding scheme: areas with no damage are coded as 0; light, moderate, extensive, and catastrophic are coded as 1–4, respectively; saturated areas are coded as 5; and completely flooded areas are coded as 6. While this conflat-ing of flood levels and damage seems unusual, these are the data as provided by FEMA. My IV results are robust to alternative coding strategies. The first-stage esti-mates are shown in panel 2 of Table 7. Column 1 shows the predictive power of the FEMA damage assessment just within the sample of Orleans evacuees. Even within Orleans, damage predicts exit from the MSA with a t-statistic of three. Within the whole sample, the t-statistic is eight. Since I have at most one instrument, I cannot instrument for all four possible endogenous location choices for Orleans evacuees. Therefore for the IV results I simplify the specification to distinguish only between Orleans evacuees who locate outside the New Orleans MSA relative to all other New Orleans evacuees.

The second-stage estimates are shown in columns 3 and 4 of Table 7. The IV estimates of gains from leaving the New Orleans MSA are 0.47 for math scores and 0.46 for ELA scores. These gains are larger than the OLS estimates, but the differ-ences are not statistically significant.14

A final concern is that perhaps test scores should not be standardized at the state level if evacuees are reducing their peers test scores, thereby making relative

14 Reduced-form effects of the instrument on test score growth are shown in columns 5 and 6 of the upper panel.

Table 8—Effect of Evacuee Status on Enrollment in A Four Year College

Enrolled in four-year college

tenth grade cohorts

Enrolled in any year college

tenth grade cohorts

Enrolled in four-year college

eighth grade cohorts

Enrolled in any year college

eighth grade cohorts

(1) (2) (3) (4)New Orleans Katrina −0.016 −0.042 0.011 −0.013 evacuee (0.010) (0.010)** (0.008) (0.009)Katrina evacuee, not 0.025 0.028 0.034 0.018 New Orleans (0.008)** (0.007)** (0.007)** (0.007)**

Rita evacuee 0.002 −0.025 0.055 0.032(0.010) (0.010)* (0.010)** (0.010)**

Observations 192,233 192,233 253,562 253,562R2 0.30 0.27 0.29 0.28

Notes: These estimates are from equation (3) in the text. All regressions include high school fixed effects and year dummies and controls for race and gender. The hurricane effect is estimated as the interaction between being a high school, which is evacuated times being in a cohort that graduates after 2005. Thus, the effect of the hurri-canes is estimated as a difference-in-differences. The effect is measured as the change in college going for cohorts that graduate pre- versus post-hurricane for high schools that evacuate relative to the pre- versus post-change for the high schools that do not evacuate. Samples include all tenth or all eighth graders in Louisiana during the years 2001–2005. The tenth graders have three pre-cohorts and two post-cohorts. The eighth graders have one pre-cohort and four post cohorts. All cohorts are initially observed pre-hurricane and in their pre-hurricane schools. All col-umns are OLS regressions, though probit regressions yield very similar results. Standard errors are clustered at the school × year level. Robust standard errors in parentheses

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level

Page 22: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

130 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

performance the wrong measure. This concern is addressed in Appendix Table 2, which runs equation (2) using growth in raw test scores rather than growth in stan-dardized test scores. The Orleans evacuees experience a raw math test score gain of 2.3 by 2009. The standard deviation of raw math test scores is 10, so an increase of 1.9 is consistent with the results shown in Table 5.

Effects on college Going.—In addition to test scores, I also consider whether col-lege enrollment rates for the evacuees are affected. One advantage of this analysis is the ability to follow all evacuees regardless of current location. A second benefit is that college-going is a longer term outcome than year-to-year variation in a stu-dent’s test taking ability.

My two outcome measures are dummy variables for enrolling in a four-year col-lege and for enrolling in any college. Table 8 shows estimates from equation (3). Again, this estimates the effect of the hurricanes as the difference in college-going for pre-hurricane and post-hurricane cohorts in evacuated schools relative to the same difference for nonevacuated schools. Using the tenth grade cohorts, Orleans evacuees experience a statistically insignificant 1.6 percentage point decline in atten-dance at four-year colleges. This is against a mean pre-hurricane attendance rate of about 27 percent. These same cohorts of Orleans evacuees experience a 4.2 percent-age point decline in the rate at which they attend any college, i.e. two- or four-year colleges. However, when we examine the students in the eighth grade cohorts, the decline in college attendance for the Orleans evacuees is not statistically significant.

The suburban Katrina evacuees see modest increases in college attendance relative to their predecessor cohorts from the same high schools. For example, using the tenth grade cohorts, suburban Katrina evacuees experience a gain of 2.5 percentage points in attendance at four-year colleges (compared to the base rate 29 percent). Within the eighth grade cohorts, the suburban Katrina evacuees see gains of 3.4 percentage points.

It is fairly easy to think of reasons why the disruption from Katrina may have harmed college going for the Orleans evacuees. In particular, the traditional links between Orleans students and local community colleges were likely disrupted and, in some cases, these colleges were not replaced by alternative institutions. Consider the tenth grade cohort of 2003 from all Orleans Parish high schools. Eight hundred seventy-three of these students attended a two year school. Five hundred seventy-nine (i.e., 66 percent) of those students went to Delgado Community College. In the subsequent (Katrina affected) cohort, only 770 students attended a two-year school, with 496 students enrolling in Delgado Community College.

It is more difficult to think of reasons why Katrina would modestly increase col-lege going for students in suburban schools. One possibility is that labor market disruptions caused students to choose college attendance over the labor force. Another, perhaps less likely, possibility is that admissions offices actively recruited evacuees in an effort to aid them.

V. Discussion and Conclusion

Hurricanes Katrina and Rita had significant impacts on the academic perfor-mance and college going of evacuees. In the first year following the hurricanes,

Page 23: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 131SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

Orleans evacuee math scores dropped 0.17 standard deviations relative to other Louisiana students. This is not terribly surprising given the massive disruptions caused by the hurricanes, and the fact that the median student lost around five weeks of school.

Perhaps what is more surprising is how quickly the Orleans Parish evacuees recov-ered from the experience and actually began to see gains in test scores. By 2008, the Orleans evacuees are doing as well academically as they were in 2004 and 2005. By 2009, the Orleans evacuees are about 0.18 standard deviations ahead of their base-line position in the test score distribution. One natural explanation is that the New Orleans schools were so deficient, that in the long run the New Orleans evacuees saw increased academic achievement as a result of being kicked out of their original schools. On average, the Orleans evacuees moved from schools with average test scores that were 0.45 standard deviations below the state average to schools that were only 0.08 standard deviations below average. And the median family income in their school’s zip code rose from $33,000 per year to $40,000 per year.

When I split the Orleans evacuees by current location, those that remain outside the New Orleans MSA have the largest test score gains. The gains for evacuees accrue to students who were initially in the bottom two quintiles of the test score distribution.

The positive long-run effects for Orleans evacuees are the same order of magni-tude as those found from moving a student to a successful charter school, to a smaller classroom, or to a teacher with value added that is one standard deviation higher. For example, Abdulkadiroglu et al. (2009) find that winning a charter school lottery in Boston raises ELA test scores by 0.20 standard deviations and math test scores by 0.40 standard deviations. Hoxby and Murarka (2009) examine charter schools in New York and find that across all charters, the average treatment effect from attend-ing one is 0.09 standard deviation in math and 0.04 standard deviations in english.15 And Dobbie and Fryer (2009) find that the charter schools in the Harlem Children’s Zone raise student achievement by 0.80 standard deviations in math over two years and 0.25 standard deviations in English over the same time period.

Kane, Rockoff, and Staiger (2008) find that a standard deviation of teacher effects in the Los Angeles Unified School district is about 0.219 in math and 0.175 in English Language Arts. Krueger and Whitmore (2001) find that being assigned to a small class in the Tennessee Star Experiment raises test scores by 5 percentile points within the distribution. In my data 5 percentile points on the national scale is approximately 0.22 standard deviations within Louisiana. So again, the long-run positive effects for Orleans evacuees are comparable to being assigned a terrific teacher or being assigned to a smaller class in the STAR experiment.16

Unfortunately gains to rates of college-going are not present for the Orleans evac-uees. There is a 0 to −1.6 percentage point effect on the rate of four-year college going for the Orleans evacuees, and they experience a −1.3 to −4.2 percentage

15 As Hanushek et al. (2007) point out, the missions, effects, and effectiveness of charter schools varies greatly. Indeed, Hoxby and Murarka (2009) find that more than half of NYC charter students attend a school that has a treatment effect on math that is between 0.1 and 0.3 standard deviations.

16 These are such different populations that I do not mean to suggest that all these studies are estimating the same parameter. Rather, I am saying that the effects seen here are of a similar magnitude to effects in these dif-ferent contexts.

Page 24: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

132 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

point drop in the rate of enrollment in any college. One sensible reconciliation of the different effects for test scores and college-going is that the students who experience much of the test score gains were not the ones on the margin of attending college. And the disruption of moving away from an available and known set of community colleges may have harmed two-year college enrollment for the Orleans evacuees.

This paper provides another example in which moving students out of schools with low average test scores and into better performing schools had a meaningful effect on test scores. Overall these results provide one of the first looks at how stu-dents were affected by one of the largest student relocations in US history.

Appendix

Table A1—Relative Performance of Students in Schools that Evacuate in the Future

Standardized value of

math score (2000)

Standardized value of

math score (2001)

Standardized value of

math score (2002)

Standardized value of

math score (2003)

Standardized value of

math score (2004)

Standardized value of

math score (2005)

(1) (2) (3) (4) (5) (6)Orleans −0.276 −0.290 −0.420 −0.332 −0.269 −0.229 school (0.061)** (0.058)** (0.049)** (0.048)** (0.052)** (0.051)**

Non-Orleans 0.003 0.070 0.108 −0.007 −0.026 −0.070 Katrina school (0.036) (0.041) (0.039)** (0.043) (0.040) (0.040)Rita school 0.044 0.118 0.070 0.054 0.060 0.041

(0.043) (0.037)** (0.044) (0.038) (0.037) (0.038)

Observations 55,494 61,304 59,870 58,323 55,338 56,882R2 0.2144 0.1933 0.2087 0.1999 0.1964 0.1715

Standardized value of

ELA score (2000)

Standardized value of

ELA score (2001)

Standardized value of

ELA score (2002)

Standardized value of

ELA score (2003)

Standardized value of

ELA score (2004)

Standardized value of

ELA score (2005)

(1) (2) (3) (4) (5) (6)Orleans −0.322 −0.290 −0.390 −0.346 −0.264 −0.257 school (0.066)** (0.057)** (0.052)** (0.052)** (0.052)** (0.051)**

Non-Orleans −0.028 −0.005 0.026 −0.007 −0.040 −0.055 Katrina school (0.028) (0.035) (0.029) (0.035) (0.038) (0.034)Rita school 0.146 0.187 0.135 0.093 0.106 0.104

(0.036)** (0.035)** (0.039)** (0.038)* (0.039)** (0.034)**

Observations 55,527 61,340 59,876 58,331 55,350 56,909R2 0.2155 0.1753 0.2150 0.1833 0.1826 0.1604

Notes: I limit the sample to successive cohorts of fourth graders. All regressions include controls for race, free lunch, and gender. These regressions are generated by running cross-sectional regressions on fourth graders for each year. This analysis includes all fourth graders in all schools and does not limit the sample to students who are observed post hurricane. Robust standard errors in parentheses

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level

Page 25: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 133SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

REFERENCES

Abdulkadiroglu, Atila, Joshua Angrist, Susan Dynarski, Thomas J. Kane, and Parag Pathak. 2009. “Accountability and Flexibility in Public Schools: Evidence from Boston’s Charters and Pilots.” National Bureau of Economic Research Working Paper 15549.

Alexander, Karl L., Susan L. Dauber Norc, and Doris R. Entwisle. 1996. “Children in Motion: School Transfers and Elementary School Performance.” Journal of Educational Research, 90(1): 3–12.

Anderson, Nick. 2010. “Education Secretary Duncan Calls Hurricane Katrina Good for New Orleans Schools.” the Washington Post, January 30.

Angrist, Joshua D., and Kevin Lang. 2002. “How Important are Classroom Peer Effects? Evidence from Boston’s Metco Program.” National Bureau of Economic Research Working Paper 9263.

Betts, Julian R., Andrew C. Zau, and Lorien A. Rice. 2003. determinants Of Student Achievement: New Evidence From San diego. San Francisco: Public Policy Institute of California.

Clotfelter, Charles T., Helen F. Ladd, and Jacob L. Vigdor. 2006. “Teacher-Student Matching and the Assessment of Teacher Effectiveness.” Journal of Human Resources, 41(4): 778–820.

Table A2—Results Using Raw Scores Rather Than Standardized

Math growth baseline to

2006

Math growth baseline to

2007

Math growth baseline to

2008

Math growth baseline to

2009(1) (2) (3) (4)

New Orleans evacuee −1.394 −0.354 0.733 2.310(0.480)** (0.538) (0.545) (0.517)**

Suburban Katrina evacuee −1.114 −1.143 −0.375 0.339(0.251)** (0.302)** (0.320) (0.324)

Rita evacuee −0.785 −0.206 0.116 −0.294(0.254)** (0.307) (0.321) (0.345)

Constant −14.774 14.914 −20.503 −0.504(14,057.824) (3.216)** (0.616)** (0.636)

Observations 59,211 59,211 59,211 59,211R2 0.4328 0.1737 0.3636 0.3649

English growth baseline to

2006

English growth baseline to

2007

English growth baseline to

2008

English growth baseline to

2009(1) (2) (3) (4)

New Orleans evacuee −3.685 −1.961 0.203 2.763(0.679)** (0.734)** (0.643) (0.686)**

Suburban Katrina evacuee −0.419 −1.288 −0.407 0.838(0.264) (0.299)** (0.261) (0.326)*

Rita evacuee −0.560 −1.739 −1.208 −1.401(0.388) (0.383)** (0.298)** (0.450)**

Constant 20.408 32.067 29.920 29.526(0.756)** (24,344.645) (0.971)** (0.707)**

Observations 59,220 59,217 59,209 59,188R2 0.2275 0.1755 0.5191 0.7152

Notes: These are the estimates from equation (2) in the text, but using raw scores rather than standardized scores. I follow two fixed cohorts of fourth graders over time from 2004 or 2005 through 2009. I distinguish between Katrina evacuees from Orleans Parish School District, Katrina evacuees from suburban districts, and Rita evacuees. All other Louisiana students in the cohorts are also included and are the omitted category. The dependent variable is the growth in raw score from baseline (2004 or 2005) to the current year. Regressions include dummies for race, gen-der, year, grade, and free lunch. Standard errors are clustered at the school level. Standard errors are in parentheses.

*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level

Page 26: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

134 AmERIcAN EcONOmIc JOuRNAL: APPLIEd EcONOmIcS JANuARy 2012

Clotfelter, Charles T., Helen F. Ladd, and Jacob L. Vigdor. 2007. “How and Why do Teacher Creden-tials Matter for Student Achievement?” National Bureau of Economic Research Working Paper 12828.

Dobbie, Will, and Roland G. Fryer Jr. 2009. “Are High Quality Schools Enough to Close the Achieve-ment Gap? Evidence from a Social Experiment in Harlem.” National Bureau of Economic Research Working Paper 15473.

Engec, Nacati. 2006. “Relationship between Mobility and Student Performance and Behavior.” Jour-nal of Educational Research, 99(3): 167–78.

Gabe, Thomas, Gene Falk, Maggie McCarty, and Virginia W. Mason. 2005. Hurricane Katrina: Social-demographic characteristics of Impacted Areas. Congressional Research Service (CRS) Report for Congress. Washington, DC, November.

Gordon, Robert, Thomas J. Kane, and Douglas O. Staiger. 2006. “Identifying Effective Teachers Using Performance On The Job.” Hamilton Project Discussion Paper 2006-01.

Groen, Jeffrey A., and Anne E. Polivka. 2008. “Hurricane Katrina Evacuees: Who They Are, Where They Are, and How They Are Faring.” monthly Labor Review, 131(3): 32–51.

Hanushek, Eric A., John F. Kain, Daniel O’Brien, and Steven G. Rivkin. 2005. “The Market for Teacher Quality.” National Bureau of Economic Research Working Paper 11154.

Hanushek, Eric A., John F. Kain, and Steven G. Rivkin. 2004. “Disruption versus Tiebout Improve-ment: The Costs and Benefits of Switching Schools.” Journal of Public Economics, 88(9–10): 1721–46.

Hanushek, Eric A., John F. Kain, Steven G. Rivkin, and Gregory F. Branch. 2007. “Charter School Quality and Parental Decision Making with School Choice.” Journal of Public Economics, 91(5–6): 823–48.

Harris, Douglas, and Tim R. Sass. 2006. “Value-Added Models and the Measurement of Teacher Qual-ity.” Unpublished.

Heinlein, Lisa Melman, and Marybeth Shinn. 2000. “School Mobility and Student Achievement in an Urban Setting.” Psychology in the Schools, 37(4): 349–57.

Hoxby, Caroline M., and Sonali Murarka. 2009. “Charter Schools in New York City: Who Enrolls and How They Affect Their Students’ Achievement. National Bureau of Economic Research Working Paper 14852.

Hoxby, Caroline M., and Jonah E. Rockoff. 2004. “The Impact of Charter Schools on Student Achieve-ment.” Unpublished.

Imberman, Scott, Adriana D. Kugler, and Bruce Sacerdote. 2009. “Katrina’s Children: Evidence on the Structure of Peer Effects from Hurricane Evacuees.” National Bureau of Economic Research Working Paper 15291.

Jacob, Brian A. 2004. “Public Housing, Housing Vouchers, and Student Achievement: Evidence from Public Housing Demolitions in Chicago.” American Economic Review, 94(1): 233–58.

Kane, Thomas J., Jonah E. Rockoff, and Douglas O. Staiger. 2008. “What Does Certification Tell Us about Teacher Effectiveness? Evidence from New York City.” Economics of Education Review, 27(6): 615–31.

Katz, Lawrence F., Jeffrey R. Kling, and Jeffrey B. Liebman. 2001. “Moving to Opportunity in Boston: Early Results of a Randomized Mobility Experiment.” Quarterly Journal of Economics, 116(2): 607–54.

Kling, Jeffrey R., Jens Ludwig, and Lawrence F. Katz. 2005. “Neighborhood Effects on Crime for Female and Male Youth: Evidence from a Randomized Housing Voucher Experiment.” Quarterly Journal of Economics, 120(1): 87–130.

Koedel, Cory, and Julian R. Betts. 2007. “Re-Examining the Role of Teacher Quality In the Educa-tional Production Function.” University of Missouri-Columbia Department of Economics Working Paper 07-08.

Krueger, Alan B., and Diane M. Whitmore. 2001. “The Effect of Attending a Small Class in the Early Grades on College-Test Taking and Middle School Test Results: Evidence from Project STAR.” Economic Journal, 111(468): 1–28.

Ludwig, Jens, Greg J. Duncan, and Paul Hirschfield. 2001. “Urban Poverty and Juvenile Crime: Evidence from a Randomized Housing-Mobility Experiment.” Quarterly Journal of Economics, 116(2): 655–79.

McIntosh, Molly Fifer. 2008. “Measuring the Labor Market Impacts of Hurricane Katrina Migration: Evidence from Houston, Texas.” American Economic Review, 98(2): 54–57.

Nelson, Perry S., Jane M. Simoni, and Howard S. Adelman. 1996. “Mobility and School Functioning in the Early Grades.” Journal of Educational Research, 89(6), 365–69.

Page 27: When the Saints Go Marching Out: Long-Term … · When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita† 109 I. Existing Literature

VOL. 4 NO. 1 135SAcERdOtE: WHEN tHE SAINtS GO mARcHING Out

Pane, John F., Daniel F. McCaffrey, Nidhi Kalra, and Annie J. Zhou. 2008. “Effects of Student Dis-placement in Louisiana During the First Academic Year After the Hurricanes of 2005.” Journal of Education for Students Placed at Risk, 13(2–3): 168–211.

Pane, John F., Daniel F. McCaffrey, Shannah Tharp-Taylor, Gary J. Asmus, and Billy R. Stokes. 2006. Student displacement in Lousiana After the Hurricanes of 2005: Experiences of Public Schools and their Students. RAND Gulf States Policy Institute Technical Report. Santa Monica, November.

Paxson, Christina, and Cecilia Elena Rouse. 2008. “Returning to New Orleans after Hurricane Katrina.” Princeton University, Industrial Relations Section, Working Paper 522.

Pribesh, Shana, and Douglas B. Downey. 1999. “Why Are Residential and School Moves Associated with Poor School Performance?” demography, 36(4): 521–34.

Rivkin, Steven G., Eric A. Hanushek, and John F. Kain. 2005. “Teachers, Schools, and Academic Achievement.” Econometrica, 73(2): 417–58.

Rockoff, Jonah E. 2004. “The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data.” American Economic Review, 94(2): 247–52.

Rosenbaum Paul R., and Donald B. Rubin. 1983. “The Central Role Of The Propensity Score In Observational Studies For Causal Effects.” Biometrika, 70(1):41–55

Rumberger, Russell W., and Katherine A. Larson. 1998. “Student Mobility and the Increased Risk of High School Dropout.” American Journal of Education, 107(1): 1–35.

Sacerdote, Bruce. 2008. “When The Saints Come Marching In: Effects of Hurricanes Katrina and Rita on Student Evacuees.” National Bureau of Economic Research Working Paper 14385.

Sacerdote, Bruce. 2012. “When the Saints Go Marching Out: Long-Term Outcomes for Student Evac-uees from Hurricanes Katrina and Rita: Dataset.” American Economic Journal: Applied Econom-ics. http://dx.doi.org/10.1257/app.4.1.109.

Sastry, Narayan. 2007. “Tracing the Effects of Hurricane Katrina on the Population of New Orleans: The Displaced New Orleans Residents Pilot Study.” RAND Corporation Working Paper 483.

Swanson, Christopher B., and Barbara Schneider. 1999. “Students on the Move: Residential and Edu-cational Mobility in America’s Schools.” Sociology of Education, 72(1): 54–67.

Temple, Judy A., and Arthur J. Reynolds. 1999. “School Mobility and Achievement: Longitudinal Findings from an Urban Cohort.” Journal of School Psychology, 37(4), 355–77.

Vigdor, Jacob L. 2007. “The Katrina Effect: Was There a Bright Side to the Evacuation of Greater New Orleans?” B.E. Journal of Economic Analysis and Policy: Advances in Economic Analysis and Pol-icy, 7(1): Article 64.

Wood, David, Neal Halfon, Debra Scarlata, Paul Newacheck, and Sharon Nessim.1993. “Impact of Family Relocation on Children’s Growth, Development, School Function, and Behavior.” Journal of American medical Association, 270(11): 1334–38.

Wright, Dan. 1999. “Student Mobility: A Negligible and Confounded Influence on Student Achieve-ment.” Journal of Educational Research, 92(6): 347–53.

Zissimopoulos, Julie, and Lynn Karoly. 2007. “Employment and Self-Employment in the Wake of Hur-ricane Katrina.” RAND Corporation Working Paper 525.