1 The Impact of the Promise of Scholarships and Altering School Structure on College Plans,...
-
Upload
emory-allen -
Category
Documents
-
view
212 -
download
0
Transcript of 1 The Impact of the Promise of Scholarships and Altering School Structure on College Plans,...
11
The Impact of the Promise of Scholarships The Impact of the Promise of Scholarships and Altering School Structure on College and Altering School Structure on College
Plans, Preparation, and EnrollmentPlans, Preparation, and Enrollment
ByByJerald R. Herting, Charles Hirschman and Nikolas Pharris-CiurejJerald R. Herting, Charles Hirschman and Nikolas Pharris-Ciurej
Department of Sociology and Center for Studies in Demography and Ecology,Department of Sociology and Center for Studies in Demography and Ecology, University of WashingtonUniversity of Washington
Thanks to the Andrew W. Mellon and the Bill and Melinda Gates Foundations and to the Tacoma Public Schools, their staff and students
22
Washington State Achievers ProgramWashington State Achievers Program::
Increase college attendance rates through various Increase college attendance rates through various means (e.g. scholarship programs, school means (e.g. scholarship programs, school redesign, and mentoring programs) at selected redesign, and mentoring programs) at selected high schoolshigh schools
Reduce financial barriers for low income students Reduce financial barriers for low income students that are motivated to attend collegethat are motivated to attend college
Schools targeted were historically lower achievers, Schools targeted were historically lower achievers, lower SES population, and higher population of lower SES population, and higher population of minority youthminority youth
33
Program alteredProgram altered
MotivationMotivation PreparationPreparation AbilityAbility
Mechanisms Mechanisms • ScholarshipsScholarships• MentorMentor• Restructuring schoolsRestructuring schools
44
Research Question:Research Question:
We are interested in assessing the We are interested in assessing the effect of the WSA program on college effect of the WSA program on college plans, actual preparation, and college plans, actual preparation, and college attendance.attendance.
55
Evaluating the Achievers Program:Evaluating the Achievers Program:A Serendipitous “Natural Experiment”A Serendipitous “Natural Experiment”
Two data sources available:Two data sources available:1.1. UW Beyond High School ProjectUW Beyond High School Project
1.1. Prior to Implementation: 2000 Senior SurveyPrior to Implementation: 2000 Senior Survey
2.2. Post Implementation: 2002 to 2005 Senior SurveysPost Implementation: 2002 to 2005 Senior Surveys
3.3. 3 Program High Schools and 2 Non-program High 3 Program High Schools and 2 Non-program High Schools. Schools.
2.2. School records from large metropolitan school School records from large metropolitan school district from mid 1990’s to presentdistrict from mid 1990’s to present
1.1. Prior: School enrollment data for the class of 2000Prior: School enrollment data for the class of 2000
2.2. Post: Enrollment data for the classes of 2001 to 2005 Post: Enrollment data for the classes of 2001 to 2005
3.3. 3 Program Schools and 2 Non-program schools3 Program Schools and 2 Non-program schools
66
The Natural Experiment:The Natural Experiment: Given lack of true experimental design the natural Given lack of true experimental design the natural
experiment allows a pre- vs post-program evaluationexperiment allows a pre- vs post-program evaluation
Our expectations are that relative to pre-program, post-Our expectations are that relative to pre-program, post-program educational attitudes/behaviors should increaseprogram educational attitudes/behaviors should increase
And that we should observe this increase in program And that we should observe this increase in program schools but schools but notnot in non-program schools in non-program schools• This observed change vs non-change provides evidence of a This observed change vs non-change provides evidence of a
WSA effectWSA effect• This observed change should narrow the gap between the WSA This observed change should narrow the gap between the WSA
schools and the non-WSA schoolsschools and the non-WSA schools
77
The comparisons we would like to The comparisons we would like to make have problemsmake have problems
Comparisons of WSA schools to non-WSA schoolsComparisons of WSA schools to non-WSA schools• Schools were selected on specific criteriaSchools were selected on specific criteria• Schools have different “input” populationSchools have different “input” population
Direct comparisons of WSA scholarship recipients Direct comparisons of WSA scholarship recipients to non-recipients poses problems to non-recipients poses problems • Without a strict experimental design comparisons are Without a strict experimental design comparisons are
problematic problematic Youth are not necessarily comparable (i.e. not all differences Youth are not necessarily comparable (i.e. not all differences
among them have been nullified by randomization)among them have been nullified by randomization) Selection bias and endogeneitySelection bias and endogeneity
88
Given these limitations our best comparison is Given these limitations our best comparison is youth in the same school pre-program to youth in youth in the same school pre-program to youth in the same school post-programthe same school post-program
We expect change in post-program WSA schools We expect change in post-program WSA schools that narrows the gap in educational oriented that narrows the gap in educational oriented behaviors between the WSA and non-WSA behaviors between the WSA and non-WSA schoolsschools
We might expect differences in WSA effects We might expect differences in WSA effects across settingsacross settings
99
Pattern of Change ExpectedPattern of Change ExpectedE
DU
CA
TIO
NA
L O
UT
CO
ME
Time
Start WSA Program in 3 Selected Schools
Pre-WSA Post-WSA
Year 1
Year 2
Gap between WSA and Non-WSA closes/disappears in presence of WSA
Initial gap due to selection of low socio-economic schools as WSA schools
WSA schools
Non-WSA schools
Year 3
1010
Data Sources:Data Sources:
Focus on race/ethnic disparities in transition to collegeFocus on race/ethnic disparities in transition to college 2000, 2002, 2003, 2004 and 2005 Senior Surveys 2000, 2002, 2003, 2004 and 2005 Senior Surveys
• 5 high schools in metropolitan school district5 high schools in metropolitan school district• Paper and pencil survey: modeled on NELSPaper and pencil survey: modeled on NELS• About 70-80% coverage of all potential high school seniorsAbout 70-80% coverage of all potential high school seniors
High school seniors – an elusive concept and population High school seniors – an elusive concept and population In school survey and multiple follow upsIn school survey and multiple follow ups Only 2 percent refusal rateOnly 2 percent refusal rate
• One year follow up survey: 90% response rateOne year follow up survey: 90% response rate• 4,300 (approx) seniors in merged file: 2000, 2002 to 20054,300 (approx) seniors in merged file: 2000, 2002 to 2005
1. UW Beyond High School Project1. UW Beyond High School Project
1111
Description of Data Sources (cont.):Description of Data Sources (cont.):
School district in which UWBHS survey was conductedSchool district in which UWBHS survey was conducted
School records for each semester from 1994 to 2005School records for each semester from 1994 to 2005
Can track individual students, courses, grades, credits Can track individual students, courses, grades, credits earned, and school attendedearned, and school attended
Trace the progress of 4 cohorts of 9Trace the progress of 4 cohorts of 9thth graders: graduating graders: graduating classes of 2000 to 2005 classes of 2000 to 2005
Roughly 9,000 students in the analysisRoughly 9,000 students in the analysis
2. School records from large metropolitan 2. School records from large metropolitan school district in Puget Sound areaschool district in Puget Sound area
1212
Outcomes of InterestOutcomes of Interest College Plans and Preparation: College Plans and Preparation:
• Plan to attend 4-year college? Plan to attend 4-year college? • Have taken the SAT/ACT? Have taken the SAT/ACT?
College Enrollment -- one year laterCollege Enrollment -- one year later• Enrolled in any college: 2 year or 4 yearEnrolled in any college: 2 year or 4 year• Enrolled in a four year college or universityEnrolled in a four year college or university
1313
Pattern of Change ExpectedPattern of Change ExpectedE
DU
CA
TIO
NA
L O
UT
CO
ME
Time
Pre-WSA Post-WSA
Year 1
Year 2
Initial gap due to selection of low socio-economic schools as WSA schools
WSA schools
Non-WSA schools
Year 3
Year 1*WSA effect
Year 2*WSA effect
1414
Control variables:Control variables:• Race/EthnicityRace/Ethnicity• GenderGender• Generational Status (Senior Survey only)Generational Status (Senior Survey only)• Social Class (Parental Education, Home ownership Family Social Class (Parental Education, Home ownership Family
Income above/below poverty level)Income above/below poverty level)• Family Structure (Senior Survey only) Family Structure (Senior Survey only)
1515
FindingsFindings The social experiment to increase the The social experiment to increase the
transition from high school to college is transition from high school to college is effecting students’ behaviors consistently in effecting students’ behaviors consistently in 2 of 3 WSA high schools.2 of 3 WSA high schools.• An observed effect on 4 year college plans An observed effect on 4 year college plans
• Strong effect by 2004 and 2005Strong effect by 2004 and 2005
• An observed effect on SAT/ACT An observed effect on SAT/ACT • Strong effect by 2004 and 2005Strong effect by 2004 and 2005
• An observed effect on attending 4 year collegeAn observed effect on attending 4 year college• Consistent impact for HS #1 for 2003, 2004, and 2005Consistent impact for HS #1 for 2003, 2004, and 2005• Consistent impact of HS #3 by 2004, 2005Consistent impact of HS #3 by 2004, 2005
1616
Figure 1. Probability of Taking a College Placement Exam (SAT or ACT) by High School Attended
.00
.10
.20
.30
.40
.50
.60
.70
2000 2002 2003 2004 2005Year
Pre
dic
ted
Pro
bab
ility
Non-WSA HS #1 HS #2 HS #3
1717
Figure 2. Probability of Attending a Four Year College by High School Attended
.00
.10
.20
.30
.40
.50
2000 2002 2003 2004 2005
Year
Pro
babi
lity
of E
nrol
ling
Non-WSA HS #1 HS #2 HS #3
1818
ImplicationsImplications
WSA initiatives appear to have WSA initiatives appear to have success on key outcomes of plans, success on key outcomes of plans, preparation, and college attendancepreparation, and college attendance
Since initiatives are implemented in Since initiatives are implemented in different settings with different forms different settings with different forms of implementation--not all strategies of implementation--not all strategies and settings were equal in outcomesand settings were equal in outcomes
One setting failed to show significant effectsOne setting failed to show significant effects Expectation of delayed effectsExpectation of delayed effects
1919
Pattern of Change ExpectedPattern of Change ExpectedE
DU
CA
TIO
NA
L O
UT
CO
ME
Time
Pre-WSA Post-WSA
Year 1
Year 2
Initial gap due to selection of low socio-economic schools as WSA schools
WSA schools
Non-WSA schools
Year 3
Year 1*WSA effect
Year 2*WSA effect
2020
Thanks and CommentsThanks and Comments
2121
2222
Table 2. Distribution of College Plans Preparation, and Attendance in WSA & Non-WSA schools: 2000, 2002, 2003, 2004, and 2005.
2000 to 2005 change in
Educational Outcomes 2002 2003 2004 2005 2000 2002 2003 2004 2005 % differencee
Four Year College Plansa:
High School #1c 38.0 44.8 46.7 54.9 54.7 -2.4% 4.4% 6.3% 14.5% 14.3% 16.7% High School #2 33.0 † 30.2 40.7 36.6 25.3 -7.5% -10.3% 0.3% -3.8% -15.2% -7.7% High School #3 20.2 * 24.0 19.2 41.0 33.7 -20.2% -16.4% -21.2% 0.6% -6.7% 13.5%High School #4 and 5 40.4 40.7 41.6 44.9 44.9 -- (0.3%) (1.2%) (4.5%) (4.5%) (4.5%)
College Preparationb: High School #1 53.5 * 61.2 64.0 70.2 72.2 -9.5% -1.8% 1.0% 7.3% 9.3% 18.7% High School #2 48.6 *** 48.0 54.3 49.3 54.1 -14.3% -14.9% -8.7% -13.6% -8.9% 5.5% High School #3 35.5 *** 41.1 38.6 56.8 50.6 -27.4% -21.8% -24.3% -6.1% -12.3% 15.1%High School #4 and 5 62.9 60.4 59.8 61.2 63.9 -- -(2.5%) -(3.1%) -(1.7%) (0.9%) (0.9%)
Attended Any College: High School #1 71.1 69.7 73.8 77.9 80.4 0.7% -0.7% 3.4% 7.5% 10.0% 9.3% High School #2 62.7 † 58.9 55.5 52.6 49.7 -7.7% -11.5% -14.9% -17.8% -20.7% -13.0% High School #3 45.9 *** 54.5 52.7 64.3 49.1 -24.5% -15.9% -17.7% -6.1% -21.3% 3.2%High School #4 and 5 70.4 73.0 69.2 69.3 70.5 -- (2.6%) -(1.2%) -(1.1%) (0.1%) (0.1%)
Attended Four Year College: High School #1 32.3 38.2 41.9 49.4 46.4 -3.9% 2.0% 5.7% 13.2% 10.2% 14.1% High School #2 27.2 * 25.0 29.0 26.3 20.1 -9.0% -11.2% -7.2% -9.9% -16.1% -7.1% High School #3 17.8 *** 17.9 14.6 32.6 27.6 -18.4% -18.3% -21.7% -3.6% -8.6% 9.8%High School #4 and 5 36.2 35.0 33.4 33.4 36.6 -- -(1.2%) -(2.8%) -(2.8%) (0.4%) (0.4%)Note: a) Four year college plans is measured by whether the student listed a four year college as the first school choice which they planned to attended the following year. b) College preparation is measured by whether the student has taken the SAT or ACT college placement exam by spring of their senior year. c) High schools #1, #2, and #3 are part of the WSA program. High schools #4 and #5 are not part of the WSA program d) In comparing the WSA and non-WSA schools there are two potential referent categories: non-WSA schools in 2000 or non-WSA schools in each year. The use of either referent category yields nearly identical results. Thus, we use non-WSA schools in 2000 as a referent as it allows for continuity with the rest of our analysis. e) The difference between WSA schools and non-WSA schools between 2000 and 2005. f) Indicated levels of significance are for the logistic regression models of each outcome regressed on the three WSA school dummy variables using non-WSA schools as the referent category.† Significant at the .10 level with a two tailed test
* Significant at the .05 level with a two tailed test** Significant at the .01 level with a two tailed test*** Significant at the .001 level with a two tailed test
2000f
Percent point difference between WSA schoolsby year and non-WSA schools in 2000d
Percent distribution by year for WSA and Non-WSA schools
2323
Effects of WSA on Educational OutcomesEffects of WSA on Educational Outcomes
Educational Outcomes
Background factors and other covariates
Unknown or unmeassured
factors
-Year-2000
20012002200320042005
SchoolWSA
Non-WSA
A
B
C
2424
Source: Stoops 2004: 2
2525
Underlying Analytical DesignUnderlying Analytical Design
Y= B0 + B1 (WSA) + B2 (Year 2) + Y= B0 + B1 (WSA) + B2 (Year 2) + B3 (Year 3) + B4 (Year 4) + B5 (Year 5) + B3 (Year 3) + B4 (Year 4) + B5 (Year 5) + B6(Year 2 * WSA) + B7 (Year 3 * WSA) + B6(Year 2 * WSA) + B7 (Year 3 * WSA) + B8 (Year 4 * WSA)B8 (Year 4 * WSA) + + B9 (Year 5 * WSA)B9 (Year 5 * WSA) + + Bn (Xn)Bn (Xn)
This model is applied separately for each This model is applied separately for each WSA program school WSA program school
2626
Interpreting Results:Interpreting Results:
We want to see a significant value We want to see a significant value greater than 0 for greater than 0 for B4 (Year 1 * WSA), B4 (Year 1 * WSA), B5 (Year 2 * WSA) and B5 (Year 3 * B5 (Year 2 * WSA) and B5 (Year 3 * WSA)WSA) which implies that being in a which implies that being in a WSA school post-program WSA school post-program implementation has increased the implementation has increased the probability of these behaviors compared probability of these behaviors compared to pre-programto pre-program