Back to the Basics:

28
Back to the Basics: Saul Geiser University of California, Berkeley In Defense of Achievement (and Achievement Tests) in College Admissions

description

Back to the Basics:. In Defense of Achievement (and Achievement Tests) in College Admissions. Saul Geiser University of California, Berkeley. “Readiness” for college: Two views. Achievement : performance in college-preparatory subjects, mastery of curriculum content - PowerPoint PPT Presentation

Transcript of Back to the Basics:

Page 1: Back to the Basics:

Back to the Basics:Back to the Basics:

Saul GeiserUniversity of California, Berkeley

Saul GeiserUniversity of California, Berkeley

In Defense of Achievement (and Achievement Tests)

in College Admissions

In Defense of Achievement (and Achievement Tests)

in College Admissions

Page 2: Back to the Basics:

““Readiness” for college: Two viewsReadiness” for college: Two views

• Achievement: performance in college-preparatory subjects, mastery of curriculum content

• Ability: generalized reasoning and problem-solving skills

Page 3: Back to the Basics:

UC findingsUC findings

Admissions criteria that tap student mastery of curriculum content, such as high-school grades and achievement tests, are stronger predictors of success in college and fairer to low-income and minority applicants than tests of general reasoning such as the SAT.

Page 4: Back to the Basics:

Underrepresented minorities as a percent Underrepresented minorities as a percent of new UC freshmen, 1995 to 1998of new UC freshmen, 1995 to 1998

Page 5: Back to the Basics:

Correlation of SAT I, SAT II and HSGPACorrelation of SAT I, SAT II and HSGPAwith socioeconomic factorswith socioeconomic factors

Family Income

Parent’s Education

High SchoolRank

SAT I Reasoning .32 .40 .40

SAT II Subject Tests .23 .29 .33

High School GPA .04 .06 .01

Page 6: Back to the Basics:

% of underrepresented minority applicants % of underrepresented minority applicants by SAT I vs. HSGPA decilesby SAT I vs. HSGPA deciles

SAT I HSGPA

Top decile 4% 9%

9th 6% 11%

8th 7% 13%

7th 9% 14%

6th 12% 16%

5th 15% 17%

4th 18% 19%

3rd 22% 20%

2nd 29% 23%

Bottom decile 45% 28%

Page 7: Back to the Basics:

HSGPA

SAT I SAT IISchoolRank SES

BioScience .35 .10 .23 .05 .03

PhysSci/Math .33 -.02 .25 .04 .03

SocSci/Hum .31 .08 .17 .04 .07

Other .32 .05 .21 .05 .07

ALL FIELDS .31 .03 .21 .06 .06

Relative weight of different admissions Relative weight of different admissions factors in predicting freshman GPAfactors in predicting freshman GPA

Sample: New UC freshmen in 1996-1999 who completed first year. N = 68,602.

Page 8: Back to the Basics:

HSGPA

SAT I SAT IISchoolRank SES

BioScience .22 .06 .14 .04 .00

PhysSci/Math .25 -.01 .22 .07 .03

SocSci/Hum .17 -.03 .07 .07 .09

Other .18 -.10 .10 .09 .09

ALL FIELDS .20 -.03 .12 .08 .07

Relative weight of different admissions Relative weight of different admissions factors in predicting 4-year graduationfactors in predicting 4-year graduation

Sample: New UC freshman in 1996-1999. N = 74,618.

Page 9: Back to the Basics:

Minority participation in AP/honors classes Minority participation in AP/honors classes among California college-bound seniorsamong California college-bound seniors

# AP/HonorsSubjects Taken

UnderrepresentedMinority %

None 32.3%

1 - 4 30.5%

5 - 8 23.7%

9 or more 17.3%

Page 10: Back to the Basics:

HSGPA SAT I

SAT IISchoolRank SES

# AP Classes

AP Exams

BioScience .21 .07 .14 .01 .05 .01 .19

PhysSci/Math .29 .03 .08 -.02 .05 .06 .22

SocSci/Hum .26 .08 . 09 .06 .08 .02 .11

Other .23 .07 .06 .04 .09 .01 .15

ALL FIELDS .24 .03 .08 .03 .08 .02 .15

Relative weight of different admissions Relative weight of different admissions factors in predicting cumulative UC GPAfactors in predicting cumulative UC GPA

Sample: New UC freshmen in 2002 who completed two years. N = 14,976.

Page 11: Back to the Basics:

Achievement tests and UC outreachAchievement tests and UC outreach

• Alignment with K-12 curriculum

• Diagnostic function

• Message to students

Page 12: Back to the Basics:

UC policy changes after Prop 209UC policy changes after Prop 209

• 1998: Increased weight for HSGPA and SAT II, reduced weight for SAT I scores in statewide eligibility index

• 2001: Introduced Top 4% Plan and Comprehensive Admissions Review

• 2002: Adopted policy favoring achievement tests over tests of general reasoning or aptitude

Page 13: Back to the Basics:

Underrepresented minorities as a percent Underrepresented minorities as a percent of new UC freshmen, 1995 to 2007of new UC freshmen, 1995 to 2007

Page 14: Back to the Basics:

Restoring the role of high-school recordRestoring the role of high-school record

• Increase emphasis on high-school grades and class rank over admissions tests

• Eliminate “bonus point” for Advanced Placement classes except where students take and pass AP exams

Page 15: Back to the Basics:

Limitations of New SAT as Limitations of New SAT as an achievement testan achievement test

• Less curriculum-based than other achievement tests

• Remains norm-referenced

• Limited diagnostic value

• Prediction is no better than old SAT

Page 16: Back to the Basics:

An expanded role for subject testsAn expanded role for subject tests

• Most content-intensive of all nationally available assessments

• Align with and reinforce classroom instruction

• Best predictors, after high-school record, of student performance in college

Page 17: Back to the Basics:

Back to the basicsBack to the basics

• Criterion-referenced vs. norm-referenced assessment

• Accent on achievement and mastery of foundational subjects

Page 18: Back to the Basics:
Page 19: Back to the Basics:

Additional slides for Q & Additional slides for Q & AA

Page 20: Back to the Basics:

Rothstein, J. (2004). “College performance predictions and the SAT." Journal of Econometrics, volume 121, 297-317.

“The results here indicate that the exclusion of student background characteristics from prediction models inflates the SAT’s apparent validity, as the SAT score appears to be a more effective measure of the demographic characteristics that predict [college grade-point average] than it is of preparedness conditional on student background. … [A] conservative estimate is that traditional methods and sparse models [i.e., those that do not take into account demographic characteristics] overstate the SAT’s importance to predictive accuracy by 150 percent.”

Page 21: Back to the Basics:

1996 1997 1998 1999 1996-1999

Predictor Variables/Equations:

(1) HSGPA 17.0% 16.7% 14.7% 12.9% 15.4%(2) SAT I 13.8% 10.8% 12.2% 14.2% 13.3%(3) SAT II 16.4% 14.4% 15.6% 16.4% 16.0%(4) SAT I + SAT II 16.7% 14.4% 15.6% 16.8% 16.2%(5) HSGPA + SAT I 21.9% 20.1% 19.2% 20.4% 20.8%(6) HSGPA + SAT II 23.0% 21.7% 21.1% 21.5% 22.2%(7) HSGPA + SAT I + SAT II 23.2% 21.7%* 21.1%* 21.9% 22.3%

SAT I increment: [(7)-(6)] 0.2% 0.0% 0.0% 0.4% 0.1%

* SAT I not statistically signif icant in prediction equation; all other variables are statistically signif icant at <.01 level.

Percent of Variance in UC Freshman GPA Explained by HSGPA, SAT I and SAT II Scores

Page 22: Back to the Basics:

HSGPASAT IM + V

SAT IIWriting

SAT IIOther

APExams

SchoolRank SES

Asian .28 -.03 .13 .05 .23 .03 .00

Black .20 -.09 .05 .06 .21 .13 .06

Latino .28 .05 .13 .05 .07 .08 .05

White .32 -.04 .14 .05 .17 .05 .06

Relative weight of different admissions factors Relative weight of different admissions factors in predicting freshman GPA by race/ethnicityin predicting freshman GPA by race/ethnicity

Sample: New UC freshmen in 1998-2001 who completed first year. N = 57,377.

Page 23: Back to the Basics:

HSGPA

SAT I SAT IISchoolRank SES

Berkeley .31 -.05 .24 .03 .09

Davis .35 .06 .24 .09 .03

Irvine .28 .08 .15 .03 .04

Los Angeles .32 .00 .20 .07 .08

Riverside .34 .11 .11 -.02 .02

San Diego .33 .06 .19 .06 .07

Santa Barbara .39 .06 .18 .05 .07

Prediction weights for 4Prediction weights for 4thth year GPA by campus year GPA by campus

Sample: New 1996-1999 UC freshmen completing 4 years. N = 58,539.

Page 24: Back to the Basics:

Mean SD Mean SD Mean SD

SAT I V 585.4 100.1 591.5 92.2 1.0% -7.9%

SAT I M 617.9 95.8 629.1 88.1 1.8% -8.0%

SAT II W 567.5 104.1 574.5 97.3 1.2% -6.5%

SAT II M 599.0 101.4 612.0 94.0 2.2% -7.3%

SAT II 3rd 606.6 111.8 616.7 105.4 1.7% -5.7%

SAT I * 1203.4 173.6 1220.6 156.9 1.4% -9.6%

SAT II WM ** 1166.5 180.8 1186.5 165.8 1.7% -8.3%

SAT II *** 1773.0 261.8 1803.2 239.4 1.7% -8.6%

HSGPA 3.75 0.49 3.86 0.41 2.9% -16.3%

* Composite of SAT I V + SAT I M** Composite of SAT II W + SAT II M*** Composite of SAT II W + SAT II M + SAT II 3rd

N = 144,082 N = 58,064

Freshmen - Applicants

Means and Standard Deviations of SAT Scoresfor UC Applicants and Enrolled Freshmen

Fall 1997 through Fall 1999

Applicants Enrolled Freshmen% Difference,

Page 25: Back to the Basics:

Racial/Ethnic Composition of CA High SchoolsBased on School Average SAT I Scores

21%

46%

62%

74%

83%

79%

54%

38%

26%

17%

0% 25% 50% 75% 100%

Bottom Quintile

Quintile 2

Quintle 3

Quintile 4

Top Quintile

White, Asian and Other Black, Latino and American Indian

Page 26: Back to the Basics:

School HSGPA SAT I SAT IIAPI Quintile

5 (high) .33 -.01* .204 .32 .01* .203 .29 .03* .252 .28 .07 .221 (low ) .25 .12 .18

All Schools .27 .07 .23

* Not statistically signif icant at <.01 level.

Standardized Regression Coefficients for HSGPA, SAT I and SAT II Scoresby School API Quintile, 1996-1999

Regression equation: UCGPA = HSGPA + SAT I + SAT II

Page 27: Back to the Basics:

HSGPA SAT I

SAT IISchoolRank SES

# AP Classes

AP Exams

1998 Cohort .28 -.04 .10 .04 .07 .04 .24

1999 Cohort .28 -.01 .07 .08 .08 .03 .22

2000 Cohort .30 .03 . 05 .05 .08 .03 .21

ALL YEARS .29 .00 .07 .05 .08 .03 .22

Relative weight of different admissions Relative weight of different admissions factors in predicting freshman GPAfactors in predicting freshman GPA

Sample: New UC freshmen who completed first year. N = 41,116.

Page 28: Back to the Basics:

HSGPA SAT I SAT IISchoolRank SES

BioScience .34 .11 .23 .05 .02

Phys/Math .36 .02 .23 .05 .02

SocSci/Hum .33 .07 .20 .04 .08

Other .34 -.03 .23 .07 .07

ALL FIELDS .33 .04 .20 .05 .06

Relative weight of different admissions factors Relative weight of different admissions factors in predicting cumulative 4-year UC GPAin predicting cumulative 4-year UC GPA

Sample: New 1996-1999 UC freshmen completing 4 years. N = 58,539.