The National Board on Educational Testing and Public Policy

Volume 1 Number 5 June 2000

s t a t e m e n t s

The Impact of Score Differences onthe Admission of Minority Students:

An IllustrationDaniel Koretz

National Board on Educational Testing and Public PolicyPeter S. and Carolyn A. Lynch School of Education

Boston College

In this paper, we discuss one of the arguments that has been ad-vanced against the use of standardized college admissions tests: the notionthat their use leads admissions officers to reject non-Asian minority stu-dents on the basis of small and insignificant differences in scores. We do notdiscuss in detail any of the numerous other arguments about college admis-sions testing, although we do briefly comment on two of them: the possibil-ity of test bias against minorities, and the relative size of the gap betweenminority and non-minority students on admissions tests compared to thegaps shown on other achievement tests.

NBETPP The Impact of Score Differences on the Admission of Minority Students

2

Daniel Koretz is a Senior Fellow with

the National Board on Educational

Testing and Public Policy and a

Professor of Educational Research,

Measurement, and Evaluation in the

Lynch School of Education at Boston

College.

The average differences in performance between non-Asianminority students and majority students are very large and havea major effect on the selection of students and the compositionof the selected student population. This effect becomes progres-sively larger as schools become more selective, not because ofany idiosyncrasy of the tests used for selection, but simply be-cause of the distribution of student performance. Some students(in any racial or ethnic group) will always fall on the margin ofacceptance, of course, and for those individual students, a smallchange in test scores might tip the balance toward acceptanceor rejection. This fact, however, should not obscure the magni-tude of the average differences between the groups. In the aggre-gate, the disadvantage minority students face as a result of theirtest scores is not a matter of small differences at the margin.Efforts to improve access for minority students must addressthat fact.

Are Admissions Tests Biased or Atypical?

A full discussion of possible bias in college admissions test-ing is beyond the scope of this paper, as is an evaluation of theagreement between admissions tests and other measures of stu-dent achievement. The large differences in scores discussedhere, however, cannot be interpreted without some informa-tion on these two questions, and a brief synopsis is providedhere.

A simple mean difference in test scores between groups –that is, a finding that the average score of one group is mark-edly lower than that of another – does not in itself indicate testbias. Bias arises when the scores for one or more groups aremisleading – for example, if they are low because of unfair ques-tions.

Copious research has not shown admissions tests to be bi-ased against minorities. Admissions tests are used to predictperformance in college, and they are most often validated byassessing how well they predict the early performance of stu-dents accepted to a given college – specifically, their freshman-year grades or grade-point averages (GPAs). If the tests werebiased against minority students, one would expect to find thatminority students perform better in college, on average, than

In the aggregate, the

disadvantage minority

students face as a result of

their test scores is not a

matter of small differences

at the margin. Efforts to

improve access for

minority students must

address that fact.▲

The Impact of Score Differences on the Admission of Minority Students NBETPP

3

The lack of bias and rather

typical mean differences

shown by current college

admissions tests do not

mean that all possible

admissions tests would

yield the same results.

Indeed, substituting

different tests for the SAT-I

or ACT would improve the

prospects for some

students while lessening

them for others.

their scores predict. But that is not the case. In the case of malestudents, research finds the opposite: on average, black andHispanic students achieve somewhat lower freshman gradesthan their scores predict. Black and Hispanic women achieveon average about the GPAs their test scores predict.

The average differences between minority and non-minor-ity students on admissions tests are not atypically large com-pared to the differences typically found on tests of educationalachievement. In 1999, the mean differences between black andwhite students on the SAT-I were 0.89 standard deviation onthe verbal scale and 1.0 on the mathematics scale.1 (When dif-ferences are expressed in standard deviations, they can be com-pared across tests.) A recent review of large-scale studies ofsecondary-school students showed that black-white differencesin composite scores (that is, scores summing across subjects)ranged from .82 to 1.18 standard deviations.2 The differencesin individual subjects varied somewhat more. While the gapbetween blacks and whites on achievement tests has narrowedin recent decades, it remains very large.3 For example, in 1994the gap in the National Assessment of Educational Progress(NAEP) trend assessment still ranged from 0.66 standard de-viation in reading to 1.08 in science.4

The similarity in the results of admissions tests and otherlarge-scale achievement tests also argues against bias. Consid-erable work has gone into limiting bias in these many tests.Moreover, examination of the questions of which the tests con-sist shows large performance differences between blacks andwhites on items that are clearly not biased, such as simple math-ematics problems.

The lack of bias and rather typical mean differences shownby current college admissions tests do not mean that all pos-sible admissions tests would yield the same results. Indeed,substituting different tests for the SAT-I or ACT would improvethe prospects for some students while lessening them for oth-ers. It does suggest, however, that the current test results arenot misleading in the aggregate and that only substantialchanges in admissions testing, such as the measurement ofimportant skills or content not currently assessed, could greatlychange group differences.

NBETPP The Impact of Score Differences on the Admission of Minority Students

4

For purposes of this paper, then, we accept that collegeadmissions tests show fairly typical group differences and thatthese differences are not biased against minorities as predica-tors of college grades. These differences are large relative tothe distribution of achievement within each group. They couldstem from a variety of factors, which we will not examine here.We will merely explore the size and effects of these differencesin practical terms.

Before analyzing the effects of test scores on admissions,we will translate the differences in standard deviations intoanother metric that is easier to understand. A mean differenceof 0.60 standard deviations, which is smaller than any of theblack-white differences noted above, would place the averageblack student at the 27th percentile among white students (Table1). That is, only 27 percent of white students would score aslow as or lower than the average black student. A more typi-cal difference of 0.80 standard deviation would place the av-erage black student at the 21st percentile among whites. A gapof a full standard deviation – the size of the gap on the SAT-Imathematics scale – places the average black student at the16th percentile among whites.

Black-White Mean Score Differences

Differences in White percentile ofstandard deviations average black student

.60 27

.80 21

1.00 16

1.20 12

Table 1

The Impact of Score Differences on the Admission of Minority Students NBETPP

5

We then examined the

effects of several simple

admissions rules that

depended solely on scores.

We set a number of cut

scores on the test, and all

students scoring above the

cut were “accepted,” while

all those below it were

“rejected.”

▲

How We Carried Out Our Analysis

We wanted our analysis to reflect the general pattern ofgroup differences in performance rather than any idiosyncra-sies of any particular test or test-taking group. Accordingly,rather than using data from the SAT or ACT testing programs,we simulated data that mirror the typical differences found inlarge-scale testing programs.

We created databases that had a mean difference of 0.80standard deviation between blacks and whites – a differencemodestly smaller than those found on the SAT but larger thansome of the most recent differences found in NAEP. We madethe scores of simulated black students a little less variable thanthose of whites.5 This is consistent with the pattern shown innumerous studies.6 For simplicity, we set the mean of the scoresto zero and the standard deviation to 1. Thus, a score of zero inour data corresponds roughly to an SAT-I score of 500.

We then examined the effects of several simple admissionsrules that depended solely on scores. We set a number of cutscores on the test, and all students scoring above the cut were“accepted,” while all those below it were “rejected.” We con-sidered no other characteristics of students. These are overlysimple selection rules that no colleges follow, and indeed usingthem would be inconsistent with accepted professional stan-dards. These unrealistically simple rules, however, isolate theeffect of test scores.

The Effect of Test Scores: Three Scenarios

We present three scenarios. For simplicity, all consider onlyblack and white applicants. The first sets the cut score at theoverall mean and uses equal numbers of black and white ap-plicants. The second retains the equal numbers of applicantsbut imposes a higher cut score, set arbitrarily at one standarddeviation above the mean, roughly the 84th percentile. Thesetwo show the pure effect of test-based selection, independentof the smaller number of black applicants at most colleges.Comparison of these two scenarios shows the effect of greaterselectivity. The final scenario maintains the cut score at one stan-dard deviation above the mean but reduces the number of blackapplicants to a more realistic 15 percent of the total.7

NBETPP The Impact of Score Differences on the Admission of Minority Students

6

SCENARIO 1: 50 percent black applicants, cutscore at the overall mean

The distributions of scores in our first simulated case, likethose in many actual test databases, roughly follow the normalcurve (Figure 1). The mean score for all students is set to zero,and the standard deviation is one. Thus, a value of +1.0 repre-sents a score one standard deviation above the overall mean,or roughly the 84th percentile in the entire population, while avalue of –1.0 represents a score one standard deviation belowthe overall mean, or roughly the 16th percentile in the entirepopulation.

Because black applicants have an average score .8 standarddeviation below that of white applicants, black students areclustered around an average that is well below the overall mean(roughly .68 standard deviation below the mean; see Figure 1).White students are clustered around their mean, which is mod-estly (.12 standard deviation) above the overall mean. Becausethe scores of black students vary somewhat less than those of

white students, black applicantsare bunched a little more tightlyaround their average scores thanare white students. The dashedvertical line in Figure 1 repre-sents the cut score, which is setat the overall mean score of 0.Everyone with scores above thisline was “accepted,” while allstudents below the line were “re-jected.”

Even with the relatively lowcut score of 0 (the overall meanscore), a much smaller percent-age of black than of white stu-dents is accepted: About 20 per-cent, compared with 55 percentof white applicants (Figure 2).The percentage accepted, shownas a bar chart in Figure 2, isequivalent to the portions of thedistributions above the cut scorein Figure 1.

Cut Score at the Overall Mean, EqualNumbers of Black and White Applicants

Cut Score

Whites

Blacks

Score in Standard Deviations

-3 -2 -1 0 1 2 3

Figure 1

The Impact of Score Differences on the Admission of Minority Students NBETPP

7

Figure 3

Figure 2Percentages Accepted by Race, Equal Numbers ofApplicants and Cut Score at Mean

60

50

40

30

20

10

0Black Students White Students

Perc

ent

Use of this low cut score causes blacks to be underrepre-sented in the admitted student body by roughly a factor of two,relative to their representation in the pool of applicants. Al-though they constitute half of the applicants, they constituteonly 27 percent of the selected students (Figure 3).

Composition of Applicant Pool and Admitted Group,Equal Numbers of Applicants and Cut Score at Mean

100

90

80

70

60

50

40

30

20

10

0Applicant Pool

Perc

ent

Admitted Group

White StudentsBlack Students

NBETPP The Impact of Score Differences on the Admission of Minority Students

8

SCENARIO 2: 50 percent black applicants,cut score at 1 standard deviation above theoverall mean

The second scenario retained the same applicant pool anddistribution of scores but was more selective, setting the cutscore at one standard deviation above the mean (Figure 4). Thiswould equal 616 on the SAT I-Verbal and 625 on the SAT I-Mathematics – appreciably above the 25th percentile of scoresof freshmen at the University of Pennsylvania on the SAT I-Verbal (560) but below the 25th percentile of those students onthe SAT I-Mathematics (650).8

Raising the cut score from the mean to one standard devia-tion above sharply reduces the percentage of both white andblack applicants accepted (Figure 4). This reduction is particu-larly severe, however, for black applicants. Roughly 17 percentof white students are accepted (Figure 5), compared with 55percent when the cut score is at the mean. Only about 1 per-cent of black applicants are accepted (Figure 5), compared with

about 20 percent when the cutscore is at the mean.

Raising the cut score also hasa dramatic effect on the racialcomposition of the accepted stu-dent population. While the ap-plicant pool is constructed to behalf black and half white, blackstudents constitute barely 6 per-cent of the accepted students(Figure 6). With the cut score atthe mean, blacks constitutedabout 27 percent of the students.In other words, with a cut scoreat the mean blacks areunderrepresented in the studentpopulation by a factor of abouttwo; with a cut score at one stan-dard deviation above the mean,blacks are underrepresented byroughly a factor of eight.

Cut Score at +1 Standard Deviation, EqualNumbers of Black and White Applicants

Figure 4

Score in Standard Deviations

Cut Score

Whites

Blacks

-3 -2 -1 0 1 2 3

The Impact of Score Differences on the Admission of Minority Students NBETPP

9

Figure 6

Figure 5

Composition of Applicant Pool and Admitted Group,Equal Numbers of Applicants and Cut Score at1 Standard Deviation Above Mean

100

90

80

70

60

50

40

30

20

10

0Applicant Pool

Perc

ent

Admitted Group

White StudentsBlack Students

Percents Accepted by Race, Equal Numbers ofApplicants and Cut Score at 1 Standard DeviationAbove Mean

20

18

16

14

12

10

8

6

4

2

0Black Students

Perc

ent

White Students

NBETPP The Impact of Score Differences on the Admission of Minority Students

10

SCENARIO 3: 15 percent black applicants,cut score at 1 standard deviation above theoverall mean

The final scenario again uses a cut score of one standarddeviation above the mean but reduces the black applicant poolto a more plausible 15 percent of the total. The result is shownin Figure 7. This is identical to Figure 4 except for the smallernumber of black applications.

Because the cut score and the average score for each groupremain unchanged, the percentage of black and white studentsaccepted remains the same: about 17 percent of white appli-cants but only 1 percent of black applicants. The smaller poolof black applicants increases the homogeneity of the acceptedstudent population. While the applicant pool is 15 percent black,the accepted student body is roughly 99 percent white (Figure 8).

Cut Score at +1 Standard Deviation,15% Black Applicants

Score in Standard Deviations

-4 -3 -2 -1 0 1 2 3 4

Cut Score

Whites

Blacks

Figure 7

The Impact of Score Differences on the Admission of Minority Students NBETPP

11

Figure 8Composition of Applicant Pool and Admitted Group,15% Black Applicants and Cut Score at 1 StandardDeviation Above Mean

100

90

80

70

60

50

40

30

20

10

0

Perc

ent

White StudentsBlack Students

Applicant Pool Admitted Group

NBETPP The Impact of Score Differences on the Admission of Minority Students

12

Implications

These simulations illustrate that when test scores countheavily in admissions, the large differences in scores betweenblack and white students have a major impact both on the prob-ability that black students will be admitted and on the compo-sition of the accepted student population. These effects becomeprogressively more severe as the selectivity of admissions in-creases. For example, with a cut score at the overall mean, blackstudents would be underrepresented by a factor of two in thestudent body; with a cut score at one standard deviation abovethe mean, they would be underrepresented by a factor ofroughly eight. The relatively small number of black applicantsto college, which stems in part from their relatively small num-bers in the cohort of college-age students, changes neither theprobability that black students will be accepted nor the pro-portional underrepresentation of black students in the studentbody. It does, however, further increase the homogeneity of thestudent body.

Of course, few if any schools select students solely on thebasis of a cut score on an admissions test, and more commonselection processes that give weight to other factors will oftenplace black students at less of a disadvantage. Nonetheless,unless test scores are given very little weight or are offset byother factors on which minority students have an advantagerelative to whites, the average test-score disparity will gener-ally have a severe impact on admission to selective colleges.

The values used in the simulation were chosen to be repre-sentative of a broad range of tests rather than any single col-lege admissions test. Because the mean difference betweenblacks and whites is somewhat larger on the SAT-I than in thesesimulated databases, using values from the SAT-I would exac-erbate the results presented here, albeit not greatly.

These simulations illustrate

that when test scores count

heavily in admissions, the

large differences in scores

between black and white

students have a major

impact both on the

probability that black

students will be admitted

and on the composition of

the accepted student

population. These effects

become progressively

more severe as the

selectivity of admissions

increases.

▲

The Impact of Score Differences on the Admission of Minority Students NBETPP

13

A parallel simulation for Hispanic students nationwidewould also show a severe impact, but smaller than that forblacks. Data from large-scale assessments typically show His-panic students scoring somewhat higher, on average, thanblacks.

It is important to emphasize that the progressively moresevere impact that accompanies greater selectivity does not stemfrom any peculiarity of college admissions tests. It stems pri-marily from the roughly normal distribution of scores – that is,from the fact that most students have scores quite close to theaverage for their group, while few have scores much higher orlower than the average. This pattern is slightly exacerbated bythe fact that black students show modestly less variable testscores than do white students. That is, the percentage of blackstudents scoring either much higher or much lower than theblack average is smaller than the corresponding percentage forwhite students.

These results illustrate the difficulty inherent in reconcilingacademic selectivity with increased equity of access to post-secondary education for non-Asian minority groups, particu-larly at selective colleges and universities. Most colleges con-sider a variety of other factors in addition to test scores inmaking admissions decisions, and to the extent that those fac-tors are not strongly correlated with test scores, the problemsillustrated here will be ameliorated somewhat. The differencesbetween minority and majority students in academic perfor-mance as measured by diverse standardized tests are so large,however, and their effects are so substantial at academicallyselective colleges, that it will be difficult to offset their impactwithout confronting them directly.

These results illustrate the

difficulty inherent in

reconciling academic

selectivity with increased

equity of access to post-

secondary education for

non-Asian minority groups,

particularly at selective

colleges and universities.

▲

NBETPP The Impact of Score Differences on the Admission of Minority Students

14

About the National Board on EducationalTesting and Public Policy

Created as an independent monitoring system for assess-ment in America, the National Board on Educational Testingand Public Policy is located in the Peter S. and Carolyn A. LynchSchool of Education at Boston College. The National Board pro-vides research-based test information for policy decision mak-ing, with special attention to groups historically underservedby the educational systems of our country. Specifically, theNational Board

Monitors testing programs, policies, and products

Evaluates the benefits and costs of testing programs inoperation

Assesses the extent to which professional standards fortest development and use are met in practice

The Impact of Score Differences on the Admission of Minority Students NBETPP

15

notes1 The College Entrance Examination Board (1999). 1999 College-

Bound Seniors, National Report. New York: author.

2 Hedges, L. V., & Nowell, A. (1998). Black-white test score conver-gence since 1965. In C. Jencks and A. Phillips (Eds.), The BlackWhite Test Score Gap, (pp. 149-181). Washington, D. C.: Brookings.

3 Cambell, J. R., Voelkl, K.E., & Donahue, P.L. (1997). Report in Brief:NAEP 1996 Trends in Academic Performance, Washington, D.C.:National Center for Education Statistics (NCES 97—986); Hedgesand Nowell, op. cit.,; Koretz, D. (1986), Trends in EducationalAchievement, Washington, D.C.: Congressional Budget Office.

4 Hedges and Nowell, op. cit.

5 Specifically, we set the black/white variance ratio to .81.

6 See Hedges and Nowell, op cit.

7 Note that the distributions of scores used in all three scenarios arebased on a population that is 15 percent black and 85 percentwhite. This results in a white average that is slightly above theoverall average and a black average that is much lower than theoverall average. If we had regenerated data for the scenarios withequal numbers of applicants based on a population that is halfblack and half white, the mean difference between blacks andwhites would have remained the same, but both group meanswould have increased relative to the overall mean. (They wouldhave been equidistant from the overall mean). This would havebeen unrealistic and would have confounded the comparisonsamong the scenarios.

8 The Princeton Review (1998), The Best 311 Colleges, 1999 Edition.New York: Random House.

Visit us on theWorld Wide Web atnbetpp.bc.edu formore articles, thelatest educationaltesting news, andinformation onNBETPP.

The National Board on Educational Testingand Public PolicyLynch School of Education, Boston CollegeChestnut Hill, MA 02467

Telephone: (617)552-4521 • Fax: (617)552-8419

Email: nbetpp@bc.edu

Visit our website at nbetpp.bc.edu formore articles, the latest educational news,and for more information about NBETPP.

About the National Boardon Educational Testing

and Public Policy

Created as an independent monitoring systemfor assessment in America, the National Board onEducational Testing and Public Policy is located inthe Peter S. and Carolyn A. Lynch School ofEducation at Boston College. The National Boardprovides research-based test information for policydecision making, with special attention to groupshistorically underserved by the educationalsystems of our country. Specifically, theNational Board

• Monitors testing programs, policies, andproducts

• Evaluates the benefits and costs of testingprograms in operation

• Assesses the extent to which professionalstandards for test development and use aremet in practice

This National Board publication series is supported bya grant from the Ford Foundation.

The National Board on Educational Testing and Public Policy

The Board of Directors

Peter LynchVice ChairmanFidelity Management and

Research

Paul LeMahieuSuperintendent of EducationState of Hawaii

Donald StewartPresident and CEOThe Chicago Community Trust

Antonia HernandezPresident and General CouncilMexican American Legal Defense

and Educational Fund

Faith SmithPresidentNative American Educational

Services

BOSTON COLLEGE