Chapter 4 Alignment Among Secondary and … · ... (ninth grade language and composition and...

63

Chapter 4 Alignment Among Secondary and Postsecondary Assessments

in Georgia

The Georgia Assessment Environment

Over the past 10 years, Georgia’s testing program has undergone many changes.

In 1991, the Georgia Assembly established a new set of tests, the Georgia High School

Graduation Tests (GHSGTs), which replaced the previous graduation exam, the

minimum competency Basic Skills Test. The GHSGTs are intended to measure the

learning objectives specified in the state standards (i.e., Quality Core Curriculum), and to

determine whether a student should receive a high school diploma. The new GHSGTs

differed from the previously required Basic Skills Test in that they assessed achievement

in social studies and science (in addition to reading, mathematics, and writing). State law

also mandated that the GHSGTs be a more rigorous test than the Basic Skills Test, and

extended the range of item difficulty found on the GHSGTs.

The GHSGTs graduation tests were phased in gradually. The first group of

students to be affected was those who entered the ninth grade between July 1991 and July

1993 (these would have been students expected to graduate in spring 1995 and 1996).

These students had to pass the English/language arts, mathematics, and writing tests to be

eligible to receive a diploma. The next group, students who entered the ninth grade

between July 1993 and July 1994 (the expected class of '97), was required to pass the

social studies test. And finally, the science test was required for those students who

entered the ninth grade after July 1994 (expected class of '98).

Students have five opportunities to take each of the tests before the end of the

12th grade. There are two testing periods during the junior year. The Writing test is

offered in the early fall semester and again in the spring semester for those students who

need to retake it. Content-specific exams are administered only during the spring

semester of the junior year. After that, the students have three more opportunities to pass

the writing test (summer, fall and spring of their senior year) and the content-specific

exams (summer, fall and spring of their senior year). Students who do not pass all the

required tests but have met all other graduation requirements are eligible to receive a

Certificate of Performance or a Special Education Diploma. These students may then

64

return to take any graduation test(s) necessary in order to qualify for a high school

diploma. There is no limit for these students on the number of times they may retake any

of the tests.

Currently, there are plans to replace the GHSGTs with new assessments

developed as part of the End-of-Course Testing (EOCT) program. Slated for full

implementation in 2003, the EOCT will be multiple-choice, end-of-course exams

assessing student knowledge of course standards in eight core courses, specifically math

(algebra and geometry), ELA (ninth grade language and composition and American

language and composition), social studies (U.S. history and economics/business), and

science (biology and physical science). Although there are plans to make passing some

or all of the end-of-course tests a condition for graduation, there has not been a final

decision regarding how the ECOT will affect graduation requirements.

Georgia Assessments Included in this Study

For this study, we did not include any EOCT assessments because they were not

available when this study was initiated. However, we were able to examine the reading,

editing, math, and writing sections of the GHSGTs. The reading, editing, and math

sections of the GHSGTs assess student knowledge with multiple-choice items, whereas

the writing section requires a written composition. Although each section of the test has

a recommended time limit, the time limit may be extended at the discretion of the test

proctor if students appear to be making progress and need additional time.

In addition to the GHSGTs, students applying to a college in Georgia are often

required to take placement tests in math and/or English. Because the kinds of placement

tests given are likely to vary by the selectivity of the institution, we obtained placement

tests from both a highly selective university (e.g., University of Georgia) and a less

selective institution (e.g. community colleges). Many of the community colleges in

Georgia administer the COMPASS, which is a system of computer-adaptive assessments

that measure student readiness for a broad range of math and English courses.1 Unlike

traditional paper-and-pencil tests, which administer the same set of items to all

examinees, computer-adaptive measures administer different items to examinees based

1 We examined a set of secured items from the COMPASS item pool. COMPASS is published by ACT.

65

on their demonstrated proficiency level. Because students at different proficiency levels

may encounter differ item content mixes, findings about content coverage or cognitive

demands should be interpreted with caution.

For ELA, we examined assessments in writing skills and reading. For math, we

examined tests in numerical skills/prealgebra and algebra. The COMPASS Writing

Skills, COMPASS Reading, and COMPASS Numerical Skills/Prealgebra tests are used

to determine whether students need to enroll in remedial courses, but the COMPASS

Algebra can be used more broadly to place students into an appropriate math course up to

trigonometry.

The University of Georgia does not use a commercial placement test, but instead

administers their own departmentally-created post-admittance exams; we include their

math and English placement tests. At the University of Georgia, all incoming freshmen,

except those with AP Calculus credit or those who have transferred college-level calculus

credit from another postsecondary institution, are required to take a 45-minutes, 26-item

multiple-choice exam, which is used to place students in an appropriate mathematics

course (hereafter this test will be referred to as the UOG Math Placement Test). The

analogous English test, referred to hereafter as the UOG ELA Placement Test, consists of

two sessions: a morning session in which students are given 55 minutes to complete 60

multiple-choice items, and an afternoon session in which students are to compose an a

writing sample in one hour. The writing session is required only of students whose

performance during the morning session is below a predetermined cut-off score.

High school students may also choose to take the Georgia Early Math Placement

Test (GEMPT) in their junior year. The purpose of this measure is to assess students’

current mathematics proficiency level in relation to the skills required by college-level

mathematics courses. Students are made aware of deficiencies that would require them to

take a remedial mathematics course in college, and are encouraged to take the appropriate

preparatory courses during their senior year in high school so that the students do not

have to take the remedial course at the postsecondary level.

Tables 4.1 and 4.2, organized by test function, list these testing programs and the

type of information we were able to obtain for this study. For most tests, we used a

single form from a recent administration or a full-length, published sample test. In a few

66

instances where full-length forms were unavailable, we used published sets of sample

items. This was the case for the GHSGTs. For the ELA tests, Table 4.2 specifies

whether the test includes each of three possible skills: reading, editing, and writing

67

Tab

le 4

.1

Tec

hnic

al C

hara

cter

isti

cs o

f th

e M

athe

mat

ics

Ass

essm

ents

T

est

Tes

t Typ

e M

ater

ials

E

xam

ined

T

ime

Lim

it

Num

ber

and

Typ

e of

Ite

ms

Too

ls

Pur

pose

C

onte

nt a

s S

peci

fied

in T

est

Spe

cifi

cati

ons

Geo

rgia

Hig

h S

choo

l Gra

duat

ion

Tes

t (G

HSG

T)

Sta

te a

chie

vem

ent

Sam

ple

item

s 18

0 m

inut

es

60 M

C

Cal

cula

tor

Mon

itor

stu

dent

ac

hiev

emen

t to

war

d st

ate-

appr

oved

con

tent

st

anda

rds

Num

ber

and

Com

puta

tion

(17

-19

%),

Dat

a A

naly

sis

(19-

21%

),

Mea

sure

men

t and

Geo

met

ry (

32-

34%

), A

lgeb

ra (

28-3

0%)

AC

T

Col

lege

ad

mis

sion

s Fu

ll s

ampl

e fo

rm

60 m

inut

es

60 M

C

Cal

cula

tor

Sel

ecti

on o

f st

uden

ts f

or h

ighe

r ed

ucat

ion

Pre

alge

bra

(23%

), e

lem

enta

ry

alge

bra

(17%

), in

term

edia

te

alge

bra

(15%

), c

oord

inat

e ge

omet

ry (

15%

), p

lane

geo

met

ry

(23%

) an

d tr

igon

omet

ry (

7%)

SA

T I

C

olle

ge

adm

issi

ons

Full

sam

ple

form

75

min

utes

35

MC

15 Q

C

10 G

R

Cal

cula

tor

Sel

ecti

on o

f st

uden

ts f

or h

ighe

r ed

ucat

ion

Ari

thm

etic

(13

%),

alg

ebra

(3

5%),

geo

met

ry, (

26%

), a

nd

othe

r (2

6%)

SAT

II

Mat

hem

atic

s L

evel

IC

Col

lege

ad

mis

sion

s Fu

ll s

ampl

e fo

rm

60 m

inut

es

50 M

C

Cal

cula

tor

S

elec

tion

of

stud

ents

for

hig

her

educ

atio

n

Alg

ebra

(30

%),

geo

met

ry (

38%

,

spec

ific

ally

pla

ne E

ucli

dean

(2

0%),

coo

rdin

ate

(12%

), a

nd

thre

e-di

men

sion

al (

6%))

, tr

igon

omet

ry (

8%),

fun

ctio

ns

(12%

), s

tati

stic

s an

d pr

obab

ilit

y (6

%),

and

mis

cell

aneo

us (

6%)

SAT

II

Mat

hem

atic

s L

evel

II

C

Col

lege

ad

mis

sion

s Fu

ll s

ampl

e fo

rm

60 m

inut

es

50 M

C

Cal

cula

tor

Sel

ecti

on o

f st

uden

ts f

or h

ighe

r ed

ucat

ion

Alg

ebra

(18

%),

geo

met

ry (

20%

,

spec

ific

ally

coo

rdin

ate

(12%

) an

d th

ree-

dim

ensi

onal

(8%

)),

trig

onom

etry

(20

%),

fun

ctio

ns

(24%

), s

tati

stic

s an

d pr

obab

ilit

y (6

%),

and

mis

cell

aneo

us (

12%

)

Tab

le c

onti

nues

67

68

Tes

t T

est T

ype

Mat

eria

ls

Exa

min

ed

Tim

e L

imit

N

umbe

r an

d T

ype

of I

tem

s T

ools

P

urpo

se

Con

tent

as

Spe

cifi

ed in

Tes

t S

peci

fica

tion

s

CO

MP

AS

S

Alg

ebra

C

olle

ge p

lace

men

t

Full

sam

ple

form

N

o ti

me

lim

it

Num

ber

of M

C

item

s gi

ven

depe

nds

on

abil

ity

Non

e A

sses

s st

uden

t pr

ofic

ienc

y in

al

gebr

a,

inte

rmed

iate

al

gebr

a, a

nd

coor

dina

te

geom

etry

Alg

ebra

, int

erm

edia

te a

lgeb

ra,

coor

dina

te g

eom

etry

CO

MP

AS

S

Num

eric

al

Ski

lls/

Pre

alge

bra

Col

lege

pla

cem

ent

Full

sam

ple

form

N

o ti

me

lim

it

Num

ber

of M

C

item

s gi

ven

depe

nds

on

abil

ity

Non

e A

sses

s pr

ofic

ienc

y in

pre

alge

bra

Pre

alge

bra

Geo

rgia

Ear

ly

Mat

h P

lace

men

t T

est (

GE

MP

T)

Col

lege

pla

cem

ent

Ful

l sam

ple

form

45

min

utes

32

MC

C

alcu

lato

r A

sses

s cu

rren

t m

athe

mat

ics

prof

icie

ncy

leve

l in

rela

tion

to c

olle

ge-

leve

l mat

hem

atic

s co

urse

s

Pre

alge

bra,

ele

men

tary

alg

ebra

, ge

omet

ry, i

nter

med

iate

alg

ebra

Uni

vers

ity

of

Geo

rgia

Mat

h P

lace

men

t Tes

t (U

OG

Mat

h P

lace

men

t Tes

t)

Col

lege

pla

cem

ent

Ful

l sam

ple

form

45

min

utes

26

MC

C

alcu

lato

r P

lace

men

t of

stud

ents

into

ap

prop

riat

e m

ath

cour

se

Ari

thm

etic

, alg

ebra

, geo

met

ry,

trig

onom

etry

(se

e ht

tp:/

/ww

w.m

ath.

uga.

edu/

~cur

r/P

lace

men

tTop

ics.

htm

l for

a

com

plet

e li

st o

f to

pics

)

Not

es.

MC

= m

ultip

le-c

hoic

e O

E =

ope

n-en

ded

G

R =

gri

d-in

Q

C =

qua

ntita

tive

com

pari

son

68

69

Tab

le 4

.2

Tec

hnic

al C

hara

cter

isti

cs o

f th

e E

nglis

h/L

angu

age

Art

s A

sses

smen

ts

Tes

t T

est F

unct

ion

Mat

eria

ls

Exa

min

ed

Tim

e L

imit

N

umbe

r an

d T

ype

of I

tem

s P

urpo

se

Rea

ding

S

ecti

on?

Edi

ting

S

ecti

on?

Wri

ting

S

ecti

on?

Geo

rgia

Hig

h S

choo

l G

radu

atio

n T

est

(GH

SGT

) E

ngli

sh/L

angu

age

Art

s

Sta

te

achi

evem

ent

Sam

ple

item

s 18

0 m

inut

es f

or

both

rea

ding

and

ed

itin

g

Add

itio

nal t

ime

allo

wed

if

stud

ent i

s m

akin

g pr

ogre

ss

50 M

C f

or b

oth

read

ing

and

edit

ing

Mon

itor

stu

dent

ac

hiev

emen

t tow

ard

stat

e st

anda

rds,

pro

vide

hig

h sc

hool

dip

lom

a

Y

Y

N

Geo

rgia

Hig

h S

choo

l W

riti

ng T

est

(GH

SWT

)

Sta

te

achi

evem

ent

Sam

ple

item

s 90

min

utes

for

w

riti

ng s

ecti

on

Add

itio

nal t

ime

allo

wed

if

stud

ent i

s m

akin

g pr

ogre

ss

1 O

E w

riti

ng

Mon

itor

stu

dent

ac

hiev

emen

t tow

ard

stat

e st

anda

rds,

pro

vide

hig

h sc

hool

dip

lom

a

N

N

Y

AC

T

Col

lege

ad

mis

sion

s Fu

ll s

ampl

e fo

rm

80 m

inut

es

--35

min

utes

re

adin

g

--45

min

utes

ed

itin

g

40 M

C r

eadi

ng

75 M

C e

diti

ng

Sel

ecti

on o

f st

uden

ts f

or

high

er e

duca

tion

Y

Y

N

AP

Lan

guag

e an

d C

ompo

siti

on

Col

lege

ad

mis

sion

s

Full

sam

ple

form

18

0 m

inut

es

--60

min

utes

re

adin

g

-- 1

20 m

inut

es

wri

ting

52 M

C r

eadi

ng

1 O

E r

eadi

ng

2 O

E w

riti

ng

Pro

vide

opp

ortu

niti

es f

or

HS

stu

dent

s to

rec

eive

co

lleg

e cr

edit

and

ad

vanc

ed c

ours

e pl

acem

ent

Y

N

Y

SA

T I

C

olle

ge

adm

issi

ons

Full

sam

ple

form

75

min

utes

40 M

C r

eadi

ng

38 M

C e

diti

ng

Sel

ecti

on o

f st

uden

ts f

or

high

er e

duca

tion

Y

Y

N

Tab

le c

onti

nues

69

70

Tes

t T

est F

unct

ion

Mat

eria

ls

Exa

min

ed

Tim

e L

imit

N

umbe

r an

d T

ype

of I

tem

s P

urpo

se

Rea

ding

S

ecti

on?

Edi

ting

S

ecti

on?

Wri

ting

S

ecti

on?

SA

T I

I L

iter

atur

e C

olle

ge

adm

issi

ons

Full

sam

ple

form

60

min

utes

60

MC

rea

ding

S

elec

tion

of

stud

ents

for

hi

gher

edu

cati

on

Y

N

N

SA

T I

I W

riti

ng

Col

lege

ad

mis

sion

s Fu

ll s

ampl

e fo

rm

60 m

inut

es

-- 4

0 m

inut

es

edit

ing

-- 2

0 m

inut

es

wri

ting

60 M

C e

diti

ng

1 O

E w

riti

ng

Sel

ecti

on o

f st

uden

ts f

or

high

er e

duca

tion

N

Y

Y

CO

MP

ASS

Rea

ding

C

olle

ge

plac

emen

t Fu

ll s

ampl

e fo

rm

No

tim

e li

mit

N

umbe

r of

MC

qu

esti

ons

give

n de

pend

s up

on

abil

ity

Ass

ess

whe

ther

adm

itte

d st

uden

ts p

osse

ss e

ntry

le

vel r

eadi

ng s

kill

s

Y

N

N

CO

MP

AS

S W

riti

ng

Skil

ls

Col

lege

pl

acem

ent

Full

sam

ple

form

N

o ti

me

lim

it

Num

ber

of M

C

ques

tion

s gi

ven

depe

nds

upon

ab

ilit

y

Ass

ess

whe

ther

adm

itte

d st

uden

ts p

osse

ss e

ntry

le

vel e

diti

ng s

kill

s

N

Y

N

Uni

vers

ity

of G

eorg

ia

Eng

lish

/Lan

guag

e A

rts

Pla

cem

ent T

est (

UO

G

EL

A P

lace

men

t Tes

t)

Col

lege

pl

acem

ent

Full

sam

ple

form

O

ne 5

5-m

inut

e M

C s

essi

on f

or

both

rea

ding

and

ed

itin

g

One

60-

min

ute

wri

ting

ses

sion

60 M

C f

or b

oth

read

ing

and

edit

ing

1 O

E w

riti

ng

Ass

ess

whe

ther

adm

itte

d st

uden

ts p

osse

ss e

ntry

le

vel E

ngli

sh s

kill

s

Y

Y

Y

Not

es.

MC

= m

ultip

le-c

hoic

e O

E =

ope

n-en

ded

70

71

Alignment Among Georgia Math Assessments

In this section, we describe the results of our alignment exercise for the math

assessments. The results are organized so that alignment among tests with the same

function is presented first, followed by a discussion of alignment among tests with

different functions.

Alignment is described by highlighting similarities and differences with respect to

technical features, content, and cognitive demands. That is, we first present how the

assessments vary along characteristics such as time limit, format, contextualized items,

graphs, diagrams, and formulas. We then document differences with respect to content

areas, and conclude with a discussion of discrepancies in terms of cognitive requirements.

Table 4.3 presents the alignment results for the math assessments. The numbers

in Table 4.3 represent the percent of items falling into each category. As an example of

how to interpret the table, consider the SAT I results; 58% of its items are multiple-

choice, 25% are quantitative comparisons, and 17% are grid-in items. With respect to

contextualization, 25% of the SAT I questions are framed as a real-life word problem.

Graphs are included within the item-stem on 7% of the questions, but graphs are not

included within the response options (0%), and students are not asked to produce any

graphs (0%). Similarly, diagrams are included within the item-stem on 18% of the

questions, but diagrams are absent from the response options (0%), and students are not

required to produce a diagram (0%). With respect to content, the SAT I does not include

trigonometry (0%), and assesses elementary algebra (37%) most frequently. In terms of

cognitive demands, procedural knowledge (53%) is the focus of the test, but conceptual

understanding (32%) and problem solving (15%) are assessed as well. Results for the

other tests are interpreted in an analogous manner.

72

Tab

le 4

.3

Alig

nmen

t A

mon

g th

e T

echn

ical

, Con

tent

, and

Cog

niti

ve D

eman

ds C

ateg

orie

s fo

r th

e M

ath

Ass

essm

ents

For

mat

Con

text

Gra

phs

D

iagr

ams

F

orm

ulas

Con

tent

Cog

niti

ve

Dem

ands

T

est

MC

Q

C

GR

O

E

C

S

RO

P

S

RO

P

M

G

P

A

EA

IA

C

G

PG

T

R

SP

M

ISC

CU

P

K

PS

Sta

te A

chie

vem

ent T

est

GH

SG

T

100

0 0

0

62

9

0 0

21

0

0

11

6

25

18

2 9

29

0 18

0

29

69

2

Col

lege

Adm

issi

ons

Tes

ts

AC

T

100

0 0

0

22

5

2 0

13

0

0

15

0

17

22

5 15

25

8

3 5

40

53

7

SA

T I

58

25

17

0

25

7 0

0

18

0 0

1

8

13

37

2 6

19

0 13

11

32

53

15

SAT

II

Mat

h L

evel

IC

10

0 0

0 0

18

8 0

0

26

0 0

12

0

2

30

10

12

28

4 8

6

34

58

8

SAT

II

Mat

h L

evel

II

C

100

0 0

0

12

12

2

0

2 0

0

10

0

2 14

22

12

14

18

6

12

26

54

20

Col

lege

Pla

cem

ent T

ests

CO

MP

AS

S

Alg

ebra

10

0 0

0 0

5

16

0

0

0 0

0

16

5

11

37

21

26

0 0

0 5

5

90

5

CO

MP

AS

S

Num

eric

al

Skil

ls/P

re-

alge

bra

100

0 0

0

41

0

0 0

0

0 0

0

0

88

0 0

0 0

0 12

0

12

82

6

GE

MP

T

100

0 0

0

6

0 3

0

6 0

0

13

3

3 44

31

9

13

0 0

0

0 10

0 0

UO

G M

ath

Pla

cem

ent

Tes

t 10

0 0

0 0

8

0

8 0

19

0

0

27

0

0 27

27

15

12

19

0

0

23

77

0

. Form

at

Con

text

ualiz

atio

n D

iagr

ams

MC

= m

ultip

le-c

hoic

e ite

ms

C

=

cont

extu

aliz

ed it

ems

S

=

grap

h/di

agra

m w

ithin

item

-ste

m

QC

= q

uant

itativ

e co

mpa

riso

n ite

ms

R

O

=

grap

h/di

agra

m w

ithin

res

pons

e op

tions

G

R =

fill

-in-

the-

grid

item

s

P

=

grap

h/di

agra

m n

eeds

to b

e pr

oduc

ed

Tab

le c

onti

nues

72

73

Not

es.

Form

at

Con

text

ualiz

atio

n D

iagr

ams

MC

= m

ultip

le-c

hoic

e ite

ms

C

=

cont

extu

aliz

ed it

ems

S

=

grap

h/di

agra

m w

ithin

item

-ste

m

QC

= q

uant

itativ

e co

mpa

riso

n ite

ms

R

O

=

grap

h/di

agra

m w

ithin

res

pons

e op

tions

G

R =

fill

-in-

the-

grid

item

s

P

=

grap

h/di

agra

m n

eeds

to b

e pr

oduc

ed

OE

= o

pen-

ende

d ite

ms

OE

= o

pen-

ende

d ite

ms

Form

ulas

C

onte

nt A

reas

C

ogni

tive

Dem

ands

M

=

for

mul

a ne

eds

to b

e m

emor

ized

PA

=

prea

lgeb

ra

C

U

=

conc

eptu

al u

nder

stan

ding

G

=

for

mul

a is

pro

vide

d

EA

=

elem

enta

ry a

lgeb

ra

PK

=

pr

oced

ural

kno

wle

dge

IA

=

inte

rmed

iate

alg

ebra

P

S =

pr

oble

m-s

olvi

ng

CG

=

coor

dina

te g

eom

etry

P

G =

pl

ane

geom

etry

T

R =

tr

igon

omet

ry

SP

=

stat

istic

s an

d pr

obab

ility

MIS

C

=

mis

cell

aneo

us to

pics

73

Tab

le c

onti

nues

74

Alignment Among Tests With the Same Function

State Achievement Tests

Only one state achievement test is included in this analysis, the GHSGT Math. It

is a 3-hour, 60-item, multiple-choice test assessing math knowledge up to geometry.

Most of its items assess procedural knowledge (69%), but a moderate proportion assesses

conceptual understanding as well (29%).

College Admissions Tests

We examined four college admissions tests: the ACT, SAT I, SAT II Math Level

IC, and SAT II Math Level IIC. All tests, except the SAT I, have a one-hour time limit.

The SAT I has a 75-minute time limit. All four exams are also predominantly multiple-

choice, although the SAT I includes quantitative comparison (25%) as well as grid-in

(17%) items. Contextualized questions are most prevalent on the SAT I (25%) and least

prevalent on the SAT II Math Level IIC (12%). Students are rarely asked to work with

graphs, and questions that contain graphs within the item-stem constitute no more than

12% of items on the college admissions measures. Questions that include diagrams

within the item-stem are more prevalent, comprising 26%, 18%, and 13% of items on the

SAT II Math Level IC, SAT I, and ACT, respectively. However, questions with

diagrams are infrequent on the SAT II Math Level IIC (2%). Formulas are also

uncommon, but there are differences with respect to the extent to which formulas are

necessary. Whereas the ACT, SAT II Math Level IC, and SAT II Math Level IIC include

items in which a memorized formula is needed (15%, 12%, and 10%, respectively), these

items are largely absent from the SAT I (1%).

Although college admissions exams generally sample from the same content

areas, they do not do so to the same extent. Elementary algebra comprises most of the

SAT I items (37%). The SAT II Math Level IC also emphasizes elementary algebra

(30%), but focuses on planar geometry as well (28%). The ACT shows a similar content

emphasis as that of the SAT II Math Level IC; 22% of its items assess elementary algebra

and 25% assess planar geometry. The SAT II Math Level IIC, on the other hand, draws

from more advanced content areas, such as intermediate algebra (22%) and trigonometry

(18%).

75

In terms of cognitive demands, all four tests assess procedural knowledge to a

similar degree. Procedural knowledge items constitute between 54% and 58% of the

items found on college admissions measures. However, there is more variation among

the exams with respect to emphasis on problem solving. The SAT I and SAT II Math

Level IIC place relatively greater emphasis on problem solving (20% and 15%,

respectively) than do the ACT and SAT II Math Level IC (7% and 8%, respectively).

College Placement Tests

There are four college placement tests in this analysis: the COMPASS Algebra,

COMPASS Numerical Skills/Prealgebra, GEMPT, and UOG Math Placement Test. All

four tests consist only of multiple-choice items, but the COMPASS Algebra and

COMPASS Numerical Skills/Prealgebra are computer-adaptive measures whereas the

GEMPT and UOG Math Placement Test are paper-and-pencil measures. The

COMPASS Numerical Skills/Prealgebra contains many items framed in a real-life

context (41%), but contextualized items are largely excluded from the COMPASS

Algebra, GEMPT, and UOG Math Placement Test (5%, 6%, and 8%, respectively).

Questions in which graphs are included within the item-stem are absent on all of the

college placement measures except the COMPASS Algebra (16%). Items that contain

diagrams are excluded from the COMPASS Algebra and COMPASS Numerical

Skills/Prealgebra, but comprise a small proportion of the GEMPT (6%), and a moderate

proportion of the UOG Math Placement Test (19%). With respect to items requiring a

memorized formula, the COMPASS Numerical Skills/Prealgebra excludes such items,

but the other college placement measures include this type of question to some extent

(13%-27%).

With the exception of the COMPASS Numerical Skills/Prealgebra, which is

narrowly focused on a single topic area, namely prealgebra (88%), the other college

placement measures show a broad range of content sampling. The COMPASS Algebra

and the GEMPT assess elementary algebra most frequently, (37% and 44%, respectively)

but also assess intermediate algebra to a moderate extent (21% and 31%, respectively).

Content differences between these two tests are most evident with respect to sampling of

coordinate geometry, which comprises 26% of items on the COMPASS Algebra, but 9%

76

of items on the GEMPT. As with the COMPASS Algebra and the GEMPT, the UOG

Math Placement Test focuses most on elementary algebra and intermediate algebra (27%

of items on each content area). However, the UOG Math Placement test also assesses

trigonometry (19%), a topic that is devoid from all other college placement measures.

Few differences are observed with respect to cognitive requirements. Procedural

knowledge problems constitute the majority of the college placement measures, ranging

from 77% of the UOG Math Placement Test questions to 100% of the GEMPT questions.

Problem-solving items are uncommon (0%-6%), as are conceptual understanding items,

although conceptual understanding is represented on the UOG Math Placement test to a

moderate extent (23%).

Alignment Among Tests with Different Functions

With the exception of the SAT I, none of the math assessments requires students

to generate their own answers. Excluding the COMPASS Numerical Skills/Prealgebra

(41%), questions framed within a realistic context typically represent a small proportion

of college placement tests (5%-8% if the COMPASS Numerical Skills/Prealgebra is

excluded), a small to moderate proportion of college admissions measures (12%-25%),

and is most prevalent on the state achievement test (62%). Questions that contain graphs

within the item-stem are relatively uncommon, comprising no more than 16% on any of

the studied exams. Questions that contain diagrams within the item-stem are more

prevalent, but typically represent only a small or moderate fraction of a test. Questions

that contain diagrams within the item-stem represent 2%-26% of college admissions

items, 0%-19% of college placement items, and 21% of the GHSGT. Items requiring a

memorized formula are generally rare, and comprise less than 16% on all but one test.

The exception is the UOG Math Placement Test, where memorized formulas constitute

27% of its items.

With respect to the content category, college admissions exams assess logic

(coded as miscellaneous) and trigonometry more frequently than do the state achievement

or college placement measures. Trigonometry items, for example, are included on 4%-

18% of college admissions tests (if the SAT I is excluded), 0% of the GHSGT, and 0% of

college placement tests, the UOG Math Placement Test notwithstanding. The

77

anomalously high proportion of trigonometry items found on the UOG Math Placement

Test (19%) reflects the fact that this exam is used to place students into calculus, whereas

the other college placement measures are used to place students into either remedial

courses (COMPASS Numerical Skills/Prealgebra) or math courses no higher than

trigonometry (COMPASS Algebra). The state achievement test, the GHSGT, is broadly

distributed across a range of content areas, particularly prealgebra and planar geometry

(25% and 29%, respectively).

In terms of cognitive requirements, all tests emphasize procedural knowledge

most frequently, although there is variation with respect to extent. Procedural knowledge

items are most common on college placement tests (77%-100%), followed by the state

achievement test, the GHSGT (69%), and least common on college admissions tests

(53%-58%). Problem-solving items are most prevalent on college admissions (7%-20%)

and are rarely included on either the GHSGT (2%) or college placement tests (0%-6%).

Conceptual understanding items are also most likely on college admission exams,

comprising between 26%-40% of the items.

Discussion

Below, we discuss the implications of the discrepancies among the math

assessments. We begin by highlighting instances in which differences are justified, then

address whether there were any misalignments that may send students confusing signals.

We also explore the possibility that state achievement tests can inform postsecondary

decisions.

Which Discrepancies Reflect Differences in Test Use?

As noted in Chapter 1, content discrepancies may reflect differences in intended

test use. To illustrate, consider the SAT II Math Level IIC and GHSGT. Although both

the SAT II Math Level IIC and GHSGT include topics from a wide variety of courses,

the SAT II Math Level IIC includes many trigonometry and problem-solving items (18%

and 20%, respectively), whereas the GHSGT rarely includes such material (0% and 2%,

respectively). In this particular case, the SAT II Math Level IIC and GHSGT have

disparate functions, and content differences reflect variations in purpose. Because the

78

SAT II Math Level IIC is used to select among higher-achieving students for entrance

into universities and colleges, it includes many problem-solving and trigonometry items

in order to distinguish among higher-proficiency examinees. The GHSGT, on the other

hand, is used to monitor student achievement statewide, and therefore requires items of

more moderate difficulty that can be attempted by students with a wider range of

proficiency levels.

The above example represents justifiable discrepancies across tests with different

purposes. However, there are also instances in which discrepancies within tests of

similar purposes are warranted as well. As mentioned earlier, that trigonometry is

prevalent on the UOG Math Placement Test but absent from the COMPASS Algebra is

justifiable given that the former test is used to place students into calculus, whereas the

latter test is used to place students into math courses no higher than trigonometry.

Likewise, differences between the SAT I and ACT with respect to emphasis on problem-

solving or non-routine logic problems is indicative of the fact that the SAT I is intended

to be a reasoning measure, but the ACT is intended to assess content found in high-school

math courses.

Is There Evidence of Misalignment?

As defined in Chapter 1, misalignments refer to those discrepancies that are not

attributable to test function, and therefore send students confusing signals regarding the

kinds of skills that are needed to perform well on a given test. In our analysis of the

math tests, we could not find any examples of misalignments, as discrepancies among

college admissions, college placement, and state achievement measures appear to have

stemmed from variations in test use. Additionally, differences among tests of similar

purposes are either small or moderate, or reflect nuances in purpose (e.g., see the above

UOG Math Placement Test and COMPASS Algebra example).

Can State Achievement Tests Inform Postsecondary Admissions and Course Placement Decisions?

Although there are many discrepancies among exams of different functions, it

may still be possible that a test can serve multiple purposes satisfactorily. Currently,

79

some measures are used for more than one purpose. Many postsecondary institutions, for

example, allow students to submit scores from college admissions exams such as the SAT

I or ACT as a means of exemption from a remedial college placement test. Potentially,

state achievement tests can be used for similar purposes, but no postsecondary institution

to date has made use of GHSGT scores for placement decisions. However, policymakers

have advocated using scores on some graduation tests for purposes beyond monitoring

student achievement (Olson, 2001b; Schmidt, 2000) because such a policy change would

not only reduce testing burden, but it would also motivate students to focus on state

standards rather than on external tests like the SAT I or ACT (Healy, 2001; Olson, 2001a;

Standards for Success, 2001). Below, we discuss the potential of the GHSGT for guiding

college placement and admissions decisions. We also discuss the possibility that testing

burden can be eased by using scores from the SAT II Math Level IIC for other purposes,

namely placement into an appropriate math course.

It may be possible to use GHSGT scores to exempt students from remedial

courses, as its content is more rigorous than that of the COMPASS Numerical

Skills/Prealgebra. Because the GHSGT assesses elementary algebra and geometry more

frequently than does the COMPASS Numerical Skills/Prealgebra, a sufficiently high

score on the GHSGT should logically excuse students from having to take the

COMPASS Numerical Skills/Prealgebra.

However, the GHSGT cannot be used to place students into an appropriate math

course because it rarely assesses material beyond geometry. Compared to the

COMPASS Algebra (which assesses intermediate algebra) or the UOG Placement Test

(which assess trigonometry), the GHSGT would not provide sufficient information

regarding whether an examinee has the necessary background to enroll in a high-level

math course such as trigonometry or calculus.

For the same reason, the GHSGT is not a viable alternative to any of the college

admissions measures. It contains fewer intermediate algebra and trigonometry items

than either the SAT II Math Level IC or SAT II Math Level IIC, and contains a lower

proportion of problem-solving items than either the SAT I or ACT. Because the GHSGT

does not sample as extensively from these content areas, its discriminating power is likely

to be more limited than that of the college admissions exams.

80

It may be possible to ease students’ testing burden in other ways beyond

expanding the use of the GHSGT scores. For instance, the SAT II Math Level IIC can be

used to place students into an appropriate math course up to calculus. The SAT II Math

Level IIC contains approximately the same proportion of intermediate algebra items as

the COMPASS Algebra, as well as the same proportion of trigonometry items as the

UOG Math Placement Test. The SAT II Math Level IIC also has the added advantage of

assessing problem-solving to a greater extent than do these two other tests. Conceivably,

academic counselors can use the scores from the SAT II Math Level IIC to advise

students which math course is most appropriate. To determine the feasibility of the SAT

II Math Level IIC as a measure that informs placement decisions, more research is

needed to explore the interrelationships between the SAT II Math Level IIC, the UOG

Math Placement Test, and grades in higher-level math courses (i.e., calculus).

Alignment Among Georgia ELA Assessments

Below we present the ELA results. As with math, we discuss discrepancies both

within and across test functions. The results are also organized by skill, namely reading,

editing, and writing. In some instances, there are only two tests that assess the same skill

and share the same function, so it is important to recognize that patterns or comparisons

between these tests may not be indicative of more general trends within this category of

tests.

Alignment is characterized by describing differences with respect to technical

features, content, and cognitive demands. Specifically, we discuss differences in time

limit and format, then document discrepancies with respect to topic, voice, and genre of

the reading passages, before concluding with variations in cognitive processes.

The alignment results for tests that measure reading skills are presented in Tables

4.4-4.5. Tables 4.6-4.7 provide the results for exams that assess editing skills, and Tables

4.8-4.9 provide the findings for exams that assess writing skills. For each table, the

numbers represent the percent of items falling in each category. To provide a concrete

example of how to interpret the findings, consider the content category results for the AP

Language and Composition, presented in Table 4.4. With respect to topic, 50% of the

reading passages included on the AP Language and Composition are personal accounts,

81

whereas 25% of the topics are about humanities, and the remaining 25% are about natural

science. It does not include topics from fiction or social science (0% each). In terms of

the author’s voice, 75% of the passages are written in a narrative style, whereas the other

25% are written in an informative manner. With respect to genre, only essays (100%) are

used; passages on the AP Language and Composition are not presented as letters, poems,

or stories (0% each). Results for the other tests are interpreted in a similar manner.

82

Tab

le 4

.4

Alig

nmen

t W

ithi

n th

e C

onte

nt C

ateg

ory

for

the

Rea

ding

Pas

sage

s

Top

ic

V

oice

Gen

re

Tes

t

Fic

tion

H

uman

itie

s N

atur

al

Sci

ence

S

ocia

l S

cien

ce

Per

sona

l A

ccou

nts

N

arra

tive

Des

crip

tive

Per

suas

ive

Info

rmat

ive

L

ette

r E

ssay

P

oem

St

ory

Sta

te A

chie

vem

ent T

est

GH

SG

T

4

5 9

9 14

23

73

0 2

25

2

55*

16*

30

Col

lege

Adm

issi

ons

Tes

ts

AC

T

25

25

25

25

0

25

25

25

25

0 75

0

25

AP

Lan

guag

e an

d C

ompo

siti

on

0

25

25

0 50

0 25

25

0

0

100

0 0

SA

T I

20

40

20

20

0

20

40

20

20

0

80

0 20

SA

T I

I L

iter

atur

e

63

0 0

13

25

63

0

0 13

13

25

50

13

Col

lege

Pla

cem

ent T

ests

CO

MP

AS

S R

eadi

ng

13

25

25

25

13

38

0 0

63

0

88

0 13

UO

G E

LA

Pla

cem

ent T

est

0

50

0

0 50

50

0 0

50

0

100

0 0

Not

e.

* O

ne p

assa

ge c

onta

ined

bot

h an

ess

ay a

nd a

poe

m.

82

83

Reading Measures

Alignment Among Tests of the Same Function


The only state achievement test included in our sample is the GHSGT, which

assesses reading proficiency solely with multiple-choice items. Because the GHSGT

does not contain separate sections for editing and reading items, we cannot determine

testing time earmarked specifically for assessing reading skills, although testing time

devoted to assessing both types of skills is 180 minutes (see Table 4.2).

Most of the reading passages on the GHSGT are fiction (45%), written in a

narrative style (73%), and presented as an essay (55%) (see Table 4.4). The GHSGT

reading items assess inference skills (70%) most frequently, although a moderate

proportion of questions also assess recall skills (30%) (see Table 4.5).

Table 4.5 Alignment Within the Cognitive Demands Category for Tests

Measuring Reading Skills


Four college admissions exams assess reading proficiency: the ACT, AP

Language and Composition, SAT I, and SAT II Literature. With the exception of the AP

Language and Composition, no college admissions test assesses reading skills with open-

ended items. Testing time devoted to measuring reading skills is 60 minutes for both the

Test Recall Inference Evaluate Style

State Achievement Test

GHSGT 30 70 0


ACT 58 42 0

AP Language and Composition 23 77 0

SAT I 18 83 0

SAT II Literature 13 80 7


COMPASS Reading 75 25 0

UOG ELA Placement Test 46 54 0

84

SAT II Literature and AP Language and Composition. Because the SAT I does not

contain separate sections for editing and reading items, we cannot determine testing time

earmarked specifically for assessing reading proficiency, although testing time devoted to

assessing both types of skills is 75 minutes (see Table 4.2).

Reading passage topics also vary from one measure to the next (see Table 4.4).

The SAT II Literature emphasizes fiction (63%) whereas the AP Language and

Composition emphasizes personal accounts (50%). The SAT I favors humanities (40%),

but the ACT is evenly distributed among fiction, humanities, natural science, and social

science (25% each). Narrative pieces are included on all college admissions measures,

and range from 40% of the SAT I passages to 100% of the SAT II Literature passages.

Essay is generally the most common genre, appearing on 75% of the ACT, 80% of the

SAT I, and 100% of the AP Language and Composition passages. However, the SAT II

Literature is more likely to include poems (50%) than essays (25%). With the exception

of the ACT, college admission exams place greatest emphasis on interpretation and

analysis of the reading passages. Inference items range from 42% of the ACT questions

to 83% of the SAT I questions (see Table 4.5).


Two college placement tests, the COMPASS Reading and the UOG ELA

Placement Test, contain reading items. Both assess reading proficiency with the

multiple-choice format. As with the GHSGT and SAT I, the UOG ELA Placement Test

does not specify testing time devoted specifically to measuring reading proficiency.

Because the COMPASS Reading is a computer-adaptive measure, testing time varies by

examinee (see Table 4.2).

Reading topics on the COMPASS Reading are distributed among humanities,

natural science, and social science (25% each), but reading topics on the UOG ELA

Placement Test are distributed between humanities and personal accounts (50% each)

(see Table 4.4). With respect to author’s voice, the COMPASS Reading favors

informative works (63%), whereas the UOG ELA Placement Test is evenly split between

narrative and informative pieces (50% each). Essay is the most commonly used genre on

either test (88%-100%). For the cognitive demands category, the COMPASS Reading

85

emphasizes recall to a great extent (75%), but the UOG ELA Placement Test focuses on

both recall and inference (46% and 54%, respectively) (see Table 4.5).

Alignment Among Tests of Different Functions

Across all measures, reading skills are assessed primarily with multiple-choice

items. Among the assessments that contain separate sections for reading, testing time

ranges from 35 minutes for the ACT to 60 minutes for the SAT II Literature and AP

Language and Composition. All assessments contain reading passages on two or more

topics, and every reading assessment includes a topic from humanities except the SAT II

Literature. Every measure also includes both narrative and informative passages. Essay

is the most prevalent genre, comprising 55% of the GHSGT, 88% -100% of college

placement, and 75%-100% of college admissions tests, the SAT II Literature exam

notwithstanding. Instead, the SAT II Literature favors poems. Both the GHSGT (70%)

and most of the college admissions measures focus heavily on inference questions (77%-

83%, excluding the ACT), whereas college placement measures place relatively greater

emphasis on recall (46%-75%).

Editing Measures



The GHSGT is the only measure that falls within this category. Again, we cannot

determine testing time devoted to assessing editing skills because editing and reading

items are combined within a single section (see Table 4.2). As with reading, editing

proficiency is assessed only with multiple-choice items. Most of the passages included

on the GHSGT editing section are fictional works (60%) in which the author uses a

narrative voice (80%). Story is the most common genre (60%), although a significant

number of passages are also presented as essays (40%) (see Table 4.6). With respect to

cognitive demands, inference skills are assessed most frequently (44%), although a

moderate proportion of items also assess recall (30%) and evaluate style (26%) skills.

86

Tab

le 4

.6

Alig

nmen

t W

ithi

n th

e C

onte

nt C

ateg

ory

for

the

Edi

ting

Pas

sage

s

Top

ic

V

oice

Gen

re

Tes

t

Fic

tion

H

uman

itie

s N

atur

al

Sci

ence

S

ocia

l S

cien

ce

Per

sona

l A

ccou

nts

N

arra

tive

Des

crip

tive

Per

suas

ive

Info

rmat

ive

L

ette

r E

ssay

P

oem

St

ory

Sta

te A

chie

vem

ent T

est

GH

SG

T

60

20

20

0

0

80

0 0

20

0

40

0 60

Col

lege

Adm

issi

ons

Tes

ts

AC

T

0

60

20

0 20

40

0 0

60

0

100

0 0

SA

T I

N/A

N/A

N/A

SA

T I

I W

riti

ng

0

100

0 0

0

50

0 0

50

0

100

0 0

Col

lege

Pla

cem

ent T

ests

CO

MP

AS

S W

riti

ng S

kill

s

0 0

50

50

0

0 0

0 10

0

0 10

0 0

0

UO

G E

LA

Pla

cem

ent T

est

N

/A

N

/A

N

/A

86

87

Table 4.7 Alignment Within the Cognitive Demands Category for Tests

Measuring Editing Skills Test Recall Inference Evaluate Style


GHSGT 30 44 26


ACT 48 4 48

SAT I 0 100 0

SAT II Writing 50 3 47


COMPASS Writing Skills 80 3 17

UOG ELA Placement Test 100 0 0


Items measuring editing skills are included on three college admissions tests, the

ACT, SAT I, and SAT II Writing. The exams are predominantly multiple-choice, with

testing time ranging from 40 minutes for the ACT to 45 minutes for the SAT II Writing

(see Table 4.2). As mentioned earlier, the SAT I does not specify the specific amount of

testing time devoted to measuring editing skills.

The SAT I does not include a reading passage, but instead uses a few sentences as

item prompts. In contrast, the ACT and SAT II Writing include reading passages. These

reading passages are typically essays about humanities, and written in either a narrative

or informative voice (see Table 4.6). The ACT and SAT II Writing items are equally

distributed among recall (48% and 50%, respectively) and evaluate style items (48% and

47%, respectively), but the SAT I assesses only inference skills (100%) (see Table 4.7).


Two college placement tests, the COMPASS Writing Skills and UOG ELA

Placement Test, assess editing skills via multiple-choice items. Neither test specifies the

time limit devoted solely to measuring editing skills (see Table 4.2). The UOG ELA

Placement Test uses sentences to assess editing skills, but the COMPASS Writing Skills

includes passages. Topics on the COMPASS Writing Skills are evenly split between

88

natural science and social science (50%), and passages are typically essays (100%) in

which the author uses an informative voice (100%) (see Table 4.6). In terms of cognitive

demands, both tests emphasize recall items (80%-100%).


Across all measures, editing skills are assessed solely with multiple-choice items.

All tests, except the SAT I and UOG ELA Placement Test, include reading passages.

(The SAT I and UOG ELA Placement Test use sentences as prompts). Of those

measures that include a reading passage, all include an essay, written in a narrative or

informative voice. More variation is observed with respect to reading passage topics.

The GHSGT favors fiction (60%) whereas humanities topics are most likely to appear on

the ACT or SAT II Writing (60% and 100%, respectively). In contrast, natural science

and social science are the most frequent topics on the COMPASS Writing Skills (50%

each topic).

Both the college placement and college admissions exams tend not to span the full

spectrum of the cognitive demands category. The ACT and SAT II Writing emphasize

recall and evaluate style items, but are generally devoid of inference items, whereas the

reverse is true for the SAT I. College placement measures focus heavily on recall items

(80%-100%) and rarely include inference and evaluate style items. In contrast, the

GHSGT assesses all three levels of the cognitive demands category; the test is almost

evenly distributed among recall, inference, and evaluate style items (30%, 44%, and 26%,

respectively).

Writing Measures



The GHSWT requires students to provide one writing sample within a 90-minute

testing session (see Table 4.2). Topics are typically drawn from humanities or personal

accounts (see Table 4.8). With respect to scoring criteria, GHSWT requires students to

demonstrate mechanics, word choice, style, organization, and insight (see Table 4.9).

89

Table 4.8 Alignment Among the Writing Prompt Topics

Table 4.9 Alignment Among the Scoring Criteria for Tests Measuring Writing Skills


Of the college admissions measures, only the SAT II Writing and AP Language

and Composition require a writing sample. The SAT II Writing provides students with a

one- or two-sentence writing prompt on a topic (usually humanities), and allows 20

minutes for students to respond (see Tables 4.2 and 4.8). In contrast, prompts on the AP

Language and Composition are typically reading passages, and students are required to

provide a total of three writing samples in over two hours (see Table 4.2).2 Topics can

2 The AP Language and Composition requires a total of three writing samples, two of which are produced during the 120-minute writing session, and one during the 60-minute reading session. However, because examinees also respond to a set of multiple-choice items during the reading session, it is unknown the amount of time students devote specifically to the writing sample.

Topic

Test Fiction Humanities Natural Science

Social Science

Personal Accounts


GHSWT X X


AP Language and Composition X X

SAT II Writing X

College Placement Test

UOG ELA Placement Test X X X X

Scoring Criteria Elements Test

Mechanics Word Choice Organization Style Insight


GHSWT X X X X X


AP Language and Composition X X X X X

SAT II Writing X X X X

College Placement Test

UOG ELA Placement Test X X X X X

90

vary, but are usually about humanities or personal accounts (see Table 4.8). The AP

Language and Composition emphasizes all elements of the scoring criteria, but the SAT

II Writing downplays the importance of insight (see Table 4.9).


Only one college placement exam, the UOG ELA Placement Test, requires a

written composition. Students are to write a single composition within a 60-minute

session (see Table 4.2). Topics vary widely from humanities and natural science to social

science and personal accounts (see Table 4.8). The scoring rubric emphasizes mechanics,

word choice, organization, style, and insight.


Writing measures can vary from 20 minutes for a single writing sample (SAT II

Writing) to over 2 hours for three writing samples (AP Language and Composition).

Humanities and personal accounts are the most common topic, as every test includes a

writing prompt from one, if not both, of these areas. All but one writing test emphasizes

mechanics, word choice, organization, style, and insight. The exception is the SAT II

Writing, which omits insight from its scoring criteria.

Discussion

Our discussion of the discrepancies among ELA assessments parallels that of the

math discussion. We first identify examples of discrepancies that are justifiable, then

discuss the implications of the misalignments. We also discuss the feasibility of using

state achievement tests to inform admissions decisions.

Which Discrepancies Reflect Differences in Test Use?

As in math, some discrepancies among the ELA assessments reflect differences in

purpose. Consider, for instance, discrepancies between the scoring standards of the

GHSWT and the AP Language and Composition. For the former test, maximum scores

are awarded to writing samples that have minor diction errors, mechanics lapses, and

underdeveloped paragraphs. Under the AP Language and Composition guidelines, such

91

compositions might receive adequate scores, but would not be viewed as exemplary

papers. Because the AP Language and Composition is used to award academic credit to

students who demonstrate college-level proficiency, whereas the GHSWT is used to

monitor the achievement of all students within the state, including those not planning to

attend a postsecondary institution, discrepancies between their scoring criteria are

warranted.

Even when two measures have similar test functions, discrepancies may still be

warranted. For example, the large discrepancy between the SAT I (100%) and the ACT

(4%) and the SAT I and SAT II Writing (3%) with respect to inference items is

attributable to subtleties in purpose. The SAT I is intended to be a measure of reasoning

proficiency, so great emphasis on inference questions is justifiable. The ACT and SAT II

Writing, on the other hand, are curriculum-based measures, so relatively greater focus on

skills learned within English classes (i.e., recall and evaluate style skills) is to be

expected.

Is There Evidence of Misalignment?

Although the majority of the ELA discrepancies is small or moderate, or stems

from variations in test function, one instance of misalignment pertains to the scoring

criteria of the SAT II Writing. Insight is included within the scoring criteria of the

GHSWT, AP Language and Composition, and UOG ELA Placement test, but is omitted

from the scoring rubrics of the SAT II Writing. Given that insight is included in the

standards of most English courses, it appears that the SAT II Writing standards are

incongruent with those that are typically expressed. Potentially, this misalignment can

send students mixed messages about the importance of insight with respect to writing

skills. If the developers of the SAT II Writing were to add insight to the scoring criteria,

or provided a clear rationale of why insight has been omitted from the scoring rubrics,

students would receive a more consistent signal about the importance of insight with

respect to writing proficiency.

92

Can State Achievement Tests Inform Postsecondary Admissions Decisions?

As mentioned earlier, policymakers are exploring the possibility that scores on

graduation tests can be used to inform college admissions decisions. In reading, that the

GHSGT assesses inference skills to approximately the same extent as college admission

measures may suggest that the GHSGT might be a viable alternative to college

admissions tests. However, inference items can vary with respect to cognitive

sophistication elicited. A previous study by Education Trust (1999) showed that ELA

inference items could vary greatly with respect to nuance of interpretations. Given the

differences in the intended test uses, it is very likely that inference items on the GHSGT

may not be as complex as that elicited by college admissions exams. More research

needs to be conducted to determine whether the GHSGT can discriminate among higher-

achieving examinees as well as college admissions exams.

With respect to assessing writing skills, the GHSWT holds more promise as an

alternative to college admissions tests. Neither the ACT nor the SAT I requires a writing

sample, and the SAT II Writing allows 20 minutes for a writing sample. Given the short

time limit, the SAT II Writing composition represents a very limited indicator of writing

proficiency. The GHSWT, on the other hand, allows students 90 minutes for a

composition. Arguably, the GHSWT would allow admissions officers to better judge

applicants’ writing skills than the ACT, SAT I, or SAT II Writing. However, as

discussed earlier, the current scoring rubrics for the GHSWT may not be rigorous enough

to be of use for some institutions, especially the selective ones. Therefore, changes to the

scoring guidelines may need to be implemented if the GHSWT writing samples were to

be used to inform admissions decisions at these higher-selectivity schools. Again, any

policy changes regarding the use of the GHSWT to inform admissions decisions will

require more research, particularly the relationship between the GHSWT scores and first-

year college grade point average.

Chapter 4 Alignment Among Secondary and … · ... (ninth grade language and composition and...

Documents

Transcript of Chapter 4 Alignment Among Secondary and … · ... (ninth grade language and composition and...