The Effect of Testing on Student Achievement 1910-2010 Slide Show

7/21/2019 The Effect of Testing on Student Achievement 1910-2010 Slide Show

1/34

2012, Richard P PHELPS World Association of Education Research, 17th Congress,Reis, !une, 2012

1

"he effect of testing on student achie#eent$

1%10&2010

Richard P' PHELPS


2/34


2

(eta&anal)sis

* A ethod for

suari+ing a largeresearch literature,ith a single,

co-ara.le easure'


3/34

2012, Richard P PHELPS World Association of Education Research, 17th Congress,Reis, !une, 2012 /

"he effect of testing on student

achie#eent

* 12&)ear long stud)

* anal)+ed close to 700 se-arate studies,and ore than 1,00 se-arate effects

* 2,000 other studies ere re#ieed andfound inco-lete or ina--ro-riate

* lacing sufficient tie and one),hundreds of other studies ill not .ere#ieed


4/34


Looing for studies to include in the

eta&anal)ses

1' 3ncluded onl) those studies that found an effect fro

testing on student achie#eent or on teacher

instruction4


5/34

2012, Richard P PHELPS World Association of Education Research, 17th Congress,Reis, !une, 2012 5

Studies included in the eta&anal)ses

2' 4hen$

* a test is nel) introduced, or nel) reo#ed* 6uantit) of testing is increased or reduced

* test staes are introduced or increased, or reo#ed orreduced


6/34


Studies included in the eta&anal)ses

/' 4-lus -re#ious research suaries e'g'8

* 9uli, 9uli, :angert&;rons, aster) testing,> fre6uenc) of testing, and

> -rogras for high&ris uni#ersit)students

* :asol < !ohanson 200%8 on testing

fre6uenc)* !ae)ung Lee 20078 on cross&state

studies

* W'!' Ha)nie 20078 in career&tech ed


7/34


?u.er of studies of effects,

.) ethodolog) t)-e

(ethodolog) t)-e?u.er of

studies?u.er of

effects

@uantitati#e 177 0

Sur#e)s and -u.lico-inion -olls S < Canada8

27 =1/

@ualitati#e 25 25

"B"AL % 1%=


8/34

2012, Richard P PHELPS World Association of Education Research, 17th Congress,Reis, !une, 2012 =

Effect si+e$ Cohens d

d D E& C8 F S-ool

ED ean, eG-eriental grou-

CD ean, control grou-S-ooled D standard de#iation


9/34

2012, Richard P PHELPS World Association of Education Research, 17th Congress,Reis, !une, 2012 %

Effect si+e$ Bther forulae

d D tn1In2Fn1n28J0'5

d D 2rF1&rK8J0'5

d D E-re&E-ost&C-reI

C -ost8FS-ooled -ost


10/34


Effect si+e$ 3nter-retation

* d .eteen 0'25 < 0'50 ea effect

* d .eteen 0'50 et 0'75 ediu effect

* d ore than 0'75 strong effect


11/34


@uantitati#e studies

-o-ulation co#erage 7 illion -ersons8


12/34


@uantitati#e studies$ Effect si+e

* M:are .onesN calculation$

d I0'55 4a ediu effect

* :are .ones effect si+e adOusted for easureent error

d I0'71 4a stronger effect

* sing sae&stud)&author aggregation

d I0'== 4a strong effect


13/34

2012, Richard P PHELPS World Association of Education Research, 17th Congress,Reis, !une, 2012 1/

Which -redictors atter

Treatment Group4Mean

Efect Size

4is made aware o perormance, and controlgroup is not

+0.98

4receies targeted instruction !e.g.,remediation"

+0.9#

4is tested wit$ $ig$er sta%es t$an control group +0.8&4is tested more re'uentl( t$an control group +0.8)


14/34


1

(ore (oderators > Source of "est

*umero

Studies

MeanEfectSize

esearc$er orTeac$er

8& 0.9-

*ational / 0.8&

ommercial -8 0.8

State or 1istrict 22 0.&

Total 160


15/34


15

(ore (oderators > S-onsor of "est

*umero

Studies

MeanEfectSize

3nternational ) 2.0

4ocal 99 0.9-

*ational /) 0.82

State 22 0.#/Total 160


16/34

2012, Richard P PHELPS World Association of Education Research, 17th Congress,

Reis, !une, 2012

1

(ore (oderators & Stud) ;esign

*umer oStudies

MeanEfectSize

5re6post 2 0.9&E7periment, uasi6e7periment

20& 0.9/

Multiariate # 0.80

E7periment, posttestonl(

& 0.#0

5re6post !wit$ s$adowtest"

8 0.)8

Total 160


17/34


Reis, !une, 2012

17

(ore (oderators > Scale of Anal)sis

*umero

Studies

MeanEfect

Sizeggregated 9 2.#0

Small6scale 228 0.92

4arge6scale -- 0.)&

Total 160


18/34


Reis, !une, 2012

1=

(ore (oderators > Scale of Adinistration

*umero

Studies

MeanEfect

Sizelassroom 22) 0.9)

Mid6scale # 0.&

4arge6scale -9 0.&2

Total 160


19/34


Reis, !une, 2012

1%

Sur#e)s and o-inion -olls


20/34


Reis, !une, 2012

20

Percentage of sur#e) ites,

.) res-ondent grou- and t)-e of sur#e)


21/34


Reis, !une, 2012

21

?u.er and -ercent of sur#e) ites,

.) test staes and target grou-

Test stakes Number % Target group Number %

High 507 2 Students /%/

(ediu 1= 2/ Schools 2=1 //

Lo // "eachers 11 1

nnon =% 11 ?o staes 7

"B"AL =1/ "B"AL =5


22/34


Reis, !une, 2012

22

B-inion -olls, .) )ear

* 2 .eteen 1%5=&&200=, in the 'S' < Canada* =1/ uni6ue 6uestion&res-onse co.inations

* close to 700,000 indi#idual res-ondents


23/34


Reis, !une, 2012

2/

Surveys and opinion polls:

Regular standardized tests, performance tests

Regular tests(N !"#

$erformance tests(N "#

Res-ondent o-inion d d

Achie#eent is increased 1'2 1'0

4eighted .) si+e of stud) -o-ulation 1'% 0'5

3nstruction is i-ro#ed 1'0 1'

4eighted .) si+e of stud) -o-ulation 0'% 0'%

"ests hel- align instruction 1'0 1'0

4eighted .) si+e of stud) -o-ulation 0'5 0'%


24/34


Reis, !une, 2012

2

@ualitati#e studies$ Suar)

Bne cannot calculate an effect si+e'8


25/34


Reis, !une, 2012

25

@ualitati#e studies, .) ethodolog) t)-e

&et'odologyNumber of

studies Q

Case stud) 120 /

EG-erient or -re&-ost stud) 21 7

3nter#ies indi#idual or grou-8 75 27

!ournal 2 1

Re#ie of official records, docuents, re-orts // 12

Research re#ie = /

Sur#e) 22 =

"B"AL 2=1 100


26/34


Reis, !une, 2012

2

@ualitati#e studies$

Effect on student achie#eent

irection of effectNumber of

studies $ercent of studies$ercent )it'out

t'e inferred

Positi#e 20 = %/

Positi#e inferred 2 10

(iGed 5 2 2

?o change = /

?egati#e / 1 1

"B"AL 2 100 100

2 studies conducted in the -ast centur) in o#er /0 countries


27/34


Reis, !une, 2012

27

@ualitati#e studies$ "esting i-ro#es student achie#eent

and teacher instruction

*c'ievement is improvedNumber of

studies %

es 200 %5

(iGed results 1 1

?o 10 5

"B"AL 211 100

+nstruction is improvedNumber of

studies %

es 15= %

?o 7

"B"AL 15 100


28/34


Reis, !une, 2012

2=


ariation .) rigor and test staes

irection of effect

evel of rigor

Total'ig' medium lo)

Positi#e %5 7 2 20

Positi#e inferred 10 = 2

(iGed / 1 1 5

?o change / 1 =

?egati#e 1 1 1 /"B"AL 11/ =0 51 2

irection of effect

Stakes

Total'ig' medium lo) unkno)n

Positi#e 1// 27 /= 20

Positi#e inferred 12 5 7 2

(iGed 1 5

?o change 2 1 5 =

?egati#e / /

"B"AL 15 // 51 2


29/34


Reis, !une, 2012

2%


Regular standardi+ed tests and -erforance tests

Regular tests(N -./#

$erformance tests(N - /0#

Study results % %

Tenerall) -ositi#e %/ %5

High&staes tests 71 2

High le#el of stud) rigor =

Student attitudes toard test -ositi#e 0 71

"eacher attitudes toard test -ositi#e 55 =0

Student achie#eent i-ro#ed %5 %5

3nstruction i-ro#ed %2 100

Large&scale testing = =


30/34


Reis, !une, 2012

/0

An enorous research literature

* :ut, assertions that it does not

eGist at all are coon

> Soe clais are ade .)

those ho o--ose standardi+ed

testing, and a) .e ishful

thining

> Bthers are MfirstnessN clais


31/34


Reis, !une, 2012

/1

;isissi#e research re#ies

* With a disissi#e researchliterature re#ie, a

researcher assures all that

no other researcher has

studied the sae to-ic


32/34


Reis, !une, 2012

/2

Uirstness clais

* With a firstnessclai, a researcherinsists that he orshe is the first toe#er stud) a to-ic


33/34


Reis, !une, 2012

//

Social costs are

enorous

* Research conducted .) those

ithout -oer or cele.rit) isdisissed && ignored and lost

* Pu.lic -olicies are seed, .asedeGclusi#el) on the research results of

those ith -oer or cele.rit)

* Societ) -a)s again and again forresearch that has alread) .een done


34/34


Reis, !une, 2012

/

"he effect of testing on student achie#eent$

1%10&2010

Richard P' PHELPS

The Effect of Testing on Student Achievement 1910-2010 Slide Show

Documents

Transcript of The Effect of Testing on Student Achievement 1910-2010 Slide Show