Tristan K.

IS 3 THEMAGIC NUMBER?

Timothy Masscockman (Dance) (09/02/2013)

CRITERIA

1. Item difficulty2. Item discrimination3. Test-score

reliability4. Test validity

Change number of options / MC item

“develop as many functional distractors

as are feasible”

EARLIER RESEARCH

p. 59, Haladyna & Downing, 1989

METHOD

where:- p1 is the mean item difficulty for items in one

format- p2 is the mean item difficulty of an alternate

format with one fewer option

Tdiff = p2 – p1

p mean item difficulty

Positivedifference

Item difficulty index from Number of options

Easieritem

Meta-Analysis of 80 years of research (1925 – 1999) 27 studies, 56 indep. Trials total n = 12,591

RESULTS

Rodriguez, 2005

Moving from 4- to 3-option items item difficulty by .04 item discrimination by .03 reliability slightly by .02.

“Three options are optimal for MC items in most settings.“

Rodriguez (2005) limited to “achievement- or aptitude type items“

Other domains?

Other populations?

quality of distractors in highly speeded tests?

Distractors providing diagnostic information

PROBLEMS

REFERENCES

Haladyna, T. M., & Downing, S. M. (1989). Validity of a taxonomy of multiple-choice item writing rules. Applied Measurement in Education, 2, 51–78. Rodriguez, M. C. (2005) Three Options Are Optimal for Multiple- Choice Items: A Meta-Analysis of 80 Years of Research

Image:Timothy Masscockman (Dance). Money & Time wallpaper. Retrieved from:http://www.pickywallpapers.com/1600x900/miscellaneous/money/time-and-money-wallpaper/