Writing Test Items

8/9/2019 Writing Test Items

1/23

Writing Test Items

Introduction

Essential Characteristics of Item Writers

o Knowledge and Understanding of the Material Being Tested

o Continuous Awareness of Objectives

o Continuous Awareness of Instructional Model

o Understanding of the Students for Who the Ites are Intended

o S!ill in Written Counication

o S!ill in Techni"ues of Ite Writing

General Tips

o #$%ress Ites as &recisel'( Clearl' and Si%l' as &ossible

o Include all )ualifications *ecessar' to &rovide a +easonable Basis for

+es%onding

o #%hasi,e -eneral Tas!s +ather than Sall .etails

o Avoid /argon and Te$tboo! 0anguage

o 0ocate and .elete Irrelevant Clues

o #liinate Irrelevant Sources of .ifficult'

o &lace all Ites of a -iven T'%e Together in the Test

o &re%are Ke's or Model Answers in Advance of Test Adinistration

o Arrange for Co%etent +eview of the Ites

Writing Specific Types of Items

o Multi%le1Choice Ites

State the &roble in the Ste

Include One Correct or Most .efensible Answer

1
https://www.msu.edu/dept/soweb/writitem.html#introhttps://www.msu.edu/dept/soweb/writitem.html#esscharhttps://www.msu.edu/dept/soweb/writitem.html#knowmathttps://www.msu.edu/dept/soweb/writitem.html#awareobjhttps://www.msu.edu/dept/soweb/writitem.html#awaremodhttps://www.msu.edu/dept/soweb/writitem.html#studentshttps://www.msu.edu/dept/soweb/writitem.html#wrtskillhttps://www.msu.edu/dept/soweb/writitem.html#technicshttps://www.msu.edu/dept/soweb/writitem.html#tipshttps://www.msu.edu/dept/soweb/writitem.html#precisehttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#generalhttps://www.msu.edu/dept/soweb/writitem.html#jargonhttps://www.msu.edu/dept/soweb/writitem.html#clueshttps://www.msu.edu/dept/soweb/writitem.html#irrelevhttps://www.msu.edu/dept/soweb/writitem.html#typehttps://www.msu.edu/dept/soweb/writitem.html#keyshttps://www.msu.edu/dept/soweb/writitem.html#reviewhttps://www.msu.edu/dept/soweb/writitem.html#specifichttps://www.msu.edu/dept/soweb/writitem.html#mcitemhttps://www.msu.edu/dept/soweb/writitem.html#stemhttps://www.msu.edu/dept/soweb/writitem.html#corranshttps://www.msu.edu/dept/soweb/writitem.html#esscharhttps://www.msu.edu/dept/soweb/writitem.html#knowmathttps://www.msu.edu/dept/soweb/writitem.html#awareobjhttps://www.msu.edu/dept/soweb/writitem.html#awaremodhttps://www.msu.edu/dept/soweb/writitem.html#studentshttps://www.msu.edu/dept/soweb/writitem.html#wrtskillhttps://www.msu.edu/dept/soweb/writitem.html#technicshttps://www.msu.edu/dept/soweb/writitem.html#tipshttps://www.msu.edu/dept/soweb/writitem.html#precisehttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#generalhttps://www.msu.edu/dept/soweb/writitem.html#jargonhttps://www.msu.edu/dept/soweb/writitem.html#clueshttps://www.msu.edu/dept/soweb/writitem.html#irrelevhttps://www.msu.edu/dept/soweb/writitem.html#typehttps://www.msu.edu/dept/soweb/writitem.html#keyshttps://www.msu.edu/dept/soweb/writitem.html#reviewhttps://www.msu.edu/dept/soweb/writitem.html#specifichttps://www.msu.edu/dept/soweb/writitem.html#mcitemhttps://www.msu.edu/dept/soweb/writitem.html#stemhttps://www.msu.edu/dept/soweb/writitem.html#corranshttps://www.msu.edu/dept/soweb/writitem.html#intro


2/23

Select .iagnostic 2oils or .istracters

O%tions Should be &resented in a 0ogical( S'steatic Order

O%tions should be -raaticall' &arallel and Consistent with the Ste

O%tions Should be Mutuall' #$clusive

Insure that Correct +es%onses are not Consistentl' Shorter or 0onger than

the 2oils

#liinate -raatical or 3erbal Clues

&resent the &roble in *ovel Ters

Use *egativel' Stated Ites Infre"uentl'

Beware of 4*one of These(4 *one of the Above(4 4All of these(4 and 4Allof the Above4

Alter Ite .ifficult' b' Ma!ing O%tions More Ali!e or 0ess Ali!e in

Meaning

o Alternative1+es%onse Ites

Include Onl' One Idea in #ach Ite

#liinate &artl' True1&artl' 2alse Ites

#liinate S%ecific .eteriners

Insure that True and 2alse Ites are A%%ro$iatel' #"ual in 0ength

Balance the *uber of True Ites and 2alse Ites

#liinate 3ague Ters of .egree or Aount

Use Caution in Writing *egative Ite Stateents

o Matching Ites

Include 5oogeneous Material in #ach #$ercise

Include at 0east Three to 2ive but no More than #ight to Ten Ites in a

Matching Set

#liinate Irrelevant Clues

2
https://www.msu.edu/dept/soweb/writitem.html#diagnosthttps://www.msu.edu/dept/soweb/writitem.html#orderhttps://www.msu.edu/dept/soweb/writitem.html#grammathttps://www.msu.edu/dept/soweb/writitem.html#exclusivhttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#elimcluehttps://www.msu.edu/dept/soweb/writitem.html#novelhttps://www.msu.edu/dept/soweb/writitem.html#negativhttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#tfitemshttps://www.msu.edu/dept/soweb/writitem.html#oneideahttps://www.msu.edu/dept/soweb/writitem.html#parttruhttps://www.msu.edu/dept/soweb/writitem.html#specdethttps://www.msu.edu/dept/soweb/writitem.html#lengthhttps://www.msu.edu/dept/soweb/writitem.html#balancehttps://www.msu.edu/dept/soweb/writitem.html#vaguehttps://www.msu.edu/dept/soweb/writitem.html#negstathttps://www.msu.edu/dept/soweb/writitem.html#matchitmhttps://www.msu.edu/dept/soweb/writitem.html#homogenhttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#irrelevhttps://www.msu.edu/dept/soweb/writitem.html#diagnosthttps://www.msu.edu/dept/soweb/writitem.html#orderhttps://www.msu.edu/dept/soweb/writitem.html#grammathttps://www.msu.edu/dept/soweb/writitem.html#exclusivhttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#elimcluehttps://www.msu.edu/dept/soweb/writitem.html#novelhttps://www.msu.edu/dept/soweb/writitem.html#negativhttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#tfitemshttps://www.msu.edu/dept/soweb/writitem.html#oneideahttps://www.msu.edu/dept/soweb/writitem.html#parttruhttps://www.msu.edu/dept/soweb/writitem.html#specdethttps://www.msu.edu/dept/soweb/writitem.html#lengthhttps://www.msu.edu/dept/soweb/writitem.html#balancehttps://www.msu.edu/dept/soweb/writitem.html#vaguehttps://www.msu.edu/dept/soweb/writitem.html#negstathttps://www.msu.edu/dept/soweb/writitem.html#matchitmhttps://www.msu.edu/dept/soweb/writitem.html#homogenhttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#irrelev


3/23

&lace #ach Set of Matching Ites on a Single &age

+educe the Influence of Clues and thereb' Increase Matching Ite

.ifficult'

Co%ose the +es%onse 0ist of Single Words or 3er' short &hrases

Arrange the +es%onses in S'steatic Order6 Al%habetical( Cronological(

etc7

o The &roof of the Ite Writing is in the Ite Aanal'sis

Introduction

Writing test ites is a atter of %recision( %erha%s ore a!in to co%uter %rograing than towriting %rose7 A test ite ust focus the attention of the e$ainee on the %rinci%le or construct

u%on which the ite is based7 Ideall'( students who answer a test ite incorrectl' will do so

because their aster' of the %rinci%le or construct in focus was inade"uate or inco%lete7 An'

characteristics of a test ite which distract the e$ainee fro the ajor %oint or focus an ite(reduce the effectiveness of that ite7 An' ite answered correctl' or incorrectl' because of

e$traneous factors in the ite( results in isleading feedbac! to both e$ainee and e$ainer7

A %oet or writer( es%eciall' of fiction( relies on rich ental iager' on the %art of the reader to%roduce an i%act7 2or ite writers( however( the tas! is to focus the attention of a grou% of

students( often with widel' var'ing bac!ground e$%eriences( on a single idea7 Such

counication re"uires e$tree care in choice of words and it a' be necessar' to tr' the ites

out before %robles can be identified7

8 To%9

Essential Characteristics of Item Writers

-iven a tas! of %recision counication( there are several attributes or ind sets that arecharacteristics of a %roficient ite writer7

Knowledge and Understanding of the Material Being Tested

At the Universit' level( the de%th and co%le$it' of the aterial on which students are tested

necessitates that onl' facult' ebers full' trained in a %articular disci%line can write concise(unabiguous test ites in that disci%line7 2urther( the nuber of %ersons who can eaningfull'

criti"ue test ites( in ters of the %rinci%les or constructs involved( is liited7 An agreeent b'

colleagues to review each others: tests will li!el' i%rove the "ualit' of ites considerabl' %rior

to the first tr'1out with students7

3
https://www.msu.edu/dept/soweb/writitem.html#singlehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#wordshttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#proofhttps://www.msu.edu/dept/soweb/writitem.html#tophttps://www.msu.edu/dept/soweb/writitem.html#singlehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#wordshttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#proofhttps://www.msu.edu/dept/soweb/writitem.html#top


4/23

Continuous wareness of !"#ecti$es

A test ust reflect the %ur%oses of the instruction it is intended to assess7 This "ualit' of a test(

referred to as content validit'( is assured b' s%ecif'ing the nature and;or nuber of ites %rior toselecting and writing the ites7 Instructors soeties develo% a chart or test blue%rint to hel%

guide the selection of ites7 Such a chart a' consider the odules or bloc!s of content as wellas the nature of the s!ills a test is e$%ected to assess7

In the case of criterion1referenced instruction( content validit' is obtained b' selecting a sa%leof criteria to be assessed7 2or content1oriented instruction( a balance a' be achieved b'

selecting ites in %ro%ortion to the aount of instructional tie allotted to various bloc!s of

aterial7 An e$a%le of a test blue%rint for a fift'1ite test is shown below7

Types of Tests %elia"ility &alidity Correlation Total

Knowledge of terms < = = = >

Comprehension of principles < ? ? ? =>pplication of principles @ ? > =

nalysis of situations = @ @ @

E$aluation of solutions 1 @ @ @

Total =< => =? >D

The blue%rint s%ecifies the nuber of ites to be constructed for each cell of the two1wa' chart72or e$a%le( in the above test blue%rint( four ites are to involve the a%%lication of the

%rinci%les of reliabilit'7

Continuous wareness of Instructional Model

.ifferent instructional odels re"uire ites of "uite different characteristics for ade"uateassessent7 2or e$a%le( a%%ro%riate ite difficult' in a aster'1odel situation ight be a

a$iu value of @D Etwent'1%ercent of the students answering incorrectl'F7 On the other hand(

ites written for a norative odel ight have an a%%ro%riate average difficult' of the order of


5/23

aster' odels( ite difficult' ust not be so low as to %rovide no challenge whatever to an'

e$ainee in a class7

It is generall' easier to adjust the difficult' than to adjust the discriination of an ite7 Ite

discriination de%ends to a degree on the range of e$ainee abilit' as well as on the difficult' ofthe ite7 It can be difficult to write aster'1odel ites which do not discriinate when the

range of abilities aong e$ainees is wide7 0i!ewise( hoogeneous abilities a!e it ore

difficult to write norative1odel ites with acce%tabl' high discriinations7

*o atter what the instructional odel or the range of abilities in a class( the onl' wa' toidentif' a%%ro%riate ites is to select the on the basis of subjective judgent( adinister the(

and anal',e the results7 Then onl' ites of a%%ro%riate difficult' and discriination a' be

retained for future use7

S'ill in Written Communication

An ite writer:s goal is to be clear and concise7 The level of reading difficult' of the ites ust

be a%%ro%riate for the e$ainees7 Wording ust not be ore co%licated than that used in

instruction7

S'ill in Techni(ues of Item Writing

There are an' hel%ful hints and lists of %itfalls to avoid which a' be hel%ful to the itewriter7 This is an area where easureent s%ecialists a' be %articularl' hel%ful7 The reainder

of this hand1out will be devoted to ite1writing ti%s7

8 To%9

GE)E%* TI+S

E,press Items as +recisely- Clearly and Simply as +ossi"le

Unnecessar' aterial reduces the effectiveness of an ite b' forcing e$ainees to res%ond to the

irrelevant aterial and %erha%s be distracted b' it7 2or e$a%le( the following ite6

In carr'ing out scientific research( the t'%e of h'%othesis which indicates the direction in which

the e$%erienter e$%ects the results to occur once the data has been anal',ed is !nown as aEnF 777

could be written

An h'%othesis which indicates the e$%ected result of a stud' is called aEnF 777

Include all .ualifications )ecessary to +ro$ide a %easona"le Basis for %esponding

The ite

What is the ost effective t'%e of test iteG

ight be rewritten

5
https://www.msu.edu/dept/soweb/writitem.html#tophttps://www.msu.edu/dept/soweb/writitem.html#top


6/23

According to #bel( the ost versatile t'%e of objective ite for easuring a variet' of

educational outcoes is the 777

The second version s%ecifies whose o%inion is to be used( narrows the tas! to consideration of

objective ites( and focuses on one ite characteristic7 The first version %oses an alosti%ossible tas!7

Emphasi/e General Tas's %ather than Small 0etails

The ite

The %roduct1oent coefficient of correlation was develo%ed b'

=7 /ohn -osset

@7 Sir +onald 2isher


7/23

>7 has re%resentative nors7

The %hrase 4easures what it %ur%orts to easure4 is considered to be a easureent cliche

which would be "uic!l' recogni,ed b' students in the area7 The ite ight be rewritten6

The validit' of a test a' be deterined b'

=7 easuring the consistenc' of its scores7

@7 co%aring its scores with those of a %arallel for7


8/23

>7 valid7

Eliminate Irrele$ant Sources of 0ifficulty

Other e$traneous sources of difficult' a' %lague e$ainees in addition to the ite faults

entioned above7 Students a' isunderstand the test directions if the test forat is co%le$and;or the students are not failiar with it7 When res%onse !e's are coon to two or ore

ites( care ust be ta!en that students are ade aware of the situation7 If a set of ites using acoon !e' e$tends to a second %age( the !e' should be re%eated on the second %age7 Then

students will not forget the !e' or have to turn bac! to an earlier %age to consult the !e'7

Whenever co%le$ or unfailiar test forats are used( e$ainees should have an o%%ortunit' to

%ractice res%onding to ites %rior to the actual test whose results are used for grading7 Such a

%ractice adinistration will also give the ite writer an indication of difficulties students a' be

having with directions or with the test forat7

+lace all Items of a Gi$en Type Together in the Test

-rou%ing li!e test ites allows e$ainees to res%ond to all ites re"uiring a coon ind1setat one tie7 The' don:t have to continuall' shift bac! and forth fro one t'%e of tas! to another7

2urther( when ites are grou%ed b' t'%e( each ite is contiguous to its a%%ro%riate set ofdirections7

+repare Keys or Model nswers in d$ance of Test dministration

&re%aring a !e' for objective1t'%e ites or a odel answer to essa' or short answer ites is an

e$cellent wa' to chec! the "ualit' of the ites7 If the are ajor flaws in ites( the' are li!el' tobe discovered in the !e'ing %rocess7 &re%aring a odel answer %rior to adinistering the test is

es%eciall' i%ortant for essa' or other o%en1end ites because it allows the e$ainer to develo%

a frae of reference %rior to grading the first e$aination7

rrange for Competent %e$iew of the Items

An'one who has atte%ted to %roof his or her own co%' !nows that it is uch better to have theaterial %roofed b' another %erson7 The sae %rinci%le a%%lies to %roofing test ites7 5owever(

it is i%ortant that the outside reviewer be co%etent in the subject atter area7 Unfortunatel'(

critical review of test ites is a deanding and tie1consuing tas!7 Ite writers a' a!ereci%rocal agreeents with colleagues or a' find advanced students to criti"ue their ites7 Test

construction s%ecialists a' %rovide hel%ful coents with res%ect to general ite

characteristics7

8 To%9

Writing Specific Types of Items

The reainder of this handboo! will deal with s!ills hel%ful in writing s%ecific t'%es of ites7

There is an alost infinite variet' to the fors test ites a' ta!e7 Test ites are often grou%ed

into two ain categories6 objective ites and constructed1res%onse ites7 Objective ites arethose in which the e$ainee recogni,es a best answer fro o%tions %resented in the ite7

8
https://www.msu.edu/dept/soweb/writitem.html#tophttps://www.msu.edu/dept/soweb/writitem.html#top


9/23

Objective ites include ulti%le1choice ites( alternative1res%onse ites and atching ites7

Constructed1 res%onse ites include restricted1res%onse ites( short1answer ites( co%letion

ites and essa' ites7 #ach t'%e of ite will be considered in turn on the following %ages7

Multiple2Choice Items

A ulti%le1choice ite %resents a %roble or "uestion in the ste of the ite and re"uires the

e$ainee to select the best answer or o%tion7 The o%tions consist of a ost1correct answer andone or ore distracters or foils7 Consider the following e$a%le7

The stateent 4Attitude toward su%%ort of %ublic schools is easured b' %erforance at the%olls4 is an e$a%le of

=7 a theor'7

@7 induction7

7 a deduction or 4if then4 stateent7

The ste is the %hrase 4The stateent Attitude toward su%%ort of %ublic schools is easured b'%erforance at the %olls: is an e$a%le of74 The nubered res%onses are the o%tions( with o%tion

nuber four being the correct answer( and o%tions one( two( three and five are foils or

distracters7 *ow let us consider soe hints for constructing this ulti%le1choice t'%e of ite7

State the &roble in the Ste The ite

Multi%le1choice ites

=7 a' have several correct answers7

@7 consists of a ste and soe o%tions7


10/23

?7 ste and a correct answer7

A student who has been given the objective of recogni,ing the co%onents of a ulti%le1choice

ite will read the ste( and iediatel' !now the correct answer7 The onl' reaining tas! is to

locate the o%tion which contains the co%lete list of co%onents7

Include One Correct or Most .efensible Answer The ite below would be a good basis for

discussion but %robabl' should not be included in an e$aination7

The ost serious as%ect of the energ' crisis is the

=7 %ossible lac! of fuel for industr'7

@7 %ossibilit' of wides%read une%lo'ent7

7 cost of develo%ing alternate sources of energ'7

Such an ite ight be rewritten to focus on a ore s%ecific and;or a as%ect of the energ' crisis7

It ight also be written to focus on the o%inion of a recogni,ed e$%ert6

According to &rofessor Koenig( the ost serious as%ect of the energ' crisis is the

=7 %ossible lac! of fuel for industr'7

@7 %ossibilit' of wides%read une%lo'ent7

7 cost of develo%ing alternative sources of energ'7

Select .iagnostic 2oils or .istracters Such as 11

Cliches

Coon Misinforation

0ogical Misinter%retations

&artial Answers

Technical Ters or Te$tboo! /argon

10


11/23

The ajor %ur%ose of a ulti%le1choice ite is to identif' e$ainees who do not have co%lete

coand of the conce%t or %rinci%le involved7 In order to acco%lish this %ur%ose( the foils or

distracters ust a%%ear as reasonable as the correct answer to students who have not asteredthe aterial7 Consider the following ite6

A terinal a' be defined as

=7 a final stage in a co%uter %rogra7

@7 the %lace where a co%uter is !e%t7


12/23

=7 diagnostic test7

@7 criterion1referenced tests7

7 subjective test7

The e$ainee is led directl' to o%tion < b' the last word in the ste which re"uires an o%tionwith its first word beginning with a vowel7 O%tion @ is rendered ore i%lausible b' the

singular1 %lural inconsistenc'7 The ite ight be rewritten as follows6

A test( which can be scored b' a cler! untrained in the content area of the test( is said to be

=7 diagnostic7

@7 criterion1referenced7

7 subjective7

O%tions Should be Mutuall' #$clusive A !nowledgeable e$ainee ust be able to locate onl'

one o%tion which will contain the correct or best answer7 Consider the fault' ite below7

What should be the inde$ of difficult' for an effective aster'1odel test iteG

=7 0ess than =D

@7 0ess than @D


13/23


14/23

?7 find out if the test is reliable7

>7 evaluate the overall difficult' of the test7

The word 4anal',e4 in o%tion @ would be associated with 4ite anal'sis4( leading e$ainees to

the correct o%tion through e$traneous inforation7 The %roble could be solved( in this case( b'changing the word 4anal',e4 in o%tion @ to 4consider47 2urther( an association ight be %rovided

for un!nowledgeable e$ainees b' changing o%tion < to 4anal',e content of the test47

&resent the &roble in *ovel Ters Most often( we are interested in easuring co%le$ ental

%rocesses such as a%%lication of %rinci%les rather than ere eor' of factual !nowledge7 If weatte%t to easure a co%le$ ental tas! such as evaluation( but we use e$a%les with which

the students are failiar( we reduce the tas! of one of sheer eor'7

In %resenting e$aination ites in novel ters( we ust ta!e care not to a!e the ites

e$treel' difficult relative to e$a%les used in instruction7 If we use novel ite t'%es( we usta!e sure that students understand the new %rocess and that the' have had an o%%ortunit' to

%ractice the re"uired s!ill7

Use *egativel' Stated Ites Infre"uentl' There are situations in which the ost eaningful tas!

we can re"uire of an e$ainee is to identif' the e$ce%tion in a set of o%tions7 The ite ste willoften as!6

Which of the following is *OT an e$a%le of HHHHHHHHH G

A ajor %roble with a negativel'1stated ite is that students a' iss the negation when

reading the ste7 A negativel'1stated ite does re"uire an e$aine to switch his or her ind setfro that of loo!ing for the best answer to that of locating the ost definite non1answer7 Ites

with negativel' stated stes can often be rewritten as effective %ositivel'1stated ites7

2or e$a%le( the negativel'1stated ite

Which of the following is *OT a ethod of deterining test reliabilit'G

=7 Coefficient of e"uivalence

@7 Coefficient of stabilit'

7 Test1criterion intercorrelation

a' be re%hrased as a %ositivel'1stated ite7

Which of the following is a ethod of deterining the validit' of a testG

=7 Coefficient of e"uivalence

14


15/23

@7 Coefficient of stabilit'

7 Test1criterion correlation

The correct answer to each of the two above ites is o%tion >7

Beware of 4*one of These(4 *one of the Above(4 4All of these(4 and 4All of the Above74 Theo%tions 4*one of these4 and 4*one of the above4 should not be used when the e$ainee is to

select the best( but not necessaril' absolutel' correct answer7 The' should onl' be used when one

of the o%tions would be agreed u%on b' e$%erts as absolutel' correct7 The ite below illustratesthe ina%%ro%riate use of the 4*one of the above4 o%tion7

What is an ideal level of difficult' for an objective test iteG

=7 =D

@7 @D

7 *one of the above

The correct answer would de%end on the test odel used as a frae of reference7 The ite ight

be i%roved b' rewriting it as follows7

What level of ite difficult' would allow for a$iu discriinationG

=7 =D

@7 @D

D

?7 D

>7 *one of the above

Ma$iu discriination could occur onl' at the level of difficult' of >D7 Therefore( 4*one of

the above4 is a ore a%%ro%riate o%tion than in the earlier version of the ite7

4All of these4 and 4All of the above4 tend to be less useful o%tions than the 4*one of these4 t'%e

of o%tion7 When 4All of these4 or 4All of the above4 are used with a five1o%tion ulti%le choiceite( an e$ainee has onl' to recogni,e an' two of the four o%tions as correct to be led to the

15


16/23

correct answer7 Conversel'( the e$ainee needs onl' to recogni,e one of the o%tions as incorrect

in order to reject the 4All of the above4 o%tion7

If *one of the above4 and 4All of the above4 are to be used as o%tions( the' ust be used

occasionall' as the correct o%tion7 If the' are seldo or never !e'ed as correct( the e$aineeswill soon recogni,e the as 4fillers4 and ignore the7

Alter Ite .ifficult' b' Ma!ing O%tions More Ali!e or 0ess Ali!e in Meaning Ite o%tionswhich are ore ali!e in eaning %rovide a ore difficult choice than do those which are ore

obviousl' different7 Consider the three following ites7

E#asiestF The "ualit' of a test which indicates how consistentl' the test easures is called

=7 objectivit'7

@7 reliabilit'7


17/23

An alternative1res%onse ite is a s%ecial case of theulti%le1choice ite forat7 There are an'

situations which call for either1or decisions( such as deciding whether a s%ecific solution is right

or wrong( whether to continue or to sto%( whether to use a singular or %lural construction( and soon7 2or such situations( the alternative res%onse ite is an ideal easuring device7 Since onl'

two o%tions are %ossible( alternative1res%onse ites are generall' shorter( and( therefore( re"uire

less reading tie7 Students a' res%ond to ore alternative1res%onse ites than other t'%es of

ites in a given length of tie7

A ajor disadvantage of alternative1res%onse ites is the fact that students have fift'1fift'

%robabilit' of answering the ite correctl' b' chance alone7 When carefull' written and

%retested( alternative1res%onse ites a' be written which e$ceed ulti%le1 choice ites in

abilit' to discriinate7 -enerall'( however( it is considered necessar' to have a larger nuber ofalternative1choice ites than of other t'%es of ites in order to achieve a given level of test

reliabilit'7

There are two ain t'%es of alternative1res%onse ites7 One t'%e is essentiall' a two1o%tionulti%le1choice ite7 An e$a%le follows7

To deterine the degree of relationshi% between two continuous variables( one ust co%ute the

=7 %roduct1oent coefficient of correlation7

@7 ran!1order coefficient of correlation7

A second t'%e of alternative1res%onse ite is the co%lete stateent forat( the ost failiar

e$a%le of which is the true1false ite7

To deterine the degree of relationshi% between two continuous variables( one ust co%ute the

ran!1order coefficient of correlation7

=7 True

@7 2alse

The correct answer to the two1choice version is o%tion @( and the correct answer to the co%lete1stateent version is o%tion @( 2alse7

A ajor distinction between the co%lete1stateent( true1false t'%e of ite and ites in

ulti%le1choice or two1choice forats( is that the co%lete1stateent ite contains no criterion

for answering the ite7 The criterion is outside of the ite( rooted in the characteristics and

e$%eriences of each individual e$ainee7 #ach e$ainee ust as! the "uestion( true or falsewith res%ect to whatG It follows that each co%lete1stateent( true1false ite ust be

une"uivocall' true or une"uivocall' false7 It sees intuitive that %ro%er wording and the

eliination of e$traneous clues are ore crucial with the true1false t'%e of ite than with an'other ite forat7 2ollowing are soe %oints to consider in writing ore effective alternative1

res%onse ites7

Include Onl' One Idea in #ach Ite Alternative1res%onse ites testing two ideas siultaneousl'%rovide little useful feedbac! to the e$ainee or to the e$ainer7 If such 4double1barreled4 ites

17


18/23

are answered incorrectl'( it is not certain whether one of the ideas or both of the are res%onsible

for the confusion7 A two1idea ite is given below7

Test validit' is a function of test reliabilit'( which can be i%roved b' using fewer ites7

=7 True

@7 2alse

The answer to the ite is false since ore ites would be needed to i%rove test reliabilit'7 Theite should be s%lit into two true1false ites7

#liinate &artl' True1&artl' 2alse Ites 0ac! of absolute truth or falsit' of an ite generall'

results fro an ite writer:s failure to consider all %ossible fraes of reference fro which anite a' be answered7 Consider this e$a%le7

To be valid( a test ust be content1balanced7

=7 True

@7 2alse

When considered fro the %oint of view of an e$ainer of student achieveent( the ite is true7

5owever( an e$ainer interested in student a%titude would li!el' consider the ite false( sincehe or she would be %riaril' interested in accurate %rediction of a criterion( not on the content of

the test7 The solution for the sa%le ite above is si%le7

To be valid( an achieveent test ust be content balanced7

=7 True

@7 2alse

The ite a' now be !e'ed true7

#liinate S%ecific .eteriners In the real world( it is difficult to find things about which we can

a!e absolute stateents( such as 4It is never true4 or 4It is alwa's true74 3er' often( we need to

"ualif' our stateents( with words li!e often( as a rule( soe ties( a'( and so on7 Students usethe following generali,ations when answering true1false ites7 If an ite contains an absolute

s%ecific deteriner11such as alwa's or never11ar! the ite false7 If an ite contains a

"ualif'ing s%ecific deteriner11such as soeties or %robabl'11ar! it true7

Writers of alternative1res%onse ites ust be e$treel' cautious in words which a' serve ass%ecific deteriners7 In fact( ite writers should atte%t to use s%ecific deteriners in a wa'

which will cause test1wise but un!nowledgeable students to answer the ite incorrectl'7 An

e$a%le of an ina%%ro%riate use of a s%ecific deteriner is given below7

All valid tests are reliable tests7

18


19/23

=7 True

@7 2alse

On the basis of the s%ecific deteriner( 4all(4 an e$ainee would answer the ite false( the

!e'ed answer7 A ore a%%ro%riate version of the ite is given below7

All valid a%titude tests are reliable tests7

=7 True

@7 2alse

In this case the s%ecific deteriner 4all4 would lead a test1wise e$ainee to select the 4false4

answer( but the !e'ed answer is 4true74

Insure that True and 2alse Ites are A%%ro$iatel' #"ual in 0ength Since we often have to

"ualif' stateents in order to a!e the une"uivocall' true( test1wise students often use ite

length as an e$traneous clue7 #fforts should be ade to write false ites of about the saelength as true ites7

e7 Balance the *uber of True Ites and 2alse Ites The nuber of true and false ites shouldbe a%%ro$iatel' balanced so that test1wise students will a%%roach an ite on the basis of its

content rather than on the %robabilit' of its being true or false7 Soe authors argue that the

%ro%ortion of false ites should e$ceed the nuber of true ites7 The' argue that( in general(

res%ondents tend to agree with stateents rather than disagree with the7 Thus the false iteswill tend to be soewhat ore difficult and the total test will be soewhat ore reliable7

#liinate 3ague Ters of .egree or Aount Words li!e 4fre"uentl'4 and 4seldo4 are

es%eciall' o%en to inter%retation in true1false ites( which have no built1in frae of reference7 Itis generall' %ossible to edit such vague ters out of true1false ites7 5ere is an e$a%le7

+eliabilit' is fre"uentl' deterined b' the s%lit1halves ethod7

=7 True

@7 2alse

This difficult1to1answer ite a' be rewritten as follows7

The s%lit1halves ethod is used to deterine test reliabilit'7

=7 True

@7 2alse

The revised ite a' be !e'ed =7 True7

19


20/23

Use Caution in Writing *egative Ite Stateents The use of negative alternative1res%onse ite

stateents involves the sae %erils as does the use of negatives in ulti%le1choice ite stes7

#s%eciall' to be avoided is the double1negative( which soeties %o%s u% in alternative1res%onse ite stateents7 An e$a%le is given below7

There is no advantage in not using s%ecific deteriners in alternative1res%onse ites7

=7 True

@7 2alse

The ite should be rewritten7

S%ecific deteriners should be balanced between true and false ites7

=7 True

@7 2alse

The ite is now !e'ed =7 True7

Matching Items

A atching ite consists of two coluns6 one colun of stes or %robles to be answered( and

another colun of res%onses fro which the answers are to be chosen7 Traditionall'( the colun

of stes is %laced on the left and the colun of res%onses is %laced on the right7 An e$a%le isgiven below7

.irections6 Match the data gathering %rocedures in the ite colun on the left with the nae of

the data gathered in the res%onse colun on the right7 &lace 'our answer in the blan! to the leftof each %rocedure7 #ach answer a' be used onl' once7

0ata Gathering +rocedure Type of 0ata

EaF =7 Adinister two fors of a test a7 Coefficient of e"uivalence

EdF @7 #stiate reliabilit' on the basis of ite data b7 Coefficient of stabilit'EcF


21/23

*ote that( in the above e$a%le( it is necessar' to answer the ulti%le1choice ite in order to

answer the %arent atching ite7 *ote also that the res%onses Eite co%onentsF in the list at the

right have a EsF added to each res%onse in order to eliinate singular1%lural e$traneous clues7

Because of the nature of the atching tas!( naes with events( for e$a%le( it is clear thatatching ites often easure recognition of factual !nowledge rather than higher level ental

%rocesses7 5ere are soe hints for writing atching ites7

Include 5oogeneous Material in #ach #$ercise Of all %ossible ite t'%es( atching ites are

the ost li!el' to contain e$traneous clues to the correct answers( es%eciall' when heterogeneousaterial is included in one atching ite7 Consider the following e$a%le7

EaF =7 Measures factual !nowledge a7 Matching ite

EbF @7 One ste and several o%tions b7 Multi%le choice ite

EdF


22/23

These rules are illustrated b' the ite revision shown below7

.irections6 Match the data gathering %rocedures in the ite colun on the left with the nae of

the data gathered in the res%onse colun on the right7 &lace 'our answer in the blan! to he left of

each %rocedure7 #ach res%onse a' be used ore than once7

0ata Gathering +rocedure Type of 0ata

EaF =7 Adinister two fors of a test a7 Coefficient of e"uivalence

EdF @7 #stiate reliabilit' on the basis of ite data b7 Coefficient of stabilit'EcF


23/23

&rinted handouts are available fro the Scoring Office( ==? Co%uter Center( which describe the

ite anal'sis %rint1out and how it a' be used to guide and i%rove instruction( and to evaluate

and i%rove ites7

23

Writing Test Items

Documents

Transcript of Writing Test Items