Completion, Completion, Short-Answer, and Short-Answer, and True-False ItemsTrue-False Items

Educational Assessment of Educational Assessment of StudentsStudentsChapter 7Chapter 7

Anthony J. NitkoAnthony J. Nitko(Brenda Stewart)(Brenda Stewart)

Three Fundamental Three Fundamental Principles for Crafting Principles for Crafting

AssessmentsAssessments Focus each assessment task entirely on Focus each assessment task entirely on

important learning targetsimportant learning targets Craft each assessment task to elicit from Craft each assessment task to elicit from

students only the knowledge and students only the knowledge and performance that are relevant to the performance that are relevant to the learning targets you are assessinglearning targets you are assessing

Craft each assessment task to neither Craft each assessment task to neither prevent nor inhibit a student’s ability to prevent nor inhibit a student’s ability to demonstrate attainment of the leaning demonstrate attainment of the leaning targets you are assessingtargets you are assessing

Short-Answer ItemsShort-Answer Items

Require a word, short phrase, number, Require a word, short phrase, number, or symbol response.or symbol response.

Three types of short-answer items:Three types of short-answer items: Question – What is the capital of Texas?Question – What is the capital of Texas? Completion – The capital of Texas is Completion – The capital of Texas is

_________._________. Association – On the blank next to the state Association – On the blank next to the state

name, write the abbreviation for that state.name, write the abbreviation for that state. TexasTexas TXTX CaliforniaCalifornia CACA

Usefulness of Short-Usefulness of Short-Answer ItemsAnswer Items

Abilities AssessedAbilities Assessed Lower-order thinking skillsLower-order thinking skills

RecallRecall Comprehension of informationComprehension of information

Higher-level abilitiesHigher-level abilities Make simple interpretations of data and Make simple interpretations of data and

applications of rulesapplications of rules Ability to solve numerical problems in science Ability to solve numerical problems in science

and mathematicsand mathematics Ability to manipulate mathematical symbols Ability to manipulate mathematical symbols

and balance mathematical and chemical and balance mathematical and chemical equationsequations

Usefulness of Short-Usefulness of Short-Answer ItemsAnswer Items

StrengthsStrengths Relatively easy to constructRelatively easy to construct Can be scored objectivelyCan be scored objectively Lowers the probability of getting the Lowers the probability of getting the

answer correct by random guessinganswer correct by random guessing ShortcomingsShortcomings

Not free of subjectivityNot free of subjectivity Cannot anticipate all possible responsesCannot anticipate all possible responses Tends to lower the reliability of the Tends to lower the reliability of the

obtained scoreobtained score

Crafting Short-Answer Crafting Short-Answer ItemsItems

Importance of what is assessedImportance of what is assessed How does it match the test blueprintHow does it match the test blueprint Question format is the preferred format Question format is the preferred format

for a short-answer item. Focus the item on for a short-answer item. Focus the item on the specific knowledge soughtthe specific knowledge sought

Word the items specifically and clearlyWord the items specifically and clearly Put the blank near the end of the sentencePut the blank near the end of the sentence If possible do not copy statements If possible do not copy statements

verbatimverbatim

Crafting Short-Answer Crafting Short-Answer ItemsItems

A completion item should omit A completion item should omit important words and not trivial important words and not trivial wordswords

Limit blanks to one or two Limit blanks to one or two Keep all blanks the same lengthKeep all blanks the same length Specify the precision you expect in Specify the precision you expect in

your answeryour answer Avoid irrelevant cluesAvoid irrelevant clues

True-False ItemsTrue-False Items

Consists of a statement or proposition Consists of a statement or proposition the student may judge as true or falsethe student may judge as true or false

Six varieties:Six varieties: True-FalseTrue-False Yes-NoYes-No Right-WrongRight-Wrong CorrectionCorrection Multiple True-FalseMultiple True-False Yes-No with explanationYes-No with explanation

Usefulness of True-False Usefulness of True-False ItemsItems

AdvantagesAdvantages Certain aspects of the subject matter readily lend Certain aspects of the subject matter readily lend

themselves to verbal propositions that can be themselves to verbal propositions that can be judged true or falsejudged true or false

Relatively easy to writeRelatively easy to write Scored easily and objectivelyScored easily and objectively Can cover a wide range of content within a Can cover a wide range of content within a

relatively short periodrelatively short period CriticismsCriticisms

Poorly constructed assess only specific, frequently Poorly constructed assess only specific, frequently trivial factstrivial facts

Ambiguously wordedAmbiguously worded Answered by random guessingAnswered by random guessing Encourage students to study and accept only Encourage students to study and accept only

oversimplified statements of truth and factual oversimplified statements of truth and factual detailsdetails

Usefulness of True-False Usefulness of True-False ItemsItems

Generalizations in a subject areaGeneralizations in a subject area Comparisons among conceptsComparisons among concepts Casual or conditional propositionsCasual or conditional propositions Relationships between two events, Relationships between two events,

concepts, facts, or principlesconcepts, facts, or principles Explanations for why events or Explanations for why events or

phenomena occurredphenomena occurred Instances or examples of a concept or Instances or examples of a concept or

principleprinciple

Usefulness of True-False Usefulness of True-False ItemsItems

Evidential statementsEvidential statements Predictions about phenomena or Predictions about phenomena or

eventsevents Steps in a procedure of processSteps in a procedure of process Computations (or other kinds of Computations (or other kinds of

results obtained from applying a results obtained from applying a procedure)procedure)

Evaluations of events of phenomenaEvaluations of events of phenomena

Validity of True-False Item Validity of True-False Item FormatFormat

Essence of educational achievement is Essence of educational achievement is the command of useful verbal knowledgethe command of useful verbal knowledge

All verbal knowledge can be expressed in All verbal knowledge can be expressed in propositionspropositions

Proposition is any sentence that can be Proposition is any sentence that can be said to be true or falsesaid to be true or false

Particular area of knowledge is indicated Particular area of knowledge is indicated by success in judging truth or falsity of by success in judging truth or falsity of propositionspropositions

Guessing on True-False Guessing on True-False Items:Items:

An Ill Wind?An Ill Wind? Guessing – getting 50% correctGuessing – getting 50% correct

Blind (complete random guessing) quite Blind (complete random guessing) quite unlike informed guessing (based on partial unlike informed guessing (based on partial knowledge)knowledge)

Well-motivated guess blindly only on a Well-motivated guess blindly only on a small percentage of questions on a testsmall percentage of questions on a test

Difficult to obtain a good score on a test by Difficult to obtain a good score on a test by blind guessing onlyblind guessing only

A given true-false test has a high reliability A given true-false test has a high reliability coefficient – evidence blind guessing does coefficient – evidence blind guessing does not seriously affect the scoresnot seriously affect the scores

Guessing on True-False Guessing on True-False Items:Items:

An Ill Wind?An Ill Wind? Random guessing – sometimes called Random guessing – sometimes called

blind guessingblind guessing Partial knowledge – make an informed Partial knowledge – make an informed

guessguess Higher than 50-50 chance of successHigher than 50-50 chance of success Measures of partial knowledgeMeasures of partial knowledge

Full knowledge – 100% chance of Full knowledge – 100% chance of answering correctlyanswering correctly

Percentages of correct answers go down Percentages of correct answers go down with the number of blind guesses madewith the number of blind guesses made

Suggestions for Getting Suggestions for Getting Started ProperlyStarted Properly

Create pairs of items, one true and Create pairs of items, one true and one falseone false

Rephrase evaluative judgments into a Rephrase evaluative judgments into a comparative statementcomparative statement

Write false statements that reflect the Write false statements that reflect the actual misconceptions held by actual misconceptions held by studentsstudents

Convert a multiple-choice item into Convert a multiple-choice item into two or more true-false itemstwo or more true-false items

Suggestions for ImprovingSuggestions for ImprovingTrue-False ItemsTrue-False Items

Importance of what is assessedImportance of what is assessed How does it match the test blueprintHow does it match the test blueprint Assess important ideas, rather than Assess important ideas, rather than

trivia, general knowledge, or common trivia, general knowledge, or common sensesense

Make sure it is either definitely true or Make sure it is either definitely true or definitely falsedefinitely false Use short statements when possibleUse short statements when possible Use exact languageUse exact language Use positive statements and avoid double Use positive statements and avoid double

negativesnegatives

Suggestions for ImprovingSuggestions for ImprovingTrue-False ItemsTrue-False Items

Avoid copying sentence verbatimAvoid copying sentence verbatim True and false statements should have True and false statements should have

approximately the same number of wordsapproximately the same number of words Don’t present in a repetitive or easily Don’t present in a repetitive or easily

learned patternlearned pattern Do not use verbal clues that give away the Do not use verbal clues that give away the

answeranswer Attribute the opinion in a statement to an Attribute the opinion in a statement to an

appropriate sourceappropriate source Focus on one ideaFocus on one idea

Crafting Multiple True-Crafting Multiple True-False ItemsFalse Items

Looks like a multiple-choice item – Looks like a multiple-choice item – followed by several alternativesfollowed by several alternatives

No single correct or best answerNo single correct or best answer Response to each alternative is true Response to each alternative is true

or falseor false Each alternative is scored correct or Each alternative is scored correct or

incorrectincorrect

Crafting Multiple True-Crafting Multiple True-False ItemsFalse Items

AdvantagesAdvantages Two or three multiple true-false Two or three multiple true-false

responses in the time it takes to make responses in the time it takes to make one multiple-choice responseone multiple-choice response

Multiple true-false tests created from Multiple true-false tests created from multiple-choice items have higher multiple-choice items have higher reliability than original multiple-choicereliability than original multiple-choice

Can assess same abilities as straight Can assess same abilities as straight multiple-choice items that are crafted to multiple-choice items that are crafted to assess parallel contentassess parallel content

Crafting Multiple True-Crafting Multiple True-False ItemsFalse Items

AdvantagesAdvantages Students believe multiple true-false items Students believe multiple true-false items

do a better job of assessing knowledge do a better job of assessing knowledge than multiple-choice itemsthan multiple-choice items

Students perceive them to be harderStudents perceive them to be harder May be easier to write – not limited to one May be easier to write – not limited to one

correct answercorrect answer LimitationsLimitations

Shares some of same limitations as Shares some of same limitations as multiple-choicemultiple-choice

Works CitedWorks Cited

Nitko, A. J., (2004). Educational Nitko, A. J., (2004). Educational Assessment of Assessment of Students, 4Students, 4thth ed. ed. Upper Saddle River, NJ: Upper Saddle River, NJ: Pearson Pearson Education, Inc.Education, Inc.