Analysis of test formats, poor and good items
Transcript of Analysis of test formats, poor and good items
![Page 1: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/1.jpg)
Analysis of Test formats
Authors:Hector Rojas
Daniel BlancoYuly Caraballo
![Page 2: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/2.jpg)
Matching
STRENGTHS
Objective Easy to design Easy to include a
representative example
WEAKNESSES
Easy to guess Only for grammar,
vocabulary, pronunciation
Not good for testing the skills.
Students need to decide on what answers accepted as correct.
![Page 3: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/3.jpg)
Gap-filling
STRENGTHS Objective Easy to correct Provides clear contexts It can be used to
correct different strategies.
WEAKNESSES Extremely difficult to
design an item with only one possible answer (generally solved by providing a list of terms)
Need to think about possible answers (if a list is not provided)
![Page 4: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/4.jpg)
Cloze
STRENGTHS
Easy for the teacher to create
Easy for the students to understand
Easy to correct and design .
WEAKNESSES
Unexpected correct answers are possible
Mainly for vocabulary It can be irritating for students
![Page 5: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/5.jpg)
Completion
STRENGTHS It is easy to construct Requires simple
recognition of information (eliminatting the guessing factor)
It is good to evaluate listening for specific information
WEAKNESSES Depend on knowledge
of language Difficult to grade
depending on evaluation criteria
It can be irritating for students
![Page 6: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/6.jpg)
Multiple Choice
STRENGTHS Easy to correct and grade Scores are less affected
by guessing Good for checking
understanding Usually it requires less
time for test takers to answer
WEAKNESSES Difficult to construct It does not allowed test
takers to demonstrate knowledge beyond the options provided
Emphasize in recognition and guessing rather than in production
![Page 7: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/7.jpg)
True-False
STRENGTHS Easy to write Can cover a wide
range of difficulty levels
Usually it requires less time for test takers to answer
Usually it is easy to grade and score
WEAKNESSES Guessing element The score can not be
reliable It does not allow test
takers to demonstrate broad range of knowledge
![Page 8: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/8.jpg)
Transformations
STRENGTHS
Good for testing some structures
Easy to write and administer
Specifically tackle one grammatical objective
Allow recognition of a connection between “grammar” and “meaning”
WEAKNESSES
Very artificial It can be more than
one possible transformation in some cases
![Page 9: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/9.jpg)
Questions
STRENGTHS
It is good for checking if students comprehend texts.
It is useful to test the student's ability to analyze a concept.
WEAKNESSES
It is not easy to score It takes more time to
correct and answer.
![Page 10: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/10.jpg)
Recommendations for item design
FORMAT RECOMMENDATIONS
Matching • Give more options than questions.• Each answer should have distracters.• Try meaningful items.• Make sure there’s only one possible selection.• Do not ask to draw lines to match.
Gap filling • No blank at the beginning.• Give enough context.• Avoid clues.• Only one answer • Give line with the same length
Cloze Don’t use nth word, for example : 5th , 3rd
![Page 11: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/11.jpg)
Completion • Avoid the statement becomes ambiguous.• Use blanks of the same length throughout the test so that
the length is not a clue.• If possible, put the blank at the end of a statement rather
than at the beginning. Asking for a response before the student understands the intent of the statement can be confusing and may require more reading time.
Multiple choice
• Be clear and concise in your word and phrase choices.• Make sure that there is only one clearly correct answer
from the options given to the student.• Randomly distribute the correct answer options i.e. A, B, C,
D etc so that there is not a clear pattern that becomes obvious to the student.
True-False • Make sure that the answer is clear and that it could not be either or.• Try not to use negative questions such as “this novel was
not written by....”• Use a random order of true and false responses with your
test questions to avoid creating a pattern.• Use more false questions than true questions as they have
been proven to cater towards higher cognitive level students.
Questions • It has to be easy to understand.• Check how you will judge answers.
![Page 12: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/12.jpg)
Poor & Good items
Multiple choice items
All words should belong to the same grammatical category.
She had to help the ______ old man up the stairs.
A. weak B. slowly C. try D. wisdom
A. weak B. strong C. energetic D. athletic
Adverb Verb Noun
![Page 13: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/13.jpg)
Poor & Good items ii
Grammatical clues make the answer obvius.
She needs to get up earlier so she’s buying an ______ clock.
A. time B. alarm C. watch D. bell
She needs to get up earlier so she’s buying ______ clock.
A. a time B. an alarm C. a watch D. a bell
Deleting the indefinite article would make this item better in terms of objectivity.
![Page 14: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/14.jpg)
Poor & Good items iii
The level of difficulty of the distractors highlights the correct alternative.
They needed lots of training to operate such ______ equipment.
A. easy B. sophisticated C. blue D. wise
“Sophisticated” clearly contrasts with the rest of the options, so students might select it right away.
![Page 15: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/15.jpg)
Poor & Good items iv
Options that collocate with the verb in the sentence. (Differentiators)
She sent the _______ yesterday.
A. letter B. gift C. food D. books
She wrote a _______ yesterday.
A. letter B. gift C. friend D. books
“Post”, “friend” and “courage” are good replacements for “gift”, “food” and “books” that are items that can also be sent or mailed.
![Page 16: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/16.jpg)
Poor & Good items v
Completion
The absence of context will lead to the acceptance of unexpected correct answers.An advanced learner could get to the conclusion of
completing with the letter “y” to form the adjective “nervy”, which means “bold” or “offensive”.
He was a very nerv_____ person.
He was always alert. He was a ________ person.
In this case, given the context, the correct option is “nervous”.
![Page 17: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/17.jpg)
Poor & Good items vi
Words that may take two opposite suffixes lead to ambiguity.
That was a care______ answer.
Enough context will permit the student know which alternative is the most appropiate one.
Yesterday he got on the wrong bus. So today, he was care_______ to find the right one.
![Page 18: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/18.jpg)
Poor & Good items vii
True-FalseColumbus discovered America.
Columbus sailed to the new world in 1942.
The use of trivial information must be avoided . Given the fact that they are more likely to be true than ambiguous information.
If used, they must be presented in a more objective way.
![Page 19: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/19.jpg)
Poor & Good items viii
Since penguins can’t fly, they cannot swim very well either.
Since penguins are unable to fly, they swim badly.
Avoiding the use of negative statements will prevent confusion and contradiction.
![Page 20: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/20.jpg)
Poor & Good items ix
Rattlesnakes live in North America, have fangs, feed on small rodents and crush their prey.
The first three clauses are true, however the last one is not.
The entire sentence must be either true or false, otherwise it will create confusion.
Rattlesnakes crush their preys.
![Page 21: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/21.jpg)
Poor & Good items x
Matching
___1. Sit-and-reach___2. 50-yard dash___3. Pull-up___4. Shuttle run___5. Balke treadmill___6. Dyer backboard volley___7. Disch putting___8. Biopsy
c. Muscle fibersh. Golff.Tennisa.Vo₂maxe.Agilityg.Arm strengthd.Speedb.Flexibility
Items are too heterogeneus, making the answers too obvious.
Contain the same number of items, so the last item could be answered by elimination.
Match column 1 and comlumn 2:
The instructions do not clarify how many times items in column 2 may be used or a basis for matching.
![Page 22: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/22.jpg)
Poor & Good items xi
TransformationChange the sentences to the future
simple:
Doug studied English yesterday.Doug went to the movies with his family.He eats a hamburger.They enjoy the day in the mall.
The activity lacks context
The sentences should be provided in the same tense, to prevent confusion.This exercise requieres more contextualized
instructions, such as:
This is what Doug usually does on the weekend, write about what he is going to do next the weekened.
![Page 23: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/23.jpg)
Item and test analysis.
Item analysis: measure the performance of each item in a test and help us to revise and improve both, items and test as a whole.
Use: -To build reliablity and validity into a test.- More diagnosis information on students.
![Page 24: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/24.jpg)
Features of item analysis
Item difficulty: deals with the number of students who answer correctly. The best items are that are neither too difficult or too hard.
IF = # of students answering the item correctly
Total of students responding that item.
![Page 25: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/25.jpg)
Features of item analysis
Item discrimination : differenciates those test takers with high - abilities from those with low - abilities.
![Page 26: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/26.jpg)
Features of item analysis
Distractor efficiency: deals with the way a distractor lures test takers, especially those with low abilities.
![Page 27: Analysis of test formats, poor and good items](https://reader036.fdocuments.us/reader036/viewer/2022062319/558a3163d8b42a32138b45fe/html5/thumbnails/27.jpg)
References
Brown, Douglas (2004) Language assessment: Principles and Classroom prractices. Longman, London.
Cunningham, George (1998) Assessment in the Classroom: Constructing and interpreting Tests. Routledge, London.
Morrow, James and others (2010)Measurement and Evaluation in Human Performance. Human kinetics, Illinois.
Madsen, Harold (1983) Techniques in testing. Oxford University Press, New York.