Test item writing and analysis
-
Upload
melanio-florino -
Category
Education
-
view
94 -
download
4
Transcript of Test item writing and analysis
September 10, 2011UIC
TEST ITEM WRITING and ITEM ANALYSIS
The Systematic Planning Process:
Identify Instructional Goals
Analyze Learners
Identify Objectives
Plan instructional activities
Choose Instructional Media
Design Assessment Tools
Implement Instruction
Revise Instruction
Steps in Test Construction
Identify Objectives Develop TOS Write Items
Illustrations on Sampling
Population(Universe of
Characteristics)
Sample( Best estimates of the
Population)
Test
Illustrations on Sampling
Universe of Characteristics
A Better Model
Best Model
A Poor Model
Illustrations on SamplingInvalid and Unreliable Test•Too many errors
Valid but Unreliable Test• Errors causes test scores to be unstable and inconsistent
Valid and Reliable Test• Observed errors are not significant (or negligible)
THREE-STAGE MODEL OF CLASSROOM MEASUREMENT
CONTENT VALIDITY
Stage 1 Stage 2 Stage 3
OBJECTIVES ACTIVITIES TEST
Test items must validly measure instructional
objectives
A. Research Designs 1. Differentiate Quantitative and Qualitative reseach.2. Classify the various research designs3. Identify common research designs4. Contruct research problems in each research designs.
total
Topics / Objectives K C A AN S E total
THINKING SKILLS
1,2
3,4,5,67,8,9,1011,12
Item Placement
2 2
2 2 41 2 1 4
2 2
60
Hint:# of items = # of days per topic / total # of days X total # of items
1 6
2 121 61 6
10
DAYS IDEAL
SAMPLE TABLE OF SPECIFICATION
WORKSHOP 1
You need to have:a. Sample syllabusb. Test Questionnairec. Paper and Pen
Instructions:Choose a syllabus and a
questionnaire from your compilation
Match the items in the questionnaire with the competencies measured in the syllabus
Make an evaluation about the content validity of the test – whether it measures what it is suppose to measure.
Volunteers will share their summary of evaluation
Matching Requirements:
Item versus objectiveThinking Skill versus
objectiveProportion of items per
objective versus the total number of items, total number of objectives, and the time element.
TemplateObjectiv
esList of Items
Skills Time Frame
Total
A. Solve the ….
5, 3, 5
ok 3
What do you see in the picture?
How many legs does this elephant have?
Multiple Choice Type of
Test
AdvantagesA large number of ideas can
be addressed in a short period of response time.
These questions are easily and quickly scored.
Questions can elicit responses from all cognitive levels, from knowledge to evaluation.
Questions can be improved over time by analyzing them in light of student performance.
DisadvantagesIt is time-consuming to write good items, especially those at higher cognitive levels.
Test-wise and English fluent students tend to be favored.
varieties of multiple-choice items
A. The correct answer variety
Who invented the sewing machine?
a. Fulton*b. Howec. Singerd. Whitee. Whitney
B. The best answer variety
What was the basic purpose of a test?
a.Measure teacher’s abilityb.determine students’ achievementc. Assess learning outcomesd. Identify students’ weaknesses
The multiple response variety
What factors are principally responsible for the clotting of blood?
a.contact of blood with a foreign substance*b. contact of blood with injured tissuec. oxidation of hemoglobin*d. presence of unchanged pro thrombin
Use of Analogies
Man is to woman as boy is to
a.Fatherb.Motherc.Girld.Lady
The incomplete statement variety
In 2008, millions of pesos worth of corn, rice, and wheat are destroyed in the Philippines by
a. ratsb. grasshoppersc. birds*d. pests
The negative variety
Which of these is NOT true of viruses?
a. Viruses live only in plants and animals.b. Viruses reproduce themselves.*c. Viruses are composed of very large living cells.d. Viruses can cause diseases.
The substitution varietyPassages to be readSurely the forces of education should be fully utilized to acquaint youth with the real nature of the dangers to democracy, (1) for no other place offers (2) as good or better opportunities than the school for; a (3) rational consideration of the problems involved.
Items to be answered1. *a. , for
b. . Forc. —ford. no punctuation needed
2. a. As good or better opportunities thanb. as good opportunities or better thanc. as good opportunities as or better than*d. better opportunities than
3. *a. rationalb. radicalc. reasonabled. realistic
The combined response variety
In what order should these sentences be written in order to make a coherent paragraph?A. A sharp distinction must be drawn between table manners and sporting manners.B. This kind of handling of a spoon at the table, however, is likely to produce nothing more than an angry protest against squirting grapefruit juice about.C. Thus, for example, a fly ball caught by an outfielder in baseball or a completed pass in football is a subject for applause.D. Similarly, the dexterous handling of a spoon in golf to release a ball from a sand trap may win a championship match.E. But a biscuit or a muffin tossed and caught at the table produces scorn and reproach.a. A. B, C. D. E*b. A, C. E, D. Bc. A, E. C. D, Bd. B, E, D. C. A
Basic Guidelines
Adolescents obtain information about marriage
A. in order to reduce the possibility of pregnancyB. from parentsC. from same-sex friends*D. in a haphazard fashion
PRESENT A CLEAR PROBLEM IN THE STEM
Most adolescents obtain information about marriage from
A. same sex friends*B. parentsC. health service personnelD. teachers
Before the Civil War, the South's _____was one of the major reasons manufacturing developed more slowly than it did in the North.
A. emphasis on staple-crop production*B. lack of a suitable supply of raw materialsC. short supply of personnel capable of operating the necessary machinery
PUT ALTERNATIVES AT THE END OF THE QUESTION
Before the Civil War, manufacturing developed more slowly in the South than in the North. One of the major causes of this was the South's___
A. emphasis on staple-crop production*B. lack of a suitable supply of raw materialsC. short supply of personnel capable of operating the necessary machinery
In objective testing, the term objective
A. refers to the method of identifying the learning outcomesB. refers to the method of selecting the test contentC. refers to the method of presenting the problemD. refers to the method of scoring the answers*
PUT MOST OF THE WORDING IN THE STEM
In objective testing, the term objective refers to the method of
A. identifying the learning outcomesB. selecting the test contentC. presenting the problemD. scoring the answers*
For almost a century, the Rhine river has been used by Europeans for a variety of purposes. However, in recent years, the increased river traffic has resulted in increased levels of diesel pollution in the waterway. Which of thefollowing would be the most dramatic result if, because of the pollution, the Council of Ministers of the European Community decided to close the Rhine to all shipping?
A. closure of the busy river Rhine ports of rotterdam, Marseilles and GenoaB. increased prices for Ruhr products*C. reduced competitiveness of the French Aerospace IndustryD. shortage of water for Italian industries
AVOID UNNECESSARY WORDINESS
Which of the following would be the most dramatic result if, because of diesel pollution from ships, the river Rhine were closed to all shipping?
A.closure of the busy river Rhine ports of Rotterdam, Marseilles and GenoaB. increased prices for Ruhr products*C. reduced competitiveness of the French Aerospace IndustryD. shortage of water for Italian industries
Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, which of the following should not happen?
A.Children should not believe all of their behavior is bad.B. Children should understand the reason(s) why they are being punished.C. Children should understand that the teacher, not the children, controls when the punishment will end.*
AVOID NEGATIVELY WORDED STEMS
Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, it is important that the children understand:
A.that it may be a long time before happy times return to the classroomB.the reason(s) why they are being punished*C. that the teacher, not the children, controls when the punishment will end
Good
Thurstone's 7 primary mental abilities include all of the following EXCEPT:
A. word fluencyB. reasoningC. social interaction*D. perceptual speed
Which of the following men contributed most towards he defeat of Hitler‘s Germany in World War II?
A. Winston ChurchillB. Josef StalinC. Franklin D. RooseveltD. George Patton
AVOID REQUIRING PERSONAL OPINION
What is the area of the rectangle described in Question 1?
A. 1050 sq. cm.B. 7396 sq. cm.C. 7654 sq. cm.*D. 8188 sq. cm.
AVOID LINKED OR CLUED ITEMS
What is the official state bird of Pennsylvania?
A.mountain laurelB.B. PhiladelphiaC. ruffed grouse*D. ibex
ALL OPTIONS SHOULD BE HOMOGENEOUS
What is the official state bird of Pennsylvania?
A. goldfinchB. robinC. ruffed grouse*D. wild turkey
Who succeeded Giscard d'Estaing as President of France in l981?
A. Georges PompidouB. Charles De GaulleC. Francois Mitterand*D. Mick Jagger
ALL OPTIONS SHOULD BE PLAUSIBLE
Which of the following is the best definition of a seismograph ?
A. An apparatus for measuring sound waves.B. An apparatus for measuring heat waves.C. An apparatus for measuring earthquake waves.*D. An apparatus for measuring ocean waves.
PUT REPEATED WORDS IN THE STEM, NOT THE
OPTIONSA seismograph is an apparatus for measuring:
A. earthquake waves*B. heat wavesC. ocean wavesD. sound waves
What type of waves does a seismograph measure?
A. earthquake*B. heatC. oceanD. sound
An angle of 90o is called a
A.acute angleB.obtuse anglesC.right angle*
MAKE ALL OPTIONS GRAMMATICALLY
CONSISTENT WITH STEM
Angles of 90o are called
A.acute anglesB.obtuse anglesC.right angles*
The average difference between males and females in the attainment of pubertyis:
A. 1 yearB. 2 years*C. 3 yearsD. no difference
ORDER OPTIONS LOGICALLY
RULES FOR WRITING MULTIPLE-CHOICE ITEMS
Design each item to measure an important learning outcome.
Present a single clearly formulated problem in the stem of the item.
Put the alternatives at the end of the question, not in the middle nor at the beginning.
Put as much of the wording as possible in the stem.
Eliminate unnecessary wordiness
Avoid negatively worded stems. "Which of the following is not..........“
Avoid requiring personal opinion. Other item types are more suitable for this.
Avoid textbook wording. Do not have linked or clued items. All options should be homogeneous.
All options should be plausible. Put repeated words in the stem, not in the options
Punctuation should be consistent. Make all options grammatically consistent with the stem of the item.
List options vertically.
Order options logically. Use the option "all of the above" sparingly.
Use the option "none of the above" sparingly.
Make All distracters the same length
Exercise:Joseph Estrada was an
ActorFormer PresidentDriverScientist
In what year did humans first set foot on the moon?
1975195719691963
Some test items Are too difficultAre objectiveAre poorly constructedHave multiple defensible answers
Ferdinand Magellan came to the Philippines in a
CarBoatAirplaneballoon
WORKSHOP 2
You need to have:a. Sample syllabusb. Test Questionnairec. Paper and Pen
Instructions: Evaluate your items using the guidelines in item writing as your criteria.
Revise poor items.Identify the objective/competence in every revised item.
Build an item bank