1. Writing Good Evaluation Questions

2. Types of QuestionsSelected Response Constructed ResponseTrue/False Fill-in-the-blankMatching EssayMultipleChoice/Response

3. e ss o f this LPart figuring out assessmentPart figuring out answer More of th is

4. (True or False)The physical relationshipbetween most petroleumproducts and purely (True or False)aqueous solutions is As a general rule, oil andgenerally such that water do not mix.physical interaction anddiffusion of the two isseverely limited. Which is easier to process for the student? Why?

5. (True or False) The physical relationship between most petroleum products and purely (True or False) aqueous solutions is As a general rule, oil and generally such that water do not mix. physical interaction and diffusion of the two is severely limited.(A) keeps the language simple. Unless the purpose of the test is to survey the extent ofthe students vocabulary, there is no point in using words that are unfamiliar to studentsor in phrasing questions so that they are difficult to understand. Students will bejustifiably angry and frustrated if they get answers wrong because they could notunderstand what was being asked rather than because they did not know the rightanswer.

6. (True or False) (True or False)The area of a rectangle is A 2 x 4 rectangle has anfound by multiplying the area of 8ft2.length by the width. Which asks the student to demonstrate at a higher cognitive level? Why?

7. (True or False) (True or False) The area of a rectangle is A 2 x 4 rectangle has an found by multiplying the area of 8ft2. length by the width.(B) Ask students to apply, rather than simply recall, information.If students can apply the information they learned, it is a safe bet they have committed itto memory. It does not follow, however, that simply because students have memorizedinformation they can also apply it. This being the case, it is better to aim questions atapplication rather then simple recall.

8. 5) The number of square feet in a room nine feet long and twelve feet wide is: A) 3. B) 21. C) 81. D) 108. E) 144.6) At $2.00 per square foot, what would it cost to carpet the room described in question #5? A) $6 B) $42 C) $162 D) $216 E) $288 Here are two successive questions on an assessment. What is our opportunity for improvement?

9. 5) The number of square feet in a room nine feet long and twelve feet wide is: A) 3. B) 21. C) 81. D) 108. E) 144. 6) At $2.00 per square foot, what would it cost to carpet the room described in question #5? A) $6 B) $42 C) $162 D) $216 E) $288Make sure that each item is independent.Check questions to be sure that one question does not provide a clue to some otherquestion or that the answer to one question is not crucial to another.Any student missing #5 probably missed #6. Nothing gained by linking questionscombine.

10. A B C D1)2)3)4)5)6)7)8)What is the answer to #8?

11. A B C D 1) 2) 3) 4) 5) 6) 7) 8)(D) Do not establish or follow a pattern for correct responses. Regardless ofhow clever an answer pattern is, some student will eventually discover it andcompromise the test results.(You know someone has done it, or it wouldnt be a rule!)

12. The acronym USDA stands for: A) United States Dairy Association. B) United States Department of Agriculture. C) United States Department of Administration. D) United States Dietitians Association. What is our opportunity for improvement?

13. The acronym USDA stands for: A) United States Dairy Association. B) United States Department of Agriculture. C) United States Department of Administration. D) United States Dietitians Association.(B) Do not include trick or trivial questions. Sometimes evaluators are tempted to askquestions that require extended effort for correct interpretation that deal with unimportantpoints. Trick or trivial questions not only reduce the validity and reliability of tests, but theymay have a powerful negative effect if they antagonize students.This should be something important and relevant something the participant should knowrather than looking up in a reference document.

14. How should you respond to a question from a studentonce the assessment has begun?

15. Do not answer questions after the test has started unless it is done publically.Sometimes individual students will seek further clarification of a question as they work throughthe test. If additional information is provided to that student, the test results may be biased,since the student will have had access to direct help when others did not. It is wise to make ageneral announcement prior to each test concerning your reluctance to answer questionsduring a test. This, plus careful proofreading of test prior to their administration, shouldforestall most questions. IF it does become necessary to answer a question about some testitem, assume that other students may be equally confused about it and call everyonesattention to the clarification. Remember however, that interrupting students during a testdestroys their train of thought and thus should be avoided whenever possible.

16. 12) John Adams was: A) the second president of the United States. B) the third president of the United States. C) the fourth president of the United States. D) none of the above.13) The second president of the United States was: A) John Adams. B) Thomas Jefferson. C) James Madison. D) none of the above. How are these two questions different?

17. 12) John Adams was: A) the second president of the United States. B) the third president of the United States. C) the fourth president of the United States. D) none of the above. 13) The second president of the United States was: Stem A) John Adams. Answer B) Thomas Jefferson. C) James Madison. Distractors D) none of the above.If the stem does its job properly, it gives the student an idea of what is sought before readingthe options. In #12, the student doesnt know what is sought until theyve read each option. Furthermore, more time is consumed because the student must read the repetitive wording inefficient testing.In Question #13, the stem is clear and concise and gives the student sufficient time to beginthinking about the answer.

18. 13) The second president of the United States was: A) John Adams. B) Thomas Jefferson. C) James Madison. D) none of the above. What could be done to improve this question?

19. 13) The second president of the United States was: A) John Adams. B) Thomas Jefferson. C) James Madison. D) none of the above.Make distractors reasonable and plausible.Distractors should be equally appealing to someone who is purely guessing. Someonevaguely familiar with the early presidents would recognize the names and eliminateoption D, thereby increasing their odds of guessing a right answer from 1 in 4 to 1 in 3.

20. The area of a circle with a radius of 3 feet is:3 A) 9.42 ft2. B) 18.85 ft2. C) 28.27 ft2. D) 29.6 ft2. Why are these distractors plausible?

21. The area of a circle with a radius of 3 feet is: = *3 3 A) 9.42 ft . 2 = 2*3 B) 18.85 ft2. = *32 C) 28.27 ft2. = 2*3 D) 29.6 ft2.One way to construct good distractors is based on the type of errors most students arelikely to make. In this case, each distractor is an incorrect calculation based on differenterrors.

22. 15) To make an effective presentation, the presenter should: A) identify the audiences needs in advance. B) meet everyone in person after the presentation. C) summarizes the main points in a title slide. D) looks at the slide to focus attention on it. What is our opportunity for improvement?

23. 15) To make an effective presentation, the presenter should: A) identify the audiences needs in advance. B) meet everyone in person after the presentation. C) summarizes the main points in a title slide. D) looks at the slide to focus attention on it.Make sure that unintentional clues are not provided.Sometimes a student can eliminate an option simply because it is grammatically incorrect.For example a stem ending with An vs aIncorrect tenses or forms of words can provide clues

24. 16) Lake effect snow storms are most likely to occur when: A) the temperature drops. B) snow is created when a cold wind blows over the warmer water in the lake causing the evaporating water to crystalize. C) it gets cold. D) it gets warm. E) it has recently rained. What is our opportunity for improvement?

25. 16) Lake effect snow storms are most likely to occur when: A) the temperature drops. B) snow is created when a cold wind blows over the warmer water in the lake causing the evaporating water to crystalize. C) it gets cold. D) it gets warm. E) it has recently rained.Another example of unintentional clues. Distractors that are consistentlylonger or shorter than the other options.The use of always or never is usually a big giveaway, as is the user of allof the above or none of the above

26. 18) Who was the second president of the United States? A) John Adams B) Warren Harding C) Thomas Jefferson D) James Madison19) The second president of the United States was: A) John Adams. B) Warren Harding. C) Thomas Jefferson. D) James Madison. Which punctuation is correct?

27. 18) Who was the second president of the United States? A) John Adams B) Warren Harding C) Thomas Jefferson D) James Madison 19) The second president of the United States was: A) John Adams. B) Warren Harding. C) Thomas Jefferson. D) James Madison.If the stem is a complete question, it should end with a question mark. The answer andthe distractors would be capitalized, but not have periods unless they were completesentences.If the stem is part of a statement that will be completed by one of the choices, it shouldend with a colon. Colons precede lists and the choices constitute a list. If the choicescomplete the statement, each should begin with a lower case letter and end with aperiod because the stem plus the choice will make a complete statement.

28. 19) Who was the second president of the United States? A) John Adams B) Warren Harding C) Thomas Jefferson D) James Madison What is our opportunity for improvement?

29. 19) Who was the second president of the United States? A) John Adams B) Warren Harding C) Thomas Jefferson D) James MadisonFormat Skip a line between the stem and the choices to make it easier for students tovisually separate the two components. Also keep both the stem and the choices on thesame page, if there is not enough room, move the whole question to a new page.

30. (True or False)If every teacher were given his or her own computer,most of the difficult problems currently limiting wideruse of computer-assisted instruction would be solved. What is the opportunity for improvement?

31. (True or False) If every teacher were given his or her own computer, most of the difficult problems currently limiting wider use of computer-assisted instruction would be solved.Most of the difficult problems? How many is that? 99%? 52%?True False items are most appropriate for lower-level cognitive skills. (Keep in mind it is a50/50 guess) Although True-False items are often misused, it is possible to construct true-false items that cover the analysis level of Blooms Taxonomy.Other points to keep in mind when writing True/False items:Be sure every item is wholly true or wholly false.Whenever possible, avoid such terms as generally or usually. (Although not as much as agiveaway as always or never, they are still open to varying interpretations.)Be sure that items are not dependent upon insignificant facts. Make sure each item askssomething of importance and worth remembering.Be sure that correct items are not consistently longer or shorter than incorrect items.Avoid the use of double negatives, but if you use any negative at all, emphasize it.

32. Connect the state with its state bird. ___Connecticut A) Robin ___Nevada B) Mountain Bluebird ___Idaho C) Bluebird ___Ohio D) Cardinal ___Oregon E) Western Meadowlark ___Mississippi F) Mockingbird ___Texas G) Chickadee ___Wisconsin H) Lark Bunting ___Minnesota I) Common Loon ___North Dakota J) Brown Thrasher ___South Carolina K) Great Carolina Wren ___Iowa L) Eastern Goldfinch What is our opportunity for improvement?

33. Connect the state with its state bird. ___Connecticut A) Robin ___Nevada B) Mountain Bluebird ___Idaho C) Bluebird ___Ohio D) Cardinal ___Oregon E) Western Meadowlark ___Mississippi F) Mockingbird ___Texas G) Chickadee ___Wisconsin H) Lark Bunting ___Minnesota I) Common Loon ___North Dakota J) Brown Thrasher ___South Carolina K) Great Carolina Wren ___Iowa L) Eastern GoldfinchMatching Items are used most easily to measure low-level cognitive skills such as recall orcomprehension. Link People to dates, capitals to states, labels for charts, etc.In this question, neither list is in alphabetical order so students must continually scan the listif they know the answer. If the list is 1-1 match, student can guess right answers byeliminating some options. So

34. Connect the state with its state bird. ___Connecticut A) Bluebird ___Idaho B) Brown Thrasher ___Iowa C) Cardinal ___Minnesota D) Chickadee ___Mississippi E) Common Loon ___Nevada F) Eastern Goldfinch ___North Dakota G) Great Carolina Wren ___Ohio H) Lark Bunting ___Oregon I) Mockingbird ___South Carolina J) ) Mountain Bluebird ___Texas K) Robin ___Wisconsin L) Western Meadowlark1. Sort both lists chronologically2. Add 1-many options in this case, some birds are used multiple times for different states, and some not at all. Alternatively, have more choices than matches. This prevents guessing by elimination. (Note this in the instructions too!)3. Ensure all items are around one topic (seems obvious, but again)4. Shorten list usually no more than 10

35. Fill-in-the-Blank vs. EssayHow is grading different for these two items?

36. Fill-in-the-Blank vs. EssayFor completion items, write items that can be completed with a single word or short phrase. When students are required to fill in more than a few words, the grading of the item is nolonger black and white, and it ceases to be a completion item.

37. The first World War began in ____________. What is our opportunity for improvement?

38. The first World War began in ____________.State? Country? Continent? Year? Decade?Be sure that only one word or phrase can correctly complete the sentence.

39. 21) ______________ was the second president of the United States.22) The second president of the United States was ______________. Which completion item is easier for the student?

40. 21) ______________ was the second president of the United States.22) The second president of the United States was ______________. Put blanks near the end of the sentence so the student is guided toward the correct response.

41. The letter from the 1776 General Congress to KingGeorge that severed all political connections betweenAmerica and Britain is known as the_________________ ___ ___________________. What is our opportunity for improvement?

42. The letter from the 1776 General Congress to King George that severed all political connections between America and Britain is known as the _____________ ______________ _______________.Make all blanks the same length. Making the length of the lines correspond tothe words, gives away hints for students who are guessing.

43. __________ __________ wrote _________ ___________. What is our opportunity for improvement?

44. __________ __________ wrote _________ ___________.No more than 2 blanks in an item. The more blanks in an item, the greater thechance the student will be unable to determine the question.

45. Benefits and Risks of Essays Advantages Disadvantages1. In-depth sampling of students 1. Teacher fatigue, knowledge of a specific topic. subconscious biases, and2. Students must interpret, other extraneous variables evaluate, and organize data; can affect grade. draw conclusions; make 2. Essay tests inherently biased inferences; and express in favor of students who can thoughts coherently. write quickly, neatly, and3. Emphasizes higher-level effectively. cognitive skills such as 3. Reduced reliability and synthesis and evaluation. validity because only a few4. Easier to construct than questions are asked. objective tests. 4. Longer time to grade. 5. Increased possibility of bluffing.

46. M. A. Lorber and W. D. Pierce (1990). Objectives, Methods, andEvaluation for Secondary Teaching (3rd Edition). Englewood Cliffs, NJ:Prentice Hall.