Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014
-
Upload
liz-norman -
Category
Education
-
view
790 -
download
7
description
Transcript of Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014
![Page 1: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/1.jpg)
Writing MCQs
Liz NormanMassey University
![Page 2: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/2.jpg)
MCQs – why?, why not?
![Page 3: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/3.jpg)
Sampling advantages
• Can test a wide breadth of subjects in a time-efficient manner
• Sampling across the subject may therefore be more representative – validity
• Longer tests (more items) have better reliability
![Page 4: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/4.jpg)
Applied vs fact recall
Fact recall:Questions capable of being answered by reference to one paragraph in a text or notes (or several paragraphs for questions requiring recall of several facts)
Applied (higher order)Questions that require the use of facts or concepts, the solution of a diagnostic or physiologic problem, the perception of a relationship, or other process beyond recalling discrete fact
From: Peitzman et al. (1990). Academic Medicine, 65(9), S59-60.
![Page 5: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/5.jpg)
Positive consequences of MCQs on learning
• All examinations reinforce learning including MCQ examinations – MCQ formats may provide a feasible way to increase
formative assessment– MCQs formats facilitate timely feedback when used
formatively
• MCQs foster a deep approach to learning
![Page 6: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/6.jpg)
Other advantages
• Can construct examinations of known difficulty (assuming psychometric analysis carried out)
• Established methods for setting cutpoints
![Page 7: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/7.jpg)
Efficiency of MCQ
• Efficient and cost effective for large numbers of candidates
• Possibility of automated development
![Page 8: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/8.jpg)
![Page 9: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/9.jpg)
Disadvantages of MCQ
Realising the advantages requires procedures which makes them resource intensive and expensive
– Creation of a large question bank– Pretesting and statistical analysis of Qs– Post examination statistical analysis
![Page 10: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/10.jpg)
Problems that can occur with MCQs
• Fact recall Qs are easier to write therefore tend to dominate
• Beginning writers will usually write fact recall Qs• Some topics are particularly difficult to write
MCQs for
![Page 11: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/11.jpg)
Problems that can occur with MCQs
• If the test is poorly crafted, then the validity of the interpretation made of the resulting score is questionable
• Candidates can’t indicate their interpretation of the Q
• Identification of a correct response requires a different type of thinking from candidates than generation of a response
• Guessing can be rewarded
![Page 12: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/12.jpg)
Other problems
• What is correct is still a subjective decision
![Page 13: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/13.jpg)
Leastcorrect
Mostcorrect
D E AC B
![Page 14: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/14.jpg)
Other problems
• Circulating recall papers may reduce even higher order Qs to recall Qs
![Page 15: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/15.jpg)
![Page 16: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/16.jpg)
Negative consequences of MCQs on learning
• False data present in MCQ distractors can result in learning of incorrect information
![Page 17: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/17.jpg)
Item writing flaws
• 36-65% of questions • 10-15% are serious enough to influence pass-
fail decisions • high-achieving students were more likely than
borderline students to be penalised by flawed items.
• reducing item flaws does improve the performance of MCQs
![Page 18: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/18.jpg)
Avoiding irrelevant difficulty
![Page 19: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/19.jpg)
Focus on one thing per Q
• One thing• Something important – common or catastrophic• Application not recall• Something that would happen in real life• NOT something tricky or overly complex
![Page 20: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/20.jpg)
Keep options short
Iris prolapse is a common sequel to penetrating corneal wounds or ruptured corneal ulcers. Which of the following steps is NOT appropriate for the treatment of iris prolapse?
A. primary closure of the corneal laceration with 8-0 vicryl and treatment with topical antibiotics to control infection.
B. placement of a nictitans flap and treatment with systemic antibiotics to control infection.
C. placement of a corneal graft with an overlying conjunctival pedicle graft and treatment with systemic antibiotics to control infection.
![Page 21: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/21.jpg)
A 7-year-old mare has had intermittent signs of moderately severe colic for the past 48 hours. Heart rate is 56 beats/min. Hydration, acid-base balance, and electrolyte parameters are near normal. On rectal examination, the left dorsal and ventral colon feels distended and is felt coursing in a dorsocranial direction. The spleen is displaced caudomedially. Which of the following is the most likely diagnosis?
A. Cecocolic intussusception or cecal inversion
B. Displacement of the left colon over the nephrosplenic ligament
C. Ileocecal intussusception
D. Infarction of the large colon
E. Volvulus of the large colon and the cecum
![Page 22: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/22.jpg)
Pose a clear task
Chyle is :-
A. The semifluid mass which empties from the stomach into the duodenum.
B. The lymph containing fat droplets found in the lacteals of the small intestine.
C. The contents of the gall bladder.
D. The rounded piece of chewed food which passes down the oesophagus when the animal swallows.
![Page 23: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/23.jpg)
Make all distractors plausible and homogenous
Which of the following statements regarding hepatic encephalopathy is true?
A. Patients typically present with asymmetrical neurological deficits
B. The most effective and appropriate anticonvulsant to use for a patient that is seizuring due to hepatic encephalopathy is phenobarbital
C. Abdominal radiographs of dogs with portosystemic shunts will often show an enlarged liver
D. Cats with portosystemic shunts often exhibit ptyalism as a clinical sign
E. An appropriate treatment for hepatic encephalopathy is intravenous neomycin
![Page 24: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/24.jpg)
Include only ONE best answer
Which of the following is true?
A. Bananas are green
B. Motorbikes are faster than cars
C. Boys are taller than girls
![Page 25: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/25.jpg)
Include only ONE best answer
Leastcorrect
Mostcorrect
D E AC B
![Page 26: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/26.jpg)
Avoid none of the above
Which of the following is true regarding ligament injuries?
A. Ligament injuries are appropriately referred to as “strains” or “sprains”.
B. Surgical intervention is indicated for treatment of second-degree sprains with demonstrable instability.
C. The elastic nature of ligaments allows 30% elongation before permanent deformation.
D. Following surgical repair of ligaments, immobilization via ESF or external coaptation is contraindicated, as range of motion is critical to successful repair.
E. None of the above.
![Page 27: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/27.jpg)
Avoid negative framing
Which of the following statements is false regarding arthrotomies?
A. When detachment of a ligament is necessary, this should be performed by osteotomy of the bony origin rather than transection of the ligament.
B. Complete closure of the synovium is necessary to prevent synovial fluid leakage into subcutaneous tissue.
C. Surgical removal of osteophytes is often followed by their relatively rapid regrowth, and has questionable value.
D. Monofilament absorbable suture material has a lower risk for long term infection than does braided nonabsorbable suture.
E. None of the above.
![Page 28: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/28.jpg)
Avoid double options
2-year-old male neutered Border Collie is presented with the following history and neurologic signs: …….. Which one of the following neuroanatomic localizations and diagnoses is CORRECT?
A. Left C1-C5 myelopathy, intervertebral disk rupture
B. Right C6-T2 myelopathy, fibrocartilagenous embolism
C. Right C1-C5 myelopathy, intervertebral disk rupture
D. Left C1-C5 myelopathy, fibrocartilagenous embolism
![Page 29: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/29.jpg)
How many options is enough?
The probability of selecting the correct answer by chance in a three-option multiple choice question is:
A. 33%
B. 50%
C. 75%
The probability of achieving a score of 70% by chance in a test comprising 30 three-option multiple choice questions is:
D. 0.00356%
E. 3.56%
F. 35.6%
![Page 30: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/30.jpg)
3 options is enough – but sometimes you do need more
A horse suffering from an acute intestinal accident is MOST likely to have
A. primary respiratory acidosis
B. primary respiratory alkalosis
C. primary metabolic alkalosis
D. primary metabolic acidosis
![Page 31: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/31.jpg)
Avoiding irrelevant ease“testwiseness”
![Page 32: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/32.jpg)
Flaws to watch out for include
• Grammatical cues: – one or more distractors don’t follow grammatically from
the stem
• Logical cues: – a subset of the options are collectively exhaustive
• Absolute and vague terms: – terms such as “always”, “never” or “all” used in options.– terms such as “usually” and “frequently” used in
options.
![Page 33: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/33.jpg)
Flaws to watch out for include
• Long correct answer: – correct answer is longer, more specific, or more
complete than other options
• Word repeats: – a word or phrase is included in the stem and in the
correct answer
• Convergence: – the correct answer includes the most elements in
common with the other options
![Page 34: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/34.jpg)
Flaws to watch out for include
• Numeric data not stated consistently• Options are in an illogical order• Stems (lead-ins) are tricky or unnecessarily
complicated
![Page 35: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/35.jpg)
A content free test
• From Phil Race who got this from Roger Lewis who cant remember where it came from…..
![Page 36: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/36.jpg)
Q1
The usual function of grunge prowkers is to remove
A. grunges.
B. snarts.
C. trigs.
D. grods.
![Page 37: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/37.jpg)
Q1
The usual function of grunge prowkers is to remove
A. grunges.
B. snarts.
C. trigs.
D. grods.
Don’t include a word from the stem in one of the options, or it will make people think it is the correct option
![Page 38: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/38.jpg)
Q2
Antigrottification occurs
A. on spring mornings.
B. on summer evenings provided there is no rain before dusk.
C. on autumn afternoons.
D. on winter nights
![Page 39: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/39.jpg)
Q2
Antigrottification occurs
A. on spring mornings.
B. on summer evenings provided there is no rain before dusk.
C. on autumn afternoons.
D. on winter nights
This option looks more probable, because it is longer, and because it is the only one which is qualified
![Page 40: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/40.jpg)
Q3
Lurkies suffer from trangitis because:
A. their prads are always underdeveloped.
B. all their brizes are horizontal.
C. their curnpieces are usually imperfect.
D. none of their dringoes can ever adapt.
![Page 41: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/41.jpg)
Q3
Lurkies suffer from trangitis because:
A. their prads are always underdeveloped.
B. all their brizes are horizontal.
C. their curnpieces are usually imperfect.
D. none of their dringoes can ever adapt.
Don’t mix indefinites with definites – indefinites are more likely to be correct. (Indefinites: usually, sometimes, often, seldom…;
Definites: all, never, none, and so on)
![Page 42: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/42.jpg)
Q4
Non-responsive frattling is usually found in an:
A. gringle.
B. janket.
C. kloppie.
D. uckerpod
![Page 43: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/43.jpg)
Q4
Non-responsive frattling is usually found in an:
A. gringle.
B. janket.
C. kloppie.
D. uckerpod
Don’t allow the indefinite article (‘an’) to give away the right answer
![Page 44: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/44.jpg)
Q5
Which are exceptions to the law of lompicality?
A. The miltrip and the nattercup.
B. The bifid pantrip.
C. The common queeter.
D. The flanged ozzer
![Page 45: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/45.jpg)
Q5
Which are exceptions to the law of lompicality?
A. The miltrip and the nattercup.
B. The bifid pantrip.
C. The common queeter.
D. The flanged ozzer
Don’t ask for something plural, then make just one option plural
![Page 46: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/46.jpg)
Q6
Which must be present for parbling to take place?
A. Phlot and runge.
B. Runge.
C. Stuke and runge.
D. Runge and trake
![Page 47: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/47.jpg)
Q6
Which must be present for parbling to take place?
A. Phlot and runge.
B. Runge.
C. Stuke and runge.
D. Runge and trake
If it’s not this option, it can’t be any of them (one way of thinking about it!!)
![Page 48: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/48.jpg)
Q7
One common disorder of an overspragged uckerpod is:
A. copious vezzling.
B. intermittent weggerment.
C. non-responsive frattling.
D. uneven yerkation
![Page 49: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/49.jpg)
Q7
One common disorder of an overspragged uckerpod is:
A. copious vezzling.
B. intermittent weggerment.
C. non-responsive frattling.
D. uneven yerkation
Be careful about overlapping questions! Candidates should look at both questions, and get maximum thinking
![Page 50: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/50.jpg)
Q7
One common disorder of an overspragged uckerpod is:
A. copious vezzling.
B. intermittent weggerment.
C. non-responsive frattling.
D. uneven yerkation
Be careful about overlapping questions! Candidates should look at both questions, and get maximum thinking
![Page 51: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/51.jpg)
Q8
Which of these is the correct answer?
A.
B.
C.
D.
![Page 52: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/52.jpg)
Q8
Which of these is the correct answer?
A.
B.
C.
D.
There was a pattern. ‘D’ is due to come up again1: A; 2: B; 3: C; 4: D; 5: A; 6: B; 7: C; 8: D
![Page 53: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/53.jpg)
Writing guidelines
![Page 54: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/54.jpg)
Item writing guidelines
• Test application of knowledge not recall• Don’t test “trivial” knowledge• Focus on real life problems• Use novel material - clinical vignettes are good
as long as they are realistic• Use a blueprint
![Page 55: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/55.jpg)
Item writing guidelines
• Minimize the amount of reading in each item• Avoid unnecessary words• Use correct grammar, punctuation,
capitalization, and spelling.• Edit and proof items
![Page 56: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/56.jpg)
Item writing guidelines
• Include the central idea in the stem instead of the choices.
• Ensure that the directions in the stem are very clear.
• Word the stem positively, avoid negatives such as NOT or EXCEPT. If negative words are used, use the word cautiously and always ensure that the word appears capitalized and boldface
![Page 57: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/57.jpg)
Item writing guidelines
• Place choices in logical or numerical order.• Keep choices independent; choices should not be
overlapping.• Keep choices homogeneous in content and
grammatical structure.• Keep the length of choices about equal.• Make sure that only one answer is right or clearly
better than any other• Make all distractors plausible
![Page 58: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/58.jpg)
Sample lead in and option lists for clinical vignettes
• Which of the following is (abnormal)?Options sets could include sites of lesions; list of nerves; list of muscles; list of enzymes; list of hormones; types of cells; list of neurotransmitters; list of toxins, molecules, vessels, spinal segments.
• Which of the following findings is most likely?Options sets could include list of laboratory results; list of additional physical signs; autopsy results; results of microscopic examination of fluids, muscle or joint tissue; DNA analysis results; serum levels.
Constructing Written Test Questions For the Basic and Clinical Sciences, 3rd edn, 2001, National Board of Medical Examiners (NBME), USA http://www.nbme.org/publications/item-writing-manual.html
![Page 59: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/59.jpg)
Sample lead in and option lists for clinical vignettes
• Which of the following is the most likely cause?Options sets could include list of underlying mechanisms of the disease; medications that might cause side effects; drugs or drug classes; toxic agents; hemodynamic mechanisms, viruses, metabolic defects.
• Which of the following should be administered?Options sets could include drugs, vitamins, amino acids, enzymes, hormones.
• Which of the following is defective/deficient/nonfunctioning?Options sets could include list of enzymes, feedback mechanisms, endocrine structures, dietary elements, vitamins.
http://www.nbme.org/publications/item-writing-manual.html
![Page 60: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/60.jpg)
Reviewing MCQs
![Page 61: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/61.jpg)
Reviewing MCQs
• Peer review– Both by subject experts and non-subject experts
• Evaluate Q performance after the exam– Q difficulty – how many got the Q right– Discrimination - How high performing and low
performing students answered each Q– Utility of distracters
![Page 62: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/62.jpg)
Item difficulty and the purpose of testing
• If we are trying to rank students (norm-referenced) we need to create items with a range of difficulty to try and distinguish them, including some items we expect are well above the standard required .
• If we are trying to determine competence or attainment of a learning outcome (criterion-referenced) then items need be no more difficult than required to determine the difference between attainment and non-attainment.
• Remember that items can be difficult for the wrong reasons
![Page 63: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/63.jpg)
“Effective item writers are trained, not born … “
Downing and Haladyna 2006, Handbook of test development ,p. 11
![Page 64: Writing MCQs, Liz Norman, SAVS-CSU Question Writing Workshop, February 2014](https://reader036.fdocuments.us/reader036/viewer/2022062307/554dcd2ab4c905c2488b55e2/html5/thumbnails/64.jpg)
http://www.slideshare.net/liznorman