Reverse+multiple choice+method--presentation 6-feb12
-
Upload
advancing-a-massachusetts-culture-of-assessment -
Category
Business
-
view
369 -
download
0
Transcript of Reverse+multiple choice+method--presentation 6-feb12
![Page 1: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/1.jpg)
Reverse Multiple-Choice Method (RMCM)
Indu M. AnandMathematics DepartmentUniversity of Massachusetts – Lowell
AMCOA Conference, February 9, 2012
![Page 2: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/2.jpg)
RMCM -- The MethodReverse Multiple-Choice Method:
Presents content for Teaching or Testing
Introduced in Patent No. US 7,033,182
Uses multiple-choice question format
Formalizes reinforcement &
explanationHarnesses power of “wrong”
answers
![Page 3: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/3.jpg)
RMCM -- The Method (contd)Requires examinee to:
(i) examine each answer choice in the context of the query;
(ii) identify key information that makes the answer correct or incorrect; then,
(iii) change the given facts to make an incorrect answer choice the "correct" answer to changed query.
![Page 4: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/4.jpg)
RMCM Method (contd)…Correct answer, no modification
For a “wrong” answer choice Examinee:--
Must examine answer choiceMust understand why answer is
incorrect Must communicate why it is incorrect
More efficient than Open format / essay May be automated for gradingMay be used for most subject areas
![Page 5: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/5.jpg)
How it works – Examinee’s view…Examinee, typically, is asked: To select correct answerThen --select one (or more) wrong answerchange the query so that
selected wrong answer is the correct answer for changed query
Modification criteria may be provided
![Page 6: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/6.jpg)
How it works – RMCM QuestionQuestion is key data structure of
system. Components of the Question object are:
Query – “Narrative” + Call of the question
Answer choices – Multiple , One correct others incorrect
RMCM Questions:May be written from the scratch, orMay be developed from other multiple-
choice questionsStored in persistent databanks of
questions
![Page 7: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/7.jpg)
RMCM Question (contd)
Special features of RMCM question:
Fact Objects (FO’s) = Facts / segments of the query on which the answer turns;
FO = True or False for an answer choice, depending on its relevance to answer choice
Fact Object Values (FV’s) – Values assigned to relevant FO’s for original or modified query.
![Page 8: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/8.jpg)
Structure…
Test Question
Query
AnswerChoice
Body
Call
FactObject FactValue
Value Score
1
1..*
1..*
![Page 9: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/9.jpg)
RMCM Question - StructureAnswer Choices should be closely related
1 Answer choice
FO/FV
2 5
3 Query 4
![Page 10: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/10.jpg)
How it works – Examiner’s view…Select incorrect answer choices
strategically in order to --Emphasize the facts provided in
the questionIdentify “Fact Objects” for each
answer choice; Provide “Fact Values” of the
“True” Fact Objects ( for each answer choice)
Provide score / partial score for each FO for each answer choice
![Page 11: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/11.jpg)
RMCM – Examiner’s view (contd)Examiner must:Define the Objectives for the question
Figure out what concept to assessKnow where and how students miss the
conceptForm answer choices around common errors
Work BackwardsCorrelate facts / segments in query with errors
Determine importance of each FOApportion relative credit for the facts / segments
![Page 12: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/12.jpg)
How it works – System viewThe RMCM system:Assists Examiner with
Editorial support to write or import questions;
Selecting “Fact Objects” for an answer choice with (proprietary) segmentation software;
Maintaining FO/FV databases, and credit generation databases
Generating computer-assisted queries
Assists Examinee with navigation
![Page 13: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/13.jpg)
How it works System view (contd)Compare Examinee’s answers with
Examiner’s data Identifying FO’s for selected wrong answer
choice + Fact Values for (True) FO’s for selected
wrong answer
Calculate score for question based on comparison Identification, or not, of correct answer;
and Identification of FO’s for selected wrong
answer Fact Values for FO’s for the selected wrong
answer
Calculate score for test
![Page 14: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/14.jpg)
How the system works – Logic I
![Page 15: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/15.jpg)
How the system works – Logic II
![Page 16: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/16.jpg)
Example 1What is the area of a rectangle with sides 3 and 4?
A. 5B. 12C. 14D. 16E. 9
![Page 17: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/17.jpg)
Screen mock up for Example 1
![Page 18: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/18.jpg)
Screen mock up continued
![Page 19: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/19.jpg)
Screen mock up continued
![Page 20: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/20.jpg)
Screen mock up continued
![Page 21: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/21.jpg)
Example 2Divide and simplify the expression: (10) 2 =
(A) 8(B) 22(C) 5(D) 25(E) 23 This question is taken from REA’s SAT preparation guidebook (1990)
![Page 22: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/22.jpg)
Example 2 (contd)Correct answer is C.
(A) would be the correct answer for the expression: (10-2);
(B) would be correct for the expression: 2 * ( 5 - 1 );
(D) would be correct for the expression: (10 * 2);
(E) would be correct for the expression:
(10 + 2).
![Page 23: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/23.jpg)
Example 3
![Page 24: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/24.jpg)
Example 3 (contd)What is the current in the 4 Ω resistor while switch S is open?
(A) Is correct if 12 V is changed to 0 V;
(B) Is the correct answer;(C)Is correct if 12 V is changed to
24 V;(D)Is correct if 12 V is changed to
40 V;(E) I correct if 12 V is changed to 60
V.
![Page 25: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/25.jpg)
Example 3 & DiscussionOther variants possible with different
sets of Fact Objects; e.g., Resistors with other resistance values;
or Changes of both resistance and
voltage.One interesting case is where a Fact
Object is “Switch S is open or closed.”In general, FO holds information –
may be numeric or non-numericExamples: Mathematical operation;
Word order in a sentence; Sequence of lab activity
![Page 26: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/26.jpg)
Example 4 – Math word problem*The sophomore class plans to sell T-
shirts with the school’s name on them. The cost of each T-shirt alone is $3.50, and the printing cost of each shirt is $0.75. If the class plans on selling each printed T-shirt for $11, what expression can you use to calculate the class profit for selling n printed T-shirts?
A. 11.00 – (3.50 + 0.75)nB. 11.00n – (3.50 + 0.75)C. 11.00 – 3.50 – 0.75nD. (11.00 – 3.50 – 0.75)n* From 2001 MCAS test for 10th grade
![Page 27: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/27.jpg)
Example 4; non-numeric FOThe sophomore class plans to sell T-
shirts with the school’s name on them. The cost of each T-shirt alone is $3.50, and the printing cost of each shirt is $0.75. If the class plans on selling each printed T-shirt for $11, what expression can you use to calculate the class profit for selling n printed T-shirts?
A. 11.00 – (3.50 + 0.75)nB. 11.00n – (3.50 + 0.75)C. 11.00 – 3.50 – 0.75nD. (11.00 – 3.50 – 0.75)n
![Page 28: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/28.jpg)
Non-numeric FO highlightedThe sophomore class plans to sell T-
shirts with the school’s name on them. The cost of each T-shirt alone is $3.50, and the printing cost of each shirt is $0.75. If the class plans on selling each printed T-shirt for $11, what expression can you use to calculate the class profit for selling n printed T-shirts?
A. 11.00 – (3.50 + 0.75)nB. 11.00n – (3.50 + 0.75)C. 11.00 – 3.50 – 0.75nD. (11.00 – 3.50 – 0.75)n
![Page 29: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/29.jpg)
Other question FO’s highlighted
The sophomore class plans to sell T-shirts with the school’s name on them. The cost of each T-shirt alone is $3.50, and the printing cost of each shirt is $0.75. If the class plans on selling each printed T-shirt for $11, what expression can you use to calculate the class profit for selling n printed T-shirts?
A. 11.00 – (3.50 + 0.75)nB. 11.00n – (3.50 + 0.75)C. 11.00 – 3.50 – 0.75nD. (11.00 – 3.50 – 0.75)n
![Page 30: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/30.jpg)
Examiner’s selection & save of FOThe sophomore class plans to sell T-
shirts with the school’s name on them. The cost of each T-shirt alone is $3.50, and the printing cost of each shirt is $0.75. If the class plans on selling each printed T-shirt for $11, what expression can you use to calculate the class profit for selling n printed T-shirts?
A. 11.00 – (3.50 + 0.75)nB. 11.00n – (3.50 + 0.75)C. 11.00 – 3.50 – 0.75nD. (11.00 – 3.50 – 0.75)n
selling each printed T-shirt
![Page 31: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/31.jpg)
RMCM query modifications…A: Change ‘selling each printed shirt’
to ‘selling all printed shirts.’B: Change ‘cost of each T-shirt’ to ‘cost
of all T-shirts,’ and change ‘printing cost of each shirt’ to ‘printing cost of all shirts.’
C: Change ‘cost of each T-shirt’ to ‘cost of all T-shirts,’ and change ‘selling each printed shirt’ to ‘selling all printed shirts.’
D: (may indicate that student lacks understanding of the material tested and may lead to complete loss of credit for the question).
![Page 32: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/32.jpg)
Another RMCM embodiment of Example 4…Examinee Task:
choose an answer that you believe is incorrect.
find facts /segments from LIST, so that chosen incorrect answer becomes correct answer to changed question
LIST: --‘cost of each T-shirt;’ ‘alone;’ ‘$3.50;’ ‘printing cost of each shirt;’ ‘$0.75;’ ‘selling each printed shirt;’ ‘$11;’ ‘profit;’ ‘selling n printed shirts’
![Page 33: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/33.jpg)
LIST: ‘cost of each T-shirt;’ ‘alone;’ ‘$3.50;’ ‘printing cost of each shirt;’ ‘$0.75;’ ‘selling each printed shirt;’ ‘$11;’ ‘’profit;’ ‘selling n printed shirts.’
Expected answers:A: ‘selling each printed shirt.’B: ‘cost of each T-shirt’ and ‘printing cost of
each shirt.’C: ‘cost of each T-shirt’ and ‘selling each
printed shirt.’D: (test-taker lacks understanding of
subject matter)
![Page 34: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/34.jpg)
Wrap up - Advantages of RMCM
Multiple-choice format used for RMCM:--
Allows context definition economically
Questions may be reusedScoring may be automatedOther advantages of automation
e.g. computer-aided generation of queries
Forces analysis/ compares / contrasts of similar and related concepts
![Page 35: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/35.jpg)
Wrap up - Disadvantages mitigated by RMCM
Unlike traditional Multiple-choice RMCM:
Discourages answering without careful reading
Encourages analysis of questions
Finds depth of student’s learning
Databank of questions has lasting value
![Page 36: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/36.jpg)
Disadvantages of open format/ essay mitigated by RMCM
Encourages identification of context
Discourages padding of answers with irrelevant writing
More efficient
Computer-aided assessment possible
![Page 37: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/37.jpg)
InvitationIf you would like to get involved
--
Effort is underway to:Assemble groups of educators /
others Assemble RMCM Questions
databanks Develop self-study coursesDevelop instructor-led courses
You are also invited to contribute to further design and development of RMCM software
![Page 38: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/38.jpg)
![Page 39: Reverse+multiple choice+method--presentation 6-feb12](https://reader035.fdocuments.us/reader035/viewer/2022062419/5589a229d8b42ac00c8b45e9/html5/thumbnails/39.jpg)
Thank You!Contact
Indu M. Anand