QUALITY ASSURANCE IN OBJECTIVE TESTING
44
QUALITY ASSURANCE IN OBJECTIVE TESTING Isaac Olakanmi Abimbola Department of Science Education University of Ilorin Ilorin, Nigeria [email protected]
-
Upload
robin-mitchell -
Category
Documents
-
view
25 -
download
0
description
QUALITY ASSURANCE IN OBJECTIVE TESTING. Isaac Olakanmi Abimbola Department of Science Education University of Ilorin Ilorin, Nigeria [email protected]. OUTLINE. Introduction Types of Test Essay Objective Forms of Objective Test Quality Assurance - PowerPoint PPT Presentation
Transcript of QUALITY ASSURANCE IN OBJECTIVE TESTING
QUALITY ASSURANCE IN OBJECTIVE TESTINGIsaac Olakanmi Abimbola
Whatever is implemented in instruction should guide testing
Teachers should only test what was intended for learning and what was actually taught
Lesson objectives Instruction Test
TYPES OF TEST
Answered in writing only
TYPES OF TEST (CONTD.)
(i) Matching items
(ii) Multiple-choice items
1. Two-choice items
(i)True/False items
Shooting questions
Statements to be judged and identified by examinees as True or False
Forms of Objective Test (Contd.)
1. Two-choice items
Example:
The name of the first spacecraft launched in 1957 by the former Soviet Union is Sputnik I _____ (True/False)
Forms of Objective Test (Contd.)
1. Two-choice items
Ensure balance between true and false items
Each statement should be clearly true or false, not partially true or false
Forms of Objective Test (Contd.)
1. Two-choice items
Measure only one idea in one statement—no double-barreled item
Forms of Objective Test (Contd.)
1. Two-choice items
Quality Assurance (Contd.)
Use one judgement mode at a time—True or False, Yes or No, Correct/Incorrect
Avoid direct quotations from books or other materials
Do not give clues
1. Two-choice items
Example:
(i) Matching items
The response column is shuffled against the premise column
Examinee matches each premise with a corresponding response
Measures understanding of concepts, principles and classification schemes
Forms of Objective Test (Contd.)
2. More than two-choice items
(i) Matching items (Contd.)
Example:
Match the weather instruments in Column 1 with their uses in Column 2 by writing the appropriate letters (A-F) in the spaces provided:
Forms of Objective Test (Contd.)
2. More than two-choice items
(i) Matching items (Contd.)
7.
8.
10.
F
Measures the degree of hotness or coldness of a living body
Forms of Objective Test (Contd.)
2. More than two-choice items (Contd.)
(i) Matching items (Contd.)
Quality Assurance
Premise should be on the left-hand column with numbers and responses on the right-hand column with letters
Items should be homogeneous
Responses should be more than premises
Provide no clue in the two columns
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items
Stem could be a question or statement
3-5 alternatives/options are available with only one correct or appropriate answer (The Key)
Incorrect or inappropriate answers are called distractors/distracters/foils
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items(Contd.)
Scoring is highly objective—lay person/machine
Chance for guessing is minimal
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items(Contd.)
Examples:
1) When human beings breathe out, what do they breathe out?
A. air B. oxygen C. nitrogen D. carbon dioxide
2) An example of a living thing is _________.
A. water B. fire C. egg D. wind
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items(Contd.)
All the options should appear plausible
Letters of the correct options should vary
Avoid “None of the above” and “All of the above “ as options
Do not start a stem with a dash
QUALITY ASSURANCE IN OBJECTIVE TESTING
1. Statement of lesson objectives
2. Table of Specification/Test Blueprint to ensure fair representation of course content
Objectives/Contents can be stated as percentages, number of items or item numbers.
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
% of Representation on Test
3. Determine the item format to be used
4. Item Writing
(i) Use terms whose definitions are precise from examinees’ point of view
(ii) Stem should not start with a dash or blank space
(iii) Stem should not contain negatives, especially, double negatives
(iv) Use students’ misconceptions and alternative conceptions as distractors
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
4. Item Writing (Contd.)
Example (Misconceptions/Alternative Conceptions):
Energy is an abstract idea that is not easy to define. Which of the following statements best represents what energy is?
A. energy is force B. Energy is a quantity
C. energy behaves as a fluid D. energy is fuel
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
4. Item Writing (Contd.)
Example (Misconceptions/Alternative Conceptions—Multiple levels):
Identify which of the following are characteristic of the vertebrate respiratory surface:
1. moist 2. vascularized 3.semi-permeable 4. freely
permeable 5. dry
4. Item Writing (Contd.)
A. 1,2,3. B. 1,2,5. C. 2,3,5. D. 2,4,5. E.1,3,5.
5. Review of Test Items
Present the test items to others for review. HOD should review the items to spot grammatical errors, omissions and gender friendliness of the items.
Regulation on multiple examiners is useful here
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
6.Trial Run of the Test
Try out the test on a representative sample of the target population and score the test in readiness for item analysis.
7. Item Analysis
Arrange candidates’ scripts according to their scores. Different authors divide the scores into three or four equal parts.
(i) Item Difficulty/Facility Index
(ii) Item Discrimination Power
(iv) Further Item Validity
7. Item Analysis (Contd.)
(i) Item Difficulty Index (Contd.)
If 15/20 of the upper group and 5/20 of the lower group got item 3 correct (B), therefore, 20 (15+5) persons out of 40 (20+20) got the item correct.
Item 3
ITEM 3
7. Item Analysis (Contd.)
Item Difficulty = 100x (R/T) = 100x20/40 = 50%
where R = the number of persons in the upper and lower thirds combined who got the item right, and
T = total number of persons in the upper and lower thirds combined
Therefore, item 3 is moderately difficult.
Difficulty Index of < 10% =Too difficult; > 90% = Too easy
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
(ii) Item Discrimination Power
Extent to which each item discriminates between those in the upper third and the lower third, respectively.
Item Discrimination Power = Ru – Rl /½ T, where:
Ru = no. of those in the upper third who got the item right
Rl = no. of those in the lower third who got the item right
T = total number of persons in the upper and lower thirds
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
The DP for item 3 above is
Ru – Rl = 15-5 = 10 =0.50
½ T ½(20+20) 20
Range of DP = -1.00 to +1.00
Interpretation: 0.40 and above =Very good item
0.30 to 0.39 = Reasonably good but room for improvement
0.20 to 0.29 = Marginal, improve
< 0.19 = Poor, reject or revise
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
(iii) Effectiveness of Distractors
Every option of an item is chosen by at least one person in the upper or lower third of the class while each distractor is chosen more frequently by those in the lower third than in the upper third.
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
Option D is the correct answer in item 5
Options A, B, & C are all functional distractors with at least one candidate picking each of them
Options A & B discriminate rightly while option C discriminates wrongly
Item 5
Item 5
Item 5
7. Item Analysis (Contd.)
(iv) Coefficient of Item Validity (Yarroch, 1991)
This involves a post-test clinical interview of examinees to find out “the relationship between the examinees’ knowledge and the knowledge the examination item was intended to measure” (p. 621). Essay items equivalent to the multiple-choice items could also be administered to the examinees to further test their proper understanding.
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
Civ = (% correct for correct reason) +
(% incorrect for correct reason)
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
8. Amend the items as necessary based on the results of item analysis and reject items that are not suitable
9. Select and compile the final test items duly following rules guiding objective testing.
Ethical principles in evaluation
The teacher must display a sense of duty towards the stakeholders
The method of evaluation must protect the integrity of examinees and examiners
The examinees must be provided with adequate information about the test and the method for its preparation
Ethical principles in evaluation contd.
All tests items should be vetted by the Heads of Department and preferably scored twice by two independent scorers as stipulated in the university regulations
There should be sufficient check against errors in adding marks and preparation of mark sheets
There should be continuous research on improving the evaluation programme by individual lecturers and other stakeholders
The actual test items should never be revealed to any unauthorized persons
Ethical principles in evaluation contd.
Test material should never be revealed to the students in advance directly or indirectly
The test scores or trend in the test performance of a student should not be revealed to an unauthorized person. (Sidhu, 2005, p. 357)
CONCLUSION
Objectives tests, especially multiple-choice tests, are good for preparing global citizens since they are popular among many countries of the world
They can be used to test practically any subject matter, they only require some training
Objective tests are fun to prepare, with adequate practice, especially using continuous assessment tests as starting points.
They take off the drudgery of marking from lecturers
Students have access to their results in good time
THANK YOU
BIBLIOGRAPHY
Abimbola, I. O. (1996). Advances in the development and validation of instruments for assessing students’ science knowledge. In G. A. Badmus & P. I. Odor (Eds.), Challenges of managing educational assessment in Nigeria (pp. 33-40). Kaduna: JAMB, NABTEB, & NBEM.
Abiri, J. O. O. (2006). Elements of evaluation, measurement and statistical techniques in education. Ilorin: Library and Publications Committee, University of Ilorin, Ilorin, Nigeria
Gillette, F. (n.d.). Multiple choice tests: Composition, assembly and analysis. San Francisco, CA: San Francisco State University. http://www.sfsu.edu . Retrieved 21/9/14.
BIBLIOGRAPHY (CONTD.)
BIBLIOGRAPHY (CONTD.)
Tests: Objective tests. San Luis Obispo, CA: Academic Skills Center, California Polytechnic State University
Whatever is implemented in instruction should guide testing
Teachers should only test what was intended for learning and what was actually taught
Lesson objectives Instruction Test
TYPES OF TEST
Answered in writing only
TYPES OF TEST (CONTD.)
(i) Matching items
(ii) Multiple-choice items
1. Two-choice items
(i)True/False items
Shooting questions
Statements to be judged and identified by examinees as True or False
Forms of Objective Test (Contd.)
1. Two-choice items
Example:
The name of the first spacecraft launched in 1957 by the former Soviet Union is Sputnik I _____ (True/False)
Forms of Objective Test (Contd.)
1. Two-choice items
Ensure balance between true and false items
Each statement should be clearly true or false, not partially true or false
Forms of Objective Test (Contd.)
1. Two-choice items
Measure only one idea in one statement—no double-barreled item
Forms of Objective Test (Contd.)
1. Two-choice items
Quality Assurance (Contd.)
Use one judgement mode at a time—True or False, Yes or No, Correct/Incorrect
Avoid direct quotations from books or other materials
Do not give clues
1. Two-choice items
Example:
(i) Matching items
The response column is shuffled against the premise column
Examinee matches each premise with a corresponding response
Measures understanding of concepts, principles and classification schemes
Forms of Objective Test (Contd.)
2. More than two-choice items
(i) Matching items (Contd.)
Example:
Match the weather instruments in Column 1 with their uses in Column 2 by writing the appropriate letters (A-F) in the spaces provided:
Forms of Objective Test (Contd.)
2. More than two-choice items
(i) Matching items (Contd.)
7.
8.
10.
F
Measures the degree of hotness or coldness of a living body
Forms of Objective Test (Contd.)
2. More than two-choice items (Contd.)
(i) Matching items (Contd.)
Quality Assurance
Premise should be on the left-hand column with numbers and responses on the right-hand column with letters
Items should be homogeneous
Responses should be more than premises
Provide no clue in the two columns
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items
Stem could be a question or statement
3-5 alternatives/options are available with only one correct or appropriate answer (The Key)
Incorrect or inappropriate answers are called distractors/distracters/foils
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items(Contd.)
Scoring is highly objective—lay person/machine
Chance for guessing is minimal
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items(Contd.)
Examples:
1) When human beings breathe out, what do they breathe out?
A. air B. oxygen C. nitrogen D. carbon dioxide
2) An example of a living thing is _________.
A. water B. fire C. egg D. wind
Forms of Objective Test (Contd.)
More than two-choice items (Contd.)
(ii) Multiple-choice Items(Contd.)
All the options should appear plausible
Letters of the correct options should vary
Avoid “None of the above” and “All of the above “ as options
Do not start a stem with a dash
QUALITY ASSURANCE IN OBJECTIVE TESTING
1. Statement of lesson objectives
2. Table of Specification/Test Blueprint to ensure fair representation of course content
Objectives/Contents can be stated as percentages, number of items or item numbers.
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
% of Representation on Test
3. Determine the item format to be used
4. Item Writing
(i) Use terms whose definitions are precise from examinees’ point of view
(ii) Stem should not start with a dash or blank space
(iii) Stem should not contain negatives, especially, double negatives
(iv) Use students’ misconceptions and alternative conceptions as distractors
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
4. Item Writing (Contd.)
Example (Misconceptions/Alternative Conceptions):
Energy is an abstract idea that is not easy to define. Which of the following statements best represents what energy is?
A. energy is force B. Energy is a quantity
C. energy behaves as a fluid D. energy is fuel
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
4. Item Writing (Contd.)
Example (Misconceptions/Alternative Conceptions—Multiple levels):
Identify which of the following are characteristic of the vertebrate respiratory surface:
1. moist 2. vascularized 3.semi-permeable 4. freely
permeable 5. dry
4. Item Writing (Contd.)
A. 1,2,3. B. 1,2,5. C. 2,3,5. D. 2,4,5. E.1,3,5.
5. Review of Test Items
Present the test items to others for review. HOD should review the items to spot grammatical errors, omissions and gender friendliness of the items.
Regulation on multiple examiners is useful here
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
6.Trial Run of the Test
Try out the test on a representative sample of the target population and score the test in readiness for item analysis.
7. Item Analysis
Arrange candidates’ scripts according to their scores. Different authors divide the scores into three or four equal parts.
(i) Item Difficulty/Facility Index
(ii) Item Discrimination Power
(iv) Further Item Validity
7. Item Analysis (Contd.)
(i) Item Difficulty Index (Contd.)
If 15/20 of the upper group and 5/20 of the lower group got item 3 correct (B), therefore, 20 (15+5) persons out of 40 (20+20) got the item correct.
Item 3
ITEM 3
7. Item Analysis (Contd.)
Item Difficulty = 100x (R/T) = 100x20/40 = 50%
where R = the number of persons in the upper and lower thirds combined who got the item right, and
T = total number of persons in the upper and lower thirds combined
Therefore, item 3 is moderately difficult.
Difficulty Index of < 10% =Too difficult; > 90% = Too easy
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
(ii) Item Discrimination Power
Extent to which each item discriminates between those in the upper third and the lower third, respectively.
Item Discrimination Power = Ru – Rl /½ T, where:
Ru = no. of those in the upper third who got the item right
Rl = no. of those in the lower third who got the item right
T = total number of persons in the upper and lower thirds
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
The DP for item 3 above is
Ru – Rl = 15-5 = 10 =0.50
½ T ½(20+20) 20
Range of DP = -1.00 to +1.00
Interpretation: 0.40 and above =Very good item
0.30 to 0.39 = Reasonably good but room for improvement
0.20 to 0.29 = Marginal, improve
< 0.19 = Poor, reject or revise
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
(iii) Effectiveness of Distractors
Every option of an item is chosen by at least one person in the upper or lower third of the class while each distractor is chosen more frequently by those in the lower third than in the upper third.
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
Option D is the correct answer in item 5
Options A, B, & C are all functional distractors with at least one candidate picking each of them
Options A & B discriminate rightly while option C discriminates wrongly
Item 5
Item 5
Item 5
7. Item Analysis (Contd.)
(iv) Coefficient of Item Validity (Yarroch, 1991)
This involves a post-test clinical interview of examinees to find out “the relationship between the examinees’ knowledge and the knowledge the examination item was intended to measure” (p. 621). Essay items equivalent to the multiple-choice items could also be administered to the examinees to further test their proper understanding.
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
7. Item Analysis (Contd.)
Civ = (% correct for correct reason) +
(% incorrect for correct reason)
QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)
8. Amend the items as necessary based on the results of item analysis and reject items that are not suitable
9. Select and compile the final test items duly following rules guiding objective testing.
Ethical principles in evaluation
The teacher must display a sense of duty towards the stakeholders
The method of evaluation must protect the integrity of examinees and examiners
The examinees must be provided with adequate information about the test and the method for its preparation
Ethical principles in evaluation contd.
All tests items should be vetted by the Heads of Department and preferably scored twice by two independent scorers as stipulated in the university regulations
There should be sufficient check against errors in adding marks and preparation of mark sheets
There should be continuous research on improving the evaluation programme by individual lecturers and other stakeholders
The actual test items should never be revealed to any unauthorized persons
Ethical principles in evaluation contd.
Test material should never be revealed to the students in advance directly or indirectly
The test scores or trend in the test performance of a student should not be revealed to an unauthorized person. (Sidhu, 2005, p. 357)
CONCLUSION
Objectives tests, especially multiple-choice tests, are good for preparing global citizens since they are popular among many countries of the world
They can be used to test practically any subject matter, they only require some training
Objective tests are fun to prepare, with adequate practice, especially using continuous assessment tests as starting points.
They take off the drudgery of marking from lecturers
Students have access to their results in good time
THANK YOU
BIBLIOGRAPHY
Abimbola, I. O. (1996). Advances in the development and validation of instruments for assessing students’ science knowledge. In G. A. Badmus & P. I. Odor (Eds.), Challenges of managing educational assessment in Nigeria (pp. 33-40). Kaduna: JAMB, NABTEB, & NBEM.
Abiri, J. O. O. (2006). Elements of evaluation, measurement and statistical techniques in education. Ilorin: Library and Publications Committee, University of Ilorin, Ilorin, Nigeria
Gillette, F. (n.d.). Multiple choice tests: Composition, assembly and analysis. San Francisco, CA: San Francisco State University. http://www.sfsu.edu . Retrieved 21/9/14.
BIBLIOGRAPHY (CONTD.)
BIBLIOGRAPHY (CONTD.)
Tests: Objective tests. San Luis Obispo, CA: Academic Skills Center, California Polytechnic State University
