Post on 06-Feb-2016
description
An Analysis of NYS Regents Physics Exam Issues
J. ZawickiScience Education, BSCSTANYS DAL, Physics Mentor
9/25/03NYS Regents Physics -- Zawicki2
Additional Researchers
Michael Jabot, SUNY Fredonia Kathleen Falconer, SUNY BSC Dan MacIsaac, SUNY BSC David Henry, SUNY BSC Rita Fischer, Elba Central School
9/25/03NYS Regents Physics -- Zawicki3
Presentation Overview
Issues from the field Exam findings Issues and events Future directions…
9/25/03NYS Regents Physics -- Zawicki4
Issues From the Field
What are issue(s) are uppermost in your mind at this time?
9/25/03NYS Regents Physics -- Zawicki5
Exam Findings
Call for student papers, STANYS, NYS Section AAPT
>2000 student papers were submitted Results from across NYS
9/25/03NYS Regents Physics -- Zawicki6
Initial Distribution of Papers
NYC (& Manhattan) 0Catskill-Leatherstocking 5Central-Western 74Eastern 74Mohawk Valley 80Nassau 324North Central 17Northeastern 24Northwestern 114Southeastern 126Southern 71Southwestern 271Suffolk 18Westchester 21Western 473
9/25/03NYS Regents Physics -- Zawicki7
Rasch Analysis
Parts A and B-1; B-2 and C presented together Concepts
– Difficulty – how well did the students do?– Discrimination – how well did an item distinguish
between high and low performing students?– Rasch transformation – log transformation
9/25/03NYS Regents Physics -- Zawicki8
Item Diffi culty Range
-4.0000
-3.0000
-2.0000
-1.0000
0.0000
1.0000
2.0000
3.0000
4.0000
0 10 20 30 40 50 60 70 80
Ranked Item Number
9/25/03NYS Regents Physics -- Zawicki9
“Easy” Items – A, B-1
Item Key R1 R2 R3 R4 %C N DE
Q45 3 57 26 860 15 89.8 958 -2.17
Q40 3 40 3 807 106 84.2 956 -1.69
Q36 2 122 777 38 21 81.1 958 -1.46
Q38 3 12 54 759 131 79.2 956 -1.35
Q3 2 129 749 38 42 78.2 958 -1.28
9/25/03NYS Regents Physics -- Zawicki10
9/25/03NYS Regents Physics -- Zawicki11
9/25/03NYS Regents Physics -- Zawicki12
9/25/03NYS Regents Physics -- Zawicki13
9/25/03NYS Regents Physics -- Zawicki14
“Difficult” Items – A, B-1
Item Key R1 R2 R3 R4 %C N DE
Q15 1 324 13 42 576 33.8 955 0.67
Q11 3 69 76 352 458 36.7 955 0.54
Q4 4 313 20 237 387 40.4 957 0.39
Q7 2 95 394 320 148 41.1 957 0.36
Q34 4 122 371 46 418 43.6 957 0.25
9/25/03NYS Regents Physics -- Zawicki15
9/25/03NYS Regents Physics -- Zawicki16
9/25/03NYS Regents Physics -- Zawicki17
9/25/03NYS Regents Physics -- Zawicki18
9/25/03NYS Regents Physics -- Zawicki19
“Easy Items” B-2, C
Item Key R0 R1 R2 %C N DE
Q59 1 16 458 -- 96.6 474 -3.35
Q54 2 27 51 398 89.0 476 -2.84
Q65 1 27 447 -- 94.3 474 -2.81
Q58 1 34 440 -- 92.8 474 -2.56
Q69 2 23 60 391 91.0 474 -2.32
9/25/03NYS Regents Physics -- Zawicki20
9/25/03NYS Regents Physics -- Zawicki21
9/25/03NYS Regents Physics -- Zawicki22
9/25/03NYS Regents Physics -- Zawicki23
9/25/03NYS Regents Physics -- Zawicki24
“Difficult” Items B-2, C
Item Key R0 R1 R2 %C N DE
Q62 1 266 207 -- 43.8 473 0.25
Q48 1 253 223 -- 46.8 476 0.13
Q53 1 246 230 -- 48.3 476 0.07
Q74 1 243 231 -- 48.7 474 0.05
Q56 1 242 233 -- 49.1 475 0.04
9/25/03NYS Regents Physics -- Zawicki25
9/25/03NYS Regents Physics -- Zawicki26
9/25/03NYS Regents Physics -- Zawicki27
9/25/03NYS Regents Physics -- Zawicki28
Reading Level
McLaughlin-SMOG Additional Analysis Pending
9/25/03NYS Regents Physics -- Zawicki29
Initial Analysis, Reading Levels
Exam Administration Grade Equivalent
June 2000 8th
June 2001 10th (Low)
June 2002 10th (High)
June 2003 11th
9/25/03NYS Regents Physics -- Zawicki30
Conceptual Level
Modification of Bloom– Knowing– Using 1– Using 2– Integrating
9/25/03NYS Regents Physics -- Zawicki31
Exam Conceptual Level
Exam Administration Conceptual Level
June 2000 1.61
June 2001 1.59
June 2002 1.89
June 2003 1.74
9/25/03NYS Regents Physics -- Zawicki32
Exam FormatExamAdministration MC W P
June 2000 75 11 3January 2001 75 10 3June 2001 75 11 4January 2002 75 11 3June 2002 45 24 12August 2002 47 21 8January 2003 50 27 10June 2003 47 29 16
9/25/03NYS Regents Physics -- Zawicki33
AP Correlation
Approximately 0.69 Precise value varied from year to year
9/25/03NYS Regents Physics -- Zawicki34
NYS Physics/AP B
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
1 2 3 4 5
bef or e 2002
2002 & 2003
9/25/03NYS Regents Physics -- Zawicki35
Listserv Comments
Latency -- Incomplete/unfinished tests Power Tests – assessments where time does
not play a significant role in the score
9/25/03NYS Regents Physics -- Zawicki36
Issues
Impact of Scaled Scoring on student scores in the standards-based exams
Direction of future exams
9/25/03NYS Regents Physics -- Zawicki37
Sessions
NYSUT Session with Commissioner Mills Science Education Consortium – August Regents Session – September Senate Hearings NYS AAPT Meeting (SUNY Binghamton) Physics Summit (Albany Area)
9/25/03NYS Regents Physics -- Zawicki38
Additional Directions?
Where do we go from here?