Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J....
-
date post
21-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J....
![Page 1: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/1.jpg)
Assessing the Assessments: Physics
An Initial Analysis of the June 2003 NYS Regents Physics Exam
J. Zawicki, SUNY Buffalo State CollegeM. Jabot, SUNY Fredonia
![Page 2: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/2.jpg)
J. Zawicki, M. Jabot 2July 2004
Assessment Purposes
Measure knowledge Measure gain in knowledge Measure preparation (predict success) Sorting (Grading) Degree requirements (benchmarks) …
![Page 3: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/3.jpg)
J. Zawicki, M. Jabot 3July 2004
Curriculum, Assessment and Instruction
•Frameworks
•Syllabi
•Guides
•Blueprints
•Benchmarks
•Objective tests
•Performance assessments
•Portfolios
•Teacher Observations
•Group Activities
•Program Evaluations
Curriculum Standards
Assessment/Evaluation System Instructional Program
alignment
validity
correlation
•Instructional styles
•Print materials
•Equipment
•Facilities
•Technology
•Community
![Page 4: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/4.jpg)
J. Zawicki, M. Jabot 4July 2004
Concepts (Continued)
Difficulty – (Percentage or proportion that are successful on an item)FacilityDifficulty
Discrimination – (How well does the item differentiate between students who understand the subject and those who do not?)
![Page 5: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/5.jpg)
J. Zawicki, M. Jabot 5July 2004
Concepts
Validity – how well the item measures match the target construct. May be qualified as:ConstructContent (Face)Criterion Related
Typically determined by a panel of experts
![Page 6: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/6.jpg)
J. Zawicki, M. Jabot 6July 2004
Concepts (Continued)
Reliability – can the results be replicated? Inter-rater (Do two or more raters agree on the score
for an item?) Test/Re-test (Will a student earn similar scores on
different administrations?) Internal Consistency
Criterion referenced tests – have the students met the “standard”
![Page 7: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/7.jpg)
J. Zawicki, M. Jabot 7July 2004
Concepts (Continued)
Latency – (How long do students take to complete the test?)
Equitable (Fair) Timed tests (Power tests)
![Page 8: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/8.jpg)
J. Zawicki, M. Jabot 8July 2004
Types of Analysis
Traditional (difficulty, discrimination) Rasch Analysis (item difficulty is equated
to student ability) Cognitive Level (Bloom’s taxonomy
simplified: knowing, using, integrating)
![Page 9: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/9.jpg)
J. Zawicki, M. Jabot 9July 2004
Initial AnalysisPhysics 2004 Format Credit Key R/C 0 R/C 1 R/C 2 R/C 3 R/C 4 R/C (%) Facility Difficulty DE P
Item # 01 MC 1 4 0 1 1 1 181 98 0.98 0.02 -4.10 0.98
Item # 39 MC 1 1 0 178 4 1 0 97 0.97 0.03 -3.39 0.96
Item # 46 MC 1 4 1 72 25 20 64 35 0.35 0.65 0.63 0.30
Item # 20 MC 1 3 0 30 6 56 92 30 0.30 0.70 0.83 0.26
Item # 60 CR 1 2 58 1 0 0 33 0.33 0.67 0.73 0.28
Item # 63 CR 2 3 2 55 1 0 31 0.31 0.69 0.79 0.27
Item # 68 CR 2 37 6 18 0 0 11 0.11 0.89 2.05 0.09
Item # 55 CR 1 50 10 1 0 0 7 0.07 0.93 2.66 0.05
![Page 10: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/10.jpg)
J. Zawicki, M. Jabot 10July 2004
Difficulty Rankings
Easier MC: 1, 39 More Difficult MC: 46, 20 Easier CR: 60, 63 More Difficult CR 68, 55
![Page 11: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/11.jpg)
J. Zawicki, M. Jabot 11July 2004
Easier Multiple Choice
![Page 12: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/12.jpg)
J. Zawicki, M. Jabot 12July 2004
Easier Multiple Choice
![Page 13: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/13.jpg)
J. Zawicki, M. Jabot 13July 2004
More Difficult Multiple Choice
![Page 14: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/14.jpg)
J. Zawicki, M. Jabot 14July 2004
More Difficult Multiple Choice
![Page 15: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/15.jpg)
J. Zawicki, M. Jabot 15July 2004
Easier Constructed Response
![Page 16: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/16.jpg)
J. Zawicki, M. Jabot 16July 2004
Easier Constructed Response
![Page 17: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/17.jpg)
J. Zawicki, M. Jabot 17July 2004
More Difficult Constructed Response
![Page 18: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/18.jpg)
J. Zawicki, M. Jabot 18July 2004
More Difficult Constructed Response
![Page 19: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/19.jpg)
J. Zawicki, M. Jabot 19July 2004
Common Threads – Easier Items
Graphing Using charts, tables, graphs Classification (vectors, scalars, …)
![Page 20: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/20.jpg)
J. Zawicki, M. Jabot 20July 2004
Common Threads – More Difficult Items
Mathematical relationships (Modeling) Direct Inverse Linear Exponential
Electric, Magnetic, and Gravitational Interactions; Fields (CSEM, Modeling, Castle, Knight)
Energy vs. Force (FCI, PET, Modeling)
![Page 21: Assessing the Assessments: Physics An Initial Analysis of the June 2003 NYS Regents Physics Exam J. Zawicki, SUNY Buffalo State College M. Jabot, SUNY.](https://reader038.fdocuments.us/reader038/viewer/2022110322/56649d575503460f94a36251/html5/thumbnails/21.jpg)
J. Zawicki, M. Jabot 21July 2004
Next Steps
Questions Implications for classrooms (program review) Additional Resources
BSC: SCI685, Evaluation in Science Education SUNY Fredonia: EDU503, Evaluation in the Schools SUNY Buffalo: LAI534, Measurement & Evaluation Of
Science Instruction STANYS 2004 Annual Conference, November 7-9, 2004 Rochester Science Educator’s Conference NYSSELA Perspectives