Video-Based Assessment of a Coach’s Ability to Fault-Check Performance
13
American Educational Research Association Annual Meeting AERA 53.069 – TICL 6: Advancing the State of the Art: Computer-Based Assessment and Learning Environments New York, NY - March 27, 2008 John J. Lee, Girlie C. Delacruz, Jesse Elmore, & William L. Bewley Video-Based Assessment of a Coach’s Ability to Fault-Check Performance
description
Video-Based Assessment of a Coach’s Ability to Fault-Check Performance. John J. Lee, Girlie C. Delacruz, Jesse Elmore, & William L. Bewley. American Educational Research Association Annual Meeting - PowerPoint PPT Presentation
Transcript of Video-Based Assessment of a Coach’s Ability to Fault-Check Performance
American Educational Research AssociationAnnual Meeting
AERA 53.069 – TICL 6: Advancing the State of the Art: Computer-Based Assessment and Learning EnvironmentsNew York, NY - March 27, 2008
John J. Lee, Girlie C. Delacruz, Jesse Elmore, & William L. Bewley
Video-Based Assessment of a Coach’s Ability to Fault-Check Performance
• Background
• Research base
• Assessments
• Design
• Analysis
• Demo
Overview
Background1. Context: Fault Checking Performance on the Range
2. Two Weapon Manipulations Skills: Speed Reload and Tactical Reload
3. Using video-based assessment from four different angles
4. Main research question: Does assessment response format make a difference?
Research Base• Expert-Novice
1. Experts have schemas to reduce cognitive load
2. Expertise Reversal Effect: some task formats may impede performance for experts when inconsistent or unaligned with existing schema
3. Scaffolding: helpful to novices who have little or no prior knowledge
4. Distributed Representations: task’s response format as scaffold
• Assessment Validation
1. Validity – reduce construct irrelevant variance as much as you can
2. Response fidelity more predictive validity than more abstract
Video-Based Assessments
1. Shooter-Skill Diagnostic Task:
Checklist Version
Open-ended Version
2. Shooter-Skill Performance Task:
Proper / Improper Execution
Shooter-Skill Performance Task: Drag and Drop Order
Video-Based Assessments
Video-Based Assessments
Video-Based Assessments
Video-Based Assessments
Video-Based Assessments
Sensitivity to Expertise
Research Question Independent Variable
Dependent Variable
Prediction
1. Do experts perform better on the SSDT that is open-ended because it is more like what they do on the firing line?
Expertise level (four levels)
SSDT Open-ended scores
Experts (PMIs)> Unit Coaches > Coaches Course Post-test > Novices (SLR non-coaches)
2. Do novices perform better on the SSDT that uses a checklist because that format provides a scaffold for performance?
Expertise level (four levels)
SSDT Checklist scores
Novices (SLR non-coaches) > Coaches Course Post-test > Unit Coaches > Experts (PMIs)
3. Is the SSPT (proper/improper) sensitive to expertise?
Expertise level (four levels)
SSPT proper/improper scores
Experts (PMIs) > Unit Coaches > Coaches Course Post-test > Novices (SLR non-coaches)
4. Is the SSPT (drag and drop) sensitive to expertise?
Expertise level (four levels)
SSPT drag and drop scores
Experts (PMIs) > Unit Coaches > Coaches Course Post-test > >Novices (SLR non-coaches)
Sensitivity to Instruction
Research Question Independent Variable
Dependent Variable Prediction
1. Is the SSDT open-ended sensitive to instruction?
Administration (pre- test versus post-test)
SSDT Open-ended scores
Post-instruction > Pre-instruction
2. Is the SSDT checklist sensitive to instruction?
Administration (pre- test versus post-test)
SSDT Checklist scores Post-instruction > Pre-instruction
3. Is the SSPT (proper/improper) sensitive to instruction?
Administration (pre- test versus post-test)
SSPT proper/improper scores
Post-instruction > Pre-instruction
4. Is the SSPT (drag and drop) sensitive to instruction?
Administration (pre- test versus post-test)
SSPT drag and drop scores
Post-instruction > Pre-instruction
![Compressed gaseous fuel measuring systems for vehicles ...fault exceeding the applicable fault limit value [OIML V 1:2013, 5.14] 3.1.13 durability ability of the measuring instrument](https://static.fdocuments.us/doc/165x107/60c189919f6154633d311b2d/compressed-gaseous-fuel-measuring-systems-for-vehicles-fault-exceeding-the-applicable.jpg)
