Authentic Performance Assessment

Allison D.Spinelli,D.O.

Assistant Professor of Anesthesiology

Residency Program Director

Authentic Performance Assessment

Background What is competency How to assess competency Examples of APA model Competency rating scale Pros/cons Looking ahead

Background

Why change evaluation system Standardizes faculty approach to

evaluating residents Helps to define resident’s competency

level for a particular year/rotation Better defines competency

What is Competency

‘Know it when I see it”

How to Assess Competency

Authentic Performance Assessment Model

Authentic Performance Assessment Model

Real-time assessment of resident performance during actual patient care

Clearly defines the skills/knowledge the resident must possess to be deemed competent at a task

Specific to resident year of training Specific to rotations

Examples of APA Model

CAY 1 Mentorship period Occurs 2 weeks after start of 1st year Resident should be able to set-up

room,perform machine check,start iv,intubate uncomplicated airway

Examples of APA Model

Four month APA for CAY1 resident All of previous skills/knowledge plus: Induction of GA

Examples of APA Model

8 month APA for CAY1 resident All of previous skills/knowledge plus Maintenance of GA and intra-op

differential dx of vital sign abnormalities

Examples of APA Model

12 month APA for CAY1 resident All of previous knowledge/skills as well

as emergence of patient from GA and post-op care

Examples of APA Model CAY 2 resident on cardiac anesthesia

rotation Pre-op,line placements,induction,airway Manage prebypass period Manage postbypass period Manage transfer of care to CSICU Diagnose and treat postop

complications

Examples of APA Model

CAY 3 residents on any rotation and for the most acute and complex patients

Pre-op assessment and anesthetic plan Line placement Induction and airway management Intra-op management of data Emergene and transfer of care Post-op complications

Competency Rating Scale Rubric system Highly competent Competent Becoming competent Not competent Gauges the degree of resident’s ability

to perform independently vs. need for attending intervention

Examples of Competency Rating Scale

Highly competent The resident is smooth and efficient The resident adapts

knowledge,judgement,and skill to start the case and induce anesthesia with no significant errors and most appropriately

The resident does not require attending intervention

Examples of Competency Rating Scales

Competent The resident lacks smoothness and

efficiency but adapts knowledge,judgement,and skill to start the case and induce anesthesia with no significant errors and most appropriately

The resident does not require attending interevention

Examples of Competency Rating Scales

Becoming competent The resident lacks smoothness and

efficiency and some knowledge,judgement or skill and exhibits some limits in adaptability.

The resident makes some signifiant errors but is able to start the case although not most appropriately

May require attending intervention

Examples of Competency Rating Scales

Not competent The resident can only start the case and

induce anesthesia with extensive coaching.The resident requires attending intervention

APA Model Logistics

Scheduled to occur at the 4,8,and 12 month intervals for the CAY1 and CAY3 residents

Scheduled to occur at the end of each subspecialty month for theCAY2 residents

Requires 3 different cases and faculty members

APA Model

Goal is for all residents to be competent or above

Becoming competent can be acceptable at start of new rotation but need to see improvement otherwise,remediate

Not competent=remediation/probation/ Daily evaluations til competent or no

credit

APA Model Pros Helps to simply define competency Residents know what is expected of them

and what goals are Helps faculty gauge competence in a

uniform way Easy form to fill out Helps faculty give timely and constructive

feedback Works well with one-to-one system

APA Model Cons Can be difficult to match patient /case

complexity to resident year Faculty/resident cooperation to assure

timeliness of assessments Residents free to pick and choose

attendings who evaluate them Residents(and faculty)wait until they have

a perfect case to assess Creates paperwork

Future of APA Model

Randomize assessments Assessment teams Expand concept to use for resident

evaluation of faculty teaching