ABPN Task Force on Clinical Skills Verification Rater Training

ABPN Task Force on Clinical Skills Verification Rater Training

Co-ChairsKaren BroquetMichael Jibson

AADPRTMichael JibsonNyapati RauRichard Summers

AAPKaren BroquetJoan Anzia

AACAPEugene BeresinJeffrey Hunt

AACDPWilliam LawsonDavid Wilson

ACPDavid KayeTony Rostain

ABPNLarry FaulknerBarbara Schneidman

APADeborah HalesSandra Sexson

Task Force Timetable

• June 2008 – ABPN convened large group of psychiatric and neurologic educators to review literature on direct observation of learners and set goals for educating faculty raters. Task Force on Clinical Skills Verification Rater Training formed, with members from major psychiatric educational organizations.

• September 2008 – Task Force met and outlined curriculum. Meeting summary and action plan, and curricular outline available via these links on AADPRT website.

CSV Rater Training Curriculum

• 5 short Modules

• Each with10-15 minute didactic portion

• Some with video vignettes

• Some with narrative/facilitator’s guide

• All with pre- and post test

• Designed to be suitable for group presentation or self-study

CSV Rater Training Modules

1. Introduction 

2. Training Faculty Evaluators

3. Setting the Examination Climate and Giving Feedback

4. Strategies for Clinical Skills Assessment

5. CSV Performance Parameters

Clinical Skills Verification rater TrainingModule 1

Introduction

Sandra Sexson, M.D.

Tony Rostain, M.D., M.A.

Pre-Test Module 1

1. Which of the following is correct regarding the Psychiatry RRC requirements for the clinical skills verification exam?

a. The resident must demonstrate competence in at least 5 evaluations

b. The evaluations must be conducted by a Board-certified psychiatrist

c. The successful evaluations must be achieved within 1 year of completing training

Pre-Test Module 1

2. Which of the following is acceptable as part of a clinical skills verification examination?

a. videotaped interview

b. standardized patient

c. one-way mirror

d. established patient of the resident

Pre-Test Module 1

3. The in-residency clinical skills verification exam provides an opportunity to:

a. insure direct observation of resident interviews in training

b. provide early assessment of clinical skills with time for remediation

c. attempt to develop standards of assessment across residency training programs

d. all of the above

The Context

• ABPN• The last ABMS member organization to include a live

patient exam in the certifying exam• Standardized patient attempts unsuccessful

• The Concern• The live patient exam unreliable• Many candidates had trouble on interview section• Anecdotal and RRC reports suggested that some residents

were never or very infrequently observed by faculty interviewing patients

The Context

• The Challenge• Would earlier, more formalized,

assessment of this competency allow for remediation and improve the competency of our graduates?

What was our immediate response as psychiatry program directors?

We took our lead from Nancy Reagan and initially said:

But then, we seized the opportunity

And we are here today to join with Obama and say:

RRC Link—V.A.l.h

h) In at least three evaluations with any patient type, in any clinical setting, and at any time during the program, residents must demonstrate satisfactory competence in: establishing an appropriate doctor/patient relationship, psychiatric interviewing, performing the mental status examination and in case presentation. Each of the three required evaluations must be conducted by an ABPN-certified psychiatrist, and at least two of the evaluations must be conducted by different ABPN-certified psychiatrists. Satisfactory demonstration of the competencies during the three required evaluations is required prior to completing the program

Where Are We?

• FIRST RESPONSE• JUST SAY NO!!

• NOW!!!!• YES, WE CAN!!

• Opportunity to:– Insure actual interviews in our programs– Attempt standardization of assessments across

programs– Provide assessments of these clinical skills earlier in

training to provide for potential for remediation– Develop methods for remediation of these specific

competencies– Conduct education research in assessment and

credentialing

What Is Required?

• PGY-1 2007 or PGY-2 2008• Passage of at least 3 CSV evaluations during training

• For residents who enter CAP in PGY IV, one CAP CSV can count toward the 3 Psychiatry CSVs

• Exams must be administered by board certified psychiatrists and no more than two exams can be done by any one psychiatrist

• Time – Minimum 45 minutes – 30 min interview + 15 min presentation, but can be longer

Patient Issues

• Unknown to resident but can be known to faculty (brief overview in morning report ok but no interview of any kind)

• Introductory patient material (intake forms) are ok to utilize

• No simulated or standardized patients• No videotape reviews• One-way mirrors are ok• ABPN silent about whether done during regular work

or in special sessions so ok either way

Evaluation Forms

• Require ABPN approval

• Approved forms are posted on ABPN website (ABPN.com) and AADPRT website

• Programs may submit their own form if they prefer for ABPN approval—procedure on ABPN website

• CSV forms maintained in resident files for review by ACGME site visitor and possibility of ABPN review

What Happens When Part II is No Longer Offered?

• Documentation of satisfactory performance on the clinical skills competencies must be verified by a current program director of an ACGME accredited Psychiatry program to the ABPN as a part of the credentialing process to take the single computerized ABPN Psychiatry exam.

Post-Test Module 1

1. Which of the following is correct regarding the Psychiatry RRC requirements for the clinical skills verification exam?

a. The resident must demonstrate competence in at least 5 evaluations

b. The evaluations must be conducted by a Board-certified psychiatrist

c. The successful evaluations must be achieved within 1 year of completing training

Post-Test Module 1

1. Which of the following is correct regarding the Psychiatry RRC requirements for the clinical skills verification exam?

b. The evaluations must be conducted by a Board-certified psychiatrist

Each of the three required evaluations must be conducted by an ABPN-certified psychiatrist, and at least two of the evaluations must be conducted by different ABPN-certified psychiatrists. Satisfactory demonstration of the competencies during the three required evaluations is required prior to completing the program.

Post-Test Module 1

2. Which of the following is acceptable as part of a clinical skills verification examination?

a. videotaped interview

b. standardized patient

c. one-way mirror

d. established patient of the resident

Post-Test Module 1

2. Which of the following is acceptable as part of a clinical skills verification examination?

c. one-way mirror

The patient must be unknown to the resident. One-way mirrors are acceptable. Videotaped interviews or standardized patients are not.

Post-Test Module 1

3. The in-residency clinical skills verification exam provides an opportunity to:

a. insure direct observation of resident interviews in training

b. provide early assessment of clinical skills with time for remediation

c. attempt to develop standards of assessment across residency training programs

d. all of the above

Post-Test Module 1

3. The in-residency clinical skills verification exam provides an opportunity to:

D. all of the above

Assurance of direct observation of resident interview skills, early recognition of potential deficiencies with time for remediation, and the opportunity to develop standards of assessment across residency training programs are all possible outcomes