Physical Examination Education in Graduate Medical Education – A Systematic Review of the...

Physical Examination Education in Graduate Medical Education – A Systematic Review of the Literature

• Somnath Mookherjee, MD• Lara Pheatt, MA• Sumant R. Ranji, MD• Calvin Chou, MD, PhD• SDRME Summer Meeting 2012

Importance of PE skills?

• CXR from “The Practioner, 1904”• "Laennec examines a consumptive patient with a stethoscope in front of his students at the Necker Hospital". Painting by Théobald Chartran.• Physical diagnosis: a guide to methods of clinical investigation. George Alexander Gibson, William Russell. D. Appleton & Co., 1891. New York.• A pocket book of physical diagnosis: for the student and physician. Edward Tunis Bruen. P. Blakiston, 1881. Philadelphia

Background Methods Results Comment Discussion

• Fletcher RH, Fletcher SW. Has medicine outgrown physical diagnosis? Ann Intern Med. 1992;117(9):786-7.

• Flegel KM. Does the physical examination have a future? CMAJ. 1999;161(9):1117-8.

• Jauhar S. The demise of the physical exam. N Engl J Med. 2006;354(6):548-51.

(Un)importance of PE skills?

ACGME Internal Medicine program requirements


Resurgence of PE

Physical examination is still importantHypothesis basedEvidence based

• Verghese, A., Culture shock--patient as icon, icon as patient. N Engl J Med, 2008. 359(26): p. 2748-51.• Verghese, A., A touch of sense. Health Aff (Millwood), 2009. 28(4): p. 1177-82.• Yudkowsky, R., et al., Residents anticipating, eliciting and interpreting physical findings. Med Educ, 2006. 40(11): p. 1141-2.• Yudkowsky, R., et al., A hypothesis-driven physical examination learning and assessment procedure for medical students: initial validity

evidence. Med Educ, 2009. 43(8): p. 729-40.• McGee, S., Evidence Based Physical Diagnosis. Second Edition ed2007, St. Louis: Saunders - Elsevier.• David L. Simel and D. Rennie, The Rational Clinical Examination: Evidence Based Clinical Diagnosis. 2009, New York: McGraw-Hill

Professional.


But skills are inadequateTrainee Type of Exam ? References

Pediatrics Ausculation Gaskin et al. Pediatrics, 2000Dhuper et al. Clinical Pediatrics, 2007.

Emergency medicine Cardiac exam Jones et al. Academic Emergency Medicine. 1997.

Emergency medicine General PE Mangione et al. Academic Emergency Medicine. 1995.

Internal medicine General PE Li et al. Academic Medicine. 1994Johnson et al. Archives of Internal Medicine. 1986Ramani et al. JGME. 2010.

Internal Medicine Rectal Wilt et al. JGIM. 1991.

Internal medicine and Family medicine Cardiac Vukanovic-Criley et al. Archives of Internal Medicine. 2006.Mangione et al. JAMA. 1997.Mangione et al. AJM. 2001. Vukanovic-Criley et al. Clinical Cardiology. 2010.St Clair et al. Annals of Internal Medicine. 1992.

Internal medicine and Family medicine Pulmonary Mangione et al. American journal of respiratory and critical care medicine. 1999.

Internal medicine and Family medicine MSK Meenan et al. The Journal of rheumatology. 1988.

Internal medicine and Family medicine Breast Exam Chalabian et al. Annals of surgical oncology. 1998.

Internal medicine and OB / GYN Pelvis and breast Dugoff et al. American journal of obstetrics and gynecology. 2003.Heiligman et al. JAMA. 1998.

Family medicine Pelvic Lang et al. F. Family medicine. 1990.

Surgery Vascular Endean et al. Journal of vascular surgery. 1994.

↓

↓↓

↓

↓↓

↓

↓↓

↓

↓↓


Why focus on GME?

Different than students◦Little “protected time” for didactics◦Constant time pressure◦Mainly experiential learning◦Skills needed for imminent practice

New program requirements


Developmental Milestones for Internal Medicine Training – Patient Care

Green ML, Aagaard EM, Caverzagie KJ, Chick DA, Holmboe E, Kane G, et al. Charting the road to competence: developmental milestones for internal medicine residency training. Journal of graduate medical education. 2009;1(1):5-20.


Research Questions

What teaching methods are used?What assessment methods are

used?What teaching methods are

effective?

GME Program Curricula Individual practices of teachers


Search strategySearch 1 = "physical examination"[MESH] AND

"Education, Medical"[MESH]Search 2 = physical examination AND "Education,

Medical"[MESH]Search 3 = physical examination AND (resident OR intern

OR graduate OR residents OR interns OR graduates) AND (teaching OR learning OR education OR teach OR learn)

Search 4 = "physical examination"[MESH] AND (resident OR intern OR graduate OR residents OR interns OR graduates) AND (teaching OR learning OR education OR teach OR learn)

Personal files


Eligibility1. English language2. Inclusion of a description of the study population;

number of participants and level of training3. Description of an educational intervention

intention of improving PE skills4. Inclusion of assessment of efficacy5. Inclusion of a clear comparison group6. Report of data analysis (descriptions of outcomes

without statistical analysis were not included)7. Study subjects enrolled in GME


Data Extraction

Study Quality◦ Medical Education Research Study Quality Instrument (MERSQI)

Based on Best Evidence Medical Education Collaboration protocol◦ Nation◦ Type of physical examination◦ Level and numbers of learners◦ Summary of intervention

Human examinees Deliberate practice

◦ Summary of outcomes Benefit to learner Assessment methods


Deliberate Practice?

1. Repetitive performance of skills by the learner.

2. Assessment of skills by the teacher.

3. Specific feedback to the learner by the teacher.

4. Observation of improved performance in a controlled setting.

• Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic medicine : journal of the Association of American Medical Colleges. 2004;79(10 Suppl):S70-81.

• McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Medical education featuring mastery learning with deliberate practice can lead to better health for individuals and populations. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(11):e8-9.

• McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(6):706-11.

• Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, Van der Vleuten CP, Scherpbier AJ. The role of deliberate practice in the acquisition of clinical skills. BMC medical education. 2011;11(1):101.

• Papers independently scored for the presence of these elements • 0 = not reported• 1 = reported

• Global deliberate practice score• 0 = no use of deliberate practice or

unable to determine• 1 = possible use of deliberate

practice• 2 = definite use of deliberate practice


Educational Outcomes

Kirkpatrick Level◦ Level 0 = no assessment of impact◦ Level 1 = assessment of reaction to the intervention◦ Level 2a = assessment of attitudes or perceptions◦ Level 2b = assessment of knowledge or skills◦ Level 3 = assessment of changes in behavior

Classification◦ “X” = not measured, or not compared to a control group◦ “0” = not better than control group◦ “1” = beneficial (intervention group with significantly better

outcome than control)


Statistics and Analysis

Inter-rater reliability ◦ Individual elements of the MERSQI scores◦ Individual elements of deliberate practice◦Global deliberate practice score

Study quality◦Average and median MERSQI scores with

standard deviations using consensus scoresNarrative synthesis◦Group review of tabulated summaries of

studiesBackground Methods Results Comment Discussion

Search and Selection of Articles


7250 Articles Identified and Screened 1543 "physical examination"[MESH] AND "Education, Medical"[MESH] 1943 physical examination AND "Education, Medical"[MESH] 2038 physical examination AND (resident OR intern OR graduate OR residents OR interns OR graduates)

AND (teaching OR learning OR education OR teach OR learn) 1695 "physical examination"[MESH] AND (resident OR intern OR graduate OR residents OR interns OR

graduates) AND (teaching OR learning OR education OR teach OR learn) 31 Hand search and expert review

155 Citations Meeting Initial Screening Criteria

7095 Rejected Based on Initial Screening Criteria (is the study about PE education in GME?)

141 Citations Removed 109 Duplicates 32 Not Meeting Full Inclusion Criteria

14 Articles Included in Systematic Review

Study characteristics

Characteristic, N=14 # Studies

Nation USA 13 Denmark 1Type of examination Cardiac 6 Thyroid 1 Pelvic 3 Breast 1 General 1 Musculoskeletal 2Randomized 4

Characteristic # Studies

GME (a) Pediatrics 3 Internal Medicine 8 Family Medicine 2 Danish “house officer” 1Outcomes assessed in comparison to controls (a, b) Kirkpatrick level 1 3 Kirkpatrick level 2a 4 Kirkpatrick level 2b 12 Kirkpatrick level 3 2(a) not mutually exclusive(b) comparison to control not applicable for level 1 outcomes


Study Quality - MERSQI Score

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18


• Perfect inter-rater agreement [kappa = 1.0 (95% CI = 1.0, 1.0)] for all but two items. o “Sampling” [kappa =

0.44 (95% CI = -0.16, 1.0)]

o “Content validity” [kappa = 0 (95% CI = -0.52, 0.52)].

Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002-1009.

Interventions


14 Studies• Criley 2008: Web-based cardiac tutorial• Horiszny 2001: Multimedia cardiac

lecture• Iversen 2006: Multimedia cardiac lecture• Mangione 1994: PE elective, lectures• Keren 2005: Brief cardiac audio tutorial• Oddone 1993: Cardiology simulator• Smith 2005: MSK small group session

Teaching associates

• Leder 2005: Pelvic exam during clinical rotation

• Smith 2006: Cardiac exam, bedside rounds

• Freund 1998: Breast exam, breast care clinic

• Rabinovitz 1987: Pelvic exam, adolescent medicine rotation

Interaction w/ patients in a clinical context

Resident interaction w/ a human examinee, n = 7

No resident interaction w/ human examinee, n = 7

• Branch 1999: Arthritis patient educator.

• Herbers 2003: Pelvic exam patient educator

Patient volunteers

• Houck 2002: Thyroid exam on volunteer patient in workshop

Single session

Assessments


14 Studies

• Criley 2008: Computer program to assess cardiac exam

• Horiszny 2002: Recorded heart sounds• Keren 2005: Recorded heart sounds

Teaching associates

• Branch 1999: MSK exam, patient educators

• Herbers 2003: Pelvic exam, patient educators

Objective Structured Clinical Examinations

Resident interaction w/ a human examinee, n = 8

No interaction w/ human examinee,

n = 6

• Houck 2002: Thyroid exam OSCE

• Mangione 1994: Multiple PE type OSCEs

• Smith 2006: Cardiac exam OSCE

• Smith 2005: MSK exam OSCE

Patient Volunteers

• Oddone 1993: Cardiac exam testing, also used simulator

• Iversen 2006: Cardiac exam testing, patient volunteers

• Freund 1998: Chart review for frequency of breast exam

• Leder 2005: Chart review for genital exam

• Rabinovitz 1987: Survey for pelvic exam confidenceSurvey

Chart review

Multi-media

Previously published assessment tool

OutcomesWhat teaching methods are used?What assessment methods are used?What teaching methods are effective?

Effective interventions

Ineffective interventions

Kirkpatrick Level

• Summary • Summary


Outcomes – KL 3Effective interventions Ineffective interventions

Outcomes – Level 3: Behavior

Freund 1998. Breast. Weekly session at Breast Health Center with observed examination of patients. Improved documentation of clinical breast exam on chart review. (13)

Leder 2005. Pelvic. Precepted exams in clinic. No difference in completeness of documentation in chart review of suspected sexual abuse cases. (4)


1. No evident pattern for the superiority of one educational setting over anothero Bedside teaching, simulator, lecture,

workshop

Effective interventions Ineffective interventionsOutcomes – Level 2b: Knowledge / Skills

• Houck 2002. Thyroid. Workshop using patient volunteer. Better scores in OSCE describing findings. (2)

• Herbers 2003. Pelvic. Workshop with gynecological teaching associates. Improvement in observed technique and communication. (3)

• Smith 2006. Cardiac. Two types of bedside teaching rounds. Intervention groups had better technique, and one of the intervention groups had mildly better key findings than control. (5)

• Branch 1999. MSK. Patients with arthritis trained as educators. Better with checklist assessment of examination. (8)

• Criley 2008. Cardiac. Web-based multimedia tutorial with human support. Better performance in cardiac exam testing. (1)

• Horiszny 2001. Cardiac. Multi-media lecture. Better at identifying heart sounds. (12)

• Smith 2005. MSK. Small group case-based sessions. Better on OSCE PE checklists and diagnosis. (11)

• Keren 2005. Cardiac. Three minute teaching session without any practice or feedback. No improvement in accuracy. (9)

• Iversen 2006. Cardiac. Workshop with recorded heart sounds +/- advanced stethoscope. No improvement in accuracy in diagnosing abnormal heart murmurs. (10)

• Houck 2002. Thyroid. Workshop using patient volunteer. No difference in OSCE in observed technique. (2)

• Leder 2005. Pelvic. Precepted exams in clinic. No difference in knowledge score between intervention and control. (4)

• Oddone 1999. Cardiac. High-fidelity simulator mannequin (Harvey). No better at detecting findings or making the diagnosis. (6)

• Mangione 1994. General. Lecture series. No difference in PE technique or knowledge scores. (7)

Outcomes – KL 2b


2. No evident pattern that spending more time results in better outcomes o Elective rotation, multiple lecture series,

single workshop, single lecture

Effective interventions Ineffective interventionsOutcomes – Level 2b: Knowledge / Skills

• Houck 2002. Thyroid. Workshop using patient volunteer. Better scores in OSCE describing findings. (2)

• Herbers 2003. Pelvic. Workshop with gynecological teaching associates. Improvement in observed technique and communication. (3)

• Smith 2006. Cardiac. Two types of bedside teaching rounds. Intervention groups had better technique, and one of the intervention groups had mildly better key findings than control. (5)

• Branch 1999. MSK. Patients with arthritis trained as educators. Better with checklist assessment of examination technique. (8)

• Criley 2008. Cardiac. Web-based multimedia tutorial with human support. Better performance in cardiac exam testing. (1)

• Horiszny 2001. Cardiac. Multi-media lecture. Better at identifying heart sounds. (12)

• Smith 2005. MSK. Small group case-based sessions. Better on OSCE PE checklists and diagnosis. (11)

• Keren 2005. Cardiac. Three minute teaching session without any practice or feedback. No improvement in accuracy. (9)

• Iversen 2006. Cardiac. Workshop with recorded heart sounds +/- advanced stethoscope. No improvement in accuracy in diagnosing abnormal heart murmurs. (10)

• Houck 2002. Thyroid. Workshop using patient volunteer. No difference in OSCE in observed technique. (2)

• Leder 2005. Pelvic. Precepted exams in clinic. No difference in knowledge score between intervention and control. (4)

• Oddone 1999. Cardiac. High-fidelity simulator mannequin (Harvey). No better at detecting findings or making the diagnosis. (6)

• Mangione 1994. General. Lecture series. No difference in PE technique or knowledge scores. (7)

Outcomes – KL 2b


3. Little clarity or consistency in what PE competence entails and how to measure ito Technique vs. accuracyo Detection vs. diagnosis

Deliberate Practice and Outcome


Categorization of Studies by Use of Deliberate Outcome, Use of Learner Interaction with Human Examinees and Efficacy of Educational Intervention

Intervention group with better educational outcomes than control at highest Kirkpatrick level assessed

Intervention group and control group with same educational outcomes at highest Kirkpatrick level assessed

Possible or definite use of deliberate practice (Global DP score = 1 or 2)

Branch 1999 (54)Criley 2008 (55)Freund 1998 (56)*Houck 2002 (59)*Rabinovitz 1987 (64)Smith 2006 (65)*

Leder 2005 (61)

No use of deliberate practice or unable to determine (Global DP score = 0)

Herbers 2003 (57)Horiszny 2001 (58)Smith 2005 (66)

Iversen 2006 (67)Keren 2005 (60)Mangione 1994 (62)Oddone 1993 (63)

* Studies scored as “definite use of deliberate practice”Studies that used learner interaction with human examinees as part of the educational intervention in bold

• Inter-rater reliability of components of DP score

• Repetitive performance of skills by the learnero [kappa=0.86 (95% CI=0.59,

1.0)]• Assessment of skills by the

teachero [kappa=0.72 (95% CI=0.38,

1.0)]• Specific feedback to the learner

by the teachero [kappa=0.71 (95% CI=0.33,

1.0)]• Observation of improved

performance in a controlled setting o [kappa=0.81 (95% CI=0.46,

1.0)]• Global deliberate practice score

o [kappa=0.76 (95% CI=0.46, 1.0)]

Limitations

1. Deliberate practice assessment non-validated

2. Effect sizes neither calculated nor meta-analyzed

3. Small number of studies with heterogeneous outcome measurements


Findings

1. No convincing evidence for one setting: “going to the bed-side” vs. “going to the simulator-side”

2. Not just the “time spent,” but the “time well spent”

Effective Ineffective+ DP Branch 1999 (54)

Criley 2008 (55)Freund 1998 (56)*Houck 2002 (59)*Rabinovitz 1987 (64)Smith 2006 (65)*

Leder 2005 (61)

- DP Herbers 2003 (57)Horiszny 2001 (58)Smith 2005 (66)



Findings

3. Deliberate practice is well-suited to teach PE in GME

4. Interaction with human examinees may be beneficial

Effective Ineffective+ DP Branch 1999 (54)

Criley 2008 (55)Freund 1998 (56)*Houck 2002 (59)*Rabinovitz 1987 (64)Smith 2006 (65)*

Leder 2005 (61)

- DP Herbers 2003 (57)Horiszny 2001 (58)Smith 2005 (66)



Recommendations

1. UME PE education systematic review underway

2. Evaluate the use of deliberate practice and human examinees to teach PE

3. Develop a GME PE blueprintA. Specific PE skills graduating residents should

haveB. What competence in these skills entailsC. How these skills are best taught and evaluated


Acknowledgements

Society of Directors of Research in Medical Education

UCSF Division Of Hospital Medicine

UCSF – OME – Teaching Scholars Program

Background Methods Results Comment DiscussionDiscussion

Outcomes – KL 1 Effective interventions Ineffective interventions

Outcomes – Level 1: Participation / Reaction

• Houck 2002. Thyroid. Workshop using patient volunteer. Residents felt it was helpful. (2)

• Mangione 1994. General. Lecture series. Lectures helpful. (7)

• Smith 2005. MSK. Small group sessions. Participants liked it and thought it was useful. (11)

Outcomes – KL 2a Effective interventions Ineffective interventions

Outcomes – Level 2a: Attitudes / Perceptions

• Leder 2005. Pelvic. Precepted exams in clinic. Intervention group had higher confidence and comfort. (4)

• Rabinovitz 1987. Pelvic. Adolescent medicine rotation. Higher confidence in pelvic exam. (26)

• Leder 2005. Pelvic. Precepted exams in clinic. No difference in self assessed competence. (4)

• Criley 2008. Cardiac. Web-based multimedia tutorial with human support. No difference in confidence. (1)


Physical Examination Education in Graduate Medical Education – A Systematic Review of the...

Documents

Transcript of Physical Examination Education in Graduate Medical Education – A Systematic Review of the...