Web-Based Portfolios: A Powerful New Tool for Surgical Education
description
Transcript of Web-Based Portfolios: A Powerful New Tool for Surgical Education
Web-Based Portfolios:Web-Based Portfolios: A Powerful New Tool for Surgical EducationA Powerful New Tool for Surgical Education
Catherine E. Lewis MD, Areti Tillou MD,Catherine E. Lewis MD, Areti Tillou MD,
Michael W. Yeh MD, Chi Quach, Michael W. Yeh MD, Chi Quach,
Jonathan R. Hiatt MD, O. Joe Hines MDJonathan R. Hiatt MD, O. Joe Hines MD
Department of Surgery Department of Surgery David Geffen School of Medicine at UCLADavid Geffen School of Medicine at UCLA
Professional CompetenceProfessional Competence
• Competency-based model for GME - 1998 Competency-based model for GME - 1998
• ““Habit of lifelong learning” Habit of lifelong learning” – – LeachLeach
• ““Reflection in daily practice for benefit of Reflection in daily practice for benefit of individual and community”individual and community”
– – Epstein & HundertEpstein & Hundert
• Surgical educators challenged to Surgical educators challenged to teach, assess and document competenceteach, assess and document competence
Portfolios:Portfolios:Multifunctional tool for competenceMultifunctional tool for competence• Documentation:Documentation:• Collection of resident’s workCollection of resident’s work
• Demonstrate learning, achievement, deficiencyDemonstrate learning, achievement, deficiency
• Assessment:Assessment:• Reliable and validReliable and valid
• Requires well-defined criteria & structureRequires well-defined criteria & structure
• Teaching and development:Teaching and development:• Guided by mentorsGuided by mentors
• Promotes self-assessment and growthPromotes self-assessment and growth
Portfolio ModelsPortfolio Models
ModelModel ContentContent Benefits & UsesBenefits & Uses
LearningLearning (Growth)(Growth)
FlexibleFlexible Self-assessmentSelf-assessmentMentoringMentoring Lifelong learningLifelong learning
Assessment Assessment (Showcase)(Showcase)
StructuredStructured Reliable & valid assessment Reliable & valid assessment Professional certificationProfessional certification
HybridHybrid BothBoth Use for both formative &Use for both formative & summative assessmentsummative assessment facilitated by contentfacilitated by content access privilegesaccess privileges
Specific AimsSpecific Aims
• Describe UCLA portfolio systemDescribe UCLA portfolio system
• Analyze effects of the portfolio systemAnalyze effects of the portfolio system
• Survey of residentsSurvey of residents
• Evaluations by facultyEvaluations by faculty
Portfolios in Surgical EducationPortfolios in Surgical Education
UCLA Portfolio SystemUCLA Portfolio System
• Learning modelLearning model
• Web-based, password-protectedWeb-based, password-protected
• 2 GB / user2 GB / user
• Developed by expert groupDeveloped by expert group
• Development cost $25,000Development cost $25,000
• Launched May 2007Launched May 2007
Portfolios in Surgical EducationPortfolios in Surgical Education
ContentContent
Confidential Personal Public
Letters of rec Presentations Presentations
Incid. reports Publications Readings
Discipline CV Goals / objs
Eval reports
Licensing
Portfolios in Surgical EducationPortfolios in Surgical Education
ContentContent
Confidential Personal Public
Letters of rec Presentations Presentations
Incid. reports Publications Readings
Discipline CV Goals / objs
Eval reports
Licensing
Portfolios in Surgical EducationPortfolios in Surgical Education
ContentContent
Confidential Personal Public
Letters of rec Presentations Presentations
Incid. reports Publications Readings
Discipline CV Goals / objs
Eval reports
Licensing
Portfolios in Surgical EducationPortfolios in Surgical Education
ContentContent
Confidential Personal Public
Letters of rec Presentations Presentations
Incid. reports Publications Readings
Discipline CV Goals / objs
Eval reports
Licensing
Portfolios in Surgical EducationPortfolios in Surgical Education
Portfolios in Surgical EducationPortfolios in Surgical Education
Portfolios in Surgical EducationPortfolios in Surgical Education
Portfolios in Surgical EducationPortfolios in Surgical Education
Portfolios in Surgical EducationPortfolios in Surgical Education
Portfolios in Surgical EducationPortfolios in Surgical Education
Surgery Education PortfolioSurgery Education Portfolio
Portfolios in Surgical EducationPortfolios in Surgical Education
Resident Survey DesignResident Survey Design• ComprehensiveComprehensive
• VoluntaryVoluntary
• MeasuresMeasures• Understanding of systemUnderstanding of system
• AcceptanceAcceptance
• UseUse
• Perceived effect on competenciesPerceived effect on competencies
• Response rate 69%Response rate 69%
Portfolios in Surgical EducationPortfolios in Surgical Education
Resident Survey ResultsResident Survey Results
Portfolios in Surgical EducationPortfolios in Surgical Education
% Agree% Agree
Appreciate organizational functionAppreciate organizational function 100100
Purpose / functions explainedPurpose / functions explained 9595
View as important education toolView as important education tool 8787
Understand uses of systemUnderstand uses of system 8282
Aided development of Aided development of competencecompetence
4545
Faculty EvaluationsFaculty Evaluations
• Comparison pre- and post-portfoliosComparison pre- and post-portfolios• Global assessmentsGlobal assessments
• End-of-rotation End-of-rotation
• Competency basedCompetency based
• Capture, n:Capture, n:• Before implementation = 1488Before implementation = 1488
• After = 697After = 697
Portfolios in Surgical EducationPortfolios in Surgical Education
Improvement in All CompetenciesImprovement in All Competencies
Portfolios in Surgical EducationPortfolios in Surgical Education
3.7
3.8
3.9
4
4.1
4.2
4.3
4.4
4.5
Medical Medical KnowledgeKnowledge
Patient Patient CareCare
ProfessionalismProfessionalism Inter-Inter-personal personal SkillsSkills
Practice Practice Based Based LearningLearning
Systems Systems Based Based PracticePractice
PrePre
PostPost
Patient CarePatient Care
Portfolios in Surgical EducationPortfolios in Surgical Education
Pre Post p
Obtains complete H & P 4.18 4.24 0.04
Generates appropriate ddx 4.10 4.14 1.16
Develops appropriate rx plan 4.08 4.15 0.02
Provides appropriate patient care 4.24 4.25 0.38
Adequate operative prep of pts 4.18 4.23 0.07
Medical KnowledgeMedical Knowledge
Portfolios in Surgical EducationPortfolios in Surgical Education
Pre Post p
Fund of basic & clinical knowledge 4.07 4.12 0.07
Engages in ongoing learning 4.00 4.05 0.10
Prepares for operation 4.07 4.12 0.09
Understands risks and benefits 4.11 4.19 0.01
ProfessionalismProfessionalism
Portfolios in Surgical EducationPortfolios in Surgical Education
Pre Post p
Follows through on tasks 4.34 4.32 0.71
Demonstrates integrity 4.44 4.45 0.38
Well groomed, appropriate attire 4.43 4.44 0.38
Available to patients/family/faculty 4.37 4.39 0.28
Communicates w/ attending effectively 4.33 4.37 0.13
Knows limitations 4.25 4.31 0.05
Portfolios in Surgical EducationPortfolios in Surgical Education
Pre Post p
Demonstrates care & concern 4.35 4.38 0.18
Communicates effectively:
With patients 4.31 4.35 0.13
With other healthcare providers 4.27 4.32 0.08
Maintains medical record 4.28 4.33 0.07
Interpersonal and Communication SkillsInterpersonal and Communication Skills
Systems Based PracticeSystems Based Practice
Portfolios in Surgical EducationPortfolios in Surgical Education
Pre Post p
Provides cost-conscious care 4.14 4.17 0.21
Effectively uses system resources 4.20 4.23 0.20
Portfolios in Surgical EducationPortfolios in Surgical Education
Reflect Reflect criticallycritically
Aptitude Aptitude for changefor change
Teach Teach othersothers
Overall Overall PBLIPBLI
Practice Based Learning & ImprovementPractice Based Learning & Improvement
Portfolio Learning HelixPortfolio Learning Helix
Portfolios in Surgical EducationPortfolios in Surgical Education
Experiences, Experiences, achievements and achievements and
deficiencies recorded deficiencies recorded in portfolioin portfolio
Resident reflectsResident reflectsin order toin order to
learn and drawlearn and drawmeaningmeaning
New knowledgeNew knowledgeis tested on newis tested on new
experiences whichexperiences whichthen are recordedthen are recorded
PBLIPBLIMentoringMentoring
Keys to Portfolio SuccessKeys to Portfolio Success
• MentorshipMentorship
• Support and acceptance by usersSupport and acceptance by users
• Minimal time commitmentMinimal time commitment
Portfolios in Surgical EducationPortfolios in Surgical Education
Limitations of Present StudyLimitations of Present Study
• Early analysis after implementationEarly analysis after implementation
• Other improvements in residencyOther improvements in residency
Portfolios in Surgical EducationPortfolios in Surgical Education
ConclusionsConclusions
• Portfolios can aid in development Portfolios can aid in development and documentation of competenceand documentation of competence
• Favorable response by residentsFavorable response by residents
• Positive effect for all competenciesPositive effect for all competencies
• Valuable organizational toolValuable organizational tool
• Integral to lifelong learning andIntegral to lifelong learning and professional developmentprofessional development
Portfolios in Surgical EducationPortfolios in Surgical Education