Evaluating Surgical Skills And Operating Room Performance: Education/Remediation?...
-
Upload
noah-stewart -
Category
Documents
-
view
213 -
download
0
Transcript of Evaluating Surgical Skills And Operating Room Performance: Education/Remediation?...
![Page 1: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/1.jpg)
Evaluating Surgical SkillsAnd Operating Room
Performance:Education/Remediation?
Certification/Credentialling?John J. Ferrara MD
Kanav Kahol PhDPhoenix Integrated Surgical Residency
![Page 2: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/2.jpg)
Evaluating Surgical SkillsChallenges
How to maintain cardinal surgical “art and science” traditions when the sands that support educational paradigms are shifting? “Publish or perish” to “Produce (RVU’s) or perish” “Duty” hours “Public” opinion
Generational chasm “Linear” educational construct
![Page 3: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/3.jpg)
Generation X: The BridgeBoomers (46-64 years) Defined by work ethic Independent Religious Financial success Career-driven Wanna be lead dogs Kumbaya Consumer-driven/TV
Millennials (18-29 years) Defined by technology Social agenda Secular Parenthood (non-traditional) Time-driven Lead only if asked Blunt Under-consumers/TV?
The good news: they respect (boomerang back to) their elders
![Page 4: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/4.jpg)
Technical Skills EvaluationLinear Construct
![Page 5: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/5.jpg)
Technology
Simulation Environment
“Real” Environment
Technical Skills EvaluationParallel Construct
LeveragedLeveraged
ScalableScalable AdaptableAdaptable
IntegrativeIntegrative
![Page 6: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/6.jpg)
Goals Measure Technical Skills in a Simulated
Environment Create a system to measure skill set and to provide
immediate feedback to the user “Battleship down”
Measure Technical Skills in the Operating Room Develop and validate a system to analyze videos of
operations submitted to a panel for assessment
![Page 7: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/7.jpg)
![Page 8: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/8.jpg)
Objective Proficiency Measures Employ neurological and
kinesiological features to analyze task (surgical) proficiency
Construct task decomposition based feedback system Breaks complex motion
into simpler units that are: Easy to analyze Easy to comprehend Easy to modify by the user
Expert
Intermediate
Novice
Instrumentmovements
Rosen 2002
![Page 9: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/9.jpg)
Hand Motion
![Page 10: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/10.jpg)
Motorical ChunkingMeasure of Expertise
Expert Novice
![Page 11: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/11.jpg)
Dynamic Virtual Reality Systems for Cognitive Training
Train residents for attention, working memory, intermodal transfer
Modify technique simulators to include a cognitive layer
Treat surgery as a combination of psychomotor and cognitive skill Original Task (Laparoscopic Training)
Modified to target working memory
![Page 12: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/12.jpg)
Marble Mania
High (0.92) correlation with basic surgical gestures
Fine motor skills based game
Hand motions similar to laparoscopy
![Page 13: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/13.jpg)
Marble Mania
![Page 14: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/14.jpg)
CyberGlove Analysis
Non-Dominant Hand Dominant Hand Marble Mania
![Page 15: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/15.jpg)
Ambidexterity
![Page 16: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/16.jpg)
Technical Proficiency on ProMIS
5.9
3.9
6.5
2.5
6.3
1.0
2.9
![Page 17: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/17.jpg)
Novices
Intermediates
ExpertsMasters
Skills Evaluation
![Page 18: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/18.jpg)
Measuring Skills in the “Real” EnvironmentProposed Solution
Computer vision instrument automatically analyzes videos Develop means/ranges/standard deviations Set “minimal” performance grade Benchmarking?
Picks up events the naked eye misses Detailed movement analysis Cheap, “portable”, time-efficient
Web based access to rate videos for experts Web based training tools to train experts to rate videos
![Page 19: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/19.jpg)
Video CaptureLaparoscopy
• Basic apparatus• Video capture system for laparoscopic system
and hand movements• Hand movements captured by external camera• Sites: ceiling/lighting system/tripod
• De-identified videos• Our system “syncs” these two streams for
presentation and analysis
![Page 20: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/20.jpg)
Dual Capture System
![Page 21: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/21.jpg)
Skills Evaluation
Expert Intermediate Novice
Tremor
Inst
rum
ent P
ath
Ineffi
cien
cy
Between Groupsp<0.05
Between GroupsP<0.05
Expert v NoviceP<0.05
![Page 22: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/22.jpg)
Benchmarking?
![Page 23: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/23.jpg)
Web-Based Training Upload/automatically
analyze videos on www.ratethesurgeons.com
Experts view videos off-site Can provide input/feedback
Novice raters• View expert ratings • Receive instruction to
become proficient raters• Reward system: pair teaching
![Page 24: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/24.jpg)
Correlation of Subjective Measures with Various Objective Measures
1.0
0.4
![Page 25: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/25.jpg)
ValidationR=0.93 p<0.05
Experts
Intermediates
Novices
![Page 26: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/26.jpg)
Where We are Now
Validation of the technical analysis tool
Evaluation on simulators also being done with videos
![Page 27: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/27.jpg)
Future Work
Enhance Database Develop Benchmarks Expand Skill Set Instrument Family Patient Care Applications
![Page 28: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/28.jpg)
New Simulation Tasks
![Page 29: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/29.jpg)
Motion History Images
![Page 30: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/30.jpg)
Virtual World “Acute Care Surgery” Training
![Page 31: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/31.jpg)
Challenges“The Uncanny Valley”
Masahiro Mori (1970)
Avatar
![Page 32: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/32.jpg)
ChallengesThe Simulation Perfect Storm
Conventional computing is dead, and with it, the first generation (six figure) simulators
Computing life measured in months• Core processors• Naturalistic computing• Gaming consoles
How to maintain a database when evaluation instruments are constantly changing?
![Page 33: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/33.jpg)
Conclusions
We (all) need help We have no magic bullet We need genomic variation
“The Two Word Definition of Dogma is Brain Dead”Zollinger (sometime during my residency)
![Page 34: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/34.jpg)
![Page 35: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/35.jpg)
ChallengesEngineers
![Page 36: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/36.jpg)
Clinicians
![Page 37: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/37.jpg)
Video Capture• Basic apparatus• Video capture system for
laparoscopic system and hand movements• Hand movements
captured by external camera
• Sites: ceiling/lighting system/tripod
• De-identified videos• Our system “syncs” these
two streams for presentation and analysis
Mobile simulator unit
![Page 38: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/38.jpg)
We are becoming increasingly challenged with teaching new dogs old tricks
AND
We are not very good at teaching old dogs new tricks
Evaluation Poses a More Daunting Challenge
![Page 39: Evaluating Surgical Skills And Operating Room Performance: Education/Remediation? Certification/Credentialling? John J. Ferrara MD Kanav Kahol PhD Phoenix.](https://reader033.fdocuments.us/reader033/viewer/2022051820/56649e3f5503460f94b3062e/html5/thumbnails/39.jpg)
Analysis
• Basic movement tracking algorithms from computer vision, an established field with myriad algorithms to track movements and predict efficacy
• Proprietary state of the art tools analyze movements