Post on 18-Jan-2017
glass
A U G M E N T E D A N D V I RT U A L R E A L I T Y I N P L A S T I C S U R G E RY
Jonathan Kanevsky, MDMirko Gilardino, MDSamuel Lin, MD
O U T L I N E
DEVICE DEMO
REALITY SPECTRUM AR VR HARDWAREAR VR APPLICATIONS
Franz Robaud – 1812-Battle of Borodino
1838: Charles Wheatstone
Motronheilig.com
Motronheilig.com
S P E C T RU M O F R E A L I T Y
Motronheilig.com
W H AT I S M I XE D R E A L I T Y ?
MR devices combine input from multiple sensors in a wearable platform
An on-board computer integrates this information between the device, the Internet, and the outside world
The processed data is then relayed to the user through a visual display
W H AT I S V I RT U A L R E A L I T Y ?
W H AT I S V I RT U A L R E A L I T Y ?
W H AT I S A U G M E N TE D R E A L I T Y ?
• Spatial AR• Hand Held• Virtual Retina Display• Contact Lens• Head Mounted Display
W H AT I S A U G M E N TE D R E A L I T Y ? Spatial Augmented Reality
W H AT I S A U G M E N TE D R E A L I T Y ? Hand Held AR
W H AT I S A U G M E N TE D R E A L I T Y ? Contact lens AR
http://www.extremetech.com/computing/126043-us-military-developing-multi-focus-augmented-reality-contact-lenses
W H AT I S A U G M E N TE D R E A L I T Y ? Virtual Retina Display
http://www.extremetech.com/computing/126043-us-military-developing-multi-focus-augmented-reality-contact-lenses
W H AT I S A U G M E N TE D R E A L I T Y ?
www.dailymail.co.ukwww.dailymail.co.uk
Head Mounted Display
Google glass
Meta
H O LO L E N SMI C R OSOFT
H O LO L E N SMI C R OSOFT
A PPL I C AT I O N S I N PL A ST I C S U R G E RY
Voice activated head-mounted camera optimal for hands free recording form surgeon POV
Generate teaching libraries for plastic surgery eduction or record resident technique
S U R G I C A L E D U C AT I O N :
www.ucsf.eduGaster and Lin, 2014
PL A S T I C SU R G E RY A P PL I C AT I ON S
Augmented reality display for:
- Diagrams - Anatomical
markings- -AI applications
S U R G I C A L E D U C AT I O N :
Gaster , Lin, Kanevsky, Corban 2015
S U R G I C A L E D U CAT I O N :
Ma M, Fallavollita P, Seelbach I, et al. Personalized augmented reality for anatomy education. Clinical anatomy (New York, N.Y.).
2016;29(4):446-453
S U R G I C A L E D U CAT I O N :
S U R G I C A L E D U C AT I O N :
S U R G I C A L E D U C AT I O N :
Transmit visual information for diagnosis and surgical planning
Trainees can call a senior surgeon to discuss a diagnosis
Virtual assistance during procedures with other surgeons in HUD
T EL E M E D I C I N E :
internetmedicine.com
P L A S T I C S U R G E RY A P P L I C AT I O N S
http://1.bp.blogspot.com/-mVbtvRKhCJQ/UpKIR3hlw-I/AAAAAAAAHgk/sNRLYQ0n4SI/s1600/VIPAAR_Google_Glass_Virtual_Surgeon.jpg
TE L E M E D I C I N E :
DATA OV E R L AY A ND D O CU M E N TAT I O N :
internetmedicine.com
Accessing patient information: CT scans, MRIs, X-rays, patient notes, labs or vital signs all while remaining sterile in the operating room.
Generating new notes: OR dictations with embedded video during surgery
Peri-operative utility: Pre-operative markings, Post-operative photos
P L A S T I C S U R G E RY A P P L I C AT I O N S
C ON S U LTAT I O N :
http://www.illusioimaging.com/
P L A S T I C S U R G E RY A P P L I C AT I O N S
P E R I O - O P E RAT I V E :
Mezzana P, Scarinci F, Marabottini N. Augmented reality in oculoplastic surgery: first iPhone application. Plast Reconstr Surg.
2011;127(3):57e-58e.
P L A S T I C S U R G E RY A P P L I C AT I O N S
PAT I E NT C A R E :
internetmedicine.com
P L A S T I C S U R G E RY A P P L I C AT I O N S
PAT I E NT C A R E :
internetmedicine.com
Perioperative Pain and Anxiety
P L A S T I C S U R G E RY A P P L I C AT I O N S
G AT E C O N T R O L T H E O RY
L I M I TAT I O N S
Two broad areas for which AR devices can be used to interface with the EMR:
• HIPAA compliance
• Distraction of Surgeon – Delayed response time an decreased awareness
• Cost effective innovation
TH E N E XT S T E P
Two broad areas for which AR devices can be used to interface with the EMR:
Adoption by the surgical community is a major hurdleIntegration with artificial intelligence and big data
Tracking analysis
3D virtual surgical planning software (i.e. oculus rift )
www.pcworld.com
A R D E V I C E N E E DSIntegrate patient data in visual field pre, intra and post-op
Enhance consultations in real-time
Make surgical education more visual and hands- on
Synthesize images and video into medical documen-tation
"Of course nobody knows anything. You just getwhat information you can through the windows of your five senses, and then make your guesses.
When they're wrong, you pay the penalty."
R E F E R E N C E S
Two broad areas for which AR devices can be used to interface with the EMR:
1. Armstrong, D. G., Rankin, T. M., Giovinco, N. A., Mills, J. L. & Matsuoka, Y. A Heads-Up Display for Diabetic Limb Salvage Surgery A View Through the Google Looking Glass. J. Diabetes Sci. Technol. 8, 951–956 (2014).2. Berger, A. J., Gaster, R. S. & Lee, G. K. Development of an Affordable System for Personalized Video-Documented Surgical Skill Analysis for Surgical Residency Training. Ann. Plast. Surg. (2013). doi:10.1097/SAP.0b013e31827e513c3. Waxman, B. P. Medicine in small doses. ANZ J. Surg. 84, 907–907 (2014).4. Assad-Kottner, C., Hakeem, A., Fontenot, E. & Uretsky, B. F. ‘Tele-mentoring’: an interventional procedure using a wearable computer: first-in-man. J. Am. Coll. Cardiol. 63, 1022 (2014).5. Carenzo, L. et al. Disaster medicine through Google Glass. Eur. J. Emerg. Med. Off. J. Eur. Soc. Emerg. Med. (2014). doi:10.1097/MEJ.00000000000002296. Schreinemacher, M. H., Graafland, M. & Schijven, M. P. Google glass in surgery. Surg. Innov. 21, 651–652 (2014).7. Lindeque, B. G. P. et al. Emerging technology in surgical education: combining real-time augmented reality and wearable computing devices. Orthopedics 37, 751–757 (2014).8. Jeroudi, O. M. et al. Accuracy of Remote Electrocardiogram Interpretation With the Use of Google Glass Technology. Am. J. Cardiol. 0,9. Shakil, I., Tran, P. & Dehy, R. System and method for augmenting healthcare-provider performance. (2014). at <http://www.google.com/patents/US20140222526>10. Emergency providers see big potential for Google Glass. ED Manag. Mon. Update Emerg. Dep. Manag. 26, 55–58 (2014)