A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE...
-
Upload
diana-moore -
Category
Documents
-
view
212 -
download
0
Transcript of A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE...
![Page 1: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/1.jpg)
A Mixed Reality Approach for Merging Abstract and Concrete Knowledge
John QuarlesDepartment of CISE
Samsun LampotangDepartment of Anesthesiology
Ira FischlerDepartment of Psycholgy
Paul FishwickDepartment of CISE
Benjamin LokDepartment of CISE
![Page 2: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/2.jpg)
Anesthesia Education
Current Learning Process
The Virtual Anesthesia Machine (VAM) – an abstract 2D
simulation
A real Anesthesia Machine – a physical simulation
![Page 3: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/3.jpg)
Knowledge Types
Type Abstract Concrete
Skill Set Concepts that can be applied to many real machine models
Procedural, tactile, and psychomotor skills
Example Invisible gas flow Performing a procedure with a specific machine
Transfer Knowledge?
![Page 4: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/4.jpg)
Problem: Merging Knowledge Types Anesthesia machine trainees must mentally
merge: Abstract Concepts Procedural Skills
Mixed Reality (MR) can merge real and virtual spaces Virtual space = Abstract Real Space = Concrete
Can MR’s Merging of spaces improve users’ merging of abstract and concrete knowledge?
![Page 5: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/5.jpg)
Merging Abstract and Concrete
Merge the abstract physical simulation with the physical device
![Page 6: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/6.jpg)
The Augmented Anesthesia Machine
Uses a magic lens approach to merge:Abstract VAM simulationReal Anesthesia Machine TUI
![Page 7: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/7.jpg)
Overview
The Augmented Anesthesia Machine The Augmented Learning Process User Study: VAM vs. AAM
MR enables abstract + concrete knowledge merging
![Page 8: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/8.jpg)
Previous Work
Tangible User Interfaces Two Main Parts:
Computational Media Physical Interface
Interface is meaningful Representative of a part of the computational media As opposed to an abstract interface (i.e. a mouse)
AAM Computational media = VAM Physical Interface = real anesthesia machine
[Ishii 1997, Ullmer 2000]
![Page 9: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/9.jpg)
Previous Work
Magic Lenses2D to 3DApplications in AR/MR
AAMUses a magic lens to visually merge the
abstract and concrete spaces
[Bier 1993, Looser 2004]
![Page 10: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/10.jpg)
Motivation
75% of Anesthesia Machine Related Accidents resulting in death or brain damage are due to user error.Many anesthesia providers do not fully
understand how the machine functions internally
VAM was created to reduce this number
![Page 11: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/11.jpg)
Motivation for the VAM
Created by Dr. Samsun Lampotang Advantages of the VAM
visualize the invisible (i.e. gas flow) Spatial layout simplified for ease of visualization Web disseminated Increases understanding of internal function
[Fishcler et. al. 2006] 30000 Registered Users One Billion Hits per year
(http://vam.anest.ufl.edu) Problems still exist in knowledge transfer from
the VAM to the real machine
![Page 12: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/12.jpg)
Motivation VAM to AAM: A Mapping Problem
A A
B
B
![Page 13: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/13.jpg)
MR: Enabling Multiple Represetations Multiple Representations improve overall comprehension
of a concept. VAM: Abstract Representation Anesthesia Machine: Concrete Representation AAM: Combined Representation
Proposed Augmented Learning Process:
Abstract Representation
(VAM)
Combined Representations
(AAM)
Concrete Representation
(Anesthesia Machine)
![Page 14: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/14.jpg)
The AAM Representation
Tracked 6DOF magic lens shows overlaid VAM simulation Real machine interaction
![Page 15: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/15.jpg)
Anesthesia Machine Tracking System 2D Optical Tracking with OpenCV
Infrared Markers for knobs Infrared LEDs connected to buttonsColor tracking for gauges (bright red)
3 Unibrain webcams positioned around the machine in positions of minimal occlusion
![Page 16: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/16.jpg)
Magic Lens Display
HP TC1100 3.7 lbs 10” screen
Not See through Uses a 3D model of the
machine, registered to the real machine
Easier to enable consistent registration
Less hardware than video see through
![Page 17: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/17.jpg)
Magic Lens Tracking System
Outside-Looking-in Optical Tracking
Infrared Markers Specifications:
2 Unibrain Fire-I Webcams 640x480 at 30 fps
Tracking Volume: 3x3x3 m Accuracy: 1cm Jitter: 5mm Latency: 70 ms
![Page 18: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/18.jpg)
User Study
Can AAM-Concrete help users to merge abstract and concrete knowledge?
Between subjects study (n=20)VAM Training GroupAAM Training Group
![Page 19: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/19.jpg)
Population and Environment
20 Psychology Students4 males, 16 femalesReceived class creditKnew nothing about anesthesia machines
Conducted in a quiet lab environment
![Page 20: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/20.jpg)
Procedure
Informed Consent
Introduction Machine
Functions
5 Training Exercises
VAM
AAM
Spatial Cognition
Tests
Self Evaluation of Training
Written Test
Real Machine Intro
VAM
AAM
Hands-on Fault Test
Questionnaires
Day 1 (90 min)
Day 2 (60 min)
![Page 21: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/21.jpg)
Metrics
Efficiency of Training Time to complete 5 training exercises
Abstract Knowledge Acquisition Written Anesthesia Machine Test
Short Answer and muliple choice From the Anesthesia Patient Safety Foundation workbook
Concrete Knowledge Acquistion Hands-on fault test
Faulty inspiratory valve
![Page 22: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/22.jpg)
Fault Test Results Hypothesis 1: The AAM is more effective at teaching
concrete concepts.
AAM group found the faults significantly more often than VAM group
Accept Hypothesis 1: The AAM trains concrete knowledge more effectively than the VAM
Group # Participants Successful
AAM 6 out of 10
VAM 1 out of 10
![Page 23: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/23.jpg)
Written Test Results
Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts
There were no significant differences in Written test scores (p<0.21)
![Page 24: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/24.jpg)
Training Time Results
Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts
VAM Group trained significantly faster (p<0.002)
Accept Hypothesis 2: The VAM trains abstract knowledge more effectively than the AAM
![Page 25: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/25.jpg)
Discussion
Hypothesis 3: The AAM improves transfer to the real machine by enabling the merging of abstract and concrete knowledge.
AAM group performed significantly better in the fault test
To solve the fault test, participants had to merge: Concrete knowledge: notice the inspiratory valve was
missing Abstract knowledge: understand the effect on the gas
flow of the machine – harmful to the patient Accept Hypothesis 3: AAM helps to merge
abstract and concrete knowledge
![Page 26: A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira.](https://reader035.fdocuments.us/reader035/viewer/2022070401/56649f1b5503460f94c30290/html5/thumbnails/26.jpg)
Conclusions
leverage MR to merge simulation typesCombining Abstract simulation with the
corresponding physical device i.e. merging the VAM with the real machine
MR enables multiple representations MR can help users to merge abstract and
concrete knowledge Improves training transfer into real world
domains