Some Usability Issues of Augmented and Mixed Reality for e-Health Applications in the Medical Domain...
-
Upload
diamond-brailsford -
Category
Documents
-
view
212 -
download
0
Transcript of Some Usability Issues of Augmented and Mixed Reality for e-Health Applications in the Medical Domain...
Some Usability Issues of Augmented and Mixed Reality
for e-Health Applications in the Medical Domain
Reinhold Behringer
Johannes Christian
Andreas Holzinger
Steve Wilkinson
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 2
Purpose of this Paper
Highlight AR and MR applications in the Medical Domain.
Point to usability issues. Not a complete survey.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 3
Augmented Reality (AR) Principle:
Computer generates output, which is fused with the human perception of the environment.
Appears to emanate from the environment itself, to be a part of it – registration with real environment.
Visual: 3D computer graphics, employing Virtual Reality
(VR) concepts. Acoustic:
Spatial audio.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 4
Mixed Reality (MR)
Real environment is not necessarily the basic framework.
Virtual Environment (VE) is the main framework.
Objects in MR can be computer-generated or real.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 5
AR and MR in Medical Domain
AR: Data visualisation directly on patient. Guidance to physician for medical procedures.
MR: Simulation and training. Patient itself can be simulated.
Benefits: improved situational awareness
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 6
Display Types for AR AR fusion of visual sense:
Head-worn displays: Ideal merging of computer output with visual
impression of surroundings. Hand-held displays:
Either acting as “frame into the real world”, or Showing representation of real world by video /
images, onto which information is overlaid. Projective displays:
Project information directly onto object (patient). 3D is only correct for one single user at a time. Requires consideration of projection surface.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 7
Tracking of User
User’s viewing position and orientation needs to be tracked for correct display rendition.
Technologies: Active illumination outside of human-visible
spectrum (e.g. IR). Computer vision approaches, capturing either:
Environment as seen from user (head-worn camera), or
User himself, seen from cameras in the environment.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 8
Examples of Applications in Medical Domain
Data projection onto patients. Support of surgery. Simulation tools for teaching and practising. Therapy and rehabilitation.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 9
Visualisation of Ultrasound and CT
UNC, A. State (1992). Live image stream of ultrasound scans
onto body: pre-natal care, obstetric examination.
Initially 2D scans, later volumetric scans. Technique also used for needle biopsies
and minimally invasive surgery.
Stockmans (2005). Correction of bone disformities. Comparison of before-after procedure.
UNC AR demonstration
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 10
Simulation Sielhorst et al., TU Munich (2004)
Birth simulator, using MR techniques. Real object: model of torso. Graphical simulation: baby. Birth pliers as haptic interface for interaction.
Nestler et al., TU Munich (2007) Virtual patients for large-scale disaster training. Table-top display, individual patients with different injuries.
Wilkinson (2005) Using purely VR for simulation of surgical procedure on
hand. Goal: to educate patients and reduce fear.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 11
Aiding Surgery Not yet practical in clinical applications, but
promising improvements of spatial orientation, allowing more radical operative therapy.
MEDARPA project: AR and VR supporting minimally invasive surgery, to aid
the navigation of the surgeon (bronchoscopy and brachytherapy).
Future use with surgical robot.
Reitinger, TU Graz (2005) Virtual liver surgery planning system, aiding in providing
more precise measurements for tumor treatment.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 12
Therapy, Rehabilitation
Using VR to cure phobias. Learning in controlled graphical environment how
to deal with phobia-causing situations. Rehabilitation of motoric skills:
AR system evokes motor images, Guides motoric practise
Healthy living: “Persuasive Mirror” shows future projection of
self, under influence of varying live style.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 13
Edutainment
Teaching anatomy: “electronic book” for interactive display of 3D
anatomic perspective of humans. Organs can be viewed from different angles.
Simulation: To reduce fear of the unknown.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 14
Registration and Tracking Requirements:
View of patient’s body needs to be unobstructed for the physician.
Tracking needs to take into account deformable tissue of human body.
Calibration effort needs to be small, so as not to distract from medical procedure.
Tracking needs to be resilient to occlusion. Markers for visual tracking can be attached to
human body. Magnetic tracking can provide seamless tracking of
physician.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 15
Displays
Head-attached: Allows hands-free activity. Low resolution and viewing angle. Example: Variscope (Birkfellner, TU Vienna)
Hand-held: Is currently available at reasonable cost and
specifications. May interfere with actual procedures. Tablet display on boom:
Allows hands-free operation, while at the same time being cost effective.
Can be semi-transparent or video-see-through. Both display and physician need to be tracked for
correct visual view. Spatial.
MEDARPA display
Variscope
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 16
Interaction
Traditional interfaces (mouse, keyboard) are not suitable, as they distract from the task.
Preferable: Automatic ubiquitous interface, acting in the
background. E.g. speech recognition, gesture recognition. Possibly integrated into medical instrument.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 17
User-Centred Development Potential benefits of AR/MR are obvious. But for studying usability, practical workflows need
to be studied. AR application developers need to understand the
workflows of medical professionals. Usability Engineering Methods (UEM) are becoming
more important, but are still not applied very often: Software engineers work on the implementation, usability
experts work on design. Rarely do they collaborate.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 18
User-Centred Development
In most cases: Usability of AR environments is evaluated. E.g. Hix:
Criteria for assessing VE design. Usability of standard user interfaces for VE.
Effectiveness of different interactive devices. Needed:
User-Centred Development.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 19
Presence and Errors in VE
Specific to VEs: Sense of presence (immersion). Assessed through questionnaires, focusing on a
person’s sense of presence. Presence is not associated with task performance
(Slater). But measures of presence have concentrated on
user perception of VE technology. Errors in VEs can impact significantly the
perception of presence
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 20
Immersion
Effects of being immersed: Investigated in a variety of VE system
configurations: Collaborative VEs, viewpoint of self, perception of
others’ presence. Fully immersive VEs have been well investigated,
focusing on head-worn displays. Missing:
Techniques for measuring presence. Questionnaire-based assessment may be biased.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 21
Concerns about AR
Problematic: Information overload can be accompanied by
sensory overload. Overwhelming experience in AR environment. Possible lack of acceptance of this technology. Social effects (beliefs, attitudes, feelings).
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 22
Suggestions for Further Study Which are the most appropriate interaction
metaphors in AR for medical domain? Which influences do AR applications have on
performance of the end users in the medical domain?
What are the effects of adaptation that people might have to make cognitively to believe in and cope within an AR environment?
How will multimodal interaction through a number of input and output channels enhance or detract from the reality/Virtuality experience?
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 23
Specific to AR
AR must deal with physical interactions, social interaction, cognitive interactions.
Is difficult to isolate variables. Less performance constraints. Lower predictability of behaviors.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 24
Successful AR Application All involved research domains have to be
considered. User-centered design focus is important. Need to be:
Accessible. Usable for everyday end users. Follow notions of pervasive and ubiquitous
computing. Implement basic ideas of social software. Designed for users without deep IT knowledge.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 25
Summary
AR and MR have excellent potential in medical applications.
Usability issues need to be addressed in application design: Not only study usability in experience of VE
environments. Need to apply usability engineering methods in
design.
Leeds Metropolitan University
Innovation North – Faculty Of Information And Technology Slide # 26
Acknowledgment
Johannes Christian Andreas Holzinger Steve Wilkinson