Multimodal Bio-signal based Control of Intelligent Wheelchair
-
Upload
nigel-west -
Category
Documents
-
view
25 -
download
0
description
Transcript of Multimodal Bio-signal based Control of Intelligent Wheelchair
Closing conference of SYSIASS – June 17th 2014
Multimodal Bio-signal based Control of Intelligent Wheelchair
Professor Huosheng HuLeader of Activity 3
University of Essex, U.K.
Closing conference of SYSIASSJune 17th 2014 2
SYSIASS: Autonomous and Intelligent Healthcare System
Closing conference of SYSIASSJune 17th 2014 3
Aim and Objectives Multimodal Human-Machine Interface aims to make wheelchairs more
user-friendly.
The key challenge is the understanding of
the user who interacts with the wheelchair
the system (the computer and sensor technology & their usability)
the interaction between the user and the wheelchair.
A proper balance between
Functionality that is defined by the set of actions or services that it provides to its users, and
Usability that is the range and degree by which the system can be used efficiently and adequately to accomplish certain goals for certain users.
Closing conference of SYSIASSJune 17th 2014 4
System Configuration
Multimodal HMI
Voice, gesture, EMG, EEG
Closing conference of SYSIASSJune 17th 2014 5
Detection of Human Intension Inertial Sensing: to sense the position and motion of the human
hand and other body parts for use in HMI.
Audio Sensing: Using microphones to sense the sound is to interpret speech, which is the most natural modality.
Visual Sensing: Using cameras to detect the human motion, such as gestures, lip motion, gaze, facial expressions, head & other body movements.
Touch and force: This is especially important for building a proper feel of “realism” in human intension.
Muscle and Brainwave: These bio-signals are obtained noninvasively from the surface of the scalp and skin; used for wheelchair control.
Closing conference of SYSIASSJune 17th 2014 6
Wireless Voice based Control To use human voice commands; Pre-defined control phrases:
“forward”, “turn left”, “turn right”, “backwards”, “stop”;
Wireless wearable headphone device; Fast voice recognition and no pre-
training Plug and play wireless and comfort
control Obstacle avoidance with laser scanner Unlimited voice control commands
available Video show
Closing conference of SYSIASSJune 17th 2014 7
Visual Head Gesture based Control To help disabled and elderly who cannot
use joystick To realise hands-free control of the
wheelchair Obstacle avoidance to ensure user safety Functionalities:Visual detection of head gestures of users
Pre-defined commands (left, right, forward, backward, stop) for controlling a wheelchair
Laser based obstacle detection & avoidance
Video show
Closing conference of SYSIASSJune 17th 2014 8
Facial Expression & Head Movement Facial expressions are detected using
the cognitiv suite of Emotiv.
Head movements are detected by the gyroscope of Emotiv.
Pre-defined commands (left, right, forward, backward, and stop)
A simple and safe hands-free control with flexible configurations for users.
An ideal alternative to the joystick control for users with severe disability.
2F+1H 3F+1H
Closing conference of SYSIASSJune 17th 2014 9
Wireless IMU in a Baseball Cap Detect head motion for wheelchair control Minimum user head motion required Adapt to people with weak neck abilities Obstacle avoidance with laser scanner Totally replace joystick control interface
Video show
Closing conference of SYSIASSJune 17th 2014 10
Minimally Invasive Intra-Oral Palate for wheelchair User
A minimally invasive way to control wheelchair
It is comfortable for users and allows prolonged use.
It is based on an intra-oral dental retainer clip with9 force sensitive resistors10 Bit 200Hz data capture Opto-isolation is deployed.Connectivity to GPSB
Closing conference of SYSIASSJune 17th 2014 11
Conformal RFID Sensing for wheelchair control
Strain gauge RFID tag is designed to sense the stretch on skin (neck/eyebrow movement)
A tongue controlled RFID tag is designed to sense the tongue position in the mouth
To provide an on/off switch and control mechanism for controlling the wheelchair.
Closing conference of SYSIASSJune 17th 2014 12
Conclusion Human-Machine Interaction (HMI)
aims to achieve information full accessibility.
Human-Machine Interaction plays an key role in assistive technology.
Hands-free control of wheelchairs are extremely useful to general public without programming skills.
Multimodal HMI is necessary to overcome the limitations of single modality HMI.
Activity 3 of SYSIASS has been conducted successfully.