A Sensor Fusion Wearable Health-Monitoring System with ...

IntroductionSystem Architecture

Simulation Results, Conclusion and Future WorkReferences

A Sensor Fusion Wearable Health-Monitoring System with HapticFeedback

F. Sanfilippo

1and K. Y. Pettersen

2

1Department of Maritime Technology and Operations, Aalesund University College, Postboks 1517, 6025 Aalesund, Norway,[email protected]

2Department of Engineering Cybernetics, Norwegian University of Science and Technology, 7491 Trondheim, Norway,[email protected]

11th International Conference on Innovations in Information Technology (IIT’15)

F. Sanfilippo and K. Y. Pettersen A Sensor Fusion Wearable Health-Monitoring System with Haptic Feedback



Summary

1 Introduction

2 System Architecture

3 Simulation Results, Conclusion and Future Work




BackgroundA Possible Application Scenario: O↵shore OperationsMotivation FactorsUnderlying Idea

Wearable Health-Monitoring Systems (WHMS)

Wearable Health-Monitoring Systems (WHMS):

the design and development of these systems (WHMS) has received lots ofattention;

low-cost systems;

Multi-sensor fusion is one of the most suitable technologies to use.

Multi-sensor fusion and Haptics:

a system that features a multi-sensor fusion approach and also provides anintegrated haptic feedback for the user has not yet been deeply investigated.





A Possible Application Scenario: O↵shore Operations

Safety of o↵shore installations is a crucial issue:

Increasingly demanding marine operations are at the heart of the maritimeindustrial cluster.

These advanced operations are associated with a high level of uncertainty onboard of an o↵shore installation because such an installation usually operates in adynamic environment in which technical, human and organisational malfunctionsmay cause accidents.





Motivation Factors

This kind of working environment is associated with a considerable amount ofstress for the workers.

Physical stressors include noise, vibration, poor lighting and ventilation, confinedliving and working space, adverse o↵shore weather conditions, long working hoursand and shift work[1].

Psychosocial stressors cover job characteristics (work load, variety, clarity,control), perceived risk (fire, explosion, travelling by helicopter or ships, etc.), jobinsecurity, work-family balance, and the lack of certain types and sources of socialsupport.

Mandatory health and safety monitoring:

Mandatory health and safety monitoring of all crew members was put in place bythe o↵shore health and safety law in order to minimise human error and potentialresulting hazards. It is also mandatory to keep records pertinent to suchmonitoring.

There is an urgent need to develop more e�cient methods and tools that willallow for a greater accuracy and therefore more reliable modelling and simulationof risk assessment.

[1] WQ Chen, I TS Yu, and TW Wong. “Impact of occupational stress and other psychosocial factors on muscu-loskeletal pain among Chinese o↵shore oil installation workers”. In: Occupational and environmental medicine 62.4(2005), pp. 251–256.





A Wearable Health-Monitoring System for O↵shore Operators

Vibr. Motor

Push-Button

Contr. BoardECG Sensor

Acc. Sensor

LED

Temp. Sensor

A wearable health sensor monitoringsystem based on a multi-sensor fusionapproach:

Biometric and medical monitoringapplications can be performed byusing this multi-sensor device. Theembedded vibration motor makes itpossible to actuate distinctive hapticfeedback patterns.

The Light-emitting diode (LED)provides the operator with anadditional intuitive visual feedback.The provided push-button can be usedby the operator to report a potentialemergency state.

Data can be sent to a cloudcomputing system in order to performpermanent storage or visualised in realtime by sending the informationdirectly to a laptop or smart phone.




System ArchitectureClient (Chest-Worn Device)SensorsIntuitive Haptic and Visual FeedbackMulti-Threading and Multi-Level Hierarchical Server

System Architecture

Use

r con

trolle

d in

put

Body Temperature Sensor

Accelerometer Sensor

Electrocardiogram (ECG) Sensor

Colour-changing light-emitting diode

(LED)

Vibration Motor

Push-Button

Offshore Operator(provided with the

proposed wearable device)

SERVER

Remote Data Storage

cloud computing

Wi-Fi High-Level Methods

Concurrent Threads

Op. 1 Op. n...

Low-Level Methods

read write

processRequest

send

receive

Local Data

Storage

Controller Box

Inte

grat

ed S

enso

rs

Sensor Platform+

Comm. Module+

Controller Board+

Battery

Hap

tic a

nd V

isua

l Fee

dbac

k

CLIENT (chest-worn device)

A client-server pattern is adopted.The overall system design is the result of the application of specific virtualprototyping methods and simulations tools to aid us in making optimalarchitectural choices, testing procedures and validation techniques.





Client (Chest-Worn Device)

A polymer lithium ion battery isadopted in order to provide power tothe proposed controller box.

An Arduino Uno board based on theATmega328 micro-controller is usedas a client.Arduino is an open-sourceelectronics prototyping platform basedon flexible, easy-to-use hardware andsoftware.

To give communication capabilities tothe proposed wearable device, anArduino WiFi Shield is stacked on topof the adopted controller board. Indetail, the Arduino WiFi Shield allowsthe client to communicate with theserver by using the 802.11 wirelessspecification (WiFi).

To gather the operator’s biometricdata, an e-Health Sensor Shield isstacked on top of the adoptedcommunication module. The e-HealthSensor Shield allows Arduino boardsto easily gather information fromdi↵erent sensors. An open-sourcesoftware library is provided with thee-Health Sensor Shield, allowing foreasy access to the sensor data.





Sensors

N+

-+-

N

An ECG sensor kit is adopted. Thissensor input can be used as adiagnostic tool to assess the electricaland muscular functions of theoperator’s heart. This sensor hasproven to be useful in the diagnosis ofseveral cardiac pathologies rangingfrom myocardial ischemia andinfarction to syncope and palpitations.

An accelerometer sensor is adopted.This sensor is used to monitor fivedi↵erent patient positions(standing/sitting, supine, prone, leftand right). Analysing movementsduring on-board operations helps indetermining work quality and irregularpattern behaviours. Theaccelerometer sensor also help todetect fainting or falling of theoperator while working.





Sensors

A temperature sensor is provided. Body temperature is an important health stateindicator and accurate measurements can provide medical insight into thewearer’s response to di↵erent situations. The reason is that a number of diseasesare accompanied by characteristic changes in body temperature. Likewise, thecourse of certain diseases can be monitored by measuring body temperature, andthe e�ciency of a treatment can be evaluated by the physician.





Intuitive Haptic and Visual Feedback

A vibration motor is embedded on theback side of the controller box and itallows for actuating distinctive hapticfeedback patterns according to theoperator’s health state. This feature isfundamental for improving the user’srisk perception.

When the operator’s health state isnormal, the motor does not produceany vibrations. On the contrary, twodi↵erent vibration patterns areactuated to symbolise a potentiallyabnormal condition (low frequencyvibrations) or a rather dangerous stateof health (high frequency vibrations).

A colour-changing LED is embeddedon the visible side of the controllerbox. This LED provides the user withan additional intuitive visual feedbackof the current health state. Inparticular, three di↵erent colours areused: green to indicate a normalhealth state, yellow to symbolise apotentially abnormal condition and redto point out a dangerous health state.





Multi-Threading and Multi-Level Hierarchical Server

SERVERHigh-Level Methods

Concurrent Threads

Op. 1 Op. n...

Low-Level Methods

read write

processRequest

send

receive

Local Data

Storage

The High-Level methods layer includesthe processRequest control function,which handles each client request.

The Concurrent Threads level is thelayer where each client request istreated as a concurrent process. Thehealth state of each operator ismonitored by a separate thread. Eachmonitoring thread sends the currentinformation status back to thecorresponding client so that thevibration motor and the LED can beactuated.

The Low-Level methods layer includesthe functions that are used to writedata to a local database (write), toread previously acquired data from alocal database (read), to send theacquired data to the cloud (send),and to receive previously acquireddata from the cloud (receive).




Simulation ResultsConclusion and Future Work

ECG Simulation Results

vibrateMotor(low-frequency)setLEDcolor(yellow)

RGB LED Vibration Motor

Concerning ECG monitoring, we used data from the Physiobank archive[2]. Thepossibility of detecting supraventricular premature or ectopic beats (atrial or nodal) isconsidered.“Ectopic beats occur when another region of the atria depolarises before the sinoatrialnode and thus triggers a premature heartbeat”. A simply way to detect this eventconsists in comparing the current beat time interval with the average time interval ofthe previous beats.[2] Ary L Goldberger et al. “Physiobank, physiotoolkit, and physionet components of a new research resource forcomplex physiologic signals”. In: Circulation 101.23 (2000), e215–e220.





Accelerometer Simulation Results

vibrateMotor(high-frequency)setLEDcolor(red)


Arduino Serial Monitor

The possibility of detecting five di↵erent operator positions is considered. By using thesoftware library provided with the e-Health Sensor Shield, the proposed system is ableto distinguish between the following positions: standing/sitting, supine, prone, left andright.A particularly dangerous situation that can happen is fainting, which may cause theoperator to fall while working. When this happens, the corresponding server-sidemonitoring thread sends a status update to the client so that the vibration motor canvibrate with high frequency and the colour of the LED can be changed to red.





Temperature Simulation Results

vibrateMotor(high-frequency)setLEDcolor(red)


Arduino Serial Monitor

The possibility of detecting states of fever and hyperpyrexia for the operator has beenconsidered. Particularly, axillary body temperature measurements are taken at10-minutes intervals.The case of hyperthermia is considered as an example. This is quite dangerous for theoperator, therefore the system considers this case as a critical state and makes themotor the vibrate with high frequency, while the colour of the LED is changed to red.





Conclusion and Future Work

A Wearable Health-Monitoring System for O↵shore Operators:

It is possible to use the collected biometric information in real-time for monitoringthe health state of the crew members, or subsequently analyse the data tofacilitate a medical diagnosis.

The information can be sent wirelessly to a cloud computing system forpermanent cloud-based data storage or it can be visualised in real time if sent toa laptop or a smart phone.

In this preliminary work, virtual prototyping methods were fundamental in thedevelopment of the proposed system architecture.

Future work:

The use of experimental data will make it possible to certify and asses systemperformance in a real application scenario.

Advanced techniques for data mining may be adopted.

The proposed framework may be integrated with a positioning system for o↵shoreoperations that we recently developed[3].

[3] Filippo Sanfilippo and Kristin Ytterstad Pettersen. “XBee Positioning System with Embedded Haptic Feedback forDangerous O↵shore Operations: a Preliminary Study”. In: submitted to the Proc. of the OCEANS’15 MTS/IEEEConference, Genova, Italy. 2015.





Thank you for your attention

Contact:

F. Sanfilippo, Department of Maritime Technology and Operations, AalesundUniversity College, [email protected].




[1] WQ Chen, I TS Yu, and TW Wong. “Impact of occupational stress and otherpsychosocial factors on musculoskeletal pain among Chinese o↵shore oilinstallation workers”. In: Occupational and environmental medicine 62.4 (2005),pp. 251–256.

[2] Ary L Goldberger et al. “Physiobank, physiotoolkit, and physionet components ofa new research resource for complex physiologic signals”. In: Circulation 101.23(2000), e215–e220.

[3] Filippo Sanfilippo and Kristin Ytterstad Pettersen. “XBee Positioning Systemwith Embedded Haptic Feedback for Dangerous O↵shore Operations: aPreliminary Study”. In: submitted to the Proc. of the OCEANS’15 MTS/IEEEConference, Genova, Italy. 2015.


A Sensor Fusion Wearable Health-Monitoring System with ...

Documents

Transcript of A Sensor Fusion Wearable Health-Monitoring System with ...