Smart Textiles & Wearable Intelligence Anna Hristoskova 1st International Summer School on eCare,...

Smart Textiles & Wearable IntelligenceAnna Hristoskova1st International Summer School on eCare, August 25 - 29, 2014, Ghent, Belgium

2

Home automation

Sensors/devices integrated into the environment

Focus is on integration and standardized communication

Automation and monitoring of person activities

1st International Summer School on eCare, August 25 - 29, 2014, Ghent, Belgium

Magnetic door sensor

Sensor gateway

Movement detection

Light/Temp/Hum sensor

RFID reader

3

Smart Textiles and Wearable Intelligence


Bedlinen:- Can contain sensors/ actuators- Conductive wiring- Electronics + Data processing- Wireless communication- Powering- …

Mattress:- Can contain sensors/ actuators- Conductive wiring- Electronics + Data processing- Wireless communication- Powering- …

Environment:- Can contain sensors / actuators- Electronics + Data processing- Wireless communication- Powering- …

Garment:- Can contain sensors/ actuators- Conductive wiring- Electronics + Data processing- Wireless communication- Powering- …

Smartphone:- Can contain sensors/ actuators- Electronics + Data processing- Wireless communication- Powering- … Bed:

- Can contain sensors/ actuators- Electronics + Data processing- Wireless communication- Powering- …

Wardrobe:- Can contain sensors/ actuators- Electronics + Data processing- Wireless communication- Powering/Charging- …

4

SMARTpro – the project


5

Vision

Identify and address industrial needs and challenges w.r.t. the realization, integration and usability of smart textiles & wearable intelligence Combine

intelligently and efficiently textiles, electronics

and ICT


Stimulate collaboration across ICT, electronics and textile sectors Manufacturing on

an industrial scale Reliable and

durable

Build proof-of-concept demonstrators in target domains such as Safety and intervention (Home) care Sport and leisure Technical applications

Involve end-users

T

E ICT

Industry

Users

Communication

Interoperability

Integration

Components Domain

Business Opportunity

Innovative solutions Personalization

Manufacturingprocesses

6

SMARTpro – the project


Oct. 1st 2013 - Oct. 1st 2017

Multi-disciplinary consortium of research partners with complementary expertise, coordinated by industrial research centers.

A steering group of core industrial players is being established.

CROSS SECTORIAL

COOPERATION

Textile sector

Electronics sector ICT-sector

Home/care

Intervention

Sports & leisure

TechnicalTextile

Electronics ICT

7

Approach to selected targeted & collective challenges

End user 1 requirements End user 2 requirements

End user requirements + Common technological manufacturing

challenges that can be addressed with innovative technologies from

SmartPro partners

Taken up by company (together with SmartPro

partner) in separate trajectory

Company 1 Technological/manufacturing challenges Company 2 Technological/manufacturing challenges

End user 3 requirements

Company 3 Technological/manufacturing challenges

The aim is to find a critical mass in requirements + technological/manufacturing challenges

that is as much as possible relevant for all member companies


8

Partner introductionSMARTpro


9

Textiles - design

Design, development and prototyping of textiles products

Digital printing & coating, i.e., conductive inks

Production of electrically conductive yarns and fabrics

Integration of components in textiles through various confection techniques

Comfort Body Scanning


10

Textiles - Integration

Lab production of smart textiles components Sensors for monitoring heart rate,

respiration rate, sweat Textile antenna, textiles batteries Textiles for heating and electro-

stimulation Printing of conductive inks Conductive yarns & coatings Optical fibres & LEDs Fibre transistors


11

Electronics - design

Prototyping following end-user specs Definition of feasible integration

prototypes Identification of manufacturability

and quality risks of prototypes Analysis of the risk factors of

electronics in textiles Industrial available electronic

component and PBA technology for the intended applications

Basic elements of design guidelines for integrating electronics into textiles


12

Electronics - Integration


Conductive yarns (acting as sensors) in

combination with integratedreadout electronics

13

Textiles & Electronics


14

Data Intelligence


Carried by humans or worn in/on the body as smart phones, heart rate meters, Google Glass or the Nike+ FuelBand

Sensors integrated into objects, such as textiles, toothbrushes, mattresses, mirrors, thermostats, doorways, steering wheels, …

Paradigm shift to wearable and

embedded devices

Physiological measurements and daily habits Context and surroundings

Allow to access and manipulate

diverse information about

humans

Lifestyle management for chronic illnesses such as diabetes Emergency situation such as fire or flooding

Intelligent sensor-based

applications enhance decision-

making for professionals and

individuals

15

Data Intelligence


• The collection, modeling, and processing of large amounts of sensor data,• The extraction of knowledge from these data and • Its presentation to the end users

While hardware and software design remain important, capturing, processing and interpreting data becomes the main concern

• Uncover hidden insights and infer additional knowledge from data• Enable advanced visualization of trends and patterns• Reduce information overload and target proactive information delivery

Data opportunities

• Data integration from heterogeneous sources• Real time information processing and event recognition• Consider information from human behaviours and multi-modal interactions• Act on behalf of the users’ intentions

Data challenges

16

Applications


17

Interdisciplinary collaboration

Electronics

TextilesICT


Tech

nolo

gica

l

build

ing

bloc

ksIntegration

in textile

Applications

Monitoring, sensing, light

Challenges:ComfortMaintenance friendlyReliable

Challenges:Sensors, packaging and reliabilityFlexible and stretchableSubstrates … other platforms

Challenges:Conductive yarnsInterconnection with flexible and stretchable substratesWashable -> packagingIntegration

18

ScenariosSmartPro project


19

Home / care


20

Home/Care

Concept continuous monitoring of the patient and his environment, at home and

on-the-go

Target users Elderly Patients in rehabilitation phase after some medical intervention Formal and informal caregivers

Objective Provide personalised monitoring and remote care with the goal to

increase autonomy and quality of life facilitate caregivers in providing optimal support on demand

Predict deteriorations of the patient’s state through Stimulation and detection of movement and inactivity Monitoring the evolution of important predictors (e.g. walking speed)



Home/CareDemonstrator

Pressure sensor

Magnetic door sensor

Sensor gateway

Movement detection

Light/Temp/Hum sensor

RFID reader

Person: - Elderly - Patients in rehabilitation phase after some medical intervention

To be monitored Information to formal and informal caregivers

Home- With unobtrusive and subtle monitoring solution

Environment/On-the-go:- With unobtrusive and subtle monitoring solution

e.g. Position monitoring

Rooms:- Equipped with sensors/ actuators- Position monitoring system- ,,,

22

Home/Care

Monitored parameters Physiological

heart rate, body temperature, galvanic skin response, EMG

Activity type (walking, biking, running, ...), distance, speed, steps, ...

Diet fruit and vegetable intake, total calorie estimation, ...

Environment temperature, noise, light, ...

Derived knowledge Habits

sleeping, resting, eating, ...

Anomalies elevated heart rate, abnormal wake-ups, too low intensity levels, ...

Patterns causal relationships between activities and physical/mental condition


23

Home/Care

Use cases Wander detection

recognize where a patient is located and whether or not he or she is supposed to be there or feels lost

Fall detection - recognize if a patient made a fall. Monitoring evolution of important predictors (e.g. walking speed) for fall prevention for

elderly

Activity monitoring recognize whether or not a patient is acting according to his or her normal

habits

Physiological state monitor the patient’s health state, create a long term profile and track

possible deteriorations


24

Intervention


25

Intervention

Concept monitoring target users’ status, environment and behavior

Target users Team of emergency workers (e.g. firefighters, police, ...) Coordinated by a commander On-site stewards Evacuees

Objective increase situational awareness and support decision making in

case of an incident Controlling the incident and get back to normal as soon as possible Ensuring safe and efficient evacuation of the building(s) and site


26

Garment:- unobtrusive reliable monitoringe.g. temperature, oxygen levels, heart rate, stress

Smartphone:- With monitoring solutione.g. location

Environment:- With monitoring solutione.g. temperature, fire, dangerous goods

InterventionDemonstrator


InterventionDemonstrator

27

Server inside fire truck

Fire fighters On-sleeve

computation

Commander tabletremote overview

Crisis team web-basedgeneralized overview


28

Intervention


heart rate, body temperature, galvanic skin response, blood oxygen level

Environment indoor/outdoor location, people movement, temperature, noise &

lighting levels, traffic information, presence and location of dangerous goods

Resources oxygen bottles, firearms, mobile devices, GSM/WiFi infrastructure

Derived knowledge Stress & fatigue, overcrowding of emergency exits, panic

reactions, ...


29

Sports & leisure


30

Sports & leisure Concept

monitoring athletes during races, training and everyday life

Target users Athletes Coach/personal trainer Science lab personnel

Objective Provide personalised and actionable insights beyond mere tracking and visualisation

No average human, different individuals have different needs and respond differently Knowing in retrospect that you slept bad is not so useful, knowing what you can/should (not) do upfront in

order to sleep well is more important

Improve performance and prevent injuries Monitor correct execution of exercises, adherence to prescribed plan, intensity levels, recovery & sleep,

muscle imbalances, ...

Detected anomalies high resting HR, abnormal sleep patterns, failure to meet intensity levels, incorrect posture

Provide feedback on potential causes cafeine intake, insufficient time between exercise, too high training volume, too high/low intensity level, ..

Predict future evolution1st International Summer School on eCare,

August 25 - 29, 2014, Ghent, Belgium

31

Sports & leisureDemonstrator


Garment:- Unobtrusive monitoring of performancee.g. heart rate, fatigue

Environment/On-the-go:- With unobtrusive and subtle monitoring solutione.g. aerodynamic posture during time trials

32

Sports & leisure


heart rate, body temperature, galvanic skin response, EMG

Activity distance, speed, steps, cadence, movement, rest, altitude, geographic locations

Environment weather, noise, light

Derived knowledge Habits

sleeping, resting, eating, travelling

Anomalies elevated heart rate, abnormal wake-ups, too high/low intensity levels, tactics

Patterns causal relationships between activities and anomalies


33

Sports & leisure

Example use cases Real-time analysis of

aerodynamic posture during time trials, in order to provide immediate corrective feedback

fatigue and stress during game, in order to recommend substitutions

Analysis of lifestyle outside training in the build-up phase towards a big event,

guarantee optimal recovery and to appropriately tune training schedules

Monitoring of correct execution of exercises and adherence to prescribed plans

speed up rehabilitation and prevent injuries


34

Technical


35

Technical Concept

Indoor intelligent flooring/wall/car interior

Outdoor monitoring critical infrastructure

Target users Large area structures for interior use

carpets, curtains, mattress covers

Outdoor structures geotextiles, tents, sunscreens

Transport & Logistics automotive, packaging

Objective(s) Outdoor

improve safety by hazard & structural health monitoring of constructions

Indoor safety & prevention, fall and inactivity detection or prevention of inhabitants

Integration of new features, e.g. lighting, heating, cooling, energy scavenging


36

TechnicalVirtual demonstrator


Large textile structure:- Unobtrusive monitoring of activities and statee.g.fall detection, structural health monitoring

37

Technical

Monitored parameters Environment

weather (ex. wind, precipitation), moisture, pressure, temperature

Activity Outdoor

safety, data for structural health monitoring, capture structural response to extreme events (e.g. earthquake, storm), speed of degradation, energy applications (flexible solar cells)

Indoor safety, lighting (LED, OLED, optical fibers integration), heating or sensor

applications, location

Derived knowledge Durability and safety limits of outdoor structures and constructions Indoor

Support for the other cases


38

Technical

Example use cases Monitoring of wind/snow stress on tent construction

Monitoring evolution of important predictors (e.g. walking speed, sleep quality) for fall prevention for elderly

Monitoring crowded indoor events in case of crowd, control, intervention during emergency


39

Data IntelligenceSMARTpro project


40

Technological challenges

Emergency/Intervention

Sport & leisure

(Home) care

TechnicalIntelligent data processing

Unobrusive m

onitoring

Electronics into textile

Textile as carrier for electronics


Data fusion & mining

41

Intelligent data processing – challenges Capture, store and fuse large amounts of data obtained from

continuously monitoring people and their environment physiological as well as non-physiological measurements context and surroundings

Identify trends in order to construct user profiles typical amount of sleep, most active periods of day/week, frequently visited

POIs with typical visiting times, heart rate zones, resting heart rate, muscle activitation, ...

Explore methods to build prediction models estimated recovery period, overtraining, evolution in walking speed,

predicted energy consumption for weight loss, physiological evolution, ...

Detect anomalies in real-time geo-fencing, fall detection, abnormal inactivity, increased stress level, ...


42

Intelligent data processing – needs Historical data

for training robust models that can predict the future based on the past

Sufficient & suitable data not necessarily ‘big’ but naturally ‘relevant’ w.r.t. requirements

Reliable data reduced level of noise appropriate frequency standardized format

End user requirements Lacking knowledge of the application domain and end-user

needs, even the most intelligent algorithms are useless


43

Context modelling method

Context definition by domain experts Field studies execution Mental models creation Scenario description Co-creation workshops Goal-directed task analysis

Formal translation into an ontology & rules by ontology engineers in collaboration with domain experts long track record of developing ICT solutions for specific domain accurate and complete reflection of the daily work practices

Deployment of the architecture by domain experts

Processing of usability feedback


Domain experts

SoftwareEngineer

User researchers

Observations

Co-creation workshops

Scenarios

44

Formal domain definition supported by semantics Common vocabulary and a shared understanding of the

structure of the information Support communication Explicit domain description of

Concepts Properties and Attributes of concepts Constraints on properties and attributes Individuals Rules to express general workflows & intelligent behavior

Enable reuse of domain knowledge Introduce standards allowing interoperability

Capture knowledge formally Reasoning and extraction of new knowledge


Domain concepts

Rule set

45

Interpretation & decision support

Domain Modelling

Rule definition

Reasoning on the data

Support actions

Disease

Treatment

Antibiotics

has_treatment *

Symptomhas_symptom *

Is a

Fever

Temp

Curable Disease == Disease AND has_treatment.Treatment

IF night → close the blinds AND dim the lights

Interpretation

The flu IS Curable Disease

Domainconcepts

Treatment: Stay at homeRestAntiviral medicationsAntibitiotics

Decision support

Recovery in a week

Stay at home

1st International Summer School on eCare, August 25 - 29, 2014, Ghent, BelgiumRule set

46

Data processing steps1. Pre-processing and cleaning of the sensor and device data

Collect distributed heterogeneous data from various devices and sensors Clean up the data Remove faulty measurements

2. Aggregation of several data measurements from various sources Averages of measurements Combination of multiple data points

3. Interpretation of the situation based on the gathered data Interpret on the exact meaning of the data Evaluate the user’s context

4. Decision support based on the user context Depending on the interpretation, the person’s situation can be inferred and

decision taken on which type of actions should be taken

5. Triggering of actions depending on the situation User profiling Alarms Predictions Recommendations

Electronics layer

36°C 37°C 15°C X,Y

36°C 37°C X,Y

36.5°C X,Y

Temperature

Position


47

Home/Care: data processing

Interpret sensor data ... recognises a person is wearing the textile (based on temp. data), where he

or she is (based on localisation data) and

what he or she is doing (based on activity data)

... and turn it into knowledge identifies patterns & trends

sleep/awake pattern typical POIs and visiting times walking speed/steps per day health state

identifies anomalies/deteriorations abnormal sleeping behaviour (e.g. during the day) geo-fencing moving from more active to more passive lifestyle (e.g. prolonged sitting instead of

walking)


48

Technological challenges


Vakgroep Textielkunde

49

Questions?Anna Hristoskova


Smart Textiles & Wearable Intelligence Anna Hristoskova 1st International Summer School on eCare,...

Documents

Transcript of Smart Textiles & Wearable Intelligence Anna Hristoskova 1st International Summer School on eCare,...