Partners - Funding

• Coordination:– CERTH-ITI– CMRR, CHU Nice

• Associated Partners– CMRR, CHU Nice, EA CoBTeK UNS– UB1 – Cassidian (CS)– INRIA– LTU – DCU– IBM Israel– Philips (PENB)– Vistek ISRA Vision (VIV)– Link Care Services (LCS)

• Funding:– FP7-ICT 2011. European Commission

3

Centre for Research and Technology Hellas Informatics & Telematics Institute (CERTH-ITI)

• Expertise– semantic multimedia analysis– Semantic Web (ontology and rules engineering, reasoning, uncertainty)– personalisation for multimedia applications (profile learning, semantic

recommendations)

• Role in the project – project Co-ordinator, WP5 leader (Medical Ambient Intelligence), PSB

member

• Contribution– clinical and generic knowledge representation [WP5]– semantic, aggregated patient behaviour interpretation [WP5]– patient profiling, knowledge enrichment, personalised feedback [WP5]– event detection/interpretation [WP4/WP5]

4

Université Bordeaux 1 (UB1) (and IPB as third party)

• Expertise– image and video indexing and retrieval– signal processing– image and video interpretation

• Role in the project – WP4 leader (Situational Analysis of Daily Activities)

• Contribution– visual sensing & instrumental activities analysis [WP4]– wearable sensors monitoring [WP4]– multimedia & multimodal data fusion [WP4/WP5]

5

CASSIDIAN SAS

• Expertise– multi-source information exploitation and handling (acquisition, mining,

analysis, filtering, knowledge management)– integrated system solutions

• Role in the project – WP7 leader (System Development), PSB member

• Contribution– data mining and fusion association for early aggregation of multiple

sensor observations, early detection of typical/alarming patterns [WP3]– high-level interpretation, semantic reasoning for complex event

processing, semantic behaviour interpretation and profiling [WP5]– feasibility studies, system architecture, development & integration [WP7]

6

Institut National de Recherche en Informatique et en Automatique (INRIA)

• Expertise– video understanding, behaviour analysis, multi-sensor fusion– mobile object perception and tracking– real-time systems, system evaluation

• Role in the project – T4.3 leader (Instrumental Activities Monitoring)

• Contribution– analysis/evaluation of human-interactions/behaviors from fixed video

cameras [WP4]– modelling activities of daily living (ADLs) at home [WP4] – learning/interpretation of behaviour profiles [WP4/5]– evaluation/clinical validation pilots in France [WP8]

7

Luleå University of Technology (LTU)

• Expertise– life-logging and activity recognition– context-awareness in group communication– health science– dementia prototyping and piloting from FP6 COGKNOW

• Role in the project – WP6 leader (Client-side Interaction), PSB member

• Contribution– life-logging research and prototype from the MemoryLane project,

improved activity recognition, client side interaction components [WP6]– responsible for running the pilots in Sweden [WP8]– ethics & confidentiality, evaluation of usefulness and impact on daily

life [WP2/WP8]– exploitation planning with SMEs [WP9]

8

Dublin City University CLARITY: Centre for Sensor Web Technologies

• Expertise– sensor technology for life-logging, processing of sensor data – clinical practice (Memory Works), working with people with dementia

and technologies for dementia care– ethics and technology, clinical ethics (Institute of Ethics)

• Role in the project – WP2 leader (Requirements and Impact), PSB member

• Contribution– life-logging, lifestyle monitoring and analysis tasks [WP3/4]– running of the pilot trials in Ireland [WP8]– clinical requirements and ethical issues pertaining to remote monitoring

[WP2]

9

IBM Israel - Science & Technology LTD Speech Technologies Group, IBM Haifa Research Lab

• Expertise– speech analysis, modelling and classification– speaker identification, speech recognition and synthesis– spoken information retrieval

• Role in the project – T4.2 leader (Audio Sensing)

• Contribution– audio classification & speaker tracking [WP4]– voice-based mental & emotional state markers, dysarthric speech

detection [WP4]– text-to-speech synthesis for patient closed loop [WP6]

10

PHILIPS

• Role in the project – WP3 leader

• Contribution– health and lifestyle monitoring and analysis (signal processing, data

fusion, data mining) [WP3]– development of security and privacy mechanisms [WP7]– development of service concepts and industry cases [WP9]

11

Memory Resource and Research Centre, University Hospital in Nice (CHU DE NICE)

• Expertise– clinical research, dementia-specific

behavioural and affective disturbances

• Role in the project – WP8 leader (Pilots, Evaluation & Clinical

Validation), PSB member

• Contribution– clinical protocols & scenarios definition,

training data annotation, submission to administrative authorities [WP2]

– closed loops for people with dementia and for clinicians [WP6]

– evaluation protocols, realisation of clinical protocols with patients (pilots in France) [WP8]

12

Vistek ISRA Vision Yapay Görme ve Otomasyon Sanayi ve Ticaret A.S.

• Expertise– image and video indexing and retrieval– image and video interpretation– activity monitoring

• Role in the project – WP9 leader (Dissemination & Exploitation), member of PSB

• Contribution- tools and algorithms for feedback to people with dementia [WP6]

- feasibility study and architectural design [WP7]

- exploitation strategy and dissemination [WP9]

13

Link Care Services (LCS)

• Expertise– video surveillance– computer-based monitoring and alerting– service to dependent elderly and their caregivers

• Role in the project – T9.2 leader (Exploitation Strategy)

• Contribution– sensors and algorithms for video surveillance and alerting [WP6]– participation in pilot projects [WP8]– exploitation plan, market analysis and marketing strategy [WP9]

14

A closed loop remote care solution for people with dementia outside hospitals and care centers

Key Innovations

• multi-sensor (vital signs, audio-visual, lifestyle) monitoring and analysis of patient behaviour • comprehensive, objective

monitoring

• patient-tailored, time-evolving behaviour profiling & interpretation• customized reactive and

proactive treatment

• adaptive, contextualized feedback for personalized treatment and support

Patient Closed Loop

At home treatment

Clinician Closed Loop

Personalized Feedback

Multi-sensor Monitoring, Behaviour Profiling

Hea

lth S

tatu

s A

ss

ess

men

t

15

Overall Project Objective

• Comprehensive multi-sensor remote monitoring of daily life activities, behavioural, lifestyle patterns of people with dementia for effective at-home care and increased independence.

• Time-evolving profiling of behaviour and lifestyle patterns and intelligent multi-parametric interpretation, tailored to each person based on the conjunction of multi-sensor outputs and clinical knowledge.

• Development of closed loop for people with dementia with easy-to-use interfaces that provides personalised adaptive feedback for remote treatment and closed loop for clinicians with monitoring outputs and interpretations.

• Clinical validation through stage-wise development and evaluation• three phases of short- and long-term pilots in three countries, in

collaboration with experts on dementia.

16

• medical knowledgeassisted multi-sensormonitoring of people with dementia

• knowledge structures, reasoning, clinical expertise Þ profiling, intelligent

interpretation of behavioural patterns

• closed loops for personalised feedback to people with dementia and to clinicians

Physiology sensors Voice sensors Visual sensors

Health conditionLifestyle

Emotional stateFunctional stateMental state

Behavioural profileLifestyleDaily Activity Recognition

Clinical RequirementsScenarios definitionProtocols

Lifelogging

Medicalknowledge

Knowledge and Reasoning

High level InterpretationMedical Assessment

Feedback

Interactive feedback between clinicians and people with dementia

Dem@Care Overview

17

Project Structure (WPs)

WP1: Project Management

WP2: Requirements and Impact

WP5: Medical Ambient Intelligence

WP6: Client-Side Interaction

WP3: Physiology and WellBeing Sensor

Collection and Processing

WP4: Situational Analysis

of Daily Activities

WP9: Dissemination and Exploitation

WP8: Pilots, Evaluation and Clinical Validation

WP7: System Development

WP1: Project Management

WP2: Requirements and Impact

WP5: Medical Ambient Intelligence

WP6: Client-Side Interaction

WP3: Physiology and WellBeing Sensor

Collection and Processing

WP4: Situational Analysis

of Daily Activities

WP9: Dissemination and Exploitation

WP8: Pilots, Evaluation and Clinical Validation

WP7: System Development

18

Evaluation and Clinical Validation (Pilots)

Pilots in France

• Initial: short term in Bordeaux and Nice memory clinics• Intermediate: short term in homes and care residences in Bordeaux• Final: long term in homes and care residences in Nice

Pilots in Ireland (Dublin)

• Initial: short term in DCU campus flat• Intermediate: short term in homes• Final: long term in homes

Pilots in Sweden

• Intermediate: short term in homes and nursing residences in Luleå• Final: long term in homes in Norbotten, Västerbotten and Luleå Norbotten

Nice

Bordeaux

Dublin

Luleå

Västerbotten