Transforming

Standard

Homes

Into

Cognitive

Prosthesis

Pervasive

Assistance for

Cognitively I

mpaired People

Sylvain

Giroux

Plan

Context

Objectives and Approach

Pervasive and mobile computingTangible User interface

From homes…

Hardware level >>> networks, sensors, effectors

… to smart homes …

Middleware level >>> pervasive infrastructure

… to smart care !

Application level >>> cognitive assistance & tele-monitoring

Validation

Usability and clinical studies

Conclusion

ContextP

eople suffering of cognitive impairments in Quebec Alzheimer disease : 5.1% of people over 65 years old Head trauma : 3000 new cases each yearSchizophrenia : 1% of the population

In many cases, they would be able to stay at home

if light assistance was provided.

But healthcare resources are scarce. So relatives have to take responsibility for care.

It then turns to an exhausting burden.Hence relatives and caregivers urge for help.

Objectives

Provide adapted and personalized environmental cues to

Foster the autonomy of cognitively impaired people

Reduce risks and hazards

Pervasive computing & Tangible user interfaces

Keep ensuring cognitive assistance outside people’s home

Mobile computing & Location-based services

Help relatives and caregivers to stay in touch at distance with cognitively impaired people

From homes…S

mart homes are augmented environments

Heterogeneous networks

Sensors networks

Embedded processors in appliances, clothes, jewels…

Information devices

Networked communicating objects

DOMUS: an augmented apartment

… smart homes …S

mart homes are augmented environments

Heterogeneous networks

Wireless : WiFi, Bluetooth, RFID, UWB...Wired : Ethernet, Electrical wires, X10, power line…

ServersFull control over audio and video streams

… smart homes …S

mart homes are augmented environments

Sensors networks

Identification and localization of objects and people

Ubisense tags, UWBSmart tags (RFID)

… smart homes …S

mart homes are augmented environments

Embedded processors in devices and clothes

… smart homes …S

mart homes are augmented environments

Information devicesFixed: smart boards, Icebox… Mobile: laptop, wireless screen, PDAs…

… smart homes …S

mart homes are augmented environments

Networked communicating objectsSightHearing

Smell (not yet investigated)Touch (not yet investigated)

Taste (is it possible ?)

From

homes to smart

homes

Smart home : the middleware

Some issues investigated at DOMUS

Spontaneous networkingHeterogeneous networksAutonomic computingMobile code and agentsLocation and context awareness

Security and privacy (not yet)

Some prototypes towards a pervasive infrstructure

A pervasive reminder systemMulti-channel delivery of services

A

Pervasive

Reminder

System

for

Smart

Homes

How to localize a user from simple sensors information ?

How to achieve pervasiveness?

Follow-me : transparent user friendly migration of sessions

How to use spontaneous networking and service discovery to build zero-configuration system ?

How to cope with heterogeneity of devices, networks, and OS ?

How to keep the system clean?

Multi-channel

delivery of services

On-the-fly generation of user interfaces from raw code

Means to control complex interactions with a user

Ready to use service delivery infrastructure

… to smart careS

mart homes can assist cognitively impaired people foster their autonomy

The whole home then becomes a true cognitive prosthesis

At distance, smart homes can

help caregivers to grant better care give a sense of security to residents and their relatives

Cognitive assistance in smart environments

What is the available information ?

Identification and localization of people and objectsObjects involved in an activityPrimitive actions performed by the resident

Who the resident is ?

Personalization

What is the user doing ?

Activity recognition

How to assist the resident ?

Taditional user interfacesTangible user interfaces

Activity recognition I

Based on plan recognition

Hierarchical descriptions of tasksEpitalk, Markov models, etc.Monitoring of one activity by one personThe assistance system reacts after the user “error”

Lattice-based modelsManage concurrent activities recognitionPredict the user behaviour (and ideally anticipating his “errors”)Disambiguate activities from observed eventsIdentify opportunities for assistance

Activity recognition II

Based on involved objects

Perkowitz et al., Mining Models of Human Activities from the Web, WWW 2004, May 17-22, 2004, New York, NY USA.

…enhanced with contextual information

Personalization

User modelling

Health status, cognitive deficits to address…Tasks to monitorRelatives and caregivers networketc.

Cognitive modelling based on episodic memory

To know from life habits, how one usually performs an activity

method usually used to achieve a taskestimations on time (average time of completion…)most likely location to perform a task

Towards interactive environments

Interactive maps

9

Towards interactive environments

>> Show-me objects + « Follow-me » applied to objects

The lamp turns off when the object is too far away

The lamp turns on to highlight the searched red book

Prototype I Pervasive cognitive assistance

In collaboration with France TelecomJean-Pierre Savary, Projet SynapsePatent pending

Cognitive

Assistance

What kind of cognitive deficits can we address ?

Initiation : inactive periods whereas the person is supposed to perform actionsMemory : difficulties to remember the activity to perform, the steps, the locations of tools and materials involvedPlanning : difficulties to perform an appropriate sequence of actions in the right order to achieve a goalAttention : shifts of attention from the activity under progress to a stimulus causing interference

How to interact with the resident ?

Traditional GUI vs Tangible user interfaces : Transforming the whole house into a cognitive prosthesis

Who initiate the interactions?

Resident vs Assistant

Telemonitoring

Asynchronous vs Synchronous

Prototype II Using a PDA-based personal agenda as a

tool for

cognitive assistance

tele-monitoring

medical assessment

coordination of caregivers

Architecture

Data

Server

Workstation

Ethernet WirelessIEEE802.11

BluetoothGPS

Using a personal agenda for cognitive assistance:

the user perspective

Reminding activities of daily living (ADLs)Confirming a given ADL is performed

Using a

personal agenda

for

medical assessment:

the user perspective

Monitoring side-effects of medication

Using a

personal agenda

for

medical assessment:

the user perspective

Asking for help from caregivers

Using a

personal agenda

for remote

monitoring:

the caregiver perspective

Monitoring many patientsMonitoring one patient activities

Monitoring one activity of a patient & caregiver coordination

Activities of daily living:the caregiver perspectiveSpecifying activities of daily living (ADLs)Assigning to patients ADLs to monitor

Usability and clinical studies

Fernand-Séguin Research Center (June 2006)L-H Lafontaine psychiatric hospital, Montreal

Gathering ecological dataMonitoring ADLs

3 schizophrens + 3 caregivers (occupational therapists or social workers)

DOMUS Laboratory (September 2006)

Pervasive cognitive assistance

Intellectual disabilities

Center for rehabilitation Estrie (November 2006)

Gathering ecological dataMonitoring ADLs Assistance outside the home using location-based services

Head trauma

Conclusion

Pervasive computing and tangible user interfaces can help to transform home into smart homes adapted to cognitively impaired people

Going beyond the usual view of computing as “PC-based”

Pervasive computing enables a seamless integration of assistance in residents’ everyday life

Going beyond traditional human computer interfaces

TUI helps to turn the whole house into a cognitive prosthesis

Smart environments and mobile applications can

Foster people’s autonomy

Lead to smarter care

Future

works

Radical changes are needed

transform and adapt the environment for cognitively impaired people

as our society did in the early 80s for physically disabled people

Change the healthcare approach by providing continuous feedback to people to foster prevention and better life habits

Cognitive assistance and telemonitoring can be the core part of the solution

Initiation deficitsInitiation deficits leads to inactive periods

whereas the person is supposed to perform actions

Wandering for a long time could be attributed to an initiation deficit

>> Prompt the resident

Planning deficitsD

ifficulties to perform an appropriate sequence of actions in the rightorder to achieve a goal.

>> Prepare_coffee = {take_milk, take_cup…}

>> Show to the user where to perform the next action

Attention deficitsS

hifts of attention from the activity under progress

to a stimulus causing interference

The current activity may be forgotten and never completed

>> Remind the activity under progress to the resident

Our team

Researchers from the Faculties of science, engineering, and administration

Sylvain Giroux, Ph. D. in Computer scienceHélène Pigot, Ph. D. in Computer science and B. in occupational therapy André Mayers, Ph. D. in Computer science and M. inpsychologyPhilippe Mabilleau, Ph.D. in engineeringClaude Caron (geo-business)

Analyst

Francis Bouchard

Students

6 Ph. D. students12 M. Sc. students6 B.Sc. students, 2 international trainees (M. Sc. level)

Some collaborations

CRE, Centre de réadaptation Estrie Centre de recherches Fernand Séguin, Computer scienceUniversité Joseph Fourier, Grenoble

France TelecomAriane Controls, Canada

Thank you for your attention

http://domus.usherbrooke.ca/