het collectief centrum van de Belgische technologische industrie

Tom Tourwé, Elena Tsiporkova (Sirris, BE)

Ville Antila (VTT, FI)

Monique Hendriks (Philips, NL)

Context-aware Personal Attentive Systems

Smarcos – Goal & motivation

• Smarcos aims to improve the interusability of interconnected

embedded devices

• Smarcos relies on the solid technological basis for system-

level interoperability resulting from other EU R&D projects

• Eg. ARTEMIS-Sofia and FP7-SmartProducts

• Smarcos addresses challenges arising from user-level

interaction of interconnected systems

• Cross-platform interaction

• Multi-user and multi-device collaboration

• Synchronisation of system actions to maintain fluent task flow

• Application and service adaptation to changing situational

contexts

Smarcos - Research topics

Design methods for embedded UIs

Interface, interaction & context models

Runtime UI & context composition

UI migration

(Distributed) Context processing

Distributed functionality

Model-based UI generation

Personal attentive systems

• People have difficulty incorporating healthy habits into their

daily life

• Office workers: working on a computer, prolonged sitting,

deadlines, meetings, travel by car

• Diabetes type II patients: monitoring of glucose levels, effects of

food, medicine adherence, physical activity, learning to cope with

the disease

• Doctors and lifestyle coaches can provide advice, but they

cannot

• Intervene at the right moment

• Provide feedback on behaviour directly afterwards

• Have intricate knowledge of people‟s daily routine

PAS in Smarcos

Monique – can you provide a

higher quality picture? Can it

be an alternative one with

real devices?

Context Interpreter

Decision Engine

Context

Reasoner

Context

Models

Data Layer

Conceptual architecture

Capture raw data from device

sensors

Transform raw data into infor-

mation about the user context

Infer new knowledge from

information

Determine content, timing &

modality of feedback

Life-pattern detection

through life logging

• Low-power context logging software

• Semantic location detection using cell-id

(low power, always available)

• Device usage detection (algorithms for

mining location relative to smartphone

application usage)

Data Layer

Ville: can you explain the “semantic” part?

The user needs to provide the semantics, right?

Detecting important

places and daily routines

• Estimation of life patterns

such as the semantic

location of the user (e.g.

“home”, “office”)

• Detection of device usage

in different locations

• Detection of physical

activity in different

situations

Context Interpreter

Can be used to …

• Determine the devices that surround the user

• e.g. at work, the user has access to his personal computer

• Time and adapt system feedback based on the situation

• e.g. time-shift notification to where user is more receptive

• Log context changes and react appropriately

• e.g. user entering an interesting location/situation such as „office‟

Context Interpreter

Domain knowledge for PAS

• Such domain knowledge is any factual information about

persons, activities, locations and devices which influences the

decision about at what time, on which device and through

which modality feedback should be delivered to the user

• Such domain knowledge is captured in a context model

Context Models

Example domain knowledge

• If the user is cycling then

• available devices are mobile devices (phone & activity monitor)

• availability for receiving messages is low

• A pill reminder message is

• an urgent message

• best delivered through the audio modality

Context Models

Infer new knowledge from

information

• The goal of the reasoner is to classify devices as being ACCESSIBLE,

AVAILABLE and/or USED

• using the OWL reasoner and SWRL rules

• ACCESSIBLE > AVAILABLE > USED

• A device is an ACCESSIBLE_DEVICE if the user has access to it

• A device is an AVAILABLE_DEVICE if it is an ACCESSIBLE_DEVICE which is

ON

• A device is a USED_DEVICE if it is being used by a user

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

THING

elenas_television THING

elenas_livingroom

is_performed_by

is_performed_on

is_accessible_from

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

THING

elenas_television THING

elenas_livingroom

is_performed_by

is_performed_on

is_accessible_from

performs

provides_access_to

Inverse properties

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_television THING

elenas_livingroom

is_performed_by

is_performed_on

is_accessible_from

performs

provides_access_to

WATHINGTELEVISION is_performed_on only TELEVISION

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

TELEVISION is_accessible_from only HOME

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

performs o is_performed_on -> uses

uses

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

Inverse properties

uses used_by

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

uses o is_accessible_from -> is_located_in

uses used_by

is_located_in

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

Inverse properties

uses used_by

is_located_in

is_location_of

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

USED_DEVICE

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

USED_DEVICE used_by min 1 USER

uses used_by

is_located_in

is_location_of

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

AVAILABLE_DEVICE

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

AVAILABLE_DEVICE is superclass of USED_DEVICE

uses used_by

is_located_in

is_location_of

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

ACCESSIBLE_DEVICE

elenas_television HOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

ACCESSIBLE_DEVICE is superclass of AVAILABLE_DEVICE

uses used_by

is_located_in

is_location_of

is_performed_by

performs

Context Reasoner

Determine content, timing

& modality of feedback

• IF the situation is suitable, THEN send the message (as it is)

• IF the situation is not suitable AND the message is not urgent,

it should be time-shifted

• IF the situation is not suitable AND the message is urgent,

then the content should be adapted to the situation

Decision Engine

as_is(Message, User, Device) :-

send_message(User, Message),

user_availability(User, high),

has_access_to(User, Device),

supports(Device, Modality),

requires_modality(Message, Modality).

adapt(Message, User, Device, Alternate_Modality) :-

send_message(User, Message),

urgent(Message),

has_access_to(User, Device),

requires_modality(Message, Preferred_Modality),

not(supports(Device, Preferred_Modality)),

supports(Device, Alternate_Modality).

Conclusion

• Presentation mainly reports on research results obtained by

VTT and Sirris within the PAS use case defined by the Dutch

consortium

• Future research

• Implementation of the decision engine in a Java-based logic

engine

• Integration of all individual components in a common prototype

• Explore distributed context processing application

• Carry out field studies to validate the requirements