Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion

Smartphones as ubiquitous devices for behaviour analysis

and better lifestyle promotion

PhD Student: Matteo CimanSupervisor: Ombretta Gaggi

Outline

Ubiquitous Computing

4


4

The most profound technologies are those that disappear. They

veal themselves into the fabric of everyday life until they are indistinguishable from it


5

Ubiquitous computing for health

6

Health costs

7

Lifestyle problem

8

Ubiquitous computing for health

9

Wearables for health

10

Ubiquitous Computing for health

11

Smartphones for healthcare

12

Smartphones for healthcare

12

Gyro Sensor

Proximity Sensor

AccelerometerGeomagnetic

Sensor

Light Sensor

Barometer

Temperature Sensor

Cross-Platform frameworks

Users in health applications

14

Market fragmentation

15

Aim at reducing the problem of market fragmentation and different OS and APIs

Write once, run everywhere approach

Four different architectures

Cross-platform frameworks

16


17


17

Web Approach


17

Web Approach Hybrid Approach


17


Interpreted Approach


17


Cross-Compiled ApproachInterpreted Approach

Frameworks analysis

18

Frameworks analysis

18

Developer Point-Of-View

Frameworks analysis

18

User Point-Of-ViewDeveloper Point-Of-View

Game with animations, sounds, etc., developed with all the analysed frameworks.

Considered aspects:

Licenses

APIs

Community/Support

Code Complexity

Supported devices

etc.

Developer point-of-view

19

Game with animations, sounds, etc., developed with all the analysed frameworks.

Considered aspects:

Licenses

APIs

Community/Support

Code Complexity

Supported devices

etc.

Developer point-of-view

19

jQuery PhoneGap Titanium MoSync

API 2 2 4.5 3

Complexity 4 4 5 2

IDE - - 5 4

Devices 5 4 4 2

20

User point-of-viewObjective analysis of energy consumption for data acquisition from smartphone sensors.

Tests with two Android and two Apple devices.

Results:

Cross-platform frameworks require higher energy from battery

Different frameworks categories employ different consumption

Smartphone for stress assessment

Mood recognition with smartphones

22

The idea

23

Human-smartphone interaction

24


25


25

Tap

Scroll

Swipe

Text WritingDouble Tap

Rotate

Zoom

Pinch

Long press

Search & Write Task

26

Protocol

27

Initial Relax

(5)

RelaxedTasks(~30)

Self Assessment

Negative ValenceLow energy

Not Stressed

Positive ValenceHigh Energy

Stressed

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

ESM 1 ESM 2 ESM 3 ESM 4 ESM 5

Stressor(5-10)

StressedTasks(~10)

Mathematical problems

Timing pressure

Uncontrollability

Repetition

Stressor Task

28


Timing pressure

Uncontrollability

Repetition

Stressor Task

28

Big primenumber


Timing pressure

Uncontrollability

Repetition

Stressor Task

28


Timing pressure

Uncontrollability

Repetition

Stressor Task

28

Randomdecrease

Vibrations and sound

Stress Induction Analysis

29

Initial Relax

RelaxedTasks

Stressor StressedTasksESM 1 ESM 2 ESM 3 ESM 4 ESM 5

Stress Induction Analysis

29

Initial Relax

RelaxedTasks

Stressor StressedTasksESM 1 ESM 2 ESM 3 ESM 4 ESM 5

TEST t(13) p-value

ESM 3 VS ESM 4 1.99 0,007 *

ESM 3 VS ESM 5 -2.84 0,009 *

ESM 4 VS ESM 5 -2.74 0.5

Participants were stressed

Different stress level at the end of tasks

Kept stressed during stress tasks

Scroll and Swipe prediction model

30

F-measure for Scroll gesture models

MODEL DT KNN SVM NN BN

USER (AVERAGE) 0.79 0.80 0.81 0.80 0.77

GLOBAL (AVERAGE) 0.73 0.71 0.78 0.74 0.67

F-measure for Swipe gesture modelsMODEL DT KNN SVM NN BN

USER (AVERAGE) 0.86 0.86 0.79 0.87 0.85

GLOBAL (AVERAGE) 0.92 0.75 0.81 0.82 0.77

User Model: Digits size (64% of users with strong correlation) Pressure/Size ratio (55% of users with strong

correlation)

Global Model: Wrong Words / Total words ratio (p-value =

0,028) Digits time distance (p-value = 0,012) Digit duration (p-value = 0,08)

Text writing statistical significance

31

Activity Recognition and Serious Games for behaviour change

The Piano Stairs Project

33

From stairs to smartphone

34

Stair steps recognition in real-time

Single player and multi players mode

Both competition and collaboration with other players

ClimbTheWorld

35

ClimbTheWorld

36

Data collected from 7 different users with their own smartphone

8000 windows, 1500 stair steps

Different methods from gravity removal, different data frequencies and different learning algorithms

Stairstep recognition

37

Data collected from 7 different users with their own smartphone

8000 windows, 1500 stair steps

Different methods from gravity removal, different data frequencies and different learning algorithms

Stairstep recognition

37

0.65

0.7

0.75

0.8

0.85

0.9

Mizell Linear OurMethod



20Hz 30Hz 50Hz

F-score

DT KNN KOMD

Energy consumption

38

11500

12000

12500

13000

13500

14000

14500

15000

Slidingwindow Datasegmentation

Energyconsumption(Ah)

} 1 hour of batterylife

User test with 13 participants

Divided into two groups, each one followed a particular program of game usage

9 days length, the firsts and the last two days using only a stair step counter without the game

Questionnaire for user engagement, logger to understand the number of stair steps made depending on the game mode

User engagement and behaviour change

39

Serious games for medical diagnosis

Children as Digital Native Speakers

Serious games and mobile devices for diagnosis tests instead of normal ones designed for adults

More engaging games can make tests longer and more precise

Children and technology

41

PlayWithEyes

42

Tested with 65 children

Evaluation both from children and diagnosis point of view

Two children identified with eye problems

PlayWithEyes

43

Tested with 65 children

Evaluation both from children and diagnosis point of view

Two children identified with eye problems

PlayWithEyes

43

3 years old 4 years old 5 years old

First eye (left) 427 513 342

Second eye (right) 302 306 315

Total Time 729 819 657

Conclusions & Future works

Analysis of cross-platform frameworks for mobile analysis

Frameworks still not mature for a strong adoption

User Interface update is the most expensive task

Future works

constantly monitor frameworks evolution

Cross-platform frameworks

45

Stress assessment using human-smartphone interaction, in particular using scroll, swipe and text writing

F-measure between 73-85% for stress recognition

Future works

In-the-wild approach monitoring behaviour with smartphone

Stress reduction strategies using mobile devices

Stress assessment

46

Serious games are a good strategy for behaviour change

Medical diagnosis can reach same results but in a more engaging way

Future works

evaluation of long term effects of serious games for behaviour change

Serious games usage for diagnosis or rehabilitation for different diseases

Serious games for health

47

Smartphones as ubiquitous devices for behaviour analysis

and better lifestyle promotion

PhD Student: Matteo CimanSupervisor: Ombretta Gaggi

Thank You!

Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion

Education

Transcript of Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion