Automated Sleep Tracking

using Wearable, Mobile Phone Coupled Sensors

Dr Amos Folarin

● Sleep tracking is clinically useful for a range of Mental Health disorders

● Clinical activity monitoring devices (e.g. Actiwatch) o expensive and have limited numbers of sensors, manual

data offload, technology largely behind the curve ● Consumer activity monitoring devices (e.g. Fitbit,

Jawbone)o cheap, wireless data offload, shorter battery life

● Mobile phones o wide range of sensors

● Mobile Phone linked wearable monitors could be used for:o self-monitoringo measuring response to treatment/adverse drug reactionso triggering interventiono stratifying patients e.g. clinical trials

Preamble...

BBC Horizon: Monitor Me

Blaine price, Open Uni, GadgetsEric Topolhttp://vimeo.com/72575830

Mobile Physiological Monitoring

ihealthlabs.com

Schizophrenia Relapse● Schizophrenia is a severe, chronic, relapsing condition ● Mainly managed in the community● Prompt intervention required to avoid lengthy hospital

admissions● Est. annual costs £3.9 billion to the NHS1

● Sleep dysregulation is widely recognised early sign of relapse in psychosis (often in hindsight) so monitoring of sleep-wake activity shows promise as an early relapse marker

Aims - Quantify SleepManual sleep logging is hard to do accurately and sustain!!● delay between sleep log and actual sleep● forgetting to log on/off● burdensome

Goals:● Automate prediction of sleep and wake states to improve utility of clinical

applications o "wear-and-forget"

● Measure sleep quantity (duration) & quality (restless, interruption)● Create a flexible software platform for:

o building use case specific mobile appso integrating newly available monitorso processing and reporting data

Activity phases

inactive,static location

Zzzzz...

low activity,static location

Restless

low activity,static location

Sedentary Work

high activity,dynamic location

Moving or Active Work

Hard to differentiate

purely based on activity

Devices

Fitbit Accelerometer- Steps- Activity types [light, fair, very]- Sedentary- Sleep start/end [manual marked]- Sleep Quality [restless, awake]

Fitibit Sitehttp://www.fitbit.com/ukFitbit APIhttps://www.fitbit.com/dev/devFitbit Wikihttps://wiki.fitbit.com/display/API/Fitbit+API

GALAXY S4 sensors- GPS location- Accelerometer- Gyroscope- Steps- Barometer- Proximity- Humidity- Temperature- Light

PR App: https://play.google.com/store/apps/details?id=edu.northwestern.cbits.purple_robot_manager Purple Robot Docs.http://tech.cbits.northwestern.edu/2013/10/purple-robot-importer-purple-robot-warehouse/

Has a mature API for programmatic data access!

Fitbit Data Catalogue (Accelerometer Probe)

Fitbit Data

id, timestamp, eventDateTime, insertedTime,TIMESTAMP

ACTIVE_SCORE, ACTIVITY_CALORIESMARGINAL_CALORIES,

SEDENTARY_MINUTES, SEDENTARY_RATIO,LIGHTLY_ACTIVE_MINUTES, LIGHTLY_ACTIVE_RATIO, FAIRLY_ACTIVE_MINUTES, FAIRLY_ACTIVE_RATIOVERY_ACTIVE_MINUTES, VERY_ACTIVE_RATIOSTEPS, TOTAL_DISTANCE,

SLEEP_MEASUREMENTS_DT_AWAKENINGS_COUNT, SLEEP_MEASUREMENTS_DT_AWAKE_COUNT, SLEEP_MEASUREMENTS_DT_DURATION**, SLEEP_MEASUREMENTS_DT_MINUTES_ASLEEP, SLEEP_MEASUREMENTS_DT_MINUTES_AWAKE, SLEEP_MEASUREMENTS_DT_MINUTES_IN_BED_AFTER, SLEEP_MEASUREMENTS_DT_MINUTES_IN_BED_BEFORE, SLEEP_MEASUREMENTS_DT_RESTLESS_COUNT, SLEEP_MEASUREMENTS_DT_TIME_IN_BED,

Fitbit (Manual data)Slots for many other manually inputed data

(not listed here for brevity, but includes things like food, weight, heart rate, blood pressure etc..)

Fitbit "always worn" continuous measure of ⇒activity vs. patchy phone accelerometer

Hardware Sensor Probes

Accelerometer (m/s^2)

Gyroscope (miliradians per sec, 3x axes)

Location (lat, lon, altitude, speed)

Pressure (on touch-screen)

Light (lux)

Ambient Temperature (c)

Ambient Humidity (%)

Proximity (phone distance from objects [cm])

Magnetic Field (micro-Tesla)

External Devices Probes

Visible Bluetooth

WiFi

Media Router

External Environment Probes

Visible Satellites

Current Weather Conditions

Sunrise & Sunset (calc day/night depending on geo location)

Personal Information Probes

Significant Location Distances(calc from local address book)

Call & Message info

Communication Events

Date Calendar

Call History Stats

External Services Probes

Google Places

Fitbit Measurements

Facebook

Twitter

Instagram

LinkedIn

Foursquare

Purple Robot Probes Catalogue

probes provided with purple robot are quite diverse (phone dependant).

e.g. LocationProbe table includes these columns:id, timestamp, eventDateTime, insertedTime, ACCURACY,GPS_AVAILABLE, GUID, LATITUDE, LONGITUDE, NETWORK_AVAILABLE,PROVIDER, TIMESTAMP, TIME_FIX, ALTITUDE, BEARING, SPEED,CLUSTER

With all probes 'on', a GS4 handset would generate > 1GB /day

Fitbit Device

Android Phone

SLaM Sleep App

Fitbit App

Purple Robot App

<configures>

OAuth Process, Sleep Classifier, Reports, Dashboard

Fitbit data flowPurple Robot data flow

Fitbit ServerPurple Robot

Warehouse

ingest

config.scm

Purple Robot (PRI/PRW) Data Flow

Purple Robot App

Sample Set (JSON)

Purple Robot Importer

Purple Warehouse(PostgreSQL)

SQL

Analysis & Visualization

SQL queryR, MATLAB, SAS, Dashboard, Custom App etc

ingest one postgres database per user id-hash

cache

emit

SLaM Sleep App

PR App Dev FrameworkFramework:● Mobile App development tools (PhoneGap)● "talk to" PR app, e.g modify probe config.● Probe (i.e. sensor) interface mechanisms

Trigger an Action e.g. questionnaire ● Date -- fire at preselected intervals (specified in standard iCalendar format)● Sensors -- fire on matching predefined pattern (or learned model)

core.brc.iop.kcl.ac.uk

northwestern.edu

SLaM Sleep App

Purple Robot

Warehouse

Purple Robot

Warehouse

Future

for currenttesting

https://github.com/KHP-Informatics/slam_sleep_test

even packaged into an Amazon EC2 AMI image

Preliminary Data:

Toy Dataset

● From two group members, ~1 month● Data was collected using Purple Robot and

the Fitbit ● Manually log each night start of sleep and

end of sleep● Attempt to see if we can classify the

manually marked sleep state.

Fitbit a 24hr sliceAwake Start (manual)

Sleep Start (manual)

PR → R

Purple Robot

Warehouse

RPostgreSQL

Sensor Table dataframes

Sensor Table dataframes

1. sort by timestamp2. timestamp → as.POSIXct date

3. merge on "timestamp", "event_Date"

4. zoo package na.approx for interpolation (handy time series object too)5. runmed for median filter (?)

merged Sensor Table dataframes

interpolated Sensor Table

timeseries

Machine LearningClassificationetc...

https://github.com/KHP-Informatics/slam_sleep_r

Pre-processing

● Epoch alignment each table o Only fitbit and location probe for now o JOIN tables on timestamp

● Interpolationo Probes not synchronised, so interpolation requiredo However, interpolation may smudge boundaries of

SLEEP_MEASUREMENTS_DT_DURATION (our sleep log)● Wake|sleep state overrun

o stripped out with heuristic filter

Sleep Log Variable

● Double tap on fitbit to log sleep start & end● SLEEP_MEASUREMENTS_DT_DURATION (total

millisecs) for last sleep period (0 or >>0)● k-means, cluster into 2 classes

sleep=0, wake=1

Sle

epW

ake

sed

enta

ry m

ins

light

ly a

ctiv

e

min

sm

oder

ate

a

ctiv

e m

ins

hig

hly

activ

e m

ins

Lat

itude

Spe

ed

GPS

?

Analysis

Analysis

Goals:1. Construct a predictor that classifies sleep or wake states, based on

the range of signals collecteda. automate est. of duration of sleep

2. Look at the quality of sleep measures (restlessness, interruptions)

Data:● Data from 2 group members wearing fitbits and galaxy S4 + purple

robot app● upto ~1 month of data in each case● Subset of probe data used (fitbit and location)

Sleep-Wake classifierx "timestamp", "LATITUDE", "LIGHTLY_ACTIVE_MINUTES", "ACCURACY", "SPEED", "FAIRLY_ACTIVE_MINUTES",

"SEDENTARY_MINUTES", "VERY_ACTIVE_MINUTES", "VERY_ACTIVE_MINUTES", "event_Hour"

y SleepWake [0=sleep, 1=wake]

n = random 10,000 timepoints from person 1

classifier <- train(x,y,'nb', trControl=trainControl(method='cv', number=10))

Resampling: Cross-Validation (10 fold)

Resampling results across tuning parameters:

usekernel Accuracy Kappa Accuracy SD Kappa SD FALSE 0.833 0.658 0.00984 0.0202 TRUE 0.958 0.915 0.0069 0.0141

n = random 2,000 timepoints from person 2

classifier <- train(x,y,'nb', trControl=trainControl(method='cv', number=10))

Resampling: Cross-Validation (10 fold)

Resampling results across tuning parameters:

usekernel Accuracy Kappa Accuracy SD Kappa SD FALSE 0.819 0.627 0.0249 0.0501 TRUE 0.932 0.848 0.0244 0.0544

Predicted

0 1

Actual 0 25563

131

1 1724 19348

sleep-wake classificationActualPredicted

Predicted

0 1

Actual 0 2785 148

1 420 5689

person 1

person 2

Some Early Thoughts● Improve classifier

o move beyond a toy training dataseto errors clustered around Sleep-Wake boundary

problem with sleep log accuracy or interpolation effect?

● Can probably improve by considering a time-series window rather than instantaneous classification

● GPS data can periodically be noisy -- why?o location of sleeping typically constrained geographically so quite

usefulo look at GPS "accuracy" metric provided in LocationProbe tableo changed GPS sensor from: moderate → high accuracy (gps + wi-fi +

mobile-network)

● Incorporate other sensor values

Next Stepsother devices and signals

Basis Monitor: advanced sleep analysis

New monitors now regularly appearing on market- heartrate- skin temperature- perspiration- actigraphy

→ Automated sleep classification→ REM, Light, Deep, interruption

"Advanced Sleep Analysis"

however Basis does not have a Formal API….at the moment anyway

http://www.mybasis.com/

a Portable Sleep Lab?

Polysomnography Traces

Basis vs Polysomnography

correlation (r = 0.92)

Deep REM Light

http://www.mybasis.com/wp-content/uploads/2014/04/Validation-of-Basis-Science-Advanced-Sleep-Analysis.pdf

http://www.huffingtonpost.com/dr-christopher-winter/sleep-tips_b_4792760.html

We now want to properly test some Mobile Monitor use cases:

1) First, some feasibility and validation studies● Will patients wear these things..?● Validate against current gold standards (actiwatch, polysomnography)

2) Clinical utility● Clinical detection of relapse based on sleep monitoring● Monitoring in the community● Patient self-monitoring● Targeted intervention for clinical teams

Schizophrenia Relapse and Sleep

Acknowledgments http://core.brc.iop.kcl.ac.uk

InformaticsDr Stephen NewhouseDr Caroline JohnstonDr Zina IbrahimDr Richard J Dobson

App DevelopmentMark BegaleChristopher KarrProf. David Mohr Center for Behavioural Intervention

Technologies CBITs

ClinicalDr Nick MeyerProf. Till WykesProf. James MacCabe

References[1] Andrews A, ; Knapp, M.; McCrone, P.; Parsonage, M.; Tractenberg, M. Effective interventions in schizophrenia the economic case: A report prepared for the Schizophrenia Commission. London: Rethink Mental Illness, 2012.

END