Addressing Stress and Addictive Behavior in the Natural Environment Using AutoSense Santosh Kumar...

Addressing Stress and Addictive Behavior in the Natural Environment

Using AutoSense

Santosh Kumar Computer Science, University of Memphis

Our Team

Behavioral Science Engineering

04/21/23Santosh Kumar, University of Memphis2

Dr. Mustafa al’Absi, UMN Dr. J Gayle Beck, Memphis Dr. David Epstein, NIDA, NIH Dr. Tom Kamarck, Pittsburgh Dr. Satish Kedia, Memphis Dr. Kenzie Preston, NIDA, NIH Dr. Marcia Scott, NIAAA, NIH Dr. Saul Shiffman, Pittsburgh Dr. Annie Umbricht, Johns

Hopkins Dr. Kenneth Ward, Memphis Dr. Larry Wittmers, UMN

Dr. Anind Dey, CMU Dr. Emre Ertin, Ohio State Dr. Deepak Ganesan, UMass Dr. Greg Pottie, UCLA Dr. Justin Romberg, Georgia

Tech Dr. Dan Siewiorek, CMU Dr. Asim Smailagic, CMU Dr. Mani Srivastava, UCLA Dr. Linda Tempelman, Giner

Inc. Dr. Jun Xu, Georgia Tech

Students & Postdocs

Memphis CMU, OSU, UCLA, Georgia Tech., UMN


Dr. Andrew Raij (now at USF)

Dr. Kurt Plarre Dr. Karen Hovsepian Amin Ahsan Ali Santanu Guha Monowar Hussain Somnath Mitra Mahbub Rahman Sudip Vhaduri

Dr. Motohiro Nakajima, UMN

Patrick Blitz, CMU Brian French, CMU Scott Frisk, CMU Nan Hua, Georgia Tech Taewoo Kwon, OSU Moaj Mustang, UMass Siddharth Shah, OSU Nathan Stohs, OSU

Paradigm Shift in Disease Prevalence


Infectious diseases, and those from poor hygiene & nutrition not as prevalent

They are replaced by diseases of slow accumulation Heart diseases Cancer, Ulcer Depression, Migraine

Growing Epidemic – Stress & Addiction


Stress & addictive behavior lead to or worsen diseases of slow accumulation Stress: headaches, fatigue, heart failures,

hypertension, depression, addiction, anxiety, rage

Smoking: cancer, lung diseases, heart diseases Yet, both continue to be widespread

Stress: 43% adults suffer adverse health effects Smoking: responsible for 20% of deaths in US

An urgency to help individuals reduce stress & abstain from addictive behavior

Addressing Stress & Addiction


An unobtrusively wearable sensor suite called AutoSense So, individuals can wear it in natural environment

Robust inference of stress from physiological measures Automatically measure physiological and psychological stress

Automatic inference of addictive behaviors Smoking, drinking, drug usage from sensor measurements

Detect addiction urges to provide timely intervention Craving for smoking and drug usage Contexts/cues that may lead to craving and eventual relapse

Infer other moderating behavioral & social contexts Conversation, physical activity, traffic stressors, etc.

Outline


Hardware and Software Platforms AutoSense sensor suite FieldStream mobile phone framework

Inferring Stress Detecting stress from physiology Predicting perceived stress

Ongoing User Studies Detecting smoking, drinking, craving, drug usage,

etc. Roadmap & Long-term Vision

AutoSense Wearable Sensor Suite


Chestband sensors: ECG, Respiration, GSR, Ambient & Skin Temp. , Accelerometer

Armband sensors: Alcohol (WrisTAS) , GSR, Temp., Accelerometer

Android G1 Smart Phone

Key Features of AutoSense Hardware


Ultra low power Six sensors (ECG, GSR, Resp., Temp, Accel) consume 1.75 mA Overall current consumption < 3mA (for 10+ days of lifetime)

Sampling and transmission of 132 samples/sec (i.e., 1.8 kbps)

Reliable radio ANT with integrated quality of service and duty cycling

Reliable and timely wireless transmissions in crowding scenarios Antenna impedance is matched for human body

Power loss reduced from 33% (for free space configuration) to 0.1% Operates at 2480-2524 MHZ band to be immune to Wi-Fi

Average packet loss rate of 0.57% even when Wi-Fi activity is intense

FieldStream – Mobile Phone Framework


For use in conducting scientific user studies In both supervised lab settings and in uncontrolled field

settings It collects measurements

Sensor measurements from wearable and phone sensors Self-reports from subjects

Computes tens of features and various statistics over them (e.g., HR, HRV, RR, Minute Ventilation)

Makes inferences using machine learning algorithms Stress, posture, activity, conversation, and commuting Detects sensor detachments and loosening

Is reconfigurable So, no need for change in source code for use in a new user

study


Converts stream of sensor measurements into packets & delivers to intended recipient

Provides a common interface to all sensors & populates buffers for feature computation

Computes base features (e.g., R-R interval) & statistics over them

Deployment Experiences and Findings


21 subjects in UMN - completed Lab session on stress; 10-14 hours per day for 2 days in

field 36 subjects in Memphis - completed

3 consecutive days in field with daily visits to the lab

Some findings on human behaviors in our subject pool Stress occurrence in daily life (Plarre et. al., in ACM IPSN’11)

Subjects were psychologically stressed 26-28% of time Natural conversations (Rahman et. al., in ACM Wireless

Health’11) Frequency of conversations : 3 per hour Avg. duration of a conversation: 3.82 minutes Avg. Time between conversations: 13.3 minutes

Outline






Measuring Stress in the Field Self-reports have been used for a long time

Questionnaires or surveys Measures perceived stress

Strengths and limitations (+) Captures detailed information (+) Proximal predictor of mental health (-) Distal predictor of physical health (-) Discrete sampling (-) Burden to participant

Need an automated approach for continuous stress measurement in the field

14 Santosh Kumar, University of Memphis 04/21/23

Continuous Measure of Stress


Can use physiological measurements to assess stress, but

Physiology is affected by several factors, not only stress

How to map physiology to psychology?

Activity, change in posture, speaking, food, caffeine, drink, etc.

How to separate out the changes in physiology due to stress?

The Quest for Automated Stress Measure

Predicting psychological state from physiology William James – pioneering work (1880) John Cacioppo and others – revitalized interest (1990)

Several studies on emotion and stress prediction Identified physiological markers of stress and emotion

Example: Heart rate, skin conductance response But, confined to controlled settings

Few studies in uncontrolled environments M. Myrtek’96 , J. Healey’05, J. Healey’10, Either no validated stressors, no lab session to train

models, not able to account for confounders, or tried to match self-reports directly


In the AutoSense Project We developed a new wearable sensor suite Conducted a scientific study with validated stress

protocol 21 participants, 2 hour lab study, 2 day field

study Protocol designed by behavioral scientists Stressors used are validated and known to produce stress Self-reports designed by expert behavioral scientists

Developed new stress models to measure Physiological response to stress

To measure adverse physiological effects of stress Perception of stress in mind

To derive a continuous rating of perceived stress


Lab Study – Stress Protocol 2 hour lab session

Subjects exposed to three types of stressors Public speaking – psychosocial stress Mental arithmetic – mental load Cold pressor – physical stress

Physiological signals recorded at all times Using AutoSense

Also, collected self-reported stress rating 14 times

18

Baseline

10 Min 10 10 4 4 4 4 4 4 4 10 10 104 555

RecoveryPublic

Speaking Cold

Pressor

Start End

Mental Arithmetic

Santosh Kumar, University of Memphis 04/21/23

Self-Report Measures of Stress

Self-report questions related to affective state

Santosh Kumar, University of Memphis19

Question Possible Answer Code

Cheerful? YES yes no NO 3 2 1 0

Happy? YES yes no NO 3 2 1 0

Frustrated/Angry? YES yes no NO 0 1 2 3

Nervous/Stressed? YES yes no NO 0 1 2 3

Sad? YES yes no NO 0 1 2 3

04/21/23

Our Aproach


Identified 22 Features from Respiration

22

Inhalation Duration

Exhalation Duration

Respiration Duration

Stretch

Mean

Median

Quartile Deviation

80th Percentile

Breathing Rate

Minute Ventilation

Insp./Exp. Ratio

Basic Features Statistical Features


Computed 13 Features from ECG

23

Mean

Median

Quartile Deviation

80th Percentile

RR Intervals Power in low/medium/high frequency bands

Ratio of low frequency/high power

Variance

RSA

Basic Features Statistical Features


Feature and Classifier Selection

Used Weka for Training Evaluated Decision Tree, DT with Adaboost, and

Support Vector Machine Using 10-fold cross validation, and training/test data

Classification results using 35 features

After feature selection, 13 features 8 Respiration, 5 ECG

24

J48 Decision Tree

J48 with Adaboost

SVM

87.67% 90.17% 89.17%


Classification Accuracy on Lab Data


Our Aproach


Perceived Stress Model Use a binary Hidden Markov Model

To reduce number of parameters, we approximate by

models the gradual decay of stress with time models the accumulation of stress in mind due to

repeated exposures to stress Both and are person dependent and are

learned from self-reported ratings of stress27

stress perceived is 1,0ts

,ˆˆ 1 ttt x

ts

tx valuestress perceived is ,,|1P 11 ttt xxs

t


Evaluation of the Model (on Lab Data)

Correlation of perceived stress model and self-report rating in the lab session

Over 21 participants

Median correlation 0.72


Field Study Protocol Participants wore AutoSense continuously for 2

days Going about their daily life (home, school, etc.) Except when sleeping at night

Field self-reports Participants responded to self-reports 20+ times each

day Same questions about affect state as in the lab

Additional context information

Additional behaviors automatically collected Speaking, from respiration patterns Physical activity, from accelerometer


Realities of Natural Environment

30

Data eliminated 37% affected by

activity 30% by bad

quality

Less than 4 min consecutive data

4 subjects missing data or self-report


Evaluation of the Model (Field)

31

Compared average stress ratings over both days

Accumulation model versus self-report

Linear interpolation


Outline






Ongoing User Studies


Memphis Study 40 daily smokers and social drinkers A lab study followed by one week in the field

Stress, drinking, smoking, and craving for cigarettes marked National Institute on Drug Abuse (NIDA) Study

20 drug addicts undergoing treatment Two lab sessions and 4 weeks in field

Smoking, craving, and stress marked in lab; Craving, stress, and drug usage reported in the field

Johns Hopkins Study 10 drug addicts in residential treatment Drug injection in lab, daily behaviors marked in the

field To develop detectors for smoking, craving, and

drug usage

Roadmap


The near-term goal is to develop personalized stress and addiction assistants on the mobile phone to Help reduce stress, e.g., least stressful route for driving Break addiction urges where and when they occur

But, these applications will impact someone’s health Will it indeed be helpful to each user and not hurt anyone?

Will it help maintain healthy behaviors even after the novelty phase? How do we generate evidence for its validity, efficacy, safety?

Within reasonable time and effort, unlike multiyear RCTs How do we design it so it has greater chance of success?

Various theories exist (e.g., stages of change, social cognitive theory)

But, no overall theory for designing adaptive interventions exist today

Long-term Vision


Use these experiences to discover the scientific principles that can be used broadly in mobile health (mHealth) To design and develop

New mHealth measures that are robust enough for field usage

New mHealth treatments and interventions that work To generate evidence of validity, efficacy, and

safety of mHealth

Contribute to the newly emerging science of mHealth

Further Reading


1. E. Ertin, N. Stohs, S. Kumar, A. Raij, M. al'Absi, T.Kwon, S. Mitra, Siddharth Shah, and J. W. Jeong, “AutoSense: Unobtrusively Wearable Sensor Suite for Inferencing of Onset, Causality, and Consequences of Stress in the Field,” ACM SenSys, 2011.

2. Md. Mahbubur Rahman, Amin Ahsan Ali, Kurt Plarre, Mustafa al'Absi, Emre Ertin, and Santosh Kumar, “mConverse: Inferring Conversation Episodes from Respiratory Measurements Collected in the Field,” ACM Wireless Health, 2011.

3. Mohamed Mustang, Andrew Raij, Deepak Ganesan, Santosh Kumar and Saul Shiffman, “Exploring Micro-Incentive Strategies for Participant Compensation in High Burden Studies,” to appear in ACM UbiComp, 2011.

4. K. Plarre, A. Raij, M. Hossain, A. Ali, M. Nakajima, M. al'Absi, E. Ertin, T. Kamarck, S. Kumar, M. Scott, D. Siewiorek, A. Smailagic, and L. Wittmers, “Continuous Inference of Psychological Stress from Sensory Measurements Collected in the Natural Environment,” ACM IPSN, 2011.

5. Andrew Raij, Animikh Ghosh, Santosh Kumar and Mani Srivastava, “Privacy Risks Emerging from the Adoption of Inoccuous Wearable Sensors in the Mobile Environment,” In ACM CHI, 2011.

Nominated for best paper award

Nominated for best paper award

Outline


Hardware and Software Platforms AutoSense sensor suite mStress mobile phone framework



etc. Privacy Issues in mHealth research

Behavior Revelation from Sensors


Accelerometer & gyroscopes can be used to monitor activity level Can infer movement pattern and place from these sensors

See SenSys’10 paper on AutoWitness Could also infer epileptic seizures

Respiration sensor can be used for activity monitoring or estimating the extent of pollution exposure Can use it to infer conversation, smoking, and stress Inferring of public speaking episodes could even pinpoint

the identity of the subject Development of other behavioral inferences in

progress

How Concerned are Study Participants?


Conducted a 66 subject (36 in NS) study

Evaluated their concern level as their personal stake in the data is increased

Also, how their concern level changes as modalities are added/removed

Awareness & Concern


Sharing of stress, commuting, and conversation generate higher concern than the sharing of place

Effect of Privacy Transformations


Disassociating time is more critical than disassociating place of occurrance

Even reducing timestamp to duration helps

Addressing Stress and Addictive Behavior in the Natural Environment Using AutoSense Santosh Kumar...

Documents

Transcript of Addressing Stress and Addictive Behavior in the Natural Environment Using AutoSense Santosh Kumar...