Addressing Stress and Addictive Behavior in the Natural Environment Using AutoSenseSantosh Kumar Computer Science, University of Memphis

Santosh Kumar, University of Memphis

Our TeamBehavioral ScienceEngineering*Santosh Kumar, University of Memphis*Dr. Mustafa alAbsi, UMNDr. J Gayle Beck, MemphisDr. David Epstein, NIDA, NIHDr. Tom Kamarck, PittsburghDr. Satish Kedia, MemphisDr. Kenzie Preston, NIDA, NIHDr. Marcia Scott, NIAAA, NIHDr. Saul Shiffman, PittsburghDr. Annie Umbricht, Johns HopkinsDr. Kenneth Ward, MemphisDr. Larry Wittmers, UMN

Dr. Anind Dey, CMUDr. Emre Ertin, Ohio StateDr. Deepak Ganesan, UMassDr. Greg Pottie, UCLADr. Justin Romberg, Georgia TechDr. Dan Siewiorek, CMUDr. Asim Smailagic, CMUDr. Mani Srivastava, UCLADr. Linda Tempelman, Giner Inc.Dr. Jun Xu, Georgia Tech

Santosh Kumar, University of Memphis

Students & PostdocsMemphisCMU, OSU, UCLA, Georgia Tech., UMN*Santosh Kumar, University of Memphis*Dr. Andrew Raij (now at USF)Dr. Kurt PlarreDr. Karen HovsepianAmin Ahsan AliSantanu GuhaMonowar HussainSomnath MitraMahbub RahmanSudip VhaduriDr. Motohiro Nakajima, UMNPatrick Blitz, CMUBrian French, CMUScott Frisk, CMUNan Hua, Georgia TechTaewoo Kwon, OSUMoaj Mustang, UMassSiddharth Shah, OSUNathan Stohs, OSU

Santosh Kumar, University of Memphis

Paradigm Shift in Disease Prevalence*Santosh Kumar, University of Memphis*Infectious diseases, and those from poor hygiene & nutrition not as prevalent

They are replaced by diseases of slow accumulationHeart diseasesCancer, UlcerDepression, Migraine

Santosh Kumar, University of Memphis

Growing Epidemic Stress & Addiction*Santosh Kumar, University of Memphis*Stress & addictive behavior lead to or worsen diseases of slow accumulationStress: headaches, fatigue, heart failures, hypertension, depression, addiction, anxiety, rageSmoking: cancer, lung diseases, heart diseasesYet, both continue to be widespreadStress: 43% adults suffer adverse health effectsSmoking: responsible for 20% of deaths in US

An urgency to help individuals reduce stress & abstain from addictive behavior

Santosh Kumar, University of Memphis

Addressing Stress & Addiction*Santosh Kumar, University of Memphis*An unobtrusively wearable sensor suite called AutoSenseSo, individuals can wear it in natural environmentRobust inference of stress from physiological measuresAutomatically measure physiological and psychological stressAutomatic inference of addictive behaviorsSmoking, drinking, drug usage from sensor measurementsDetect addiction urges to provide timely interventionCraving for smoking and drug usageContexts/cues that may lead to craving and eventual relapseInfer other moderating behavioral & social contextsConversation, physical activity, traffic stressors, etc.

Santosh Kumar, University of Memphis

Outline*Santosh Kumar, University of Memphis*Hardware and Software PlatformsAutoSense sensor suiteFieldStream mobile phone frameworkInferring StressDetecting stress from physiologyPredicting perceived stressOngoing User StudiesDetecting smoking, drinking, craving, drug usage, etc.Roadmap & Long-term Vision

Santosh Kumar, University of Memphis

AutoSense Wearable Sensor Suite*Santosh Kumar, University of Memphis*Android G1 Smart Phone

Santosh Kumar, University of Memphis

Key Features of AutoSense Hardware*Santosh Kumar, University of Memphis*Ultra low powerSix sensors (ECG, GSR, Resp., Temp, Accel) consume 1.75 mAOverall current consumption < 3mA (for 10+ days of lifetime)Sampling and transmission of 132 samples/sec (i.e., 1.8 kbps)Reliable radio ANT with integrated quality of service and duty cyclingReliable and timely wireless transmissions in crowding scenariosAntenna impedance is matched for human bodyPower loss reduced from 33% (for free space configuration) to 0.1%Operates at 2480-2524 MHZ band to be immune to Wi-FiAverage packet loss rate of 0.57% even when Wi-Fi activity is intense

Santosh Kumar, University of Memphis

FieldStream Mobile Phone Framework*Santosh Kumar, University of Memphis*For use in conducting scientific user studiesIn both supervised lab settings and in uncontrolled field settingsIt collects measurementsSensor measurements from wearable and phone sensorsSelf-reports from subjectsComputes tens of features and various statistics over them (e.g., HR, HRV, RR, Minute Ventilation)Makes inferences using machine learning algorithmsStress, posture, activity, conversation, and commutingDetects sensor detachments and looseningIs reconfigurableSo, no need for change in source code for use in a new user study

Santosh Kumar, University of Memphis

*Santosh Kumar, University of Memphis*Converts stream of sensor measurements into packets & delivers to intended recipientProvides a common interface to all sensors & populates buffers for feature computationComputes base features (e.g., R-R interval) & statistics over them

Santosh Kumar, University of Memphis

Deployment Experiences and Findings*Santosh Kumar, University of Memphis*21 subjects in UMN - completedLab session on stress; 10-14 hours per day for 2 days in field 36 subjects in Memphis - completed3 consecutive days in field with daily visits to the labSome findings on human behaviors in our subject poolStress occurrence in daily life (Plarre et. al., in ACM IPSN11)Subjects were psychologically stressed 26-28% of timeNatural conversations (Rahman et. al., in ACM Wireless Health11)Frequency of conversations : 3 per hourAvg. duration of a conversation: 3.82 minutesAvg. Time between conversations: 13.3 minutes

Santosh Kumar, University of Memphis

Outline*Santosh Kumar, University of Memphis*Hardware and Software PlatformsAutoSense sensor suiteFieldStream mobile phone frameworkInferring StressDetecting stress from physiologyPredicting perceived stressOngoing User StudiesDetecting smoking, drinking, craving, drug usage, etc.Roadmap & Long-term Vision

Santosh Kumar, University of Memphis

Measuring Stress in the FieldSelf-reports have been used for a long timeQuestionnaires or surveysMeasures perceived stressStrengths and limitations(+) Captures detailed information(+) Proximal predictor of mental health(-) Distal predictor of physical health(-) Discrete sampling(-) Burden to participantNeed an automated approach for continuous stress measurement in the field

*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Continuous Measure of Stress*Santosh Kumar, University of Memphis*Can use physiological measurements to assess stress, butPhysiology is affected by several factors, not only stress

How to map physiology to psychology?

Activity, change in posture, speaking, food, caffeine, drink, etc.How to separate out the changes in physiology due to stress?

Santosh Kumar, University of Memphis

The Quest for Automated Stress MeasurePredicting psychological state from physiologyWilliam James pioneering work (1880)John Cacioppo and others revitalized interest (1990)Several studies on emotion and stress predictionIdentified physiological markers of stress and emotion Example: Heart rate, skin conductance responseBut, confined to controlled settingsFew studies in uncontrolled environmentsM. Myrtek96 , J. Healey05, J. Healey10,Either no validated stressors, no lab session to train models, not able to account for confounders, or tried to match self-reports directly *Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

In the AutoSense ProjectWe developed a new wearable sensor suiteConducted a scientific study with validated stress protocol21 participants, 2 hour lab study, 2 day field studyProtocol designed by behavioral scientistsStressors used are validated and known to produce stressSelf-reports designed by expert behavioral scientistsDeveloped new stress models to measure Physiological response to stress To measure adverse physiological effects of stressPerception of stress in mindTo derive a continuous rating of perceived stress

*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Lab Study Stress Protocol2 hour lab sessionSubjects exposed to three types of stressors Public speaking psychosocial stressMental arithmetic mental loadCold pressor physical stress

Physiological signals recorded at all timesUsing AutoSenseAlso, collected self-reported stress rating 14 times*Baseline10 Min101044444441010104555RecoveryPublic Speaking Cold PressorStartEndMental Arithmetic Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Self-Report Measures of StressSelf-report questions related to affective state

Santosh Kumar, University of Memphis**

QuestionPossible AnswerCodeCheerful?YES yes no NO 3 2 1 0Happy?YES yes no NO 3 2 1 0Frustrated/Angry?YES yes no NO 0 1 2 3Nervous/Stressed?YES yes no NO 0 1 2 3Sad?YES yes no NO 0 1 2 3

Santosh Kumar, University of Memphis

*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Our Aproach*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Identified 22 Features from Respiration*Inhalation DurationExhalation DurationRespiration DurationStretchMeanMedianQuartile Deviation80th PercentileBreathing RateMinute VentilationInsp./Exp. RatioBasic FeaturesStatistical FeaturesSantosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Computed 13 Features from ECG*MeanMedianQuartile Deviation80th PercentileRR IntervalsPower in low/medium/high frequency bandsRatio of low frequency/high powerVarianceRSABasic FeaturesStatistical FeaturesSantosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Feature and Classifier SelectionUsed Weka for Training Evaluated Decision Tree, DT with Adaboost, and Support Vector MachineUsing 10-fold cross validation, and training/test dataClassification results using 35 features

After feature selection, 13 features8 Respiration, 5 ECG

*Santosh Kumar, University of Memphis*

J48 Decision TreeJ48 with AdaboostSVM87.67%90.17%89.17%

Santosh Kumar, University of Memphis

Classification Accuracy on Lab Data*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Our Aproach*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Perceived Stress ModelUse a binary Hidden Markov Model

To reduce number of parameters, we approximate by

models the gradual decay of stress with time models the accumulation of stress in mind due to repeated exposures to stressBoth and are person dependent and are learned from self-reported ratings of stress

*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Evaluation of the Model (on Lab Data)Correlation of perceived stress model and self-report rating in the lab session

Over 21 participants

Median correlation0.72

*Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Field Study ProtocolParticipants wore AutoSense continuously for 2 daysGoing about their daily life (home, school, etc.)Except when sleeping at night

Field self-reports Participants responded to self-reports 20+ times each daySame questions about affect state as in the labAdditional context information

Additional behaviors automatically collectedSpeaking, from respiration patternsPhysical activity, from accelerometer *Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Realities of Natural Environment*Data eliminated37% affected by activity30% by bad quality

Less than 4 min consecutive data

4 subjects missing data or self-reportSantosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Evaluation of the Model (Field)*Compared average stress ratings over both days

Accumulation model versus self-report

Linear interpolation

Santosh Kumar, University of Memphis*

Santosh Kumar, University of Memphis

Outline*Santosh Kumar, University of Memphis*Hardware and Software PlatformsAutoSense sensor suiteFieldStream mobile phone frameworkInferring StressDetecting stress from physiologyPredicting perceived stressOngoing User StudiesDetecting smoking, drinking, craving, drug usage, etc.Roadmap & Long-term Vision

Santosh Kumar, University of Memphis

Ongoing User Studies*Santosh Kumar, University of Memphis*Memphis Study40 daily smokers and social drinkersA lab study followed by one week in the fieldStress, drinking, smoking, and craving for cigarettes markedNational Institute on Drug Abuse (NIDA) Study20 drug addicts undergoing treatmentTwo lab sessions and 4 weeks in fieldSmoking, craving, and stress marked in lab; Craving, stress, and drug usage reported in the fieldJohns Hopkins Study10 drug addicts in residential treatmentDrug injection in lab, daily behaviors marked in the fieldTo develop detectors for smoking, craving, and drug usage

Santosh Kumar, University of Memphis

Roadmap*Santosh Kumar, University of Memphis*The near-term goal is to develop personalized stress and addiction assistants on the mobile phone toHelp reduce stress, e.g., least stressful route for drivingBreak addiction urges where and when they occurBut, these applications will impact someones healthWill it indeed be helpful to each user and not hurt anyone?Will it help maintain healthy behaviors even after the novelty phase?How do we generate evidence for its validity, efficacy, safety?Within reasonable time and effort, unlike multiyear RCTsHow do we design it so it has greater chance of success?Various theories exist (e.g., stages of change, social cognitive theory)But, no overall theory for designing adaptive interventions exist today

Santosh Kumar, University of Memphis

Long-term Vision*Santosh Kumar, University of Memphis*Use these experiences to discover the scientific principles that can be used broadly in mobile health (mHealth)To design and develop New mHealth measures that are robust enough for field usageNew mHealth treatments and interventions that workTo generate evidence of validity, efficacy, and safety of mHealth

Contribute to the newly emerging science of mHealth

Santosh Kumar, University of Memphis

Further Reading*Santosh Kumar, University of Memphis*E. Ertin, N. Stohs, S. Kumar, A. Raij, M. al'Absi, T.Kwon, S. Mitra, Siddharth Shah, and J. W. Jeong, AutoSense: Unobtrusively Wearable Sensor Suite for Inferencing of Onset, Causality, and Consequences of Stress in the Field, ACM SenSys, 2011.

Md. Mahbubur Rahman, Amin Ahsan Ali, Kurt Plarre, Mustafa al'Absi, Emre Ertin, and Santosh Kumar, mConverse: Inferring Conversation Episodes from Respiratory Measurements Collected in the Field, ACM Wireless Health, 2011.

Mohamed Mustang, Andrew Raij, Deepak Ganesan, Santosh Kumar and Saul Shiffman, Exploring Micro-Incentive Strategies for Participant Compensation in High Burden Studies, to appear in ACM UbiComp, 2011.

K. Plarre, A. Raij, M. Hossain, A. Ali, M. Nakajima, M. al'Absi, E. Ertin, T. Kamarck, S. Kumar, M. Scott, D. Siewiorek, A. Smailagic, and L. Wittmers, Continuous Inference of Psychological Stress from Sensory Measurements Collected in the Natural Environment, ACM IPSN, 2011.

Andrew Raij, Animikh Ghosh, Santosh Kumar and Mani Srivastava, Privacy Risks Emerging from the Adoption of Inoccuous Wearable Sensors in the Mobile Environment, In ACM CHI, 2011.Nominated for best paper awardNominated for best paper award

Santosh Kumar, University of Memphis

Outline*Santosh Kumar, University of Memphis*Hardware and Software PlatformsAutoSense sensor suitemStress mobile phone frameworkInferring StressDetecting stress from physiologyPredicting perceived stressOngoing User StudiesDetecting smoking, drinking, craving, drug usage, etc.Privacy Issues in mHealth research

Santosh Kumar, University of Memphis

Behavior Revelation from Sensors*Santosh Kumar, University of Memphis*Accelerometer & gyroscopes can be used to monitor activity levelCan infer movement pattern and place from these sensorsSee SenSys10 paper on AutoWitnessCould also infer epileptic seizuresRespiration sensor can be used for activity monitoring or estimating the extent of pollution exposureCan use it to infer conversation, smoking, and stressInferring of public speaking episodes could even pinpoint the identity of the subjectDevelopment of other behavioral inferences in progress

Santosh Kumar, University of Memphis

How Concerned are Study Participants?*Santosh Kumar, University of Memphis*Conducted a 66 subject (36 in NS) study

Evaluated their concern level as their personal stake in the data is increasedAlso, how their concern level changes as modalities are added/removed

Santosh Kumar, University of Memphis

Awareness & Concern*Santosh Kumar, University of Memphis*

Sharing of stress, commuting, and conversation generate higher concern than the sharing of place

Santosh Kumar, University of Memphis

Effect of Privacy Transformations*Santosh Kumar, University of Memphis*

Disassociating time is more critical than disassociating place of occurranceEven reducing timestamp to duration helps

Santosh Kumar, University of Memphis

*