Large-Scale Evaluation of Call-Availability Prediction

Research

Martin Pielot, Telefónica ResearchACM UbiComp’14, Sep, 2014, Seattle, USAWed, Sep 17, 2014 – 14:00 – 15:30Session: Interruptability & Notifications

"2007Computex e21Forum-MartinCooper" by Rico Shen. Via Wikimedia. CC-SA 3.0.

Phone calls reach us

anywhere,

anytime

makelessnoise. Phon-ey Call. via Flickr, Jul 07, 2006 (CC BY 2.0)

30% of all calls are missed

WilliamTheaker. Hamhung cyclist. via Wikipedia, Apr 17, 2012 (CC BY 2.0)

Please raise hands:

Who of you would take a call right now?

Kaz. Hände. Via Pixabay, Nov 26, 2013. (Public Domain CC0)

Interruption!

Callers want to know: Location and time, physical, social,

emotional availability,

and current activity.De Guzman et al. 2007

"White Diamonds Party" by Club Skirts Dinah Shore Weekend - Own work. via Wikimedia Commons. CC BY-SA 3.0 -

Callers want to know: Location and time, physical, social,

emotional availability,

and current activity.De Guzman et al. 2007

"White Diamonds Party" by Club Skirts Dinah Shore Weekend - Own work. via Wikimedia Commons. CC BY-SA 3.0 -

Callees react depend. on: Location and time, Presence of

others, and current activity.Danninger et al. 2006

"No trespassing" by Djuradj Vujcic - Own work. via Wikimedia Commons. CC BY-SA 3.0.

People have concerns sharing too much contextual information Knittel et al. 2011

Availability

Location

Presence of others

Time

Current Activity

…

Machine-LearningUser Model

…

Related Approaches

Horvitz et al. 2005

Using calendar details from Outlook to predict cost of interruption by call

Related Approaches

Horvitz et al. 2005

Using calendar details from Outlook to predict cost of interruption by call

Rosenthal et al. 2011

Use ESM to train phones to mute ringer in certain situations

Related Approaches

Horvitz et al. 2005

Using calendar details from Outlook to predict cost of interruption by call

Rosenthal et al. 2011

Use ESM to train phones to mute ringer in certain situations

Pejovic and Musolesi 2014

Identifying opportune moments for mobile device-based interruptions

Opportunities to Advance Line of Work

(1)Actual reaction

(2) Sample selection & size

Study

On shake

Anonymous logs

Sep 17, 2014, 15:20

Screen Status

Reaction to the call

Proximity Sensor

Day and Time

Ringer mode

Charging

Results

31,311 callsfrom 418 distinct

user

Extracted 15 Basic FeaturesCategory FeatureLast Active Last ringer change (time)Last Active Last screen change (time)Last Active Last (un)plugged (time)Last Active Last call (time)Currently Active Screen statusCurrently Active Pitch of phoneRelationship How often called by callerContext Day of the weekContext Hour of the dayContext Charger (un)pluggedContext Ringer modeContext Last call silencedContext Activity / AccelerationContext Screen (not) coveredContext Last call picked

Prediction

Random Forest (10 trees)Classes: available | not available

Accuracy 83.2% (κ=.646)(10-fold cross-validation)

19,175 calls picked up

61.2% baseline accuracy

Advantages of large data set

Model accuracy over time

100 4001600

6399.9999999999925600

10240065

70

75

80

85

Number of instances (phone calls)

Accu

racy (

%)

Random Forest Model

Personalized model

Subset of 120 calls: 87.0% (κ=.640)

Features Ranked by Prediction PowerCategory Feature Mean RankLast Active Last ringer change (time) 1Last Active Last screen change (time) 2Currently Active Screen status 3.6Last Active Last (un)plugged (time) 5.4Last Active Last call (time) 6.8Context Activity / Acceleration 7.3Relationship How often called by caller 7.6Context Day of the week 9.4Context Hour of the day 10Context Charger (un)plugged 10.1Context Ringer mode 11.4Context Last call silenced 12.4Currently Active Pitch of phone 12.5Context Screen (not) covered 13Context Last call picked 14.1

Application

Mute the ringer

CommunicateNon-Availability

CommunicateNon-AvailabilityLikely to be

unavailable

Large-Scale Evaluation of Call-Availalability Prediction.

First large-scale study (31,311 calls) of call-availability prediction

Prediction possible with 15 basic features83% accuracy (generic models)87% accuracy (personalized models)

Strongest 5 predictors4 features regarding time of last activityScreen status

Use casesMute ringer on unavailabilityAllow caller to check availability

Large-Scale Evaluation of Call-Availability Prediction

ACM UbiComp’14, Sep, 2014, Seattle, USAWed, Sep 17, 2014 – 14:00 – 15:30Interruptability & Notifications

Martin Pielot, Telefónica Research

martin.pielot@telefonica.com

Q&A