Slide 1

Kyungmin Lee, Jason Flinn, and Brian Noble University of Michigan The Case for Operating System Management of User Attention

Slide 2

Trend in mobile app interaction 2 Kyungmin Lee Using apps while performing primary tasks Apps initiate interactions

Slide 3

Interaction in various user contexts 3 Kyungmin Lee Users current primary activity Application is unaware of users context! ?

Slide 4

Existing solution: Let user decide 4 Kyungmin Lee Set policy for each appDisable all interactions Too coarse grained! All or nothing

Slide 5

Our proposed approach 5 Kyungmin Lee Mobile OS Mobile sensors Interactions -Deliver now -Modify format -Defer -Deliver now -Modify format -Defer Extract users context Interactions

Slide 6

Outline Motivation Our vision Our proposed approach Challenges 6 Kyungmin Lee

Slide 7

Our vision 7 Kyungmin Lee Can you pick up milk? From: Your wife Users current contextInterrupt? Do not interrupt!

Slide 8

Our vision 8 Kyungmin Lee Dangerous road conditions ahead Users current context Interrupt! via audio interaction Interrupt?

Slide 9

Our vision 9 Kyungmin Lee Can you pick up milk? From: Your wife Users current context Interrupt! Interrupt?

Slide 10

Manage user attention as a resource 10 Kyungmin Lee Visual Auditory Cognitive Haptic Attention demand 100% Visual Auditory Cognitive Haptic Attention level 100% Visual Auditory Cognitive Haptic Attention level 100% Interaction Users activity Its a scheduling problem!

Slide 11

Our proposed approach Priority Attention level 11 Kyungmin Lee Very lowLowMediumHighVery high Users current context Visual Auditory Cognitive Haptic Attention level 100%

Slide 12

Our proposed approach 12 Kyungmin Lee Can you pick up milk? From: Your wife Interrupt? Priority Attention demand 100% Visual Auditory Cognitive Haptic Very lowLowMediumHighVery high

Slide 13

No delivery! Our proposed approach Attention level after delivery 13 Visual Auditory Cognitive Haptic Attention level 100% Can you pick up milk? From: Your wife Medium priority High priority

Slide 14

Dangerous road conditions ahead Our proposed approach Attention level after delivery 14 Visual Auditory Cognitive Haptic Attention level 100% Very high priority High priority

Slide 15

Deliver! Dangerous road conditions ahead Our proposed approach Attention level after delivery Change to audio modality 15 Visual Auditory Cognitive Haptic Attention level 100% Very high priority High priority

Slide 16

Deliver! Our proposed approach Attention level after delivery Cognitive attn. load has dropped 16 Visual Auditory Cognitive Haptic Attention level 100% Can you pick up milk? From: Your wife Medium priority High priority

Slide 17

Challenges in determining priority 17 Kyungmin Lee Med. priority From: A colleague Friends request High priority From: A colleague Low priority Friends request

Slide 18

Learn from users behavior 18 Kyungmin Lee High priority Low priority High priority Low priority

Slide 19

Interactions attention demand Extend AMC (Mobisys 13) 19 Kyungmin Lee Button size Button closeness Text contrast ratio Word count Animation Scrolling

Slide 20

Interactions attention demand Extend AMC (Mobisys 13) 20 Kyungmin Lee Visual Auditory Cognitive Attention demand Demand level

Slide 21

Estimating users attention level 21 Kyungmin Lee Visual Auditory Cognitive Haptic Attention level 100% Very high priority Low priority Lowly engaged activity Highly engaged activity Same activity, but different priority level

Slide 22

Conclusion Our vision: Right interaction at the right time Our proposed approach Treat user attention as a shared resource Determine priorities of interaction and activity Consider Attention level supply vs. demand 22 Kyungmin Lee

Slide 23

23 Questions? Kyungmin Lee