Shift: A Technique for Operating Pen-Based Interfaces
Using Touch Daniel Vogel
University of TorontoPatrick Baudisch
Microsoft Research
Advantages of the Pen
Pen Finger
unique contact point
remove hand from screen
ambiguous contact point
finger occludes target
Disadvantages 1. no visual feedback until contact, need to estimate
offset 2. makes some display areas inaccessible3. unexpected offset affects walk-up-and-use scenarios
Offset Cursor (Potter et al. 1988)
Benefit 1: Aim for the Target
Users expect to click on the target itself. allows switching between pen and touch walk-up and use with kiosk
Users expect to click on the target itself. allows switching between pen and touch walk-up and use with kiosk
Benefit 1: Aim for the Target
Benefit 2: All Areas Accessible
Callout is relative to finger, so it can go anywhere. no edge problems
Callout is relative to finger, so it can go anywhere. no edge problems
Benefit 2: All Areas Accessible
Callout only used when necessary same speed as unaided touch screen for large targets
Benefit 3: Fast For Large Targets
occlusion
smaller occlusionthreshold larger
TargetSize
Tim
e
Touch Screen
Offset CursorShift
Performance Model
Model
Based on selection ambiguity with fallback to hesitation.
ST = Target Size, SF = Finger occlusion threshold
ST << SF high selection ambiguity no delay
ST >> SF no selection ambiguity long delay
ST ≈ SF “ambiguous selection ambiguity” short delay
ST
SF
de
lay
(s)
ST - SF0
0
1.2
-25 25
Escalation
Escalation
Based on selection ambiguity with fallback to hesitation.
ST = Target Size, SF = Finger occlusion threshold
ST << SF high selection ambiguity no delay
ST >> SF no selection ambiguity long delay
ST ≈ SF “ambiguous selection ambiguity” short delay
ST
SF
de
lay
(s)
ST - SF0
0
1.2
-25 25
user’s view hardware’s view
Perceived Input Point CorrectionUsers expect selection point to be higher.
input point
input point
Iterative estimate for a correction vector V using difference between initial contact point P1 and final lift off point P2
Vt+1 = Vt + w(P2 - P1)
Experimental Design
3 techniques (Shift, Touch, Offset Cursor) x2 finger styles (nail, tip) x3 blocks x6 target sizes (6, 12, 18, 24, 48, 96px) x4 target directions (NW, NE, SW, SE)
Error
96482418126Square Target Size (px)
100
80
60
40
20
0
Mea
n E
rror R
ate
(%)
Touch Screen
Offset Cursor
Shift
Tip Nail
Time
96482418126
Square Target Size (px)
2.5
2.0
1.5
1.0
0.5
0.0
Med
ian
Tim
e (s
)
Selection Time: Fingernail
Touch Screen
Offset CursorShift
Time
96482418126
Square Target Size (px)
2.5
2.0
1.5
1.0
0.5
0.0
Med
ian
Tim
e (s
)
Selection Time: Finger Tip
Touch Screen
Offset CursorShift
Corrective Movements
96482418126Square Target Size (px)
20
10
0
Dis
tan
ce
(p
x)
Off
set
To
uch
Sh
ift
OffsetTouch Shift
(a) FingerTip corrective movements by target size
(b) FingerTip
Corrective Movements
96482418126Square Target Size (px)
20
10
0
Dis
tan
ce
(p
x)
Off
set
To
uch
Sh
ift
OffsetTouch Shift
(a) FingerTip corrective movements by target size
(b) FingerTip
Discussion
Able to select small targets reliably (like Offset Cursor)Fast for large targets (like unaided Touch Screen)
However, biggest benefit may be simpler mental model: “Just aim for the target”
Thanks to members of the ASI and VIBE groups at MSR, special thanks to Raman Sarin, Ed Cutrell, and David Thiel.
Estimating Occlusion ThresholdDon’t know actual finger size, so estimate it over time when ST ≈ SF short delay … means user can choose to
use escalation by hesitating or not
if they hesitate and use escalation make SF larger
if they just click without escalation make SF smaller
ST
SF
Top Related