Shift: A Technique for Operating Pen-Based Interfaces Using Touch Daniel Vogel University of Toronto...

Shift: A Technique for Operating Pen-Based Interfaces

Using Touch Daniel Vogel

University of TorontoPatrick Baudisch

Microsoft Research

Motivation

Small Targets

Advantages of the Pen

Pen Finger

unique contact point

remove hand from screen

ambiguous contact point

finger occludes target

Possible Solutions …

Offset Cursor (Potter et al. 1988)

Pen Offset Cursor

Disadvantages 1. no visual feedback until contact, need to estimate

offset 2. makes some display areas inaccessible3. unexpected offset affects walk-up-and-use scenarios


Benefit 1: Aim for the Target

Users expect to click on the target itself. allows switching between pen and touch walk-up and use with kiosk

Users expect to click on the target itself. allows switching between pen and touch walk-up and use with kiosk

Benefit 1: Aim for the Target

Benefit 2: All Areas Accessible

Callout is relative to finger, so it can go anywhere. no edge problems

Callout is relative to finger, so it can go anywhere. no edge problems

Benefit 2: All Areas Accessible

Callout only used when necessary same speed as unaided touch screen for large targets

Benefit 3: Fast For Large Targets

Design Iterations

occlusion

smaller occlusionthreshold larger

TargetSize

Tim

e

Touch Screen

Offset CursorShift

Performance Model

Model

First Prototype

Revision and Visuals

Based on selection ambiguity with fallback to hesitation.

ST = Target Size, SF = Finger occlusion threshold

ST << SF high selection ambiguity no delay

ST >> SF no selection ambiguity long delay

ST ≈ SF “ambiguous selection ambiguity” short delay

ST

SF

de

lay

(s)

ST - SF0

0

1.2

-25 25

Escalation

Escalation

Based on selection ambiguity with fallback to hesitation.

ST = Target Size, SF = Finger occlusion threshold

ST << SF high selection ambiguity no delay

ST >> SF no selection ambiguity long delay

ST ≈ SF “ambiguous selection ambiguity” short delay

ST

SF

de

lay

(s)

ST - SF0

0

1.2

-25 25

user’s view hardware’s view

Perceived Input Point CorrectionUsers expect selection point to be higher.

input point

input point

Iterative estimate for a correction vector V using difference between initial contact point P1 and final lift off point P2

Vt+1 = Vt + w(P2 - P1)

(a) (b) (c)

Experiment

Experimental Design

3 techniques (Shift, Touch, Offset Cursor) x2 finger styles (nail, tip) x3 blocks x6 target sizes (6, 12, 18, 24, 48, 96px) x4 target directions (NW, NE, SW, SE)

Error

96482418126Square Target Size (px)

100

80

60

40

20

0

Mea

n E

rror R

ate

(%)

Touch Screen

Offset Cursor

Shift

Tip Nail

Time

96482418126

Square Target Size (px)

2.5

2.0

1.5

1.0

0.5

0.0

Med

ian

Tim

e (s

)

Selection Time: Fingernail

Touch Screen

Offset CursorShift

Time

96482418126

Square Target Size (px)

2.5

2.0

1.5

1.0

0.5

0.0

Med

ian

Tim

e (s

)

Selection Time: Finger Tip

Touch Screen

Offset CursorShift

Corrective Movements

96482418126Square Target Size (px)

20

10

0

Dis

tan

ce

(p

x)

Off

set

To

uch

Sh

ift

OffsetTouch Shift

(a) FingerTip corrective movements by target size

(b) FingerTip

Discussion

Able to select small targets reliably (like Offset Cursor)Fast for large targets (like unaided Touch Screen)

However, biggest benefit may be simpler mental model: “Just aim for the target”

High Accuracy Enhancements

Added Zooming and CD-Ratio Manipulation

Added Zooming and CD-Ratio Manipulation

High Accuracy Enhancements

Thanks to members of the ASI and VIBE groups at MSR, special thanks to Raman Sarin, Ed Cutrell, and David Thiel.

Appendix

Estimating Occlusion ThresholdDon’t know actual finger size, so estimate it over time when ST ≈ SF short delay … means user can choose to

use escalation by hesitating or not

if they hesitate and use escalation make SF larger

if they just click without escalation make SF smaller

ST

SF

Prototypes

Shift: A Technique for Operating Pen-Based Interfaces Using Touch Daniel Vogel University of Toronto...

Documents

Transcript of Shift: A Technique for Operating Pen-Based Interfaces Using Touch Daniel Vogel University of Toronto...