Input Techniques and Models

Input Techniques and Models

Properties of an input device

Evaluation and analysis of input devices

Input device states

Interaction modalities

Hinckley, Ken., Input Technologies and Techniques, Chapter 7. Handbook of Human-Computer Interaction, ed. by Andrew

Sears and Julie A. Jacko. Lawrence Erlbaum & Associates.

10/5/2010 2

What‘s an input device

• Input devices sense physical properties of people, places or things

• However, they do not operate in isolation, i.e. need visual feedback

– otherwise similar to a pen without paper

• Must include:

– the physical sensor (positioning wheels)

– the feedback presented to the user (cursor)

– the ergonomic of the device (fits in hand)

– interplay between all the interaction techniques supported by a system (clicking, moving, selecting, etc.)

• Need an understanding of input technologies to design interaction techniques that match a user‘s natural workflow

10/5/2010 3

Input Device Properties

• Several properties characterize most

devices:

– Property sensed

– Number of dimensions

– Indirect vs. direct

– Device acquisition time

– Gain

10/5/2010 4

Property Sensed

• Most devices sense– Linear position (tablets sense position of pen)

– Motion (mice sense change in position)

– Force (Isometric joysticks, IBM Trackpoint)

– Angle or change in angle (rotary input)

• Absolute input device– position sensing

• Relative input device– motion sensing

• Relative device requires visual feedback but also can be inefficient due to clutching

10/5/2010 5

Number of Dimensions

• Devices sense one or more input dimensions

–Two linear dimensions (mouse, x/y)

– Angular dimension (knob)

– 6 degree-of-freedom (magnetic tracker, senses 3-position

dimensions and 3 orientation dimensions)

• A pair of knobs is a 1D+1D device, mouse with scroll

wheel is a 2D+1D multi-channel device

• Multiple degree-of-freedom devices sense three or

more simultaneous dimensions of spatial position or

orientation

10/5/2010 6

Degrees of Freedom vs. Dimensions

the bat

Ware, C. and Jessome, D. (1988), Using the Bat: A Six Dimensional Mouse for Object

Placement. IEEE Computer Graphics and Applications. November 8-6, 65-70.

Usually confuse the idea of dimensions and degrees-of-freedom

10/5/2010 7

Mapping Degrees of Freedom to Dimensions

A Two-Ball Mouse Affords Three Degrees of Freedom

I. Scott MacKenzie, R. William Soukoreff, & Chris Pal

CHI 97 Electronic Publications: Late-Breaking/Short Talks

3DOF Mouse

10/5/2010 8

Mapping Degrees of Freedom to Dimensions

Ravin Balakrishnan, Thomas Baudel, Gordon Kurtenbach, George W.

Fitzmaurice. (1997). The Rockin'Mouse: Integral 3D manipulation on a plane.

Proceedings of ACM CHI 1997 Conference on Human Factors in Computing

Systems, p. 311-318.

4DOF Mouse

10/5/2010 9

Degrees of Freedom vs. Dimensions

Hinckley, K., Sinclair, M., Hanson, E., Szeliski, R., Conway, M., The

VideoMouse: A Camera-Based Multi-Degree-of-Freedom Input Device, ACM

UIST'99 Symposium on User Interface Software & Technology, pp. 103-112.

5DOF Mouse

10/5/2010 10

Indirect vs. Direct

• Mouse is indirect, i.e. must move mouse to

move pointer on screen

• Touch-screens or tablets are direct input

devices, i.e. unified input and display

surface

– Occlusion is typically a problem

Diversity of input ‗devices‘

Positioning

Absolute positioning

Relative positioning

clutching

Input mode

Direct input

motor & visual space

Indirect input

motor space

visual space

Direct Indirect

Relative

Absolute

Central question

How to simultaneously leverage properties

of each input or positioning mode for

interaction?

Dual-Surface Input:Augmenting One-handed Interactions with

Coordinated Front and Behind-the-screen

Input

Xing-Dong Yang, Edward Mak,

Pourang Irani, Walter Bischof

ACM MobileHCI 2009

One-handedness is common

Over 80% of mobile users employ or prefer

one-handed operation [Karlson & Bederson, 2006]

Input device: Thumb

Reach

Karlson at al, 2006

Input ―mismatch‖

High occlusion + low accuracy

Input ―mismatch‖

user view hardware view

Shift

Vogel & Baudisch, ACM CHI 2007

Dual-surface input

Dual-surface model

Direct input

Move cursor with absolute input on front

Indirect input

‗Adjust‘ with relative on back

Key point: coordinated

Nanotouch

Baudisch & Chu, ACM CHI 2009

Insights #1

Coordinated front (absolute) + back (relative) superior than either alone

For back, indirect works better

Hardware solution for fluid move between

modes works

ARC-Pad:Absolute + Relative Cursor Positioning for

Large Displays with a Mobile Touchscreen

David McCallum, Pourang Irani

ACM UIST 2009

Large Displays

Reach is problematic

Custom hardware

Not always available or expensive

Coupling a mobile device

Relative

large distance to travel

lots of clutching!

Absolute

imprecise

a b c

d e

Hybrid Pointing

Forlines et al, ACM UIST 2006

ARC-Pad

Insights #2

Seamless transition between absolute &

relative positioning is crucial

no explicit mode switch

via interpretation of input gesture

Software based solution requires some training

Guidelines

Avoid explicit mode switching, should be

implicit

software based – proper interpretation of input

hardware based – novel augmentations

Seamlessly coordinate input modes

Proper ergonomics to accommodate novelty

10/5/2010 48

Device Acquisition Time

• Acquisition time: average time to move

hand to device

• Homing time: average time to return to a

‗home‘ position, i.e. mouse to keyboard

• For common desktop workflows, pointing

and selecting dominate acquisition/homing

time

– integration of pointing with keyboard may not

improve overall performance

10/5/2010 49

Gain

• Control-to-display gain or C:D ratio

– distance moved by an input device/distance

moved on the display

• Composite measurement taking into

account device size and display size

• Gain has very little effect on the time to

perform pointing movements

– not a commonly used metric

10/5/2010 50

Pointing Devices

• Mouse:– Invented in ‘67

– Used for pointing

– Picks up changes in x, y

– As good as pointing with finger

– Integrated with buttons/wheels etc

• Trackball:– Senses relative motion of partially exposed ball in

2DOF

– Engage different muscle groups than the mouse, but an alternative for those who experience discomfort

10/5/2010 51

Pointing Devices

• Isometric joystick:

– Force sensing

– Rate of cursor is proportional to the force exerted

– Returns to center when released

– Good when space is at a premium

• Isotonic joystick:

– Sense angle of deflection

– Different than isometric joystick

J. Lipscomb and M. Pique (1993). Analog Input Device

Physical Characteristics. SIGCHI Bulletin 25 (3): 40-45.

10/5/2010 52

Pointing Devices

• Touchpads:

– Small touch-sensitive devices found on laptops

– Use relative mode for cursor control

– Can operate in absolute mode by dragging

finger and leaving it on edge of the pad

– Necessitates multiple clutchings

• Touchscreens/pen-operated devices:

– Fingers, or electromagnetic digitizers

– Parallax error, mismatch between sensed input

position and apparent position

10/5/2010 53

Input Device States

• Disaccord between states of a GUI and

states and events sensed by devices

• Difficult to support interface primitives such

as click, drag, double-click, and right-click

• Useful to diagram device states

– Identifies relationship between events sensed by

input device and interaction technique

demands

10/5/2010 54

Three-State Model of Graphical Input

• Buxton's 3-state model for graphical input devices

• Expression of the operation of computer pointing

devices in terms of state transitions

• Expressive vocabulary for exploring the relationship

between pointing devices and the interaction

techniques they afford

10/5/2010 55

Input Device States

10/5/2010 Comp 4020 - HCI 2 (PPI) 56

Three-State Model of Graphical Input

• Three-states:– The states are Out of range (State 0, for clutching or repositioning a mouse on a mouse pad;

– Tracking (State 1) for moving a tracking symbol such as a cursor about a display

– Dragging (State 2) for moving an icon on the desktop or for grouping a set of objects or a range of text

• Seems simple and obvious but can add insight to the existing body of pointing device research

– can be extended to multi-button interaction, stylus input, and direct vs. indirect input

10/5/2010 57

Input Device States

• Based on Buxton‘s model, the mouse &

touch sensitive devices are a two-state

device

10/5/2010 58

Activity: Input Device States

• Based on the previous state diagram, can you

describe a limitation of touch-sensing input (PDA‘s)

• Do you know of a device that supports all 3 states

10/5/2010 Comp 4020 - HCI 2 (PPI) 59

Input Device States

• Can fully capture core interaction by

extending the 3-state model

10/5/2010 60

Evaluation and Analysis of Input Devices

• Keystroke-Level Model

• Fitts‘ Law

• Hick-Hyman Law

• Accot‘s Steering Law

10/5/2010 61

Models and Modeling

• A model is a simplification of reality, but useful only if

it helps in understanding some phenomenon or

behavior:

– Design, evaluate or help understand complex behavior

• Models sit on a continuum:

Analogy &

Metaphor

Mathematical

Equations

Descriptive

Models

Predictive

Models

Fitt’s LawGuiard’s Model

10/5/2010 62

Keystroke Level Model

• Only address one aspect of task performance: time

• Predicts expert error-free task-completion time with the following inputs:

– a task or series of subtasks

– method used

– command language of the system

– motor-skill parameters of the user

– response-time parameters of the system

• Predict time an expert would take to execute the tasks– Assuming no errors

10/5/2010 63

KLM operators

• Six operators

– Keystroke• Avg time determined by std typing tests

– Pointing• Pointing with a mouse or other device on a display to select an

object.

• Varies from 0.8 – 1.5 seconds

– Homing• Bring ‗home‘ hands on the keyboard or other device

• 0.4 seconds based on various studies

– Mental• 1.35 seconds, experimentally determined

– Response

• Texecute = TK + TP + TH + TM + TR

10/5/2010 64

Encoding Methods

• E.g., replace 5 letter word with another in a text

editor

• Reach for mouse Hmouse

• Point to word Pword

• Select word K

• Home on keyboard Hkeyboard

• Call replace cmd M,Kreplace

• Type new 5 letter word 5Kword

• Texecute = TM + TP + 2TH + 7Tk

10/5/2010 65

Where does KLM apply?

• Where users perform tasks at expert level

– users have mastered a skill

– users are not problem solving

– users know what to do, just act on the steps

• Qualitatively

– Used for designing training programs, help systems

– Focus design on problem areas

• Quantitatively

– Good predictions of performance time

– Maybe some predictions on learning

10/5/2010 66

KLM weaknesses

• The model applied to skilled users, not to beginners or intermediates

• The model doesn't account for either learning of the system or its recall after a period of disuse

• Even skilled users occasionally make errors; however, the model doesn't account for errors

• Mental workload is not addressed in the model

10/5/2010 67

Hick-Hyman Law

• Law for choice reaction-time, given in the form of a prediction equation

• Given a set of n stimuli (flashing objects), associated with n responses (selecting object), reaction time (RT) can be given as follows:

RT = a + b log 2(n) – a, b empirically determined constants

• Examples include:– measuring and predicting time to select items in a

hierarchical menu (Landauer & Nachbar, 1985)

– predicting text-entry rates on soft keyboards with non-qwerty layouts, since users have to visually scan the layout (MacKenzie et al., 1995, 1999)

– mode selection on tabletPCs (Ruiz & Lank, 2006)

10/5/2010 68

A Quiz Designed to Give You Fitts

•http://www.asktog.com/columns/022DesignedToGive

Fitts.html

• Microsoft Toolbars offer the user the

option of displaying a label below each

tool. Name at least one reason why

labeled tools can be accessed faster.

(Assume, for this, that the user knows

the tool and does not need the label just

simply to identify the tool.)

http://www.asktog.com/columns/022DesignedToGiveFitts.html

http://www.asktog.com/columns/022DesignedToGiveFitts.html

10/5/2010 69


1. The label becomes part of the target. The

target is therefore bigger. Bigger targets, all

else being equal, can always be

acccessed faster. Fitt's Law.

2. When labels are not used, the tool icons

crowd together.

10/5/2010 70


• You have a palette of tools in a graphics

application that consists of a matrix of 16x16-

pixel icons laid out as a 2x8 array that lies

along the left-hand edge of the screen.

Without moving the array from the left-hand

side of the screen or changing the size of the

icons, what steps can you take to decrease

the time necessary to access the average

tool?

10/5/2010 71


1. Change the array to 1X16, so all

the tools lie along the edge of the

screen.

2. Ensure that the user can click on

the very first row of pixels along

the edge of the screen to select a

tool. There should be no buffer

zone.

10/5/2010 72


• Microsoft offers a Taskbar which can be oriented

along the top, side or bottom of the screen,

enabling users to get to hidden windows and

applications. This Taskbar may either be hidden or

constantly displayed. Describe at least two reasons

why the method of triggering an auto-hidden

Microsoft Taskbar is grossly inefficient.

10/5/2010 73


• Screen edges are prime real estate. You don't waste an entire edge that could be housing a couple of dozen different fast-access icons just for one object, the Taskbar

• The auto-hidden Taskbar is entirely too easy to display by accident. Users are constantly triggering it when trying to access something that is close to, but not at, the edge

• The Taskbar would not have any of these problems, yet be even quicker to get to if it were located at any one of four corners of the display. Throw the mouse up and to the left, for example, and you'll have a taskbar displayed. Fast access without the false triggering

10/5/2010 74

Fitts‘ Law

• Robust and highly adopted model of human movement

• Originated as interest of applying information theory to the

analysis and understanding of difficulty of movement tasks &

human rate of information processing

• Used Shannon‘s law for information capacity

C = B log2(S / N + 1)

» S is the signal power and N is the noise power

• Based on the following analogies:

– Amplitude of aimed movement == electronic signal

– Spatial accuracy of movement == electronic noise

10/5/2010 75

Fitts‘ Law

• Described the analogy in two

papers:

– a serial, or reciprocal, target

acquisition task wherein subjects

alternately tapped on targets of

width W separated by amplitude A

– experiment using a discrete task,

wherein subjects selected one of two targets in response to a stimulus

light

Fitts, P. M. (1954). The information capacity of the human motor

system in controlling the amplitude of movement. Journal of

Experimental Psychology, 47, 381-391.

Fitts, P. M., & Peterson, J. R. (1964). Information capacity of

discrete motor responses. Journal of Experimental Psychology, 67,

103-112.

10/5/2010 76

Fitts‘ Law

• Quantify a movement task's difficulty — ID, the

index of difficulty

ID = log2(A / W + 1) (bits)

A = amplitude, W = width of object

• Movement time to complete a task is predicted

using a linear equation

MT = a + b * ID (secs)a & b are empirically determined using linear regression

• Throughput (TP) or Index of Performance (IP) is

TP = ID / MT (bits/sec)

10/5/2010 77

Fitts‘ Law

• To determine a and b design a set of tasks with

varying values for A and W (conditions)

• For each task condition

– multiple trials conducted and the time to execute each is recorded and stored electronically for statistical analysis

• Accuracy is also recorded

– either through the x-y coordinates of selection or

– through the error rate — the percentage of trials selected

with the cursor outside the target

10/5/2010 78

Fitts‘ Law

Same ID → Same Difficulty

Target 1 Target 2

10/5/2010 79

Fitts‘ Law

Smaller ID → Easier

Target 2Target 1

10/5/2010 80

Fitts‘ Law

Larger ID → Harder

Target 2Target 1

10/5/2010 81

Fitt’s Law

A (pixels) W (pixels) ID (bits) Device 'A' Device 'B'

ER (%) MT (ms) ER (%) MT (ms)

40 10 2.32 2.08 665 1.25 1587

40 20 1.58 3.33 501 2.08 1293

40 40 1.00 1.25 361 0.42 1001

80 10 3.17 2.92 762 2.08 1874

80 20 2.32 1.67 604 2.08 1442

80 40 1.58 1.67 481 0.83 1175

160 10 4.09 3.75 979 2.08 2353

160 20 3.17 5.42 823 1.67 1788

160 40 2.32 4.17 615 0.83 1480

Mean: 2.40 2.92 644 1.48 1555

Example data sets for two devices from a Fitts' law experiment

10/5/2010 82

Fitt‘s Law

10/5/2010 83

Fitt‘s Law

• If primary goal in Fitts‘ law experiment is to determine performance between devices/interaction techniques, then throughput (TP) is best criterion

– TP = ID/MT

• If for a given device ID = 4.09 bits and task is executed in MT = 979 ms– human rate of information processing for that task is 4.09 / 0.979 =

4.18 bits/s or TP = 4.18 bits/s

• Mean throughput across all the A-W conditions for Device 'A' is TP = 2.40 / 0.644 = 3.73 bits/s

• For Device 'B', TP = 2.40 / 1.555 = 1.57 bits/s

• Using throughput we conclude users' performance with Device 'A' was about 3.73 / 1.57 = 2.4 times better than performance with Device 'B'

10/5/2010 84

Setting it up

– MacKenzie, I. S. (1995). Movement time prediction in human-computer interfaces. In R. M. Baecker, W. A. S. Buxton, J. Grudin, & S. Greenberg (Eds.), Readings in human-computer interaction (2nd ed.) (pp. 483-493). Los Altos, CA: Kaufmann. [reprint of MacKenzie, 1992]

– http://www.yorku.ca/mack/GI92.html

• Vary A,W values for at least 4 ID conditions– Small A, small W– Small A, large W

– Large A, small W

– Large A, large W

– 2-4 variations in between

• Clicking start position presents object to click on– Record whether user missed

– Record time to click on stimulus

• Design with several repetitions and several blocks

10/5/2010 85

Case Study #1: Text Entry Rates on Mobile Phones

• Can we predict text entry rate on mobiles using Fitts‘ Law?

10/5/2010 86


• Two main approaches:– Multi-tap:

• presses each key one or more times to specify the input character

• large overhead: 33 key presses 15 characters of text

• "on average" multi-tap method requires 2.034 keystrokes per character

77 88 444 222 55 0 22 777 666 9 66 0 333 666 99

q u i c k _ b r o w n _ f o x

– One-key disambiguation:• Add linguistic knowledge to make best guess

• Can be ambiguous in some cases, have to correct

7 8 4 2 5 0 2 7 6 9 6 0 3 6 9

q u i c k _ b r o w n _ f o x

10/5/2010 87


• Text entry on a mobile phone, for example, consists

of aiming for and acquiring (pressing) a series of

keys "as quickly and as accurately as possible―

• Time to press any key, given any previous key, can

be readily predicted using Fitts' law

• For index finger input = MT = 165 + 52 ID

• and for thumb input = MT = 176 + 64 ID

10/5/2010 88


• Elements to build a text-entry prediction model are:

– Information on position and size of keys (ruler)

– Letter assignment to keys (any phone)

– Relative probabilities of digrams (probabilities of letter pairs) in target language (sources)

A B C D Z Space

A 0.00002 0.00130 0.00290 0.00360 0.00011 0.00047

B 0.00130 0.00013 0.00000 0.00000 0.00000 0.00034

C 0.00340 0.00000 0.00012 0.00000 0.00000 0.00044

D 0.00099 0.00001 0.00000 0.00035 0.00000 0.02500

Z 0.00003 0.00000 0.00000 0.00000 0.00008 0.00002

Space 0.01800 0.00960 0.00810 0.00480 0.00002 0.00000

t-h or e-space have high P

g-k or f-v have low P

10/5/2010 89


Method Predicted Expert Entry Rate (wpm)

Index Finger Thumb

Multi-tap

- wait for timeout

- timeout kill

22.5

27.2

20.8

24.5

One-key with disambiguation 45.7 40.6

• Time to enter each i-j sequence is predicted using

Fitts‘ law giving MTij, weighted by the probability

of the digram in the target language Pij

MTL = ∑∑ (Pij × MTij )

WPM = MTL × (60 / 5) (avg 5 chars/word)

2 assumptions:

- all words are in dictionary

- when ambiguity arises the intended word is the most probable

10/5/2010 90

Case Study: Using Fitts‘ to redesign text entry

http://www.exideas.com/ME/

http://www.exideas.com/ME/Pressfolder/PressReleaseDec2-2003.html

Nesbat, S. “A System for Fast, Full-Text Entry for Small Electronic

Devices“, Proceedings of the Fifth International Conference on Multimodal

Interfaces, ICMI 2003 (ACM-sponsored), Vancouver, November 5-7, 2003.

MessagEase Onscreen Keyboard

Example of an interface design

which can be adapted to multiple

devices

10/5/2010 91

Diverging Text Entry Technologies

Device Text Entry Technology

Cell phones Multi-tap

PDAs and Tablet computers Graffiti, Jot, QWERTY

Email devices QWERTY

TV remote controllers, watches Scroll and Pick

Car navigation systems Several proprietary

Point of Sale devices ABCD, multi-tap

http://www.nec.com.tw/product/image/pda.jpg

http://www.fujitsupc.com/images/pentablets/ltp600/angle.jpg

10/5/2010 92

Using Letter Frequency

10/5/2010 93

Nine Most Frequent Letters: Double Click

E

T

N

R

O

I

A

S

H

10/5/2010 94

Eight Less Frequent Letters: Two Taps

D

C

U

P

G

B

Q

J

10/5/2010 95

Remaining Nine Letters: Two Taps

F

M

Y

W

V

X

K

Z

10/5/2010 96

Adding Space, Shift, and Mode

10/5/2010 97

Special Characters

38 special characters entered by two taps;

6000+ characters can be entered with combine.

10/5/2010 98

Soft Keyboard Design

Hard Key Soft Key

Most Frequent Letters Double Click Single Tap

Less Frequent Letters Two Clicks Single Drag

The same

mapping

used for

letters

10/5/2010 99

Special Characters

Entered with a single drag

10/5/2010 100

Optimization and Evaluation

• Exhaustively simulated all permutations of letters within each group

• The configuration with the max speed was selected

10/5/2010 101

Fitts‘ Law

Movement Time from one

key to another:

MT = a + b*log2(A/W+1)

WA

10/5/2010 102

Digraph Probability

The probability Pij that letter j will

follow letter i in a body of text:

Pij = 1

A B C D Z Space

A 0.00002 0.00130 0.00290 0.00360 0.00011 0.00047

B 0.00130 0.00013 0.00000 0.00000 0.00000 0.00034

C 0.00340 0.00000 0.00012 0.00000 0.00000 0.00044

D 0.00099 0.00001 0.00000 0.00035 0.00000 0.02500

Z 0.00003 0.00000 0.00000 0.00000 0.00008 0.00002

Space 0.01800 0.00960 0.00810 0.00480 0.00002 0.00000

10/5/2010 103

Performance Measure – Hard Key

Calculation of max theoretical entry speed:

• Movement Time

–MT = a + b log2(A/W+1)

• Total time (2 Clicks)

–CT = MT1 + MT2

• Total time (Dble Click – no

movement)

–CTDC = 2a + b log2(A/W+1)

• Average Time

–CTav = (Pij × CTij)

• Speed

–WPM = (1/ CTav) ×(60/5)

10/5/2010 104

Performance Measurement – Soft Key

– Most frequent characters – Single tap: • TLi = (1/4.9) log2 [(D0-i/W) + 1]; if D0-i> 0,

• TLi = a; if D0-i = 0

– Less frequent characters – Drag:• TLjk = t0-j + tdown + tj-k + tup

• TLjk = (t0-j + tdown+ tup) + (tj-k + tup+ tdown) –(tdown+ tup)

• TLjk = TLj + TLk– a t0-j: time to move to key j

tj-k: time to move from key j to key k

tdown: time to move stylus down

tup: time to move stylus up

10/5/2010 105

Hard Key (Cell Phone) Comparison

User Study

0

2

4

6

8

10

12

Multi-tap MessagEase

WP

M

Theoretical

0

10

20

30

Multi-tap MessagEase

WP

M

130%

209%

10/5/2010 106

10

30

3638

43

50

0

10

20

30

40

50

60

Graffiti

Comparison with Other Soft Keyboards

Includes all special

characters

Graffiti QWERTY Fitaly Opti Metropolis MessagEase

(all based on Fitts’ law except for Graffiti)

10/5/2010 107

Comparison with Other Soft Keyboards

• Given the same area MessagEase has:

– Only 11 keys, but enters more characters

– Its Keys are 3-5 times bigger; therefore faster!

10/5/2010 108

Advantages and Disadvantages

Disadvantages Advantages

New letter assignment, requires

learning a new pattern

Applicable to both hard-key and Soft-

key

Some keys may become clutteredFull text entry:

letters, numbers, special Chars.

Size and language agnostic

Deterministic and unambiguous

One handed (hard-key) operation

10/5/2010 109

Applications:

Any small mobile device that cannot sport a full QWERTY keyboard

10/5/2010 110

Watches

•Scroll-and-Pick is too slow

•QWERY is too small

•MessagEase will have buttons

large enough for a stylus

10/5/2010 111

Car Navigation Systems

•One handed touch typing

•Unifying an otherwise

fragmented text entry systems

used

10/5/2010 112

Example: Expanding targets

• Acquisition of Expanding Targets. Proceedings of

ACM Conference on Human Factors in Computing

Systems (CHI) 2002, pages 57-64

10/5/2010 113

Example: Expanding targets

Furnas

Generalized fisheye views

CHI 1986 Bederson

Fisheye Menus

UIST 2000

Mackinlay, Robertson, Card

The Perspective Wall

CHI 1991

10/5/2010 114

Mac OS X ―dock‖

Does this make acquisition easier ?

• Size of the interface widget (viewing region)

changes dynamically

– Provide the user with a magnified target area at their focus

of attention (area around the cursor)

– Expanding toolbar implemented in latest Apple OS X

operating system

10/5/2010 115

Advantages and Disadvantages

• Advantages

– Icons are displayed in reduced size to solve the

increasing number of commands and icons

– Larger amount of screen real estate devoted to

the display of the underlying data

• Disadvantages

– Can reduce the user‘s ability to select the

desired icon efficiently

10/5/2010 116

Fitts‘ Law

• 3 different scenarios describing what Fitts‘ Law is modeling

– Iterative corrections model

– Impulse variability model

– Optimized initial impulse model

10/5/2010 117

Iterative Corrections Model

• States that the movement consists of many discrete

sub-movements

• Each sub-movement takes the user closer to the

target

• Sub-movements are triggered by feedback

indicating the target has not been reached yet

10/5/2010 118

Impulse Variability Model

• Movement consists of initial impulse delivered by

the muscles towards the target, flinging the limb

towards the target

• Last part of movement time consists of limb

coasting towards target

• Either type of explanation cannot explain the Fitt‘s

Law completely given a range of tasks

10/5/2010 119

Optimized Initial Impulse Model (1)

• Most complete and successful explanation to

the Fitts‘ Law

• Combination of the iterative corrections and

the impulse variability models

– movement is initiated towards the target

– task is completed if the movement lands at the

target

– another movement is required if it lands outside

the target

– same processes will be carried out until the

target is reached

10/5/2010 120

Distance

Speed

What does Fitts‘ Law really model?

W

UndershootOvershoot

Open-loop

Closed-loop

10/5/2010 121

Expanding Targets

• Basic Idea:– Big targets can be acquired faster, but take up

more screen space

– So: keep targets small until user heads toward them

– Can this be used for devices with small viewing space?

Cancel

Okay

Click Me !

10/5/2010 122

Experiment Goals

The experiment was designed to answer the following

questions for a typical expanding target selection

task:

1. Can such a task be modeled by Fitts‘ law?

2. If it can be modeled by Fitts‘ law, is it possible to predict

performance in such tasks from a base set of data where

no expansion takes place?

3. Is movement time dependent on the final target width and not the initial one at onset of movement?

4. At what point should the target begin expanding?

5. Do different target expansion strategies affect

performance?

10/5/2010 123

Experimental Setup

Target

Start Position

W

A

10/5/2010 124

Experimental Setup

Expansion:

How ?

Animated

Expansion

10/5/2010 125

Experimental Setup

Expansion:

How ?

Fade-in

Expansion

10/5/2010 126

Experimental Setup

Expansion:

How ?

When ?P = 0.25

10/5/2010 127

Experimental Setup

Expansion:

How ?

When ?P = 0.5

10/5/2010 128

Experimental Setup

Expansion:

How ?

When ?P = 0.75

10/5/2010 129

Pilot Study

• 7 conditions:

– No expansion (to establish a, b values)

– Expanding targets

• Either animated growth or fade-in

• P is one of 0.25, 0.5, 0.75

– (Expansion was always by a factor of 2)

10/5/2010 130

Pilot Study

7 conditions

x 16 (A,W) values

x 5 repetitions

x 2 blocks

x 3 participants

= 3360 trials

10/5/2010 131

Pilot Study: Results

Time

(seconds)

ID (index of difficulty)

10/5/2010 Comp 4020 - HCI 2 (PPI) 132


Time

(seconds)


)1(log2 W

Aba

10/5/2010 Comp 4020 - HCI 2 (PPI) 133


Time

(seconds)


)1(log2 W

Aba

)12

1(log2

W

Aba

10/5/2010 134


Time

(seconds)


P = 0.25

10/5/2010 135


Time

(seconds)


P = 0.5

10/5/2010 136


Time

(seconds)


P = 0.75

10/5/2010 137

• Pilot Study suggests the advantage of

expansion doesn‘t depend on P

• So, set P = 0.9 and perform a more rigorous

study

Implications

10/5/2010 138

Full Study

• 2 conditions:

– No expansion (to establish a, b values)

– Expanding targets, with

• Animated growth

• P = 0.9

• Expansion factor of 2

10/5/2010 139

Full Study

2 conditions

x 13 (A,W) values

x 5 repetitions

x 5 blocks

x 12 participants

= 7800 trials

10/5/2010 140

Results

Time

(seconds)

A, W values

10/5/2010 141

Results

Time

(seconds)

10/5/2010 142

Results

Time

(seconds)

10/5/2010 143

Results

Time

(seconds)

10/5/2010 144

Results

Time

(seconds) P = 0.9

10/5/2010 145

Implications

• For single-target selection task,

– Expansion yields a significant advantage, even

when P=0.9

• What about multiple targets ?

10/5/2010 146

Implications for Design (1)

• Experimental results can influence the design of

buttons, menus, or other selectable widgets

• Interface with multiple expanding targets does not

need to predict cursor's trajectory to anticipate

which widgets to expand

– Instead, just expand widgets as the cursor approaches

them

10/5/2010 147

Implications for Design (2)

• Expansion Strategies for adjacent

widgets (e.g. toolbars)

• Expanding a widget around its

center will cause overlap &

occlusion with nearby widgets– Expanding a group of widgets around a

group’s center

– Expand nearest widgets and move

adjacent widgets away

– Expand nearest widgets, but allow some

overlap as well as expand adjacent widgets

so they are easier to see

10/5/2010 148

Summary

• Expanding targets acquisition can be accurately

modeled by Fitts‘ Law

• User performance is aided by target expansion

• Targets that are always expanded can be acquired

just as fast as targets that expand just as the user

reaches them

• Implications of results can be applied towards the

design of UI widgets for devices with limited viewing

space

10/5/2010 149

Improvement to Fitts‘: Bubble Cursor

TO READ:

• Tovi Grossman, Ravin Balakrishnan. The Bubble

Cursor: Enhancing target acquisition by dynamic

resizing of the cursor’s activation area, ACM

CHI, 2005, p. 281-290.

10/5/2010 150

Bubble Cursor

10/5/2010 151

Bubble Cursor

Improvements by

– Decreasing A

• Drag-and-pop [Baudisch et al.]

• Object pointing [Guiard et al.]

– Increasing W

• Area cursor [Kabbash & Buxton]

• Enhanced area cursor [Worden at al]

• Expanding targets [McGuffin & Balakrishnan]

– Decreasing A and Increasing W

• Semantic pointing [Blanch et al]

10/5/2010 152

Design of Bubble Cursor

Modification to area cursor

Problem! Circular cursor resolves problem

10/5/2010 153


Size problem

10/5/2010 154


10/5/2010 155


10/5/2010 156


10/5/2010 157


10/5/2010 158

Results

10/5/2010 159

Results

10/5/2010 160

Experiment 2

10/5/2010 161

Results

10/5/2010 162

Models for Trajectory-Based HCI Tasks

• Trajectory tasks are becoming more common

– Navigating through nested menus

– Drawing curves

– Moving in 3D worlds

• Cannot be successfully modeled using Fitts‘ law

• ―Steering through tunnel‖ as paradigm to represent

trajectory-based tasks

“Beyond Fitts’ Law: Models for trajectory based HCI tasks.”

Proceedings of ACM CHI 1997 Conference

10/5/2010 163

Beyond pointing: Trajectory based tasks

10/5/2010 164


• Experimental paradigm focused on is steering

between boundaries (constrained motion)

• It appears that the time to produce trajectories sets

the relative speed-accuracy ratio: the larger the

amplitude, the less precise the result is.

• Want to derive and validate quantitative

relationships between completion time and

movement constraints in trajectory-based tasks

10/5/2010 165


10/5/2010 166


10/5/2010 167


10/5/2010 168

Steering Law

10/5/2010 169

Crossing Based Interfaces: Motivation

• Pointing is most universal and best adapted

interaction paradigm in human computer

interfaces

• However some disadvantages exist:

– time-consuming if object pointed to is small

– pointing-driven widgets consume screen real estate

– double-clicking is not trivial for novice users, due to rapid

succession of clicks (temporal dependence)

“More than dotting the i's --- foundations for crossing-based interfaces.” Johnny Accot ,

Shumin Zhai. Proceedings of the SIGCHI conference on Human factors in computing systems.

April 2002

10/5/2010 170

Crossing interfaces: example designs

10/5/2010 171

Crossing interfaces: example designs

10/5/2010 172

Descriptive Models

• Do not yield empirical or quantitative measure of user performance– Not predictive

• Provide a framework/context for thinking methodologically about a problem or situation

• Typically a verbal or graphical description of classes or identifiable features in an interface– Facilitates categorization

• Purpose is to give the designer a tool for studying and thinking about user-interaction experience

• Two examples:– KAM

– Mapping DOF to Dimensions

– Guiard‘s Theory of Bimanual Interaction

10/5/2010 173

Key-Action Model

• Keyboards are essential in interacting with a

computer

• 101-keys categorized by function keys, numeric

keys, characters, control keys, etc.

• Define a descriptive model referred to as Key-

Action Model (KAM) as follows:

– Symbol keys – deliver graphical symbols (a,z,1,?)

– Executive keys – invoke actions in the application or at the

system level (ENTER, F1, ESC)

– Modifier keys – set up a condition to modify effect of

pressing a key (ALT, SHIFT, CTRL)

10/5/2010 174

Key-Action Model

• Some questions to consider regarding this model:

– Is the model correct?

– Is it flawed?

– Do all keyboard keys fit the model?

– Are there additional categories or sub-categories?

– Can it be improved to become more comprehensive?

– Is the model really useful?

10/5/2010 175

Guiard‘s Model

• Originated from the area of motor behavior

referred to as bimanual control or laterality

• Both hands perform a different set of tasks

• Given this knowledge and handedness of people,

interesting to evaluate how interaction

accommodates best the division

• Results in descriptive model of bimanual skill, given

by Guiard in 1987 paper

Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action:

The kinematic chain as a model. Journal of Motor Behavior, 19, 486-517.

10/5/2010 176

Guiard’s Model

Hand Role and Action

Non-preferred • leads the preferred hand

• sets the spatial frame of reference for the preferred hand

• performs coarse movements

Preferred

• follows the non-preferred hand

• works within established frame of reference set by the non-

preferred hand

• performs fine movements

10/5/2010 177

Guiard‘s Model

• Example:

– a right-handed artist sketches a design of car

– acquires a template with left hand (non-preferred hand

leads)

– template is manipulated over the workspace (coarse

movement, sets the frame of reference)

– Right hand picks stylus (preferred hand follows) and placed

close to the template (works within frame of reference set

by the non-preferred hand)

– Artist sketches (preferred hand makes precise movements)

10/5/2010 178

Guiard‘s Model

• People naturally gravitate to using two hands

• Performance times are reduced

• Can be used for interfaces that employ:

– Drawing designs

– Fabricating virtual objects

– Positioning

– Reshaping

10/5/2010 179

Bimanual Control and Desktop Computer Affordances

• How does the distribution of keys on a keyboard facilitate

task division between right/left hands?

– where does interaction with the mouse fit into the model?

• Right side bias toward power keys (executive keys +

modifier keys, marked in red dots)

10/5/2010 180


• Dominance on right hand side, good

for the 80s but how does this work with

GUIs and pointing devices that are now

commonplace?

• Right handed have to reach over with

left or leave the mouse

• Is there an advantage for left-handed

users?

10/5/2010 181


Task Leading Movement Trailing/Overlapping

Movement

Delete

Left hand — manipulate pointer

with mouse and select text/object

by double clicking or dragging

Right hand — press DELETE

(probably with little finger)

Select an option

in a window


with mouse and click on an option

Right hand — press ENTER

(Note: OK button is the default)

Click on a link in a

browser

Right hand — navigate to link via

PAGE UP and/or PAGE DOWN

keys

Left hand — manipulate

pointer with mouse and select

link by clicking on it

Open a file, open

a folder, or launch

a program


with mouse and single click on icon

Right hand — press ENTER

(Note: avoids error prone

double-click operation)

Common tasks performed by a left-handed user manipulating mouse in the left hand

10/5/2010 182


• Tasks described previously are faster for left-handed users than right-handed users

• When pointing is juxtaposed with power key activation (excluding SHIFT, ALT, & CONTROL), the desktop interface presents a left-hand bias

10/5/2010 183


• Scrolling typically accomplished by dragging ―elevator‖ of scrollbar along the right-hand side of an application’s window

•Takes up to 2 secs per trial and is obtrusive and non-transparent

• In perspective of Guiard’s model of bimanual control, we can delegate scrolling to non dominant hand

Task Characteristics

Scrolling• precedes/overlaps other tasks

• sets the frame of reference

• minimal precision needed (coarse)

Selecting, editing, reading,

drawing, etc.

• follows/overlaps scrolling

• works within frame of reference set by scrolling

• demands precision (fine)

10/5/2010 184

Redesigning the Scrolling Interface

10/5/2010 185

What did we cover

• Text entry rate prediction on mobile phones

• Bimanual control and analysis of task division

between right and left hands

– Application to guide design choices

• Application of Fitts, expanding targets

• MessagEase: a commercial application

making use of Fitts‘ law for improving text

entry rates

Input Techniques and Models

Documents

Transcript of Input Techniques and Models