1

Human Abilities

This material has been developed by Georgia Tech HCI faculty, and continues to evolve. Contributors include Gregory Abowd, Jim Foley, Diane Gromala, Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris Shaw, John Stasko, and Bruce Walker. Permission is granted to use with acknowledgement for non-profit purposes. Last revision: March 2011 by Valerie Summet, Emory University.

2/61

Outline

• Human capabilities Senses Motor systems Information processing

• Memory• Cognitive Processes

Selective attention, learning, problem solving, language

• Simple predictive models

3/61

Typical Person?

•Do we really have limited memory capacity?

4/61

Basic Human Capabilities

• Do not change very rapidly Not like Moore’s law!

• Have limits, which are important to understand• Our understanding of human capabilities does

change, ie Cognitive neuroscience Theories of color perception Effect of groups and situation on how we act and

react

5/61

Human Capabilities

• Why do we care? Better design! Want to improve user performance

• Knowing the user informs the design Senses Information processing systems Physical responding

Time and effort expendedto complete tasks

6/61

Overview

I. Senses A. Vision B. Hearing C. Touch D. Smell

III. Motor system A. Hand movement B. Workstation Layout

II. Information processing A. Perceptual B. Cognitive 1. Memory a. Short term b. Medium term c. Long term 2. Processes a. Selective attention b. Learning c. Problem solving d. Language

7/61

Senses (Our Input System)

Sight, hearing, touch important for current HCISmell, taste ?

Abilities and limitations affect design

8/61

Key concepts for Senses

Just noticeable difference (jnd)How much of a change in stimulus is

needed before can be sensedTends to be logarithmic - Weber’s Law

Magnitude of physical stimulus versus perceived magnitude(Doubling number of photons does not

double perceived intensity)

9/61

Vision (Covered in greater detail in section on visual design)

• Visual System Eye

Retina

Neural pathway– ~ 80% of brain’s operation

10/61

Color JND

Color, from 400 to 700 mm

V I B G Y O R

JND

, m

m

11/61

Audition (Hearing)

• Capabilities (best-case scenario) pitch - frequency (20 - 20,000 Hz) loudness - amplitude (30 - 100dB) location (5° source & stream separation) timbre - type of sound (lots of instruments)

• Often take for granted how good it is(disk whirring)

• Implications ?

12/61

Auditory JND

• Pitch• Loudness• Location

• Temporal variation (eg, songs) is another dimension

13/61

Touch

• Three main sensations handled by different types of receptors: Pressure (normal) Intense pressure (heat/pain) Temperature (hot/cold)

• Sensitivity, Dexterity, Flexibility, Speed

• Where important? Mouse, Other I/O, VR, surgery

14/61

Touch JND

• Spatial - relevant for reading braille• Pressure• Temperature

15/61

Smell?

Joseph Kaye, “Making scents: aromatic output for HCI” ACM interactions Volume 10, Number 1 (2004), Pages 48-61 Solenoid-controlled scent bottles

16/61

Motor System (Our Output Sys.)• Capabilities

Range of movement, reach, speed,strength, dexterity, accuracy

Workstation design, device design

• Often cause of errors Wrong button Double-click vs. single click

• Principles Feedback is important Minimize eye movement

• See Handbooks for data

17/61

Work Station Ergonomics – to Facilitate I/O

18/61

Outline

• Human capabilities Senses Motor systems Information processing

• Memory• Cognitive Processes

Selective attention, learning, problem solving, language

• Simple predictive models

19/61

The “Model Human Processor”

• A true classic - see Card, Moran and Newell, The Psychology of Human-Computer Interaction, Erlbaum, 1983 Microprocessor-human analogue using results

from experimental psychology Provides a view of the human that fits much

experimental data But is a partial model

• Focus is on a single user interacting with some entity (computer, environment, tool) Neglects effect of other people

20/61

Block DiagramLONG-TERM MEMORY

SHORT-TERM (WORKING) MEMORY

AUDITORY IMAGESTORE

VISUAL IMAGESTORE

R = SemanticD = InfiniteS = Infinite

R = AcousticD = 1.5 [0.9-3.5] sS = 5 [4.4-6.2] letters

R = VisualD = 200 [70-1000] msS = 17 [7-17] letters

R= Acoustic or VisualD (one chunk) = 73 [73-226] sD (3 chunks) = 7 [5-34] sS = 7 [5-9] chunks

R = RepresentationD = Decay TimeS = SizeC = Cycle Time

PERCEPTUALPROCESSOR

C = 100 [5-200] ms

COGNITIVEPROCESSOR

C = 70 [27-170] ms

MOTORPROCESSOR

C = 70 [30-100] MS

Eye movement (Saccade) = 230 [70-700] ms

21/61

Memory

• Perceptual “buffers” Brief impressions

• Short-term (working) memory Conscious thought, calculations

• Long-term memory Permanent, remember everything that

ever happened to us

22/61

Recognize-act cycle

• Contents of Working Memory trigger actions held in Long Term Memory

23/61

Perceptual

• Memory structures Image Stores - Holds fixed image of outside

world long enough for some analysis(will come back to this)

• Processes - Info goes to brain for more processing e.g. Pattern recognition Uses context & knowledge

24/61

Perceptual Image Stores

• Visual and auditory impressions Visuospatial sketchpad, phonological loop

• Very brief, but accurate representation of what was perceived Details decay quickly (70 - 1000 ms visual; 0.9 -

3.5 sec auditory) Limited capacity (7 - 17 letters visual; 4 - 6

auditory) Rehearsal prevents decay Another task prevents rehearsal - interference

25/61

Short Term Memory

• Use “chunks”: 5-9 units of information

• Display format (picture, text, sound) should match memory system used to perform task

• New info can “push out” old info - interference

26/61

About Chunks

• A chunk is a meaningful grouping of information – allows assistance from LTM

• 4793619049 vs. 404 894 7328• NSAFBICIANASA vs. NSA FBI CIA NASA• My chunk may not be your chunk

User and task dependent

27/61

Long-Term Memory

• Seemingly permanent & unlimited

• Access is harder, slower -> Activity helps (we have a cache)

• Retrieval depends on network of associations

File system full

28/61

LT Memory Structure

• Episodic memory Events & experiences in serial form

– Helps us recall what occurred

• Semantic memory Structured record of facts, concepts &

skills– One theory says it’s like a network– Another uses frames & scripts (like record

structs)

29/61

Memory Characteristics

• Things move from STM to LTM by rehearsal & practice and by use in context

• We forget things due to decay and interference

Unclear if we ever really forget something - just lose link to the info

Lack of use

Similar gets inway of old

30/61

Outline

• Human capabilities Senses Motor systems Information processing

• Memory• Cognitive Processes

Selective attention, learning, problem solving, language

• Simple predictive models

31/61

Processes

• Four main processes of cognitive system: Selective Attention Learning Problem Solving Language

32/61

Selective Attention

• We can focus on one particular thing Cocktail party chit-chat

• Salient visual cues can facilitate selective attention Examples?

33/61

Learning• Two types:

Procedural – How to do something Declarative – Facts about something

• Involves Understanding concepts & rules Memorization Acquiring motor skills Automotization

– Tennis– Driving to work

• Even when don’t want to

– Swimming, Bike riding, Typing, Writing

34/61

Learning

• Facilitated By structure & organization By similar knowledge, as in consistency in UI design By analogy If presented in incremental units Repetition

• Hindered By previous knowledge

– Try moving from Mac to Windows

=> Consider user’s previous knowledge in your interface design

35/61

Observations

• Users focus on getting job done, not learning to effectively use system

• Users apply analogy even when it doesn’t apply Or extend it too far - which is a design

problem– Dragging floppy disk icon to Mac’s trash can

does NOT erase the disk, it ejects disk!

36/61

Problem Solving

• Storage in LTM, then application• Reasoning

Deductive -

Inductive -

Abductive -

• Goal in UI design - facilitate problem solving! How??

If A, then BGeneralizing from previouscases to learn about new ones

Reasons from a fact to theaction or state that caused it

37/61

Observations

• We are more heuristic than algorithmic We try a few quick shots rather than plan

– Resources simply not available

• We often choose suboptimal strategies for low priority problems

• We learn better strategies with practice

38/61

Implications

• Allow flexible shortcuts Forcing plans will bore user

• Have active rather than passive help Recognize waste

39/61

Language

• Rule-based How do you make plurals?

• Productive We make up sentences

• Key-word and positional Patterns

• Should systems have natural language interfaces? IBM's Watson

40/61

Recap

I. Senses A. Sight B. Sound C. Touch D. Smell

II. Information processing A. Perceptual B. Cognitive 1. Memory a. Short term b. Medium term c. Long term 2. Processes a. Selective attention b. Learning c. Problem solving d. Language

III. Motor system A. Hand movement B. Workstation Layout

41/61

Class Discussion:Model Human Processor

• What are the three major subsystems and their functions?

• What does it mean to say that certain subprocessors have “variable rates?”

• What is the “recognize-act” cycle? Is it like the “fetch-decode-execute” of a CPU?

• What are some of the other assumptions underlying the MHP model?

• How good is the model?

42/61

People

• Good Infinite capacity LTM LTM duration &

complexity High-learning

capability Powerful attention

mechanism Powerful pattern

recognition

• Bad Limited capacity STM Limited duration STM Unreliable access to

LTM Error-prone processing Slow processing

43/61

Outline

• Human capabilities Senses Motor systems Information processing

• Memory• Cognitive Processes

Selective attention, learning, problem solving, language

• Simple predictive models

44/61

Simple User Models

• Idea: If we can build a model of how a user works, then we can predict how s/he will interact with the interface Predictive model predictive evaluation

• No mock-ups or prototypes!

45/61

Two Types of User Modeling

• Stimulus-Response Hick’s law Practice law Fitt’s law

• Cognitive – human as interperter/predictor – based on Model Human Processor (MHP) Key-stroke Level Model

– Low-level, simple

GOMS (and similar) Models– Higher-level (Goals, Operations, Methods, Selections)– Not discussed here

46/61

Power law of practice

• Tn = T1n-a

Tn to complete the nth trial is T1 on the first trial times n to the power -a; a is about .4, between .2 and .6

Skilled behavior - Stimulus-Response and routine cognitive actions

– Typing speed improvement– Learning to use mouse– Pushing buttons in response to stimuli– NOT learning

47/61

Uses for Power Law of Practice

• Use measured time T1 on trial 1 to predict whether time with practice will meet usability criteria, after a reasonable number of trials How many trials are reasonable?

• Predict how many practices will be needed for user to meet usability criteria Determine if usabiltiy criteria is realistic

48/61

Hick’s law

• Decision time to choose among n equally likely alternatives T = Ic log2(n+1)

Ic ~ 150 msec

49/61

Uses for Hick’s Law

• What did you think of?• Menu selection

• Which will be faster as way to choose from 64 choices? Go figure: Single menu of 64 items Two-level menu of 8 choices at each level Two-level menu of 4 and then 16 choices Two-level menu of 16 and then 4 choices Three-level menu of 4 choices at each level Binary menu with 6 levels

50/61

Fitts’ Law

• Models movement times for selection tasks in one dimension

• Basic idea: Movement time for a selection task Increases as distance to target

increases Decreases as size of target increases

51/61

Fitts’ Law

• Selection task made up of 3 parts: start/stop time time for gross motor movement time for find adjustment

• Fitt's Law models all of these and gives you a numerical estimate of time.

52/61

Original Experiment

• 1 Dimensiond w

53/61

Components

• ID - Index of difficulty

• ID is an information theoretic quantity Based on work of Shannon – larger target =>

more information (less uncertainty)

ID = log2 (d/w + 1.0)

bits result

width (tolerance)of target

distance to move

54/61

Components (p. 322)

• MT - Movement time

• MT is a linear function of ID a and b are experimental constants

MT = a + b*IDMT = a + b *log2 (d/w + 1.0)

55/61

Exact Equation• Run empirical tests to determine a

and b in MT = a + b* ID• Will get different ones for different

input devices and device uses

MT

ID = log2(d/w = 1.0)

56/61

Questions

• What do you do in 2D?

w x h rect:one way is ID = log2(d/min(w, h) + 1)

Should take into account direction of approach

57/61

Uses for Fitt’s Law

• Menu item size• Icon size• Scroll bar target size and placement

Up / down scroll arrows together or at top and bottom of scroll bar

58/61

Keystroke-Level Model (KSLM)

• KSLM - developed by Card, Moran & Newell, see their book* and CACM* The Psychology of Human-Computer

Interaction, Card, Moran and Newell, Erlbaum, 1983

• Skilled users performing routine tasks• Assigns times to basic human operations -

experimentally verified• Based on MHP - Model Human Processor

59/61

KSLM Accounts for

• Keystroking TK

• Mouse button press TB

• Pointing (typically with mouse) TP

• Hand movement betweenkeyboard and mouse TH

• Drawing straight line segments TD

• “Mental preparation” TM

• System Response time TR

60/61

Using KSLM

• Skilled users• Performing routine tasks

The user has done it many times before No real learning going on Some modest “thinking” as captured by “Ms”

(mental preparation)

• Rules for placing “Ms” are heuristics• Best use is for comparing alternatives

Sometimes predictions are off But rankings of faster - slower tend to be accurate

61/61

Cognitive models - many flavors

More complex than KSLMHierarchical

GOMS - Goals, Operators, Methods, SelectorsCCT - Cognitive Complexity Theory

LinguisticTAG - Task Action GrammarCLG - Command Language Grammar

Cognitive architecturesSOAR, ACT