© John M. Abowd 2005, all rights reserved Household Samples John M. Abowd March 2005.
The Human Processing and Memory Human Computer Interaction, 2 nd Ed. Dix, Finlay, Abowd, and Beale...
-
Upload
rosamond-dickerson -
Category
Documents
-
view
221 -
download
0
Transcript of The Human Processing and Memory Human Computer Interaction, 2 nd Ed. Dix, Finlay, Abowd, and Beale...
The HumanProcessing and Memory
Human Computer Interaction, 2nd Ed.
Dix, Finlay, Abowd, and Beale
Chapter 1
Model Human Processor + AttentionRecall, “purely and engineering abstraction”
• Sensory store– Rapid decay “buffer” to hold
sensory input for later processing
• Perceptual processor– Recognizes symbols, phonemes– Aided by LTM
• Cognitive processor– Uses recognized symbols– Makes comparisons and
decisions– Problem solving– Interacts with LTM and WM
• Motor processor– Input from cog. proc. for action– Instructs muscles– Feedback
• Results of muscles by senses
• Attention– Allocation of resources
Overview
• Will look at elements of human information processing from a slightly different orientation than “engineering abstraction”
• A bit more fine grained analysis, following from psychological studies– But, it is these psychological studies from which the “engineering
abstraction” is derived
• 3 stage model of human memory– Iconic buffer, STM, LTM
• Models of LTM
• Reasoning
• Problem solving
Model Human Processor + AttentionRecall, “purely and engineering abstraction”
• Sensory store– Rapid decay “buffer” to hold
sensory input for later processing
• Perceptual processor– Recognizes symbols, phonemes– Aided by LTM
• Cognitive processor– Uses recognized symbols– Makes comparisons and
decisions– Problem solving– Interacts with LTM and WM
• Motor processor– Input from cog. proc. for action– Instructs muscles– Feedback
• Results of muscles by senses
• Attention– Allocation of resources
3-Stage Model of Human Memory
• Sensory (here, iconic) memory – “very” short term memory– lasts 1-2 seconds, infinite capacity
• Short-term memory (Working memory)– lasts ~ 18 seconds, holds 1.75 (7+/-2 items)
• Long-term memory– infinite capacity; short of damage is permanent– Recall vs. Recognition (Remember vs. Know)
• Retrieval cues• Will demonstrate later in class … http://www.if.uidaho.edu/~marbjm/class%202.pdf
“Executive” - Attention
• Central “executive” controls tasking – Pays, or allocates, attention– Bandwidth of attention is limited
• Tasks that require the same resources interfere with one another
• Attention is both a low-level and high-level property of visionhttp://www.if.uidaho.edu/~marbjm/class%202.pdf
Sensory Memory: “Very” Short Term Memory
• Sensory buffers for stimuli received through senses– iconic memory: visual stimuli– echoic memory: aural stimuli– haptic memory: tactile stimuli
• Examples– “sparkler” trail– stereo sound
• Continuously overwritten – demo follows
A Test – of Visual Iconic Memory
• Will present figure briefly (~1/2 second)
• Try to remember as many elements as you can
• Write them down
The Phenomenon
• After presentation, did you continue to “see” the items?
– Some purely physiological based “seeing”:• Afterimage• Bleaching of pigments• “bright, or colored, stuff”
– But also, there is a more “memory-based” image (process further downstream in memory system)• Iconic memory• “dark, or veridical, stuff”• Reading from the iconic buffer
Reading from the Iconic Buffer, 1
• Typically can list 3 – 7 items named
• Short lived visual, or iconic, buffer– holds the image for a second or two
• Read images and place in STM– 3-stage model
• Can get about 5-7 items until run out of short term (working) memory capacity
• Limitation of 5-7 comes from:– Decay of iconic memory– Rate can read from visual buffer– Capacity of working memory
• In each fixation between saccadic eye movements, image of world captured
Set of miscellaneous symbols
a
VisualSearch orMonitoringStrategy
EyeMovementControl
Useful VisualField of View
Reading from the Iconic Buffer, 2
• Again, Limitation of 7 comes from:– Decay of iconic memory– Rate can read from visual buffer– Capacity of working memory
• From each image, – brain must identify objects, – match them with objects previously
perceived, and – take information into working memory for
symbolic analysis
• Search light model of attention (for vision) – Visual information is acquired by pointing
fovea at regions of visual field that are interesting
– Then using a scanning process in which objects are read from an image buffer from more extensive processing
Set of miscellaneous symbols
a
VisualSearch orMonitoringStrategy
EyeMovementControl
Useful VisualField of View
Attention
• Spotlight metaphor– Spotlight moves serially from one input channel to another– Can focus attention (and perceptual processor) on only one input channel at a
time• Location in visual field, voice in auditory field, …, anything
• Visual dominance: – Easier to attend to visual channels than auditory channels
• All stimuli within spotlighted channel are processed in parallel– Whether you want to or not– Can cause “interference” - demo
Say the Colors of the Words
• Easy enough – didn’t take too long
Say the Colors of the Words
• Took longer … Stroop effect• For design:
– Choose secondary characteristics of display to reinforce message
Again, Human Memory Stages
• Sensory (here, iconic) memory– lasts 1-2 seconds, infinite capacity
• Short-term memory (Working memory)– lasts ~ 18 seconds, holds 1.75 (7+/-2 items)
• Long-term memory– infinite capacity; short of damage is permanent– Recall vs. Recognition (Remember vs. Know)
• Retrieval cueshttp://www.if.uidaho.edu/~marbjm/class%202.pdf
Short-Term Memory (STM)
• “Scratch-pad” (or buffer) for temporary recall– rapid access ~ 70ms– rapid decay ~ 200ms– limited capacity - 7± 2 chunks
• Chunking, recoding, etc. – affects amount of information retained, entering LTM
Example - Chunking
HEC ATR ANU PTH ETR EET
Long-term Memory (LTM)
• Repository for all our knowledge– slow access ~ 1/10 second– slow decay, if any– huge or unlimited capacity
• Episodic and semantic memory– Episodic (episodes): Serial memory of events– Semantic (“meanings”): Structured memory of facts, concepts, skills
• Also, procedural and declarative memory– “Processes” vs. “facts”
LTM – Models of Semantic Memory
• Semantic memory structure– Contains LTM knowledge of world– Provides access to information– Generic knowledge -- specific details lost– Represents relationships between bits of information– Important for rule-based behavior
• Supports inference
• Many models, theories, accounts, schemata proposed
• Semantic network model (example next slide): – E.g., Inheritance – child nodes inherit properties of parent nodes– Relationships between bits of information explicit– Supports inference through inheritance
• Other Models (examples follow):– Scripts, frames, production rules
Early Model of Semantic MemoryCollins and Quillian
• Collins & Quillian’s Teachable Language Comprehender
• Semantic memory is organized as a network of interrelated concepts
• Each concept is represented as a node
• Concepts are linked together by pathways
• Economy of representation
• Activation of one concept spreads to interconnected nodes
• Remind you of anything from computer science?
Early Model of Semantic MemoryCollins and Quillian
• Collins & Quillian’s Teachable Language Comprehender
Early Model of Semantic MemoryCollins and Quillian
• Spreading Activation
• Working memory is activated LTM
• When a concept becomes active, activation spreads to all other interconnected nodes
• Activation spreads to all related nodes
• How do you evaluate sentences like “Is a robin is an animal”?
Early Model of Semantic MemoryCollins and Quillian
• Spreading Activation
• Activation spreads from each of the concept nodes (Robin & Animal)
• When two spreading activations meet, an intersection is formed
• Robins ==> BIRD <== Animals
• If no intersection, relatively fast no
• If intersection, decision stage operates to determine if sentence is valid
Is a robin an animal?
Tests of Spreading Activation
• Sentence verification task– Time to respond yes or no
• Takes time for activation to spread
• Greater distances ==> longer RT
• Verification time for items 0, 1, and 2 links
But, it’s not that simple ...
• E.g. typicality effects - how many links separate:– A canary is a bird?– A robin is a bird?– A chicken is a bird?– An ostrich is a bird?
• But, RT varied - less typical birds took longer than more typical birds
FYI - Semantic Relatednes and Semantic Priming
• Semantic relatedness– Spreading activation between related concepts– Activation of one concept partially activates
semantically related concepts
• Semantic priming– Stimulus 1 ==> Stimulus 2– (Prime) ==> (Probe)– Test spreading activation by manipulating
semantic relationship between prime & probe
• Concepts linked by spreading activation• Prime: Probe:• Doctor Nurse• Bread Butter• Doctor Butter• Bread Nurse
• Sometimes prime facilitates processing
FYI - Semantic Relatednes
• Recall, semantic relatedness– Spreading activation
between related concepts– Activation of one concept
partially activates semantically related concepts
• So, can focus on relatedness, without explicitly indicating links
Spreading Activation - Fin
• Semantic priming commonplace– Can exploit in design– Indeed, in design can exploit all information about how human operates
• Spreading activation is thought to be automatic
• Governed by data-driven aspects of processing
• How do expectancies affect semantic access?– Automatic vs Conscious Strategies (attentional)– Fast vs Slow– Effortless vs Effortful– Benefits vs Costs & Benefits
Fyi – Another semantic network
Models of LTM – Frames, or Schemata
• Information organized in “memorial data structures”
• Schemata– Stored frameworks or body of knowledge– Conceptual framework for interpreting
information– Biased information processing to relate new
material to what we already know– Alters way we perceive things– Individual differences in perception and memory
• Frames– Slots in structure instantiated with values for
instance of data– Type–subtype relationships
DOG
Fixed legs: 4
Default diet: carniverous sound: bark
Variable size: colour
COLLIE
Fixed breed of: DOG type: sheepdog
Default size: 65 cm
Variable colour
Models of LTM - Scripts
• Model of stereotypical information required to interpret situation
• Script has elements that can be instantiated with values for context
Script for a visit to the vet
Entry conditions: dog illvet openowner has money
Result: dog betterowner poorervet richer
Props: examination tablemedicineinstruments
Roles: vet examines diagnoses treatsowner brings dog in pays takes dog out
Scenes: arriving at receptionwaiting in roomexaminationpaying
Tracks: dog needs medicinedog needs operation
Models of LTM - Production Rules
• Representation of procedural knowledge.
• Condition/action rules if condition is matched
then use rule to determine action.
IF dog is wagging tail, THEN pat dog
IF dog is growling, THEN run away
LTM - Storage of information
• LTM much studied in psychology:
• Rehearsal– information moves from STM to LTM
• Total time hypothesis– amount retained proportional to rehearsal time
• Distribution of practice effect– optimized by spreading learning over time
• Structure, meaning and familiarity– information easier to remember
LTM - Forgetting
• Decay– information is lost gradually but very slowly
• Interference– new information replaces old: retroactive interference– old may interfere with new: proactive inhibition
• So, ... may not forget at all, memory is selective …!
• Also, affected by emotion – can subconsciously `choose' to forget
LTM - Retrieval
• Should be familiar from heuristics ...
• Recall– information reproduced from memory can be assisted by cues,
e.g. categories, imagery
• Recognition– information gives knowledge that it has been seen before– less complex than recall - information is cue
Thinking – Cognitive Processing
• Humans reason, process information, like, well, humans– Recall, any theory is an abstraction and, thus, captures some elements of
phenomenon, and misses others– Question is …
• Is the account (theory, model) useful in the context and for the purpose for which it is used?
Thinking – Cognitive Processing
• Humans reason, process information, like, well, humans– Recall, any theory is an abstraction and, thus, captures some elements of
phenomenon, and misses others– Question is …
• Is the account (theory, model) useful in the context and for the purpose for which it is used?
• Basic forms of reasoning, or, forming inferences, are useful in understanding broad outlines of human cognition– Deduction– Induction– Abduction
• Problem solving– Gestalt– Problem Space– Analogy– Skill acquisition
Reasoning
• Deduction:– derive logically necessary conclusion from given premises
• e.g., If it is Friday, then she will go to work - It is Friday, therefore she will go to work
– Logical conclusion not necessarily true:e.g., If it is raining, then the ground is dry - It is raining, therefore the ground is dry
• Induction:– Generalize from cases seen to cases unseen
• e.g., All elephants we have seen have trunks - therefore all elephants have trunks.– Unreliable (but useful):
• Can only prove false not true
• Abduction:– Reasoning from event to cause e.g., Sam drives fast when drunk. If I see Sam driving fast, assume drunk.
– Unreliable:• can lead to false explanations
Induction vs. Deduction
• Deduction: Formulate hypothesis first, then test hypothesis
– Via experiment and accept/reject
– Data collection more targeted than in induction
– Only limited data mining opportunities
Mueller, 2003
Induction vs. Deduction
• Induction: Make observations first, then draw conclusions
– Organized data survey (structured analysis, visualization) of raw data provide basis for interpretation process
– Interpretation process will produce knowledge that is being sought
– Experience of individual scientist (observer) is crucial
– Important: selection of relevant data, collection method, and analysis method
– Data mining is an important knowledge discovery strategy
• ubiquitious data collection, filtering, classification, and focusing is crucial Mueller, 2003
Problem Solving
• Process of finding solution to unfamiliar task using knowledge– Complex, time consuming process– Selections not immediately obvious– May require many steps– May involve insight– May use analogy– Solutions often counterintuitive– Several theories, or accounts
• Gestalt – Problem solving both productive and reproductive– Productive draws on insight and restructuring of problem– Attractive but not enough evidence to explain “insight” etc.– Move away from behaviourism and led towards information processing theories
• Others: Insight, Functional fixedness, Analogy
Problem Solving Cycle
• One schema – consider “task performance”
Insight
• Early - Kohler (a Gestalt psychologist) in Canary islands in WWI– Studied problem solving in chimpanzees– Sultan and the Banana:
• Learned how to get banana with longer pole• Then given shorter poles that wouldn’t reach• Flash of “insight”, Sultan put the poles together
• Sudden perception of useful or proper relations– Solutions will sometimes “spring to mind”– Pieces fall into place
• First attempts to solve don’t work– Production hindered by unwarranted assumptions– Insight occurs when the assumption is removed
Problem Solving (cont.)
• Problem space theory
– Problem space comprises problem states
– Problem solving involves generating states using legal operators
– Heuristics may be employed to select operatorse.g. means-ends analysis
– Operates within human information processing systeme.g. STM limits etc.
– Largely applied to problem solving in well-defined arease.g. puzzles rather than knowledge intensive areas
Problem Solving (cont.)
• Analogy
– Analogical mapping:
• novel problems in new domain?
• use knowledge of similar problem from similar domain
– Analogical mapping difficult if domains are semantically different
• Skill acquisition – e.g., “expert” performance
– Skilled activity characterized by chunking
• lot of information is chunked to optimize STM
– Conceptual rather than superficial grouping of problems
– Information is structured more effectively
Individual Differences
• Long term– Gender, physical and intellectual abilities
• Short term– Effect of stress or fatigue
• Changing– Age
• Dix says ask: – Will design decision exclude section of user population?– (or, more generally) How does design differentially affect sections of the
population?
– i.e., Universal Usability
Card’s “Model Human Processor + Attention” isSimilar to Ware (2004) Model
• … one more model of cognitive (and visual) processing– All are in fact much the same,
but focus on different goals
Card’s “Model Human Processor + Attention” isSimilar to Ware (2004) Model
• … one more model of cognitive (and visual) processing– All are in fact much the same,
but focus on different goals
• Card model - context of predicting user performance– E.g., set parameters and
perform simulation
• Ware’s model includes much the same elements …– But focuses on those which are
most relevant for processing of visual information in context of task performance
Card’s “Model Human Processor + Attention” isSimilar to Ware (2004) Model
• Sensory store– Rapid decay “buffer” to hold
sensory input for later processing
• Perceptual processor– Recognizes symbols, phonemes– Aided by LTM
• Cognitive processor– Uses recognized symbols– Makes comparisons and
decisions– Problem solving– Interacts with LTM and WM
• Motor processor– Input from cog. proc. for action– Instructs muscles– Feedback
• Results of muscles by senses
• Attention– Allocation of resources
A Model of Perceptual ProcessingQuick Overview
• An information processing (the dominant paradigm) model– “Information” is transformed and processed
• Physical light does excite neurons, but at this “level of analysis” consider information
– Gives account to examine aspects important to visualization• Here, clearly, many neural subsystems and mapping of neural to ip is pragmatic
– In spirit of visualization as evolving discipline, yet to develop its theories, laws, …
• Stage 1: Parallel processing to extract low-level properties of the visual scene• Stage 2: Pattern perception• Stage 3: Sequential goal-directed processing
What we do is design information displays!
Stage 1: Parallel Processing to Extract Low-level Properties of Visual Scene
• (Very first) neurons fire
• Visual information 1st processed by – large array of neurons in eye – primary visual cortex at back of brain
• Individual neurons selectively tuned to certain kinds of information– e.g., orientations of edges or color of
light– Evoked potential experiments
• In each subarea large arrays of neurons work in parallel– extracting particular features of
environment (stimulus)
Stage 1: Parallel Processing to Extract Low-level Properties of Visual Scene
• At early stages, parallel processing proceeds involuntarily– Largely independent of what choose
to attend to (though not where look)
• Is rapid, – If want people to understand
information fast, should present in way so is easily detected by these large, fast computational systems in brain
• Stage 1 processing is:– Rapid and parallel– Entails extraction of features,
orientation, color, texture, and movement patterns
– “transitory”, briefly held in iconic store– Bottom up, data-driven
Stage 2: Pattern Perception
• Rapid processes
• Divide visual field into regions and simple patterns, e.g.,– Continuous contours – Regions of same color – Regions of same texture– …
• “Active”, but not conscious processes
• Specialized for object recognition– Visual attention and memory
• E.g., for recognition must match features with memory
– Task performing will influence what perceived
– Bottom up nature of Stage 1, influenced by top down nature of Stage 3
Stage 2: Pattern Perception
• Specialized for interacting with environment– E.g., tasks involving eye-hand coordination
• “Two-visual system hypothesis”– One system for locomotion and eye-hand
coordination --- The “action system”– One system for symbolic object
manipulation --- The “what system”
• Characteristics:– Slower serial processing– Involvement of both working (vs. iconic)
and long-term memory– Both bottom up and top down
• More emphasis on arbitrary aspects of symbols than Stage 1
• Top-down processing– Different pathways for object recognition
and visually guided motion
Stage 3: Sequential Goal-Directed Processing
• At highest level of perception are the objects held in visual memory by demands of active attention
• To use an external visualization, we construct a sequence of visual queries that are answered through visual search strategies
• Only a few objects can be held in memory at a time
• They are constructed from available patterns providing answers to the visual queries
Stage 3: Sequential Goal-Directed Processing
• They are constructed from available patterns providing answers to the visual queries– E.g., if use a road map to look for a
route, the visual query will trigger a search for connected red contours (representing major highways) between two visual symbols (representing cities)
• Are other subsystems, as well– Visual object identification process
interfaces with the verbal linguistic subsystems of the brain so that words can be connected to images
– The perception-for-action subsystem interfaces with the motor systems that contril muscle movements
End
• .