Thinking with Visualizations Ware Chapter 11 University of Texas – Pan American CSCI 6361, Spring...
-
Upload
rafe-morton -
Category
Documents
-
view
218 -
download
0
Transcript of Thinking with Visualizations Ware Chapter 11 University of Texas – Pan American CSCI 6361, Spring...
Thinking with Visualizations
Ware Chapter 11
University of Texas – Pan AmericanCSCI 6361, Spring 2014
Overview
• Cost of knowledge
• Information Foraging Theory – Pirolli and Card– “wildly popular”
• Cognition and computers
• Ware’s account (“theory”) of how thinking (and problem solving) can be augmented by visual queries on visualizations of data– Based on 3-stage account presented in book
• “Creativity” and visualization
Cost of Knowledge, Cognition, and Computers, 1
• Information systems (primarily computer) and “cost” of acquiring knowledge
– A first principle of information system design– “Cognitive information ergonomics”
• Efficiency/productivity gain/usability/…
• There is (and has always been) a cost to acquire information / knowledge
– cost = user/worker time +, e.g., machine cost, db access charge, book
– Ware’s chapter concerned with • “economics of cognition and the
cognitive cost of knowledge”
Cost of Knowledge, Cognition, and Computers, 2
• There have been many studies that fail to document increased profit directly from implementation of (single) information system
– However, there is no doubt that worker productivity in late 20th century dramatically increased
– Productivity greatly enhanced by pervasive use electronic information systems (computers)
• Visualization is one element of information system use– As such, as it adds to efficiency/effectiveness it is economically useful– I.e., it is part of the equation
• As noted in 1st lecture, entire paradigm of information access, as indicated by order of magnitude of information accessible, seems to have changed
– Ware: “cognitive cyborgs”– And, if you think about it, I guess so …
• Even preparing this lecture, looking at big screen, going to old papers and presentations, going via internet for images, going via internet to papers, …
• “Siri, where’s the best place bar-b-que?”
Informavores and Information Foraging
• That human information seeking is innate and adaptive is well known
• Humans are informavores– George Miller, 1983, “… magic number 7 + 2”– Organisms that hunger for information about the world
and themselves
• “A wealth of information creates a poverty of attention and a need to allocate it efficiently”– Herb Simon, AI, Nobel prize, cognition
• Consider analogy of acquiring knowledge with animals seeking food– Pirolli, P. and S. Card (1995). Information Foraging in
Information Access Environments, in CHI '95, p. 518– Pirolli, P. (2007) ….. Book …..
Information Foraging Theory (IFT)
• Information Foraging Theory (IFT)– Pirolli and Card – Xerox PARC … and lots of biologists– “an approach to the analysis of human activities involving information
access technologies”– Derives from optimal foraging theory (oft) in biology and anthropology
• Analyzes adaptive value of food-foraging strategies
• Analyzes trade-offs in value of information gained against the costs of performing activity in human-computer interaction tasks– And need models and analysis techniques to determine value added by
information access, manipulation, and presentation techniques
• Real information system design problem is not how to collect more information, but how to optimize user’s time– Increase relevant information gained per unit time expended
• IFT provides a relatively “formal” (quantitative) account
IFT – Time Scales
• Considers “adaptiveness of human-system designs in the context of the information ecologies in which tasks are performed”
– Ecology, as system, here, information
• Time scales of information seeking and sense making activities
– Cognitive band (~100 ms – 10 s)– Rational band (minutes to hours)– Social band (days to months)
• Have seen much of cognitive, now others
Time Scales of Analysis
Time scale (s)Psychologicaldomain
10-1000 • Problem solving• Decision making
1-100• Visual search• Motor behavior
Pete Pirolli's Home Page
Peter Pirolli. ... Palo Alto, CA 94304 USA phone: +1-650-812-4483 fax: +1-650-812-4241
email: [email protected] This page updated December 18, 2000.
www.parc.xerox.com/istl/members/pirolli/pirolli.html - 9k - Cached - Similar pages
.100-1• Visual attention• Perceptual judgment
User Interface Domain
IFT – An Ecological Perspective
• Again, time scales of information seeking and sense making activities– Cognitive band (~100 ms – 10 s)– Rational band (minutes to hours)– Social band (days to months)
• As time scale increases, less regard for how internal processing accomplishes linking of actions to goals
• Assumes behavior governed by “rational principles and shaped by constraints and affordances of the task environment”
• An ecological perspective– I.e., that behavior is “adaptive” in that it accomplishes some goal
IFT – Metaphor and Quantitative
• IFT - name a metaphor and straightforward use of biological “optimal foraging theory”
• Metaphor:– Animals adapt behavior and structure through evolution
• (humans don’t have to wait that long!)– Animals adapt to increase their rate of energy intake, etc.
• To do this they evolve different methods• E.g., wolf hunts prey, spiders build webs and wait
• And there are analogies to this– E.g., hunting = active information seeking, waiting = information filtering– Humans (and others) hunt in groups when variance of food is high
• Accept a lower expected mean to minimize probability of days without food– Also, on social time scale, sharing of information
Optimal Foraging Theory - Biology
• Developed in biology for understanding opportunities and forces of adaptation– P&C use elements of the theory to help in understanding existing
human adaptations for gaining and making sense of information– Also, aid in task analysis for creating new interactive information system
designs
• Optimality models include:– Decision assumptions
• Which of the problems faced by an agent are to be analyzed• E.g., whether to pursue a particular type of information (or prey) when
encountered, how long to spend– Currency assumptions
• How choices are to be evaluated, e.g., information value (food value)– Constraint assumptions
• Limit and define relationships among decision and currency variables• E.g., from task structure, interface technology, user knowledge
Information Foraging Theory (IFT)
• Information foraging usually a task
embedded in context of other task– Value and cost structure defined in
relation to the embedding task– Value of external information may
be in improvements to outcomes of
embedding task
• Usually, embedding task is some ill-structured problem– Additional knowledge needed to better define goals, available actions, heuristics, etc.– E.g., choosing a graduate school, developing business strategy
• Though use optimality model, not imply human behavior is classically rational
– I.e., have perfect information and infinite computational resources– Rather, humans exhibit bounded rationality, or make choices based on satisficing
IFT – Information Patch ModelFormal Model
• Information patch model – from optimal foraging theory
• Rate of currency (“utility”) intake, R = U / (Ts + Th)– U = net amount of currency gained
– Ts = time spent searching
– Th = time spent exploiting
• Net currency gain, U = Uf - Cf
– Uf = overall currency intake (gross amount foraged)
– Cf = currency expended in foraging
• Average rate of currency intake u = Uf / Ts
– If assume information workers/foragers/consumers encounter information as linear function of time
– Total n items encountered = Ts, where is rate of encounter with items
IFT – Information Patch ModelFormal Model - Details
• Average cost of handling items:
• Let s = search cost per unit time, then total cost of search = sTs
• Then, substituting in equation for R, rate of currency intake:
• So, can express R in terms of – Average rate of currency intake, u– Search cost per unit time, s– Cost of handling items, h
IFT – Information Patch Model
• And so forth …, cf. ‘95 CHI paper
An Example: Scatter Gather
• Hierarchical clustering of document– Users see “overview” of document clusters
– Allows user to navigate through clusters and overviews
Scatter/Gather Task
Scatter/GatherWindow
Law
World News
AI
CS
Medicine
Nat. Lang.
Robots
Expert Sys
Planning
Bayes. Nets
Display TitlesWindow
Optimal Foraging Time in a Patchor, how long to spend before moving on
• gi(t), cumulative information gain function– Amt of information gained in time t
– gA(t) = random order of encounter
– gB(t) and gc(t) = ordered by relevancy• gC encounters most relevant “faster” than (before) gB
• p, rate of encounter with relevant items
• x-axis, travel time between patches
• RB and RC = rate of return– Again, differential rate rel. item encounter
• tc and tb optimal foraging time– Foraging longer in the “patch” not optimal
IFT - Cost of Knowledge
• Foraging Efficiency– Animals minimize energy expenditure to get required gain in sustenance– Humans minimize effort to get necessary gain in information
• Again, foraging for food has much in common with seeking information– Like edible plants in wild, useful information items often grouped together, but
separated by long distances in an “information wasteland”
• Also, information “scent” – a very popular metaphor– Like scent of food, information in current environment that will assist in finding more
information clusters
• Activities analyzed according to value gained and the cost incurred– The “cost of lost (or chosen) opportunity”
– Resource costs • Expenditures of time and cognitive effort incurred
– Opportunity costs• Benefits that could be gained in engaging in other activities• E.g., if not gaining information about visualization, could be gaining information about
software design
IFT – Evolution of Systems
• Information processing systems evolve so as to maximize the gain of valuable information per unit cost– Sensory systems (vision, hearing)
– Information access (card catalogs, offices)
information valuecost of interaction( )maximize
Information Scent
Tokyo
San Francisco
New YorkCues that facilitate orientation, navigation, assessment of information value
Information Scent
cell
patient
dose
beam
new
medical
treatments
procedures
InformationNeed
Text snippet
• Spreading activation– Derived from models of human memory– Activation reflects likelihood of relevance
given past history and current context– Approximates Bayesian network
0
1
2
3
4
5
6
7
1 2 3 4 5 6 7 8 9 10
Rank
Pro
bab
ilit
y r
ele
van
t
Observed Rating
Predicted Rating
Optional:Cluster selection (optimal diet model)
Rank profitability
Rel
evan
t do
cum
ents
/sec
ond
0
2
4
6
8
10
12
14
16
0 1 2 3 4 5 6 7 8 9 10
Number of relevant documents in cluster
Time to process cluster =
Total relevant documentsTotal timeR =
R
Optimum
Choose clusters (in descending rank ) if > R
Optional:Enrichment vs. Exploitation
0 200 400 600 800 10000
.01
.02
.03
.04
.05
.06
R*SG
R*D
Time (sec)
Rat
e o
f ga
in
R*SG > R*
D R*D > R*
SG
relevant documentstime cost
if user chooses to display clusters now
if user choosesto display later(after more Scatter/Gather)
R=
Cost of Knowledge Characteristic Function
• Cost of knowledge characteristic function
– Webforager at right
• Measures access properties of a workspace
– Plot n objects can be accessed as a function of time cost of accessing them
– Expect a “balanced” workspace will exhibit an exponential relationship,
• Most conveniently displayed as a straight line in semi-log coordinates.
• For Webforager example, computation assumes
• Page in the Focus Position (hence the maximal occlusion),
• Desk is full• One row of pages from each of the discrete
Z-distances in the space is visible – Design of the space has been carefully set
up to permit this
Navigation as a Cost of Knowledge
• Intra-saccade – (0.04 sec) – Query execution
• An eye movement – (0.5 sec) – < 10 deg : 1 sec> 20 deg.
• A hypertext click – (1.5 sec, but loss of context)
• A pan or scroll – (3 sec, but don’t get far)
• Walking – (30 sec., don’t get far)
• Flying – (faster, can be tuned)
• Zooming, t = log – (scale change)
• Fisheye – (max 5x)
Models
• A popourri …
Human-Machine Problem Solving System(quick review – Ware’s “first principles” model)
• From first lecture or so:– Human is good at
• Hypothesis formation• Goal-directed search• Pattern recognition• Decisions in the presence of error and uncertainty
• Interface– Visual channel is highest-bandwidth from computer to human– Haptic channel is the only bi-directional modality
• Problem solving loop– People solve problems with diagrams differently from the way they do it without diagrams– Visualizations functions as memory extensions– Visualizations enable cognitive operations that would otherwise be impossible
- Computer is good at:•Perfect recall of facts•Quantitative display of complex models
Human Memory Structures and Visualization Augmentation
• Iconic Memory– Image of last thing seen still in
pictorial form– Trace of last burst of sound heard
• Working Memory– 7 +/- 2 “chunks” can be stored– Augmentation:
• Objects hold information for cognitive operations
– Can refresh from visual scene without concentration of “forming a mental image”
• Visual scene can be much more complex and accurate than that reconstructed in working memory
• Long-Term Memory– Episodic memory, motor skills,
reading, etc.– Network of linked concepts– Augmentation
• Visual images activate long-term memory of images and objects (recognition vs. recall)
• Example –as simple as maps:– Driving route for vacation trip
• Presents alternative routes precisely• Highlighted path preserves
information about the route that has been planned so far
– Ship route planning through a harbor• Precise plots of ship and fixed
obstacle positions• Accurate simulation of turning of ship
– Frees pilot of one cognitive task to concentrate on others
Unified Cognitive Model
• A task/human-factors oriented model:
– Kieras and Myers, 1997
• Note that linkages with computers are asymmetric with respect to bandwidth
– 10’s of MB/sec from computer to human visual channel
– Relatively low-bandwidth return paths– Lends itself to steerable computing,
surfing the web
Visual Spatial Reasoning
• External representations guide and constrain thinking– Which way would you like to play Tic-Tac-Toe?
• Cognitive operations can take place directly in the visual representation– No internal model is needed for Tic-Tac-Toe a)– Internal computation needed for version b)
• The problem can be partially externalized– Frees user from having to do some operations– Provides increase in overall capacity
Rensink’s Model of Visual Attention
• Ware’s account of problem solving draws heavily from this
• ALL should be familiar:– Parallel processing of elements– “Hardwired” / built-in detection of
elements– Extraction of proto-objects
• For object recognition– Setting and object systems
provide part of Ware’s “middle stage”
– Attention as spotlight– Etc., …
Problem Solving with Visualizations,Ware’s (perceptually focused) Account
• Ware’s account (“theory”) of how thinking (and problem solving) can be augmented by visual queries on visualizations of data
• Based on 3-stage account presented in book:– 1st (lowest) stage,
• massively parallel processing of visual scene into elements of form, opponent colors, and elements of texture and motion
– 2nd (middle) stage, • pattern formation, providing basis for object and pattern
perception– 3rd (highest) stage,
• mechanism of attention pulls out objects and critical patterns form the pattern analysis subsystem to execute a visual query
• Content of visual working memory consists of “object files”
– a visual spatial map in egocentric coordinates that contains residual information about a small number of recently attended object
• Also present is visual query pattern that forms the basis for active visual search through the direction of attention
DisplayFeatures
Proto-objects andPatterns
VisualWorkingMemory
GIST
VisualQuery
VerbalWorkingMemory
Egocentric object andPattern map
Interaction Loop
Problem Solving with Visualizations,Ware’s Account
• Somewhat generic account of problem solving with emphasis on roles of visualization (and visual system)
• Set of embedded processes:
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Problem Solving with Visualizations,Ware’s Account – more detail than last week, ex. follows
• Key features of visual thinking (and problem solving) process …
1. Problem components are identified that have potential solutions based on visual pattern discovery
– These are formulated into visual queries consisting of simple patterns
2. Eye-movement scanning strategies are used to search display for query patterns
3. Within each fixation, query determines which patterns are pulled from flux of pattern-analysis subsystems
– Patterns and objects are formed as transitory object files from proto-object and proto-pattern space
– Only a small number of objects or pattern components retained from one fixation to the next • These object files also provide links to verbal-propositional information in verbal working memory
– A small number of cognitive markers are placed in a spatial map of the problem space to hold partial solutions where necessary.
• Fixation and deeper processing are necessary for these markers to be constructed.
4. Links to verbal-propositional information are activated by icons or familiar patterns, bringing in other kinds of information
Problem Solving with Visualizations,Ware’s Account
• Somewhat generic account of problem solving with emphasis on roles of visualization (and visual system)
• Set of embedded processes:
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
An Example – Edinburg to Chicago
• Planning trip with aid of map– Find route/destinations
• From here to Chicago– Have some extra time and
want to visit some places– So, not just minimum time
• Will examine each element in problem solving loop:
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Problem-solving Strategy
• To “solve problem”, first formulate set of requirements:
– Say, will take 10 days– See cities and different regions– Weight driving time with interest
of places to visit– Cost of mileage weighted less
than cost of lodging– Will use information sources
such as Internet & Barnes and Noble travel guides
– Etc.
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Visual Query Construction
• Establish location of various cities through series of preliminary visual queries
– Fixating city icon and label helps establish connection to verbal-propositional knowledge about city
– Little remains in working memory, but primed for later reactivation
• Now, path planning begins with major alternative routes
– Trading off travel time for “interest”– Note that roads are coded as major
and cities with size in visual representation
– More than just graph traversal• Paths have “value”, e.g., interest, as
well as “cost”, e.g., time
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Pattern-finding Loop – “exploration”
• Now find all acceptable routes, as defined in previous step
• Essentially, finding “contours”– E.g, EP, Denver, …, or, Hou, NO, St.
Louis
• Can hold 2 or 3 in working memory
• When route found, “code” and hold in verbal-propositional memory
– E.g., western, central (Miss. Valley), etc.
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Pattern-finding Loop – “refinement”
• Now find all acceptable routes, as defined in previous step
• Essentially, finding “contours”– E.g, EP, Denver, …, or, Hou, NO, St.
Louis
• Can hold 2 or 3 in working memory
• When route found, “code” and hold in verbal-propositional memory
– E.g., western, central (Miss. Valley), etc.
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Scr Pattern-finding Loop – “refinement”
• Now find all acceptable routes, as defined in previous step
• Essentially, finding “contours”– E.g, EP, Denver, …, or, Hou, NO, St.
Louis
• Can hold 2 or 3 in working memory
• When route found, “code” and hold in verbal-propositional memory
– E.g., western, central (Miss. Valley), etc.
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Eye Movement Control Loop
• Detailed execution of pattern-finding carried out through series of eye movements to capture major continuous paths meeting criteria
• Eye movements planned using task-weighted spatial map of proto-patterns
– Promising partial paths are given attention, starting with most significant
– Partial solutions marked by placeholders in egocentric spatial map
• E.g., mark Denver, Dallas or NO and explore around
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
Intrasaccadic Scanning Loop
• Here, information available through single fixation is processed
• Sections of lines representing roads are processed through selective tuning of pattern-finding mechanism
– E.g., those going in wrong direction or are minor are rejected
– Only 3 or 4 small sections held in working memory at a time
• City names also processed, contributing to overall process
-Problem-solving strategy
-Visual query construction
-Pattern-finding loop
-Eye movement control loop
-Intrasac. image-scanning
“Creativity” and Problem Solving
• From Ware, perspective on problem solving …– “Well used”…– Provides differing emphasis/orientation than previous
• Preparation: Acquire background information– Exploratory data analysis: getting the lay of the land– Visualization is especially important here, providing multiple views of the problem with different characteristics– Preparation through multiple representations …
• Production: Propose a Solution– How to support tentative, sketchy interact. in visualization?– Probably by better interface devices and interaction
metaphors
• Judgment: How well does it work?– Comparing hypotheses against the data– Discard incorrect, already known, and irrelevant models– Sift out novel ones leading to practical solutions– Visualization enables direct visual comparison of model and
data
Wrapping Up:Tools to Help Scientists Build Better Models
• Computer’s roles
• Estimate model from experiment data using image analysis
• Quantitatively simulate instrument’s view of a model
• Display combined experiment and model-derived data to enable direct comparison
Wrapping Up:Supporting Human Cognition
• Provide visual display of relevant factors– Layout should reflect important relationships
• Provide tools to answer common questions– Relieves user of mental calculation– Provides more precise and rapid calculation
• Iteration is unavoidable– Adding visualization changes the task the user is doing– Adding visualization changes the user’s capabilities
• Avoid misleading the user– Lies of Commission: More verisimilitude than veracity– Lies of Omission: Hiding important information
Really Wrapping UP: About Visualization:Usefulness and Truthfulness, not Ornamentation
• “A toolmaker succeeds as, and only as, the users of his tools succeed with his aid. However shining the blade, however jeweled the hilt, however perfect the heft, a sword is tested only by cutting. That swordsmith is successful whose clients die of old age.”
– F. P. Brooks, Jr., "Grasping Reality from Illusion"
End
• .