COGS 121HCI Programming Studio

Week 03

Direct Manipulation

Principles of Direct Manipulation

1. Continuous representations of the objects and actions of interest with meaningful visual metaphors.

2. Physical actions or presses of labeled buttons, instead of complex syntax.

3. Rapid, incremental, reversible actions whose effects on the objects of interest are visible immediately.

6-15

Examples of Direct-Manipulation Systems

Command line vs. display editors and word processors • Training times with display editors are much less than line editors • Line editors are generally more flexible and powerful • The advances of WYSIWYG word processors:

• Display a full page of text • Display of the document in the form that it will appear when the

final printing is done • Show cursor action • Control cursor motion through physically obvious and intuitively

natural means • Use of labeled icon for actions • Display of the results of an action immediately • Provide rapid response and display • Offer easily reversible actions

6-3

Examples of Direct-Manipulation Systems

WYSIWYG word processing

6-4

Examples of Direct-Manipulation Systems

Spreadsheet

6-7

6-10

Examples of Direct-Manipulation Systems

Guitar Hero VideoGame

Continuing evolution of Direct-Manipulation Systems

Direct-Manipulation interfaces are being used in a wide range of applications, e.g. management dashboard for a retail store

6-12

Interactive Data Visualization?

ObamaBudgethttp://www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html

CongressionalInfluenceshttp://www.brightpointinc.com/interactive/political_influence/index.html

ParallelCoordinateshttp://bl.ocks.org/mbostock/1341021

Cross-Filterhttp://square.github.io/crossfilter/

How people do things (Hutchins, Hollan, Norman)

1. To get something done, you start with some notion of what is wanted – the goal to be achieved

2. Then you do something to the world – take action to move yourself or manipulate someone or something

3. Finally, you check to see that your goal was made

● Human action has two primary aspects • Execution: doing something • Evaluation: comparison of what happened to what was

desired (to our goal)

Hutchins, Edwin L., James D. Hollan, and Donald A. Norman. "Direct manipulation interfaces." Human–Computer Interaction 1.4 (1985): 311-338.

Execution / Evaluation Action cycle

Stages of Execution

● Goals do not state precisely what to do • Where and how to move, what to pick up

● To lead to actions, goals must be transferred into intentions • A goal is something to be achieved • An intention is a specific set of actions to get to the goal

• Yet even intentions are not specific enough to control actions

Stages of Execution Example

● “I am reading a book and decide to need more light”

1. My goal: get more light 2. Intention: push the switch button on the lamp 3. Action sequence (still a mental event) to satisfy intention: move

my body, stretch to reach the switch extend my finger 4. Physical execution: action sequence executed

• Note that I could satisfy my goal with other intention and action sequences

• Instead of pushing the switch, ask another person to switch on the light • My goal hasn’t changed, but the intention and the resulting action sequence

have

Stages of Evaluation Example

● “I am reading a book and decide to need more light”

1. My goal: get more light 2. Intention: push the switch button on the lamp 3. Action sequence (still a mental event) to satisfy intention: move

my body, streach to reach the switch extend my finger 4. Physical execution: action sequence executed

5. Perceive whether there is more light in room 6. Decide whether the lamp turned on 7. Decide whether the resulting amount of light is sufficient

Seven stages of action● 1 for goals, 3 for execution and 3 for evaluation

• Note: only an approximatemodel

1. Forming the goal 2. Forming the intention 3. Specifying an action 4. Executing the action 5. Perceiving the state of the world 6. Interpreting the state of the world 7. Evaluating the outcome

Seven stages of action

Visual Perception

Human Perception & Cognition

• We perceive what we expect

• Our vision is optimized to see structure

• We seek and use visual structure

• Reading is unnatural

• Our color vision is limited

• Our peripheral vision is poor

We perceive what we expect

• Our perception is biased by:

• Our experience

• The context

• Our goals

\

We perceive what we expect

TopHat Question

We perceive what we expect

Next

We Perceive What We Expect!

Page 1 2 3 4 !

Back

TopHat Question

We perceive what we expect

Next

We Perceive What We Expect!

Page 1 2 3 4 !

Back

Our Vision is Optimized to See Structure

Gestalt Principles of Visual Perception

• Proximity: occurs when elements are placed close together. They tend to be perceived as a group.

• Similarity: occurs when objects look similar to one another. People often perceive them as a group or pattern.

• Continuity: occurs when the eye is compelled to move through one object and continue to another object.

• Closure: occurs when an object is incomplete or a space is not completely enclosed. If enough of the shape is indicated, people percieve the whole by filling in the missing information

• Symmetry: occurs when the mind perceives objects as being symmetrical and forming around a center point. It is perceptually pleasing to divide objects into an even number of symmetrical parts.

• Figure/ground: The eye differentiates an object form its surrounding area. a form, silhouette, or shape is naturrally perceived as figure (object), while the surrounding area is perceived as ground (background).

• Common fate: occurs when objects are perceived as lines that move along the smoothest path

Gestalt Principle: Closure• We tend to see whole, closed objects, not

collections of fragments

• Overlapping circles & triangles, not odd fragments

Gestalt Principle: Symmetry• We tend to see simple figures rather than

complex ones

•

• E.g., two overlapping diamonds; not other, more complex combinations

We Seek & Use Structure• Structured information is easier to perceive

We Seek & Use Structure!

Flight: United 237, Auckland ➔ San FranciscoDepart: 14:30 Tue 15 Oct Arrive: 11:40 Tue 15 Oct

You are booked on United flight 237, which departs from Auckland at 2:30pm and arrives at San Francisco at 11:40am Tuesday 15 Oct

Flight: United 237, Auckland ➔ San Francisco Depart: 2:30pm Tue 15 Oct Arrive: 11:40am Tue 15 Oct

We Seek & Use Structure• Visual hierarchy gets people to goal faster

20

Reading is Unnatural• We’re pre-wired for language

• Brain learns language easily in childhood

• Nearly everyone learns a language

• We are not pre-wired for reading

• Brain has no special facility for reading

• Learning reading is like learning other skills: writing, arithmetic, reading music, kung fu

• Many people never learn to read well, or at all

TopHat Attendance

Gestalt Principles

TopHat Question

Colors● Color is a powerful (visual) communication medium

• shapes our perception, interpretation and memory or what we see • it can enhance the effectiveness of a message • likewise, it may impair it

● Human color perception has both strengths and limitations • vision optimized to detect contrast (edges) not absolute brightness • our ability to distinguish colors depends on how colors are presented • color-blindness • the user’s display and the environmental conditions affect color perception

Vision is Optimized for Edge Contrast, Not Brightness

● Opponent color process (subtractions) make our visual system much more sensitive to differences in color and brightness (edgecontrast) than to absolute brightness levels

• Compare the two circles: are they the same? • they are exactly the same (the size and the color shade) • demonstrates insensitivity to absolute brightness

Vision is Optimized for Edge Contrast, Not Brightness

Insensitivity to brightness and sensitivity to contrast by E. H. Adelson

ThesquaresmarkedAandBarethesamegray.WeseeBaswhitebecauseitis“shaded”bythecylinder!(checkwithaneyedropper)

TopHat Question

Ability to Discriminate Colors Depends on how Colors are Presented• Even our ability to detect color differences is limited

• Three presentation factors affect our ability to distinguish colors from each other

A. Paleness: The paler (less saturated) two colors are, the harder is to tell them apart

B. Color patch size: The smaller or thinner objects are, the harder it is to distinguish their colors

C. Separation: The more separated color patches are, the more difficult it is to distinguish their colors, especially if the separation is great enough to require eye motion between patches

Our Color Vision is Limited• A lot of people have color-blindness

● Approximately 8% of male and around 0.5% female population suffer some form of color-blindness

• E.g., colors that would be hard for red-green colorblind people to distinguish

Color-blindness● Being color-blind does not mean seeing grey or black and

white

● It means that one or more ‘color subtraction channels’ do not function normally • It becomes difficult to distinguish certain pairs of colors

• The most common type of colorblindness is red/green • This means that your boss or even worse your investor is potentially affected

normalcolorvision(1%ofmalepopulation)(6%ofmalepopulation)(1%ofmalepopulation)

Opponent ColorsSeparatestrongopponentcolors

• placing opponent colors right next to or on top of each other causes a disturbing flickering sensation

Our Peripheral Vision is Poor

Our Peripheral Vision is Poor, so…

• Common methods of getting seen • Information where users are looking • Near the error • Use red for errors • Use error symbol

• Heavy artillery: use with parsimony • Popup in error dialog box • Audio: beep • Flash or wiggle briefly (not continuously)

Our Peripheral Vision is Poor, so…

• Common methods of getting seen • Information where users are looking • Near the error • Use red for errors • Use error symbol

• Heavy artillery: use with parsimony • Popup in error dialog box • Audio: beep • Flash or wiggle briefly (not continuously)

ExploitingHumanPerceptualFacilitiesUsetheeyeforpatternrecognition;peoplearegoodatscanningrecognizingrememberingimages

Graphicalelementsfacilitatecomparisonsvialengthshapeorientationtexture

AnimationshowschangesacrosstimeColorhelpsmakedistinctionsAestheticsmaketheprocessappealing

Interface-Building ToolsVisual Thinking and Icons

• An icon is an image, picture, or symbol representing a concept

• Icon-specific guidelines • Represent the object or action in a familiar manner • Limit the number of different icons • Make icons stand out from the background • Consider three-dimensional icons • Ensure a selected icon is visible from unselected icons • Design the movement animation • Add detailed information • Explore combinations of icons to create new objects or

actions

6-16

Next Steps• Technical Lecture: Thu 4/14

-Data Mining, Data Processing and Data Modeling + Assignment 2 bootstrapping

• Technical Discussions (required), Fri 4/15-Troubleshooting with Assignment 2-Quiz on Week 2 Content

• Readings (required)- Shneiderman (Designing the User Interface) - Chapter 5 - Johnson (Designing With the Mind in Mind) - Chapter 1, 2, 3, 4, 5

• Next Lecture: Tue 4/19-The Art of Web and Visual Desing

• Now: Interactive Activity -Gestalt and Perception