Introduction –

Human Capabilities:

Cognition and Perception

Dr Joan CondellSupported by Ms Anne Hinds

Advanced HCI – COM719M1

Module Information

http://www.infm.ulst.ac.uk/~joan

See Specific Reading List forvarious texts and journals

•UU library•‘My library’ of books•Internet

Recommended Reading

Module Topics

Weeks 1 to 6 Lectures on specific topics:

Human Capabilities

HCI Accessibility

Tasks and Contexts

Participatory Systems Design

Usability Evaluation

Weeks 7 and 8 Individual Project Presentation/Demos

Weeks 9 to 11Group Presentations

Week 12Revision for exam

Individual project: Design and implement a simple interactive system Individual presentation Prototype and Report Submission during week 6 Presentations run weeks 7 and 8 Examples from previous years:

Belfast Shipping Ltd Quoting System Commissions Management: Hotel Bookings Investment Banking - The Client On-Boarding Process - Financial

product setup ‘URLoans’ – Loans system Credit Agreement System

Module Assessments Assignment 1: Prototype (30%)

Group work: Critical evaluations and comparison of selected individual projects

Group presentation

Presentations run weeks 9, 10 and 11

Report Submission during week 12

Module Assessments Assignment 2: Group Critique Report (20%)

Module Assessments Exam

3 hour Examination 50%

Advanced HCI Module Aims

Study Significant Issues in HCI Theory, experimental psychology, reflection Cognitive theories / psychology Computing / interface design

Collate and analyse materials Reflect and develop critical analysis

Encourage professionalism: verbal / written Individual presentation skills Group work

Develop interactive system Based on practical / theoretical knowledge

Lecture 1 Overview

Human Capabilities; People; Technology

HCI and Cognitive Frameworks Perception Attention / Memory Knowledge / Conceptual Models Learning

‘Seeing isn’t done by the eyes but the brain’

Cognitive Computing and

its applications in HCI

Human Computer InteractionPreece, J. Addison & Wesley

What is HCI?

What is HCI?HUMAN MACHINE

GOALS

functions

actions

CONTEXT

Central Aim and Approach of HCI

Approach: User-Centred Users should not have to adapt to the interface Interface should be intuitive/natural to learn/use.

“Talking to users is not a luxury, it’s a necessity”

Aim: To optimise performance of human and computer together as a system

The Parts of User Interface Development

Development of the user interface

Development of the interaction component

COGNITION

Development of the interface software

e.g. PRINCIPLES

ConstructionalBehavioural

Human - Computer

Usability

• Learnability• Efficiency• Memorability• Errors• Satisfaction

Relationship between user’s goals - required actions –results must be meaningful, not arbitrary

Therefore VISIBLE or TRANSPARENT

Visibility (Norman, 1988, 1992)

UserGoals

UserActions

ResultsInterfaceComponents(controls)

FunctionsOf thesystem

Feedback

What does the system do, NOT how does it do it

Affordance (Norman, 1988, 1992)

Normans defines affordance as

“….. A technical term that refers to the properties of objects – what sorts of operations and manipulations can be done to a particular object”

Perceived affordance – what a person thinks an object can do

Door Piece of paper Water tap Button

Poor interface may cause:

Increased mistakes in data entry / system operation

Inaccessible functionality

User frustration: low productivity and/or under utilisation

System failure because of user rejectionNearly half of entire software development effort relates to the user interface.

(Myers and Rosson, 1992)

Cognitive scientists analyse “What the mind does…”

Investigate mechanisms/processes from which mental phenomena emerge and nature/structure of human knowledge

Mental phenomena such as…… Perceiving – learning – remembering Controlling actions Thinking - Emotion Planning – Imagining – Creating Understanding language and others Communicating with others Making decisions – Solving problems

Model Human Processor

Perception (Visual) Knowledge Attention Language etc

Model Human Processor

The Model Human Processor

Brain viewed as three interacting subsystems

(each with memory store and processor)

Perceptual system (storage of signals from senses e.g. images, sounds; brief time memory = sensory register)

Cognitive system (‘think’, analyse, recall from LTM; storage in STM/working memory)

Motor system (called on when physical response is required – transmit signals to muscles)

(Card, Moran and Newell, 1983)

Perception –

Why do things look the way they do?

Perception is

translation of physical world into pattern of neural activity that can be used by brain to guide behavior.

Perception –McGraw-Hill

‘Perceptual organisation’

Understanding of how we put together the basic features

(edges, colours, motion, depth etc.)

to see a coherent organised world of things/surfaces.

Perception in Computing

Fundamental for HCI Perceive info presented at/through interface Theories of perception influence interface

design Present info in easily

understandable/unambiguous manner i.e. user-centred

Active process

Why Study Perception ?

Practical reasons: Identify/correct environmental hazards; Design devices

to optimize perceptual performance; for people with perceptual impairments.

Perception and pleasure: Visual art/music/gourmet cuisine engage sensory

system in ways that we find pleasurable Understanding perception allows us to heighten

our sensory experiences.

Perception and intellectual curiosity

Visual Perception: Perceptual world can be different from real world!

Human system is capable of perceiving objects brightest of sunlight/darkest of night rapidly moving objects/rapidly decaying events

Human system cannot perceive Bullet being shot / Plant growing / Infrared light

So we can obtain info from displays • which vary in quality, size and other

characteristics• but not with uniform efficiency across

whole spectrum and all speeds

Properties of Vision – People draw distinctions between what is seen

“Object recognition” This could mean “is this a fish or a bicycle?” It could mean “is this George Washington?” It could mean “is this poisonous or not?” It could mean “is this slippery or not?” It could mean “will this support my weight?” Great mystery

How to build programs that draw useful distinctions based on image properties.

Vision application: Pedestrian finding

many pedestrians look like lollipops (hands at sides, torso wider than legs) most of the time

classify image regions, searching over scales

But what are features?

MAKE computer perceive pedestrians as lollipops?

Vision application: Pedestrian finding Result: Pedestrians found, search over scales

Figure from, “A general framework for object detection,” by C. Papageorgiou, M. Oren and T. Poggio, Proc. Int. Conf. Computer Vision, 1998, copyright 1998, IEEE

Figure from, “A general framework for object detection,” by C. Papageorgiou, M. Oren and T. Poggio, Proc. Int. Conf. Computer Vision, 1998, copyright 1998, IEEE

Perception = ‘The way we see’:Constructivist vs. Ecologist (1)

• Intervention of representations/memories• View world by constructions from info in

environment / previously stored knowledge• Don’t see replica/copy of world but model

created by transforming, enhancing, distorting and discarding info / retinal images.

• Effect = constant view of world• (Gregory, 1970; Marr, 1982; Easterby, 1970)

• Decompose or partition images into separate entities so readily recognizable

• Perception of objects• Our ability to interpret meaning of scene and

objects is based on innate laws of organisation

• (Gestalt: proximity, similarity, closure, continuation etc)

• Background-ground discrimination grouping can be seen in terms of allocating some elements

to a figure, some to ground (Muller-Lyer etc)

Constructivist – Context and Gestalt Psychology

Construction! What does this scene contain?

Extract shapes/objects (early); recognise (late)Process visually (early); recognise patterns (late)

Visual Perception – EARLY and LATE phasesPicture: http://whalen.psych.udel.edu/cognition/perception.html

Example of visual perception – ‘ink blobs’ or ‘dog in park’???

Picture: http://whalen.psych.udel.edu/cognition/perception.html

Did you ‘see’ Dalmatian dog? How long did it take you? Can you now see picture as anything else?

Interpretation due to prior knowledge of Dalmatian Without it we wouldn't be able to make sense

of picture

Active Construction of Image = process of making sense of black shapes

‘The way we see’:Constructivist vs. Ecologist (2)

• Process of picking up / exploring / detecting info from environment…..

• …not requiring processes of construction or elaboration (Gibson, 1979)

• Concerned with how we deal with continuous events over time; What we need to know to carry out our activities and how this might be known.

• Central concept = Affordance (Gibson, 1979; Gaver, 1991; Norman, 1988)

The Eye as an Optical Instrument

Aspects of Human Vision

a - Sharp vision

b - Unsharp vision

c - Only movement seen

Visual FieldEye contains different

kinds of receptors: Rods and Cones in Retina

Most of colour sensitive cones found in FOVEA CENTRALIS (2mm dia.)

FC in centre of retinaReceives light from objects Puts objects in focus

three layers: fibrous tunic,

vascular tunic, retina.

divided into 2 chambers

contains iris, pupil, and lens

muscles control motions.

Human Eye: Structure

Picture: http://guide.discoveronline.org/discover/guide01/1sec1a.htmPicture: http://www.medicinenet.com/black_eye/page4.htm

The structure of eyes is highly related to their function

Human Eye: Iris, Lens, Retina

Iris: regulates amount of light received by eye and depth of field

Lens: accommodates near and far vision

Retina : contains photoreceptors that convert light energy into neural signals

Human Eye: Macula (in Retina)

Vision is most acute at macula, located at center of retina

When we look directly at an object, its image is centered within macula

Human Eye: Muscle

Picture from Perception PPT by McGraw-Hill

Involuntary movements prevent our eyes from standing still

This is important for vision, because stable retinal images disappear Muscles in eye

Where should eyes be placed ? Frontal eyes: Depth perception. Lateral eyes: See more of world at

once

Picture from Perception PPT by McGraw-Hill

Sensitivity to Colour Combinations (3 Cones sensitivity to colour/light of equal intensity)

Violet Blue Green Yellow Orange Red

Sensitivity

Most sensitive

Least sensitive

Sensitive

Sensitivity

Human Perception of Colour

Eye most sensitive to green/yellow; least sensitive to blueComplex perceptual system

Cone response e.g.17:44:39 is blue, 61:39:0 is yellow, 50:45:5 is white

Colour impairment: defective colour vision 8% males, 0.4% females Red/green blindness is most common

HCI designer must take this into account Use other symbolic forms of coding - Shape, size, texture,

symbols E.g. Traffic lights use position/brightness as well as colour!

‘Chromatic Aberration’ Phenomenon

Different colours bent by different amounts when light passes through lens of eye Difficult for eye to focus on all colours in object at

same time Can lead to eye strain

Avoid extreme colour pairs red (most bent) and blue (least bent) green and magenta

Desaturate (i.e. add white) to reduce effect Use pastel or darker shades for large areas

RGB

Colour Pairs – avoid extremes

Extreme Colour Pairs

Saturated Colour Pairs

ADD WHITE

Causes of colour and their appearance

Sensation of color is caused by brain. Some ways to get this sensation include pressure on eyelids,

dreaming, hallucinations, etc. Main way to get it is response of visual system to presence/absence of light at various wavelengths.

Color appearance is strongly affected by (at least): other nearby colors, adaptation to previous views “state of mind”

Several demonstrations follow.

Background Colour and Apparent Brightness and Size

Colour selection can effect presentation of info Colours look brighter and larger against black Colours look darker and smaller against white

STROOP EFFECT(Colour naming is affected by more than just physics)

Read ALOUD and FAST1) colour names of XXXs2) colour names of RHS (word/colour same)3) colours of RHS4) words of centre (word/colour differ)5) colours of centre

Colour Appearance and Surrounding Colours (1)

Which is most appealing?

Why?

Are grey figures same colour?

Colour Appearance and Surrounding Colours (2)

Grey figures ARE same

Contrasting backgrounds affect perception of colours

Need more than pure physics

Adaptation phenomena:Changing operating point of receptors

Response of color system depends on spatial contrast what it has seen before i.e. adaptation

Common example: walk inside from a bright day everything looks dark for a bit then takes its conventional brightness.

Guidelines for Use of Colour in Applications:Visual Density & Balance

How `tightly packed’ text is on screen Measure of amount of ‘White Space’ in

relation to amount of text High density = lots of text and little

white space More difficult to read

People perceive patterns/structures in displaysSO present info accordingly to increase readability/ease of search

Text Density

On paper, density of 70-80% common 70% of page covered in text Readable but on computer screen would be difficult

On screen, 15-20% recommended

On its own not sufficient measure of readability Way text is structured also very important (e.g.

search info) E.g. phone directory in tabular form to make easier

to search

Text Legibility (1)

Use standard rules for UPPER and/or lower case Longer text passages are easier read using standard

capitalization rules rather than using all capital letters

READING ALL CAPITAL LETTERS CAN TAKE LONGER AS WORDS LOSE CHARACTERISTIC SHAPE - WORDS BECOME RECTANGULAR

Improper grammar/punctuation slows reading speed

Text Legibility (2)

Some font styles might look decorative but they are not necessarily legible Stick to common fonts Rule: don’t use >3 fonts and font sizes per document

If yu mst use abbrev mke sre th usrs r famlr wth thm

Keep text lines short Long line lengths can be difficult to read

Visual and Other Coding

Differing Intensity (brightness, lightness) Shape e.g. box frame (recognise/pickout) Colour and/or shadingshading Underlining (not recommended in printed text –

can slow reading) Character size and font Movement e.g. ‘micons’ Sound and/or synthesized speech

Example of Screen Design – Visualisation / Aid to Accessibility

(Source: http://www.cs.umd.edu/projects/hcil/chi96/paper/cps1txt.htm)

(Source: http://www.cs.umd.edu/projects/)

Improved Design – What features aid readability /

understanding?

Causes of color and their appearance

The sensation of color is caused by the brain.

Color appearance is strongly affected by (at least): other nearby colors, adaptation to previous views “state of mind”

Colour Attracts Eye: Old PCs didn’t have power/RAM to use colour; Falling costs

Colour on computer screens? Aesthetically pleasing (prefer to Black/White) Improve human efficiency by adding extra coding dimension Easier to find / distinguish e.g. search for ‘targets’ - characters,

words etc Memory better Provides (useful) redundant coding e.g. Standard background

colour for screens

Limited advantage over monochrome display – only add colour to enhance interface – should function without it

Can be misused e.g. Inappropriate colours/’colour pollution’

Value of Colour in (Mainly) Text Screens

Help user search/distinguish between items Used to segment screen into related info Relates separate fields of info

i.e. same type = same colour

Used to Categorise e.g. Actual and projected figures More or less recent data Caption or data field User or computer provided data Status - correct or error, normal or urgent

Guidelines for Use of Colour - Screens Containing Mainly Text (1)

Don’t use too many colours (<4/5)

Use colour coding to support user’s task (not hinder) Identify similar instances Exceptional instances e.g. warnings (bright to

emphasise) Common coding scheme

Green - normal or OK Orange - caution Red – problem/danger/hot

No universal/multicultural interpretations of colour in chemical plant, red colour might just mean hot in West black = mourning, in China white = mourning

Emphasise Bright colours emphasise, Less bright de-emphasise

Relevance and ‘Known’ – Give user control with meaning

Colour coding scheme must be relevant and known to user Enables selective attention

Differences and similarities – Be consistent User will notice these regardless of whether they have task-related

meaning. If no meaning then user wastes time trying to work out meaning

Colour refuses to be irrelevant – Choose carefully Irrelevant colour increases search time red on blue vibrate yellow on blue pale at edges red on green or yellow on blue shadows

Red

YellowRed

Guidelines for Use of Colour - Screens Containing Mainly Text (2)

‘Optimal Colour Combinations’ chart Which colours can be used effectively against which background colours

Lecture 1: Review

HCI and Cognitive Frameworks Perception Attention / Memory Knowledge / Conceptual Models Learning

NOW: Lab for practical work