Human-Machine Interfaces for Piloted

SystemsENAE 380 Flight Software systems

Content

• Human-Machine Interface Definition

• Human Capacities and Limitations

• Guidelines and Theories

• Interface Types

• Application in Piloted Systems

• Evaluating Human-Machine Interfaces

Human-Machine Interface (HMI)

• Where people and technology meet...

• Multi-disciplinary

• Psychology

• Ergonomics

• Electrical/Computer Engineering

• Human Computer/Robot Interaction

• Standards for HMI:

• ISO 9241 - The ergonomics of human system interaction

• ISO 13407 - Human-centered design process

• Akin’s Law nu. 12: “There is never a single right solution. There are always multiple wrong ones, though.”

Humans Machines

Remember principles and strategies

Count and measure

Sense low-level stimuliSense stimuli outside human’s

range

Respond to emergenciesPerform repetitive actions

reliably

Reason inductively Reason deductively

Adapted from [1]

Human Short Term Memory Capacity

• Humans have limited short term memory

• Miller (1952):

• 7 +- 2 “chunks” of information

• 15-30 seconds

• Size of “chunks” depends on familiarity with info.

• Read one set of binary digits, then close your eyes and count to 20:

• 7 digits: 1011010

• 14 digits: 00110101110110

Humans and Colors• Use of color can speed or slow task

• Use colors consistently and conservatively

• Use color only to add meaning

• Use colors to signify changes and draw attention to warnings and errors

• ~ 8% of male population red-green color blind

red

blue

Human Movement -Fitts’ Law

• Based on Shannon’s theorem 17 [2]

• Part of Information Theory

• Information capacity of a communications channel is limited by the effect of noise

• Fitts (1954) extended to information capacity of human motor system, claimed [3]:

• Distance moved (A) equivalent to signal power

• Tolerance/Width constraint on motion (W) equivalent to noise power

MT = a + (b x ID),

ID = log2(A / W + 1)

MT - Mean Time to complete taskID - Index of Difficultya, b - Experimentally determined constants

This is an improved version of the original Fitt’s law, and is known as the Shannon formulation.

Interpreting Fitts’ Law

• Can estimate time taken for fast, targeted movements

• Trade-off between distance and accuracy of movement

• Humans have a constant maximum information processing rate (Index of Performance)

From [3]

IP = ID / MT (bits / sec)

IP - Index of Performance

Amplitude, A

Width, W

Using Fitts’ Law Each point represents average

of all subjects for that task

Task:

One standard deviation for task

Linear fit (correlation coefficient r2) with MT-axis intercept a and slope b

. . .

Application of Fitts’ Law• Menu and screen layout

• Frequent buttons/actions should be large and close to cursor’s (or hand’s) average position

• Edges of a screen are “infinitely targetable” (cursor typically stops at edge)

• Run Fitt’s law experiment to compare input devices

• I.P. is a measure of effectiveness of device in general

• Can generalize about comparison, as based on human performance model

• Limitations

• Only for fast, targeted movements (discrete or reciprocal)

• Does not take into account system response time and mental preparation made

8 Golden Rules of Interface Design

• Consistency

• .. of sequences of actions, terminology and colors

• Aim for Universal usability

• Identify and accommodate all potential users

• Give Informative feedback

• .. for every user action

• Yield closure

• Defined beginning and end of tasks

Adapted from [1]

8 Golden Rules of Interface Design

• Prevent user errors

• E.g. grey out options not appropriate

• Easy reversal of actions

• If possible. Reduces anxiety and promotes exploration

• Support internal locus of control

• Operator is the initiator of actions and feels in control

• Reduce short-term memory load

• Consolidate displays and minimize window motion

Adapted from [1]

Command Language Interfaces

• Good for fast access, if you know commands and syntax

• e.g. “http://www.aero.umd.edu”

• 3 basic structures

• Guidelines for use:

• Use meaningful and distinctive names

• Aim to make it hierarchical

• Limit commands and avoid duplicates

1) Simple Command Set Example

Task Key Combination

Save ctrl-x-s

Quit ctrl-x-c

Search ctrl-s

2) Command + Argument ExampleCOPY FILEA, FILEBDELETE FILEAPRINT FILEB -OPTIONC

3) Hierarchical Command Structure Ex.

Action Object Destination

COPY File Screen

DISPLAY Process Printer

MOVE Directory File

One rule of formation

Menu Interfaces• Access to large amount of items without

need for memorizing syntax and commands

• Categorical organization recommended

• Can halve nu. of errors compared to unstructured menus [1]

• Faster response time than alphabetical

• Tree-structured menus

• Response time and errors increase as depth increase (same nu. items)

• ~ 3-4 levels max (preferred)

• Grouping suggestions: Logically similar, non-overlapping and using familiar terminology

Linear Sequence (e.g. online checkout proc. and wizards)

Tree Structure

(most common)

Network (e.g. WWW)

Cyclic

Acyclic

Depth = 3 (number of levels)

Breadth = 2 (items at this level)

Speech Interfaces

Pro Con

Hands-free Recognition errors

“Natural”Not useful in noisy

environments

Eyes-offLow information

content

Good in cramped conditions

More demanding on working memory

Direct Control Interfaces

Isometric Isotonic “Elastic”

Measured Force Position Deflection

Physical Motion Very little No limit Bounded

Transfer function Typically non-lin. Linear Typically linear

Main Strength “Static” Fast and intuitive Accurate

Main Weakness Inaccurate Fatiguing “Slow”

Typically: - rate control: Isometric and Elastic- position control: Isotonic

2D vs 3D Displays• 3D pros

• “Natural” - humans used to 3D representations

• Good for 3D tasks - e.g. CAD

• 3D cons

• Occlusions - e.g. 3D barcharts

• Inefficient use of screen space

• Can we go beyond 3D represenations?

• “X-Ray” vision

• “Augumented” reality

Microsoft Task Gallery

Example of Augmented Reality in Teleoperation

Side View

Orbital Replacement Unit (ORU)

End Effector View

From [5]

Piloted Systems Requirements

• What makes piloted HMI different?

• High density of controls

• Requires very high situational awareness

• Operator performs extensive multi-tasking

• Successful design a balance:

• Keep pilot/operator workload low

• But, pilot /operator must remain effective

HMI in Piloted Systems- Eurofighter

• Modern fighter avionics suites produce very large amounts of data

• Fusion of data from sensors needed to reduce operator information load

• “Autonomy” can be included for housekeeping tasks

• Want pilot to focus on environment

• Helmet mounted displays can be used to increase situational awareness

• Voice feedback and input (selecting targets, route manipulation etc)

Hostile aircraft example:may be simultaneously detected on:- radar- infrared tracker- defensive aids system- data link from other aircraft and sensors

Autonomy example:- know fuel, fuel flow, performance capabilities of aircraft- can easily tell pilot when its time to turn around..

what is a machine good at again?

Speech input:- traditionally: HOTAS (Hands On Throttle And Stick)- now: VTAS (Voice, Throttle And Stick) for time-critical but infrequent controls

hmm, seems to be a good application of speech interface, but may be limited by recognition errors and mental workload related to generating correct speech command

- Also included in F35 Lightning II, but not on F-22 yet.

From [6]

HMI in Piloted Systems - Space Shuttle

• “Glass cockpit” (MEDS upgrade ~2000)

• “Painted” instruments on screens

• Pilots still use multiple screens to assimilate information

• Compare numbers with printed que cards

• Cockpit Avionics Upgrade (CAU)

• Improve human-machine interface

From [4]

Reaction Control System Display

From [4]

Original CAU

Horizontal Situation Display

From [4]

Original CAU

- Top-down view of shuttle relative to airfield

Old: - No info on whetehr shuttle has enough speed and altitiude to make it to a given landing site

New: - “Energy footprint” - Inside “homeplate”: achievable with nominal flying procedures- Yellow and outside: achievable with “risky” flying procedures (minimizing entry losses) - Situational awareness - Reduced workload - Used 10 colors, 5-7 better

CAU Lessons

• Organize data to ease information assimilation

• Use colors mainly to get users attention and increase information density

• In high-risk systems, situation awareness is a safety issue - HMI design matters..

• CAU will unfortunately not be implemented, as Shuttles are to be retired early

Evaluating HMI Designs• Expert Reviews

• Let peers or customers try interface to pinpoint problem areas

• Not good at simulating first-time users

• Usability Study

• More extensive well-documented study involving real users performing tasks based on realistic scenarios

• Known to speed up and reduce costs of projects

• Controlled Experiment

• Yields very narrow results, but reliable

• E.g. Fitts’ law test comparing two hand-controllers

References

• [1] Shneiderman, B. and Plaisant, C. , ”Designing The user Interface”, Fourth Edition, Addison Wesley, 2004.

• [2] Shannon, C. and Weaver, W., “The mathematical theory of communication”, Urbana: Univ. of Illinois Press, 1949.

• [3] Fitts, P.M., “The Information Capacity of the human Motor System in Controlling the Amplitude of Movement” Journal of Exp. Psych., Vol. 47, No. 6, p. 381-391, 1954.

• [4] McCandless, J.W., et al., “New Displays for the Space Shuttle Cockpit”. Ergonomics in Design, Vol. 13, Nu. 4, p. 15-20, 2005.

• [5] Maida, J.C. et al., “Improving Robotic Operator Performance Using Augmented Reality”, Proceed. of Human Factors and Erg. Soc. 51st Annual Meeting, Baltimore, 2007.

• [6] BAE Systems [internet], “The Eurofighter Typhoon”, http://www.eurofighter-typhoon.co.uk/, accessed 11/30/07.