DESIGNING MULTIPLE UAV DISPLAYSBASED ON THE HUMAN CAPACITY TO

PROCESS INFORMATION

PRESENTED BY:

Dr. William MarshakJeff S. Collier

SYTRONICS, IncorporatedDayton OH

PRESENTED FOR:

Cognitive Engineering Research Institute’sFourth Annual Human Factors of UAVs Workshop

“Manning the Unmanned”21-24 May 2007

SYTRONICS, Incorporated 2

Contents

• Project Background• Human Cognitive Operating Characteristic• Multiple UCAV/UAV Cognitive Task Demands• Design Strategies

– Voice Input/Output Using Context– Agent Mediated “Shallow” Information Access– Temporal Mission Event Management– Distinguishing Plan from Hypothetical Replanning– Recognition Primed Decision-making (RPD) Display– Visual Thinking / External Cognition (VT/EC)

• Summary

SYTRONICS, Incorporated 3

Multiple UAV AGEncy (MAGE)

Tactical SituationSystem Display

Decision SupportSystem Display

Sandbox SystemDisplay

Purpose: Develop means for integrating net-centric information to facilitate multiple UCAV/UAV control

Employs intelligent agent software and theory-based design strategy

Transition to current systems and systems in development

Sponsor: Air Force Research Lab / Human Effectiveness Directorate (Dr. Mark Draper & Ms. Gloria Calhoun)

SYTRONICS, Incorporated 4

Human Cognitive Operating Characteristic

• Perception bandwidth– Vision-high, Audition-medium, Touch/Taste/Position-low

• Attention driven sampling from sensory stream• Limited Short-Term Memory (STM) Capacity

(Miller's 7±2)• Large Long-Term Memory (LTM) Capacity (if

consolidated)• Communication occurs within common cognitive

framework• Impedance matching between user and interface

facilitates information flow– Matching includes perception and cognitive (way we

think)

SYTRONICS, Incorporated 5

Multiple UCAV/UAV Cognitive Task Demands

• Vehicle states– Internal status of propulsion,

sensors, communications, etc.

– Consumables status: fuel, power

• Vehicle environment– Flight characteristics, traffic

deconfliction– Weather– Threats

• Mission demands– Ground engagement

(surveillance, search, weapons strike)

– Coordination (C3, Ground/JTAC, Intel, Air, ATC)

– Lost Link

SYTRONICS, Incorporated 6

Voice Input/Output with Context

• Voice input using Nuance®; speaker independent, vocabulary limited system– Switch actuated recognition made at any time

• Voice output using Microsoft Speech SDK– Text-To-Speech (TTS) speaks answers to queries

• Context manager maintains elements of previous conversation, filling in omissions in subsequent queries– Speech suitable responses delivered by TTS– Non-verbal responses available in Decision Support

System• Context correspondence maintained by selective

echoing– Ubiquitous way people maintain context

SYTRONICS, Incorporated 7

Agent Mediated Shallow Info Access

• Information requirements driven by both mission event and operator

• MAGE queries internal and net-centric sources to gather needed information

• Most important information loaded into workspace in expandable window

• Other information organized under tasks and accessible with one click

• Avoids multi-layered menus• Reminds operator about

relevance• Operator queries supplement

agent generated information

SYTRONICS, Incorporated 8

Temporal Mission Event Management

• One strategy for workload management is avoiding events from happening at the same time

• Identifying time conflict among mission events requires a temporal display

• Display includes time line for each vehicle, “now” indicator, typed event strategy

• Automatic or manual “scrolling” through mission time

SYTRONICS, Incorporated 9

Distinguishing Mission from Hypothetical Replanning

• How to replan without losing awareness of the current executing plan

• Separate “Tactical Situation” and hypothetical “Sandbox” displays

• Separates reality from hypothetical, but allows side-by-side comparison

• Includes spatial and time line renditions of the mission plans

Tactical Display

Sandbox Display

SYTRONICS, Incorporated 10

Recognition Primed Decision-Making Display

• Based on Klein (1989) observation experts recognize what to do

• MAGE agency detects need for intervention • Assembles a Decision Support Template to aid

and accelerate decision process• Instigation and decision alternatives presented • Workspace allows “recognition” of the solution• Shallow menu and voice query system make

more information available• MAGE operator “picks” the most desirable

solution• Enables better and faster decisions in high

workload environment

SYTRONICS, Incorporated 11

Decision Support Template

SYTRONICS, Incorporated 12

Visual Thinking / External Cognition (VT/EC)

• Arnheim (1969) studies artists’ creative thinking and postulates higher cognitive function evolved from vision– Visual representations consistent with visual processing

are easier to understand, mimic higher order reasoning– “Blink” and RPD may be manifestations of Visual

Thinking• Examples include nomographs and DSTs

– Not only provide solutions, but reveal process of solution• McKim (1972) proposes concrete principals governing

VT• McEachren (1995) applies concept to maps• Card, S. K., Mackinlay, J. D., and Shneiderman, B.

(1999) “rediscover” VT/EC• Expand visual thinking to other senses broader term of

External Cognition

SYTRONICS, Incorporated 13

Complex Judgements by Visual Thinking

SYTRONICS, Incorporated 14

Evaluation of the VT Display

• Brief study comparing bar graph representation of alternative mission metrics with VT Widget (VTW)

• Observers taught criterion for judging 3 alternatives on 7 dimensions, including minimum acceptability

• Then judged sets of 12 trios of mission alternatives either in a block of tables, bar graphs or in a block single VTWs

• Choices were to rank three alternatives, detect if all three violate required minimums, and decision time recorded

• Deviations from mathematical ideals, correct rejections of minimum violations, and decision times were compared between blocks in a dependent t-test

SYTRONICS, Incorporated 15

Summary

• Cognitive Systems Engineering of human cognitive operating characteristic and system displays reduces impedance and increases bandwidth

• This is accomplished by– Voice input/output with Context– Agent facilitated shallow information access– Temporal mission event management– Recognition primed decision-making aiding

display– Visual Thinking / External Cognition displays

• MAGE integrates net-centric feeds and multiple vehicle control, overcoming much of the workload overload