Visualisation issues in the context of Information Fusion

Jean-Rémi DuquetLockheed Martin Canada

Jean-Yves FisetSystèmes Humains-Machines Inc

Hélène L’Heureux

École Polytechnique de Montréal

Plan

Data Fusion - overview

Data Fusion – human factors issues

Tagci – an HMI Design method

Research – adapting Tagci for data fusion applications

Data Fusion - Overview

Data / Information Fusion can be seen as:1. Knowledge Composition (e.g. Feature

Extraction)

2. Evidence Aggregation

3. Decision (e.g. plot symbol as AIR HOSTILE, put on threat list)

Data Fusion OverviewFLIR Shape-Matching Classifier SAR Hierarchical Classifier

80 % Merchant20 % Unknown

70 % Destroyer20 % Frigate10 % Unknown

Intelligence reports(Location, ID, Emitters, Freq)

Surface Radar (Location)

Human Analysis

Data Fusion – Human Factors Issues

Use & display of fuzzified / uncertain identity statements Large tree of “Identity Propositions” created by aggregation processConflicts management, recovery from errors and deceptionOutput stability (e.g. friend /foe oscillations, multi-scan tracking)Algorithmic issues related to HUMINT, human analysis, self-intoxication Human inputs as part of fusion process (e.g. correlation,

aggregation) Fusion process (self- ?) monitoring

Usability / function allocation issues (SA display, cues and alerts )

Data Fusion – Human Factors Issues

Test Bed Human-Machine Interface

Data Fusion – Human Factors Issues

Test Bed Human-Machine Interface

HMI Methodology Requirements for DF

Simple to learn and apply

Smooth interface with SW engineering

Allows technological uncertainty

Information Requirements: obvious “common ground” between DF development and CE methodologies

HMI

G

g1 g2

g3

Tagci – An HMI Design Method

Generic model of the operator’s

task

Matchingprocessmodels

Operator’sbehaviour

HMI design principles

HMI Content and Organization

Information architecture

Tagci

Getting on with the job

Tagci

D

d1 d2 d3

d12d11 d33d32d31

G

g1 g2 g3

g11 g12 g31 g32 g33

p1 p2 p3

D

d1 d2 d3

d12d11 d33d32d31

D

d1 d2 d3

d12d11 d33d32d31

G

g1 g2 g3

g11 g12 g31 g32 g33

p1 p2 p3

Goal Modeling, Detection Task and DF

Data Fusion

An example Tagci-based display sketch

Hierarchy of Military Goal(s)

Area, personnel and

equipment

Info.Srce

Info.Srce

Info.Srce

Info.Srce

G

g1 g2 g3

g4 g5 g6

Tagci-based display• Goal monitoring• Support for compensation

Research Agenda

Fine tuning of generic tasks

Built-in operator strategies

Enhanced visualization primitives

ConclusionData fusion is a complex process in itselfNumerous human factors complicate HMI designHMI design must consider domain, tasks, operator, HMI design rulesTagci integrates several sources of knowledge to design HMIs for monitoring and controlling complex systems Potential for monitoring and controlling DF

processs

Optimizations to Tagci for DF applications are being examined