Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE...

52
Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004 (Version 2004.02)

Transcript of Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE...

Page 1: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion for the Army Knowledge Fusion Research Workshop

Dale A. WalshPrincipal Engineer

The MITRE CorporationFusion SME for Army DCS G-2

October 2004 (Version 2004.02)

Page 2: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Definition(s) of Fusion• JCS Pub 1-02

The process of examining all sources of intelligence and information to derive a complete assessment of an activity.

• TextbookA process dealing with the association, correlation, and combination of data and information from single and multiple sources to achieve refined position and identity estimates, and complete and timely assessments of situations and threats, and their significance. The process is characterized by continuous refinements of its estimates and assessments, and the evaluation of the need for additional sources, or modification of the process itself, to achieve improved results. (Llinas/Hall)

• Textbook, SimplifiedA process of combining data or information to estimate or predict entity states. (Steinberg/White)

• Army’s “Usable” A series of processes performed to transform observational

data into more detailed and refined information, knowledge, and understanding. (Walsh/Jones)

These processes, by their very nature, will involve both automated processes and human cognition. Also included as part of fusion are the databases, human interfaces and information portrayal, and the control and feedback of the fusion process.

Page 3: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion “Truisms”

• Fusion is not new; automating some of it is

• Fusion is a “critical enabler”

• Fusion is driven by the Commander’s Priority Intelligence Requirements (PIRs)

• Fusion takes place simultaneously at all echelons

• Fusion facilitates Actionable Intelligence

Page 4: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Definitions• The JDL Fusion Model – A taxonomy and concept initially

developed in the 1980’s by the then-Joint Directors of (DoD) Laboratories – Designed to allow the diverse fusion efforts to have a common

language and framework – has worked very well to that purpose

– Is NOT a system architecture, notional or otherwise– Is NOT intended to define problem sets that clearly vary from

armed service to armed service– The “boundaries” between fusion “levels” are not intended to

be completely distinct (so some processes may touch adjoining levels)

– Is now shepherded by a panel of “graybeards” who still debate and modify the model on occasion

Page 5: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

The Data Fusion Model(from Joint Directors of Laboratories, as modified)

Level1

Fusion

Level3

Fusion

Level2

FusionLevel0

Fusion

Databases(Fusion & Support)

Sources&

Sensors

Level4

Fusion

HumanComputerInteraction

Level5

Fusion

Page 6: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

What do these levels do for me?• Level 1 fusion tells you what is physically out there

• Level 2 fusion tells you how they’re working together and what they’re doing now

• Level 3 fusion tells you what it all means, what will happen next, and how it affects your plans

• Level 4 fusion tells you what you need to do to improve the information from levels 1-3

• Level 5 fusion allows you to look at and interact with the information from levels 1-4

Page 7: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Entities vs. Fusion Levels 1-3

Tier 1 - Direct Observables(equipment, facilities, physical infrastructure)

Tier 2 - Reportable Items(lower echelon units, individuals, specific events)

Tier 3 - Conclude Existence(upper echelon units, organizations, comms nets,

interrelationships between other entities)

Tier 4 - Post-analysis(plans/courses of action, intentions, behavior at both the enterprise and sub-enterprise level)

Entity Tiers We have taken the types or classes of entities and other knowledge goals and categorized them into

“tiers”. On one end, Tier 1 entities are ones that can (in general) be

observed directly; conversely, Tier 4 entities (really knowledge goals

more than entities) are items of knowledge that can never be

directly observed but can only be developed by cognitive analysis.

Page 8: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Entities vs. Fusion Levels 1-3

Tier 1 - Direct Observables(equipment, facilities, physical infrastructure)

Tier 2 - Reportable Items(lower echelon units, individuals, specific events)

Tier 3 - Conclude Existence(upper echelon units, organizations, comms nets,

interrelationships between other entities)

Tier 4 - Post-analysis(plans/courses of action, intentions, behavior at both the enterprise and sub-enterprise level)

Entity TiersBattlefield entities, and further

knowledge and understanding about them and their actions/intents, are developed in Fusion Levels 1, 2,

and 3 only.

Levels 0 and 5 are simply pre- or post-cursors to that goal, not

“lower” or “higher” levels of fusion.

Level 4 is the feedback loop that uses what is learned in Levels 1-3 to drive improvements in processing

(or collection tasking)

Page 9: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Entities vs. Fusion Levels 1-3

Tier 1 - Direct Observables(equipment, facilities, physical infrastructure)

Tier 2 - Reportable Items(lower echelon units, individuals, specific events)

Tier 3 - Conclude Existence(upper echelon units, organizations, comms nets,

interrelationships between other entities)

Tier 4 - Post-analysis(plans/courses of action, intentions, behavior at both the enterprise and sub-enterprise level)

Entity Tiers

Level1

Fusion

Level2

FusionLevel

3Fusion

Nouns --> Verbs --> Sentences --> Paragraphs

Knowledge is developed by continuous cycling of current knowledge and incoming data through Levels 1, 2, and 3 - and this cycle can occur at each echelon.

Page 10: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Current Fusion “State of the Art”• Level 0 fusion handled by sensor communities

• Generating objects is not automated for all domains

• Level 1 fusion largely conquered - automated correlation and update techniques in use for 10+ years

• Level 2 fusion is largely unautomated, yet current and future quantities of observations call for some levels of machine assistance

• Automated processes need “templates” for aggregation rules

• Level 3 fusion will remain the G2/G3 analysts’ domain for the near future

• Level 4 fusion is handled by the collection management arena

• Level 5 fusion is a “visualization thing”; user interaction for feedback/control is at various depths

Page 11: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

How much Automation?How much Cognition?

• The desired or achievable balance between machine and Soldier has led to much debate

• An Army Working Group will be stood up to try and address how much automation is “enough”

• Four Modes of Operation have been defined:1. Manual: No applications software to help

2. Computer Assisted: Tool Box of applications

3. Semi-Computer Controlled: Manned Assembly Line (series of applications run under user supervision)

4. Total Computer Controlled: Automatic Assembly Line

Page 12: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Notional Fusion Informational Flow

Sensing/Reporting Domain-specific

Correlation (Level 1)

GenericCorrelation (Level 1)

EntitiesEqmt/Unit/Facility/Organization/Individual/Event

Entities w/tech data

Pixels, waveforms, grunts/squeals, etc.

Object Aggregation/Analysis (Level 2A)

Correlated Objects

Event/ActivityAggregation/

Analysis(Level 2B)

CorrelatedEvents

COP

HypothesizedCurrent Enemy COA(s) Tgt

SA

AggregatedobjectsSU

RE

Entity ExtractionEntities w/tech data

Entities without

Entities Entitiesw/o tech data

CurrentBehavior

Assessment(Level 2C)

Note: Decomposing Level 2 into 2A/B/C is an Army-uniqueconcept to permit more detailed analysis of the classic L2 notions

Page 13: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Intelligence Domains

All Source (Integrating) Domain

Multi-INT (Intra-domain) Exchanges

HumanHumanDomainDomain

SignaturesSignaturesDomainDomain

SignalsSignalsDomainDomain

ImageryImageryDomainDomain

Soldiers asSensors

CI/HUMINT

MASINT(exc. Imagery)

COMINTExternals

COMINTInternals

ELINT

Visible

Infrared

Radar(incl. SAR, MTI)

Multi/HyperSpectralOpen Source

Page 14: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

An Analogy(Thanks to Rex Williams, USAIC&FH DCD)

• The COP is like taking a snapshot of a soccer game: it will show you where the players and the ball are at that point in time, but it doesn’t show the flow of the game

• The Intel Running Estimate (RE) is like a video of a soccer game: it shows the flow of the game and allows the viewer to project how they think the game is going and will go

Page 15: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

030440Z - Seismic detection

030440Z - Acoustic detection

030441Z - Tracked vehicle on move

030443Z - Acoustic detection

030444Z - Tracked vehicle westbound

030445Z - MTI reports 8 vehicles

030447Z - UAV Video shows tanks on road

030448Z - Radio intercept of 3rd Tank BN

There’s something out there...There’s some vehicles out there...There’s some vehicles moving west...There’s 8 tanks moving west... 3rd Tank BN is moving west Elements of the 42nd Tank BDE are moving west The 42nd Tank BDE is moving towards our positionLooks like the 42nd Tank BDE is going to attack our communications facility at Bradlehofer

03 Sep 0440Z03 Sep 0441Z03 Sep 0444Z03 Sep 0445Z03 Sep 0448Z03 Sep 0450Z

Page 16: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Now, A Word from Our Sponsor

While the last slide showed a very realistic operational example, fusion in today’s Army cannot be constrained to the simple problems of observing, analyzing, and predicting movement of conventional units. The tasks of fusing information regarding non-linear battlespaces, terrorism cells, and other 21st century matters must also be added to fusion’s “plate”.

Page 17: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

An Army Operational Extension to the JDL Data Fusion Model

Solid lines – primary data flow

Dashed lines – alternate data flow

– human interaction

– displays, reports, etc.

Level3

Level2Level

1

Level4

Level5

Sensors

Level0

DataRepository

Analysts

Warfighter

COP/RE,BCS

Page 18: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion from “Space to Mud”

EACCORPSDIVBDEBNCOPLTSQD OFW

FCSwith

DCGS-Aembeddedsoftware

DCGS-A

Home Station(HSOCs, IDC)

OrganicSensors

OrganicSensors

OrganicSensors

Organic/Nat’l

Sensing Σ

Σ

Σ

Σ

UA

UE

Tactic

al O

verw

atch

Tactic

al O

verw

atch

Page 19: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Tactical Overwatch Concept• Overwatch is a new, advanced concept for providing

dedicated, focused intel to operationally engaged UA units (BDE/BN)

• Overwatch “spotlights tactical forces with the full power of the theater, joint, and national set of intel capabilities”

• Overwatch provides the UA high-level fusion along with collection management and targeting support

• Overwatch provides sustained culturally-aligned global awareness conducted daily (and operates in peacetime and during war)

• Overwatch provides support to engaged UAs while anticipating transitions and future operations

• MI Brigades (as UEy) are the platforms from which Overwatch is executed, with INSCOM’s IDC acting as a hub for all Overwatch activities

Page 20: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Part Two: Fusion Issues

Page 21: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion for Dismounted Squads

• Fusion Goals– Locally-focused Level 1 fusion on organic sensing

to produce the local COP/SA– Integration (and resolution) of the SA/SU products

from FCS/UA above– Semi-autonomous Level 4 asset management– Soldier-supportive visualization/reporting

• Organic Sensing– EO/IR, acoustic/magnetic/seismic, CBRN, soldiers

• Efforts– Objective Force Warrior, Warriors’ Edge (ARL)

Page 22: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion for Unit of Action• Fusion Goals

– Regionally-focused Level 1 fusion on a huge multi-INT organic sensing flow, leading to generation of COP/SA

– Integration and deconfliction of the SA from squads below and the SA/SU of tactical overwatch support from above

– Effective cross-cuing to maximize utility of organic sensors– Level 2/3 fusion to see the aggregated red and blue picture and

then develop predictions on how/where the focused conflict might develop (ie, SU)

– Intelligent management of “steerable” collection assets– Seamless use of fusion products between ISR and C2

• Organic Sensing– EO/IR/SAR, video, LADAR, acoustic, seismic, magnetic, radars,

GMTI, CBRN, Met, EMTI• Efforts

– FCS with DCGS-A embedded software, FBKFF STO (ARL/I2WD, Level 2/3)

Page 23: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion for Unit of Employment• Fusion Goals

– Globally-focused Level 1 fusion within each discipline feeding an all-source fusion process which also leverages COP/SA/RE/SUs from below/above/joint

– Level 2/3 fusion to aggregate forces then predict future developments, put them in context with terrain/weather/plans, and develop knowledge about intended behavior/driving forces and their impact

– Joint operations (true at all echelons, but critical here)

• Organic Sensing– COMINT, ELINT, IMINT, MASINT, HUMINT, OSINT,

Soldiers

• Efforts– DCGS-A, FBKFF STO, DARPA RAID (Level 3)

Page 24: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion for Overwatch• Fusion Goals

– Extract relevant information from the large “take” available at the overwatch level

– Make that information usable and actionable • Provide strategic-level SA/SU

• Provide tactical overwatch support (to echelons as low as UA) in a timely, adjudicatable form

– Provide models, guidance, context information, etc. to support fusion processing at all echelons

• Sensing– Vast amounts of strategic/national intel

– Some organic sources

• Efforts– INSCOM’s IDC work and research, NGIC

Page 25: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion “Longpoles” (Page 1)• ATR/AiTR - The quantity of imagery and video is exploding,

yet the ability to extract objects and information from those feeds in an automated way trails badly. Techniques like change detection are available but not implemented on a large scale.

• Cross Domain - Intelligence domains breed classification domains, which restrict the timely flow of critical intelligence to those who need it. The resolution of this issue is as much political/procedural as it is technical. Intelligent agents can work to provide critical information while still protecting sources and methods - if their implementations are approved for use.

• Sensor Webs - A simple web of 20 deployable MASINT sensors can produce an overwhelming amount of raw data - so much that it becomes critical for Level 1 fusion first be accomplished within the web. This fusion must still produce a product that can be efficiently fused with other info/product held upstream.

Page 26: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion “Longpoles” (Page 2)• Text/Speech Translation - Translation of printed, handwritten,

and spoken languages is needed to begin to exploit the vast amounts of available non-formatted information sources. Translation support must be able to have a small footprint at SQD (for limited capabilities) and support high volumes for overwatch.

• Text/Speech Exploitation - More robust capabilities are needed to first parse (given post-translation grammatical constructs of each language) and then to extract/make sense of “free text” data. Eventually the ability to infer meaning into jargon, etc. needs to be explored.

• Scalability of Level 1 - While ASAS and TES provide very viable (and yet different) Level 1 correlation capabilities today, the sensing capabilities of an FCS UA will increase the input stream 100-fold. The scalability of current approaches/software has not been determined.

Page 27: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion “Longpoles” (Page 3)• Distributed Level 1 Architecture, with Tactical Overwatch -

Given the required analytical timelines and the realities of future communications throughputs, Level 1 fusion cannot be centralized for a UE or even a UA. Distributing fusion within a UA implies that issues of adjudication between varying non-synchronized views must be dealt with, and only in a cohesive metadata/data tagging environment. This problem has not be solved nor even sufficiently researched to date. The ability to effectively cross-cue from both single domain and all source results will also be critical. And the addition of tactical overwatch products into the fusion at a UA will complicate a distributed fusion architecture even more.

• Reconstruction of Functional Networks - The application of varied techniques to sort through quantities of information and discover/confirm/assess the non-obvious, n-deep interrelationship networks is not widespread, nor is it readily manageable by average analysts. This capability is critical for many current operations.

Page 28: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Fusion “Longpoles” (Page 4)• Level 2 Fusion at Tactical Echelons - Given insufficient staffs at UA

echelons, the need to perform Level 2 fusion analysis with sufficient automated support on the constrained problem view of the tactical commander is critical.

• Level 2 Fusion at Overwatch Echelons - At higher echelons, the ability to constrain the problem view must be lifted. More robust Level 2 capabilities, especially for urban, asymmetric, stabilization, and other types of operations, will need to be developed.

• Red Models - In order to support Level 2 fusion at both tactical and overwatch echelons, a significant amount of prerequisite knowledge of enemy doctrine/beliefs/structure/methods/etc. will need to be first developed and then formulated for use by automated fusion processes.

• Simplified User Interfaces - Tactical users see/sense much information, yet have no simple, quick way of inputting that information into the fusion process. Similarly, they need easy, understandable access to the results and products of fusion.

Page 29: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Mapping “Longpoles” to EchelonsLongpole Dsmtd SQD Unit of Action Unit of Empl Overwatch

ATR/AiTR

Cross Domain

Sensor Webs

Text/Speech Translate

Text/Speech Exploitation

Distributed L1 Arch/ Tactical Overwatch

Scalability of Level 1

Reconstruction of Functional Networks

Level 2 - Tactical

Level 2 - Overwatch

Red Models Use Use Build

Simplified User I/F

SBU Coll SCI plus

Page 30: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Part Three: Level 1 Fusion Concepts and Architecture

Page 31: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Level 1 Fusion Flowchart

CorrelatedEntities

HCI

Reports/Observations

SpatialAlignment

TemporalAlignment

IdentityAlignment

TechnicalAlignment(s)

Alignment Correlation

Combination(State Estimation)

CandidateRetrieval

CandidateScoring

CandidateAssessment

New, add

Already seen

Determinationof “best”Resultant

record

•COP•Targets/BDA•Sit Alarms•Level 2 Processes

Kn

owle

dge

Bas

es

Page 32: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

The Advantage of Multi-INT(a notional example, not anyone’s official chart of sensor capabilities!)

Type of |Entity Type | Process| Classes of Knowledge |

INT |U Q F I E O | Speed | Loc Mvmt Time Type ID Comp Status Intent|

COMINT Ext x fast

COMINT Int x x x x x x slow

ELINT x fast

IMINT EO/IR x x x * * med

IMINT MTI x fast

HUMINT EPW x x x x x x slow

MASINT ASM x x fast

MASINT MS/HS x x x slow

OSINT x x x x x x slow

ZSU-23-4, seen by

EO and MTI med

EO and MTI and ELINT med

EO and MTI and ELINT and COMINT Int

slow

Page 33: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Part Four: Level 2 Fusion Concepts and Architecture

(with thoughts on Level 3)

(Version 2)(still a work in progress)

Page 34: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Level 2/3 Fusion at UA• Level 1 fusion correlates inputs and produces a set of tracks

170,000 reports => 105 tracks

• Level 2 first associates and aggregates those tracks into forces

105 tracks => 15 “units”

• and those forces into force structures

15 “units” => a BN-sized force

• which will threaten our forces and require the battle to be “shaped”

threatening our forces here

• then proposes that the force is exerting (L2) /will exert (L3) itself in some way

moving to contact this way

Scenario taken from USAIC&FH MAPEX

Page 35: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Why Level 1 Fusion Isn’t Enough

PLT COP

UE

C o m p l e x i t y

6K+ Reports/Hour

Plus…Information from echelons above UA

Report count based on DCGS-A MAPEX

results using Caspian Sea Scenario

Report count based on DCGS-A MAPEX

results using Caspian Sea Scenario

Bde COP

Bn COP

18K+ Reports/Hour

56K+ Reports/Hour

170K+ Reports/Hour

Co COP

Sensor Acquired DataSensor Acquired DataFused to Level 1Fused to Level 1

Reports generated from FCS EO/IR and COMINT Sensors only.Add MASINT sensors and reporting at UA goes to @ 600K/hour.

PLT COP

UE

C o m p l e x i t y

C o m p l e x i t y

6K+ Reports/Hour

Plus…Information from echelons above UA

Report count based on DCGS-A MAPEX

results using Caspian Sea Scenario

Report count based on DCGS-A MAPEX

results using Caspian Sea Scenario

Bde COP

Bn COP

18K+ Reports/Hour

56K+ Reports/Hour

170K+ Reports/Hour

Co COP

Sensor Acquired DataSensor Acquired DataFused to Level 1Fused to Level 1

Reports generated from FCS EO/IR and COMINT Sensors only.Add MASINT sensors and reporting at UA goes to @ 600K/hour.

If only Level 1 fusion is done, there is no roll-up of entities, and the Bde COP

carries all the “little things”.

With only L1: 500+ iconsWith L1 and L2: >50 icons

Page 36: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Level 2/3 Fusion State-of-the-artMid 1980’s

Mid 1990’s

2000-2001

Dec 2001

Jan 2002

June 2002

Oct 2002

JDL Fusion model proposes concept of Levels 2 and 3

ASAS makes small inroads to automating some Level 2 fusion

FCS/OF concept emerges - trades armor for ISR/Knowledge (aka fusion)

DARPA Level 2/3 Symposium - sets an 8-year path for research(RAID planned for FY05 start)

G-2 says 8 years too long; develop ‘03 STO

Notional L2/3Architecture presented

Joint I2WD/ARL FBKFF STO approved

Level1

Fusion

Level3

Fusion

Level2

FusionLevel0

Fusion

Databases(Fusion & Support)

Sources&

Sensors

Level4

Fusion

HumanComputerInteraction

Level5

Fusion

Level 2 - Situation Refinement(how they work together and what they’re doing)

Level 3 - Threat Refinement(what it means and how it

affects our plans)

Page 37: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Level 2 Fusion Flowchart(a work in progress)

ObjectAggregation Aggregate

Analysis

EventDerivation

EventAggregation

ContextInterpretation

“Linked” Correlated Entities

“Linked”Events/Activities

Environment

Red Models

Behavior Hypotheses(large and small)

HCICorrelated Simple Objects

Correlated Complex Objects

Event Reporting

Kn

owle

dge

Bas

es

Learning

Hypotheses of the current behavior/ motive/objectives of the enterprise and its constituents, with some sense of validity/score of each viable hypothesis

Aggregated,linked setof objects

EventInterpretation

& AnalysisHypothesisGeneration

HypothesisAssessment

DeceptionAssessment

Page 38: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Level 2, Decomposed

ObjectAggregation Aggregate

Analysis

EventDerivation

EventAggregation

ContextInterpretation

“Linked” Correlated Entities

“Linked”Events/Activities

Environment

Red Models

Behavior Hypotheses(large and small)

HCICorrelated Simple Objects

Correlated Complex Objects

Event Reporting

Kn

owle

dge

Bas

es

Learning

Hypotheses of the current behavior/ motive/objectives of the enterprise and its constituents, with some sense of validity/score of each viable hypothesis

Aggregated,linked setof objects

EventInterpretation

& AnalysisHypothesisGeneration

HypothesisAssessment

DeceptionAssessment

“Level 2A” – Object Aggregation/Analysis

“Level 2B” -Event/

ActivityAggregation/

Analysis

“Level 2C” – Current Behavior Assessment

Page 39: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Some Object Aggregation Techniques

• Template-based• Example: Observations of lower-tier units or equipment spawn

inferred units of one higher echelon

• Traffic Analysis• Example: Piecing together hundreds of observations of emissions to

reconstruct a communications network

• Geolocation/Movement• Examples: a unit that stays at a facility may be using the facility; a

unit whose equipment is moving west is likely also moving west

• Other Data Mining• Example: What n-deep relationships can two people have if they talk

to the same people or share the same religious center?

Page 40: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Activity “Trees” of Events

Tank #3719moving N

Tank #3717moving N

Tank #3713moving N

+ association of these three tanks to the 33 TK CO

“3 Events”

Event: one object performing one simple* action(* be at, move, shoot, emit, etc.)

Page 41: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Activity “Trees” of Events

Tank #3719moving N

Tank #3717moving N

Tank #3713moving N

+ association of these three tanks to the 33 TK CO

33 TK COmovingNorth

(Conf 85%)

ADA atBridge #36(Conf 90%)

Comms between33 TK CO

and ADA unit(Conf 95%)

“Event Cluster”Event Cluster: complex object

performing one action

Page 42: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Activity “Trees” of Events

Tank #3719moving N

Tank #3717moving N

Tank #3713moving N

+ association of these three tanks to the 33 TK CO

33 TK COmovingNorth

(Conf 85%)

ADA atBridge #36(Conf 90%)

Comms between33 TK CO

and ADA unit(Conf 95%)

ACTIVITYMODEL #121 –River Crossing

Unit movingtowards river

Bridge on unit’sprobable path

ADA deployedat/near bridge

ACTIVITYMODEL #121 –River Crossing

Unit movingtowards river

Bridge on unit’sprobable path

ADA deployedat/near bridge

33 TK COto conduct

rivercrossingoperation

Bridge #36approx.1345Z

(Conf 73%)

“Activity”

Activity: upper ‘tree’construct built from

multiple objects performing a complexaction (derived from

a set of related simple actions)

Page 43: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Level 2 Fusion Flowchart(a work in progress)

ObjectAggregation Aggregate

Analysis

EventDerivation

EventAggregation

ContextInterpretation

“Linked” Correlated Entities

“Linked”Events/Activities

Environment

Red Models

Behavior Hypotheses(large and small)

HCICorrelated Simple Objects

Correlated Complex Objects

Event Reporting

Kn

owle

dge

Bas

es

Learning

Hypotheses of the current behavior/ motive/objectives of the enterprise and its constituents, with some sense of validity/score of each viable hypothesis

Aggregated,linked setof objects

EventInterpretation

& AnalysisHypothesisGeneration

HypothesisAssessment

DeceptionAssessment

Page 44: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Critical point:As is true between Levels 1, 2, and 3,

the interaction between Levels 2A, 2B, and 2C are not linear nor necessarily

sequential.Knowledge is developed by a continual and ad hoc application of all these levels

of fusion processes.

Page 45: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Level 3 Fusion

Takes the Course of Action hypotheses from Level 2C, vetted by the evidence from Levels 1, 2A, and 2B, and projects future Courses of Action hypotheses and assesses them vis-à-vis the capabilities, vulnerabilities, opportunities, environment, etc. of both red and blue.

(A notional architecture remains to be tackled.)

Page 46: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Part Five: Fusion Developments Topics

Page 47: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Enabling Technologies• Ontologies provide organization of the terms and concepts, to

permit clearer cross-understanding of what things are• Intelligent Agents are software that can, given a set of

actions/reactions, provide automation to tasks such as smart dissemination, alert/alarms, sanitization/declassification, etc.

• Knowledge Bases/Management are advanced IT techniques for capturing and manipulating the “higher level” abstractions of information (environment, belief, intent, etc.)

• Service-based Architectures are the emerging means to support net-centric operations; they imply that the processes will be able to function as invokable services hosted within some network space

• Distributed Fusion Architectures are approaches to implementing fusion processing across a multi-echelon net of fusion process services without the creation and use of a central fusion node (to date, this is a very immature field of study)

Page 48: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Army Goals for Level 2 Fusion• “Unit of Action” Level 2 goals

Given that we can sense where the threat is...

– Where is it going and what are its near-term goals?

– When will they see us and decide to shoot?

• “Unit of Employment” Level 2 goals– Who are the players and where are they operating?

– What are the large-scale objectives?

– Why are they doing what they are doing?

BOTTOM LINE: We cannot hope to completely automate these goals…but given the vast amounts of observations that will be collected, we need to find ways to automate hypothesizing knowledge from amidst the information

Page 49: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Part Six: Current Army-Oriented Fusion Work

Page 50: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Army Fusion Efforts• Program Development

• Objective Force Warrior• Future Combat System• DCGS – Army (incl. ASAS, TES, CHIMS, CGS)• INSCOM/Information Dominance Center

• Research• Warrior’s Edge/Horizontal Fusion Initiative• Warrior’s Lens – ARL 6.1 Fusion Work Group• SIBRs

» Adv Viz Support to Fusion; Ontology-based Fusion Model; VMTI Tracker• STOs/ATDs/ACTDs

» FBKFF, Eye in the Sky, Netted Sensors, BTRA, Gnd Station Tech Testbed• Several smaller studies on specific topics

» Urban issues, MASINT phenomenologies, etc.» AIS/Univ Buffalo IR&D on Graph Theory

• Cooperation across DoD• DoD-level fusion community symposia and coordination• Leverage of DARPA efforts (RAID, DDB, DTT, AIM, etc.)

Page 51: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

Part Last: Conclusion

Page 52: Fusion for the Army Knowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004.

A Concluding Thought:

Twenty years ago, the automated fusion of observations and reports into a correlated set of battlefield objects was a worthy but unexplored concept - and research was begun.

Ten years ago, every service had fielded systems that automated some aspects of Level 1 fusion. The fidelity of those systems continues to improve as time goes on.

Army leadership is now preparing to trade some of its heavy armored vehicles for lighter and faster vehicles. But this will only be effective if we can produce quick, predictive views of the battlespace to protect and empower that fighting force. To do this, we must advance the state-of-the-art of fusion, especially beyond the Level 1 correlation techniques done today.

And if we want to advance that state-of-the-art in time for that exchange of armor for knowledge, we need to aggressively begin the work now.