System architecture supporting detection of threats in...
22
21.09.2011 1 System architecture supporting detection of threats in asymmetric warfare Barbara Essendorfer, M.A. Yvonne Fischer, Dipl.-Math. Outline • Introduction • An architecture for surveillance and reconnaissance – Surveillance concepts – Components and systems – Technical requirements – Network and information exchange • Critical event detection in asymmetric warfare • Conclusion and further research
Transcript of System architecture supporting detection of threats in...
21.09.2011
1
System architecture supporting detection of threats
in asymmetric warfareBarbara Essendorfer, M.A.Yvonne Fischer, Dipl.-Math.
Outline
• Introduction• An architecture for surveillance and reconnaissance
– Surveillance concepts– Components and systems– Technical requirements– Network and information exchange
• Critical event detection in asymmetric warfare• Conclusion and further research
21.09.2011
2
MOTIVATIONIntroduction
Civil and military surveillance and reconnaissance- characteristics
• Asymmetric Threats– Not limited to single regions/ areas
• Surveillance of critical areas– Heterogeneous surveillance needs
• People/Personal• Material/ Objects• Harbours/Airports • Infrastructure
– Roads, Railway Tracks– Shipping routes– Network Infrastructure
– Threat starts before the event• Timely (Planning of operations)• Space (Information Exchange, Spying)� Usage of different sensor/s (ranges) &
information
21.09.2011
3
Civil and military surveillance and reconnaissance- characteristics
• Heterogeneous Surveillance Techniques/Platforms– Long, medium, short range– Satellites, aero planes, mobile and static ground
surveillance– Cameras (IR/EO), alarm sensors (radar, PIR…),
tracking devices, humans– Images, videos, reports, texts, alarms– Infrastructure Surveillance
• IT• Change detection etc.
• Flexible Quick Reaction– Often immediate action required– Complex problems – combined specialized forces– Decision/ Reaction needs to be well coordinated in
existing hierarchies– Decision/ Reaction needs to be well coordinated
among different organizations/ nations
Civil and military surveillance and reconnaissance- characteristics
• Analyze– Timely requirements
• Ad Hoc reaction to immediate threat• Environmental analysis based on contextual
information– Critical events and mass data
• Automatic procedures• Identification of „abnormal behavior“
– Dependent on cultural/ ethnic aspects– Dependent on situation– Dependent on context
– Connection to effectors– Information exchange and archiving
• Preservation of evidence• Secondary exploitation
21.09.2011
4
Civil and military surveillance and reconnaissance
• Information Security– Not all information is meant to be shared
• Organizational restrictions• National restrictions• Hierarchical restrictions
– Especially true for fused information– Information needs to be reliable and not
corrupted– Trust – information and -source
� Surveillance of different objects/areas of interest � with different sensors and platforms� by different organizations� and share information and services adequately
DEFINITIONIntroduction
21.09.2011
5
Information Management CycleCoping With The Unexpected
VirtualKnowledge
Base
InformationInformationInformationInformation
ExploitationExploitationExploitationExploitation
Multi Multi Multi Multi Source Source Source Source
Data FusionData FusionData FusionData Fusion
Information FusionInformation FusionInformation FusionInformation FusionDecision SupportDecision SupportDecision SupportDecision Support
DisseminationDisseminationDisseminationDissemination
Data AcquisitionData AcquisitionData AcquisitionData Acquisition
Requests forRequests forRequests forRequests forInformationInformationInformationInformation
DecisionDecisionDecisionDecision
Virtual knowledge base in reconnaissance
Technical requirements
An architecture for surveillance and reconnaissance
21.09.2011
6
IT Security• Network security
– Safety from interception– Encryption of data– Network accreditation
• System security– System accreditation
• Auditing• Radiant emission• User certification
• Data security– Classification of data (Classification, Policy, Releasability)– Definition of user roles– Authentification
Rights and roles• Definition of user roles
– User rights according to roles• System access• Access of data and services
– Rights• Security classifications• Tasks
– Read, write, editing data (content)– Read, write, editing data types– Deletion and archiving of data/ system administration
• Role specific access (AOI, Time, granularity)• Nation specific access (through data classification)
21.09.2011
7
Data types
• Imagery data– Electro optical, thermal images/infrared, Synthetic Aperture Radar (SAR)– Annotated images
• Videos/ Motion Imagery– Electro optical, thermal images/infrared – Annotated Motion Imagery
• Tracks and alarms– Alarms (Predefined threshold, boundaries, rules)– Sources: magnetic, acoustic, seismic, thermal, radar, HUMINT– TracksMaps
Standardization
• Critical event detection needs different data types– Dynamic integration of sensor data and flexible data access– Quick adaption to changing requirementsBUT
• Commercial surveillance sensors are proprietary– Internal specific data formats� flexible architecture needs standards
• Critical event detection� specific requirements– Time sensitive– Different sources� Existing standards (military and commercial) need to be analyzed and eventually
adapted
21.09.2011
8
Sensor integration and information exchange
An architecture for surveillance and reconnaissance
Standardized dissemination of ISR Information
CSD ServerSTANAG 4559
ReportVideoImage Track
Standards (STANAGS)
Metadata
Client- Subscription/Query
Relevant Data
Task
Synchronization
• MAJIIC (Multi-sensor Aerospace-ground Joint ISR Interoperability Coalition) – until 2010
– 9 nations and NATO
– Operational and technical interoperability in ISR
– CSD (Coalition Shared Data)
• To store and retrieve ISR data
• STANAGS/ Standards
– Client access through standardized interfaces (Create, Query, Subscribe, Update), standardized metadata
– Product data (Video- 4609; Imagery- 4545, Reports 3596/ 3377)
– Yearly trials and further development
21.09.2011
9
Data management by metadata• Definition of a common metadata model
– Core metadata
• File information
• Releasability and security markings
• Spatial and temporal location
• General description
– Specific information
• Spectrum of imagery or video data
• Priority markings
• Status markings etc.
GOOD BUT NOT ENOUGH
Interoperability across domains needs more coordination !
Modeling interoperability
• Organizational aspects of interoperability – Legal aspects– Coordination of operational processes
• Command structures and workflow• Need to know/ Need to share
– Organisational structures• Role based access• System set up
– Service descriptions for organizational functions and structures• Metadata on creator/ source/ organization• Service Level Agreements
Transnational Level
National Level A
National Level B National Level C
Regional Level A
Regional Level B
Local Level 1 Object Surveillance
Local Level 2Object Surveillance
21.09.2011
10
Modeling interoperability
• Structural aspects of interoperability – Equipment interoperability– Standardized data formats– Standardized exchange methods
• Information exchange– Common understanding
• Definition of data values• Transformation of operational business rules in technical business rules
Modeling interoperability
• Information exchange
– Information access criteria
• User roles (esp. data validation, approval)
– Common semantic
• Common practices (when to use what)
• Meaning of values (what does it mean?)
• Service usage
• Automated transformation / Conversion of data based on standard descriptions
– Management framework
– Manage documentation and linkage between documents
21.09.2011
11
PROJECT WIMA²SCritical event detection in asymmetric warfare
Project WiMA²S a system of systems architecture
21.09.2011
12
Project WiMA²S
• Main Goal:
Support Situation Awareness of decision makers by Common Maritime Picture
• In a system of systems this is done by
– Fusion of sensor data collected from multiple platforms/sensors
– Representing the fused information about the current situation at sea
– Guiding the focus of attention of a decision maker to relevant activities
– Supporting the situation-dependent sensor and platform tasking process
Project WiMA²S
• Therefore, it is needed to:
– Define a generic fusion-architecture for maritime surveillance
– Visualize fused information
– Develop methods for situation assessment
– Calculate information needs for sensor planning
21.09.2011
13
FUSION ARCHITECTURECritical event detection in asymmetric warfare
Fusion architecture
• Challenges due to the System of Systems-architecture:
– Heterogeneous types of sensors
– Different ranges and incomplete coverage
– Different revisiting times
– Different feature extractions
– Mobile and stationary platforms– Limited connectivity and bandwidth to ground control station
21.09.2011
14
Fusion architecture
(a) Application-level modules, (b) Signal-Processing modules, (c) Sensor Deployment
OOWM: Object Oriented World Model
Fusion architecture
• Functionalities of the OOWM-Fusion-Architecture:
– Application-Independent representation of objects, their attributes anduncertainties
– Standardized interfaces
– Sensor- and data-fusion on object-level, including data association andtracking methods
– Data fusion of new information with older information
– Object classification
– Management of object instances
– Serves as an information source for application level modules
HMI
Behavior
Analysis
Anomaly Detection
Video-Based
Object Tracking
Radar-Based
Object Tracking
AIS Interface
OOWM System
21.09.2011
15
Fusion architecture
• OOWM is the basis for higher-level fusion methods (situation assessment) and thus the critical element for supporting maritime domain awareness
• Algorithm selection (data fusion, association and tracking) depends on:– sensor characteristics
– application domain
– properties of objects under surveillance
• Sensor coverage and age of information is explicitely modeled
• Probabilistic representation due to uncertainty introduced by inaccurate signal-processing and incomplete sensor coverage
Fusion architecture
OOWM Benefits:
• Algorithms are encapsulated as software modules
• Generalized object representation is independent of application and signal-processing modules
• The architecture enables– Connection of new sensors
– Independent development of new modules
– Extension of the system with improved algorithms
21.09.2011
16
SITUATION ASSESSMENTCritical event detection in asymmetric warfare
Situation assessment
• Due to the maritime domain, almost all observed ships are cooperative, but operators are Interested in non-cooperativ ships
• Higher-level Challenges:– Differentiate between relevant and irrelevant activities
– Detection and localization of non-cooperative ships
• Goals:
– Due to the lack of training data – especially for critical situations – they aredescribed by experts
– Early detection of an ongoing situation for timely alarm
– Indications of relevant objects or areas as an input for further sensorplanning
21.09.2011
17
Situation assessment
• Scenario: vessels carrying refugees on board
• Characteristics:
– Material: Wood
– Length: ~15 meters
– Many people on board
– No AIS-Signal
– Slow speed
– Start in Libya or Tunesia
– Direction Lampedusa
– Most of the time directly
Situation assessment
• Behavior classification with a Bayesian Network:
– Determines probability of each boat to carry refugees
– Based on object informations in the OOWM
– Structure and parameters are not learned – they are set due to the scenario description
21.09.2011
18
Situation assessment
• Result of calculation if following evidences has been observed:
– Heading: direction of Lampedusa
– No AIS-Signal
– Speed: about 10knots
Situation assessment
• Challenges:
– Structure modelling
– Parameter determination (transition probability)
21.09.2011
19
Situation assessment
• Current work: Dynamic Bayesian network forsituation assessment with thefocus on:
– Inclusion of temporal processes
– Conclusion on past events
– Prediction of events in thenear future
– Recognition of sequences ofstates
SUMMARY AND FURTHER WORK
21.09.2011
20
Summary I• Combination of Sensor(platforms) and data types is essential for the critical event
detection
• Dynamic critical event detection must be adaptable to tasks and user
• Interoperable data and information management has to include:
− Adaptable processes
− Decentralized design (common rules for data management)
− Distributed data- and information storage
− Distributed adaptable access
− Security management
− Metadata for categorizing to support intelligent query
− Management of all data types and associations
Summary II• Standardization is essential
− Interoperability
− Easy integration of new systems by software components
− Adaption of surveillance systems to „new“ threats
− Critical situation assessment needs
− A description of the objects and their characteristics that are known to exist in a specific domain
− Detailed analysis of the domain knowledge
− Training data
− Expert knowledge
− Classification of behavior
21.09.2011
21
Further work• Archiving of data / information and information exchange
− Collected experience to facilitate learning through test and simulation
− Foundation for future operations
• Semantic Interoperability/ Ontologies
− Standardization
• Service concepts
− Service infrastructure
− System spezific services
Comment
• Presented architecture enhances protection possibilities
• BUT
− If something is not perceived as a threat it can not be recognized (blind spot)
− Sometimes only analysis of events after occurrence
− Not everything can be perceived
− Legal restrictions
21.09.2011
22
Thank you for your attention
• Contact:• Barbara Essendorfer M.A.• Team Lead System Architecture, IAS• Fraunhofer IOSB• [email protected]• 0049-721 6091 596
