2015 Deep Dive Seminar: STK EOIR
Transcript of 2015 Deep Dive Seminar: STK EOIR
2015 Deep Dive Seminar: STK EOIR
Summary by Patrick North
Introduction
Patrick North – Passionate Image and Computer Scientist
Sensor and Image Exploitation SME– Exploitation Algorithm Development– Sensor Design and Performance
Assessment and Requirements Development
– Sensor and Mission Simulation– Calibration and Anomaly Resolution– Operations Support
www.agi.com/eoir
Overview
What is STK EOIR
What can it be used for
What do the outputs look like
How does it work
What validation and verification has been done so far
How are new features added into EOIR
Overview: What is STK EOIR?
Heritage:– Originally developed by the Space Dynamics Lab for Missile Defense
simulations
Generally Applicable:– Radiometric ray-tracer– Physics based rendering engine for 0.28 to 28 micron wavelengths
Framework:– Directly embedded in STK with direct access to the STK capabilities
Out-of-the-Box Ready to Go:– Global background with a thermal model– Exceptional point source modeling– Ability to generate analytical metrics
Overview: What can it be used for?
Remote Sensing, Imaging System, and Sensor Payload Design, Engineering, and Analysis
System Level
Individual Components
Test and Evaluation Data Generation
Proposals, Presentations, and Education
Concept Development
DesignField Test
Operations Support
Overview: What can it be used for?
System Level
– Performance prediction and assessment
– Requirements development
– System design and mission verification
Overview: What can it be used for?
Individual Components
– Radiometric budget analysis
– Component level optimization with qualitative and quantitative analysis
Overview: What can it be used for?
Test and Evaluation Data Generation
– Create exploitation data for legacy or candidate algorithms
– Forensic analysis of system or collection anomalies
Overview: What can it be used for?
Proposals, Presentations, and Education
– Quick high quality graphical output for experimentation and creating presentations
– Fast and accurate real-time analysis for presentations or training
Overview: What do the outputs look like?
Video available at www.agi.com/eoir
Overview: How does it work?
Surface Albedo / Reflectance & Emission
Atmospheric Effects
Sensor Entrance Aperture Radiance
Focal Plane Irradiance
Sensed Analog Electrical Signal
Raw Digital Signal
Processed Digital Image
Note: Not all tools need to provide all features, for example intermediate products or tap points can be
processed with different tools and a daisy-chain approach can be taken to produce the best results.
Source Illumination
Overview: How does it work?
EOIR Simulation Steps1. Create a platform2. Add a sensor3. Project FOV4. Sample all objects in FOV as area or point
source objects5. Repeat sampling N-times for each
intersected segment or object6. Calculate thermal emission and reflection
spectrally along each of the coupled segments and objects paths as well as atmospheric path radiance and transmission losses as entrance aperture radiance
7. Convert spectral radiance/irradiance to integrated focal plane irradiance
8. Apply MTF sources and convert to electron signal
Overview: How does it work?
User Interface and Interaction
STK Property Pages
EOIR GUI’s
Data Providers
Connect Commands
Object Model
Overview: How does it work?
Modeled
Objects
Synthetic
Scene
Generator
Sensor
Model
Position and Orientation,
Shape and Dimensions,
Optical Properties,
Temperature
Celestials
Satellites
Aircraft
Missiles
Line of Sight
Field of View
Wavelengths
Digitized Sensor Image Files
Performance Metric Reports
Digitized Scene Image Files
Image Interrogation Reports
Spectra
Note: Point vs Area Treatment
Overview: How does it work?
Earth Model
0.93 km resolution materials map– International Geosphere-Biosphere Program [IGBP]
Atmosphere– Simple “fast” model
• Four aerosol options
• SMARTS2 for shortwave (< 4um)
• Optical depth vertical profile (>4 um)
Diurnal Thermal Model– Based on a seasonal and daily latitude dependent
sine wave
– Varies based on land or water surface
Overview: How does it work?
Stars and other Planets
Utilizes STK star database and planetary positions
Material maps for mars and earth’s moon
Basic thermal models and simple atmospheres for all planets in solar system
Video available upon request
Detector
Optics
Detector
Detector
Focal Plane
Optics
1 Pixel =
Instantaneous
Field of View
Field of View
Vertical
Horizontal
Pixel
Pitch
Sensor Modeling – Spatial
Overview: How does it work?
High Band Edge(Longer Wavelength)
Low Band Edge(Shorter Wavelength)
Number of Intervals [ = 6 ]Sensor Modeling – Spectral
Overview: How does it work?
Longitudinal Defocus
Detector
BestImage
Entrance Pupil Diameter
Effective Focal Length
F/# = Entrance Pupil Diameter
Effective Focal Length
Detector
Sensor Modeling – Optical 1
Overview: How does it work?
Point Source(Pure Impulse Input to Optical System)
Diffraction(Optical System Response)“pixelized” by the Detector
Approximately = Sine(Diffraction Wavelength)
Diffraction Wavelength
2
The Best That Physics Allows“Diffraction Limited”
2.44 x Diffraction Wavelength xEffective Focal Length
Entrance Pupil Diameter
Including SomeOptical Aberrations
Sensor Modeling – Optical 2
Overview: How does it work?
Line Of Sight
Full Width at Half Maximum / 2.35
GaussianDistribution
Line Of Sight Jitter parameter =
Sensor Modeling – Optical 3
Overview: How does it work?
Motion Blur
•Irradiance (W m-2)
Point Source Images
•Radiance (W m-2 sr-1)
Resolved Shape Images
NEI, NER
SEI, SER
Noise Floor
Saturation
0.0E+00
5.0E-06
0.1 1 10Irra
dia
nce
W c
m-2
milli seconds
NEI Edit Table
Sensor Modeling – Radiometric
Overview: How does it work?
𝐷𝑦𝑛𝑎𝑚𝑖𝑐 𝑅𝑎𝑛𝑔𝑒 =𝑆𝑎𝑡𝑢𝑟𝑎𝑡𝑖𝑜𝑛
𝑁𝑜𝑖𝑠𝑒 𝐹𝑙𝑜𝑜𝑟
Overview: How does it work?
Target Modeling
– Position and Orientation
– Shape and Dimensions
– Temperature
– Material
Overview: How does it work?
Outputs
– Simulated Images
– Interactive Interrogation
– Reports
Overview: What V&V has been done so far?
Academic Radiometric Calculations
MODTRAN Atmospheric Terms
Commercial Space Operations Center (ComSpOC) Collections
Radiometric Slides and Worksheet
Original EOIR V&V Report
MONET Analytical Signal Estimation
Original EOIR V&V Report
Overview: What V&V has been done so far?
Video available at www.agi.com/eoir
Overview: What V&V has been done so far?
Optical Sensor Modeling: Shape and Material
Video available upon request
Overview: How are new features added?
Customer and partner feedback
AGI field team input
Conversations with potential customers and collaborations
General research and trends
Break – Next Applications and Use Cases
www.agi.com/eoir
Applications and Use Cases
Missile Defense
Space Situational Awareness
Broad Area Intelligence, Surveillance, and Reconnaissance (ISR)
Air-to-Air Battlespace Awareness
Collaborations and Partnerships
Missile Defense
Primary Needs
Global Background
Atmospheric Model
Thermal Model
Accurate Sensor and Target Modeling
STK EOIR offers a very well suited simulation and analysis framework for SDL heritage missile defense scenarios
Missile Defense
Missile Defense Problem EOIR Solution
Operational: Determine how wellIron Dome optical sensors can detect and track missile threats
Generate analysis including sensor-to-target metrics and sample images to evaluate the current operational sensor detection and tracking capabilities against the system requirements
Future System Design: Determine optimal DSP constellation to deploynew satellite sensor system
Generate detection, tracking, and chain-of-custody analysis using EOIR SNR metric constraints to support constellation design decisions
Data Exploitation R&D: Generate test data to validate Technology Readiness Level (TRL) of prototype missile characterization algorithm
Generate simulated EOIR images to evaluate performance against simulated truth to determine continuing or canceling the R&D effort
MD Recorded Scenario Creation
Space Situational Awareness
Primary Needs
Celestial Background
Atmospheric Model
Accurate Sensor and Target Modeling
Accurate Time Dynamics
STK EOIR provides the ability to model conceptual or operational systems for SSA such as our own ComSpOC to
design, optimize, and assess these SSA solutions
Space Situational AwarenessSSA Problems EOIR Solution
Operational: Determine if existing sensor systems can characterize if a detection is a single or multiple RSO
Generate analysis including sensor-to-target metrics and sample images to characterize system capabilities and operational readiness
Future System Design: Evaluate various sensor configurations to determine the best network for persistent monitoring of geostationary satellites
Generate EOIR sensor-to-target metrics reporting each geo’s maximum detected signal across the network throughout the period of analysis
Component R&D: Evaluate a new sensor prototype being developed for installation in legacy GEODDS systems
Generate before and after simulated EOIR images and sensor-to-target metrics to justify agency maintenance and upgrade investments
SSA Recorded Scenario Creation
Broad Area ISR
Primary Needs
Global Background
Target Metrics
Optimization and Product Integration
Accurate PlatformDynamics
EOIR imagery integrated with other GIS products provides the campaign view the
IC is moving towards
Broad Area ISRGround Imaging ISR Problems EOIR Solution
Operational: Determine access time, NIIRS, and illumination conditions for highvalue target during a SOF mission window across all NTM
Generate analysis including sensor-to-target metrics to be post-processed into NIIRS ratings and provide sample images to generate an integrated product for mission planning purposes
Future System Design: Perform trade study between two competing sensor systems for collecting campaign data
Generate sensor-to-target metrics for high interest targets over period of performance and develop integrated coverage and quality reports to justify system acquisition strategy and requirements definition
Data Exploitation R&D: Send UAVthermal test data to small target exploitation algorithm developers to evaluate their Technology Readiness Level (TRL)
Generate before and after simulated EOIR images and sensor-to-target metrics to evaluation small thermal target exploitation capability
Air-to-Air Battlespace Awareness
Primary Needs
Accurate Flight Dynamics
Atmospheric Model
Thermal Model
Integrated with RADAR and Comms
STK EOIR provides the framework to model concepts
such as the JSF DAS-EO sensors
Air-to-Air Battlespace AwarenessAir-to-Air Battlespace Problems EOIR Solution
Operational: Determine effectiveness of adversarial denial-and-deception against tracking systems
Generate analysis including sensor-to-target metrics and sample images to determine the effectiveness and potential threats posed in red-force/blue-force confrontations
Future System Design: Evaluate full spherical sensor concept for 360-degree situational awareness
Generate EOIR simulated images and provide to aircraft sensor integration software to evaluate future system performance and provide scenario views for qualitative evaluation and budgetary go/no-go decisions
Platform R&D: Evaluate the air-to-air surveillance capabilities of an experimental high altitude long-dwell UAV
Generate EOIR images showing area of coverage and resolution and sensor-to-target metrics for objects of interest to show capability and justify go/no-go budgetary decisions
Collaborations and Partnerships
Current Collaboration Projects
High Fidelity Infrared Focal Plane Modeling
Time-Dynamic Thermal Profiles
Custom 3D Models and Materials
MODTRAN Based Atmospheric Model
Interplanetary Laser Communications Test
Potential Collaboration Efforts
Space Debris
Wildfire Detection
Break – Next What’s New in STK EOIR 11.0
www.agi.com/eoir
What’s New in STK EOIR 11.0
Magnitude of Improvements
Integrated STK EOIR Sensors
New atmospheric model
Custom 3D models
Custom materials
Custom temperature profiles
Magnitude of Improvements
Value
Difficulty
Save Intermediate Products
STK EOIR Sensor
Custom Temperature
Profiles
Internal Reflections
Editable Solar Spectrum
Custom 3D Models
Output Documentation
Custom Materials
New Atmosphere
5 New Features and37 vs 9 Resolved
Issues from 10.1 as of 7/10/2015
…
Integrated STK EOIR Sensors
Moving from 3rd party to STK gives us
STK Property Pages
Closer Access to STK
Object Model
Access to Analysis Work Bench
New Atmospheric Model
What Was Missing
EOIR Atmospheric Parameters and Setup
Qualitative Comparisons
Quantitative Comparisons
I MODTRAN™Atmospheric model description
whitepaper and slides
Source geometry interpolation
New Atmospheric Model: What’s new?
Visibility Effects
Skylight
New Atmospheric Model: Atmospheric Setup
3) All of the Atmospheric Parameters are the same, but now there’s a 3rd
option for Atmosphere Model
1) Start by choosing the EOIR Configuration button on the toolbar
2) On the EOIR Configuration pop-up choose the Atmosphere Definition
New Atmospheric Model: Setup Aerosol Model
– Affects the type of aerosols in the atmosphere, apparent in both visible and thermal imaging
– Best atmospheres to see through in rough order:• Troposphere• Rural• Urban• Maritime
Visibility– This determines the quantity of the specific aerosols and is proportional to
how far one could see on the ground in the visible spectrum
Relative Humidity– This determines the amount of water vapor in the lower atmosphere
causing stronger H2O spectral features (reducing transmission and increasing thermal emission at certain wavelengths) at higher levels
Surface conditions are interpolated to upper atmosphere through lapse rate
relationships
New Atmospheric Model: 5 Scenarios
1) Looking Up at the Stars At Night in the Visible Spectrum, Varying Visibility
2) Looking Up at the Stars During the Daytime in the Visible Spectrum, Single Comparison
3) Looking Up at the Stars During the Daytime in the Shortwave IR Spectrum
4) Looking Down at the Earth During the Daytime in the Visible Spectrum
5) Looking Down at the Earth During the Daytime in the Midwave IR Spectrum
New Atmospheric Model: Scenario 1
Visible sensor on the ground
looking up at the stars at night
New Atmospheric Model: Scenario 1
Aerosol: RuralVisibility: 50.0 kmHumidity: 45.8 %
New Atmospheric Model: Scenario 1
Aerosol: RuralVisibility: 27.0 kmHumidity: 45.8 %
New Atmospheric Model: Scenario 1
Aerosol: RuralVisibility: 10.0 kmHumidity: 45.8 %
New Atmospheric Model: Scenario 1
Aerosol: RuralVisibility: 5.0 kmHumidity: 45.8 %
New Atmospheric Model: Scenario 1
Aerosol: RuralVisibility: 2.0 kmHumidity: 45.8 %
New Atmospheric Model: Scenario 1
Takeaways
New MODTRAN based atmospheric model properly handles visibility as one would expect and as visibility decreases the stars disappear
At around 27 km visibility the models match
Even though the Simple model looks the same in all cases the values do slightly change
New Atmospheric Model: Scenario 2
Visible sensor on the ground looking up at the stars during the
daytime
New Atmospheric Model: Scenario 2
Aerosol: RuralVisibility: 50.0 kmHumidity: 0.0 %
New Atmospheric Model: Scenario 2
Takeaways
Just as one would expect stars are not easily visible during the daytime with the MODTRAN model
There are a few stars visible in the direction farthest from the sun and it is afternoon in the winter time so the MODTRAN based model looks like it’s accurately modeling the first visible stars
The Simple model does not accurately simulate skylight (scattered path radiance)
New Atmospheric Model: Scenario 3
SWIR sensor on the ground looking up at the stars during
the daytime
New Atmospheric Model: Scenario 3
Aerosol: RuralVisibility: 50.0 kmHumidity: 0.0 %
Daytime Satellite Imaging Analysis
New Atmospheric Model: Scenario 3
Takeaways
The MODTRAN model shows more stars are visible because of the lower level of SWIR skylight, however it is definitely still present
The Simple model again still does not simulate skylight (scattered path radiance) for the SWIR spectrum
New Atmospheric Model: Scenario 4
Visible sensor on a satellite looking
down at the earth during the daytime
New Atmospheric Model: Scenario 4
Aerosol: RuralVisibility: 27.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 4
Aerosol: RuralVisibility: 15.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 4
Aerosol: RuralVisibility: 10.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 4
Aerosol: RuralVisibility: 5.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 4
Takeaways
From space the MODTRAN model seems to handle the visibility parameter realistically
At 10 km visibility the two models agree
Simple model varies with visibility but is not appear to be drastic enough for the space based simulation
New Atmospheric Model: Scenario 5
Midwave Infrared (MWIR) thermal
sensor on a satellite looking
down at the earth during the daytime
New Atmospheric Model: Scenario 5
Aerosol: RuralVisibility: 27.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 5
Aerosol: RuralVisibility: 15.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 5
Aerosol: RuralVisibility: 10.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 5
Aerosol: RuralVisibility: 5.0 kmHumidity: 25.0 %
New Atmospheric Model: Scenario 5
Takeaways
From space both the MODTRAN and Simple models appear to vary slightly with visibility, the aerosols in the rural model affect the MWIR spectrum less drastically than in the visible spectrum
The MODTRAN model shows greater contrast under the given atmospheric conditions than the Simple model
New Atmospheric Model: Quantitative
Bring Up MODTRAN
Comparison GUI
Custom 3D Models
Custom 3D Models
Custom 3D Models
Video available upon request
Custom 3D Models
Bring Up 3D Model Conversion Writeup
Create and Render 3D Model Sample
Custom Materials
EOIR BRDF equation derivation
Spectral material database
27 built in materials
Custom reflectance spectra option
Custom Materials
Custom Materials
Custom Materials
Lepidolite
Custom Materials
Caesium
Custom Materials
Drilling Fluids
Custom Materials
Custom Materials
Video available upon request
Custom Temperature Profiles
Static temperature
Time-dynamic temperature profile
Custom Temperature Profiles
Custom Temperature Profiles
Custom Temperature Profiles
Custom Temperature Profiles
Video available upon request
Custom Temperature Profiles
SEET Passive Thermal Model
Astrogator Re-entry
Break – Next EOIR Development Roadmap
www.agi.com/eoir
Future DevelopmentIn General
Faster
More Intuitive
More Reliable
More Effective
Current Limitations:
High spatial/temporal resolution imaging
Complicated 3D geometry and interactions
Fine detail system and target modeling
Future Development: What doesn’t EOIR do right now?
Future Development: What doesn’t EOIR do right now? High spatial/temporal resolution imaging
– Ground material maps for earth are roughly 1 km or ½ nmi resolution
– Oversampled rendering performed all in-memory
– 4x spatial oversampling and coarse temporal oversampling for blur
– Not taking advantage of STK light-time-delay calculations
Development Packages
Atmosphere and Broad Area Effects
World/Background Modeling
Performance
Future Development: What doesn’t EOIR do right now? Complicated 3D geometry and interactions
– Limited to thousands of polygons rather than millions
– Not making use of modern graphical algorithms or hardware optimizations
– Shadows are limited to the facet resolution of the 3D geometry
Development Packages
World/Background Modeling
Performance
High Fidelity Target Modeling
Future Development: What doesn’t EOIR do right now? Fine detail system modeling
– Optical prescription modeling from OTF, Zernike Polynomials, or Seidel Aberrations
– Low-level hardware specifications and spectral performance curves
– Outside the Field-of-View Stray-Light effects
– Specific jitter profile or PSD inputs
Development Packages
Performance
High Fidelity Sensor Modeling
Future Development: Example Value vs Difficulty Plot
Value
Difficulty
CloudsUpgraded Rendering
Engine
Directly Input Complex Sensor
Parameters
Material Maps
Sensor Param Time
Profiles
Active Laser Modeling
More Data Access
Verification
Future DevelopmentRelated Upgrade Packages
Related Development Packages
Atmosphere and Broad Area Effects
World Modeling
Metrics
Performance
High Fidelity Target Modeling
High Fidelity Sensor Modeling
Visualization and Compatibility
Exploitation Algorithms
Verification Work
Branching Out
Future Development: Atmosphere and Broad Area Effects
Cloud Modeling
Optical Refraction in Ray Tracing
Wide Area Atmospheric Effects of Turbidity (Adjacency Effects) and Stray Light
Future Development: World Modeling
City Based Light Pollution Map
Galactic Background Texture Map
Utilize DTED or STK Terrain for Global Terrain Elevation
Utilize Bing Maps or STK Background Imagery for High Spatial Resolution Texture
Future Development: Metrics
Visual Magnitude
Contrast Metric for Sensor-to-Target
Atmospheric Terms for Sensor-to-Target
Fast Analytics Capability Option
NIIRS
Standard Peak SNR Metric including MTF and Motion Blur
Surface Illumination Maps and Contour Plots
Future Development: Performance
Profile and Benchmark Entire Current Codebase
Develop and Utilize EOIR Specific Unit and Regression Tests
Develop Asynchronous Low-Fidelity and High-Fidelity Rendering Paths
Tile-Based Rendering for Larger Images and Parallel Operations
Replace Rendering Engine with Upgraded/Optimized Modern Engine
Future Development: High Fidelity Target Modeling
Improve or Read Active Thermodynamics Modeling
Integrated STK EOIR 3D Model Geometry and Articulation
Apparent Position Information Incorporating Light-Time-Delay
Future Development: High Fidelity Sensor Modeling
Automatic Integration Time Calculation Option
Implement Scan Modes
Implement Depth-of-Field Based MTF
Implement Auto-Focus
Improve Optics Model
Outside FOV Stray Light Model
Load Custom MTF File and Allow Time-Dynamic MTF Selection
Future Development: Visualization and Compatibility
Load Color Bands
Add New Colormaps (B->G->R)
Export GIS Products
New Sensor Interface w/ Access to More Parameters & Quick View
Progress Bar
Improve Image Viewer
Import OPTISIG/IESNA/other Target Intensity Distributions
Export DIRSIG Renderable Scenes
Future Development: Exploitation Algorithms
Target Tracking and Signal Characterization
Image Equalization, Flattening, and MTFC
Ground Based Geolocation PRC Generation, Calibration, and Accuracy Assessment
Celestial Registration and Space Object Pointing Accuracy
Image/Band Registration and Change Detection
Material and Target Detection and Output
Target and Anomaly Detection and Output
Temperature Estimation w/ Calibration Options and STRR
Future Development: Verification Work
Evaluate Radiometric Transition from Point to Area Sources
Atmospheric Verification Work Including Turbulence and Turbidity
Evaluate or Develop Twilight Models
Future Development: Branching Out
Ground Truth Verification of Models
Customer Outreach
Develop EOIR Training Course
Publish Analysis Whitepapers
Create Non-ITAR/Web Based Sensor Model Branch