Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the...
-
Upload
roy-palmer -
Category
Documents
-
view
219 -
download
0
Transcript of Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the...
![Page 1: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/1.jpg)
Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation
(MTWS)
Major Donald Herod, USMC
In Partnership with SEED
Training as we fight;Fighting as we train!Advisor: William Becker, Ph.d
Co-Advisor: Prof Curtis Blais
![Page 2: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/2.jpg)
• As a simulations community, we do a very good job of verifying and validating algorithmic functionality and “real world” platform-based performance in a constructive simulation
• As a simulation community, we have NEVER determined how to conduct a holistic validation of a constructive simulation so as to determine if the model as a training tool produces “real world” responses/outputs.
• This can lead to negative training transfer and unnecessary loss of life in combat.
The Problem
![Page 3: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/3.jpg)
• The Marine Corps premier constructive simulation… capable of full range of MAGTF combat, combat support and combat service support applications, including amphibious operations.
• MTWS… still meeting the doctrinal expectations of the current operational environment.
![Page 4: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/4.jpg)
System Overview
• Constructive, stochastic, aggregate level simulation of ground, air, maritime, and amphibious operations
• Computer-assisted command and control training system
• Provides exercise control services and tactical combat simulation– Multi-sided, real-time– Full range of battle space capabilities– All phases of military operations
![Page 5: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/5.jpg)
System Benefits
• User-definable parametric data (characteristics and behaviors)
• Global play box; user-definable regions for digital terrain data
• Tool for course of action evaluation in operational planning
![Page 6: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/6.jpg)
Thesis Overview
• Research Question: Can the outputs provided by a constructive simulation be data-farmed to determine if the model’s performance parameters produce valid outputs that stimulate the training audience? Additionally, is there a range of values or specific design points that cause the system to produce algorithmic anomalies (verify)?
• Abstract: The requirement to routinely assess staff training simulations is necessary to ensure the interaction within the simulation provides a realistic training environment. This project will explore methods to apply data farming to MTWS to obtain information about its computational characteristics as part of a more general study into use of data farming for simulation validation.”
![Page 7: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/7.jpg)
Getting Started• Scenario Development
– Think small… simulation is like Pandora's Box– What is the training objective?
• Identify Factors of Interest– Deep dive into the parametrics that govern computations in the
model– The number of factors will determine the number of Design Points
required.
• Methodology – Design of Experiments (DOE)
• Nearly Orthogonal Latin Hyper-cubes (NOLH)
– Batch File Replication– Simulation Run (AKA: Micro-Battles)– Data Mining– Statistical Analysis/Prediction Profiling
![Page 8: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/8.jpg)
8
Orthogonal Latin Hyper-cube
![Page 9: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/9.jpg)
MTWS Output
9
![Page 10: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/10.jpg)
Multivariate Correlation & Scatterplot
10
![Page 11: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/11.jpg)
3D Scatterplot of the Sample Space
11
![Page 12: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/12.jpg)
Stepwise Fit Model
12
![Page 13: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/13.jpg)
Stepwise Fit Model
13
![Page 14: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/14.jpg)
Prediction Profiler
14
![Page 15: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/15.jpg)
• Casting a wider net…– Data Farm all User Adjustable Factors of Interest
• NOLH with 20 factors of interest (10 per side)
Future Work
- K-Kill Hardness- Mounted Hardness- F-Kill Hardness- Cbt Power- Hit Probability
- Firing Rate- M-Kill Hardness- Vulnerability- Destructive Pwr- Range
• DOE containing 129 DP (30 reps per DP) • 3870 Micro-Battles
– Generate MTWS Batch Files• Trouble-shoot errors and dependencies
– Stepwise Fit a Model with interactions • Generate Prediction Profiles for Factors on Interest
![Page 16: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/16.jpg)
Impacts
• Present a new approach to VV&A that provides a holistic assessment of stochastic constructive simulations.
• Recommend enhancements to MTWS based on observed anomalies to verify that the model is producing expected outcomes.
• Generate a more scientific means of determining appropriate parametircs that are valid for a specific training requirement.
![Page 17: Ascertaining Validity in the Abstract Realm of Constructive Simulation Models: An Analysis of the MAGTF Tactical Warfare Simulation (MTWS) Major Donald.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1d5503460f94c33d8d/html5/thumbnails/17.jpg)
Questions