Prediction of Computational Quality for Aerospace Applicationscadmus.usc.edu › webdocs ›...
Transcript of Prediction of Computational Quality for Aerospace Applicationscadmus.usc.edu › webdocs ›...
NASA Langley Research Center - 1Workshop on UQEE
Prediction of Computational Qualityfor Aerospace Applications
Michael J. Hemsch, James M. Luckring, Joseph H. MorrisonNASA Langley Research Center
Elements of Predictability WorkshopNovember 13-14, 2003
Johns Hopkins University
NASA Langley Research Center - 2Workshop on UQEE
Outline
• Breakdown of the problem (again) with a slight twist.
• The issue for most of aerospace is that non-computationalists are doing the applications computations.
• What are they doing now? What can we do to help?
NASA Langley Research Center - 3Workshop on UQEE
Breakdown of tasks
Off-lineComputation
Measuring themeasurement system
Measuring themeasurement system
Random errorcharacterizationusing standardartifacts
Discrimination testingof the measurementsystem
Discrimination testingof the measurementsystem
Systematic errorcharacterization
QA checks againstabove measurementsduring customertesting
QA checks againstabove measurementsduring customertesting
Process outputof interest
Calibration ofinstruments
Off-lineExperimentation
Traceableoperationaldefinition ofthe process
Calibration ofinstruments
Traceability tostandards
Verifying that thecoding is correct
Verifying that thecoding is correct
Off-line
Off-line
Measuring thecomputational process
Measuring thecomputational process
Off-lineCharacterizationof processvariation usingstandard problems
Model-to-model andmodel-to-realitydiscrimination
Model-to-model andmodel-to-realitydiscrimination
Off-lineSystematic errorcharacterization
QA checks againstabove measurementsduring computation forcustomer
QA checks againstabove measurementsduring computation forcustomer
Solutionverification
NASA Langley Research Center - 4Workshop on UQEE
The key question for applications:
“How is the applications person going to convince the decision maker that the computational process is good enough?”
NASA Langley Research Center - 5Workshop on UQEE
Our tentative answer based on observation of aero engineers trying to use CFD on real-life design problems is that it is the quantitative explanatory force of any approach that creates acceptance.
NASA Langley Research Center - 6Workshop on UQEE
• How can quantitative "explanatory force“ be provided?
• Breakdown to two questions:– How do I know that I am predicting the
right physics at the right place in the inference space?
– How accurate are my results if I do have the right physics at the right place in the inference space?
NASA Langley Research Center - 7Workshop on UQEE
Airfoil Stall Classification
NASA Langley Research Center - 8Workshop on UQEE
Boundaries Among Stall Types
NASA Langley Research Center - 9Workshop on UQEE
• The applications person needs a process that can be
ControlledEvaluatedImproved
(i.e. a predictable process)
NASA Langley Research Center - 10Workshop on UQEE
Creating a predictable process …
Controllable input(assignable cause variation)
Geometry,flight conditions,
etc.
Predictedcoefficients,
flow features,etc.
Process
Uncontrolled input from the environment(variation that we have to live with,
e.g. numerics, parameter uncertainty,model form uncertainty, users)
NASA Langley Research Center - 11Workshop on UQEE
Critical levels of attainment for a predictable process
• A defined set of steps
• Stable and replicable
• Measurable
• Improvable
NASA Langley Research Center - 12Workshop on UQEE
What it takes to have an impact ...
• Historically, practitioners have created their designs (and the disciplines they work in) with very little reference to researchers.
• Practitioners who are successfully using aero computations already know what it takes to convince a risk taker.
• If we want to have an impact on practitioners, we will have to build on what they are already doing.
NASA Langley Research Center - 13Workshop on UQEE
What is takes to have an impact ...
• Good questions:– Are researchers going to be an integral
part of the applications uncertainty quantification process or are we going to be irrelevant?
– What specific impact on practitioners do I want to have with a particular project?
– What process/product improvement am I expecting from that project?
NASA Langley Research Center - 14Workshop on UQEE
What is takes to have an impact ...
• We can greatly improve, systematize and generalize the process that practitioners are successfully using right now.
• The key watchwords for applications are:– practicality, as in mission analysis and design– alacrity, as in "I want to use it right now."– impact, as in "Will my customer buy in?" and "Am I
willing to bet my career (and my life) on my prediction?"
NASA Langley Research Center - 15Workshop on UQEE
Actions
• Establish working groups like the AIAA Drag Prediction Workshop (DPW)– Select a small number of focus problems– Use those problems
» to demonstrate the prediction uncertainty strategies » to find out just how tough this problem really is
• For right now …– Run multiple codes, different grid types, multiple models, etc.– Work data sets that fully capture the physics of the
application problem of interest.– Develop process best practices and find ways to control and
evaluate them.– Develop experiments to determine our ability to predict
uncertainty and to predict the domain boundaries where the physics changes.
NASA Langley Research Center - 16Workshop on UQEE
Breakout Questions/Issues
1. Defining predictability in the context of the application2. The logical or physical reasons for lack of predictability3. Possibility of isolating the reducible uncertainties in view
of dealing with them (either propagating them or reducing them)
4. The role of experimental evidence in understanding and controlling predictability
5. The possibility of gathering experimental evidence6. The role that modeling plays in limiting predictability7. Minimum requisite attributes of predictive models8. The role played by temporal and spatial scales and
possibilities mitigating actions and models