Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... ·...
Transcript of Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... ·...
![Page 1: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/1.jpg)
Progress and Challenges in Numerical Simulation of Multi-physics Turbulent Flows in
Aerospace Applications
Parviz MoinCenter for Turbulence Research
Royal Academy of Engineering, Spain
![Page 2: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/2.jpg)
(2003 Estimate)
A Story from the aircraft industry
• In 2003, Boeing estimated that the number of wing tests for 787 would be 5, representing a significant reduction from 11 a decade earlier.
• Estimates were based in large part on the increased use of simulation and enormous increase in compute resources during the decade 1995 to 2005 (~1000x)
![Page 3: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/3.jpg)
(2005 Data)
A Story from the aircraft industry
• By 2005, the actual number of wing tests required was 11, the same as a decade earlier
• Why? computer power was not the largest source of uncertainty in their predictions: it was model fidelity.
• High fidelity methods that incorporate more “first principles” are a path to predictive simulations because they can leverage the dramatic increase in compute power available
![Page 4: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/4.jpg)
Turbulence
Turbulence is the chaotic state of fluid motion that arises when the flow speed is higher than just the creeping motion
It is the rule, not the exception, in fluid dynamics
![Page 5: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/5.jpg)
Transition prediction: DNS, 1.2B, 32k Cores
Time Evolution at Fixed Position
![Page 6: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/6.jpg)
Methods for numerical simulation of turbulent flows
9/17/2014 6
Reynolds Averaged Navier Stokes (RANS). Average
over all turbulence scales.
Direct numerical simulation (DNS). Not practical for
applications.
Large Eddy Simulation (capture large eddies and
model small scales)
![Page 7: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/7.jpg)
Large Eddy Simulation of Turbulence
Resolve the large scale motions directly and “model” the effect of small scales
Turbulent fluctuations are obtained as part of the solution in LES; only small scale
phenomena (largely universal) are modeled
Useful for prediction of large scale mixing, stirring and engulfing, pressure
fluctuations, noise, distortion of electromagnetic waves, …
![Page 8: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/8.jpg)
Multi-physics Turbulent Flows Modeling Challenges
Many modeling and simulation challenges can benefit from a high-fidelity approach (LES):
• Compressible flow with shocks and complex mixing
• Laminar/turbulent flow transition
• Two Phase flow
• Combustion dynamics and coupled thermo-acoustics
• Integrated system issues, e.g. combustor/Turbine
Goal for this talk is to illustrate where we are in many of these areas, and where we are going in the near future
![Page 9: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/9.jpg)
Filtering
Computational Grid
u u
![Page 10: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/10.jpg)
10
Differential filtering on unstructured gridsGermano 1986a,b
Previous filtering approaches on unstructured grids built weighted sums from surrounding point cloud (Marsden et al 2002, Haselbacher & Vasilyev 2002)
Not robust on arbitrary meshes & complex to implementDifficult to decouple filtration from grid topology
Alternatively, introduce a linear operator whose Green’s function can be manipulated through a specification of a filter width
,
For constant p on an unbounded domain:
![Page 11: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/11.jpg)
Elements of Large Eddy Simulation (LES)
9/17/2014 11
Traditional components
• Filtering-- constitutive equations
• Subgrid scale modeling ---- Dynamic Modeling (1991)
• Wall modeling-------Slip wall (2012)
• Numerical Methods----Continued advances since 2010
New considerations
• Interlink among above components
• Computer science
• Multiphysics (Combustion, Multiphase, cavitation…)
Stand-alone research in anyone of these areas is not
going to have large engineering impact
![Page 12: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/12.jpg)
• It is important for LES calculations to predict accurately the quantities that led to choosing LES in the first place (e.g., turbulent fluctuations, acoustic sources, mixing,…).
• Numerical dissipation present in most codes, originally designed for RANS, is inadequate for LES
• Dispersion errors important for compressible flow and prediction of aerodynamic noise
• LES imposes additional requirements on mesh quality and size
Not all LES’s are equal: Numerical Methods
![Page 13: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/13.jpg)
Decision to go unstructured
Penalty on a per-node/cv basis (5x+), however:
Complex geometry (e.g. combustor + turbine stage)
Mesh Flexibility: adaptation and refinement
Massive parallelism
9/17/2014 13
![Page 14: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/14.jpg)
14
General Unstructured Meshes----Adaptation
Solvers designed to handle:
General polyhedral grids with hanging nodes
Complex geometries with body-fitted meshes
Local refinement in regions of interest
Grid-sensitive operators to reduce numerical dissipation
Grid-sensitive operatorsLocalized mesh refinement
![Page 15: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/15.jpg)
Directional-refinement capability in Charles
9/17/2014 15
Normal shock
due to
supersonic jet
impingement
In addition to handling complex
geometries, unstructured
directional adaptation also
supports complex physics by
focusing refinement exactly where
it is required:
Turbulent shear
layer
![Page 16: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/16.jpg)
Flow physics of high speed jet impingement (ideally-expanded)
169/17/2014
Note interaction
of shear layer
and normal
shock
![Page 17: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/17.jpg)
FWH approach for noise prediction from compressible flow solver
Acoustic computations: a challenging quantitative metric
17
end capsend caps
9/17/2014
Computation (ideally-
expanded)
Measurement
Predicted OASPL : 154 dB
Measured OASPL : 156 dB
9/17/2014
![Page 18: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/18.jpg)
18
Local refinement to address SGS model shortcomings
SGS model inaccurate at sufficiently coarse resolutionTypically, global refinement is performed to obtain more accurate resultsGoal: locally increase resolution in regions to reduce stress from SGS model
Can scale SGS model contributions by k* (measure of SGS fluctuation energy) Use statistics of k* to guide adaptation
Local refinement done automatically using adapt (Cascade Tech) Parallel, anisotropic mesh refinement
![Page 19: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/19.jpg)
19
k* adaptation for LES
Refine in regions where estimates of SGS fluctuations are largeNorm of SGS stress contribution scales with k*Refine grid (filter) in regions where k* > k*crit
k* is bounded from above:
Criteria will never refine in regions where Rkk ≤ 2k*crit
k* is O(∆f2) accurate to SGS kinetic energy
... versus gradient based refinement criteriaGradients large everywhere for turbulent flowsNo self-limiting mechanism; gradients continue to grow after refinement
![Page 20: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/20.jpg)
20
k* adaptation applied to a 3D diffuserdiffuser midplane (z/H = 1.67)
logscale k* contours, instantaneous
x/H
y/H
mean k* contours
Refinement concentrated in separated shear layers (bottom/side walls)Homothetic refinement of flagged cells Adapted mesh results in increase of 15% in total cell count
![Page 21: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/21.jpg)
21
Separated shear layer visualizationsdiffuser midplane (z/H = 1.67)
spanwise velocity contours Ux = 0 (blue)
spanwise velocity contours, coarse
LES
21
![Page 22: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/22.jpg)
22
Streamwise mean velocity profilesdiffuser midplane (x/H = 1.67), post-adaptation
k* adapted mesh addresses shortcomings of coarser LES Accurate prediction of mean velocity in separated shear layer
Continued discrepancy near upper wall due to weak sep shear layer
LES, post-adaptation Kolade expt (2010)coarse LES
![Page 23: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/23.jpg)
23
SGS-based mesh adaptationk* generalizations
Re = 8900 heated cylinder in crossflow
Mean Nusselt number
(Nakamura & Igarashi 2004)
Nested near wall mesh refinement using estimate of SGS temperature fluctuations
![Page 24: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/24.jpg)
Wall Modeling- A pacing item for LES
A turbulent boundary layer and the required LES grid resolution:
![Page 25: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/25.jpg)
Wall Model Implementation
Adopted strategy for general, unstructured-grid, massively
parallel solvers.
Additional cost of wall-model is only 6-7% thanks to dual-
constraint partitioning
![Page 26: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/26.jpg)
26
Wall modeled LESA filtering perspective
Cost of LES for wall bounded flows driven by resolution of small scales in the near wall region (Choi & Moin, 2012)
(grid point estimates for attached flow over an airfoil)
Wall resolved LES: filter width tends to 0 at the wall → resolves near-wall structuresresolved scales obey no-slip boundary conditions
Wall modeled LES: filter width finite at the wall → averages over the inner layerwhat BCs do the resolved scales satisfy?
![Page 27: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/27.jpg)
27
Differential filter based wall modeling: slip velocity BC
What happens if the near-wall filter width is not small?Expanding the differential filter expression at the wall:
Wall-resolved (no-slip)
Wall-modeled (slip BC)
![Page 28: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/28.jpg)
28
Slip BC formulation (Bose and Moin, 2014)vs traditional wall models
No explicit computation of wall stress; no embedded law of the wall
No empirical parameters from inner layer RANS modelNo zonal decomposition for hybrid RANS/LES techniques
No sensitivity to matching locations or ad hoc parametersNo a priori specification of transition locations or separation points
No sensors needed to deactivate wall model to revert to no-slip
).
![Page 29: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/29.jpg)
29
NACA 4412 at near-stall conditionsRec = 1.6 X 106, AoA = 13.86o
Dynamic slip BC model for non-trivial geometry with separation + varying resolutions
NASA LaRC turbulence modeling benchmark case (Rumsey et al, 2012) Compare with the expt of Wadcock (1978) (also Wadcock & Coles 1978)
Expt measures trailing edge separation x/c ≈ 0.85-0.86
Courtesy: NASA LaRC, Wilcox k-ω
Streamwise velocity
![Page 30: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/30.jpg)
30
NACA 4412 at near-stall conditionsRec = 1.6 X 106, AoA = 13.86o
Dynamic slip BC model for non-trivial geometry with separation + varying resolutions
NASA LaRC turbulence modeling benchmark case (Rumsey et al, 2012) Compare with the expt of Wadcock (1978) (also Wadcock & Coles 1978)
Expt measures trailing edge separation x/c ≈ 0.85-0.86
Courtesy: NASA LaRC, Wilcox k-ω
Streamwise velocity
x/c = 0.953
![Page 31: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/31.jpg)
31
NACA 4412: trailing edge separationWall-model streamwise slip velocity
No-slip achieved due to shear layer refinement
Unsteady trailing edge separation
Contours blanked for Us ≤ 0 indicating separation or no-slip
“Transition”
![Page 32: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/32.jpg)
32
NACA 4412: pressure coefficientRec = 1.6 X 106, AoA = 13.86o
WM-LES
Wadcock expt 1978
Overall agreement satisfactory with experimentCaptures Cp flattening due to separation at trailing edgeOverprediction of Cp during “transition” on suction side
![Page 33: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/33.jpg)
Subgrid scale modeling in two phase flow
• Common practice in CFD to inject distributions of Lagrangian drops to represent fuel spray
• Based heavily on empirical correlations and experimental data – not predictive
• Need to be able to simulate primary atomization of fuel with high-fidelity approaches
• Physics-based subgrid scale models of fuel breakup are required
Experiment (Marmottant et al.)
Numerical Simulation
![Page 34: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/34.jpg)
349/17/2014
![Page 35: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/35.jpg)
Physical Breakup Process: pinching-off
Experiment (Tjahjadi et al. JFM 1992)
35
Refined Level set Grid Method (Herrmann 2008)
• Capillary instability leads to formation of satellite drops
• Number and size of drops can be predicted using stability theory
![Page 36: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/36.jpg)
Physical Breakup Process: pinching-off
36
Experiment (Marmottant et al.)
• Ligaments undergo similar instability, pinching off to form small drops
![Page 37: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/37.jpg)
Subgrid scale modeling concept
Method proposed by Kim & Moin (2011):
1. Detect ligament using resolution criteria
2. Locally solve stability problem with interface geometry as initial condition
3. Replace ligament with drops in Lagrangian DPM
Experiment Coarse grid Fine Grid
![Page 38: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/38.jpg)
Subgrid scale model in action
• Subgrid droplet model in action for the coaxial liquid jet simulation
Lagrangian drops
Ligament just above detection threshold Ligament replaced by satellite drops
![Page 39: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/39.jpg)
Sub-grid scale model validation
• Quantitative comparison to measured droplet pdf
Experiment
(Marmottant et al. 2004)
Pd
f (1
/mm
)
Diameter (mm)
subgrid drops
D / D = 0.01
![Page 40: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/40.jpg)
Reacting Flow Challenges
Several competing approaches differing in cost, turbulence closure, complexity
of chemical mechanism, combustion regime,
Flamelet/Progress-Variable approach
• Assumes thin flame structure
• Tabulation of complex chemistry -> Reasonable cost
• Must be extended to include complex effects
• Autoignition, heat transfer, slow species, different regimes
PDF/FDF Transport approaches
• Accurate chemistry and turbulence closure, but costly
• Issues with mixing closure
Reduced Mechanisms
• Turbulence closure problem
Advocate for a balanced approach that doesn’t preference chemical fidelity over
flow fidelity, geometric fidelity
![Page 41: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/41.jpg)
41
Sandia Flame D
Mixture
Fraction TemperatureHeat
Release NOx
![Page 42: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/42.jpg)
Growth in Computing Power
7 years
100 times
![Page 43: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/43.jpg)
Closing the Efficiency Gap
Hejaz. et al. cavitation
simulations:
70% of peak
![Page 44: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/44.jpg)
Made possible by a critical mass of technology and people co-located at Stanford
through nearly two decades of DOE/NNSA support. Acquired the multi-disciplinary
culture needed to build, run and validate LES with realistic hydrocarbon chemistry, 2-
phase flow in complex geometries, noise, aero-optics, shape optimization, flow
control, hypersonics, …
![Page 45: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/45.jpg)
45
Hot Supersonic Over-Expanded Jet (Chevron Nozzle)
![Page 46: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/46.jpg)
46
Hot Supersonic Over-Expanded Jet (Chevron Nozzle)
![Page 47: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/47.jpg)
47
Hot Supersonic Over-Expanded Jet (Chevron Nozzle)
Noise Predictions (1 Million cores)
150 deg
Near-field noise prediction
Blind comparison with UTRC experiment
Far-field noise prediction
Mj =1.35
Ma = 1.84
Tj = 1.85
Rej = 130,000
mesh: 55M cells
Temperature
![Page 48: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/48.jpg)
Conclusions and Outlook
• Numerical methods and numerical analysis (e.g. stability
of multi-physics coupling) remain critical
• Grid generation for turbulent flows
• Conservative interface tracking on unstructured mesh
• Computer power increasing at 100x/7yrs but architectures
changing rapidly due to power constraints:
• challenges in programming these heterogeneous
systems efficiently (e.g. Liszt DSL)
• challenges associated with truly massive parallelism:
e.g. 1,000,000 cores
• Physics-based subgrid models will remain an important
element of LES of multiphysics engineering systems
![Page 49: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/49.jpg)
High-fidelity reacting turbulent flow simulation for simple geometries only
Multicode/Multiphysics integration non-existent
Parallel computations with modest (~64) number of processors
e.g. gas-phase combustion only
Computational Capabilities in 1997
64 processors!
![Page 50: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/50.jpg)
Computational Capabilities in 2009
Integrated Multiphysics/Multicode Simulations to support analysis and design
e.g. realistic aircraft engine simulations using multi-code coupling on 1000’s of processors
2000 processors
Code 1
Code 2
Code 3
![Page 51: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/51.jpg)
LES Cost Estimates
NASA Vision 2030 LES cost estimates are pessimistic
• 100x: Based on reported calculations and/or optimal resolution of unstable modes and/or unstructured mesh adaptation
• 10x: Most CFD codes operate at 3-5% peak performance; we can reach 30-50% with better memory movement and code analysis in partnership with our CS (e.g. DSLs)
At least a factor 1000 too conservative
A 5-Exaflop/s machine can design & optimize, not just produce a one-off calculation!
Multi-physics WMLES of combustors is used in the propulsion industry now
Engineering use of full-wing WMLES is feasible
![Page 52: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/52.jpg)
CharLES on Sequoia + Vulcan system
Leverage of the “Early Science
program” at LLNL (PSAAP)
Unique combination of
Vulcan+Sequoia
1.9 million cores
24.5 petaflops
![Page 53: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/53.jpg)
539/17/2014
![Page 54: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/54.jpg)
54
Large-scale scalability of explicitly filtered LES
Strong scalability of coupled duct + diffuser LES, 55M cvs
Intrepid, BG/P (ANL)
~75% eff. with <2K cvs/core
Scalability achieved with differential filters, locally refined meshes, new SGS models, and coupled simulationsPoisson solver scalability still largest bottleneck
Not subject to the EAR per 15 C.F.R. Chapter 1, Part 734.3(b)(3).
![Page 55: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/55.jpg)
55
Software infrastructureUnstructured solvers for engineering applications
Differential filters and SGS models implemented in CharLES
Allow locally refined meshes with pockets of hanging nodes 3D diffuser, NACA 4412 airfoil meshes locally refined
![Page 56: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/56.jpg)
• Control with synthetic jet actuator
• CDP’s unstructured grid capability
• Spanwise vorticity ( )
velocity BC
synthetic jet actuator
uncontrolled
controlled
Flow Separation Control
56
![Page 57: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/57.jpg)
uncontrolled
controlled
Uncontrolled Controlled
LES 0.83 1.43
EXP 0.82 1.41
Lift coefficientLines: LES
Symbols: Experiments (Gilarranz et al., JFE, ’05)
uncontrolled
controlled
Velocity in the wakeSurface pressure
Flow Separation Control
57
![Page 58: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/58.jpg)
Unstructured grid LES Solver: Charles
• Unstructured LES solver Charles
• Second-order in space (low numerical dissipation & dispersion)
• Anisotropic grid refinement
• Arbitrary polyhedral elements
• Dynamic subgrid-scale LES turbulence models
• Massively Parallel MPI-basedinfrastructure (run up to 1.9Mcores)
Scaling performance
BG/P Argonne National Laboratory
- Intrepid
![Page 59: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/59.jpg)
59
Streamwise velocity fluctuationsdiffuser midplane (x/H = 1.67), post-adaptation
Fluctuation prediction accurate for x/H < 10Centerline over-prediction for x/H > 12
possible causes: statistical under-sampling, fine → coarse resolution transition
LESKolade expt (2010)
![Page 60: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/60.jpg)
60
CHARLES : Scaling on LLNL Sequoia
Problem: LES of jet
crackle
Mesh : 650M CVs
Cores : 131K to 1.05M
4000 to 620 CVs/core
Performance
88% parallel efficiency
at 1.05M cores (relative
to 131K cores)
217 = 131K
219 = 524K
220 = 1.05K
![Page 61: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/61.jpg)
61
I/O Scaling
Competition between MPI ranks during I/O can hamper
scalability for I/O intensive problems
New, more flexible I/O model can lead to 6-8X speedup
![Page 62: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/62.jpg)
Visual evidence of Numerical Dissipation in LES
From Liu et al.
AIAA J. 2009,
MILES
Supersonic Jet LES using MILES-base method
Supersonic Jet LES using low-dissipation method (Charles)
![Page 63: Progress and Challenges in Numerical Simulation of Multi-physics …“N PROF. MOIN... · 2019-02-08 · • Numerical methods and numerical analysis (e.g. stability of multi-physics](https://reader033.fdocuments.us/reader033/viewer/2022050505/5f96891c5333f56ff81e8e34/html5/thumbnails/63.jpg)
Power – and the Exaflop machine in 2020
DOE planning to build an exaflop machine by 2020 that uses 20MW (dramatically reduced power/flop)
However, scaling of our problems is hard: e.g. for a factor of 2 in grid length scale, we need a factor of ~2^4=16 in computation power, or about 4 years
For a factor of 10 in length scale, need ~13 years
In the next decade:
physics-based sub-grid modeling will remain a critical part of high-fidelity simulations
Methods should carefully focus increased fidelity to beat these estimates (e.g. unstructured grids, fidelity of chemistry)
639/17/2014