CFD Analysis of Multielement Airfoils for Wind Turbines - Aerospace
Algorithm and Scaling (Issues) for Aerospace (CFD) Codes
description
Transcript of Algorithm and Scaling (Issues) for Aerospace (CFD) Codes
![Page 2: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/2.jpg)
2
Scope of Presentation Range of aerospace CFD and related
applications Hierarchy of simulation approaches Hierarchy of algorithmic approaches Algorithm and scalability issues and
considerations
![Page 3: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/3.jpg)
3
Presentation Approach & Goals A picture is worth a thousand words We will use ten thousand words and 1
picture == eleven thousand word-equivalents
Catalog, serve as collective conscience
Discuss relationship between application needs, algorithms, modeling approaches and HPC issues and possibilities
![Page 4: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/4.jpg)
CFD++ Aerospace Applications External aerodynamics Propulsion integration Component integration Systems Cabin airflow FADEC Icing Fuel tank purge Thrust reverser Propulsion Nozzle design Jet noise
![Page 5: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/5.jpg)
CFD++ Aerospace Applications Plumes Trajectory Aerodynamic coefficients Drag polar Dynamic derivatives Store separation Canopy separation Sabot separation Stage separation Pilot seat ejection Projectiles Spinning projectiles
![Page 6: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/6.jpg)
CFD++ Aerospace Applications Synthetic jets Turbomachinery Blade design Blade cooling Pulsed detonation Flapping wings Flexible wings Entomopters Helicopters Propellers, rotors Parachutes Parachutists, sky-diving
![Page 7: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/7.jpg)
CFD++ Aerospace Applications Spacecraft launch Reentry vehicles Rocket assisted landings (Earth, Mars, Venus) X-Prize vehicles Land speed record vehicles Bullets, artillery rounds Liquid fuel breakup Liquid fuel sloshing, feed Acceleration, deceleration effects Aeroacoustics Flow Structure Interaction (FSI)
![Page 8: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/8.jpg)
What’s special about Aerospace CFD?
Extremes of scales, operating conditions, physics and chemistry, speeds, application-specific needs (extraction of useful information)
Nonlinearity is most often inherent It is not just the simulation itself that
counts If there is no information output
required, no need to do the simulation
![Page 9: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/9.jpg)
Hierarchy of problem classes Steady state/unsteady problems Small, medium and large scale
problems Entire configurations as well as
analysis of components Engineering analysis, scientific
analysis, trouble shooting All speeds, atmospheric conditions,
diverse fluids and their properties
![Page 10: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/10.jpg)
10
Physics (nature)Math Model of Physics
Numerical Model of Math ModelComputational Model Human(s) in the loop
Simulation Results
Common Elements of Simulations
![Page 11: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/11.jpg)
Common Underlying Physical Processes
ijP
11
Convection:
Production:
Dissipation:
Redistribution:
Diffusion:
Evolution:tuu ji
k
kji
xuuu
.
ij
*ij
ijd
![Page 12: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/12.jpg)
12
Summary of some HPC issues Loading the problem, saving final results Checkpointing Computational vs. communications
performance (scalability) Data extraction issues Robustness (10000-way parallel should
be as robust as serial algorithm) Data-center issues (throughput, storage) Visualization, interaction with running
case
![Page 13: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/13.jpg)
13
Modeling Hierarchy Potential flow assumption Small-disturbance approaches Inviscid-flows taken separately, and
hybridized with boundary layer theory Reynolds/Favre-averaged N-S
equations with phenomenological turbulence models
LES and hybrid RANS-LES approaches Special equations and models
![Page 14: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/14.jpg)
14
Mesh possibilities Surface mesh only (panel methods) Cartesian mesh, almost Cartesian mesh Structured mesh – hex (3D) & quad (2D) Unstructured – all cell types Hybrid structured and unstructured
meshes, hex-core meshes Patched and overset meshes Moving (dynamic) meshes Flexible boundaries and meshes
![Page 15: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/15.jpg)
15
“Extreme Grids” Aspect ratios of 10000 to 1 or more
(boundary layer resolution with Y+ < 1)
Mesh sizes of hundreds of million and more
Extreme grid spacings present in mesh
![Page 16: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/16.jpg)
16
Numerical approaches Explicit and implicit Fractional steps and factored
schemes Finite volume, finite difference
schemes Finite element schemes Spectral and spectral element
schemes “Local” schemes and “global”
schemes
![Page 17: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/17.jpg)
17
Some HPC algorithmic challenges Challenges of making implicit schemes
be really implicit on multi-CPU computations
Ensure insensitivity of results to variations in number of parallel processes used
How to make the 10000-way parallel computation as robust as the serial algorithm
How to make the 10000-way parallel computation converge as well but in much less time
![Page 18: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/18.jpg)
18
Adaptive meshes Adaptive elements (cells) Adaptive grids H-adaptation, P-adaptation, H-P-
adaptation
![Page 19: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/19.jpg)
19
Classification of Algorithms Low information density schemes –
expand stencil to improve accuracy High information density schemes –
expand information content per cell (e.g. use values and derivatives, or values at multiple collocation points)
Homogeneity (or lack of) of discretization and solution methodology
Homogeneity (or lack of) underlying physics models
![Page 20: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/20.jpg)
20
The usual scalability considerations
Computation and communication Computation versus communication Overlap of computation and
communication Bulk of communication for local
schemes can follow pattern of one to a few connectivity
Global operations – global reductions often determine scalability
![Page 21: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/21.jpg)
21
Recent Scalability Improvements CFD++ now scales well to very large number of cores The scalability improvements are universal – they apply to all
modern HPC platforms from all vendors Tests have shown effective performance all the way up to 4096
cores Even relatively small grids (e.g. 16 million cells) scale well to
2048 or even 4096 cores, depending on computer and type of case run
Goal – to demonstrate similar performance on 10000 to 40000 cores
0 200 400 600 800 1000 12000123456
# of CPU cores
Scaling Perfor-mance
0 500 1000150020002500300035004000450005
10152025303540
# of CPU cores
Scaling Per-for-
mance
Ex 1: 33M cells, Computer 1, Case 1
Ex 2: 16M cells, Computer 2, Case 2
![Page 22: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/22.jpg)
22
Some Influences on Scalability Effect of physics – increased
sophistication means more computation, often more scalability
Effect of numerics – increased accuracy means more computation, and more communication, often more scalability
Effect of grid – more grid means more computation and less communication for “local” algorithms
![Page 23: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/23.jpg)
23
Additional thoughts on Parallel Processing
Two ways of using multiple compute engines Parallel computations Pipelined computations
Pipelined algorithms have not been exploited too much at the HPC level
Process level and thread level parallelism beginning to be combined (e.g. to exploit GPGPUs)
![Page 24: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/24.jpg)
24
Load balancing issues Structured vs. unstructured grids
(usually solved by weighted domain decomposition)
Adaptive algorithms and adaptive meshes
Different physics in different regions Moving meshes and overset meshes
![Page 25: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/25.jpg)
25
Optimization considerations Parallel algorithms for optimization How to use large numbers of
processors E.g. Do many cases in parallel Pre-compute cases matrix, sensitivity,
etc. and then train neural networks or tabulate sensitivity before applying optimization procedure
![Page 26: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/26.jpg)
26
Multi-physics considerations Communications between non-
homogeneous simulation tools Communications between diverse
hardware platforms Tight coupling vs. loose coupling
considerations
![Page 27: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/27.jpg)
27
Need for Parallel I/O and File systems
Very large scale problems Very large number of processors Initial load and final save +
intermediate data output Asymmetric data extraction needs
![Page 28: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/28.jpg)
28
Typical “post-processing” needs Global information (forces and
moments, lift, drag, torque) Semi-global information (forces and
moments along wing span, along fuselage)
Reduced subsets – iso-surfaces, surface data, cut-planes
Time-averages versus instantaneous values
In-situ “post”-processing can be very useful
![Page 29: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/29.jpg)
Single and Distributed File Parallel I/O
Parallel I/O (PIO) can be accomplished in two ways
In Single-File mode, PIO reads and writes from the current full-mesh/full-solution files.
In Distributed-File mode, PIO reads and writes from a set of files (e.g. placed in subdirectories) associated with each parallel process
29
![Page 30: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/30.jpg)
30
Interactive massively parallel computing
Steady state versus Transient (unsteady) computations
Links with front-end and graphical processing
Even post processing of large scale problems may require substantial parallel computing resources
One should not just focus on the “batch” computing model
![Page 31: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/31.jpg)
31
Some elements of the balancing act
Computation Communication Memory requirements I/O requirements Accuracy requirements Robustness requirements In-situ solution processing
requirements
![Page 32: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/32.jpg)
32
Bandwidths to consider Number of cores vs. number of I/O
channels Memory bandwidth from core to
memory Memory access conflicts
![Page 33: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/33.jpg)
33
Some old ideas revisited Paying more attention to connectivity
architecture Minimization of hops Domain decomposition that minimizes
traffic between switches How many switches or hops (groups
of nodes), how many nodes, how many processors in a node, how many cores per processor
![Page 34: Algorithm and Scaling (Issues) for Aerospace (CFD) Codes](https://reader034.fdocuments.us/reader034/viewer/2022051402/56816368550346895dd441cc/html5/thumbnails/34.jpg)
34
Final thoughts The challenge of producing codes that
work in the user’s hands and computing facilities
Ease of use Scalability and effectiveness vs. just
scalability Resource maximization versus
minimization What can be done with less What can be done with more What more can be done with less
Thank you