Recent Developments in Overture2010.oversetgridsymposium.org/assets/pdf/... · Ogmg: solves scalar...
Transcript of Recent Developments in Overture2010.oversetgridsymposium.org/assets/pdf/... · Ogmg: solves scalar...
Recent Developments in Overture
Bill Henshaw
Center for Applied Scientific ComputingLawrence Livermore National Laboratory
Livermore, CA
10th Symposium on Overset Composite Grids and Solution Technology,Nasa Ames Research Center, Moffett Field, CA
October 2010.
Henshaw (LLNL) Recent Developments in Overture OGS2010 1 / 25
Acknowledgments.
Supported byDepartment of Energy, Office of Science
ASCR Applied Math ProgramLLNL: Laboratory Directed Research and Development (LDRD) program
Current Overture developersKyle ChandBill Henshaw
Major ContributorsDon Schwendeman (RPI),Jeff Banks (LLNL).
Henshaw (LLNL) Recent Developments in Overture OGS2010 2 / 25
Overview of Recent Work
Solid mechanics on overlapping grids (talk by Jeff Banks).
Efficient high-order scheme for incompressible flows (talk by KyleChand).
Grid generation: large scale (billions of grid points), parallel, andmoving.
Multigrid solvers for overlapping grids (new high-order accurateand parallel algorithms),
Maxwell’s equations (multi-material electromagnetic problems in3D),
Fluid structure interactions (coupling high speed flow withdeforming solids).
Henshaw (LLNL) Recent Developments in Overture OGS2010 3 / 25
Overture: a toolkit for solving partial differentialequations (PDEs) on overlapping grids.
high level C++ interface for rapid prototyping of PDE solvers.
built upon optimized C and fortran kernels.
library of finite-difference operators: conservative andnon-conservative, 2nd, 4th, 6th and 8th order accurateapproximations.
support for moving grids.
support for block structured adaptive mesh refinement (AMR).
extensive grid generation capabilities.
CAD fixup tools (for CAD from IGES files).
interactive graphics and data base support (HDF).
Henshaw (LLNL) Recent Developments in Overture OGS2010 4 / 25
CG: Composite Grid PDE solvers built with Overture
Different PDE solvers in the CG suite:
cgad: advection diffusion equations.
cgins: incompressible Navier-Stokes with heat transfer.
cgcns: compressible Navier-Stokes, reactive Euler equations.
cgmp: multi-physics solver (e.g. conjugate heat transfer).
cgmx: time domain Maxwell’s equations solver.
cgsm: solid mechanics (*new*)
Henshaw (LLNL) Recent Developments in Overture OGS2010 5 / 25
Grid generation.
Two major steps:1 construct component grids (Mappings).2 grid connectivity: cut holes and determine
interpolation information (Ogen).
In recent work changes have been made to support
the generation of large (billion point +) grids.
parallel moving-grid flow simulations (Ogen iscalled at each time step).
Henshaw (LLNL) Recent Developments in Overture OGS2010 6 / 25
Overture Mappings (Part I)
Annulus Airfoil CrossSection Cylinder DataPoint
Depth Elliptic Fillet Hyperbolic Intersection
Join
Normal Nurbs OffsetShell Plane
Henshaw (LLNL) Recent Developments in Overture OGS2010 7 / 25
Overture Mappings (Part II)
Quadratic
Reparameterization
RevolutionRocket SmoothedPolygon
Sphere Spline Square, Box Stretch Sweep
TFI Trimmed Unstructured
CompositeSurface
Henshaw (LLNL) Recent Developments in Overture OGS2010 8 / 25
Ogen: overlapping grid generator (grid connectivity)
Brief description of the algorithm:
physical boundaries cut holes (“implicit hole-cutting”).
grids ordered by priority; interpolation preferred from higher priority grids.
robust algorithm with backup rules and interactive error diagnostics.
Brief description of capabilities:
arbitrary stencil widths (1,2,3... fringes),
arbitrary order of interpolation (linear, quadratic, cubic,...),
fast searching algorithms and fast “inverse” map.
inverse optimized for common mappings (spheres, cylinders, ...),
optimized for Cartesian grids,
script files with embedded perl commands for "automatic" parameterizedgrid generation.
Henshaw (LLNL) Recent Developments in Overture OGS2010 9 / 25
Ogen: two examples
Grid order of grid points processors cpu (s)accuracy (nodes× p/n)
Sphere in a box 2 2.1 billion 16 (8 × 2) 136Re-entry vehicle 4 215 million 128 (16 × 8) 1990
Significant performance improvements can still be made.
Henshaw (LLNL) Recent Developments in Overture OGS2010 10 / 25
Parallel Flow Solution on Moving Grids.
Flow past a pitching andplunging airfoil.
Demonstrates new parallelmoving grid capabilities.
Henshaw (LLNL) Recent Developments in Overture OGS2010 11 / 25
Multigrid: fast in theory and practice (if careful)
For elliptic problems, multigrid algorithms can have near optimalcomplexity requiring O(N) work to solve for N unknowns (c.f.conjugate gradient: O(N3/2)).
multigrid uses a sequence of coarser and coarser meshes toaccelerate the convergence rate on the finest grid.
For overlapping grids, coarse grid generation is a difficulty.
Henshaw (LLNL) Recent Developments in Overture OGS2010 12 / 25
Ogmg: Overlapping Grid Multigrid Solver
Ogmg: solves scalar elliptic boundary value problems.
automatic coarse grid generation of “any” number of levels.
adaptive smoothing
variable sub-smooths per component gridinterpolation-boundary smoothing (IBS)over-relaxed Red-Black smoothers
Galerkin coarse grid operators (operator averaging)
numerical boundary conditions for Dirichlet and Neumann problems
WDH., On Multigrid For Overlapping Grids, SIAM J. Sci. Comput. 26, no. 5, (2005) 1547–1572.
New capabilities under development:
fourth-order accuracy
parallel
Henshaw (LLNL) Recent Developments in Overture OGS2010 13 / 25
Ogmg: Automatic Coarse Grid Generation
overlap increases interpolation accuracy reduced
Henshaw (LLNL) Recent Developments in Overture OGS2010 14 / 25
Ogmg: Fourth order accuracy.
The equations are solved to fourth-order accuracy on the finestlevel but only second-order accuracy on lower levels.
Boundary conditions must be carefully formulated to avoiddegradation in convergence rates.
Neumann boundary conditions require particular care.
Henshaw (LLNL) Recent Developments in Overture OGS2010 15 / 25
Ogmg: Results: fourth order accuracy.
10−2
10−10
10−9
10−8
10−7
10−6
10−5
10−4
h
Max
imum
err
orCircle in a Channel, Errors versus h
Slope 4
DirichletNeumann
10−2
10−1
10−7
10−6
10−5
10−4
10−3
10−2
h
Max
imum
err
or
Sphere in a Box, Errors versus h
Slope 4
DirichletNeumann
Henshaw (LLNL) Recent Developments in Overture OGS2010 16 / 25
Ogmg: Convergence rates - Cartesian grids.
0 1 2 3 4 5 6 7 8
10−5
100
105
Square 10242, Order 4, V[2,1]
max
imum
res
idua
l
multigrid cycle
← CR=0.117, ECR=0.65
← CR=0.061, ECR=0.57
CR=0.018, ECR=0.46 →
ω=1ω=1.15Op Ave, ω=1.15
0 1 2 3 4 5 6 7 810
−8
10−6
10−4
10−2
100
102
104
106
Box 2563, Order 4, V[2,1]
max
imum
res
idua
l
multigrid cycle
← CR=0.188, ECR=0.70
← CR=0.118, ECR=0.63
CR=0.042, ECR=0.50 →
ω=1ω=1.15Op Ave, ω=1.15
Figure: Multigrid convergence rates for Poisson’s equation with Dirichletboundary conditions on a 10242 square and a 2563 box with a V[2,1] cycle.The rates are significantly improved using operator averaging andover-relaxed Red-Black smoothers with a relaxation parameter ω.
Henshaw (LLNL) Recent Developments in Overture OGS2010 17 / 25
Ogmg: Convergence rates - Overlapping grids.
0 2 4 6 8 10
10−5
100
105
Two Circles in a Channel, Order 4, V[2,1]
max
imum
res
idua
l
multigrid cycle
← CR=0.064, ECR=0.60
0 2 4 6 8 1010
−8
10−6
10−4
10−2
100
102
104
Sphere in a Box, Order 4, V[2,1]
max
imum
res
idua
lmultigrid cycle
← CR=0.246, ECR=0.74
CR=0.058, ECR=0.61 →
no IBSIBS
Figure: Left: two-cylinders-in-a-channel. Right: sphere-in-a-box
Henshaw (LLNL) Recent Developments in Overture OGS2010 18 / 25
Ogmg: Parallel issues.
Each grid is distributed across a set of processors.
Coarse grids may be distributed over different processors than fine.
Load balancing - there is less work on coarse grids.
Coarse grid generation, smoothers, and fine-coarse transfers mustaccount for the parallel distribution.
0 2 4 6 8 10
10−5
100
105
Circle in a Channel, Order 4, V[2,1]m
axim
um r
esid
ual
multigrid cycle
NP 1NP 2NP 4NP 8NP 16NP 32
Cylinder-in-a-channel, 6.6M pts, Convergence for different numbers of processors (preliminary).
Henshaw (LLNL) Recent Developments in Overture OGS2010 19 / 25
Cgmx: electromagnetics solver.
a time-domain finite difference scheme.
fourth-order accurate, 2D, 3D.
Efficient time-stepping with themodified-equation approach
High-order accurate symmetric differenceapproximations.
High-order-accurate centered boundary andinterface conditions.
• WDH., A High-Order Accurate Parallel Solver for Maxwell’s Equations on
Overlapping Grids, SIAM J. Scientific Computing, 28, no. 5, (2006).
Henshaw (LLNL) Recent Developments in Overture OGS2010 20 / 25
Scattering by a 3d material interface
EX Intensity Intensity
glass
air
Uses newly developed 4th-order accurate 3D material interfaceapproximations.
Scattering of a plane wave by an interface with a bump, glass-to-air.
1 billion grid points, 32 nodes (8 processors per node) of a Linux cluster.
Henshaw (LLNL) Recent Developments in Overture OGS2010 21 / 25
Cgmp: a multi-domain multi-physics solver.
Conjugate heat transfer: coupling incompressible flow to heatconduction in solids.
overlapping grids for each fluid or solid domain,
a partitioned solution algorithm (separate physicssolvers in each sub-domain),
(cgins) incompressible Navier-Stokes equations(with Boussinesq approximation) for fluid domains,
(cgad) heat equation for solid domains,
a key issue is interface coupling.
• WDH., K. K. Chand, A Composite Grid Solver for Conjugate Heat Transfer in
Fluid-Structure Systems, J. Comput. Phys, 2009.
Henshaw (LLNL) Recent Developments in Overture OGS2010 22 / 25
The multi-domain composite grid approach for CHT
Ω1fluid
Ω3solid
Ω3solid
Ω2fluid
Ω4solid
Ω5solid
Each fluid or solid sub-domain is covered by an overlapping grid.Fluid sub-domains : cgins. Solid sub-domains: cgad.Coupled problem: cgmp.
Henshaw (LLNL) Recent Developments in Overture OGS2010 23 / 25
Deforming composite grids for Fluid-StructureInteractions (FSI)
Goal: Couple overlapping grid techniques for modeling fluids andgases (using moving grids and AMR) with linear and non-linear solidmechanics codes.
A mixed Eulerian-Lagrangian approach:
Fluids: Overlapping grid fluid solver.
Solids : overlapping-grid (or unstructured) solid solver.
Boundary fitted deforming grids for fluid-solid interfaces.
Strengths of the approach:
Maintains high quality grids for large deformations/displacements.
efficient structured grid methods optimized for Cartesian grids.
Henshaw (LLNL) Recent Developments in Overture OGS2010 24 / 25
Cgmp: deforming composite grids for FSI
Current status:
Solve Euler equations in the fluid domains on moving grids.
Solve equations of linear elasticity in the solid domains.
Fluid grids at the interface deform over time.
Adaptive mesh refinement (in progress).
Gas
Solid
Mach 2 shock in a gas hitting two elastic cylinders.
Henshaw (LLNL) Recent Developments in Overture OGS2010 25 / 25