FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
A Solution Accurate, Efficient and Stable Unsplit Staggered Mesh MHD Solver in FLASH
Dongwook Lee
University of Chicago
The Flash Center for Computational Science
Outline
Split vs. unsplit formulations
Unsplit solvers in FLASH (UHD & USM) CFL stability (reduced or full?)
Reduced/Full corner-transport-upwind (CTU) for 3D
Divergence-free magnetic fields for USM-MHD constrained-transport (CT)
Verifications, convergence, performance Runtime parameters
Summary
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 1
Dimensionally Split vs. Unsplit???
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 1
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Single-mode Rayleigh-Taylor Instability Top figures:
Dimensionally split using PLM, PPM+old limiter, PPM+new limiter
high-wavenumber instabilities grow Bottom figures:
Dimensionally unsplit using PLM, PPM+old limiter, PPM+new limiter
high-wavenumber instabilities suppressed the split solvers experience high compressions and expansions in subsequent directional sweeps where there is a local high strain rate Almgren et al, ApJ, 715, 2010
Part 1
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Weakly magnetized 2D field loop Gardiner and Stone 2005 (JCP); Lee and Deane 2009 (JCP)
Part 1
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
8-wave split MHD scheme (Powell et al. 1999) at t=2.0
Unsplit staggered mesh MHD scheme (Lee and Deane, 2009) at t=2.0
Part 1
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
What is wrong with the split formulation for MHD? In the split formulation, you cannot correctly include terms proportional to
Gardiner and Stone (2005) Dynamics of in-plane magnetic fields in x and y directions are ruined from erroneous growth of magnetic field in z direction:
Part 2
Unsplit Hydro/MHD Solvers & Algorithms
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Hydro Unit in FLASH
Hydro_Unsplit
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Unsplit Staggered Mesh (USM) MHD Solver
Shock-capturing high-order Godunov Riemann solver (Lee & Deane, JCP, 2009; Lee 2012, to be submitted)
Finite volume method New data reconstruction-evolution algorithm for high-order accuracy Adaptive mesh refinement, uniform grid 1st order Godunov, 2nd order MUSCL-Hancock, 3rd order PPM, 5th Order WENO Approximate Riemann solvers: Roe, HLL, HLLC, HLLD, Marquina, modified
Marquina, Local Lax-Friedrichs Monotonicity preserving upwind PPM slope limiter for MHD (Lee, 2010,
Astronum) Divergence of magnetic fields is numerically controlled on a staggered grid,
using a constrained transport (CT) method (Evans & Hawley, 1998) Wide ranges of plasma flows Full Courant stability limit (CFL ~ 1 for 3D)
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Unsplit Formulations
Take a deep breath!
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
MHD Governing Equations
MHD system of equations:
This can be written in a simple matrix form:
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
MHD Governing Equations
Conservative variables and fluxes:
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
A primitive form:
where the coefficient matrix is
Linearized System
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Corner Transport Upwind (CTU)
Linear systemin 3D
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Corner Transport Upwind (CTU)
Linear systemin 3D
Normal predictor Transverse corrector
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Corner Transport Upwind (CTU)
Linear systemin 3D
Normal predictor Transverse corrector
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Traditional approach (Colella 1990; Saltzman 1994)
Characteristic tracing for the normal predictor
Subsequent calls to Riemann solvers for transverse corrector
Corner Transport Upwind (CTU)
Linear systemin 3D
Normal predictor Transverse corrector
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Traditional approach (Colella 1990; Saltzman 1994)
Characteristic tracing for the normal predictor
Subsequent calls to Riemann solvers for transverse corrector
New approach (Lee and Deane 2009):
Characteristic tracing for BOTH normal predictor and transverse corrector!
A primitive form:
where the coefficient matrix is
First consider the evolution in the x-normal direction and treat the normal magnetic field separately from the other variables:
Linearized System, cont’d
Normal predictor
MHD source termFLASH Workshop
Hamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Single-step data Reconstruction-evolution in USM
Normal Predictor
Characteristic Tracing
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Characteristic tracing for Transverse corrector
A jump relationship:
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Reduced 3D CTU in USM
Characteristic Tracing for
Normal Predictor
Transverse Corrector
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Full 3D CTU in USM
Characteristic Tracing for
Normal Predictor
Transverse Corrector
F u l l C T U d i a g o n a l c o u p l i n g FLASH Workshop
Hamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Summary of Part 1
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
New approach of using characteristic tracing for BOTH normal predictor and transverse corrector
Reduced 3D CTU A direct extension of 2D CTU to 3D Requires 3 Riemann solves for 3D (6-ctu needs 6 Riemann solves) Only including second cross derivatives CFL limit ~ 0.5
Full 3D CTU Full considerations of accounting for third cross derivatives Requires 3 Riemann solves for 3D (12-ctu needs 12 Riemann solves) CFL limit ~ 1.0 20% relative performance gain compared to reduced 3D CTU
Part 2
Divergence-Free fields:Constrained Transport (CT) MHD
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 2
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
CT scheme by Balsara and Spicer, 1998:
Part 2: recall…
Conservative variables and fluxes:
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 2
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
New upwind biased modified electric field construction(upwind-MEC), Lee 2012:
Part 2
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Small angle advection of the 2D field loop:
Part 2
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Small angle advection of the 3D field loop:
Summary of Part 2
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Three CT schemes were discussed: Standard CT scheme by Balsara and Spicer, 1998:
Takes a simple arithmetic averaging Lacks numerical diffusion for magnetic fields advection
Modified electric field construction (MEC) scheme by Lee and Deane, 2009: 3rd order accurate in space Not enough numerical diffusion for field advection
Upwind biased MEC (upwind-MEC) scheme by Lee, 2012 (to be submitted) Upwind scheme of MEC Added numerical diffusion to stabilize field advection
Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Verification, convergence, and performance
Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Summary of Part 3
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Verification tests for the reduced/full 3D CTU schemes:
CFL=0.95 for all 3D simulations using the full CTU scheme
CFL=0.475 for the reduced CTU scheme
They both converge in 2nd order
20% performance gain in using the full CTU scheme:
Various choices in runtime parameters
Conclusion
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Directionally split vs. unsplit formulations for hydro and MHD
Unsplit hydro/MHD solvers in FLASH4 (also FLASH3 in part) The reduced and full 3D CTU algorithms Upwind-MEC scheme for MHD Stable solutions with 2nd order convergence with CFL=0.95 20% performance gain in the full CTU scheme over the reduced CTU
scheme
Work in progress: Fully implicit Jacobian-Free Newton-Krylov implicit solver for the unsplit
solvers More HEDP capabilities for the USM solver
Thank You
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Questions?
New Upwind PPM for Slowly Moving Shock
Upwind PPM 5th order WENO
Standard PPM Standard PPM with increasing By
larger By
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
New Upwind PPM for Slowly Moving Shock
Upwind PPM 5th order WENO
Standard PPM Standard PPM with increasing By
Lee, 2010, 5th Astronum Proceeding;
Lee, 2011, in preparation
larger By
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Block and Mesh Packages
Uniform Grid AMR with variable patch size - CHOMBO
q Mesh package can be selected at configuration time
q The basic abstraction is a block of interior cells surrounded by guard cells
q Grid unit makes sure that blocks are self contained before being given to the solvers
Oct tree based AMR - PARAMESH
FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012
Top Related