ANon-HydrostaticDynamicalCoreintheHOMMEFramework[HOMAM] ·...

1
A Non-Hydrostatic Dynamical Core in the HOMME Framework [HOMAM] Ram D. Nair 1,* , Michael D. Toy 1 and Lei Bao 2 1 National Center for Atmospheric Research and 2 University of Colorado at Boulder Introduction The High-Order Method Modeling Environment (HOMME), developed at NCAR, is a petascale hydrostatic framework, which employs the cubed-sphere grid system and high-order continuous or discontinuous Galerkin (DG) methods. Recently, the HOMME framework is being extended to a non-hydrostatic (NH) dynamical core, the “High-Order Multiscale Atmospheric Model” (HOMAM). Orography is handled by the terrain-following height-based coordinate system. To alleviate the stringent CFL stability requirement resulting from the vertical aspects of the dynamics, an operator- splitting time integration scheme based on the horizontally explicit and vertically implicit (HEVI) philosophy is adopted for HOMAM. HOMAM: DG-NH Model Based on the conservation of mass, momentum and potential temperature, the NH atmospheric model is characterized by the classical compressible Euler System. In 3D generalized curvilinear coordinates (x j ), the governing equations are: ∂ρ ∂t + 1 G ∂x j ( G ρu j ) = 0 { Summation Implied} ∂ρu i ∂t + 1 G ∂x j [ G(ρu i u j + pG ij )] = -Γ i jk (ρu j u k + pG jk ) +f G(u 1 G 2i - u 2 G 1i ) - ρg G 3i ∂ρθ ∂t + 1 G ∂x j ( G ρθ u j ) = 0 ∂ρq ∂t + 1 G ∂x j ( G ρq u j ) = 0. Where ρ is the air density, θ potential temperature, u i is the contravariant wind field, f is the Coriolis parameter and q is the tracer field. G ij metric tensor, G = |G ij | 1/2 is the Jacobian of the transform, G ij =(G ij ) -1 , and i, j, k ∈{1, 2, 3}. Γ i jk is the Christoffel symbol (metric terms) associated with coordinate transform. Shallow atmosphere approximation (x 3 = r + z,z r ). The vertical grid system relies on height-based terrain-following (ζ ) coordinates (Gal-Chen & Sommerville). Computational Domain: Dimension-split (2D + 1D) approach allows to treat the 3D atmosphere as a vertically stacked cubed-sphere layers in the ζ direction [1]. k+1 k x y z Ω i,j,k DG discretization employs GLL spectral-element grid (HOMME grid system) in the horizontal and 1D Gauss-Legendre (GL) type grid in the vertical direction With the prognostic state-vector U h =( Gρ, Gρu 1 , Gρu 2 , Gρw, Gρθ ) T , the DG spatial discretization leads to U h ∂t + ∇· F(U h )= S(U h ) d dt U h = L(U h ) in (0,T ) Contact Information * Ram Nair: [email protected] Time Integration Schemes Dimensional (Operator) Splitting The spatial DG discretization is L(U h ) is decomposed into the horizontal L H and vertical L V parts such that L(U h )= L H (U h )+ L V (U h ). For HEVI: H -part is solved explicitly via third-order SSP-RK and V -part is solved implicitly via the Diagonally Implicit Runge-Kutta (DIRK). H - V - H " sequence of operations, only one implicit solver per time step. For the implicit solver: Inner linear solver uses Jacobian-Free GMRES and it usu- ally takes 1 or 2 iterations for the outer Newton solver The effective Courant number is only limited by the minimum horizontal grid- spacing [2]. Strang-type operator splitting permits Ot 2 ) accuracy in time. The HOMME hydrostatic model relies on explicit time-stepping with excellent petascale scalability. HOMAM may retain the parallel efficiency with ‘HEVI’ 3D Advection: DCMIP [4] Tests DCMIP-12: A 3D deformational advection test, which mimics “Hadley-like” meridional circulation (Kent et al. [3]), for 1 day simulation. HOMAM setup for 1 , L60: N e = 30, N p =4 (GLL); V nel = 15, N g =4 (GL), Δt = 60s. HEVI, HEVE, un-split results are almost identical. Vertical Levels (# of grid points) 40 50 60 70 80 90 100 110 120 L 2 error norm 10 -3 10 -2 10 -1 10 0 Hadley test-case: Normalized errors HEVI HEVE FULL O(2) O(3) Horiz. resolution (degree) 0.5 1 2 L 2 error norm 10 -3 10 -2 10 -1 10 0 Hadley test-case: Normalized errors HEVI HEVE FULL O(1) O(2) Errors/Models: Mcore CAM-FV ENDGame CAM-SE HOMAM 1 2.22 1.93 2.18 2.27 2.62 2 1.94 1.84 1.83 2.12 2.43 1.64 1.66 1.14 1.68 2.16 DCMIP-13: Flow Over a Rough Orography [3]. A series of steep concentric ring-shaped mountain ranges forms the terrain. Day = 0 Day = 6 Day = 12 Day = 0 Day = 6 Day = 12 z (km) ζ (km) z (km) z (km) ζ (km) ζ (km) Error MCore (ref) HOMAM Norm 1 L120 1 L120 1 0.83 0.78 2 0.55 0.50 0.73 0.76 HOMAM 1 L120 setup: N e = 30,N p =4 (GLL); V nel = 30,N g =4 (GL). Vertical cross-sections along the equator for the tracer fields. The results are sim- ulated with HOMAM using the HEVE/HEVI scheme at a resolution 1 horizontal with 60 vertical levels (Δt = 12s). Acknowledgement The National Center for Atmospheric Research is sponsored by the National Science Foundation. This work is partially supported by KIAPS, Seoul, South Korea. Non-Hydrostatic 3D Gravity Wave Test DCMIP-31: NH gravity wave test on a reduced planet (X = 125). θ 0 after 3600s. HOMAM setup: N e = 25,N p =4,N g =4 (Δx Δz 1 km), Δt =0.20s The initial state is hydrostatically balanced and in gradient-wind balance. An overlaid potential temperature perturbation triggers the evolution of gravity waves. ICON-IAP ENDGame Reference Schär Mountain Test (2D) Schär mountain test with Higher ele- vation (slope 55%) w after 1800 s, with dt =0.175s, with p 3 -DG (GL) [2] Virtually identical results with explicit RK3 and HEVI Work In Progress: More benchmark tests including DCMIP IMEX time integration method. Design of a proper preconditioner for accelerating implicit time solver. Efficient explicit time solver for horizontal dynamics (sub-cycling). References [1] R. D. Nair, L. Bao, and M. D. Toy. A time-split discontinuous Galerkin transport scheme for global atmopsheric model. Procedia Computer Science, XXX:1–10, 2015. [2] L. Bao, R. Klöfkorn, and R. D. Nair. Horizontally explicit and vertically implicit (HEVI) time discretization scheme for a discontinuous Galerkin nonhydrostatic model. Mon. Wea. Rev., 2015. DOI:10.1175/MWR-D-14-00083.1. [3] J. Kent, P. A. Ullrich, and C. Jablonowski. Dynamical core model intercomparison project: Tracer transport test cases. Q. J. Roy. Meteol. Soc., 140:1279–1293, 2014. [4] C. Jablonowski, P. A. Ullrich, J. Kent, K. A. Reed, M. A. Taylor, P. H. Lauritzen, and R. D. Nair. https://www.earthsystemcog.org/ projects/dcmip-2012/. The 2012 Dynamical Core Model Inter- comparison Project, 2012.

Transcript of ANon-HydrostaticDynamicalCoreintheHOMMEFramework[HOMAM] ·...

Page 1: ANon-HydrostaticDynamicalCoreintheHOMMEFramework[HOMAM] · ANon-HydrostaticDynamicalCoreintheHOMMEFramework[HOMAM] Ram D. Nair1;, Michael D. Toy1 and Lei Bao2 1National Center for

ANon-HydrostaticDynamicalCore in theHOMMEFramework [HOMAM]Ram D. Nair1,∗, Michael D. Toy1 and Lei Bao2

1National Center for Atmospheric Research and 2University of Colorado at Boulder

IntroductionThe High-Order Method Modeling Environment (HOMME), developed at NCAR,

is a petascale hydrostatic framework, which employs the cubed-sphere grid systemand high-order continuous or discontinuous Galerkin (DG) methods. Recently, theHOMME framework is being extended to a non-hydrostatic (NH) dynamical core, the“High-Order Multiscale Atmospheric Model” (HOMAM). Orography is handled bythe terrain-following height-based coordinate system. To alleviate the stringent CFLstability requirement resulting from the vertical aspects of the dynamics, an operator-splitting time integration scheme based on the horizontally explicit and verticallyimplicit (HEVI) philosophy is adopted for HOMAM.

HOMAM: DG-NH ModelBased on the conservation of mass, momentum and potential temperature, the NHatmospheric model is characterized by the classical compressible Euler System. In3D generalized curvilinear coordinates (xj), the governing equations are:

∂ρ

∂t+

1√G

[∂

∂xj(√Gρuj)

]= 0 { Summation Implied}

∂ρui

∂t+

1√G

[∂

∂xj[√G(ρuiuj + pGij)]

]= −Γi

jk(ρujuk + pGjk)

+f√G(u1G2i − u2G1i)− ρg G3i

∂ρθ

∂t+

1√G

[∂

∂xj(√Gρθ uj)

]= 0

∂ρq

∂t+

1√G

[∂

∂xj(√Gρq uj)

]= 0.

Where ρ is the air density, θ potential temperature, ui is the contravariant wind field,f is the Coriolis parameter and q is the tracer field. Gij metric tensor,

√G = |Gij |1/2

is the Jacobian of the transform, Gij = (Gij)−1, and i, j, k ∈ {1, 2, 3}. Γi

jk is theChristoffel symbol (metric terms) associated with coordinate transform.

• Shallow atmosphere approximation (x3 = r + z, z � r). The vertical grid systemrelies on height-based terrain-following (ζ) coordinates (Gal-Chen & Sommerville).

• Computational Domain: Dimension-split (2D + 1D) approach allows to treatthe 3D atmosphere as a vertically stacked cubed-sphere layers in the ζ direction[1].

k+1

kxy

z

Ωi,j,k

• DG discretization employs GLL spectral-element grid (HOMME grid system)in the horizontal and 1D Gauss-Legendre (GL) type grid in the vertical direction

• With the prognostic state-vector Uh = (√Gρ,√Gρu1,

√Gρu2,

√Gρw,

√Gρθ)T ,

the DG spatial discretization leads to

∂Uh

∂t+∇ · F(Uh) = S(Uh) ⇒ d

dtUh = L(Uh) in (0, T )

Contact Information∗Ram Nair: [email protected]

Time Integration Schemes• Dimensional (Operator) Splitting

The spatial DG discretization is L(Uh) is decomposed into the horizontal LH andvertical LV parts such that L(Uh) = LH(Uh) + LV (Uh).

• For HEVI: H-part is solved explicitly via third-order SSP-RK and V -part issolved implicitly via the Diagonally Implicit Runge-Kutta (DIRK).

• “H − V −H" sequence of operations, only one implicit solver per time step.• For the implicit solver: Inner linear solver uses Jacobian-Free GMRES and it usu-

ally takes 1 or 2 iterations for the outer Newton solver• The effective Courant number is only limited by the minimum horizontal grid-

spacing [2]. Strang-type operator splitting permits O(∆t2) accuracy in time.• The HOMME hydrostatic model relies on explicit time-stepping with excellent

petascale scalability. HOMAM may retain the parallel efficiency with ‘HEVI’

3D Advection: DCMIP [4] Tests• DCMIP-12: A 3D deformational advection test, which mimics “Hadley-like”

meridional circulation (Kent et al. [3]), for 1 day simulation.• HOMAM setup for 1◦, L60: Ne = 30, Np = 4 (GLL); Vnel = 15, Ng = 4 (GL),

∆t = 60s. HEVI, HEVE, un-split results are almost identical.

Vertical Levels (# of grid points)40 50 60 70 80 90 100 110 120

L2 e

rror

norm

10-3

10-2

10-1

100Hadley test-case: Normalized errors

HEVIHEVEFULLO(2)O(3)

Horiz. resolution (degree)0.512

L2 e

rror

norm

10-3

10-2

10-1

100Hadley test-case: Normalized errors

HEVIHEVEFULLO(1)O(2)

Errors/Models: Mcore CAM-FV ENDGame CAM-SE HOMAM`1 2.22 1.93 2.18 2.27 2.62`2 1.94 1.84 1.83 2.12 2.43`∞ 1.64 1.66 1.14 1.68 2.16

• DCMIP-13: Flow Over a Rough Orography [3].A series of steep concentric ring-shaped mountain ranges forms the terrain.

Day = 0

Day = 6

Day = 12

Day = 0

Day = 6

Day = 12

z (k

m)

ζ (k

m)

z (k

m)

z (k

m)

ζ (k

m)

ζ (k

m)

Error MCore (ref) HOMAMNorm 1◦L120 1◦L120`1 0.83 0.78`2 0.55 0.50`∞ 0.73 0.76

• HOMAM 1◦L120 setup:Ne = 30, Np = 4 (GLL); Vnel =30, Ng = 4 (GL).

• Vertical cross-sections alongthe equator for the tracerfields. The results are sim-ulated with HOMAM usingthe HEVE/HEVI scheme at aresolution 1◦ horizontal with 60vertical levels (∆t = 12s).

AcknowledgementThe National Center for Atmospheric Research is sponsored by the National ScienceFoundation. This work is partially supported by KIAPS, Seoul, South Korea.

Non-Hydrostatic 3D Gravity Wave Test• DCMIP-31: NH gravity wave test on a reduced planet (X = 125). θ′ after

3600s. HOMAM setup: Ne = 25, Np = 4, Ng = 4 (∆x ≈ ∆z ≈ 1 km),∆t = 0.20s• The initial state is hydrostatically balanced and in gradient-wind balance. An

overlaid potential temperature perturbation triggers the evolution of gravitywaves.

ICON-IAP

ENDGame

Reference

Schär Mountain Test (2D)• Schär mountain test with Higher ele-

vation (slope ≈ 55%)

• w after 1800 s, with dt = 0.175s, withp3-DG (GL) [2]• Virtually identical results with explicit

RK3 and HEVI

• Work In Progress: More benchmark tests including DCMIP• IMEX time integration method. Design of a proper preconditioner for accelerating

implicit time solver.• Efficient explicit time solver for horizontal dynamics (sub-cycling).

References[1] R. D. Nair, L. Bao, and M. D. Toy. A time-split discontinuous Galerkin transport scheme for global

atmopsheric model. Procedia Computer Science, XXX:1–10, 2015.[2] L. Bao, R. Klöfkorn, and R. D. Nair. Horizontally explicit and vertically implicit (HEVI) time

discretization scheme for a discontinuous Galerkin nonhydrostatic model. Mon. Wea. Rev., 2015.DOI:10.1175/MWR-D-14-00083.1.

[3] J. Kent, P. A. Ullrich, and C. Jablonowski. Dynamical core model intercomparison project: Tracertransport test cases. Q. J. Roy. Meteol. Soc., 140:1279–1293, 2014.

[4] C. Jablonowski, P. A. Ullrich, J. Kent, K. A. Reed, M. A. Taylor, P. H. Lauritzen, and R. D. Nair.https://www.earthsystemcog.org/ projects/dcmip-2012/. The 2012 Dynamical Core Model Inter-comparison Project, 2012.