The Structure of the FV3 Solver - gfdl.noaa.gov · possible •Finite-volume, integrated form of...
Transcript of The Structure of the FV3 Solver - gfdl.noaa.gov · possible •Finite-volume, integrated form of...
TheStructureoftheFV3 Solver
LucasHarris
andtheGFDLFV3 Team
FV3WorkshopinTaiwan
4 December2017
FV3 DesignPhilosophy
• Discretizationshouldbeguidedbyphysicalprinciplesasmuchas
possible
• Finite-volume,integratedformofconservationlaws
• Upstream-biasedfluxes
• Operators“reverseengineered”toachievedesiredproperties• Computationalefficiencyiscrucial.Fastmodelscanbegoodmodels!
• Solvershouldbebuiltwithvectorizationandparallelisminmind
• Dynamicsisn’tthewholestory!Couplingtophysicsandtheoceanis
important.
Finite-volumemethodology
• InFV3,allvariablesare3Dcell- orface-means…notgridpoint values
• WesolvenotthedifferentialEulerequationsbuttheircell-integrated
formsusingintegraltheorems
• Everythingisaflux,includingthemomentumequation
• Massconservationisensured,toroundingerror
• C-Dgrid:Vorticitycomputedexactly;accuratedivergencecomputation
• Mimetic:Physicalpropertiesrecoveredbydiscretization,particularly
Newton’s3rd law
OutlineoftheFV3 solver
AbstractofFV3
• Gnomoniccubed-spheregridforscalabilityanduniformity
• Fully-compressiblevector-invariantEulerequations
• Vertically-Lagrangian dynamics
• C-Dgriddiscretization• Forward-in-time2DLin-RoodadvectionusingPPMoperators
• Fullynonhydrostatic withsemi-implicitsolver
• Runtimehydrostaticswitch
FV3 timeintegrationsequence
• FV3 isaforward-in-timesolver
• Flux-divergencetermsandphysicstendenciesevaluatedforward-in-time
• Pressure-gradientandsound-wavetermsevaluatedbackward-in-timefor
stability
• Lagrangian verticalcoordinate:flowconstrainedalongtime-evolving
Lagrangian surfaces
• Tracersaresub-cycled sincetheirstabilityconditionismuchless
restrictivethanthesound- orgravity-wavemodes
fv_dynamics()
FV3 solver
dyn_core()
Lagrangian dynamics
fv_tracer2d()
Sub-cycledtracertransport
OpenMP onk
Lagrangian_to_Eulerian()
VerticalRemapping
(i,k)OpenMP onj
c_sw(),etc.
C-gridsolver
d_sw()
ForwardLagrangian dyn.
OpenMP onk
update_dz_d()
Forwardδz evaluation
OpenMP onk
one_grad_p()/nh_p_grad()
BackwardshorizontalPGF
OpenMP onk
riem_solver()
BackwardsverticalPGF,
soundwaveprocesses
(i,k)OpenMP onj
[physics]
fv_update_phys()
Consistentfieldupdate
fv_dynamics()
FV3 solver
dyn_core()
Lagrangian dynamics
fv_tracer2d()
Sub-cycledtracertransport
OpenMP onk
Lagrangian_to_Eulerian()
VerticalRemapping
(i,k)OpenMP onj
c_sw(),etc.
C-gridsolver
d_sw()
ForwardLagrangian dyn.
OpenMP onk
update_dz_d()
Forwardδz evaluation
OpenMP onk
one_grad_p()/nh_p_grad()
BackwardshorizontalPGF
OpenMP onk
riem_solver()
BackwardsverticalPGF,
soundwaveprocesses
(i,k)OpenMP onj
[physics]
fv_update_phys()
Consistentfieldupdate
dt_atmosk_split
“remapping”loop
n_split
“acoustic”loop
PrognosticVariablesδp Totalairmass(includingvaporandcondensates)
Equal to hydrostatic pressuredepthoflayerθv Virtualpotentialtemperature
u,v HorizontalD-gridwindsinlocalcoordinate
(defined oncellfaces)
w Verticalwinds
δz Geometriclayerdepth
qi Passive tracers
Cell-meanpressure,density,divergence,andspecificheatarealldiagnostic quantitiesAllvariablesarelayer-meansinthevertical:Noverticalstaggering
Lagrangian DynamicsinFV3
Whattheyare
Whattheequationsare
Howtheyaresolved
WhatareLagrangian Dynamics?
• TheEulerequationscanbewritteninLagrangian orEulerianforms…orEulerianinthehorizontal,andLagrangian inthevertical
• Thisconstrainstheflowalongquasi-horizontalsurfaces
• Surfacesdeformduringtheintegration,representingverticalmotionandadvection“forfree”
• Doesrequirelayerthicknesstobeaprognosticvariable
5.2. Dependent variables and governing equations
Variable Description�p
⇤ Vertical difference in hydrostatic pressure, proportional to massu D-grid face-mean horizontal x-direction windv D-grid face-mean horizontal y-direction wind
⇥
v
Cell-mean virtual potential temperaturew Cell-mean vertical velocity�z Geometric layer height
Table 5.1: Solution variables in FV3
The continuous Lagrangian equations of motion, in a layer of finite depth�z and mass �p
⇤, are then given as
D
L
�p
⇤+ r · (V�p
⇤) = 0 (5.3)
D
L
�p
⇤⇥
v
+ r · (V�p
⇤⇥
v
) = 0
D
L
�p
⇤w + r · (V�p
⇤w) = �g�z
@p
0
@z
D
L
u = ⌦v � @
@x
� 1
⇢
@p
⇤
@x
���z
D
L
v = �⌦u � @
@y
� 1
⇢
@p
⇤
@x
���z
The operator D
L
is the “vertically-Lagrangian” derivative, formally equalto @�
@t
+
@
@z
(w�) for an arbitrary scalar �. The flow is entirely along theLagrangian surfaces, including the vertical motion (which deforms thesurfaces as appropriate, an effect included in the semi-implicit solver).The vertical component of absolute vorticity is given as ⌦ and the ki-netic energy is given as =
12(euu + evv), and p is the full nonhydrostatic
pressure. The nonhydrostatic pressure gradient term in the w equationis computed by the semi-implicit solver described Chapter 7. There is noprojection of the vertical pressure gradient force into the horizontal; sim-ilarly, there is no projection of the horizontal winds u, v into the vertical,despite the slopes of the Lagrangian surfaces.
The vertically-Lagrangian discretization requires expressions for ei-ther the mass or for z, as only one can be used as an independent variable;
33
DL� =@�
@t+
@
@z(w�)
Lagrangian Dynamics:Flux-formadvection
• Advectionisnotjustforpassivetracers!NearlyeverythinginFV3 isaflux
• Eventhemomentumadvectiontermscanbeexpressedasscalarfluxes
• Lin-Rood(1996)2Dadvectionscheme:
• Reverse-engineeredschemedevisedfrom1DPPMoperators
• Allowsmonotonicityandpositivityfromsubgrid reconstructions
• Monotonicityis“smart”diffusion
• Again:advectionisalongLagrangian surfaces
Lagrangian Dynamics:Traceradvectionandsub-cycling• Tracerscanbeadvected withalongertimestep thanthedynamics
• FV3 permitsaccumulationofmassfluxesduringLagrangian dynamics.
Thesefluxesarethenusedtocomputetheadvectionoftracers
beforetheverticalremapping
• Typicallyoneortwotracertimesteps issufficientforstability.
• Traceradvectionisalwaysmonotone(oratleastpositivedefinite)to
avoidnewextrema.Explicitdiffusionisnotused.
• Question: whataboutphysicaleddydiffusion(cf.PBLscheme?)
Lagrangian Dynamics:C-Dgridsolver• FV3 solvesforthe(purelyhorizontal)
D-gridstaggeredwinds.Butsolver
requiresface-normalfluxes
• Tocomputetimestep-meanfluxes,the
C-gridwindsareinterpolatedandthenadvancedahalf-timestep.
• AsortofsimplifiedRiemannsolver
• TheC-gridsolveristhesameastheD-grid,butuseslower-orderfluxesfor
efficiency
• Two-griddiscretizationandtime-centeredfluxesavoidcomputational
modes
�
D-grid winds
C-grid winds
Fluxes
Fig. 2. Geometry of the wind staggerings and fluxes for a cell on a non-orthogonal grid.The angle � is that between the covariant and contravariant components; in orthogonalcoordinates � = ⇥/2.
45
Lagrangian Dynamics:Momentumequation• FV3solvestheflux-formvectorinvariant
equations
• Nonlinearvorticityfluxterminmomentum
equation,confoundinglinearanalyses
• D-gridallowsexactcomputationof
absolutevorticity—noaveraging!
• Vorticityusessamefluxasδp:consistency
improvesgeostrophicbalance,andSW-PV
advected asascalar!
2. The Nested Grid Model126
a. Finite-Volume Dynamical Core and cubed-sphere grid127
The FV core is a hydrostatic, 3D dynamical core using the vertically-Lagrangian dis-128
cretization of L04 and the horizontal discretization of Lin and Rood (1996, 1997, hence-129
forth LR96 and LR97, respectively), using the cubed-sphere geometry of PL07 and Putman130
(2007). This solver discretizes a hydrostatic atmosphere into a number of vertical layers, each131
of which is then integrated by treating the pressure thickness and potential temperature as132
scalars. Each layer is advanced independently, except that the pressure gradient force is133
computed using the geopotential and the pressure at the interface of each layer (Lin 1997).134
The interface geopotential is the cumulative sum of the thickness of each underlying layer,135
counted from the surface elevation upwards, and the interface pressure is the cumulative136
sum of the pressure thickness of each overlying layer, counted from the constant-pressure137
top of the model domain downward. Vertical transport occurs implicitly from horizontal138
transport along Lagrangian surfaces. The layers are allowed to deform freely during the139
horizontal integration. To prevent the layers from becoming infinitesimally thin, and to ver-140
tically re-distribute mass, momentum, and energy, the layers are periodically remapped to141
a pre-defined Eulerian coordinate system.142
The governing equations in each horizontal layer are the vector-invariant equations:143
⇧�p
⇧t+⌅ · (V�p) = 0144
⇧�p�
⇧t+⌅ · (V�p�) = 0145
146
⇧V
⇧t= �⇥k̂ ⇤V �⌅
�⇥ + ⇤⌅2D
⇥� 1
⌅⌅p
⇤⇤⇤z
147
where the prognostic variables are the hydrostatic pressure thickness �p of a layer bounded148
by two adjacent Lagrangian surfaces, which is proportional to the mass of the layer; the149
potential temperature �; and the vector wind V. Here, k̂ is the vertical unit vector. The150
6
Lagrangian Dynamics:Momentumequation• FV3solvestheflux-formvectorinvariant
equations
• Nonlinearvorticityfluxterminmomentum
equation,confoundinglinearanalyses
• D-gridallowsexactcomputationof
absolutevorticity—noaveraging!
• Vorticityusessamefluxasw:consistencyimprovesnonlinearbalance,andupdraft helicityadvectedasascalar!
2. The Nested Grid Model126
a. Finite-Volume Dynamical Core and cubed-sphere grid127
The FV core is a hydrostatic, 3D dynamical core using the vertically-Lagrangian dis-128
cretization of L04 and the horizontal discretization of Lin and Rood (1996, 1997, hence-129
forth LR96 and LR97, respectively), using the cubed-sphere geometry of PL07 and Putman130
(2007). This solver discretizes a hydrostatic atmosphere into a number of vertical layers, each131
of which is then integrated by treating the pressure thickness and potential temperature as132
scalars. Each layer is advanced independently, except that the pressure gradient force is133
computed using the geopotential and the pressure at the interface of each layer (Lin 1997).134
The interface geopotential is the cumulative sum of the thickness of each underlying layer,135
counted from the surface elevation upwards, and the interface pressure is the cumulative136
sum of the pressure thickness of each overlying layer, counted from the constant-pressure137
top of the model domain downward. Vertical transport occurs implicitly from horizontal138
transport along Lagrangian surfaces. The layers are allowed to deform freely during the139
horizontal integration. To prevent the layers from becoming infinitesimally thin, and to ver-140
tically re-distribute mass, momentum, and energy, the layers are periodically remapped to141
a pre-defined Eulerian coordinate system.142
The governing equations in each horizontal layer are the vector-invariant equations:143
⇧�p
⇧t+⌅ · (V�p) = 0144
⇧�p�
⇧t+⌅ · (V�p�) = 0145
146
⇧V
⇧t= �⇥k̂ ⇤V �⌅
�⇥ + ⇤⌅2D
⇥� 1
⌅⌅p
⇤⇤⇤z
147
where the prognostic variables are the hydrostatic pressure thickness �p of a layer bounded148
by two adjacent Lagrangian surfaces, which is proportional to the mass of the layer; the149
potential temperature �; and the vector wind V. Here, k̂ is the vertical unit vector. The150
6
Manyflowsarevortical
Notjustlarge-scaleflows
NASAGoddard
MayItalktoyouabout…diffusion?
• Alldynamicalcoresrequireartificialdiffusion(implicitorexplicit)
toremoveenergycascadingtogridscale
• FV3 hasimplicitdiffusionfrommonotoneadvection,forward-
backwardtimestepping,andverticalremapping
• Exceptforverticalremapping…allexplicitandimplicitdiffusionis
alongtheLagrangian surfaces
MayItalktoyouabout…diffusion?
• C-Dgriddoesnotexplicitlyseedivergence—soexplicitdivergence
dampingisanintrinsicpartofthesolver
• Non-monotoneadvectionisusefulinnonhydrostatic dynamics
Noisecanbecontrolledbyaddingdiffusiontothevorticityfluxes
• Explicitly-dissipatedkineticenergycanbeconvertedtoheat• Divergenceandvorticitydampingcanbecontrolledseparately
• Asimplifiedsecond-orderSmagorinsky damping,withanonlinear
flow-dependentcoefficient,isalsoavailable
Backwardhorizontalpressuregradientforce• ComputedfromNewton’ssecondandthirdlaws,andGreen’sTheorem
• Errorslower,withmuchlessnoise,comparedtotraditionalevaluations
• Purelyhorizontalno along-coordinateprojection
• PGFequalandopposite—3rd law!Momentumconserved
• Curl-freeintheabsenceofdensitygradients• Nonhydrostatic andhydrostaticcomponentscanbecomputedseparately
• log(phyd)PGFmoreaccurate
Backwardhorizontalpressuregradientforce• ComputedfromNewton’ssecondandthirdlaws,andGreen’sTheorem
• Errorslower,withmuchlessnoise,comparedtotraditionalevaluations
• Purelyhorizontalno along-coordinateprojection
• PGFequalandopposite—3rd law!Momentumconserved
• Curl-freeintheabsenceofdensitygradients• Nonhydrostatic andhydrostaticcomponentscanbecomputedseparately
• log(phyd)PGFmoreaccurate
Backwardhorizontalpressuregradientforce• ComputedfromNewton’ssecondandthirdlaws,andGreen’sTheorem
• Errorslower,withmuchlessnoise,comparedtotraditionalevaluations
• Purelyhorizontalno along-coordinateprojection
• PGFequalandopposite—3rd law!Momentumconserved
• Curl-freeintheabsenceofdensitygradients• Nonhydrostatic andhydrostaticcomponentscanbecomputedseparately
• log(phyd)PGFmoreaccurate
Verticalprocesses:VerticalelastictermsandLagrangian verticalcoordinate
Semi-implicitsolver
• Verticalpressuregradientandlayerdepthδzissolvedbysemi-
implicitsolver
• Vertically-propagatingsoundwavesweaklydamped
• ThisisallthatisneededtomaketheclassicFVhydrostaticalgorithm
nonhydrostatic
• Fullycompressibleandnonhydrostatic!FullEulerequationssolved
• w,δzadvected asothervariables—consistent!
• Nonhydrostatic horizontalPGFevaluatedsamewayashydrostatic
TheLagrangian VerticalCoordinate
• Thedomainisseparatedintoanumberofquasi-horizontalLagrangian
layers (kindex)
• Allflowiswithinthelayers(“logically”horizontal)• No cross-layerlayerflowordiffusionVerticalmotiondeformsthelayersinstead
• Periodically,ahighly-accurateconservativeremappingisdoneto
avoidlayersbecominginfinitismally thin(δp→0)
• RemappingintervallikeLagrangian advection’stimestep:
longertimesteps yieldlessoverallartificialdiffusion
• Thisistheonly waycross-layerdiffusionisintroduced!!
TheLagrangian VerticalCoordinate
• Themassδpandheightδzareprognosticvariables.Heightandpressureofeachlayeraredynamicallycomputed.
• Lagrangian coordinateworksforany basecoordinate• Hybrid-pressure,hybrid-height,andhybrid-isentropiccoordinateshavebeensuccessfullyused
• Verticaladvectionisimplicitthroughtheverticalmovementoflayers
• ThereisnoCourantnumberrestrictionortime-splitting!
• Verticaladvectiondoesnotneedaseparatecomputation.Computingδpandδzissufficient.
• Implicitadvectionnotonlysavestimebutalsopermitsverythinlayerswithoutrequiringasmallertimestep
TheCubed-SphereGridThe3inFV3
TheCubed-SphereGrid
• Gnomoniccubed-spheregrid:
coordinatesaregreatcircles
• Widestcellonly√2widerthan
narrowest
• Moreuniformthanconformal,elliptic,
orspring-dynamicscubedspheres
• Tradeoff:coordinateisnon-orthogonal,andspecialhandlingneedstobedoneat
theedgesandcorners.
�
D-grid winds
C-grid winds
Fluxes
Fig. 2. Geometry of the wind staggerings and fluxes for a cell on a non-orthogonal grid.The angle � is that between the covariant and contravariant components; in orthogonalcoordinates � = ⇥/2.
45
TheCubed-SphereGrid
• Gnomoniccubed-sphereisnon-
orthogonal
• Insteadofusingnumerousmetricterms,
usecovariantandcontravariantwinds
• Solutionwindsarecovariant,advectionisbycontravariantwinds
• KEishalftheproductofthetwo• Windsuandvaredefinedinthelocal
coordinate:rotationneededtogetzonal
andmeridionalcomponents
�
D-grid winds
C-grid winds
Fluxes
Fig. 2. Geometry of the wind staggerings and fluxes for a cell on a non-orthogonal grid.The angle � is that between the covariant and contravariant components; in orthogonalcoordinates � = ⇥/2.
45
Alittlebitaboutinitialization
FV3 hasahostofutilitiestogenerategrids,orography,andinitial
conditionsfromcoldstart
• Anygridcanbegeneratedinseconds• InitializefromNCEPorECanalyses
• Remapfromdifferentverticalspacings
• Comprehensivetopographygeneration,includingsubgrid orography
• AdvancedFCTorographyfilterallowspreservationoftotaltopography,peaks,andvalleys;limitsslopesteepness;andpreventsnonzerotopographyinthe
ocean
• Atmosphericnudgingtoanalysesforsimpleassimilatedinitialization,orfor
aerosolandchemistrystudies
EssentialRuntimeOptions• dt_atmos:physicstimestep
• k_split:Numberofverticalremappings perdt_atmos• n_split:Numberof“acoustic”timesteps perk_split• hord_xx:Horizonal advectionalgorithms
• kord_yy:Verticalremappingoptions
• d4_bg,vtdm4:explicitdampingoptionsfordivergenceandfor
fluxes(exceptscalars),oforder2*(nord+1)• dddmp:Smagorinsky dampingcoefficient
• d_con:Convertsexplicitly-dampedKEtoheat
Gridrefinement(preview)FV3 supportsbothstretchingandnestingforgridrefinement
Gridstretchingissimpleandsmooth Gridnestingisefficientandflexible
• FV3isabletomimicmanyphysicalproperties,particularlyNewton’s
laws,massconservation,andexcellentvorticitydynamics
• Lagrangian dynamicsisverypowerful!
• Increasedparallelism• Implicitverticaladvection,withoutcomputation
• Muchreducedimplicitverticaldiffusion
• ImprovesPBLandsurfaceinteraction
• Solverisfullycompressible,andsohorizontallylocal
• Fullynonhydrostatic solvermaintainsexcellenthydrostaticflowwhile
consistentlyimplementingnonhydrostatic elasticterms
• Flexibleadvection,diffusion,andgridstructureoptions