The Next-Generation Flexible The Next-Generation Flexible Global Atmospheric ModelGlobal Atmospheric Model

Hann-Ming Henry JuangHann-Ming Henry JuangEnvironmental Modeling Center, NCEPEnvironmental Modeling Center, NCEP

Dynamics group: lead by H.-L. PanDynamics group: lead by H.-L. PanM. Iredell, H. Juang, S. Moorthi, J. SelaM. Iredell, H. Juang, S. Moorthi, J. SelaMentor: Don JohnsonMentor: Don Johnson

Symposium on the 50th Anniversary of Operational Symposium on the 50th Anniversary of Operational

Numerical Weather Prediction Numerical Weather Prediction June 17, 2004June 17, 2004

IntroductionIntroduction

Reasons to develop new codeReasons to develop new code Flexible to test different dynamical coresFlexible to test different dynamical cores Develop to work with others, e.g. ESMFDevelop to work with others, e.g. ESMF

ContentsContents Current design for flexibilityCurrent design for flexibility Current accomplishmentCurrent accomplishment Immediate next stepsImmediate next steps Future concernsFuture concerns

Rules of Code DesignRules of Code Design

Fortran 90 modules, libraries built.Fortran 90 modules, libraries built. C preprocessor for compiling optionsC preprocessor for compiling options NAMELIST for running-time dimensionNAMELIST for running-time dimension Data structure for generalizationData structure for generalization Pluggable physics packagesPluggable physics packages Possibly pluggable dynamics Possibly pluggable dynamics ESMF-type for super-structureESMF-type for super-structure Easy to maintain and friendly to useEasy to maintain and friendly to use

Concept of Code StructureConcept of Code Structure

Model layers

Super structure

APIs

APIs

APIs : official ESMF; local ESMF; WRF API etc.

Element of Super StructureElement of Super Structure

package

preprocessing

execution

postprocessing

atmos

atmos_initial

atmos_model

atmos_final

initial_prep

initial_input

initial_post

model_prep

model_integrate

Model_post

final_prep

final_outputfinal_post

(read namelist, allocation)

(read input and data etc)

(constant initialized)

(allocate temporary spaces)

(deallocate temporary spaces)

(prepare data for output)

(write output or restart data)

(deallocation)

model_integrate -

integrate_init

(integrate_getgrid)integrate_dynamics

integrate_dynamicsadj(integrate_getwave)

integrate_fine ( updating, time-filter etc)

( prepare coefficients )

integrate_physics(integrate_getwave)

integrate_physicsadj

integrate_radiation

integrate_dynamicpre

integrate_physicspre(integrate_getgrid)

integrate_radiationpre(integrate_getgrid)

integrate_dynamics -

dynamics_hybrid

dynamics_nonadv

dynamics_advect

( compute all values related to vertical coordinates etc )

( compute all forcingexcept 3d advection )

( Eulerian or Lagrangian )

atmos_ocean -

atmos_initial

atmos_model

atmos_final

ocean_initial

ocean_final

ocean_model

atmos2ocean_trans

ocean2atmos_trans

ocean2atmos_initial

atmos2ocean_initial

ocean2atmos_final

atmos2ocean_final

Data ConnectionData Connection

Structured Data

Variable structured data

Definition structured data

Pass structured data, not by common block.

Generalized Variable Generalized Variable StructureStructure

type atmos_dynamics_vars_typetype atmos_dynamics_vars_type

real*DYN_KIND, pointer :: valu(:,:,:)real*DYN_KIND, pointer :: valu(:,:,:)

character*5 , pointer :: name(:)character*5 , pointer :: name(:)

integer , pointer :: indx(:)integer , pointer :: indx(:)

integer integer len,lev, stp, numlen,lev, stp, num

endtype atmos_dynamics_vars_typeendtype atmos_dynamics_vars_type

Variable DefinitionVariable Definitiontype atmos_dynamics_defs_typetype atmos_dynamics_defs_type character datatype*4character datatype*4 integer , pointer :: date(:)integer , pointer :: date(:) real fhourreal fhour real , pointer :: dx(:)real , pointer :: dx(:) real , pointer :: dy(:)real , pointer :: dy(:) real , pointer :: ds(:)real , pointer :: ds(:) real , pointer :: sl(:)real , pointer :: sl(:) real , pointer :: si(:)real , pointer :: si(:) integer numx,numy,numsinteger numx,numy,nums integer numtracer,numdate,numwaveinteger numtracer,numdate,numwave endtype atmos_dynamics_defs_typeendtype atmos_dynamics_defs_type

Running-time DimensionRunning-time Dimension Pass dimension and options through Pass dimension and options through

NAMELIST asNAMELIST as &ATMOS_DEFINE&ATMOS_DEFINE datatype=’coef', datatype=’coef', wavenumber=64, layernumber=28,wavenumber=64, layernumber=28, longitudenumber=108, latitudenumber=1,longitudenumber=108, latitudenumber=1, initialtime=0., finaltime=240.,initialtime=0., finaltime=240., timestep=1800., filter=0.92,timestep=1800., filter=0.92, &END&END

Current AccomplishmentCurrent Accomplishment

Implement all designsImplement all designs Implement current operational Implement current operational

dynamics and physicsdynamics and physics Testing in three different platforms: Testing in three different platforms:

ibm-sp, origin-o2k, pc-linuxibm-sp, origin-o2k, pc-linux Stable up to monthly integrationsStable up to monthly integrations Single and multiple tasksSingle and multiple tasks

Model GridModel Grid

Spectral model with spherical transform is Spectral model with spherical transform is suitable for spherical domain.suitable for spherical domain.

Reduce Gaussian grid (Williamson and Reduce Gaussian grid (Williamson and Rosinski, 2000)Rosinski, 2000)

Reduce spherical transform (Juang 2004)Reduce spherical transform (Juang 2004) Reduce Gaussian gridReduce Gaussian grid Reduce Legendre computationReduce Legendre computation

Reduced (color)

Full (black)

Reduced - Full

Hybrid Parallel ComputingHybrid Parallel Computing Code structure is the same for single and multi Code structure is the same for single and multi

PE sPE s Only IO and spectral transform routine contain Only IO and spectral transform routine contain

MPI related callsMPI related calls 2D decomposition with 1D option2D decomposition with 1D option

2D can run more PE s2D can run more PE s 1D is faster in vector machine1D is faster in vector machine

Multi-threads for outmost do-loopMulti-threads for outmost do-loop Consider vector in inner do-loopConsider vector in inner do-loop For different # of PEs without recompilingFor different # of PEs without recompiling

24-h accumulated rainfall from FGM

Contour intervals of 4x10-4 m

Next StepsNext Steps

Generalized vertical coordinatesGeneralized vertical coordinates Implemented a generalized coordinatesImplemented a generalized coordinates Emphasized on sigma-theta coordinatesEmphasized on sigma-theta coordinates Current variables: U, V, T, P, and tracersCurrent variables: U, V, T, P, and tracers

Comparisons forecast skillsComparisons forecast skills Differences between sigma-p to sigma-thetaDifferences between sigma-p to sigma-theta Differences from the operational oneDifferences from the operational one

€

du*

dt= −

RdTmΠ ∂p /∂ζ( )

Π∂(∂p /∂ζ )

a∂λ

⎛

⎝ ⎜

⎞

⎠ ⎟

− g∂zsfca∂λ

− (Π s − Π)∂ RdTm /Π ∂p /∂ζ( ) (∂p /∂ζ )

a∂λ

⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟+ f sv

*

dv*

dt= −

RdTmΠ ∂p /∂ζ( )

Π∂(∂p /∂ζ )

a∂ϕ

⎛

⎝ ⎜

⎞

⎠ ⎟− g∂zsfca∂ϕ

−(Π s − Π)∂ RdTm /Π ∂p /∂ζ( ) (∂p /∂ζ )

a∂ϕ

⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟− f su

* −m2 s*2

asinφ

dTmdt

= κTmm2

Π ∂p /∂ζ( )V * • Π∇ ∂p /∂ζ( ) − Π ∇ •V * ∂p /∂ζ( )( )[ ]

∂∂p

∂ζ

⎛

⎝ ⎜

⎞

⎠ ⎟

∂t= −m2 V * •∇ ∂p /∂ζ( ) + ∂p /∂ζ( )∇ •V *

( ) −∂ζ

•

∂p /∂ζ( )∂ζ

Hydrostatic system in generalized vertical coordinates

P=0

P=P

P=P

P=Ps

P=A(P+B(Ps

Isentropicddt

sigma

s

=1

infinite

=0

Future ConcernsFuture Concerns

Improve Dynamics as well as PhysicsImprove Dynamics as well as Physics Un-approximated primitive equationUn-approximated primitive equation Higher resolution physicsHigher resolution physics

Improve Numerical MethodsImprove Numerical Methods Optimal between conservation and accuracyOptimal between conservation and accuracy Adopt multi-dynamical coresAdopt multi-dynamical cores

Linkage to other components of earth Linkage to other components of earth models through ESMFmodels through ESMF

Model EquationModel Equation

Nearly all global models areNearly all global models are Hydrostatic systemHydrostatic system Diagnosis vertical momentumDiagnosis vertical momentum Shallow-atmosphere assumptionShallow-atmosphere assumption

Next generation could be years laterNext generation could be years later Very high resolution is expectedVery high resolution is expected Hydrostatic is not validHydrostatic is not valid Full Coriolis effectFull Coriolis effect Deep atmosphere Deep atmosphere

Momentum EquationMomentum Equation

€

du

dt= −

1

ρ

∂p

rcosϕ∂λ+ f sv − fcw +

uv tanϕ

r−uw

r + Fu

dv

dt= −

1

ρ

∂p

r∂ϕ − fsu −

u2 tanϕ

r−vw

r + Fv

dw

dt= −

1

ρ

∂p

∂r− g + fcu +

u2

r+v 2

r+ Fw

where f s = 2Ωsinϕ and fc = 2Ωcosϕ

Un-approximated ApproachUn-approximated Approach

From energy point of viewFrom energy point of view Energy=1/2[u(du/dt)+v(dv/dt)+w(dw/dt)]Energy=1/2[u(du/dt)+v(dv/dt)+w(dw/dt)]

Coriolis force and curvature terms are cancelled Coriolis force and curvature terms are cancelled out out

Cosine component of Coriolis force contributes Cosine component of Coriolis force contributes to zonal vertical circulationto zonal vertical circulation

And r=a+z provides accurate curvature and And r=a+z provides accurate curvature and displacementdisplacement

Generalized vertical coordinates (Staniforth and Generalized vertical coordinates (Staniforth and Wood, 2003)Wood, 2003)

Full Coriolis EffectFull Coriolis Effect

Sine component of it modifies the flow Sine component of it modifies the flow pattern at middle- and high-latitudespattern at middle- and high-latitudes

Cosine component of it modifies the Cosine component of it modifies the vertical flow patterns at tropic and extra-vertical flow patterns at tropic and extra-tropic areas.tropic areas.

Coriolis force is approached to 10Coriolis force is approached to 10-4-4 from f and 10 from u, so it is about from f and 10 from u, so it is about 1010-3-3, and w is 10, and w is 10-2-2, 10% effect in w., 10% effect in w.

Not-shallow atmosphereNot-shallow atmosphere

For shallow assumptionFor shallow assumptionr = a (earth radius)r = a (earth radius)

For non-assumptionFor non-assumptionr = a + zr = a + z

Not only affect gradients, curvature terms but also Not only affect gradients, curvature terms but also displacement by wind, displacement by wind, since u = r cossince u = r cos d d/dt and v = r d/dt and v = r d/dt/dt

For example, 50 m/s wind from100 hPa (20km) to For example, 50 m/s wind from100 hPa (20km) to 0.2 hPa (60km), errors in r=a are from 0.2 to 0.2 hPa (60km), errors in r=a are from 0.2 to above 1%, thus, errors in displacement after 10 above 1%, thus, errors in displacement after 10 days are from 20 km to 432 km.days are from 20 km to 432 km.

Final NoteFinal Note

With the combinations ofWith the combinations of(1)(1) Sigma-theta coordinates to reduce the errors Sigma-theta coordinates to reduce the errors

in vertical advection in vertical advection (2)(2) Deep atmosphere to reduce the errors in Deep atmosphere to reduce the errors in

stratospheric horizontal advectionstratospheric horizontal advection(3)(3) Full Coriolis effect to have better vertical Full Coriolis effect to have better vertical

circulation along tropical areascirculation along tropical areasThe improvement on dynamical weather and The improvement on dynamical weather and

climate forecasts can be expected.climate forecasts can be expected.