GEOS-CHEM: The Next Generation - Harvard...

GEOS-CHEM: The Next Generation

Bob YantoscaSoftware EngineerAtmospheric Chemistry Modeling GroupHarvard University

Group Meeting08 October 2003

08 Oct 2003GEOS-CHEM: The Next Generation2

TopicsLife after BrendanSMVGEAR II implementationGEOS–CHEM v5-07-xx: scientific upgradesGEOS–CHEM benchmark simulationsTransition to GEOS–4 / fvDAS met fieldsIDL 6.0 and GAMAPAutomatic met data processing systemGEOS–5 (when will the madness ever end?)


Life after BrendanWe are now in Year 1 A.B. (After Brendan)

Areas that Brendan worked onSMVGEAR II prep workInstallation of P–fixers for TPCORE (old & new versions)Acetone simulationMethanol simulationAnd a lot of other stuff (incl. GISS model)

Brendan was also an extra pair of hands around hereWe are have less programmers here than we used to…


Life after BrendanWhat does this mean for the average GEOS-CHEM user?

We now ask GEOS–CHEM users to be more resourcefulLearn about Fortran 90, IDL, shell scripts, Matlab, Splus, etc.Take a more active role in debugging GEOS-CHEMLearn how to use the IDL regridding data routinesLearn how to use the IDL netCDF & HDF file I/O routinesBe prepared to write data analysis and/or plotting scriptsTake advantage of GEOS–CHEM & GAMAP documentationTake advantage of other people’s expertise w/in the group

Turnaround times may be slower than beforeBob Y. only has 2 hands ☺GEOS–4 / fvDAS integration is now the highest priorityGEOS–CHEM users outside Harvard must also be supported


SMVGEAR II implementationOverview

GEOS–CHEM v5–05–03 was the first version to use the new SMVGEAR II chemistry solver package (June 2003)

What’s new in SMVGEAR II?Can define 3 regimes (urban, free trop, strat) for chemistryCan reorder the grid boxes by stiffnessNew chemistry mechanism file formatSolver is supposedly faster than SMVGEAR I

DrawbacksOriginally written to optimize well on CRAY vector machinesSMVGEAR II seems to not optimize as well on scalar machinesIt may be accessing widely-spaced array elements at once


SMVGEAR II implementationHow have we implemented SMVGEAR II in GEOS-CHEM?

We have turned off the grid box reordering by stiffnessOn SGI, whatever speed-up you get from reordering the boxes is negated by the time it takes to physically reorder the boxesBox reordering may yield more speed-up on other platforms

Although you can do 3 types of chemistry w/ SMVGEAR II, we are currently only doing tropospheric chemistry

Gabriele Curci is working on a strat chemistry mechanism which will later be implemented in conjunction w/ our trop mechanism

We have added OpenMP parallelization into SMVGEAR II

We have freed up memory by removing obsolete COMMONs


SMVGEAR II implementationChallenges in SMVGEAR II implementation

We had to translate the SMVGEAR I chemistry mechanism file into the SMVGEAR II file format

This was done with IDL code and hand editing

The ND65 chemical prod & loss diagnostic had to be rewritten Our first implementation slowed the code down on LinuxWe rewrote it in a more efficient manner on the 2nd try

The ND40 plane-following diagnostic had to be updatedSpecies and reactions are numbered differently in SMVGEAR II than in SMVGEAR I, we had to adjust accordingly


Scientific UpgradesSMVGEAR II implementation took place between March and June 2003. During that time several important scientific upgrades to the code were put on the back burner.

Starting in July 2003, we began a series of GEOS-CHEM versions numbered v5–07–xx. Each minor release number would include one particular scientific upgrade.

Each intermediate v5–07–xx release was benchmarked with the 1-month full chemistry simulation, so that we could assess the impact of each upgrade separately.

Each intermediate v5–07–xx version was not released to the GEOS-CHEM user community (for people’s sanity!)


Scientific UpgradesList of code updates

GEOS–CHEM v5–07–01 Allow SMVGEAR II background methane to vary as a function of latitude and year. Use CMDL station data.

GEOS–CHEM v5–07–02Rewrote the ND65 chemical prod-loss diagnostic so that it executes 20-30% faster (especially on Linux)

GEOS–CHEM v5–07–03Now scale FAST-J O3 columns to monthly-mean values from EP-TOMS dataCurrently have data for 1997-2002; other years to follow soon


Scientific UpgradesList of code updates (cont’d)

GEOS–CHEM v5–07–04Null revision

GEOS–CHEM v5–07–05Update SMVGEAR II reaction rates (in the chemistry mechanism file "globchem.dat") to JPL 2002 values

GEOS–CHEM v5–07–06Instead of only dry-depositing species at the surface, apply dry deposition losses in all levels w/in the mixed layerThis is necessary since GEOS-3 has a very shallow sfc layerThis gives better treatment of HNO3 dry deposition


Scientific UpgradesList of code updates (cont’d)

GEOS–CHEM v5–07–07The γ coefficient (stickiness factor) for N2O5 hydrolysis is now a function of relative humidity, temperature, and aerosol typePreviously, γ was set to a constant value (0.1)

GEOS–CHEM v5–07–08Change stratospheric O3 flux from 400 Tg/yr to 500 Tg/yrThis was necessary to better represent the stratospheric flux when using initial conditions from GEOS–CHEM versions which contain the TPCORE P–fixer

GEOS–CHEM v5–07–08 is the last update in this series Yay! ☺


GEOS–CHEM benchmarksOverview

GEOS–CHEM v5–07–08 was benchmarked with a standard 1-year simulation

6-month spin-up (July 1, 2001 – Jan 1, 2002)12-month run (Jan 1, 2001 – Jan 1, 2002) Will compare results to both observations and other model runs

GEOS–CHEM v5–07–08 will be the version that is released to the GEOS–CHEM user community

Pending examination of 1-year benchmark results by djj

The 1-year benchmark was a result of the discussion from the GEOS–CHEM User’s Meeting, held in June 2003.


GEOS–CHEM benchmarksFull chemistry benchmark (1 year)

GEOS–3 meteorology w/ 30 layers @ 4x56-month spinup: Jul 1, 2000 – Jan 1, 20011 year run: Jan 1, 2001 – Jan 1, 2002

Diagnostic quantitiesEverything but the kitchen sinkAlso save out P(O3), L(O3) for Tagged Ox runs

We use Inna’s IDL code (same as for GMI) to plot model results vs. observations from aircraft, sondes, ship tracks, and surface sites.


GEOS–CHEM benchmarks

How does GEOS–CHEM v5–07–08 stack up?

At first glance, it doesn’t’seem a whole lot different than Rokjin’s run with v5–05–03

There are pages and pages and pages and pages more of these types of plots!


GEOS–CHEM benchmarksIn addition to the 1-year full chemistry benchmark run, we will continue to perform our existing benchmarks

1-year radon run1-month full chemistry run

These existing benchmarks are useful in code developmentThey allow us to find obvious implementation bugsThey allow us to gauge how the model is performing when porting GEOS–CHEM to other platforms

The 1-year benchmark run is time-intensiveDue to queue time limits, we have to run it in 6 separate stagesIt can take approx. 2 weeks to complete and print out resultsWe will run this only for “watershed” releases of GEOS–CHEM


Transition to GEOS–4Overview

GEOS–CHEM is driven by assimilated met data from NASA’s Global Modeling and Assimilation Office (GMAO)

GMAO has produced 4 major met data sets:GEOS-1 (1985 – 1995, 20 layers to 10 hPa)GEOS-STRAT (1995 – 1997, 46 layers to 0.1 hPa)GEOS-3 (2000 – 2002, 48 layers to 0.01 hPaGEOS-4/fvDAS (2003 –, 55 layers to 0.01 hPa)

GEOS–CHEM can presently run with GEOS–1, GEOS–STRAT, and GEOS–3 fieldsGEOS–4 support is being added now


Transition to GEOS–4 Comparison of GEOS met fields up to 3 km


Transition to GEOS–4Comparison of GEOS vertical levels to ~32 km


Transition to GEOS–4

Z = “Zulu” = abbreviation for GMT

From the fvDAS file spec on GMAO’swebsite Start of day Start of next day

GEOS-1

GEOS-S

GEOS-3

GEOS-4

A-6 fields

A-3 fields

A-6 fields

A-3 fields

Times of day when you need to read met fields


Transition to GEOS–4Differences from GEOS–3Some met fields that used to be synoptic (e.g. instantaneous

snapshots) in GEOS–3 are now averaged in GEOS–4U, V, Specific Humidity, Temperature, Surface Albedo

GEOS–4 cloud convection fields are totally different than the fields provided in GEOS–3

CLDMAS and DTRAIN are no longer providedHack (shallow) convection: HKETA and HKBETAZhang/McFarlane (deep) convection: ZMMU, ZMMD, ZMEU

GEOS–4 seems to use a different land map than GEOS–3Z0, Vegetation cover, Albedo, etc are different


Transition to GEOS–4Transport

A new fvDAS/TPCORE transport scheme from S-J Lin is available for use with either GEOS-3 or GEOS-4 winds.

fvDAS/TPCORE is now included in GEOS-CHEM v5-05-03 and higher. It can use OpenMP or MPI parallelization.

We also implemented the LLNL P-fixer (cf. Phil Cameron-Smith), which adjusts the U, V winds in order to nudge the computed pressure to the actual met field pressure.

fvDAS/TPCORE with LLNL P-fixer conserves mass when using GEOS-3 winds. GEOS-4 winds are yet untested.



For offline models, the starting surface pressure w/in a grid box is specified from the met fields (i.e. is read from disk).

However, transport will move air into the box from its neighbor boxes

At end of transport time step, the grid box pressure will not match the met field pressure (from disk)

This causes mass non-conservation.

Graphic from http://asd.llnl.gov/pfix/



P-fixer distributes the horizontal mass-flux (for hybrid pressure levels) in the vertical in proportion to the change in sigma coefficient across each layer. This ensures that the corrective mass-flux does not induce any vertical wind, which is calculated from the divergence of the horizontal winds at each level

Graphic from http://asd.llnl.gov/pfix/


Transition to GEOS–4Transport (cont’d)

The LLNL P–fixer as we have implemented it assumes that the U, V winds are synoptic.

This is true for GEOS–3. This is NOT true for GEOS–4.

We will need to modify the LLNL P-fixer in GEOS–CHEM to account for the fact that the GEOS–4 winds are 6-hr averages and not synoptic fields.

Shiliang and Bob Y. are currently working on this. We are consulting Phil Cameron-Smith at LLNL for more advice.


Transition to GEOS–4Convection

The new GEOS–4 cloud convection met fields require that a new convection scheme be implemented into GEOS–CHEM.

We will use the cloud convection scheme from NCAR’sMATCH model (cf. Phil Rasch).

We were not given a standalone code package. We are now faced with having to strip out selected subroutines from the MATCH source code. A lack of documentation is making this job difficult.

GMAO considers the NCAR scheme a “black box”, they don’t really understand how it works either. ☺


Transition to GEOS–4Convection (con’td)

The NCAR scheme uses two different parameterizations:Hack (shallow) convectionZhang & McFarlane (deep) convection

The Z&M scheme includes both updrafts & downdrafts.

We must also replace the code which performs the wet loss of soluble tracer in cloud updrafts into the GEOS–4 convection routines. This will take some extra finagling.

Shiliang and Bob Y. are working on this. Don’t expect a quick turnaround time; this will be an intricate operation. ☺


Transition to GEOS–4Processing

Every time GMAO changes met data sets, a substantial amount of recoding that must be done

Scripts must be written to download data from GMAOCode must be written to read data from HDF / HDF–EOS filesRegridding programs must be customized in order to regrid and save only the met fields that we need for GEOS–CHEM

GEOS–4 met fields files require


Automated met data systemOverview


IDL 6.0 and GAMAPOverview

We recently upgraded our IDL version from 5.3 to 6.0.IDL 5.3 was released in 1999, it’s outdated now

Problem: IDL versions higher than 5.3 do not support GIF. GIF is now a proprietary file format (you must pay for it)We needed GIF’s for TRACE-P and 2k2, so we didn’t upgrade

IDL 6.0 supports other graphic file formats, includingPNG (Portable Network Graphics)JPEGTIFFBMP (Windows Bitmap)MPEG (for animations)


IDL 6.0 and GAMAPGAMAP was recently upgraded to version 1.53

Can now read in GEOS-4 met fieldsCan save PNG, JPEG, TIFF, BMP files


GEOS–5 met fieldsOverview

GMAO is planning on releasing on GEOS–5 met fields within about a year from now

What can we expect in GEOS–5?GCM appears that it will be a mix of packages from the NCAR CCM, GEOS–3 GCM, and NSIP GCMRadiation code may be updatedThey may revert to the GEOS–3 convection parameterization

GEOS-CHEM: The Next Generation - Harvard...

Documents

Transcript of GEOS-CHEM: The Next Generation - Harvard...