Willem A. LandmanAsmerom BerakiFrancois EngelbrechtStephanie Landman

Supercomputing for weather and climate modelling:

convenience or necessity

11 June 2009Cut-off low over central SA

New Multi-Model Short-Range Ensemble System (precipitation)

• 24hour totals for:– day1 (14 members) and – day2 (6 members)

• Unified Model (different configurations and resolutions) – 10 members:

• 12km (xaana/ng/nj)• 15km (xaaha/hc)

• WRF model– 2 members:

• 12km • Non-hydrostatic mesoscale core

– 2 members:• 15km• Advanced Research WRF core

• In Test Phase– 1º NCEP model (15 members)

Probability MapsDay 1 Day2

>5mm

>10mm

Are AGCMs useful?

“trend” hits=27/33=82%

The best model is the ECHAM4.5 AGCM

Uncertainty in initial atmospheric

state

Uncertainty in future atmospheric

state

Ensemble forecast from model 1 explores part of the future uncertainty

Ensemble forecast from model 2, run from (eventhe) same set of initial states, typically explores

additional future uncertainties

Uncertainty inSST state

???

Will uncertainties in forcing SST fields better estimate the probability of each outcome?

Ensemble forecast from model 3, run from different ocean states may explore

additional future uncertainties

Shaded areas: forecast uncertainty as reflected by forecast ensemble; black line: ensemble mean; red line: model climatology

The multi-models: Skill Differences• 3 AGCM configurations:

– Forced with ca_sst, ECMWFem and ECMWFsc• 2 AGCM configurations

– Forced with ECMWFem and ECMWFsc

Positive values where MM is better than best single model (ECMWFem)

By considering (some of) the certainties in forcing SST fields the probability of forecast outcomes is better estimated (over some areas)

Coupled model on CHPC…• ECHAM4.5-MOM3 on CHPC using 8 processors, i.e., 4 processors for each model.• Simulation for one month (May 1982)• Time needed to fish the coupled run was 1074.32 sec (1.49 hrs)• Similar run for uncoupled ECHAM4.5 using 4 processors took 225.49 sec (19 min)• Seems that coupled run is slower than expected – usually double the time is

assumed for coupled run (here, just one case)

Total rainfall (mm; shaded) and 500 hPa geopotential height (m; contour)  

Resolution over southern Africa is about 60 km

ARC-CSIR-CHPC-UP-CSIRO

Meraka Institute, C4-cluster

High-resolution regional climate modelling Exp1: 60 km resolution over southern Africa

• Forcing (wind nudging) from NCEP reanalysis data

• Period simulated 1976-2005

• Time step 20 min

• Data set size: 300 GB

High-resolution panel:

40 S to 10 S

10 E to 40 E

Resolution over Australia is about 60 km

High-resolution regional climate modelling: Exp2: 8 km resolution over the southwestern Cape

• Forcing (wind nudging) from 60 km simulation

• Period simulated 1976-2005

• Time step 3 min

• Data set size: 600 GB

High-resolution panel:

35.5 S to 31.5 S

17.5 E to 21.5 E

ARC-CSIR-CHPC-UP-CSIRO

Meraka Institute, C4-cluster

Resolution over Australia is about 60 km

• Forcing (wind nudging) from 8 km simulation

• Period simulated 1976-2005

• Time step 30 sek

• Data set size: 1.8 TB

High-resolution regional climate modelling: Exp3: 1 km resolution over a portion of the

southwestern Cape

High-resolution panel:

34.31 S to 33.81 S

28.28 E to 18.78 E

ARC-CSIR-CHPC-UP-CSIRO

Meraka Institute, C4-cluster

Resolution over Australia is about 60 km

High-resolution regional climate modelling: Exp4: 200 m resolution over the Stellenbosch region

• Forcing (wind nudging) from 1 km simulation

• Period simulated 1976-2005

• Time step 6 sek

• Data set size: 1.8 TB

High-resolution panel:

33.89 S to 33.79 S

18.79 E to 18.89 E

ARC-CSIR-CHPC-UP-CSIRO

Meraka Institute, C4-cluster

A few machines...

System Configuration

The ES is a highly parallel vector supercomputer system of the distributed-memory type, and consisted of 160 processor nodes connected by Fat-Tree Network. Each Processor node is a system with a shared memory, consisting of 8 vector-type arithmetic processors, a 128-GB main memory system. The peak performance of each Arithmetic processors is 102.4Gflops. The ES as a whole thus consists of 1280 arithmetic processors with 20 TB of main memory and the theoretical performance of 131Tflops.

Global Atmosphere Simulation with MSSG-A03-08AUG2003, Horizontal resolution: 1.9 km, 32 vertical layers

The Northern Pacific Ocean Horizontal Resolution: 2.78km, Vertical Layers: 40 layers, 15 years integration Boundary condition: monthly data from NCAR monthly data from OFES simulation( 10km global simulation)

Ocean Component of Multi-Scale Simulator for the Geoenvironment:MSSG-O

Conclusion

• High-resolution, large ensemble, many models, various configuration– All require dedicated high-speed computer for

• Operational forecasts/projections• Research to better understand the weather/climate

system