Recent Progresses in Numerical Weather Prediction and HPC ... · PDF fileRecent Progresses in...

Recent Progresses in

Numerical Weather Prediction

and HPC at KMA

Hee-Dong Yoo Korea Meteorological Administration

26th WGNE Meeting

October 18-22, 2010, Tokyo

Major Changes in

Operational NWP System

Regional (Deterministic short-range)

Global (Deterministic medium-range)

Major NWP Changes

• T426L40 (GDAPS)

– GSM from JMA

– 3DVAR

– Operation since 1997

• UM N320L50

– The Unified Model from UKMO

– 4DVAR

– Operation since May 2010

• 30kmL33 (RDAPS)

– MM5 Model

– FDDA


• 10kmL40 (KWRF)

– WRF ARW Model


• UM 12kmL38

– The Unified Model from UKMO

– Initialized from Global UM

– Operation since May 2010

• 10kmL40 (KWRF)

– WRF ARW Model

– L.B.C. from Global UM

- 3 -

- 4 -

UM Implementation Background

KMA decided to import the Unified Model as a next-generation NWP

system (Q4 ’07)

UM Research license (Q4 ’07)

Collaboration Agreement – including Science Plan - between KMA and UK Met Office

(Operational License, Q2 ’08)

Routine operation of global/regional UM started (Q2 ’08)

Initialized from UK Met Office’s initial condition

Global D.A. cycle for UM including ODB implementation (’08~’09)

Migration of UM system to the 3rd supercomputer (Q4 ’09)

Parallel run of UM system on Cray XT5 Interim (March ’10~)

Operational run of UM system on Cray XT5 (14th May ’10~)

- 5 -

Cray XT5 (3rd HPC)

Global Model (UM N320L50)

10-day Forecast

Regional Model (UM 12km)

Parallel Suite : 2010.03 ~ 2010.05

Observation

Pre-Processing

Data

Assimilation

Obs. Pre-proc. System

(OPS)

Atmos. 4DVAR (VAR)

Surf. Analysis (SURF)

COMIS

GTS/FTP

Decoded Obs.

(ODB)

Pre-/Post-Server

(New)

Application/Statistical Models

Post-Processing Regional Model

(KWRF 10km)

Operational Suite : 2010.05 ~

Operational UM System

- 6 -

Regional • Horiz. : ~12km (540x432 / 0.11°x0.11°)

• Vert. : 38 layers (top ~ 39km)

• +72hrs Forecast

• Initialized from Global I.C.

• Version : UM 6.6

Global • Horiz. : N320 (~40km / 0.5625°x0.375°)

• Vert. : 50 layers (top ~ 60km)

• +252hr Forecast

• Initialized by 4DVAR

• Version : UM 6.6

UM Configuration in KMA

MODELS Horiz. Resol.

(Vert. Layers) Forecast Length Target

UM (GLOBAL) 40km (50) 10 days Global Medium-range Fcst.

GDAPS

30km (40) 10 days Global Medium-range Fcst.

55km (40) 10 days Global Medium-range (EPS)

110km (21) 120 days Seasonal Forecasting

UM (REGIONAL) 12km (38) 72 hours East-Asia Short-range Fcst.

RDAPS 30/10/5km (33) 66/24/24 hours East-Asia Short-range Fcst.

KWRF 10 km (40) 66 hours East-Asia SRF – UM-based

KLAPS 5km 12 hours Korean Peninsula

Wave Models

(Wave Watch-III)

60km 10 days GWW3(Global) – UM-based

8km 66 hours RWW3(E-Asia) – UM-based

1km 24 hours CWW3(Coastal) – UM-based

Sand Dust Model

(ADAM) 30km 72 hours

Yellow-Dust (East Asia)

– UM-based

Tide and Storm Surge

(RTSM) 8km 72 hours

Regional Tide & Storm Surge

– UM-based

Typhoon Model

(DBAR) 35km 72 hours Track & Intensity

Statistical/Digital Fcst. - 2-10 days UM-based

Main Operational Models (’10.5~)

- 8 -

UM-GLOBAL UM-REGIONAL

Governing Eq. Complete equation (Non-hydrostatic)

Horiz. Resolution N320 (40km 0.5625x0.375) 12km (0.11x0.11)

Vertical Layers L50 (top ~ 63km) L38 (top ~ 39km)

Forecast Length 10.5 days (252 hours) 3 days (72 hours)

Timestep Size 900 sec 240 sec

I.C./ Data Assimilation 4DVAR Downscaling from global initial condition

Spatial Discretization Finite Difference method

Time integ. / Advection Semi-implicit Semi-Lagrangian scheme

Radiation Process Edwards-Slingo general 2-stream scheme

Surface Process MOSES-II land-surface scheme

PBL Process MOSES-II Non-local PBL

Convection Process Mass flux convection with CAPE closure

Microphysics Mixed-phase precipitation

Gravity Wave Drag G.W. drag due to orography (GWDO)

Surface B. C. Surface Analysis + Climatology

Operation Frequency Twice daily (00/12 UTC) /

6hour D.A. cycle

Twice daily (00/12 UTC)

Operational NWP Models – Unified Model

- 9 -

GDAPS (Deterministic) GBEPS (Ensemble)

Base Model GSM (Global Spectral Model from JMA)

Governing Eq. Primitive Equation (Hydrostatic)

Horiz. Resolution T426 (0.28125x0.28125) T213 (0.5625x0.5625)

Vertical Layers L40 (top : 0.4 hPa)

Forecast Length 252 hours 240 hours

I.C./ Data Assimilation 3DVAR Breeding Method + Factor Rotation / 3DVAR

Ensemble Size - 17*2 (12 hour time-lag)

Spatial Discretization Spectral Transformation

Time integration Semi-implicit Scheme

Radiation Process SW : Lacis and Hansen (1974) / LW : JMA (Sugi et al., 1989)

Surface Process SiB (Simple Biosphere, Sellers, 1986)

PBL Process Non-Local PBL (Holtslag and Boville, 1993)

Convection Process Kuo Type (Kuo, 1974)

Microphysics Large-scale condensation (Sundqvist, 1978)

Gravity Wave Drag GWD due to orography (Iwasaki et al., 1989)

GWD due to cumulus convection (Chun and Baik, 1988)

Operation Frequency Twice daily (00/12 UTC)

6hour 3DVAR cycle

Twice daily (00/12 UTC)

6hour 3DVAR cycle

Operational NWP Models – Others / Global / Atmos.

RDAPS KWRF

30km 10km 5km

Base Model MM5 WRF ARW

Dynamic Frame Non-hydrostatic Non-hydrostatic

Horiz. Resolution 30km(171x191) 10km(160x178) 5km(141x141) 10km(574x514)

Vertical Layers 33 Layers / ~50 hPa 40 Layers / ~50 hPa

Forecast Length 66 hours 24 hours 24 hours 66 hours

I.C./Data Assim. FDDA 3hr cycle (IAU) 1-way interact. 3DVAR / DFI

Lateral B.C. Relaxation (12hr) Time & inflow/outflow dependent relaxation (3hr)

Updated every 6 hrs (global UM)

Spatial Discretization Finite Difference Finite Difference

Radiation Process Cloud Radiation RRTM

Surface Process 5-layer Soil Model Noah LSM

PBL Process MRF PBL YSU PBL

Convection Process New Kain-Fritsch None New Kain-Fritsch

Microphysics Mixed phase WSM6

Operational NWP Models – Others / Regional / Atmos.

- 10 -

Data Assimilation System – for Global UM

- 11 -

Observation Data and Global D.A.

• Observation DB : ODB CY32R3

• Observations Used

– Surface (SYNOP/Ship/Buoy), Sonde (TEMP, Pilot, Wind Profiler),

Aircraft, Satwind, AIRS (AQUA), Scatwind, IASI (MetOp), GPSRO

(COSMIC), SSMIS

– ~80,000 observations per cycle (80~85% compared to UK Met Office)

• Analysis Scheme : 4-dimensional Variational Data Assimilation

• Analysis Time : 00, 06, 12, 18 UTC

• Cut-off Time : 2 hours 25 minutes for Early Analysis

6 hours 25 minutes for Update Analysis

• Spatial Resolution (Inner Model) : N108(1.67x1.11deg) L50

• Assimilation Window : -3 hours to +3 hours of Analysis Time

- 12 -

NWP Performance - Global

Forecast error comparison (GDAPS vs. UM)

- 13 -


+5days

+3days

RMS error for 500 hPa Geopotential Height / N. Hemisphere

Old (GDAPS)

New (UM)


• Verification Period

– April ~ September 2010

• ~15% enhancement

compared to GDAPS for

500 hPa Geopotential

Height (+120 hours)

Verification against Analysis

• The Regional UM

outperforms KWRF

• Light Precipitation is

significantly over-

estimated in the UM

KWRF(10km)

UM12

UM(12km) – KWRF(10km)

Precipitation Verification (against 76 ASOS obs.) – UM versus KWRF

- 15 -

NWP Performance - Regional (Precipitation)

New Supercomputer and

Upcoming NWP Changes

NEC SX-5/28M2 1999.12 ~ 2005.11

1st Supercomputer

Theoretical performance

0.2 TFlops

×90

2nd Supercomputer


18.5 TFlops

CRAY X1E 1024/MSP 2005.12 ~ 2010.11

3rd Supercomputer


682.9 TFlops

CRAY XE6 90,240 cores 2010 ~

×37

- 17 -

Current Status (’10.10) - 1st phase of 3rd Supercomputer - Cray XT5 - 2,560 cores / 27.65 TFlops

KMA’s Supercomputer

Cray XT5 MPP System with Lustre Global Parallel File System – Processor : AMD 2.7GHz (4 core)

Initial Phase Technology Peak Perform.

TFlop/s Storage Backup

Interim Cray XT5 14 12 TB

4.5 PB Main Computational

System Cray XT5 28

0.7 PB (Tier0)

1.4 PB (Tier1)

System 2

Interim System

System 1

Main Computational System

KMA’s 3rd Supercomputer – Initial Phase

- 18 -

Cray Next Generation XE6 (Baker) MPP System – Processor : AMD 2.1GHz (12 core)

Initial Phase Technology Peak Perform.

TFlop/s Storage Backup

Interim Cray Baker 16 12 TB

4.5 PB Main Computational

System Cray Baker 683

0.7 PB (Tier0)

1.4 PB (Tier1)

KMA’s 3rd Supercomputer – Final Phase

System 1 - Operational


System 3 Interim System

System 2 – Research/Backup


- 19 -

Cray XE6

Cray XE6

• Construction : June 2008 ~ Jan 2010

• Official Opening Ceremony: Feb 2010

• Total Cost: 25.3 billion won(about 23M USD)

• Total Construction Area : 23,092㎡ / KMA HQ: ~16,500㎡

• Building area: 7,052㎡, 3rd floors

Electricity: 250 % greater than HQ

UPS: 475 % greater than HQ

Cooling : 360 % greater than HQ

KMA’s Supercomputer Center

- 21 -

Upcoming NWP Changes

Global NWP System

• Improving the TC Bogussing Scheme for Global NWP

– Current Bogussing Method : Wind Bogussing (Obs. Bogussing)

– New Development : Wind + Sea Level Pressure Bogussing

– Effect : Improvement in intensity (and track) of simulated TCs

– Operational Application : Q4 2010

Intensity error

reduced

Track error

reduced

- 22 -

Upcoming NWP Changes on the New Supercomputer

New Global Ensemble Prediction System

• GDAPS T213L40 (GBEPS)

– Model : GDAPS

– T213 (0.5625x0.5625)

– Bred Vector + Factor Rotation

– No Model Error Simulation

– 16+1 members (x2, 00/12UTC)

– 10 day Forecast

– 6-hourly 3DVAR Cycle

00, 12UTC Forecast

• UM N320L50 (MOGREPS-G)

– Model : The Unified Model

– N320 (0.5625x0.375)

– ETKF

– Stochastic Physics included

– 23+1 members

– 15 day Forecast

– Control I.C. from Deterministic Model

00, 12UTC Forecast

• Currently on its Pre-operational Real-time Test

• Operational Run : Q1 2011

– Platform : Cray XE6 (final phase of KMA’s 3rd supercomputer)

RMSE

SPREAD

Solid : OPER

Dashed : N320L50

Results for

Summer 2010

CPRSS_New

CPRSS_Old

- 23 -


Resolution Change of Global NWP Model

• UM N320L50

– N320 (640x481 / ~40km)

– 50 Vertical Levels (~63km)

– UM Version : vn6.6

– 4DVAR Inner Loop : N108

• UM N512L70

– N512 (1024x769 / ~25km)


– UM Version : vn7.5 or later

– 4DVAR Inner Loop : N144

• Operational Run : Q2 2011 on Cray XE6

N320 N512

• Max. Height in South

Korea domain

– N320 orog. : 523m

– N512 orog. : 692m

- 24 -


Regional 4DVAR UM System

• UM 12kmL38

– 12km (0.11x0.11 deg)


– UM Version : vn6.6

– Initialized from Global Model

(Cold-Start)

– 00, 12UTC Forecast (+72 hours)

• UM-4DVAR 12kmL38

– 12km (0.11x0.11 deg)



– 6-hourly Atmos. 4DVAR +

Global Surface Analysis Downscaled

– 00, 12UTC Forecast (+72 hours)

• Operational Run : Q2 2011 on Cray XE6

- 25 -


Local High-Impact Weather Forecasting System

• UM 1.5kmL70

– New NWP System

– Horizontal Resolution : 1.5km Variable Grid

– Vertical Levels : 70 Levels (~39km)


– Data Assimilation : 3-hourly 3DVAR Cycle

Radar Data

• Pre-Operational Parallel Run : Q3 2011

• Operational Run : Q2 2012

12km Domain

1.5km Domain

Two Tracks on the Future NWP at KMA

• Successful UM operation on new supercomputer • Modification and Improvement

• Development of our own NWP System • Own Techniques PBL scheme, Microphysics scheme,

Gravity wave, own dynamics (DFS)

- 26 -

- 27 -

Development of an Original NWP System

Long-Term Development Plan (’11~’19)

• 1st Phase : Initial Development Stage

– Development of Core Modules

: Dynamics Core, Physical Parameterization Schemes, etc.

• 2nd Phase : Growing Stage

– Production of a Prototype Numerical Model

: Combining Developed Modules and Preliminary Test

• 3rd Phase : Mature (Pre-Operational) Stage

– Stabilization, Performance Enhancement

: Preparation for the Operational Implementation

# New organization out of KMA!

# Invitation for the position of the head of new organization soon!

Climate Prediction Division

CPD

LRF

Climate Outlook

International

Cooperation Climate Monitoring

Technical

Development

LRF and Climate Outlook

1-month Forecast

- 3rd, 13th and 23rd of the month

3-month Forecast

- 23rd of the month

Seasonal Climate Outlook

- Feb. for Summer, May for Autumn, Aug. for Winter,

Nov. for Spring

Procedure

Global SST Prediction Model

Global Dynamic Model

Statistical Analysis Model

APCC MME(15 model, 8 contries)

WMO LC-LRFMME(12 GPCs)

Discussion (CPD)

LRF ET Meeting (season) Int’l Joint Meeting (summer, winter)

Discussion

Press Release

Briefing

Disaster Prevention

System KMA Homepage Mass Media, …

「World Best 365」 하늘을 친구처럼, 국민을 하늘처럼

WMO LC-LRFMME

http://www.wmolc.org

No. of Members

: 106 members

from 45 countries

「World Best 365」 하늘을 친구처럼, 국민을 하늘처럼

Future Plan

UM based Atmospheric Model - Operational Forecast System is going to be replaced with

the UM based atmospheric model(N144L38) by 2011

GloSea4 - According to the agreement between UKMO and KMA

on the joint seasonal forecast system(’10.6.22),

KMA is currently working to move its seasonal forecast system

to the GloSea4(HadGEM3N96L85/ORCA1L75)

< Major Plan > - Jan 2011: Migration of KMA-GloSea4 from Cray-XT5 to Cray-XE6

- 2011: Testing of KMA-GloSea4 with real-time ICs from UKMO

- 2012: Semi-operational run of KMA-Glosea4 with real-time ICs from UKMO

- 2013: Operational run of KMA-GloSea4

Thank You for Your Attention

See you again in Korea or other countries!!

Recent Progresses in Numerical Weather Prediction and HPC ... · PDF fileRecent Progresses in...

Documents

Transcript of Recent Progresses in Numerical Weather Prediction and HPC ... · PDF fileRecent Progresses in...