Results from the THORPEX Observation Impact Inter-comparison Project

Ron Gelaro1, Rolf Langland2, Simon Pellerin3, Ricardo Todling1

1NASA Global Modeling and Assimilation Office (GMAO)2Naval Research Laboratory (NRL)

3Environment Canada (EC)

Third THORPEX International Science SymposiumSept 2009 – Monterey, CA

Background / Motivation

A goal of THORPEX is to improve our understanding of the ‘value’ of observations provided by the current global network

DAOS WG proposed, as a starting point, a comparison of observation impacts in several forecast systems, facilitated by the emergence of new (adjoint-based) techniques

Experiments for a ‘baseline’ observation set were designed by NRL, NASA/GMAO, EC, ECMWF and Météo France …this talk presents results from 3 systems: NRL, GMAO, EC

• optimize the use of current observations• inform the design, deployment of new obs systems

Acknowledgements: Carla Cardinali (ECMWF), Pierre Gauthier (EC, UQAM), Florence Rabier (Météo France)

With millions of observations assimilated every analysis cycle, how do we quantify the value provided by these data?

The Challenge

QUESTIONS:

• How similar or different are the impacts of observations in one forecast system vs. another?

• Which observation types have the largest total impacts, and impacts per-observation?

• How do observation impacts vary as a function of location, channel, or other relevant attribute?

DATA DENIAL / ADDITION EXPERIMENTS (OSEs)

Typically done for a few sets of observation types

Not feasible to examine impact of ALL observations

Valid for any forecast range or measure

ADJOINT-BASED OBSERVATION IMPACT

Can evaluate impacts of all observations on a selected measure of short-range forecast error

Computationally efficient and accurate, but subject to assumptions and limitations in adjoint model

Method described in Langland and Baker (Tellus 2004) and subsequent papers by Errico 2007; Gelaro et al. 2007

Methods for Evaluating Observation Impact

Methods compared by Gelaro and Zhu (2009), Cardinali (2009)

◄

Definition of Observation Impact

Assimilation of observations moves the model state from the background trajectory to the new analysis trajectory

Energy-weighted forecast error norms

Observation impact is quantified as the difference in forecast error norms

It may be estimated as a sum of contributions from individual obs using information from the model and analysis adjoints

ff eee ba

eaf

ebf

xa

xb

xt

e

t = 0 t + 24ht 6h

observations assimilated

adjoint analysis scheme innovations

adjoint model

Calculation of Observation Impact

Based on LB04, or an extension of that method

where

Impact computed for all observations simultaneously

The impact of any subset of observations, S, can be easily quantified using a partial sum

Experiment for the Baseline Observation Set

January 2007, every 6 hours, 124 assimilation times

Baseline set of observations assimilated (next slide)

24h global forecast error - Dry total energy norm

Dry physics in adjoint model

NRL – NOGAPS forecast and adjoint T239L30NAVDAS 3D-Var analysis and adjoint 0.5°

GMAO – GEOS-5 forecast 0.5°, adjoint 1.0°GSI 3D-Var analysis and adjoint 0.5°

EC – Meso-Strat GFS forecast 0.4°, adjoint 1.5°4D-Var analysis and adjoint 1.5°

Observations in Baseline Comparison

Conventional observations

– Raobs, pibals, dropsondes– Commercial aircraft – Land surface – Ship surface– Buoys

Satellite observations

– AMSU-A (3 satellites)– Geo-satellite winds

Vis, IR and WV– MODIS polar winds – SSM/I surface wind speed – QuikScat surface wind

Each center used their usual data selection and QC procedures, error statistics and observation operators

NRL and GMAO assimilated SSM/I speeds but no Profiler winds; EC did the opposite

GEOS-5 NOGAPS

Forecast error on Background Trajectory,

Forecast error on Analysis Trajectory,

Difference of nonlinear forecast error norms

Adjoint-based estimate of global observation impact

Global domain: All assimilation times Jan 2007

Time series of forecast error norms

febfea

ff ee ba ff ee ba

FCST ERROR REDUCTION

GEOS-5 NOGAPS

Global domain: 00+06 UTC assimilations Jan 2007

Daily average observation impacts

EC-MSGFS

AMSU-A, Raob, Satwind and Aircraft have largest impact in all systems

Smaller impact of Satwind in GEOS-5 than in other systems

FCST ERROR REDUCTION

Global domain: 00+06 UTC assimilations Jan 2007

FCST ERROR REDUCTION

GEOS-5

Impacts per-observation

NOGAPS

EC-MSGFS

GEOS-5 has smaller impact per-ob, because more observations are assimilated (next slide) …recall TOTAL impacts are similar

FCST ERROR REDUCTION

>30!

Very large impact per-ob for Ships in EC system is an outlier

Global domain: 00+06 UTC assimilations Jan 2007

GEOS-5

Observation Counts

NOGAPS

EC-MSGFS

GEOS-5 assimilates more obs overall, especially AMSU-A, Aircraft, QuikScat

NOGAPS assimilates more Satwinds

Global domain: 00+06 UTC assimilations Jan 2007

Fraction of obs that reduce forecast error

GEOS-5 NOGAPS

EC-MSGFS

All observation types (except SSMI speeds in GEOS-5) are in the range of 50% to 54% beneficial

FCST ERROR REDUCTION

FCST ERROR REDUCTION

FCST ERROR INCREASE

FCST ERROR INCREASE

GEOS-5 NOGAPS

N18 AMSUA Ch 7 N18 AMSUA Ch 7

Raob T 300-700 hPa Raob T 300-700 hPa

Global domain: 00 UTC assimilation 21 Jan 2007

Scatter of observation impact vs. innovation

Most total forecast error reduction comes from observations with moderate-size innovations – not from outliers with very large positive or negative innovations

Global domain: 00+06 UTC assimilations Jan 2007

GEOS-5 NOGAPS

Observation impacts: AMSU-A channels 4 -11

Large impact from channels 5, 6 and 7 in all systems

EC-MSGFS

Small impact of channel 8 in NOGAPS; significant impact of channel 9 in EC-MSGFS

GEOS-5 NOGAPS

Observations that produce large forecast error reductions

Observations that produce forecast error increases in both models …land or ice surface contamination of radiance data?

Observation impacts: NOAA-18 channel 7Global domain: 00+06 UTC assimilations Jan 2007

Comparison experiments for GMAO, NRL and EC systems completed for baseline set of observations

Despite differences in algorithms, RTMs and data handling, overall quantitative results similar for all systems; but details of impact differ (e.g., impact-per-ob, channels)

Largest impacts provided by AMSU-A and raobs (GMAO), AMSU-A and satwinds (NRL, EC); aircraft also has large impact in all systems

Only a small majority (50-55%) of assimilated observations improve the forecast…targeting ramifications?

Common problem areas with AMSU-A noted; handling of surface properties a likely cause

Future study to include other models(?) and more recent observation types…

Summary of Comparison Results to Date