1

NOAA Testbeds and Proving Grounds:Fostering Transitions between Research

and Applications/Operations

Paula Davidson, Ph.D. Chief ScientistOffice of Science & TechnologyNOAA/National Weather Service Co-chair, NOAA Testbeds & Proving Grounds Coordinating Committee

2

Background

NOAA operates a number of testbeds (TB) and operations and services proving grounds (PG), but lacked a systematic approach for their function, mode of operation, and reporting results

March 2011- NOAA Line Office Transition Managers approved guidelines and chartered a Coordinating Committee for Testbeds & Proving Grounds (TB/PG)

• Guidelines:• Roles and responsibilities of TB/PG • Function and execution• Governance

• Coordination:• Increasing R&D leveraging, collaboration for R2O

Increasing reliance on TB/PG for advancing R2O/A/U • NOAA re-chartered COMT (2013) • CSTAR FFO links results to testing in OPG; use of NWS Virtual Laboratory (FY14)• Planning for Next Generation Global Prediction System

Brief working definitions of TB/PG:TB-- for developmental testingPG-- for pre-deployment (experimental)

testing of advanced operations, services and science and technology capabilities

R&D R&D

3

NOAA Testbed and Proving Ground Portal

                                                                                                                                            

                                                                                                                                            

                                                                                                                                            

                                                                                             

http://www.testbeds.noaa.gov (thanks to : Rich Lataitis & Barb deLuisi)

AWT CTB COMT DTC

GRPG HWT HMT JCSDA

JHT OPG SWPT

Current NOAA TB/PGCurrent NOAA TB/PG

4

Framework for Transition:Phased Implementation into NWS Operations

Phase Key Q Key Metric FacilityR&D Does it work? Peer-reviewed

PublicationUniversities, Government Labs, Private Industry

Developmental Testing

Works with operational systems?

Feasibility/ Engineering Analysis Successful

Testbed with operations-like environment

Experimental Testing

Meets operational performance criteria?

Go/No Go based on: Objective Performance (e.g. accuracy)

Subjective Feedback

Production Readiness

Operational proving ground for clinical tests and full “dress rehearsal”

Operations Maintains required performance?

Objective criteria: accuracy and reliability

Operations

TB

PG

5

Coordinating Committee

Comprised of TB/PG managers and LOTM-designated LO focal points: Co-chairs: Paula Davidson (NWS LO focal point) Mike Uhart (OAR LO focal

point)

Initial meeting (NWS and OAR members): Seattle AMS, Jan 2011 Featured Activities:

Monthly virtual meetings featuring TB/PG recent tests/results Coordination/Outreach: revamped TB/PG coordination website www.testbeds.noaa.gov,

coordinated SEE inputs, tutorials on guidelines, testing protocols , coordinated AOs Organize annual science-themed TB/PG workshops Adopted recommended approach to metrics for recognizing progress, Fall 2012:

Relevance, quality, and effectiveness/efficiency Accomplishments and performance metrics provided in annual TBPG progress reports

Summary publication (Ralph, Intrieri, et al.): The emergence of weather-focused testbeds linking research and forecasting operations. BAMS, 94, 1187-1210, 2013

6

Annual Workshops

Highlight/promote integrated testing opportunities in NOAA TB/PG Overviews from current/emerging TB/PG Integrating Science Themes:

3rd Annual TBPG Workshop: Intense Precipitation Events (2012) 4rd Annual TBPG Workshop: High-impact Environmental Events (2013) 5th Annual TBPG Workshop: Advances in Environmental Prediction (2014)

Best paper (Science-theme) competition Awarded to Faye Barthold (WPC, 2012); Tom Workoff (WPC, 2013); Chad

Gravelle (OPG, 2014) Criteria - excellence, relevance, presentation

7

FY 2014 Activities

TBPG Workshop: College Park, MD, April 16-18 Science Theme: Advances in Environmental Prediction Best Paper: Chad Gravelle, “Transitioning the GOES-R Fog and Low Stratus

Products from Research To Operations Through the NWS Operations Proving Ground”

Closer connection with CSTAR , MDL’s Virtual Laboratory and pilot activities: advanced testing

Increasing coordination, external engagement Combined Special Testing Opportunities In-residence and virtual testing Consolidation on Website: Announcements of Opportunity

8

Contact Information:Paula Davidson – Paula.Davidson@noaa.gov

Thank You!

9

Backup

10

FY13 Highlights: COMT FY13 Highlights: COMT • COMT results featured in a special issue of JGR-Oceans:

11

FY13 Highlights: DTC FY13 Highlights: DTC

Hurricane-WRF (HWRF) testing for operational implementation at NCEP

Operational forecastshttp://www.emc.ncep.noaa.gov/gc_wmb/vxt/

g

• HWRF provides guidance to the National Hurricane Center for the North Atlantic and Eastern North Pacific basins

• Upgraded HWRF increased skill in forecasts of hurricane intensity

• Regional model 27/9/3 km• HWRF has 8 components, many

used in other applications• Developmental Testbed Center

roles:– Support code to community– Code management– Testing and evaluation (R2O)

12

FY13 Highlights: GOES-R PGFY13 Highlights: GOES-R PG• NWS Central Region Fog & Low Stratus Evaluation, Aug-Dec 2012

• Products demonstrated successfully – scheduled for delivery via NWS operational systems• High latitude & Arctic testing

• snow/fog, low stratus/volcanic ash, aviation applications• GOES-R Algorithm Development & Testing

• Simulated satellite imagery: lightning mappers, nearcasting data, convective initiation, cloud-top cooling

• Partnerships • NWS: NCEP/AWT, HWT, WPC, OPC, Regions; NESDIS : SAB; CIRA, CIMSS, NASA/SPoRT, DoD,

private sector…

courtesyMike Pavalonis, NESDIS/STAR

Heritage GOES FogDetection

GOES-R AlgorithmApplied to GOES

GOES-R AlgorithmApplied to GOES-R

13

• 26 forecasters, researchers and model developers from WPC, NSSL, ESRL and HMT collected to examine alternatives for short term QPF and flood prediction

• Several operational and research models tested

12-hr mean QPF

valid 2013 07/18 0Z

FY13 Highlights: HMTFY13 Highlights: HMT

Flash Flood and Intense Rainfall Experiment (July 8-26, 2013)

Operational forecastshttp://www.emc.ncep.noaa.gov/gc_wmb/vxt/

QPE NAM- X Nest HRRR

NAM Nest NSSL WRF HRW ARW

14

FY13 Highlights: HWTFY13 Highlights: HWT

• LAPS Space and Time Multiscale Analysis System (STMAS)• Forecast return moisture and instability very useful for determining if storms will change

intensity in 0-2 hours• WDSSII Multiple-Radar/Multiple Sensor (MRMS) applications

• 2-minute updates of MRMS products helps keep situational awareness high• Greater efficiency in storm diagnosis• Aided in positioning of storm-based warning polygons

• Hail Size Discrimination Algorithm (HSDA)• Very beneficial in breaking down areas of

severe vs. non-severe hail• Spatial extent of giant hail tended to be too

large at times• GOES-R Algorithm Development and

Testing• Simulated satellite imagery: lightning mappers,

nearcasting data, convective initiation, cloud-top cooling

Experimental Warning Testing**

** additional highlights from experimental forecast testing not shown

15

Testing new snow depth algorithms led to improved snow characterization in land-surface component (NOAH) for NCEP’s operational GFS (release v3.5) and WRF models.

Improved prediction of snow depth by NOAH/GFS

•U of Arizona effort, funded by NOAA’s JCSDA external research program•Collaboration with NESDIS CRTM and NWS land data assimilation teams •Improved roughness length led to increased skill in snow depth prediction by GFS/NOAH and WFR/NOAH•Implemented in GFS/NOAH land model (latest release, v 3.5).

Black: observationsGreen: control GFSYellow: GFS + improved roughness lengthRed: GFS + all improvements

averaged (over 100-120degW, 50-60degN) for 3/2011JCSDA annual report; figure courtesy of X. Zeng, U Arizona

FY 13 Highlights: JCSDAFY 13 Highlights: JCSDA

16

11 Joint Hurricane Testbed projects completed their two years of testing during the 2013 hurricane season

Uhlhorn , Hurricane Research Division: Stepped-frequency microwave radiometer bias-corrected hurricane winds

Ginis/Bender, U Rhode Island/ Geophysical Fluid Dynamics Laboratory: Improved hurricane models

FY 13 Highlights: JHTFY 13 Highlights: JHT

17

• Virtual AWIPS proof of concept

• GOES-R Fog/Low Stratus Tool

• Support for AWC TB Experiment

• Programming for Winter Hazard Message Simplification• Finalizing Test Plan with NASA SPoRT/GOES-R for

Operational Readiness Evaluation in March 2014• Test plan developed for Hazards Services

Hydro Tool

FY 13 Highlights: OPGFY 13 Highlights: OPG

18

FY13 Highlights: SWPTFY13 Highlights: SWPT

• Tested upgrades to operational Wang-Sheeley-Arge Enlil model for prediction geomagnetic storms, now implemented

• Tested new coronal mass ejection analysis tool, now implemented • Tested Space Environmental Anomalies Expert System Real Time

for GEO satellite operators, now implemented• Evaluated and selected magnetosphere/geospace model for

transition to operations (with NASA)• Tested ionosphere and thermosphere

products • Ovation Prime aurora forecast model, now

implemented• Upgrade/expansion to US Total Electron Content

model – N America domain ready for implementation; global product in testing