Open Access to the GTS: A JCOMM Observations …meetings.wmo.int/JCOMM-5/TECO/Presentations/Session...

Open Access to the GTS: A JCOMM Observations Coordination Group Pilot Project

Kevin O’Brien University of Washington, USA NOAA/PMEL, USA

David Legler NOAA/OOMD, USA

Nadia Pinardi University of Bologna, Italy

Bill Smith, Michelle Little, Kevin Kern NOAA/NDBC, USA

Simona Simoncelli, Alessandro Grandi INGV, Italy

Vanessa Cardin, Alessandro Crise, Pierre Poulain, Elena Mauri OGS, Italy

Begona Perez, Fernando Manzano, Enrique Alvarez Fanjul Puertos del Estado, Spain

James Potemra University of Hawaii, USA

Eugene Burger, Heather Tabisola, Jessica Cross, Meghan Cronin, Noah Lawrence-Slavas NOAA/PMEL, USA

Goal is to provide data producers a simpler method of submitting real time data to GTS infrastructure and data consumers a simpler method of access

A pilot project to take well understood physical ocean data from known platform types and inject in near real-time onto the GTS for distribution globally.

Retrieve these and other data from the GTS and make available through interoperable web services.

*in situ marine observations and upper air observations are regarded as “essential” by WMO Resolution 40 and they can be exchanged without charge and with no conditions on use

JCOMM Open Access GTS Pilot Project

Introduction

• NOAA Observing System Monitoring Center (OSMC), USA

• Data management, encoding/decoding GTS data, providing GTS data access

• Puertos del Estado, Spain

• Moorings, tide gauges

• NOAA/PMEL, USA

• 5 SAILDRONE platforms (3 in Arctic, 2 in Tropical Pacific)

• National Institute of Oceanography and Experimental Geophysics (OGS), Italy

• E2M3A mooring

• US Integrated Ocean Observing System/PacIOOS, USA

• Bottom Pressure Recorder

• National Institute of Geophysics and Volcanology (INGV), Italy

• GTS Access and GTS data tests

• JCOMMOPS, France

• WMO IDs for GTS platforms


Partners

• Real time data (via GTS) has very high societal value • Local and global forecasts • Tsunami detection and warning • Resource allocation and management • Rescue missions at sea • Many more

• Carse, F., et al. (2015) Roquet, F., et al. (2013) discuss improvements using tagged seal data

• Carse, F. et al 1-6% improvement in root-mean-square error statistics using data provided by tagged seals

• Roquet,F. et al Inclusion of seal-derived data ‘substantially modifies the estimated surface mixed-layer properties and circulation patterns within and south of the Antarctic Circumpolar Current.’

• GTS is a convenient way to SHARE data GLOBALLY

• Barriers exist to distributing and accessing real time data on the GTS

Why?


GTS diagram – circa 1994 Courtesy of Mark Bushnell

• WMO ID required • Unique ID for every platform reporting on the GTS • For platforms such as Tide Gauges, can be unclear

who should generate a WMO ID

• Connection to National Data Center required

• Complexity of data format required • BUFR is replacing TAC

• Currently not within scope of short term or small

projects

Complications for GTS data insertion


Barriers

• How to get to GTS data? • BUFR decoders are required • Even decoded BUFR messages are difficult to fully

understand

• Data missing from GTS Feeds • Identifying new GTS headers difficult • Decoding issues can cause gaps

• Transition to BUFR highlights this in near term

• Area all GTS nodes created equal?

• How to ensure machine to machine access of data?

Complications for GTS data access


Barriers

• Platform observations and metadata to be available in known format (netcdf, csv, xml)

and from well –understood platforms • Metadata requires WMO ID to be assigned to platform • WMO IDs generated by JCOMMOPS

• Data and metadata will then be ingested into ERDDAP server/framework

• Participating GTS institution (NDBC) will use the defined ERDDAP API to access the

platform observation data and metadata

• Participating institution (NDBC) will encode the data and metadata using the appropriate BUFR template and submit data using custom GTS header (

• Participating institution will inject encoded BUFR message onto GTS

• This process will be automated for daily or more frequent data ingestion, as required


Data Ingestion

ERDDAP data platform

GTS

ENCODE into BUFR

Harvest from

ERDDAP

Transmit

to GTS

Add

WMO ID


Data Ingestion

WMO ID Platform Name Platform Type Institution Location (approx)

6101404 Algeciras Coastal Mooring PdE Med Sea

6201070 Langosteira Coastal Mooring PdE Atlantic (Spanish coast)

6101403 E2M3A E2M3A OGS Adriatic Sea

4802011 1001 Saildrone PMEL Arctic



5801950 1005 Saildrone PMEL TPOS

5801951 1006 Saildrone PMEL TPOS

5100275 Aloha Cabled Observatory

Bottom Pressure Recorder PacIOOS Hawaii

Gijon Tide Guage PdE

Ferrol Tide Guage PdE

Hielva Tide Guage PdE

Fuerteventura Tide Guage PdE

Palma de Malloca Tide Guage PdE


Partners

Volvo Ocean Race 2017- 2018 7 teams


Partners

• The OSMC project already pulls data

hourly from the GTS at the National Data Buoy Center and makes this data available through an ERDDAP framework

• The OSMC project will retrieve the newly inserted platform data from the GTS, decode the BUFR data with the appropriate decoder and then add the data and metadata to the OSMC database

• OSMC project will develop web-accessible tools and visualizations to illustrate the platform data in context with the other real time observations available from the GTS


Data Access

ERDDAP data

platform

ESRI Geoportal

Google Earth Live Access Server

iPython notebook

Matlab™

IOOS coastal

modeling

Metadata available

through international

standards

GTS

Data flows directly into

users’ applications

DECODE from BUFR


Data Access

Algeciras sst

Langosteira atmp

E2M3A atmp

Saildrone 2011 atmp

Saildrone 2012 atmp Saildrone TPOS atmp

Saildrone 2013 atmp

ACO sea water pressure

• Data Access test and comparison with Copernicus Marine Environment Monitoring service (CMEMS) and the Observing System Monitoring Center (OSMC) GTS feed for Mediterranean sea in month of September, 2017

• Simona Simoncelli and Alessandro Grandi, INGV


Data Access

• Ship, VOS, VOSclim data in GTS, not in CMEMS • Fixed buoys and moorings available in CMEMS not in GTS • Drifting buoys in CMEMS not in GTS (BUFR issue)? • Some CMEMS floats missing from GTS • No TSG or river flow data data in GTS


Data Access

Conclusions • Exercise illustrated importance of GTS data for forecasting activities

• Most GTS data available within 12 hours • 43% of CMEMS NRT data only available after 24 hours

• GTS availability is fundamental to increase predicative capabilities • Thanks to 2x daily observations available within 12 hours of

collection • Both GTS and CMEMS were missing some data types

• Data Access test and comparison with Copernicus Marine Environment Monitoring service (CMEMS) and the Observing System Monitoring Center (OSMC) GTS feed for Mediterranean sea in month of September, 2017

• Simona Simoncelli and Alessandro Grandi, INGV


Next Steps (ingestion)

• Lots of enthusiasm for the project • Support from JCOMM (Open Access GTS Task Team?)

• Need support from National Data Centers (GISC,

DCPC,NC) who would agree to harvest and encode data • Maybe the future would allow a “GTS encoder in a

box” - not ready now

• Need to entrain individual data collectors or global GTS “Data Assembly Centers” in use of ERDDAP


Participating National Data Center

GTS Data Assembly Center

Entrain/Enable GTS “Data Assembly Centers”




• NetCDF/HDF as potential GTS transport format • WMO has formed the Inter-Programme Expert Team on

Operational Weather Radars (IPET OWR).This team is evaluating CfRadial2 (based upon HDF/netCDF) as a probable candidate to replace the BUFR.

• GTS WIS • Task Team on Evolution of WIS (TT eWIS) • Development of WIS 2.0


Next Steps (access)

• Need support from National Data Centers (NCs, DCPCs) who would agree to decode and provide access to data • US NOAA Observing System Monitoring Center

(OSMC) already does this through ERDDAP server


Next Steps (access)


Next Steps (access)

• Need support from National Data Centers (NCs, DCPCs) who would agree to decode and provide access to data • US NOAA Observing System Monitoring Center

(OSMC) already does this through ERDDAP server

• Further investigation of GTS as a distribution mechanism • Develop GTS data flow metrics

• Recommend ERDDAP as the data platform for uniformity of both ingest and access of GTS data • Would integrate with current delayed mode DM

activities and strategy supported by JCOMM OCG

Open Access to the GTS: A JCOMM Observations Coordination Group Pilot Project

Thank You! Kevin.M.O’[email protected] [email protected]

Open Access to the GTS: A JCOMM Observations …meetings.wmo.int/JCOMM-5/TECO/Presentations/Session...

Documents

Transcript of Open Access to the GTS: A JCOMM Observations …meetings.wmo.int/JCOMM-5/TECO/Presentations/Session...