Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007...

Meeting Description

8th GHRSST-PP Science Team Meeting Melbourne, Victoria - 14th - 18th May 2007Status and Application of the HR-DDS

Dave Poulter

Ocean Observing and Climate Group

and Laboratory for Satellite Oceanography

National Oceanography Centre, Southampton

UK

[email protected]

http://www.hrdds.net

Project LogoQuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

mailto:[email protected]

http://www.hrdds.net/

Meeting Description


Dave Poulter

Status and Applications of the HR-DDS



QuickTime™ and aTIFF (Uncompressed) decompressor


Meeting Description


The HR-DDS concept



The HR-DDS system is a distributed archive and analysis system for subsets of GHRSST-PP and other data.


are needed to see this picture. QuickTime™ and aTIFF (Uncompressed) decompressor



are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

Satellite SST ObservationModel output and estimationsIn situ obervations






Meeting Description


The HR-DDS concept



Data are ingested in real time, re-gridded to a common 0.01 degree grid and reproduces in standard format over 200 2 by 2 degree areas.

















Meeting Description


The HR-DDS concept



Allows for easy intercomparison of data interactively online














Meeting Description


The HR-DDS concept



Highly configurable and expandable analysis system



Meeting Description


Why do we need and HR-DDS?



Meeting Description


Status Overview - A years progress!



•A new look GHRSST ‘style’ website

•Inclusion of L2Pcore and non GHRSST datasets

•Enhanced statistical analysis functions

•New and far more powerful graphics engine

•Redesigned processing backend - now DISTRUBUTED.

•Prototype climatology datasets

•Semantic content awareness

•Prototype real time data metrics

•Prototype direct data downloads

Meeting Description


New Distributed ArchitectureThe system is now distributed: There are many possible modes - this is a single computer HR-DDS system



SQLRegister

&Stats DB

FTPDODS

THREADSHTTP

ProcessingSystem

Meeting Description


New Distributed ArchitectureThis is the currently system running at NOCS



SQLRegister

&Stats DB

FTPDODS

THREADSHTTP

ProcessingSystem

SQLRegister

&Stats DB

ProcessingSystem

BackupDB

BackupArchive

Meeting Description


New Distributed ArchitectureThis is planned system architecture at NOCS



FTPDODS

THREADSHTTP

MODISProcessing

System

BackupProcessing

SystemBackup

DB

Register & Stats

DB

Re - Processing

System

PrimaryProcessing

System

BackupArchive

Meeting Description


New Graphics EngineThe new graphics engine allows us, for the first time, to have the following:





Error bars

Readabledates

Line styles

Font styles

Meeting Description


New Graphics EngineSignificantly, the new engine reads the netCDF HR-DDS files and generates maps on the fly.

This means that there is no need for the generation of quick-look image files - the webpage is entirely dynamic.

The major advantage is that the page can draw other fields than SST.



Meeting Description


Enhanced Statistical AnalysisThe HR-DDS can now plot apply bias estimates online






Meeting Description


Enhanced Statistical AnalysisThe HR-DDS website also records ‘centre only’ values






Meeting Description


Enhanced Statistical AnalysisAnd also ‘best quality only’ values - based on native quality flags






Meeting Description


A New Look HR-DDS Website



Meeting Description


L2P Core and Non Medspiration Data:

• Ingesting L2PCore AMSRE data from REMSS and JAXA

• Ingesting FOAM 1 degree and 1/9 degree 5m Temperature

• Ingesting AVHRR 18 LAC from NAVO

• Ingesting AVHRR 17 and 18 GAC from NAVO



Meeting Description


Climatologies Included:The system now includes the Pathfinder 4km Pentad climatology and the Reynolds 1 degree daily climatology:





Meeting Description


Semantic Content:The new design contains an interpretive layer in the backend, which ensures that ‘user’ concepts such as ‘sea surface temperture’ can equate to variable names such as ‘analysed_sst’, ‘foundation_sst’ and ‘sea_water_potential_temperature’.

Similarly - the database is aware of SST type (foundation, skin etc), SST origin (model, satellite, in situ etc) and depth (skin, 1m, blended etc)



Meeting Description


Analysis on demand:The following plots were generated by our prototype system, and will soon be available on demand.

The exact specification of these plots can be set up specifically for any user - proving a user driven metric.



Meeting Description


Analysis on demand:Standard deviation of night time SEVIRI compared to AATSR match ups. Centre values only, all quality levels. No bias correction applied and match ups +/- 3 hours only.



QuickTime™ and aTIFF (LZW) decompressor


This took 25 seconds!

Meeting Description


Analysis on demand:Bias of OSTIA to AATSR - same conditions. Blue = OSTIA Cool





15 seconds!

Meeting Description


Analysis on demand:Number of AVHRR 18 L match ups - same conditions.





38 seconds

Meeting Description


Analysis on demand:I just thought of this at lunch!:

This system could be used to return the URL of L2P files where diurnal warming is likely have occurred - in rank order:

Rank Number of Sites URL

1 38 http://www.misst.org/data…../data1.nc

2 25 http://www.misst.org/data…/data45.nc



Meeting Description


Quality Control Services:



Provided format specifications and corrections for UK Remote Sensing and Data Analysis System for AVHRR LAC products

The HR-DDS system has provided quality control and format specification to JAXA for AMSR-E L2PCore data

Meeting Description


The Future:The HR-DDS has secured funding from the European MyOceans project.



We are secure for at least end 2011. Money is set aside for operation maintenance, reprocessing and continual development.

The HR-DDS is moving from the University of Southampton to the UK Strategic Research Division

There is a clear remit with the project for user driven metrics, for example ‘traffic light’ performance indicators.

Meeting Description


The Future II:There have been good communications with the ESA GLOBColour project



A merged interface has been defined - GHRSST users will be able to retrieve OC data from the GLOBColour DDS using the GHRSST HR-DDS system.

The GHRSST HR-DDS has mirrored all GLOBColour HR-DDS sites.

Similar experimental interfaces have been defined for the MDB.

Meeting Description


What can we do for you?

You are the HR-DDS users! We are now in a position to offer specific services to specific users - G8 priority #7 ‘user led applications’.

We can provide an email alert to you if you want to know when AVHRR17 is more than a degree warmer that SEVIRI within a 1 hour match up criteria.

We can ingest data from YOUR buoy array…..

We can provide daily maps of coral bleaching conditions……



Meeting Description


Acknowledgements!

Fred Wimmer and Chris Jefferys at NOCS have helped with development.

Most science team members have helped at some point or another!



Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007...

Documents

Transcript of Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007...