Data merging benefits Globcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo Session 4...
-
Upload
audra-bridges -
Category
Documents
-
view
216 -
download
2
Transcript of Data merging benefits Globcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo Session 4...
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20071
Data merging benefits
Antoine Mangin, Stéphane Maritorena
Session 4 –GlobColour applications – November 21, 2007
for the users
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20072
Data merging benefits ?
• Improvement of spatial/temporal coverage
• Error bar estimates
• Trends analysis
Background
For the benefit of every user
For assimilation into models (but also to understand reliability of products)
For the benefit of reliable environmental reporting and carbon cycle studies
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20073
Data merging
• Improvement of spatial/temporal coverage
• Error bar estimates
• Trends analysis
It is a truism.
Outlines
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20074
Improvement of spatial/temporal coverage
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20075
Data merging benefits ?
• Improvement of spatial/temporal coverage
• Error bar estimates
• Trends analysis
It requests a careful analysis of error estimates as inputs of GSM
Outlines
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20076
Error bar estimates
L wN() source 1 LwN() source 2 L wN() source n
LwN () uncertainties source 1 LwN () uncertainties source 2 LwN () uncertainties source n
[L wN() source 1, L wN() source 2, ..., LwN() source n]
LwN () uncertainties source 1, LwN () uncertainties source 2, ..., LwN () uncertainties source n[ ]
GSM01 MODEL INVERSION
wNL bb
bb= f(
a + )
wNL
SeaWiFSMODIS
[Chl] (443)acdm (443)bbp
DATA SOURCES
CONCATENATED ARRAYS
[Chl] confidence interval (443) confidence intervala cdm (443) confidence intervalb bp
RETRIEVALS
Estimates of the uncertainties on input LwN
Estimates of the error model
Estimates of the uncertainties on outputs Chla, bbp,cdm
(Co-variance matrix between all ingredients)
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20077
Error bar estimates
Estimates of the uncertainties on input LwN
This is a direct result from the characterisation at sensors level (GC phase 1 ++)
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20078
Error bar estimates
Estimates of the full uncertainties as input of GSM
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 20079
Error bar estimates
Discussion
Interest: Relative importance (weight) of each wavelength in the inversion is a key element
Assumption: Main assumption is that there is no bias – input error is considered as a pure deviation defined by its standard deviation
Future: When we will reach the million match up points (or maybe before) – we should go to error estimates by classes of LwN
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200710
Error bar estimates - Validation
L wN() source 1 LwN() source 2 L wN() source n
LwN () uncertainties source 1 LwN () uncertainties source 2 LwN () uncertainties source n
[L wN() source 1, L wN() source 2, ..., LwN() source n]
LwN () uncertainties source 1, LwN () uncertainties source 2, ..., LwN () uncertainties source n[ ]
GSM01 MODEL INVERSION
wNL bb
bb= f(
a + )
wNL
SeaWiFSMODIS
[Chl] (443)acdm (443)bbp
DATA SOURCES
CONCATENATED ARRAYS
[Chl] confidence interval (443) confidence intervala cdm (443) confidence intervalb bp
RETRIEVALS
Estimates of the uncertainties on outputs Chla, bbp,cdm
(Co-variance matrix between all ingredients)
Use of the estimates
Nomad DB / EO data Extract samples of concomitent
LwN, Chla bbp and cdom
samples
If the Chla error estimates is reliable
should be close to a standard normal distribution
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200711
Error bar estimates - Validations
Inputs: In situ observations (Nomad)Results: very close to expectancy – no significant bias
0
10
20
30
40
50
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
Normalised std dev
Po
pu
lati
on
Results
Expected
Inputs: GC productsResults: very close to expectancy – a small bias is detected – the error estimates by GSM (with ad hoc inputs) is slightly underestimated.
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200712
Example of products uncertainties - daily
GlobColour Chla-merged product – May,15 2006
GlobColour Chla-merged product relative uncertainties – May,15 2006
100
50
0
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200713
GlobColour Chla-merged product – May 2006
Example of products uncertainties
GlobColour Chla-merged product relative uncertainties– May 2006
100
50
0
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200714
Data merging benefits ?
• Improvement of spatial/temporal coverage
• Error bar estimates
• Trends analysisDifferences between individual sensor time series for each sensors may (will?) lead to disturbances in merged time series.One aspect which is however not yet fully exploited is the correlation between individual sensor products which is rather good.
Outlines
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200715
Background
Context for this trend analysis: EEA reporting
In the frame of GSE Marcoast, reliability of EO to help environmental reporting is explored as well as consistency between missions to ensure continuity of the reporting.
Today the reporting is based on in situ observation and the metric for trend identification is, for a given area, the number of stations that have showed a significant increase/decrease of observed Chla (*) during the last 10 years.
(*) Observed Chla is an average seasonal value built on a very strict protocol.
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200716
Background – ingredients for reporting
14 eco-regions
About 6800 Chla samples in 2003-2005
… and thus the report
Within Marcoast we are working to replace/complement in situ sampling by EO (and here more specifically by
GlobColour)
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200717
Method used
Setting up of a non parametric test for detection of trends at GC pixel level.
The test is based on summation of sign of difference between one status and the previous ones (eg. season 2005 compared to 2004, 2003 etc..)
Statistical variance 2 of a white noise on such times series is analytically known.
So … any departure above (resp. below) 2 (resp. –2) from this law would indicate that a trend exist with a 95% significance level
Trends analysis
Important distinction: We are not trying here to quantify trends but to identify the probable ones.
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200718
2.5% 2.5%
Standard normal distribution
White noise at a level of significance of 95%
Trends detection
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200719
Discrepancies and Consistencies between instruments
MERISalone
MODISalone
2003-2006
2.5% 2.5%
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200720
MODIS SeaWiFS
MERIS GlobColour
Patchiness of MERISresults is probably
due to coverage
Possible trends are very consistent from one single sensor to the other
2003-2006
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200721
2.5% 2.5%
Trends detection – weighted average – the full game
Spatial consistency of possible trends are evidences of trends
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200722
Trends detection – GSM – the full game
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200723
Correlation coefficient for the seasonal figures
.6/.8/.6
.2/.5/.2
.8/.9/.9
.3/.6/.4
.8/.9/.8
.9/.9/.9.7/.8/.7
.8/.9/.8
.8/.8/.8.9/.9/.8
MERSWF/SWFMOD/MERMOD
.9/.9/.9
This gives a reasonible confidence level (or caution level!) in the merging of all sensors in order to identifiy trends
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200724
0% 60% 80%40% 100%20%
2003-2005 trends GC - 1998-2006 trends
Final reporting for EEA
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200725
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200726
Data merging benefits ?
• Improvement of spatial/temporal coverageYes, by construction
• Error bar estimatesA reliable error estimates has been derived through GSM – about to be submitted for publication
• Trends analysisAlthough GC has not yet the right quality for Climate change studies, it already provides means to detect evidence of trends for environmental reporting – under iteration with EEA within GSE-Marcoast
Outcomes/conclusions
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200727
A special thanks to
Christophe Lerebourg and Julien Demaria for data handling, impossible, hair-splitting and after-hours computations.
Acknowledgments
…. and thank you for your attention
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200728
0
10
20
30
40
50
60
70
80
90
100
-6 -4 -2 0 2 4 6 8
Results
Expected
2-sigma
2 - « sigma »
Error bar estimates – Validation – inputs : GC products
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200729
Impact of taking into account input Lwn uncertainties on:1. Retrieved Lwn (from GSM forward)
2. Retrieved Chla, CDM, Bbp GSM forward)
0.000
0.020
0.040
0.060
0.080
0.100
0.120
0.140
400 420 440 460 480 500 520 540 560 580
Wavelength (nm)
RM
S (
ab
s)
Noconsideration of error on inputLwn
Error model from characterisation isapplied on input Lwn
N RMS BIAS RATIO
Mean (%)
DIFF
Median(%)
DIFFMIN In
situMAX In
situmean In situ
median In situ
MIN Model
MAX model
mean model
median model r2 INTCPT SLOPE
CHL 259 0.21 -0.03 1.03 30.91 20.03 0.069 26.91 2.12 0.45 0.017 27.61 1.87 0.40 0.90 -0.03 1.01CHL (no error) 252 0.23 -0.09 0.91 30.92 22.62 0.069 26.91 1.87 0.41 0.019 28.48 1.58 0.29 0.89 -0.08 1.02
CDM 259 0.34 -0.05 1.14 49.29 26.93 0.008 1.97 0.15 0.04 0.001 1.38 0.13 0.04 0.74 0.29 1.26CDM (no error) 252 0.34 -0.04 1.18 51.15 27.98 0.008 1.99 0.13 0.03 0.001 1.38 0.12 0.04 0.72 0.34 1.28
BBP 26 0.103 -0.026 0.964 14.13 8.85 0.0011 0.0054 0.0028 0.0026 0.0012 0.0045 0.0025 0.0024 0.729 -0.994 0.627BBP (no error) 26 0.140 0.082 1.245 32.35 31.16 0.0008 0.0051 0.0022 0.0021 0.0012 0.0045 0.0025 0.0024 0.739 -1.278 0.497
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200730
Error bar estimates
Estimates of the model uncertainties
L wN() source 1 LwN() source 2 L wN() source n
LwN () uncertainties source 1 LwN () uncertainties source 2 LwN () uncertainties source n
[L wN() source 1, L wN() source 2, ..., LwN() source n]
LwN () uncertainties source 1, LwN () uncertainties source 2, ..., LwN () uncertainties source n[ ]
GSM01 MODEL INVERSION
wNL bb
bb= f(
a + )
wNL
SeaWiFSMODIS
[Chl] (443)acdm (443)bbp
DATA SOURCES
CONCATENATED ARRAYS
[Chl] confidence interval (443) confidence intervala cdm (443) confidence intervalb bp
RETRIEVALS
Nomad DBExtract samples of
LwN
Use of direct bio optical model to derive new LwN
Estimates of the error model
Data merging benefitsGlobcolour / Medspiration user consultation, Nov. 20-22, 2007, Oslo
Session 4 –GlobColour applications – November 21, 200731
IC ES Eco-region
0.0
0.2
0.4
0.6
0.8
1.0
Co
rrel
atio
n c
oef
fici
ent
A B C DE F G H I J K L M N
A: Greenland and Iceland SeasB: Barents SeaC: FaroesD: Norwegian SeaE: Celtic SeaF: North SeaG: South European Atlantic ShelfH: Western Mediterranean SeaI: Adriatic-Ioanan SeasJ: Aegean-Levantine SeasK: Oceanic Northeast AtlanticL: Baltic SeaM: Black SeaN: Azov Sea
MERSWF SWFMOD MERMOD
Correlation coefficient for the seasonal figures