Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Copernicus Global Land Operations
“Cryosphere and Water” ”CGLOPS-2”
Framework Service Contract N° 199496 (JRC)
QUALITY ASSESSMENT REPORT
SNOW COVER EXTENT
COLLECTION 500M CONTINENTAL EUROPE
VERSION 1.0
Issue I1.02
Organization name of lead contractor for this deliverable: ENVEO IT GmbH
Book Captain: Gabriele Schwaizer (ENVEO)
Contributing Authors: Elisabeth Ripper (ENVEO)
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 2 of 31
Dissemination Level PU Public X
PP Restricted to other programme participants (including the Commission Services)
RE Restricted to a group specified by the consortium (including the Commission Services)
CO Confidential, only for members of the consortium (including the Commission Services)
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 3 of 31
Document Release Sheet
Book captain: Gabriele Schwaizer Sign Date
Approval: Roselyne Lacaze Sign Date
Endorsement: Mark Dowell Sign Date
Distribution: Public
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 4 of 31
Change Record
Issue/Rev Date Page(s) Description of Change Release
03.03.0217 All First Issue I1.01
08.05.2017 All Update after external review I1.02
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 5 of 31
TABLE OF CONTENTS
1 Background of the document .................................................................................................. 9
1.1 Executive Summary ..................................................................................................................... 9
1.2 Scope and Objectives ................................................................................................................... 9
1.3 Content of the document............................................................................................................. 9
1.4 Related documents .....................................................................................................................10
1.4.1 Applicable documents ...........................................................................................................................10
1.4.2 Input .....................................................................................................................................................10
1.4.3 Output ..................................................................................................................................................10
1.4.4 External documents...............................................................................................................................10
2 Review of Users Requirements .............................................................................................. 11
3 Quality Assessment Method .................................................................................................. 12
3.1 Overvall procedure .....................................................................................................................12
3.2 Satellite Reference Products .......................................................................................................14
3.2.1 Landsat scenes ......................................................................................................................................14
3.2.2 Very high resolution optical satellite data ..............................................................................................17
4 Results ................................................................................................................................... 19
5 Conclusions ............................................................................................................................ 28
6 Recommendations ................................................................................................................. 30
7 References ............................................................................................................................. 31
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 6 of 31
List of Figures
Figure 3.1: Validation concept developed in the ESA QA4EO project SnowPEx (lead: ENVEO) for
the validation of the SCE product with snow reference products from high resolution optical
satellite data........................................................................................................................... 14
Figure 3.2: Overview on distribution of Landsat acquisitions selected for the generation of
reference snow maps. The fill color of each scene indicates the year of the acquisition. Black
outlined scenes show the footprint of a scene. A red outlined scene shifted towards the south-
west of a basic footprint represents a further scene available for this location of the year as
indicated by the fill color. ........................................................................................................ 16
Figure 3.3: Overview on spatial distribution of very high resolution optical satellite data (red
outlines) provided courtesy by the Copernicus Data Warehouse during the EU FP7 project
CryoLand for validation purposes. The fill color of each scene indicates the year of the
acquisition. Black outlined scenes show the footprint of a scene. A red outlined scene shifted
towards the south-west of a basic footprint represents a further scene available for this
location of the year as indicated by the fill color. .................................................................... 18
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 7 of 31
List of Tables
Table 3.1: Band numbers, wavelengths and resolution of Landsat 5 Thematic Mapper (TM),
Landsat 7 Enhanced Thematic Mapper plus (ETM+), Landsat 8 Operational Land Imager
(OLI) and Thermal Infrared Sensor (TIRS), and Sentinel-2 Multi-Spectral Instrument (MSI). . 15
Table 4.1: Summary of statistical measures resulting from the validation of the SCE product over
continental Europe with snow maps from each the total area of 110 selected Landsat scenes
and 44 very high resolution optical satellite scenes. Results are generated by ENVEO during
the ESA QA4EO project SnowPEx, and the EU FP7 project CryoLand. ................................. 19
Table 4.2: Summary of statistical measures resulting from the validation of the SCE product over
continental Europe with 110 selected Landsat scenes, separating the total non-forested and
total forested areas classified from each Landsat scene. Results are generated by ENVEO
during the ESA QA4EO project SnowPEx. ............................................................................. 20
Table 4.3: Statistical measures of the fractional snow cover extent (snow on ground) vs. Landsat
snow algorithms with the same quantity displayed from January to July (independent of the
year). Results are generated by ENVEO during the ESA QA4EO project SnowPEx. ............. 21
Table 4.4: Statistical measures derived from the validation of the SCE product over continental
Europe with reference snow maps generated by three different algorithms from Landsat
scenes. Only values for the total area covered by each scene are provided in the table. R =
correlation coefficient. Results are generated by ENVEO within the ESA QA4EO project
SnowPEx. .............................................................................................................................. 22
Table 4.5: Statistical measures derived from the validation of the continental European fractional
snow cover products with snow maps generated manually from very high resolution optical
satellite data. Analyses are performed by ENVEO within the EU FP7 project CryoLand. ....... 25
Table 4.6: Validation results of fractional SE products detecting snow on ground vs. LS snow
algorithms for different fraction sections: 0-25 %, 26-50 %, 51-75%, 76-100 %. Results are
generated by ENVEO during the ESA QA4EO project SnowPEx. .......................................... 27
Table 5.1: Compliance matrix for the continental European SCE product. .................................... 29
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 8 of 31
List of Acronyms
ABBREV. MEANING
ATBD Algorithm Theoretical Basis Document
ESA European Space Agency
ETM+ Enhanced Thematic Mapper plus
EU European Union
FP7 7th Framework Program of the European Community for research,
technological development and demonstration activities
FSC Fractional Snow Cover
MODIS Moderate Imaging Spectroradiometer
MSI Multi-Spectral Instrument
NASA National Aeronautics and Space Administration
NDSI Normalized Difference Snow Index
NDVI Normalized Difference Vegetation Index
NIR Near Infrared
NRT Near-Real Time
OLI Operational Land Imager
QA4EO Quality Assessment for Earth Observation
QAR Quality Assessment Report
RMSE Root Mean Square Error
SCE Snow Cover Extent
SRTM Shuttle Radar Topography Mission
SWIR Shortwave Infrared
TIR Thermal Infrared
TIRS Thermal Infrared Sensor
TM Thematic Mapper
VHR Very High Resolution
VIS Visible
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 9 of 31
1 BACKGROUND OF THE DOCUMENT
1.1 EXECUTIVE SUMMARY
The Global Land (GL) Component in the framework of Copernicus is earmarked as a component of
the Land service to operate “a multi-purpose service component” that will provide a series of bio-
geophysical products on the status and evolution of land surface at global scale. Production and
delivery of the parameters are to take place in a timely manner and are complemented by the
constitution of long term time series.
The Copernicus Global Land Service contains a Snow Cover Extent (SCE) Near Real Time (NRT)
product extending continental Europe at 0.005 degree resolution. The SCE is derived from medium
resolution optical satellite data, currently from Terra MODIS (Moderate resolution Imaging
Spectroradiometer). The spectral capabilities of the sensor are used in combination with a digital
elevation model, a transmissivity map describing the transmissivity of forest canopy, and surface
classification maps to detect the fraction of snow cover per pixel in percentage. The usage of the
transmissivity map allows also in forested areas to detect the snow on the ground.
This Quality Assessment Report describes the validation activities and results performed so far for
the Continental European SCE product at 0.005 degree resolution.
1.2 SCOPE AND OBJECTIVES
The document presents the results of the quality assessment of the central European SCE500 V1
product.
The quality assessment is performed on the SCE product from 2000 to 2014, covering continental
Europe (72N 11W – 35N 50E) with 0.005 degree resolution using snow maps generated from high
resolution optical satellite data acquired between 2000 and 2014 over different surface types and
at different snow conditions over Europe.
The objective is to check that the overall product performance meets the quality requirements
defined by users.
1.3 CONTENT OF THE DOCUMENT
This document is structured as follows:
• Chapter 2 recalls the users requirements, and the expected performance
• Chapter 3 describes the methodology for quality assessment, the metrics and the criteria of
evaluation
• Chapter 4 presents the results of the analysis
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 10 of 31
• Chapter 5 summarizes the main conclusions of the study
• Chapter 6 makes recommendations based upon the results
1.4 RELATED DOCUMENTS
1.4.1 Applicable documents
AD1: Annex I – Technical Specifications JRC/IPR/2015/H.5/0026/OC to Contract Notice 2015/S
151-277962 of 7th August 2015
AD2: Appendix 1 – Copernicus Global land Component Product and Service Detailed Technical
requirements to Technical Annex to Contract Notice 2015/S 151-277962 of 7th August 2015
AD3: IGOS, 2007: Cryosphere Theme Report 2007. WMO/TD-No 1405. 100 pp.
AD4: GCOS, 2006: Systematic observation requirements for satellite-based products for climate.
WMO/TD Report No. 1338, September 2006.
1.4.2 Input
Document ID Descriptor
CGLOPS2_SSD Service Specifications of the Global Component of
the Copernicus Land Service.
CGLOPS2_SVP Service Validation Plan of the Global Land Service
CGLOPS2_ATBD_SCE500-
CEURO-500m_V1
Algorithm Theoretical Basis Document of the
SCE500-CEURO-500m_V1 product
1.4.3 Output
Document ID Descriptor
CGLOPS2_PUM_SCE500-
CEURO-500m_V1
Product User Manual summarizing all information about the
SCE500-CEURO-500m_V1 product
1.4.4 External documents
ED1: Malnes, E., Buanes, A., Nagler, T., Bippus, G., Gustafsson, D., Schiller, C., Metsämäki, S.,
Pulliainen, J., Luojus, K., Larsen, H.E., Solberg, R., Diamandi, A., Wiesmann, A. (2015). User
requirements for the snow and land ice services – CryoLand. The Cryosphere, 9, 1191–1202,
doi:10.5194/tc-9-1191-2015.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 11 of 31
2 REVIEW OF USERS REQUIREMENTS
According to the applicable documents [AD3] and [AD4], and the external document [ED1], the
user’s requirements relevant for the Snow Cover Extent product for continental Europe are:
• Definition: Continental European Snow Cover Extent (SCE)
• Geometric properties:
o Spatial resolution: Minimum: 500 m, Target: 100 m
o Grid / Projection: Geographic coordinates (latitude/longitude), WGS84 datum,
WGS84 ellipsoid
• Geographical coverage: Continental Europe (72°N / 11°W – 35°N / 50°E)
• Accuracy requirements:
o Geometric accuracy: Target: Sub-pixel location error
o Thematic accuracy: Minimum: 15 %, Target: 5 %
• Temporal requirements:
o Time period: Full year
o Temporal frequency: Minimum: Daily, Target: 12 hours
o Delivery time: Minimum: 48 hours after image acquisition, Target: 12 hours after
image acquisition
According to the applicable documents [AD3] and [AD4], and the external document [ED1], the
user’s requirements relevant for the Snow Cover Extent product for continental Europe are:
• Definition: Northern hemispheric Snow Cover Extent (SCE)
• Geometric properties:
o Spatial resolution: Minimum: 4 km, Target: 500 m, 100 m in complex terrain
o Grid / Projection: Geographic coordinates (latitude/longitude), WGS84 datum,
WGS84 ellipsoid
• Geographical coverage: Northern hemisphere (85°N / 180°W – 35°N/180°E)
• Accuracy requirements:
o Geometric accuracy: Minimum: 1/3 IFOV, Target: IFOV 1 km (100 m in complex
terrain)
o Thematic accuracy: Minimum: 15 %, Target: 5 %
• Temporal requirements:
o Time period: Full year
o Temporal frequency: Minimum: Daily, Target: 12 hours
o Delivery time: Minimum: 24 hours after image acquisition, Target: 12 hours after
image acquisition
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 12 of 31
3 QUALITY ASSESSMENT METHOD
3.1 OVERVALL PROCEDURE
The snow cover extent (SCE) product for continental Europe is generated daily from medium
resolution optical satellite data. The applied algorithm [CGLOPS2_ATBD_SCE500-CEURO-
500m_V1] detects the fraction of snow cover per pixel, given in percentage of snow ranging
between 0 % and 100 %. The applied algorithm corrects for forest canopy, and thus provides in
forested areas information on the snow on ground.
The validation of the snow cover extent product for continental Europe is based on the
intercomparison of the product with reference snow maps from high resolution optical satellite data
The validation is performed on a pixel by pixel basis, following the concept and recommendations
established during the ESA QA4EO project SnowPEx (lead by ENVEO). To describe the product
quality, statistical measures derived from the intercomparison of the product with reference snow
information are used. For the validation of the continental European SCE product with snow maps
from Landsat scenes, the total area covered by each Landsat scene is used, but also different
surface classes, such as forested / non-forested areas, plains, and mountains are discriminated.
The swath width of very high resolution (VHR) optical satellite data also used for validation is in
most cases significantly smaller than that of Landsat images. Thus, for the validation of the
continental European SCE product with snow maps from VHR optical satellite data only the total
coverage of the particular VHR scene is used.
For the validation of the SCE product with snow maps from high resolution optical satellite data all
snow maps are prepared as fraction of the snow cover extent. For a pixel-by-pixel intercomparison
it is mandatory that all information is available in the same map projection and grid size. Thus, the
reference snow maps from high resolution optical satellite data are resampled to the grid size of
the SCE product. Pixels classified as water bodies, clouds, or invalid pixel information in either the
SCE product or the reference snow product are excluded from the validation process. For all other
areas, a pixel-by-pixel intercomparison is applied calculating the following statistical measures for
the total area and for particular surface classes to describe the product performance:
• The fully snow covered area of each individual product is derived by the number of
equivalent fully snow covered pixels (SCE = 100%) (Nequ_fse), which is given according to:
𝑁𝑒𝑞𝑢_𝑓𝑠𝑒 = ∑ ∑𝑆𝐸100(𝑖, 𝑗)
100
𝑥
𝑖=0
𝑦
𝑗=0
Eq. 1
where the 𝑆𝐸100 indicates only pixels with full snow cover in the SCE products.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 13 of 31
• The total snow covered area of each individual product is derived by the number of
equivalent snow covered pixels (Nequ_se), which is given according to
𝑁𝑒𝑞𝑢_𝑠𝑒 = ∑ ∑𝐹𝑆𝐶(𝑖, 𝑗)
100
𝑥
𝑖=0
𝑦
𝑗=0
Eq. 2
where the fractional snow coverage is in per cent (values between 1 to 100 %).
• For determining the Bias between two snow products, the pixels specified by 𝑁𝑢𝑖 are used
as calculation basis:
𝐵𝐼𝐴𝑆 = 1
𝑁𝑢𝑖 ∑ ∑(𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗))
𝑥
𝑖=0
𝑦
𝑗=0
Eq. 3
• The root-mean-square error, RMSE, between two snow products is calculated using all
pixels suitable for inter-comparison (Nui) according to
𝑅𝑀𝑆𝐸 = √1
𝑁𝑢𝑖 ∑ ∑(𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗))²
𝑥
𝑖=0
𝑦
𝑗=0
Eq. 4
• In addition to the RMSE, the unbiased RMSE is applied using the same input dataset (Nui)
and is described by
𝑢𝑛𝑏𝑖𝑎𝑠𝑒𝑑 𝑅𝑀𝑆𝐸 =
Eq. 5 = √
1
𝑁𝑢𝑖 ∑ ∑((𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝐸𝑋𝑇
) − (𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹 ))²
𝑥
𝑖=0
𝑦
𝑗=0
• The correlation coefficient calculation between two products (EXT = SCE in Product, REF =
Reference snow map, e.g. from Landsat) includes only the valid pixels for the inter-
comparison (𝑁𝑢𝑖) and is given by
𝐶𝑜𝑟𝑟𝐶𝑜𝑒𝑓 =
Eq. 6 = ∑ ∑ (𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝐸𝑋𝑇
𝑥𝑖=0
𝑦𝑗=0 ) (𝐹𝑆𝐶𝑅𝐸𝐹 (𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹
)
√∑ ∑ (𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝐸𝑋𝑇 )²𝑥
𝑖=0𝑦𝑗=0 ∑ ∑ (𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹
)²𝑥𝑖=0
𝑦𝑗=0
where FSC
is the average fractional snow cover value.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 14 of 31
The concept for the product validation with snow maps from high resolution optical satellite data is
shown in Figure 3.1.
Figure 3.1: Validation concept developed in the ESA QA4EO project SnowPEx (lead: ENVEO) for the
validation of the SCE product with snow reference products from high resolution optical satellite
data.
3.2 SATELLITE REFERENCE PRODUCTS
3.2.1 Landsat scenes
Archived high resolution optical satellite data from the Landsat satellite family, including Landsat 5
TM and Landsat 7 ETM+, are used to generate reference snow maps. This data base will be
extended in future with data from Landsat 8 OLI/TIRS and Sentinel-2 MSI. The different sensors
have very similar spectral band characteristics (Table 3.1). Thus, established methods and
algorithms can be applied with only very minor adaptations on data of all these sensors. The
Sentinel-2 sensor does not feature thermal infrared bands, which are useful for automated cloud
screening. Thus, areas obscured by clouds or cloud shadows in Sentinel-2 scenes are currently
masked manually. The development of new methods for automated cloud screening from Sentinel-
2 is ongoing, and can be included in the processing chain as soon as a method has been
established. Landsat images used for the processing of reference snow maps are level "L1T"
products, meaning the scenes are radiometrically and geometrically corrected by the data provider
U.S. Geological Survey (USGS), using ground control points and a digital elevation model for the
orthorectification.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 15 of 31
Table 3.1: Band numbers, wavelengths and resolution of Landsat 5 Thematic Mapper (TM), Landsat 7
Enhanced Thematic Mapper plus (ETM+), Landsat 8 Operational Land Imager (OLI) and Thermal
Infrared Sensor (TIRS), and Sentinel-2 Multi-Spectral Instrument (MSI).
Spectral
characteristic
Landsat 5 TM Landsat 7 ETM+ Landsat 8 OLI/TIRS Sentinel-2 MSI
Bd. No.
Spectral Band-width
(micro-metres)
Resolution (meters)
Bd. No.
Spectral Band-width
(micro-metres)
Resolution (meters)
Bd. No.
Spectral Band-width
(micro-metres)
Resolution (meters)
Bd. No.
Spectral Band-width
(micro-metres)
Resolution (meters)
Deep blue and violets (coastal blue, aerosol)
1 0.43 - 0.45
30 1 0.433 - 0.453
60
Visible (VIS)
Blue 1 0.45 - 0.52
30 1 0.45 - 0.52
30 2 0.45 - 0.51
30 2 0.458 - 0.523
10
Green 2 0.52 - 0.60
30 2 0.52 - 0.60
30 3 0.53 - 0.59
30 3 0.543 - 0.578
10
Red 3 0.63 - 0.69
30 3 0.63 - 0.69
30 4 0.64 - 0.67
30 4 0.650 - 0.680
10
Near Infrared (NIR)
5 0.698 - 0.713
20
6 0.773 - 0.793
20
7 0.773 - 0.793
20
4 0.76 - 0.90
30 4 0.77 - 0.90
30 5 0.85 - 0.88
30 8 0.785 - 0.900
10
8A 0.855 - 0.875
20
9 0.935 - 0.955
60
Shortwave Infrared (SWIR)
5 1.55 - 1.75
30 5 1.55 - 1.75
30 6 1.57 - 1.65
30 11 1.565 - 1.655
20
7 2.08 - 2.35
30 7 2.09 - 2.35
30 7 2.11 - 2.29
30 12 2.100 - 2.280
20
Panchromatic (PAN)
8 0.52 - 0.90
15 8 0.50 - 0.68
15
Cirrus / clouds 9 1.36 - 1.38
30 10 1.360 - 1.390
60
Thermal Infrared (TIR)
6 10.40 - 12.50
120* (30) 6 10.40 - 12.50
60** (30) 10 10.60 - 11.19
100*** (30)
11 11.50 - 12.51
100*** (30)
* TM Band 6 was acquired at 120-meter resolution, but products processed before February 25, 2010 are resampled to
60-meter pixels. Products processed after February 25, 2010 are resampled to 30-meter pixels.
** ETM+ Band 6 is acquired at 60-meter resolution. Products processed after February 25, 2010 are resampled to 30-
meter pixels
*** TIRS bands are acquired at 100 meter resolution, but are resampled to 30 meter.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 16 of 31
The generation of the reference snow maps from Landsat data includes four major steps:
(1) the pre-processing of the data in the original map projection and grid size
(2) reprojection of the pre-processed data into the map projection of the SCE products
(3) apply selected algorithms to generate high resolution reference snow maps
(4) aggregate high resolution reference snow maps to grid size of SCE products
Currently, 110 Landsat scenes acquired between 2000 and 2011 have been used to generate
reference snow maps. An overview on the spatial and temporal distribution of these scenes is
shown in Figure 3.2.
Figure 3.2: Overview on distribution of Landsat acquisitions selected for the generation of reference
snow maps. The fill color of each scene indicates the year of the acquisition. Black outlined scenes
show the footprint of a scene. A red outlined scene shifted towards the south-west of a basic
footprint represents a further scene available for this location of the year as indicated by the fill
color.
The pre-processing of the Landsat scenes includes radiometric calibration (Chander, Markham,
and Helder 2009) and topographic correction (Ekstrand, 1996) of all used bands in order to reduce
topographically induced illumination effects. The map projection of the delivered Landsat images is
Universal Transverse Mercator (UTM) on WGS84 ellipsoid and is maintained until the pre-
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 17 of 31
processing is finished. The origin coordinates of all Landsat scenes refer to the upper left corner of
the upper left pixel. This pixel reference remains unchanged for all high-resolution snow products.
The original Landsat pixel size is 30 m x 30 m and will also be preserved until pre-processing is
finished.
The topographically corrected top of atmosphere reflectance at 30 m x 30 m grid size resulting
from the pre-processing is reprojected (bilinear) into the map projection of the SCE products
(geographic coordinates / WGS84). Based on the reprojected topographically corrected top of
atmosphere reflectance at 30 m x 30 m, three snow detection algorithms are applied on each
Landsat scene to assess also the quality of the reference snow maps:
• Dozier and Painter 2004: thresholds applied on NDSI and NIR band, resulting in maps
showing binary snow cover on ground
• Klein, Hall, and Riggs 1998: thresholds applied on NDVI and NDSI, resulting in maps
showing binary snow cover on ground
• Salomonson and Appel 2004, 2006: linear model weighting the NDSI, resulting in maps
showing fractional information on viewable snow cover (in forested areas on top of forest
canopy)
The resulting high-resolution snow maps from Landsat are aggregated to the SCE product
resolution.
3.2.2 Very high resolution optical satellite data
Beside the freely available Landsat data set, a set of 44 very high resolution (VHR) optical satellite
data, such as SPOT-5, Quickbird or WorldView-1/-2, acquired between 2004 and 2013 was
courtesy provided by the Copernicus Data Ware House during the EU FP7 project CryoLand for
validation purposes. These VHR optical satellite data (0.50m – 10m resolution) cover only small
areas, and only images acquired at cloud-free conditions were selected as reference scenes.
Snow maps from these VHR optical satellite data were generated manually by the CryoLand
partners, and used in the same way as Landsat based snow maps as reference snow maps to
validate the continental European SCE product. An overview on the spatial distribution of these
VHR satellite scenes is shown in Figure 3.3.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 18 of 31
Figure 3.3: Overview on spatial distribution of very high resolution optical satellite data (red outlines)
provided courtesy by the Copernicus Data Warehouse during the EU FP7 project CryoLand for
validation purposes. The fill color of each scene indicates the year of the acquisition. Black outlined
scenes show the footprint of a scene. A red outlined scene shifted towards the south-west of a basic
footprint represents a further scene available for this location of the year as indicated by the fill
color.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 19 of 31
4 RESULTS
In this chapter, the detailed validation results derived from the pixel-by-pixel comparison of the
continental European SCE product with snow maps from high-resolution optical satellite data are
presented.
All reference snow maps are reprojected to the map projection and aggregated to a fractional snow
cover extent at the resolution of the continental European SCE product (0.005 degrees). For the
pixel-by-pixel intercomparison, the statistical measures described in detail in Section 3.1 are
calculated. Table 4.1 shows a summary of the validation results of the continental European SCE
product compared on a pixel-by-pixel basis with the 110 selected Landsat scenes of 2000 – 2011
and the 44 VHR optical satellite scenes acquired between 2004 and 2013. Most of the selected
Landsat scenes used for the generation of reference snow maps for the validation of the Pan-
European SCE product were acquired between January and May. For the validation with snow
maps from Landsat data the results are presented separately for each snow algorithm applied on
all the Landsat scenes.
Table 4.1: Summary of statistical measures resulting from the validation of the SCE product over
continental Europe with snow maps from each the total area of 110 selected Landsat scenes and 44
very high resolution optical satellite scenes. Results are generated by ENVEO during the ESA
QA4EO project SnowPEx, and the EU FP7 project CryoLand.
Snow maps from Landsat data Snow maps from VHR
satellite data
MEAN Salomonson Klein Dozier Manual mapping
unbiased RMSE 15,32 15,92 15,61 15,36
BIAS 0,15 -0,75 0,82 3,02
Correlation Coefficient 0,74 0,72 0,73 0,74
RANGE MIN MAX MIN MAX MIN MAX MIN MAX
unbiased RMSE 2,38 26,69 1,70 38,33 0,45 30,12 0,00 38,75
BIAS -19,61 13,76 -33,28 28,84 -17,10 98,47 -13,45 44,64
Correlation Coefficient 0,03 0,96 0,06 0,97 0,04 0,97 0,05 0,92
The mean unbiased RMSE of all scenes is about 15%, independently which approach was used
for the generation of the reference snow maps. Also the mean Correlation Coefficient is very
similar, around 0.73, for the validation with all reference data sets. Only the mean Bias shows
larger discrepancies depending on the snow detection approach used for the generation of
reference snow maps. Nevertheless, if the range of these statistical measures are investigated in
more detail, there are indeed partly large differences. The unbiased RMSE shows ranges between
about 0% and 39%, the Bias shows a large variance ranging from about –33% to about 98%, and
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 20 of 31
also the Correlation Coefficients reflect these differences ranging between 0.03 and 0.97. Thereby,
each of the maximum and minimum statistical measures are derived for different reference scenes.
The validation of the continental European SCE product with reference snow maps from Landsat
data has also been separately analysed with respect to forested and non-forested areas, as
forested areas are very challenging for mapping snow correctly. It has to be considered that
different snow algorithms provide different thematic information on snow in forest, which can either
be the snow on top of the forest canopy, also called “viewable snow”, or the snow on the ground.
For validation activities in forested areas only snow maps from those algorithms providing the
same thematic information as the SCE product (snow on the ground) are used. In non-forested
areas, the results of all snow algorithms can be intercompared. A summary of the validation results
for non-forested areas and forested areas, respectively, derived from intercomparison of the SCE
product with the reference snow maps from Landsat data, is shown in Table 4.2. The agreement of
the SCE product with reference snow maps from Landsat scenes is slightly higher in non-forested
areas. Also in forested areas, the performance of the SCE product is still acceptable, although
some outliers are found. Some cases with larger discrepancies between the SCE product and the
reference snow products are found for forested and non-forested areas, with unbiased RMSE
values up to 35% in non-forested areas, and up to 40%. in forested areas, but there are neither
particular spatial nor temporal patterns identifiable. For the interpretation of the results one should
keep in mind that also the performance of the reference snow maps would need a detailed
evaluation.
Table 4.2: Summary of statistical measures resulting from the validation of the SCE product over
continental Europe with 110 selected Landsat scenes, separating the total non-forested and total
forested areas classified from each Landsat scene. Results are generated by ENVEO during the ESA
QA4EO project SnowPEx.
Class Total non-forested area Total forested area
MEAN Salomonson Klein Dozier Klein Dozier
unbiased RMSE 13,47 14,36 13,69 17,73 17,42
BIAS -0,50 -1,04 0,33 -0,46 1,51
Correlation Coefficient 0,77 0,74 0,76 0,60 0,62
RANGE MIN MAX MIN MAX MIN MAX MIN MAX MIN MAX
unbiased RMSE 1,03 24,35 1,72 34,98 0,46 26,28 0,32 39,50 0,00 32,18
BIAS -16,98 12,52 -27,52 26,33 -14,82 99,55 -35,93 29,41 -22,78 97,67
Correlation Coefficient 0,04 0,98 0,02 0,98 0,01 0,98 003 0,93 0,02 0,94
The statistical measures unbiased RMSE and Bias resulting from the validation of the SCE product
with snow maps generated by the Dozier and the Klein approaches from Landsat scenes acquired
between January and July are illustrated in the scatterplots in Table 4.3. The mean values of the
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 21 of 31
unbiased RMSE and Bias shown in these plots are slightly different to the values shown in Table
4.1, as only a particular time frame is considered for generating these mean values.
Table 4.3: Statistical measures of the fractional snow cover extent (snow on ground) vs. Landsat
snow algorithms with the same quantity displayed from January to July (independent of the year).
Results are generated by ENVEO during the ESA QA4EO project SnowPEx.
Statistical
measure Dozier Klein
Bias
Unbiased
RMSE
The individual validation results of the SCE product with snow maps generated by different
algorithms from the selected Landsat scenes, using each the total area covered by the Landsat
scene, are provided in Table 4.4. The detailed validation results resulting from snow maps from
VHR satellite data, also each valid for the total area covered by the particular VHR scene, are
listed in Table 4.5.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 22 of 31
Table 4.4: Statistical measures derived from the validation of the SCE product over continental
Europe with reference snow maps generated by three different algorithms from Landsat scenes. Only
values for the total area covered by each scene are provided in the table. R = correlation coefficient.
Results are generated by ENVEO within the ESA QA4EO project SnowPEx.
Date
Landsat scene No. of pixels used
for inter-
comparison
Salomonson Klein Dozier
Sensor
Path Row R Bias unbiased
RMSE R Bias
unbiased RMSE
R Bias unbiased
RMSE
28.02.2000 LE7 185 26 132751 0,8 1,15 25,43 0,8 -8,03 26,23 0,82 -1,82 25,44
07.03.2000 LE7 193 27 132782 0,94 -4,14 16,76 0,94 -1,39 15,71 0,95 -0,93 14,56
07.03.2000 LE7 193 28 99664 0,92 1,85 17,93 0,92 -0,49 17,98 0,92 1,14 17,8
21.03.2000 LE7 195 27 126719 0,96 0,18 8,27 0,96 -0,37 8,4 0,97 -0,16 7,91
21.03.2000 LE7 195 28 101387 0,95 1,75 14,92 0,95 0 14,55 0,95 0,94 14,7
27.03.2000 LE7 173 33 110122 0,93 5,31 16,51 0,94 4,72 15,71 0,94 4,64 15,77
30.03.2000 LE7 194 19 128724 0,91 3,92 13,7 0,92 -0,62 12,79 0,89 4,24 14,43
30.03.2000 LE7 194 20 153546 0,92 -6,1 15,46 0,92 0,91 14,89 0,91 -4,58 15,12
17.05.2000 LE7 194 13 242312 0,88 4,35 19,16 0,78 12,32 26,56 0,89 1,97 18,45
21.05.2000 LE7 190 12 228796 0,86 -6,46 20,42 0,78 -10,73 24,9 0,85 -6,39 20,83
22.05.2000 LE7 181 15 152224 0,28 -19,61 26,69 0,27 -33,28 38,33 0,26 -17,1 30,12
07.03.2001 LE7 188 23 161916 0,89 -0,27 18,54 0,91 9,39 17,72 0,88 1,24 19,98
28.03.2001 LE7 191 12 269166 0,08 -3,23 6,61 0 1,09 4,99 0 1,09 4,99
29.03.2001 LE7 182 25 160308 0,94 4,7 14,26 0,93 8,15 16,62 0,95 3,59 14,2
04.05.2001 LE7 194 12 195320 0,85 -1,67 17,49 0,77 -7,41 20,81 0,84 -2,29 17,98
12.01.2002 LE7 173 33 129845 0,88 3,37 14,73 0,88 3,53 14,97 0,88 3,51 14,94
04.03.2002 LE7 194 27 130909 0,96 -0,65 13,4 0,96 -1,97 13,97 0,96 -1,98 13,32
04.03.2002 LE7 194 28 130614 0,94 0,22 16,41 0,94 -1,38 16,66 0,94 -0,85 16,28
13.03.2002 LE7 193 27 147399 0,92 -0,63 17,84 0,92 0,57 17,98 0,92 0,19 17,83
13.03.2002 LE7 193 28 144679 0,93 -0,5 15,84 0,93 0,5 15,97 0,93 -0,07 15,81
05.04.2002 LE7 178 14 234555 0,21 7,01 10 0,2 -0,63 4,73 nan 98,47 7,35
09.04.2002 LE7 174 21 164594 0,85 -1,92 21,64 0,88 8,31 20,13 0,83 -2,36 24,33
13.04.2002 LE7 170 21 56478 0,9 -0,36 16,92 0,93 -5,24 14,17 0,89 -1,43 17,83
27.05.2002 LE7 190 11 156070 0,89 -2,91 9,91 0,91 -1,87 9,09 0,91 -1,75 8,97
29.05.2002 LE7 188 12 237853 0,72 3,14 10,25 0,7 2,96 10,79 0,75 2,38 9,44
30.05.2002 LE7 195 12 265463 0,86 2,48 10,83 0,87 2,15 10,4 0,87 2,11 10,32
14.12.2002 LE7 173 33 115084 0,94 5,39 15,17 0,94 5,46 15,97 0,94 5,26 15,73
10.01.2003 LE7 202 33 124431 0,79 -7,05 16,64 0,67 -11,84 25,96 0,8 -6,56 18,47
18.01.2003 LE7 194 28 80024 0,9 -2,55 20,91 0,9 -3,15 20,82 0,91 -2,89 20,42
24.01.2003 LE7 172 33 121635 0,95 -4,04 15,15 0,94 -4,12 15,53 0,94 -4,09 15,5
03.02.2003 LE7 178 25 169585 0,33 -1,01 10,46 0,12 2,68 10,03 0,13 2,41 10,22
07.02.2003 LE7 174 36 7206 nan -0,64 2,5 nan -0,25 1,7 nan -0,02 0,45
07.02.2003 LE7 190 29 63009 0,92 1,11 17,62 0,9 -8,06 21,05 0,9 -2,02 20
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 23 of 31
Date
Landsat scene No. of pixels used
for inter-
comparison
Salomonson Klein Dozier
Sensor
Path Row R Bias unbiased
RMSE R Bias
unbiased RMSE
R Bias unbiased
RMSE
08.02.2003 LE7 197 30 88504 0,92 -0,57 10,22 0,91 1,13 10,89 0,93 -0,2 9,86
09.02.2003 LE7 172 21 180808 0,64 -5,34 16,74 0,18 9,82 20,24 0,5 -4,2 23,35
12.02.2003 LE7 193 23 113259 0,73 3,17 18,35 0,67 -10,87 18,47 0,68 4,49 23,78
14.02.2003 LE7 191 27 113591 0,61 8,2 21,81 0,61 3,1 21 0,66 -1,01 18,32
15.02.2003 LE7 198 27 133319 0,79 -0,12 2,38 0,73 -0,36 3,62 0,76 -0,08 2,64
16.02.2003 LE7 189 22 119611 0,53 -7,11 13,22 0,32 3,72 11,68 0,46 -3,41 16,59
19.02.2003 LE7 194 19 149364 0,75 -12,64 21,67 0,73 0,78 22,56 0,74 0,19 22,13
19.02.2003 LE7 194 25 88771 0,75 8,11 25,34 0,76 -3,26 25,09 0,75 6,68 26,56
19.02.2003 LE7 194 26 150273 0,85 5,87 21,02 0,87 -1,91 20,01 0,86 2,21 20,86
19.02.2003 LE7 194 27 112814 0,46 2,54 16,43 0,43 -1,78 15,64 0,49 -1,63 14,85
19.02.2003 LE7 194 28 126248 0,94 -1,1 16,68 0,94 -2,58 16,95 0,94 -1,38 15,9
21.02.2003 LE7 192 27 94449 0,72 9,59 22,49 0,73 6,21 21,24 0,76 1,61 19,32
24.02.2003 LE7 197 25 162640 0,92 -1,22 12,45 0,92 1,61 12,64 0,94 0,03 10,66
26.02.2003 LE7 195 27 107653 0,84 5,82 23,91 0,84 -0,14 23,71 0,87 1,59 21,39
26.02.2003 LE7 195 28 69767 0,93 -0,11 17,24 0,94 -1,88 16,7 0,94 -0,91 16,3
27.02.2003 LE7 186 26 158302 0,57 -3,55 21,46 0,44 7,65 22,89 0,54 0,41 22,85
27.02.2003 LE7 186 27 141617 0,58 1,63 12,99 0,58 -2,08 12,32 0,56 -0,61 13,43
27.02.2003 LE7 186 28 133052 0,93 -2,94 14,14 0,91 -5,59 15,93 0,93 -2,16 14,33
27.02.2003 LE7 186 29 91336 0,85 -5,54 22,31 0,85 -9,22 22,83 0,84 -2,92 23,31
27.02.2003 LE7 186 30 127432 0,9 1,02 19,33 0,88 -5,85 20,78 0,9 -0,04 19,59
27.02.2003 LE7 186 31 97801 0,92 1,38 18,22 0,92 4,97 19,09 0,92 1,76 18,9
27.02.2003 LE7 186 32 48518 0,93 -1,59 11,99 0,92 -2,64 13,33 0,93 -1,63 12,13
08.03.2003 LE7 169 34 105942 0,96 -4,11 13,81 0,96 -3,93 13,76 0,96 -3,92 13,76
08.03.2003 LE7 185 21 145969 0,3 10,39 15,39 0,23 -1,26 10,08 0,3 6,07 17,57
12.03.2003 LE7 181 32 86814 0,86 -1,29 9,21 0,81 -2,79 12,53 0,89 -1,04 8,55
15.03.2003 LE7 186 17 152573 0,56 -18,33 18,95 0,56 -4,16 18,16 0,53 -8,9 19,24
15.03.2003 LE7 186 20 176128 0,49 7,94 21,76 0,59 -3,96 18,46 0,5 5,41 23,32
23.03.2003 LE7 194 28 143183 0,96 -1,05 12,37 0,97 -0,35 11,93 0,96 -0,93 12,3
24.03.2003 LE7 185 23 171462 0,74 -0,13 15,43 0,74 7,02 17,99 0,62 -5,34 18,06
24.03.2003 LE7 185 31 132405 0,94 0,22 11,7 0,94 -1,65 12,05 0,94 0,76 12,05
28.03.2003 LE7 181 21 186386 0,59 11,89 21,08 0,52 -4,86 18,45 0,59 12,48 26,12
28.03.2003 LE7 181 22 184319 0,53 -0,17 21,99 0,5 10,27 20,69 0,48 1,35 25,69
28.03.2003 LE7 181 23 173309 0,88 1,66 19,95 0,9 9,25 18,67 0,86 -0,33 22,77
28.03.2003 LE7 181 24 166692 0,79 -1,36 9,76 0,82 -2,81 11,41 0,74 -0,1 10,37
01.04.2003 LE7 177 14 261073 0,21 -8,68 8,87 0,13 0,59 3,81 0,17 0,32 4,46
01.04.2003 LE7 193 27 141321 0,93 0,83 15,09 0,93 -1,26 15,51 0,93 0,41 15,12
09.04.2003 LE7 185 13 201437 0,09 -1,48 4,96 0,23 0,44 3,28 0,16 0,36 3,49
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 24 of 31
Date
Landsat scene No. of pixels used
for inter-
comparison
Salomonson Klein Dozier
Sensor
Path Row R Bias unbiased
RMSE R Bias
unbiased RMSE
R Bias unbiased
RMSE
10.04.2003 LE7 192 12 278368 0,07 -0,03 3,49 0,11 0,43 3,15 0,09 0,41 3,19
17.04.2003 LE7 193 13 269038 0,24 -2,47 7,32 0,29 0,73 5,48 0,25 0,13 6,3
17.04.2003 LE7 193 14 163481 0,73 13,76 18,54 0,74 1,16 16,35 0,73 11,69 21,92
11.05.2003 LE7 169 30 136848 0,85 -1,48 9,52 0,86 -1,32 9,45 0,86 -1,24 9,26
14.05.2003 LE7 174 13 2159 0,05 1,83 4,6 0,06 0,6 4,53 0,06 0,62 4,55
11.10.2003 LT5 192 27 122304 0,85 1,43 23,05 0,86 0,16 22,84 0,86 0,98 22,7
18.10.2003 LT5 193 27 112554 0,87 2,34 17,8 0,88 1,58 17,68 0,87 2,48 18,12
25.10.2003 LT5 194 28 113400 0,92 -1,29 18,35 0,92 -3,36 18,62 0,92 -2,35 18,36
30.05.2004 LT5 200 16 158380 0,91 3,15 14,82 0,91 2,77 14,64 0,92 2,65 14,48
30.05.2004 LT5 200 17 162872 0,92 -3,14 18,4 0,92 -2,71 18,33 0,92 -2,6 18,24
10.06.2004 LT5 197 13 192733 0,96 -0,89 12,86 0,96 -0,42 12,76 0,96 -0,37 12,72
05.04.2005 LT5 194 28 135212 0,9 0,89 17,18 0,9 0,61 17,72 0,89 1,38 18,24
17.04.2005 LT5 198 11 131059 0,62 0,29 10,45 0,6 -1,07 10,74 0,62 -0,82 10,91
25.04.2005 LT5 190 14 203426 0,85 2,93 15,68 0,8 -4,13 17,66 0,84 0,7 16,87
30.04.2005 LT5 193 14 131271 0,9 6,19 17,38 0,86 -8,42 20,72 0,9 6,94 18,59
16.05.2005 LT5 193 11 266156 0,81 0,21 8,56 0,78 -0,43 9,23 0,78 -0,05 9,44
16.05.2005 LT5 193 12 279181 0,41 0,36 6,31 0,33 -0,34 6,54 0,4 -0,2 6,56
15.04.2006 LT5 195 17 73472 0,18 11,19 18,94 0,19 2,14 16,49 0,21 2,88 16,59
30.04.2006 LT5 188 12 254569 0,75 2,99 14,56 0,65 6,63 16,55 0,71 3,63 16,09
03.05.2006 LT5 193 11 200192 0,88 -5,7 18,38 0,85 -6,56 20,82 0,85 -4,3 21,11
05.05.2006 LT5 191 12 220061 0,84 -10,2 22,97 0,72 -19,5 30,66 0,84 -7,79 23,22
05.05.2006 LT5 191 13 235105 0,74 11,97 21,28 0,54 28,84 32,94 0,77 5,42 20,85
08.05.2006 LT5 196 13 206648 0,91 5,59 19,1 0,81 11,86 27,12 0,91 3,45 18,67
11.06.2006 LT5 194 27 157030 0,61 4,83 17,57 0,6 5,17 18,48 0,62 4,53 17,37
13.06.2006 LT5 192 27 157021 0,75 -3,32 14,22 0,73 -3,46 15,15 0,75 -2,86 14,03
20.06.2006 LT5 193 27 141747 0,6 -3,1 12,62 0,6 -2,86 12,55 0,61 -2,58 12,12
29.05.2007 LT5 194 12 250863 0,94 -2,36 9,73 0,93 -2,09 10,6 0,95 -1,43 8,96
12.06.2009 LT5 193 28 153201 0,83 -2,58 13,34 0,82 -2,35 13,4 0,83 -2,18 13,23
07.03.2010 LT5 181 28 126737 0,95 2,73 14,36 0,92 4,48 17,49 0,94 3,26 14,91
23.03.2010 LT5 181 24 154306 0,71 -4,5 20,75 0,72 -13,12 20,25 0,69 -5,15 22,7
23.03.2010 LT5 181 26 156872 0,88 -12,8 17,95 0,88 -12,74 19,88 0,88 -11,81 19,32
04.04.2010 LT5 169 23 52340 0,63 4,74 16,33 0,55 8 17,9 0,58 6,72 17,9
04.04.2010 LT5 185 31 134551 0,86 -2,08 10,45 0,83 -2,32 11,68 0,86 -1,8 10,21
04.04.2010 LT5 185 32 129838 0,73 -2,55 11,16 0,69 -2,96 12,95 0,74 -2,25 10,95
29.03.2011 LT5 186 20 124324 0,03 11,91 12,31 0,08 -0,52 4,53 0,04 5,13 13,53
29.03.2011 LT5 186 21 69851 0,58 7,43 11,61 0,76 -0,68 8,09 0,69 0,69 10,36
08.05.2011 LT5 194 12 237292 0,92 -0,71 16,96 0,86 -6,61 22,4 0,93 0,51 17,03
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 25 of 31
Date
Landsat scene No. of pixels used
for inter-
comparison
Salomonson Klein Dozier
Sensor
Path Row R Bias unbiased
RMSE R Bias
unbiased RMSE
R Bias unbiased
RMSE
14.05.2011 LT5 188 12 228286 0,92 -1,52 17,21 0,88 -6,79 21,32 0,92 -1,6 17,73
21.08.2011 LT5 193 27 106941 0,57 -0,87 6,99 0,56 -0,81 6,84 0,56 -0,79 6,83
22.09.2011 LT5 193 28 108787 0,92 -3,22 13,55 0,9 -5,04 16,64 0,92 -2,8 13,91
Table 4.5: Statistical measures derived from the validation of the continental European fractional
snow cover products with snow maps generated manually from very high resolution optical satellite
data. Analyses are performed by ENVEO within the EU FP7 project CryoLand.
Acquisition Date
Scene Centre
(Lat/Lon) Satellite
Number of pixels used for
intercomparison
Correlation Coefficient
Bias Unbiased
RMSD
11.02.2004 59.65N/10.85E SPOT5 14238 nan 2.82 8.40
07.03.2004 61.76N/11.22E SPOT5 23235 nan 5.80 13.19
10.03.2004 66.04N/18.94E SPOT5 29475 nan 0.59 4.10
12.03.2004 64.39N/12.32E SPOT5 23863 nan 1.16 5.50
12.03.2004 65.21N/13.11E SPOT5 20699 nan 1.30 5.85
28.03.2004 62.84N/13.95E SPOT5 22986 nan 2.45 9.25
01.04.2004 64.43N/13.14E SPOT5 26480 nan 0.43 3.51
03.04.2004 68.63N/16.67E SPOT5 22956 nan 2.00 8.11
09.04.2004 59.96N/17.71E SPOT5 14048 nan -0.08 1.23
10.09.2004 59.56N/7.66E SPOT5 11735 nan 0.00 0.00
09.02.2005 47.99N/6.53E SPOT5 3952 0.83 6.96 15.85
09.02.2005 47.99N/6.53E SPOT5 12285 0.78 13.13 23.44
30.12.2005 45.60N/14.12E SPOT5 2796 nan 7.95 14.66
11.12.2008 37.83N/43.84E SPOT5 7053 nan 2.99 14.20
24.11.2009 43.67N/40.97E SPOT5 5554 0.38 12.96 23.27
24.11.2009 43.67N/40.97E SPOT5 15322 0.81 14.21 27.59
06.01.2010 38.82N/40.50E SPOT5 6207 nan 5.17 17.53
10.03.2010 51.78N/8.20E SPOT5 18110 0.88 11.19 18.12
24.12.2010 51.17N/4.25W SPOT5 18139 0.66 -13.45 15.16
25.12.2010 43.19N/4259E SPOT5 6255 nan 44.64 38.75
05.01.2011 49.64N/3.53E SPOT5 13598 0.85 -2.79 14.08
14.03.2011 39.30N/43.16E SPOT5 15978 0.54 -1.76 9.14
25.03.2011 42.71N/19.97E SPOT5 6656 0.90 1.03 19.03
11.04.2011 46.08N/7.31E SPOT5 15208 0.85 3.64 23.76
11.04.2011 45.11N/6.95E SPOT5 14280 0.83 7.17 25.56
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 26 of 31
Acquisition Date
Scene Centre
(Lat/Lon) Satellite
Number of pixels used for
intercomparison
Correlation Coefficient
Bias Unbiased
RMSD
01.10.2011 61.48N/8.78E Worldview2 1377 nan 0.00 0.07
31.01.2012 49.37N/20.25E SPOT5 6976 nan 4.63 11.98
31.01.2012 47.52N/24.29E SPOT5 4599 nan 4.73 11.37
31.01.2012 49.37N/20.25E SPOT5 6976 nan 4.63 11.98
03.02.2012 52.20N/4.28W SPOT5 16629 0.86 -3.89 21.78
03.02.2012 51.94N/3.63W SPOT5 17061 0.86 4.08 23.00
08.02.2012 48.42N/15.80E SPOT5 5676 nan 6.77 13.38
19.02.2012 54.93N/2.87W SPOT5 18919 0.70 -1.57 10.91
16.03.2012 43.20N/18.84E SPOT5 5465 nan 4.50 13.19
27.03.2012 41.80N/23.63E SPOT5 5509 0.92 5.43 18.31
17.04.2012 61.30N/8.94E Worldview1 209 nan -0.81 0.95
01.05.2012 61.54N/9.09E Worldview1 1074 0.83 -3.65 14.63
05.05.2012 61.30N/8.94E Quickbird 288 0.86 -5.33 15.77
30.05.2012 61.48N/8.78E Worldview2 787 0.78 -9.83 13.76
09.12.2012 43.05N/1.08W SPOT5 17171 0.86 -0.01 23.87
03.01.2013 43.04N/0.87E SPOT5 11002 0.82 10.41 26.03
14.01.2013 37.47N/3.97W SPOT5 15857 0.76 10.81 22.22
03.02.2013 42.99N/0.63W SPOT5 11403 0.65 -13.15 32.16
02.03.2013 45.67N/7.11E SPOT5 16814 0.67 -7.84 28.58
For the validation of the SCE product with reference snow maps from Landsat data also different
snow conditions are considered. The snow maps are therefore intercompared with respect to the
fraction of snow cover extent per pixel. The same statistical analyses as shown before for the total
area and different surface classes are used for this kind of validation. The unbiased RMSE and the
Bias values resulting from this validation of the SCE product with fractional snow cover maps from
all 110 selected Landsat scenes are illustrated in the scatterplots in Table 4.6. The results of the
comparison of the SCE product with Dozier snow maps from Landsat scenes shows an
underestimation of the SCE product in the 0-25 % range (bias 9.9 %) compared to the reference
snow product, and a tendency to overestimate the snow cover slightly towards higher snow cover
fractions (bias -9.28 % in the 75-100 % range) compared to the reference snow maps. The same
tendency is also apparent in the validation with the Klein snow maps from Landsat data.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 27 of 31
Table 4.6: Validation results of fractional SE products detecting snow on ground vs. LS snow
algorithms for different fraction sections: 0-25 %, 26-50 %, 51-75%, 76-100 %. Results are generated
by ENVEO during the ESA QA4EO project SnowPEx.
Dozier Klein
FSC Unbiased RMSE Bias Unbiased RMSE Bias
0 -
25 %
26 -
50
%
51 -
75
%
76 -
100
%
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 28 of 31
5 CONCLUSIONS
The continental European SCE product has so far been validated with snow maps generated from
a selection of 110 Landsat scenes acquired between 2000 and 2011, and from 44 selected VHR
optical satellite data acquired between 2004 and 2013 over different surface classes and at
different snow conditions all over Europe. The validation activities were performed in the EU FP7
project CryoLand and the ESA QA4EO project SnowPEx, each under the lead of ENVEO. The
SCE product has been validated on a pixel-by-pixel comparison using statistical measures to
describe the performance of the product, following the validation protocols developed within the
QA4EO project SnowPEx, and agreed by the international snow community.
The mean resulting unbiased Root Mean Square Error derived from the validation of the SCE
product with all reference snow maps is 15%, independently of the applied snow algorithm for the
generation of the reference snow map. The mean Bias derived for validation with Landsat snow
maps is within ± 0.8 %, while the validation with VHR snow maps results in a higher mean Bias of
about 3%. The mean correlation coefficient is in all validation cases about 0.73. Anyway, for
particular validation cases the resulting statistical measures are higher or lower, respectively.
In case of Landsat based reference snow maps, generated with snow algorithms resulting in the
same thematic information as the SCE product, namely snow on ground, also the performance
over non-forested and forested areas has been analysed separately. In forested areas, the
resulting unbiased RMSE is about 17%, while in non-forested areas it is about 14%, with mean
Bias values of about ±1.5%. The derived correlation coefficient is about 0.76 for non-forested
areas, and about 0.60 for forested areas.
Additionally, the agreement of the fraction of snow cover of the SCE product with the aggregated
fractional snow cover from the selected Landsat scenes is assessed. In case of less fractional
snow cover (0% - 25%), the SCE products underestimates the derived reference snow cover, while
for areas largely covered by snow (76% - 100%), the SCE products overestimate the derived
reference snow cover from Landsat data.
The performance of the SCE product is compared with the requirements defined by the users. The
resulting compliance matrix is shown in Table 5.1.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 29 of 31
Table 5.1: Compliance matrix for the continental European SCE product.
Requirements Minimum Target Value Range
(Min/Max) Compliant Comments
Thematic
accuracy:
15 % 5 % Unbiased
RMSE: 15 %
Bias:
± 0.8 %
Unbiased
RMSE:
0% / 39%
Bias: -3% /
98%
Partly Reference snow maps need
also validation, performance
depends on surface class
and snow conditions
Time period: Full year Full year 2000 –
present
Yes
Temporal
frequency
Daily 12 hours Daily Yes
Delivery time: 48 hours
after image
acquisition
12 hours
after image
acquisition
12 hours
after image
acquisition
Yes Depends on the availability
of the NRT satellite data
used as input
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 30 of 31
6 RECOMMENDATIONS
The validation results of the continental European SCE product at 0.005 degree resolution with
snow maps from high resolution optical satellite data indicates that the minimum thematic accuracy
requirement of 15% defined by users can on average be achieved. But, for particular surface or
snow conditions, the differences between the SCE product and the snow maps used as reference
data are larger. For the interpretation of the quality assessment of the SCE product for continental
Europe one should keep in mind that also the snow maps from (very) high resolution optical snow
maps can include some misclassifications.
When reference snow maps from high resolution optical satellite data are generated, the following
facts should be considered:
1. For the generation of a reference snow map from high resolution optical satellite data, a
snow mapping algorithm designed particularly for the spectral capabilities of the satellite
data should be used. The reference snow maps from Landsat data used so far for the
validation activities are based on algorithms originally developed for medium resolution
optical satellite data from Terra MODIS. In the snow community, it is quite common to just
adapt these approaches for Landsat data, as the sensors have similar spectral
characteristics.
2. The generated reference snow maps from high resolution optical satellite data need to be
validated themselves. As snow cover is a highly dynamic parameter which can change
within a few hours from fully snow covered to snow free, the availability of reference data
acquired over the same region and at the same time as a Landsat scene is very limited.
Thus, this task is very challenging and needs a very extensive planning and preparation
phase.
Also, the validation of the continental European SCE product at 0.005 degree resolution with in-situ
snow observations for single points can be critical, as the station of the snow measurement does
not necessarily reflect the snow conditions of a pixel covering an area of about 500m x 500m. Such
a validation of the SCE product has been performed during the QA4EO project SnowPEx by SYKE
for different surface classes within particular snow climate classes after Sturm, Holmgren, and
Liston (1995), but needs to be interpreted very carefully. A publication of these results is in
preparation.
Nevertheless, the current validation results of the continental European SCE product will in future
be extended with further snow maps from high resolution optical satellite data and in-situ
observations, depending on the availability of such data.
Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02
Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium
Issue: I1.02 Date: 08.05.2017 Page: 31 of 31
7 REFERENCES
Chander, Gyanesh, Brian L. B.L. Markham, and Dennis L. D.L. Helder. 2009. “Summary of Current Radiometric Calibration Coefficients for Landsat MSS, TM, ETM+, and EO-1 ALI Sensors.” Remote Sensing of Environment 113: 893–903.
Dozier, J., and T. H. Painter. 2004. “Multispectral and Hyperspectral Remote Sensing of Alpine Snow Properties.” Annual Review of Earth and Planetary Sciences 32: 465 – 494.
Klein, Andrew G, Dorothy K Hall, and George A Riggs. 1998. “Improving Snow Cover Mapping in Forests through the Use of a Canopy Reflectance Model.” Hydrological Processes 12: 1723–
44.
Salomonson, V.V., and I. Appel. 2006. “Development of the Aqua MODIS NDSI Fractional Snow Cover Algorithm and Validation Results.” IEEE Transactions on Geoscience and Remote Sensing 44 (7): 1747–56. doi:10.1109/TGRS.2006.876029.
Salomonson, V.V, and I Appel. 2004. “Estimating Fractional Snow Cover from MODIS Using the Normalized Difference Snow Index.” Remote Sensing of Environment 89 (3): 351–60.
doi:10.1016/j.rse.2003.10.016.
Sturm, M., J. Holmgren, and G. E. Liston. 1995. “A Seasonal Snow Cover Classification System for Local to Global Applications.” Journal of Climate 8 (5): 1261–83. doi:10.1175/1520-
0442(1995)008<;1261:ASSCCS>;2.0.CO;2.
Top Related