Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version)...
Transcript of Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version)...
![Page 1: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/1.jpg)
1 JPSS Calibration/Validation Maturity Review
Validated Maturity Science Review For VIIRS-VH
Presented by Felix Kogan Contributed by Wei Guo
Date: 10/18/2016
![Page 2: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/2.jpg)
2 JPSS Calibration/Validation Maturity Review
Outline
• Algorithm Cal/Val Team Members • Product Requirements • Evaluation of algorithm performance to specification
requirements – Evaluation of the effect of required algorithm inputs – Quality flag analysis/validation – Error Budget
• Identification of Processing Environment • Users & User Feedback • Documentations (Science Maturity Check List) • Conclusion • Path Forward
![Page 3: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/3.jpg)
3 JPSS Calibration/Validation Maturity Review
Name Organization Major Task Felix Kogan NESDIS/STAR Leader; Evaluating the validation result
Wei Guo IMSG Data collection, processing and analysis
VIIRS-VH Cal/Val Team
Algorithm Cal/Val Team Members
![Page 4: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/4.jpg)
4 JPSS Calibration/Validation Maturity Review
Vegetation Health (VH) Algorithm
![Page 5: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/5.jpg)
5 JPSS Calibration/Validation Maturity Review
VH Algorithm for SNPP and J1 components
![Page 6: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/6.jpg)
6 JPSS Calibration/Validation Maturity Review
Vegetation Health Index & applications, Oct 9, 2012
![Page 7: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/7.jpg)
7 JPSS Calibration/Validation Maturity Review
SNPP-VIIRS Vegetation Health Index, 1 km, Sep29, 2016
![Page 8: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/8.jpg)
8 JPSS Calibration/Validation Maturity Review
Requirements
• Product performance requirements from JPSS L1RD supplement (threshold) versus observed/validated
EDR Attributes JPSS L1RD Veg. Health Product System
Horizontal Cell Size Threshold – 0.036o (4km) Threshold – 0.036o (4km) Objective – 0.018o, 0.009o (2 and 1 km)
Objective – 0.018o, 0.009o (2 and 1 km)
Vertical Reporting Interval NS NS
Mapping Uncertainty, 3 sigma
Threshold – <0.036o (4km) Threshold – <0.036o (4km) Objective – <0.018o, <0.009o , <0.0045o
Objective – <0.018o, <0.009o , <0.0045o
Measurement Precision Threshold – 4.0% (For the range 0-100%)
Threshold – 4.0% (For the range 0-100%)
Objective – NS Objective – NS Measurement Accuracy Threshold – 1.0% Threshold – 1.0%
Objective – NS Objective – NS Refresh Threshold – Every 7 day period Threshold – Every 7 day period
Objective – Every 5 day period Objective – Every 5 day period
![Page 9: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/9.jpg)
9 JPSS Calibration/Validation Maturity Review
Evaluation of algorithm performance to specification requirements
• Improvements since Algorithm Readiness Review (ARR, Provisional) – Algorithm Improvements
• 4 and 1 km Climatology from AVHRR (MODIS?) – LUT / PCT updates
• Regression coefficient 2.0 (using 3 years’ data)
• Cal/Val Activities for evaluating algorithm performance: – Comparison with other algorithms – Comparison with in situ data – Validation strategy/method
• Correlation and regression analysis • Time series analysis • Comparison of VIIRS VH with other similar products • Agricultural, health and economic data • Users response
![Page 10: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/10.jpg)
10 JPSS Calibration/Validation Maturity Review
ADJUST: Comparison of SMN from VIIRS and AVHRR
The SMN from VIIRS and AVHRR are not on the same level. Adjustment is required to make the long term time series aligned.
![Page 11: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/11.jpg)
11 JPSS Calibration/Validation Maturity Review
ADJUST: Time series of SMN from VIIRS and AVHRR
The SMN from VIIRS and AVHRR are not on the same level. Adjustment is required to make the long term time series aligned.
![Page 12: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/12.jpg)
12 JPSS Calibration/Validation Maturity Review
VALID: Comparison on VHI from VIIRS, 4km vs. 500m
VHI from 4km dataset VHI from 500m dataset
Data was extracted from sampled (16km) files which were created from VIIRS 4km and 500m data sets respectively. The major difference of VIIRS 4km and 500m data sets is the high resolution climatology (using MODIS and AVHRR).
2016 week 26
VIIRS 4km VH product is an operational product running at OSPO.
![Page 13: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/13.jpg)
13 JPSS Calibration/Validation Maturity Review
VALID: Drought USDM versus VIIRS/VHI
USDM Sep 11, 2012
VIIRS/VHI Sep 14, 2012
USDM
VIIRS/VHI
![Page 14: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/14.jpg)
14 JPSS Calibration/Validation Maturity Review
VALID: SPOT dNDVI versus VIIRS/VHI
![Page 15: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/15.jpg)
15 JPSS Calibration/Validation Maturity Review
VALID: US Drought versus poor and very poor w. wheat & pasture, May 2013
![Page 16: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/16.jpg)
16 JPSS Calibration/Validation Maturity Review
VIIRS/VHI versus Economic Indicators, California 2015
![Page 17: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/17.jpg)
17 JPSS Calibration/Validation Maturity Review
Major Wheat Area of Australia
The small images with blue background are mask files generated from the polygons provided by USDA.
![Page 18: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/18.jpg)
18 JPSS Calibration/Validation Maturity Review
Yield versus VHI for Australia major wheat area 1982-2015
![Page 19: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/19.jpg)
19 JPSS Calibration/Validation Maturity Review
Users & User Feedback • User list
– USDA – Other users: bank, insurance company, research institute.
• Feedback from users – USDS used STAR VH data in their monthly report
• Downstream product list – Drought, – Moisture stress, – Thermal stress, – Fire risk, – Malaria risk – Vegetation health – Moisture condition – Thermal condition – Mosquitoes’ born diseases – Landslides – Land cover changes – Climate warming – Crop/pasture condition
• Reports from downstream product teams on the dependencies and impacts – Pending
![Page 20: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/20.jpg)
20 JPSS Calibration/Validation Maturity Review
Number of Users for the VH WEB site
http://www.star.nesdis.noaa.gov/smcd/emb/vci/VH/vh_browse.php
![Page 21: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/21.jpg)
21 JPSS Calibration/Validation Maturity Review
Evaluation of the effect of required algorithm inputs (2-5 slides)
• Required Algorithm Inputs – Primary Sensor Data
• VIIRS SDR I1,I2, I5, M3, cloud mask – Ancillary Data
• Land types, land sea mask, snow – Upstream algorithms
• NA – LUTs / PCTs
• Regression coefficients between VIIRS and AVHRR SMN (SMT)
• Evaluation of the effect of required algorithm inputs – Study / test cases – Results
![Page 22: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/22.jpg)
22 JPSS Calibration/Validation Maturity Review
-- Quality flag analysis/validation (2-5 slides) • Defined Quality Flags
– Variable – Description – Value
• Quality flag analysis/validation – Test / example / ground truth data sets – Analysis/validation results – Analysis/validation plan
1. VH products will be validated by countries & admin. Regions (not by pixels)
2. Validation data: Official products, Ag. Production, crop conditions, Malaria, Similar products, Economic indicators etc.
3. No quality flags is needed.
![Page 23: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/23.jpg)
23 JPSS Calibration/Validation Maturity Review
VALID: VIIRS-VH 1 vs 4 km, June (26) 2016
![Page 24: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/24.jpg)
24 JPSS Calibration/Validation Maturity Review
-- Error Budget (1 slide)
Compare analysis/validation results against requirements, present as a table. Error budget limitations should be explained. Describe prospects for overcoming error budget limitations with future improvement of the algorithm, test data, and error analysis methodology.
Attribute Analyzed
L1RD Threshold
Analysis/Validation Result
Error Summary
Support Artifacts
NO pixel validation
![Page 25: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/25.jpg)
25 JPSS Calibration/Validation Maturity Review
-- Identification of Processing
• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015
• Algorithm version – ATBD 1.0
• Version of LUTs used – Regression coefficient 2.0 (using 3 years’ data)
• Version of PCTs used – Need revise this page
• Description of environment used to achieve validated maturity stage – Need revise this page
![Page 26: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/26.jpg)
26 JPSS Calibration/Validation Maturity Review
Documentations (Check List, 1 slide)
Science Maturity Check List Yes ?
ReadMe for Data Product Users Yes
Algorithm Theoretical Basis Document (ATBD) Yes
Algorithm Calibration/Validation Plan Yes
(External/Internal) Users Manual Yes
System Maintenance Manual (for ESPC products) Yes
Peer Reviewed Publications (Demonstrates algorithm is independently reviewed)
Yes
Regular Validation Reports (at least. annually) (Demonstrates long-term performance of the algorithm)
will do annually report
![Page 27: Validated Maturity Science Review For VIIRS-VH...• ESPC (e.g., NDE, Okeanos) build (version) number and effective date – Version 1.1 – SPSRB briefing on 7/31/2015 • Algorithm](https://reader033.fdocuments.us/reader033/viewer/2022060307/5f09ed867e708231d4292d14/html5/thumbnails/27.jpg)
27 JPSS Calibration/Validation Maturity Review
Path Forward
• Planned further improvements – Update Regression Coefficients between VIIRS SMN
(SMT) and AVHRR SMN (SMT) – Update algorithm for VH climatology – Start developing VIIRS-VH climatology – Climate warming and VH climatology
• Planned Cal/Val activities / milestones - Validate by standard drought products - Validate by crop yield data for more locations. – Validate by comparing with other events – Validate by agricultural condition – Validate by users response – Validate by standard product – Validate by users’ responce