Validation Reportof the CAMS UV processor Issue …...between model cycles can be identifiedby...

ECMWF COPERNICUS REPORT

Copernicus Atmosphere Monitoring Service

Validation Report of the CAMS UV processor Issue #16 June-July-August (JJA) 2019 CAMS-72: Solar radiation products

Issued by: M.R.A. Pitkänen, W. Wandji, A. Arola, FMI

Date: 28/12/2019

Ref: CAMS72_2018SC1_D72.2.1.1-2019Q4_UV_VAL_201912_v1


CAMS72_2018SC1 – Validation Report of the CAMS UV processor JJA 2019 of 48

This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of CAMS on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view.



Contributors

FMI Mikko R. A. Pitkänen William Wandji Antti Arola



Contents

1. Source of data 7

1.1 Ground based measurements 7 1.2 CAMS UV data 9

2. Methodology 10

3. Results and discussions 11

3.1 Performance of the UV processor – UV Index 11 3.2 Performance of the UV processor – spectral UV 19

4. CAMS e-suite UV evaluation 25

5. Conclusions 29

6. References 30

7. Appendix 30



Summary The CAMS UV processor produces global UV index and spectral UV forecasts, whose performance is regularly monitored in a comparison against high quality ground-based reference measurements starting from 2016. The reference data is measured using broadband radiometers and spectroradiometers presently at a total of 33 locations with wide latitudinal coverage. Overall, the performance of both UV index and spectral irradiance forecasts is high in terms of high accuracy and low bias as well as high correlation in comparison to the reference data. At some measurement sites, improvements near the CAMS model cycle upgrade can be identified from the improved accuracy, while at many sites the improvement is masked behind the high natural variability of UV and behind seasonal changes in UV irradiance. Detailed global and regional changes in UV forecasts between model cycles can be identified by comparing the overlapping periods of two model runs (see reports for SON 2017 and JJA 2018 as well as this report for JJA 2019), and the changes can be associated with, for instance, aerosols, but the improvements in model cycle performance cannot currently be confirmed very effectively due to limited spatial coverage of reference data. Overall, UVI and spectral UV index forecast accuracy (median relative root-mean-square difference rRMSE 0.32) and correlation (median Pearson coefficient 0.92) have improved since 2016, and a portion of that can be accounted for improvements in the UV processor itself and in the CAMS model cycle upgrades. Median relative bias 0.01 improved significantly during JJA 2019 compared to 2018 (rBias 0.09), which was contributed by the implementation of a new, improved CAMS model cycle 46r1 in July 2019. General improvement since 2016 can be seen in CAMS spectral UV irradiance in JJA season. While a significant improvement of correlation and accuracy was observed in spectral UV forecast in Reading and Sodankylä from JJA 2017 to 2018, a decrease in correlation and accuracy between CAMS UV and observed UV was seen between JJA 2018 and 2019. The spectral UV accuracy was, however, improved at Thessaloniki measurement site from JJA 2018 to 2019.



Introduction As part of supporting the development of the ultraviolet (UV) radiation products of CAMS Services for solar radiation, an evaluation of the UV processor against ground-based spectral UV data is carried out on a periodic basis. For this purpose, the best possible quality of ground measurements is required, and data has to be provided in a reasonably short temporal delay after the acquisition.

In the previous report, CAMS UV radiation output was validated against 33 ground-based stations located in Europe, Israel, Australia and Antarctic. The European ground-based measurements are provided from the COST-713 UV Index Database and European UV Database (EUVDB) and FMI delivers data from Marambio, Antarctica. Israelian and Australian-Antarctic ground-based measurements are provided from the Israel Meteorological Service (IMS) and Australian Radiation Protection and Nuclear Safety Agency (ARPANSA), respectively. These ground-based measurements are UV Index (UVI) and spectral UV irradiances. All available data from March to May 2019 (hereafter referred as MAM 2019) were used in the previous report. Furthermore, all available data from January 2016 until May 2019 were used for long-term time series assessments.

This report is the 16th issue in a series of periodic reports, and it focuses on June to August 2019 (JJA 2019). After requesting UV measurements from data providers, all stations from the previous report are still operational and the measurements are available during JJA 2019, except Blindern, Norway, is absent due to missing data. A total of 32 ground based measurement sites were used for CAMS UV evaluation for JJA 2019 and results are compared with those obtained by a similar validation for the same calendar months from the previous three years, 2016, 2017 and 2018.



1. Source of data

1.1 Ground based measurements

Ground-based measurements are obtained from: - 9 European stations from the COST-713 UV Index Database hosted by the Finnish

Meteorological Institute (FMI) - Marambio (Antarctic) and Sodankylä (Finland) provided directly by FMI - Reading (United Kingdom) from EUVDB - Thessaloniki (Greece) station through personal contact. - 16 Australian stations from ARPANSA network - 3 Israelian stations from Israel Meteorological Service (IMS)

Figure 1 displays two maps of all stations in Europe, Israel, and Australia. Table 1 lists all the 32 stations, their corresponding abbreviations, location coordinates, and altitudes.

The data from ground-based measurements consist of UVI and spectral UV irradiance. UVI is obtained from two different types of instruments: multi-band filter radiometer and broadband detectors, while spectral UV is measured with spectroradiometers. The broadband detectors account for a weighting function that mimics that of the erythemal one.

Figure 1: Maps showing 32 stations used for the periodic validation in Europe, Israel and Australia except four stations located in the Antarctic.



Table 1: 32 stations in Europe, Israel, Australia and Antarctic ordered by decreasing latitude and as used for the periodic validation. All stations provide UVI measurements except those stations marked in bold, which are providing only spectral UV measurements. The Sodankylä station provides also spectral UV measurements.

Station Abbr. Latitude Longitude Altitude Ny-Ålesund, Norway NYA 79.0 N 11.83 E 15 m Alomar, Norway AND 69.2 N 16.0 E 380 m Sodankylä, Finland SOD 67.22 N 26.39 E 179 m Trondheim, Norway TRH 63.42 N 10.38 E 70 m Kise, Norway KIS 60.78 N 10.82 E 140 m Finse, Norway FNC 60.58 N 7.57 E 1200 m Bergen, Norway BRG 60.38 N 5.33 E 40 m Østerås, Norway OST 59.92 N 10.75 E 150 m Landvik, Norway LAN 58.33 N 8.52 E 10 m Reading, United Kingdom READ 51.44 N 0.94 W 66 m Florence, Italy FLO 43.82 N 11.20 E 50 m Thessaloniki, Greece THE 40.63 N 22.95 E 60 m Bet-Dagan, Israel BET 32.0 N 35.21 E 25 m Jerusalem, Israel JER 31.78 N 34.96 E 700 m Eilat, Israel EIL 29.55 N 34.81 E 10 m Darwin, Australia DAR 12.42 S 130.89 E 30 m Townsville, Australia TOW 19.33 S 146.76 E 10 m Alice Springs, Australia ALS 23.80 S 133.90 E 550 m Brisbane, Australia BRI 27.45 S 153.03 E 20 m Gold Coast, Australia GLD 28.00 S 153.37 E 10 m Perth, Australia PER 31.92 S 115.96 E 15 m Newcastle, Australia NEW 32.90 S 151.72 E 20 m Sydney, Australia SYD 34.04 S 151.10 E 20 m Adelaide, Australia ADE 34.92 S 138.62 E 10 m Canberra, Australia CAN 35.31 S 149.20 E 580 m Melbourne, Australia MEL 37.73 S 145.10 E 60 m Kingston, Australia KIN 42.99 S 147.29 E 50 m Macquarie Island, Australia MIS 54.50 S 158.94 E 10 m Marambio, Antarctic MAR 64.24 S 56.63 W 198 m Casey, Australia CAS 66.28 S 110.52 E 40 m Mawson, Australia MAW 67.60 S 62.87 E 10 m Davis, Australia DAV 68.58 S 77.97 E 20 m

The very best target for the uncertainty of broadband instruments falls within 10-15% (Seckmeyer et al., 2006). However, for correcting the cosine error, in addition to the solar zenith angle (SZA), the correction method also requires the total column ozone (Bodhaine et al., 1998). It is a laborious task to determine such a calibration. To our knowledge, this correction has not been fully taken into



account in those broadband measurements submitted to the COST-713 UV Index Database. Therefore, standard uncertainties can be estimated as being slightly higher with up to 20%. Norwegian ground-based UV (GUV) instruments, on the other hand, are somewhat different instruments (http://www.comm-tec.com/Prods/mfgs/Biospherical/Brochures/GUV-511&541.pdf, valid on 2019-12-27) and the network is very well maintained. Therefore, we anticipate those standard uncertainties being closer to the 10-15% range.

1.2 CAMS UV data

The UV processor is part of the CAMS Integrated Forecasting System (IFS). It provides UV estimates from July 2012 onwards. A revised version of the UV processor has been implemented and used since 3rd September 2015 (Bozzo et al., 2015). The CAMS model output is provided with 5 nm spectral resolution. Knowing the erythemal action spectrum given by McKinlay-Diffey’s definition (McKinlay and Diffey, 1987), spectral data are integrated to get the UVI. UV irradiances integrated into 5 nm spectral bands from 280 nm to 400 nm are also provided and result in 24 spectral bands. Both UVI and UV irradiances are given for clear and all-sky conditions separately. The outputs are instantaneous values.

Before 3rd September 2015, UVI estimates and UV irradiances were produced every 3 hours. Since then, UVI has been provided every hour (MARS keywords: type=fc, stream=oper, param=2.214/3.214, class=mc, expver=0001, levtype=sfc) whereas the UV irradiances are still provided in the 3–hourly output frequency (MARS keywords: type=fc, stream=oper, param=55.210, class=mc, expver=0001, levtype=ml). The modeled UVI and spectral irradiances evaluated in this report are the forecasted (00 UTC base time every day) UV values closest to observations.

In the past years the CAMS-IFS UV forecasts have been affected by several upgrades, the important ones have been listed in Table 2. Note that all model cycle upgrades, except 41R1_CAMS_hires, include also meteorological changes originating from the upgrades of the underlying IFS weather prediction model. Hence, the differences between model cycles are a combination of improvements in both meteorology and UV related data assimilation and modeling. Further information on changes in both IFS and CAMS-IFS models can be found at https://atmosphere.copernicus.eu/node/326 (valid on 2019-12-27). For information on the dissemination of the current global NRT products see https://confluence.ecmwf.int/display/COPSRV/CAMS+Global (valid on 2019-12-27). UV index forecasts can also be accessed in chart form at https://atmosphere.copernicus.eu/charts/cams/uvindex-forecasts (valid on 2019-12-27)



Table 2: Important UV related changes of CAMS IFS NRT forecasts

CAMS cycle First forecast base time

Implementations potentially affecting UV forecasts

46r1 2019-07-09 Updated emissions for anthropogenic, biogenic, dust and sea salt aerosols. New aerosol species: nitrate and ammonium. Coupling of sulphur species between chemistry and aerosol schemes.

45r1 2018-06-26 Assimilation of NO2 from GOME-2 satellite instrument. Changes in emissions and modeling of sea salt, volcanic and biomass burning and secondary organic aerosols

43r3 2017-09-26 The extraterrestrial UV spectrum Kurudz upgraded to newer ATLAS3. Updates optical properties of organic and sea salt aerosols. Improvements to aerosol modeling. Improved use of ozone information in the UV processor.

43r1 2017-01-24 Improved use of cloud overlap information in UV processor. Changes to dust, sulphate and SOA aerosol modeling. Assimilation of satellite based vertical profiles of ozone (OMPS).

41r1_hires 2016-06-21 Increase in horizontal resolution from T255 to T511 41r1 2015-09-03 New UV processor. UV coupled with prognostic aerosols

instead of climatological aerosols. UV index forecasts produced now hourly in addition to the 3-hourly forecasts. Assimilation of MODIS deep blue AOD and GOME-2 SO2, affecting UV attenuation by aerosols. Modifications in emissions of organic matter and black carbon.

2. Methodology

Before evaluating the performance of the CAMS UV processor, the best quality of the ground–based measurements should be selected. In order to achieve this goal, two main constraints have been applied on ground-based measurements. The first one is that only UV measurements with solar zenith angle (SZA) lower than 88° are used. This criterion has been also applied on ECMWF outputs. Realistically, UV measurements cannot be equal to zero. Therefore, the second constraint is that measurements should be greater than an infinitesimal threshold set empirically to 0.005 UVI. Additional constraints have been applied based on the thresholds for range of 2.5° SZA range based on the 99% and 1% percentiles of long-time series of measurements accounting for the fact that the presence of broken cloudiness in the sky may increase UV fluxes.

The validation in this report uses the full years 2016, 2017, 2018 of CAMS estimated UV data as well as all the available ground-based observational data for the respective stations. It also covers 2019



until end of July and includes the most recent three-month JJA period being the focus of this validation report.

Modeled UV values are compared with measured ones. For spectral comparisons, measured spectral irradiances are integrated over each spectral range of 5 nm to match with the spectral resolution from the UV processor. According to the spectral range of UV outputs, comparisons are then carried out over the spectral range between 290 nm and 400 nm. That results in maximum 22 spectral bands for European stations, each of 5 nm width as spectral resolution.

For each station and for both UVI or UV irradiances, deviations between modeled (estimated) and measured values are computed. Pearson correlation coefficient (CC), bias (Bias), root mean square error (RMSE), relative bias (rBias), and relative RMSE (rRMSE) normalized by the mean of measured values are derived:

𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 = 1

𝑛𝑛 ∑ 𝑌𝑌𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 (𝑗𝑗) − 𝑌𝑌𝑀𝑀𝐸𝐸𝐸𝐸𝐸𝐸𝑀𝑀𝑀𝑀𝐸𝐸𝐸𝐸 (𝑗𝑗)

𝑛𝑛𝑗𝑗=1 (1)

𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 = � 1𝑛𝑛∑ �𝑌𝑌𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸(𝑗𝑗) − 𝑌𝑌𝑀𝑀𝐸𝐸𝐸𝐸𝐸𝐸𝑀𝑀𝑀𝑀𝐸𝐸𝐸𝐸 (𝑗𝑗)�

2𝑛𝑛𝑗𝑗=1 (2)

𝑟𝑟𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 = 1 𝑌𝑌𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀�� 𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 (3)

𝑟𝑟𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 = 1𝑌𝑌𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀�� 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 (4)

where j indicates each value, and n the total number of values. The value Y can be UVI or UV irradiances for each spectral band and Ȳ represents the average value of Y. All of these statistics are computed monthly and seasonally.

3. Results and discussions

3.1 Performance of the UV processor – UV Index

There are 30 stations that provided UV index measurements for this report. Figure 2 shows two examples of the scatterplot between ground-based measurements (horizontal axis) and the CAMS estimates (vertical axis) for the period of JJA 2019 by dark green dots and similarly the same periods of 2016, 2017 and 2018 by orange, yellow and light green dots, respectively, in Østerås, Norway (left side) and in Darwin, Australia (right side). Similar plots for each station are shown in the Appendix.

Østerås (Fig. 2, left side) represents the summer conditions of midlatitude humid continental climate, where CAMS UV index shows small changes in performance and near zero bias. During JJA 2019 CAMS UVI shows lower correlation with measurements (correlation coefficient 0.91) smaller rBias (0.01 and increased scatter (rRMSE 0.42) compared to JJA 2018 (correlation coefficient 0.94,



rBias 0.04 and rRMSE 0.34). Darwin offers an example of a tropical savannah climate with sunny dry season conditions, where UVI bias seems visually smaller in Figure 2 (right side) compared to JJA 2018. The statistics confirm this as rBias are 0.06 and 0.22 in 2019 and 2018, respectively. Also, correlation coefficient and rRMSE improved from 2018 (0.95 and 0.35, respectively) to 2019 (0.97 and 0.20, respectively).

Figure 3 shows the frequency distributions of the measurements in dark blue line, CAMS estimates in orange line and the corresponding differences in light blue line for the period of JJA 2019. Consistent with most other sites, the highest frequencies in Østerås (Figure 3 left side) correspond to the differences close to zero and the 25-, 50 and 75-percentiles of UVI difference are -0.12, 0.01 and 0.18, respectively. In Darwin (Figure 3 right side), the 25-, 50 and 75-percentiles of UVI difference -0.07, 0.23 and 0.57, respectively, reveal that the absolute bias is typically higher and varies more than in Darwin. This is largely dominated by the fact that UVI in stations with smaller average UV levels tends to indicate systematically smaller absolute biases compared to UVI in station with a higher average UV levels.

Figure 4 shows the full time series of UV measurements and CAMS estimates and their corresponding absolute bias, relative bias and RMSE in a moving time window from January 2016 to July 2019 in Darwin, Australia. They were calculated for measurements close to local noon when the Sun is at the highest elevation in the sky. While the seasonal variability of UV index tends to mask some of the developments of CAMS UVI performance, Figure 4 shows that rRMSE medians are smaller in the JJA 2019 (rRMSE 0.20) compared to 2018 (rRMSE 0.35) and 2017 (rRMSE 0.28). This indicates improved accuracy for CAMS UVI. Also, notable improvement in rBias occurred JJA CAMS UVI bias from 2018 (rBias 0.22) to 2019 (rBias 0.06). A notable reduction of rBias occurred also at

Figure 2: Scatterplots between measurements and CAMS estimates of UV Index for JJA in 2019, 2018, 2017 and 2016: Østerås, Norway (left side) and Darwin, Australia (right side).



many other Australian continental stations. Figures similar to Fig. 2, Fig. 3 and Fig. 4 for each station are shown in the Appendix.

Figure 3: Frequency distributions of the measurements, CAMS estimations and their corresponding deviations of UV Index for JJA 2019: Østerås in Norway (left side) and Darwin in Australia (right side).

Figure 4: Time series of measurements and CAMS estimates of UV Index and their corresponding absolute bias, rBias and rRMSE in terms of moving average from January 2016 to August 2019 in Darwin. The labeled vertical lines indicate the upgrades of CAMS model cycles.



JJA seasonal CAMS UVI performance statistics on correlation coefficient, rBias and rRMSE are shown in Figure 5 for different years. 2016, 2017, 2018, and 2019 are given in orange, yellow, light green, and dark green, respectively. It should be noted, that Blindern has missing data for JJA 2019, but statistics for previous year are still shown. Also, the Antarctic sites Casey (CAS), Mawson (MAW) and Davis (DAV) measured maximum UV index of less than 1.0, thus data uncertainties may have a pronounced impact on CAMS UVI performance statistics. A notable decrease in correlation coefficient occurred at high latitude sites Ny-Ålesund (NYA), Bergen (BRG), Østerås (OST), Landvik (LAN), Casey (CAS) and Mawson (MAW) by -0.03 to -0.06.

For this evaluation data set, the CAMS model UVI in JJA 2019 is nearly bias free, as the 25-, 50- and 75-precentiles of rBias are -0.05, 0.01 and 0.07, respectively. The minimum rBias -0.29 was observed in Ny-Ålesund, a high latitude site in the Arctic Ocean. The highest rBiases 0.17 was observed at Davis, an Antarctic measurement site. As a new feature compared to 2017 and 2018, CAMS UVI shows only small rBias in most sites on the Australian continent, except Darwin, Alice Springs and Adelaide. However, Figure 2 (right side) and figures in Appendix reveal, that Darwin and Alice Springs nevertheless show improved CAMS performance in JJA 2019 compared to JJA 2018.

The statistics indicate a generally low level of bias by CAMS UVI. The highest CAMS UVI performance by rBias is seen in Melbourne, Perth, Newcastle, Østerås, Landvik, Sydney, Bergen, Brisbane,

Figure 5: Statistical indicators of comparisons between CAMS estimates and measurements of UV Index at each station for JJA season for years 2016, 2017, 2018 and 2019. BLI stands for Blindern, a Norwegian site absent in this report due to missing data.



Kingston, Goldcoast, Canberra, Townsville, Finse and Trondheim with rBias between ±0.05, while the lowest performance is found in Marambio and Ny-Ålesund rBias outside ±0.20. Total median rBias is significantly lower in JJA 2019 (0.01) compared to JJA 2018 (0.09) for the same sites, and the scatterplots in the Appendix give a more detailed view of changes in UV bias between years.

The median correlation coefficient 0.92 is high meaning that CAMS estimates are again well correlated with measurements. This shows no significant change compared to JJA 2018 correlation coefficient 0.93. Median rRMSE was 0.32 and 0.36 for JJA 2019 and JJA 2018, respectively, which indicates slightly improved forecast accuracy at the measurement sites.

When comparing the total seasonal statistic measures for all sites in different years, not all years may include the same measurement stations, which slightly complicates the interpretation of the results. However, when expressing seasonal total values using medians instead of averages, the total statistics are more robust against inclusion and exclusion of individual sites, which yields a statistic measure better suited for comparing the overall CAMS UV performance between years.

The heatmap in Figure 6 aims for a qualitative overview of the seasonal correlation between CAMS UVI and UVI measurements starting from MAM 2016. As the grey value corresponds the total average correlation coefficient 0.90 over all sites and seasons, then red and blue colors indicate anomalies from the mean value. Firstly, correlation increases over time, because the dominance of blue colors shifts to the dominance of red colors. Secondly, an annual cycle can be seen at the northern high latitude sites with typically negative correlation anomalies in the summer and positive anomalies in the winter seasons. A notable increase in correlation occurred in MAM 2017 after the implementation of 43r1 model cycle.

Figure 7 illustrates the same as Figure 6, but for rRMSE and color anomalies centered on total mean rRMSE 0.42. Similar features can be seen as in Figure 6 for the correlation: 1) increased accuracy (reduction of rRMSE) from 2016 towards 2019, especially after 43r1 and 2) increased accuracy during winter at the northern high latitude sites. Additionally, most Australian continental stations indicate a notable reduction in rRMSE in JJA 2019 compared to earlier JJA seasons.

A similar heatmap for rBias is shown in Figure 8, but here anomalies in blue and red color are with regard to rBias 0.0. The dominance of red and greyish colors indicates an overall shift towards near zero and positive biases near DJF 2017. Again, an annual cycle is apparent in the northern high latitude sites with negative rBiases in the summer and positive in the winter. Most notably in JJA 2019, most Australian continental stations show a reduction of rBias compared to earlier JJA seasons, which supports the same finding noted from Fig. 5.

In Fig. 8 the high rBias values in Ny-Ålesund (NYA) in MAM 2016, Blindern (BLI) during DJF 2017, and 2019 as well as in Landvik (LAN) during DJF 2018 and 2019 stand out from the heatmap, which is likely contributed by the pronounced effect of data uncertainties at low winter time UVI levels. However, the scatter is also higher at low UVI at these sites during DJF than during MAM (see scatterplots in the Appendices of this report and in CAMS72_2018SC1_D72.2.1.1-2019Q2_UV_VAL_201906_v3), which suggests there are also forecast biases present.



Figure 6: Heatmap of Pearson correlation coefficient for each site and for each quarterly period with JJA 2019 as the rightmost column. Grey color is centered at the total mean value 0.90 and white color indicates seasons with less than 50 data points. The implementations of CAMS model cycles are indicated with vertical lines and labels.



Figure 7: Similar heatmap as in Figure 6, but for CAMS UVI rRMSE. Grey color is centered at the total mean rRMSE = 0.42.



Figure 9 shows the relative bias as a function of the solar zenith angle (SZA) for the JJA 2019. In the northern hemisphere 9 (NYA, SOD, TRH, KIS, FNC, OST, FLO, JER, EIL) out of 13 sites show increasing rBias with increasing SZA and two of the sites (BRG, BET) show a decreasing rBias with SZA. In the southern hemisphere, however, 2 (GLD, NEW) out of 17 show increasing rBias with increasing SZA, while 8 (DAR, TOW, ALS, BRI, PER, SYD, MEL, MAR) out of 17 decreasing rBias with increasing SZA.

At the rest of the sites CAMS UVI shows no consistent rBias-SZA dependency. All in all, the relative bias is within approximately -0.2 to 0.3 for most of the stations and for the most of SZA range except Ny-Ålesund (NYA), Townsville (TOW), Brisbane (BRI) and Marambio (MAR) at high SZA values.

Figure 8: Similar heatmap as in Figure 6, but for CAMS UVI rBias. The grey color is centered at rBias = 0.0.



3.2 Performance of the UV processor – spectral UV

There are three stations that provided spectral UV measurements for this report, Reading (UK), Sodankylä (Finland), and Thessaloniki (Greece), with 823, 1189 and 284 observations collocated with CAMS UV, respectively. The study of the spectral UV comparisons is first done for the spectral wavelength band 310-315 nm and then for the full spectrum.

The left side of Figure 10 shows the scatterplot of measured vs. CAMS UV irradiance in the band 310-315 nm in Reading. The overall UVI accuracy in JJA 2019 is high and similar to JJA 2018. The histogram of differences in surface irradiance on the right side of Figure 10 shows that the forecast error in Reading had a near symmetric distribution around 0.0 W/m2. On the other hand, Sodankylä and Thessaloniki (see Appendix) indicate a CAMS UV irradiance low bias at 310 nm to 315 nm compared to the measurements.

Figure 9: Relative bias of UV Index in centered 10° SZA bins for all stations in JJA 2019: northern stations (left side) and southern stations (right side).



The results of spectral comparisons of UV irradiances are summarized in Figure 11 for JJA 2019. Two spectral bands below 300 nm are not displayed, because of very high relative measurement errors due to the very low absolute values of the irradiance and in Sodankylä this limit is set to 305 nm. Also, spectra from Thessaloniki end at 365 nm and from Sodankylä at 325 nm due to limitations of the instrument and data processing. For both Thessaloniki and Reading, the irradiance statistics show striking similarities as a function of wavelength.

Figure 10: Scatterplot (left side) between measurements and CAMS estimates of UV irradiance [W/m2] over 310-315 nm for JJA in 2019, 2018, 2017 and 2016 in Reading. Frequency distributions (right side) of the measurements, CAMS estimates and their corresponding deviations in JJA 2019.



Firstly, for Reading and Thessaloniki the correlation coefficient is high in all spectral bands and decreases slightly as a function of wavelength. Secondly, rBias varies in unison for Reading and Thessaloniki with a near zero average rBias between 310-365 nm. rBias is typically between -0.15 to 0.15, which is of slightly smaller range than for UV index at other stations in Figure 5, where rBias varies typically between -0.2 to 0.2. Sodankylä indicates an exception with reduced correlation coefficient at 305-310 nm and a consistent negative rBias. Thirdly, rRMSE varies as a function of the spectral bands in all stations and shows a similar tendency for each site. Characteristically, the lower rRMSE seems to contribute most to higher correlation coefficient in Thessaloniki compared to Reading, since rBias values are similar for both sites. The overall range for rRMSE of UVI in JJA 2019 is 0.13-0.62, which is slightly lower than the rRMSE range for spectral irradiance. This indicates, that UVI forecasts have slightly higher accuracy than spectral irradiance.

Figure 12, Figure 13, and Figure 14 show the comparison statistics for JJA 2019 and the same for the 3-month period from the previous two years Reading, Sodankylä, and Thessaloniki. Irradiance in Reading (Figure 12) during JJA 2019 compared to JJA 2018 shows a systematic decrease in correlation coefficient of about 0.01 to 0.05 and an increase of rRMSE of 0.02 to 0.06. However, rBias in Reading shows no consistent significant change from 2018 to 2019.

In Sodankylä, Figure 13 indicates increased rRMSE of 0.05 to 0.13, which contributes to the decrease in correlation coefficient of approximately 0.02 to 0.06 from JJA 2018 to JJA 2019. The negative rBias enhanced by -0.02 at wavelengths longer than 310 nm.

Figure 11: Statistical indicators of UV irradiance in each spectral band for JJA 2019 for Reading, Sodankylä and Thessaloniki. Each marker is centered in its 5 nm band.



In Thessaloniki from JJA 2018 to JJA 2019 (Figure 14) there is a consistent reduction of rRSME by -0.15 to -0.2 and an increase in correlation coefficient of 0.05 to 0.10. Meanwhile, rBias was reduced by approximately -0.05 in the whole spectral range.

Figure 12: Statistical indicators of UV irradiance for all spectral bands for Reading for JJA 2019 and the same for years 2016, 2017 and 2018. Each marker is centered in its 5 nm band.



Figure 15 displays the spectral rBias in 10-degree SZA bins for Reading and Sodankylä for JJA 2019 and Figure 16 shows the same for Thessaloniki. In this evaluation data set, rBias shows no consistent features or trends between the measurement sites or SZA bins.

Figure 13: Statistical indicators of UV irradiance for all spectral bands for Sodankylä for JJA 2019 and the same for years 2016, 2017 and 2018. Each marker is centered in its 5 nm band.

Figure 14: Statistical indicators of UV irradiance for all spectral bands for Thessaloniki for JJA 2019 and the same for years 2016, 2017 and 2018. Each marker is centered in its 5 nm band.



Figure 15: Relative bias of UV irradiance as a function of wavelength, for different solar zenith angle bins, and for JJA 2019. Reading site is on the left side and Sodankylä on the right side. Each marker is centered in its 5 nm band.



4. CAMS e-suite UV evaluation The CAMS data in JJA 2019 was produced with the o-suite model cycle 45r1 until 2019-07-09 when the cycle 46r1 was implemented as the new o-suite model. Prior to its implementation, 46r1 was run as e-suite cycle in parallel with the o-suite 45r1, which allows for overlapping comparison between UV forecasts from the two different model cycles. This section first compares the UV index estimates between the two CAMS model versions and then evaluates both model cycles against ground based reference UV index measurements. The UV related changes in the e-suite are summarized in Table 2. Figure 17 shows the global map of average difference of daily maximum UV index between e-suite and o-suite. e-suite 45r1 shows four distinct features in how the daily maximum UV index changed in comparison to 46r1. Firstly, there is a small -0.2 UVI to -0.6 UVI systematic decrease in UV index in the tropics approximately between the Tropic of Cancer (23.4°N) and the Tropic of Capricorn (23.4°S). This smaller UVI is more realistic at the Australian stations Darwin (12.4°S), Townsville (19.3°S), Alice Springs (23.8°S), Brisbane (27.5°S), Gold Coast (28.0°S) and Perth (31.9°S), which shows in Fig. 5 and Fig. 8 as reduced rBias when comparing CAMS UV index with measurements. Secondly, there is a small to medium -0.2 UVI to -1.0 UVI decrease in UV index in the North Sahara, Middle East and Indian domains and an even stronger reduction of -0.6 UVI to -4.9 UVI in the arid

Figure 16: Relative bias of UV irradiance as a function of wavelength, for different solar zenith angle bins, and for JJA 2019 in Thessaloniki. Each marker is centered in its 5 nm band.



and savannah type climate regions near southern Saharan Desert. This is likely explained by the coincident increase in e-suite dust aerosol optical thickness (AOD) reported by Basart et al. (2019). Thirdly, small to medium 0.2 UVI to 2.0 UVI increase occurred in west Sahara, Arabian Peninsula and in the Gobi Desert regions. These changes are likely connected with coincident decreases in dust AOD in the same regions, also reported by Basart et al. (2019). Fourthly, UVI index increased near active volcanoes, which is likely in connection with the new coupling between the chemistry and aerosol schemes of sulphur species. The rest of the globe shows near zero average change in UVI between -0.2 UVI to 0.2 UVI.

Figure 18 shows the zonal mean changes in UVI index. The zonal mean absolute UV index (left side) decreased most notably in the tropics by -0.46 UVI at greatest, which means roughly -2% to -3.5% in relative terms (right side). The negative relative difference is stronger -3 to -5% in the southern high latitudes, however, the corresponding absolute UV index was lower than 1.0 UVI due low sun winter conditions.

Figure 17: mean difference of maximum daily UV index between e-suite and o-suite.



While Figures 17 and 18 reveal the global e-suite UV index developments with regard to the overlapping o-suite UV index, Figures 19, 20 and 21 compare the statistical performance of both e-suite and o-suite UV index with regard to ground based observations. Figure 19 shows the Pearson correlation coefficient of e-suite UV index (y-axis) at each measurement site (one site per dot) compared to correlation coefficient of o-suite (x-axis). Increase in UVI correlation coefficient shows as dots above 1-1 line and ideally, all dots should line up horizontally at y=1.0. Overall, most southern station indicate no change in correlation coefficients, while most northern sites show slight increase.

Figure 18: zonal mean of maximum daily UV index (left) and the zonal mean relative difference (right) of maximum daily UV index between e-suite and o-suite.



A similar representation of rBias is shown in Figure 20. Here model UVI improvement shows as dots that are positioned away from 1-1 line towards the horizontal line y=0. Generally, 9 sites show no remarkable change in rBias (change in rBias between -0.02 to 0.02), 14 sites show a weaker rBias and 8 sites a stronger rBias in e-suite compared to o-suite.

Figure 19: correlation coefficient between CAMS UV index and measured UV index. Each dot represent one measurement station, colored by latitude. x and y axes correspond o-suite and e-suite UV index, respectively.

Figure 20: same as Fig 19, but showing UVI rBias.



Finally, a similar representation of rRMSE is presented in Figure 21. Improvement in model UVI shows here as dots below 1-1 line and ideally all dots should be located at y=0. 24 sites indicate a decreased rRMSE, while 7 sites show increased rRMSE.

5. Conclusions

For JJA 2019 period, in general, UV processor estimates of UV index are well correlated to measurements. For UVI correlation coefficients are higher than 0.85 at all but two stations with total median 0.92 showing little difference to JJA 2018 (0.93). rRMSE median for all sites was 0.32, which improved from JJA 2018 (0.36). Most notable improvement in UVI was in median rBias, which reduced from 0.09 in JJA 2018 to 0.01 in JJA 2019. This was contributed by a CAMS model cycle upgrade, which reduced model UV index in the tropics, as confirmed by UVI measurements at several Australian measurement sites.

As for UVI, the spectral UV irradiances from the UV processor also show a high forecast performance when compared against ground-based spectral measurements. The correlation coefficient is high, varying spectrally between 0.77 and 0.95 at the measurement sites in Reading, Sodankylä, and Thessaloniki during JJA 2019. Spectral rBias varies between -0.23 to 0.18, while rRMSE was mostly in range 0.18 and 0.8. In the spectral data comparison CAMS UV accuracy was lowest in Sodankylä.

Figure 21: same as Fig 19, but showing UVI rRMSE.



6. References

Bodhaine Barry A., E. G. Dutton, R. L. Mckenzie, and P. V. Johnston, "Calibrating broadband UV instruments: Ozone and solar zenith angle dependence", Journal of Atmospheric and Oceanic Technology, 15(4):916-926, 1998.

Bozzo A., Arola A., Cesnulyte V., Pitkänen M. Report on implementation of spectral UV irradiance, MACC-III deliverable D57.2, work package 122, Feb. 2015.

McKinlay, A. F. and Diffey, B. L., “A reference action spectrum for ultraviolet induced erythema in human skin” in: Commission International de l’Éclairage (CIE), Research Note, 6 (1), 17–22.

Seckmeyer G., A. Bais, G. Bernhard, M. Blumthaler, C.R. Booth, K. Lantz, R.L. McKenzie, P. Disterhoft, and A. Webb (2006), Instruments to measure solar ultraviolet radiation. Part 2: Broadband instruments measuring erythemally weighted solar irradiance. Available at: https://library.wmo.int/index.php?lvl=notice_display&id=12616, WMO/GAW No. 164 World Meteorological Organisation, Geneva.

Basart, S, A. Benedictow, Y. Bennouna, A.-M. Blechschmidt, S. Chabrillat, Y. Christophe, E.Cuevas, H. J. Eskes, K. M. Hansen, O. Jorba, J. Kapsomenakis, B. Langerock, T. Pay, A.Richter, N. Sudarchikova, M. Schulz, A. Wagner, C. Zerefos, Upgrade verification notefor the CAMS real-time global atmospheric composition service: Evaluation of thee-suite for the CAMS upgrade of July 2019, Copernicus Atmosphere Monitoring Service(CAMS) report, CAMS84_2018SC1_D3.2.1-201907_esuite_v1.pdf, July 2019,doi:10.24380/fcwq-yp50.

7. Appendix This section shows the results for all stations ordered by decreasing latitude in Europe, Australia as well as the Antarctic sites. For each station, three graphs are provided. The first graph on the top left is the scatterplot between measurements and CAMS estimates for JJA period and each year. The second graph on the top right is the frequency distribution of measurements (dark blue line), CAMS estimates (orange line) and deviations (light blue line). The third graph on the bottom is the full times series of measurements and CAMS estimates close to local noon, their corresponding bias on top and the rBias and rRMSE on the bottom in terms of moving average.


atmosphere.copernicus.eu copernicus.eu ecmwf.int

ECMWF - Shinfield Park, Reading RG2 9AX, UK Contact: [email protected]

Validation Reportof the CAMS UV processor Issue …...between model cycles can be identifiedby...

Documents

Transcript of Validation Reportof the CAMS UV processor Issue …...between model cycles can be identifiedby...