Atlantic -Iberian Biscay Irish- IBI Production Centre IBI ......the CMEMS Products User Manual...

Atlantic -Iberian Biscay Irish- IBI Production Centre

IBI_ANALYSIS_FORECAST_WAV_005_005

Issue: 4.0

Contributors: Cristina Toledano, Alice Dalphinet, Pablo Lorente, Marta de Alfonso, Malek Ghantous, Lotfi Aouf, Marcos García Sotillo

Approval date by the CMEMS product quality coordination team: 18/06/2020

QUID for IBI MFC Products


Ref:

Date:

Issue:

CMEMS-IBI-QUID-005-005

3 April 2020

4.0

/ 46

CHANGE RECORD

When the quality of the products changes, the QuID is updated and a row is added to this table. The third column specifies which sections or sub-sections have been updated.

Issue Date § Description of Change Author Validated By

1.0

23/12/2016

All

Creation of the QUID document for the IBI MFC NRT Wave forecast product (operational launch coincident with CMEMS V3 Service Release)

Marcos Sotillo, Lotfi Aouf, Marta Alfonso, Romain Renaud, Cristina Toledano, Pablo Lorente, Alice Dalphinet

Enrique Álvarez Fanjul , A. Melet

2.0 23/01/2018

All

Update of the QUID for the IBI NRT Wave forecast product to include V4 changes.

Marcos Sotillo, Lotfi Aouf, Marta Alfonso, Romain Renaud, Cristina Toledano, Pablo Lorente, Alice Dalphinet

Enrique Álvarez Fanjul , A. Melet

3.0 18/04/2019 All Update of the QUID for the new release of IBI NRT Wave forecast product (RFC CCIBI-125, CMEMS-II)

Cristina Toledano, Alice Dalphinet, Pablo Lorente, Marta de Alfonso, Malek Ghantous, Lotfi Aouf, Marcos Sotillo.

Mercator Ocean

3.1 04/09/2019 All Update of QUID to include the change in atmospheric forcing frequency (from 3-hour to hourly winds; RFC CCIBI-XX, CMEMS-II)

Cristina Toledano, Pablo Lorente, Marta de Alfonso, Marcos Sotillo.

Mercator Ocean

4.0 27/03/2020 All Update of the QUID for the new release of IBI NRT Wave forecast product (RFC CCIBI-XX, CMEMS-II)

Cristina Toledano, Malek Ghantous, Alice Dalphinet, Pablo Lorente, Marta de Alfonso, Lotfi Aouf, Marcos Sotillo.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

TABLE OF CONTENTS

I Executive summary ..................................................................................................................................... 4

I.1 Products covered by this document ........................................................................................................... 4

I.2 Summary of the results .............................................................................................................................. 4

I.3 Estimated Accuracy Numbers ..................................................................................................................... 5

II Production system description .................................................................................................................... 8

III Validation framework ............................................................................................................................... 11

IV Validation results ...................................................................................................................................... 14

IV.1 Pre-Operational Qualification ................................................................................................................ 14

V System’s Noticeable events, outages or changes ...................................................................................... 35

VI Quality changes since previous version ..................................................................................................... 42

VII References ............................................................................................................................................ 46



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

I EXECUTIVE SUMMARY

I.1 Products covered by this document

This document describes the quality of the CMEMS Atlantic IBI -Iberian Biscay Irish- Wave Analysis and Forecast product (CMEMS product catalogue identifier: IBI_ANALYSIS_FORECAST_WAV_005_005).

The operational IBI Wave Forecasting system provides a 5-days regional wave forecast, updated twice a day (cycles at 00 h and 12 h). The system is based on the MFWAM model code running at approximately a 5-Km horizontal resolution (1/20º). The IBI WAV runs are driven by 1-hourly analysed ECMWF winds and use boundary conditions (wave spectra) from the global CMEMS wave system. In this latest release, the assimilation of altimeter wave data and the ocean current forcing have been implemented. The data assimilation IBI WAV system uses an optimal interpolation algorithm with constant model and observation errors from Janson2 and 3, Saral, Cryosat2 and Sentinel 3 altimeters. The ocean currents forcing are from the IBI circulation model.

As part of this regional IBI wave product, hourly instantaneous fields are delivered, including wave Height, Period and Direction for total spectrum and fields of Wind Wave (or wind sea), Primary Swell Wave and Secondary Swell for partitioned wave spectra.

The IBI forecast product is updated twice a day. Together with the +5D forecast data from the latest forecast bulletin available, the concatenation of IBI best estimates are delivered (hindcast data for the Day-1 date), composing the historic IBI wave time series (going back to the 1st January 2018).

Further information on the latest version of the IBI-MFC Wave Forecast System and its associated product (the IBI_ANALYSIS_FORECAST_WAV_005_005) is provided in Section II of this document and in the CMEMS Products User Manual Document (CMEMS-IBI-PUM-005-005-v4.0).

I.2 Summary of the results

Quality assessment is a key issue in CMEMS. In that sense, before any new operational release, the CMEMS IBI Monitoring & Forecasting Centre (IBI-MFC) performs a qualification of the proposed future IBI system before this new model system version becomes fully operational. This qualification phase is based on scientific assessment of the hindcast component of the IBI-WAV MFC product IBI_ANALYSIS_FORECAST_WAV_005_005, measuring the quality of any new updated version of the IBI forecast system and evaluating their products together with the previous IBI operational ones in order to quantify potential added values associated to the novelties, as well as to verify the existence of non-regression in terms of quality with respect to the previous IBI solutions.

The present IBI QUID document shown outcomes from this pre-operational qualification phase. Thus, the quality of 2 years (period assess: from 01/01/2018 to 31/12/2019) of the hindcast component of the IBI WAV regional product IBI_ANALYSIS_FORECAST_WAV_005_005 has been assessed by comparison with observations, both from in-situ moorings and satellite remote sensed products. The resulting metrics are shown in the Sections IV.1 and IV.2, respectively. In these sections, different product quality information is provided, focusing also on different IBI sub-regions. EANs (Estimated Accuracy Numbers, which are essential statistics) have also been estimated, averaged over the whole



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

IBI domain, obtained from long-term comparisons with reference observations. EANs give a measure of the IBI WAV model performance along the historical time series of products available in the CMEMS Catalogue. The IBI WAV EANs are provided in Section I.3. Finally, the impact of the implementation data assimilation and current forcing is shown in Section VI.1

The wave model validation in the IBI region with in situ data has been performed through three parameters: significant wave height (SWH), mean wave period (MWP, with Tm02 estimator), and mean wave direction (MWD). The general results show a very good performance of the model. The headline results from these Qualification Phase are the following ones:

Significant wave height:

The comparison of IBI WAM model products against observations from mooring buoys and satellite altimeter shows a low bias (2 cm), RMSD of 28 cm and a high correlation coefficient (0.95). As expected, the satellite validation results are appreciably advancement for the new model with data assimilation. The metrics with insitu measurements present a slight improvement in Western Mediterranean region (WSMED sub-region).

Mean Wave Period:

In terms of mean period, the IBI model also presents a very good skill with mean values (averaged over the whole IBI region) of 0.11 and 0.95 seconds for the bias and the RMSD, respectively. The correlation coefficient is about 0.85, consistent but not as high as the obtained for SWH. The results are appereciably better for deep water buoys than for coastal buoys.

Mean Wave Direction:

The qualification of this parameter presents a consistent but a lower skill. The RMSD and scatter index for MWD are 29.4 degrees and 0.16, respectively. Again, the results are better for deep water buoys.

I.3 Estimated Accuracy Numbers

This section is devoted to providing essential statistics obtained from long-term comparisons with reference information. A consistent estimation of the relevant accuracy levels for the IBI-MFC WAV products of the CMEMS catalogue is provided. Most of the validation procedures are based on comparisons of the model solution with in-situ and satellite observational data delivered by the CMEMS In-Situ TAC and other non-CMEMS satellite products.

These Estimated Accuracy Numbers (EAN) numbers have been calculated for a 2-year period (01/01/2018-31/12/2019) using the best estimates of the IBI-WAV operational system.

In previous issues the EANs were calculated using several altimeters: Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3. In the present issue the data derived from these altimeters were assimilated into the wave model, so an independent data source was needed for validation. The accuracy numbers shown in Table 1 were thus derived from the comparison of IBI-WAV best estimates with SWH data derived from the HY-2 satellite’s altimeter. The provided values are derived from statistics computed using hourly data from 1/1/2018 to 31/12/2019, averaged over the whole IBI service domain.

Table 2, Table 3 and Table 4 show the EAN set for the significant wave height, the mean wave period and mean wave direction, respectively, computed over the whole IBI area for the period 01/01/2018-31/12/2019, using in-situ observational data from 42 mooring buoys (provided by the CMEMS In-Situ



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

TAC). Take into consideration that only part of those mooring buoys have the available observations of mean wave direction or mean wave period.

Significant Wave Height (SWH). OBS Reference: satellite-derived products

IBI Service Domain

Period 2018 2019

Bias (m) 0.02 0.03

Scatter Index (%) 10.9 10.3

Table 1: IBI EAN related to SWH (year period). IBI-WAV best estimates used in the metric computation. Observational source used as reference: HY-2 satellite-derived product.

Significant Wave Height (SWH). OBS Reference: In-situ observation from moorings.

IBI Service Domain

MOORING BUOYS BIAS (m) RMSD (m) SI

COASTAL -0.014 0.277 0.322

DEEP WATER -0.027 0.292 0.206

TOTAL -0.018 0.289 0.230

Table 2: IBI EAN related to SWH (period 01/01/2018-31/12/2019). IBI-WAV best estimates used in the metric computation. Observational source used as reference: data from 42 in situ mooring buoys. The different EANs were computed (Bias, RMSD and scatter index (SI)) using all the available buoys and considering separately the coastal and deep-water ones

Mean Wave Period (MWP). OBS Reference: In-situ observation from moorings.

IBI Service Domain

MOORING BUOY BIAS (s) RMSD (s) SI

COASTAL 0.295 1.011 0.230

DEEP WATER -0.364 -0.364 0.144

TOTAL -0.221 0.879 0.162

Table 3: IBI EAN related to MWP (period 01/01/2018-15/03/2019). IBI-WAV best estimates used in the metric computation. Observational source used as reference: data from 37 in situ mooring buoys. The different EANs were computed (Bias, RMSD and scatter index (SI)) using all the available buoys and considering separately the coastal and deep water ones.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Mean Wave Direction (MWD). OBS Reference: In-situ observation from moorings.

IBI Service Domain

MOORING BUOY BIAS (degrees) RMSD (degrees) SI

COASTAL 4.48 44.48 0.26

DEEP WATER -0.13 24.63 0.13

TOTAL 0.97 29.4 0.16

Table 4: IBI EAN related to MWD (period 01/01/2018-31/12/2019). IBI-WAV best estimates used in the metric computation. Observational source used as reference: data from 31 in situ mooring buoys. The different EANs were computed (Bias, RMSD and scatter index (SI)) using all the available buoys and considering separately the coastal and deep water ones.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

II PRODUCTION SYSTEM DESCRIPTION

Production Center

The CMEMS IBI MFC NRT ocean forecast service is generated by the IBI-MFC Production Centre. Nologin, in coordination with Puertos del Estado, is responsible of the operational suites to produce, disseminate, and validate the near real time IBI wave forecast products. The CMEMS IBI MFC Production line is:

CMEMS IBI MFC PU: Puertos del Estado (PdE) IBI-PUERTOS-MADRID-SP (Production Line)

Production System

The CMEMS IBI-MFC WAVE System

The IBI MFC wave system is based on an operational suite, run by NoLogin-Puertos del Estado in the CESGA Supercomputing facilities, with the objective to produce a near-real-time short-term (5-days) forecast of wave parameters. The wave model code used as a basis of the IBI-WAV application is the MFWAM, which is the operational wave model of Météo-France. The MFWAM model has been upgraded with IFS-ECWAM-41R2 code from the ECMWF. We recall that the MFWAM model uses the ST4wave physics as developed by Ardhuin et al. (2010) to model the dissipation by wave breaking and the swell damping source terms. The model physics has been updated with major improvements thanks to the FP7 European Research project Mywave (see Mywave final report). During phase 1 of CMEMS-IBI the ST4 wave physics of the MFWAM model has been adjusted by including a Philipps spectrum for the high frequency tail of the wave spectrum.

The MFWAM model is implemented on the IBI domain (see Figure 1.a) with a grid size of approximately 5 km (1/20º) and the bathymetry used in the IBI-wave model has been generated smoothing the ocean bathymetry of ETOPO1 (https://sos.noaa.gov/datasets/etopo1-topography-and-bathymetry/), with 1 arc-minute of spatial resolution (1/60 of degree). The wave spectrum is discretized in 24 directions and 30 frequencies ranging from 0.035 Hz and 0.56 Hz.

In the new release (July 2020) of the CMEMS IBI-MFC WAV system, the assimilation of significant wave height from altimeters have been implemented. The assimilation scheme is based on optimal interpolation as described by Lionello et al. (1992) and it is the same scheme used in the CMEMS global wave system. For the IBI ocean domain we set the correlation length and the distance of influence of the observations to 170 km and 650 km, respectively. In this release the ratio of background and observation errors are kept constant over the IBI domain. In other respects, by using empirical wave growth laws (Lionello et al. 1992) the analysed SWH after the assimilation induces a correction on the wave spectra mostly on the wind sea part in the frequency scale (Aouf and Lefevre 2015). The assimilation scheme has been adjusted with the ST4 physics used in the IBI-wave model. The new release of the IBI-wave system also includes the use of off-line surface currents forcing obtained from the IBI ocean circulation model. The upgraded IBI wave system includes improved computations of coupling parameters such as the surface stress, and wave-breaking induced turbulence in the ocean mixed layer.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

The CMEMS-IBI wave model is driven by hourly analysed ECMWF winds (1/8º spatial resolution) and uses boundary conditions (wave spectra) from the GLOBAL CMEMS wave system (1/10º spatial resolution). The IBI-wave model benefits of the most accurate boundary conditions because of the assimilation of altimeters wave data in the global CMEMS wave system.

The IBI-wave model performs a partitioning technique on wave spectra over all ocean grid points of the IBI domain. The partitioning technique is based on the watershedding method developed for image processing (Gerling, 1992). This process effectively treats the wave spectrum as a topographic map from which individual peaks in wave energy can be identified in order to define the separate wave components. First the wave spectrum is split in wind sea and swell wave spectra. Then the partitioning is applied for the swell wave spectrum. The peaks on the spectrum are isolated and they are considered as partitions. Afterward the classification of swell partitions in primary and secondary swell is performed depending on the mean energy of each partition.

The operational IBI wave system provides hourly wave parameters (twice a day) till a 5-day forecast horizon (see further details on the IBI wave product in Table 5).

Figure 1: a) Significant wave height of the IBI-MFC wave model application. Snapshot of the raw model output. The IBI_ANALYSIS_FORECAST_WAV_005_005 product is delivered through the CMEMS catalogue only for the IBI Service Domain (depicted by the red rectangle); b) example of wave spectrum with two main wave systems: a south-east wind sea and north swell.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Product Specification IBI_ANALYSIS_FORECAST_WAV_005_005

Geographical coverage 19°W -> 5°E ; 26°N ->56°N Product delivered to users on the IBI Service Domain.

Model Set-Up MFWAM code + 1h ECMWF analysed and forecasted winds as forcing + Daily CMEMS GLOBAL wave data as Boundaries conditions (wave spectra) + data assimilated from L3 CMEMS altimeter data + IBI CMEMS currents as forcing. Model domain: 20°W-17°E ; 25°N-64,6°N

Variables Spectral significant wave height (Hm0) Spectral moments (-1,0) wave period (Tm-10) Spectral moments (0,2) wave period (Tm02) Wave period at spectral peak /peak period (Tp) Mean wave direction from (Mdir) Wave principal direction at spectral peak Stokes drift U Stokes drift V Spectral significant wind wave height Spectral moments (0,1) wind wave period Mean wind wave direction from Spectral significant primary swell wave height Spectral moments (0,1) primary swell wave period Mean primary swell wave direction from Spectral significant secondary swell wave height Spectral moments (0,1) secondary swell wave period Mean secondary swell wave direction from

Analysis Yes.

Forecast Yes

Available time series 01/01/2018 – Present time (+ 5Days forecast).

Temporal resolution Instantaneous fields with hourly frequency

Target delivery time Two 5-day Forecast bulletins per day. The IBI-WAV 00H and 12H cycles available in the CMEMS catalog at 14:00UTC and 23:59UTC, respectively.

Delivery mechanism All the CMEMS User Interface mechanisms: SubSetter, FTP and DirectGetFile

Horizontal resolution 1/20º

Format Netcdf CF1.0

Table 5: IBI_ANALYSIS_FORECAST_WAV_005_005 Product Specification



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

III VALIDATION FRAMEWORK

A year trial of IBI-WAV has been performed in order to assess the quality of the hindcast product relative to available observations. This pre-operational work, described in Section IV, consists in qualification with satellite-derived products and in situ observation (mooring buoys) for the period 01/01/2018-31/12/2019, covering IBI domain: 25ºN,20ºW to 17ºE-64,6ºN.

For this period, the wave parameters that have been qualified are significant wave height (Hs) and Average zero crossing wave period (Tm02). Table 6 cross-references these wave parameters with the IBI_ANALYSIS_FORECAST_WAV_005_005 product files.

Generic name Model variable CMEMS convention long name

Significant wave height (SWH) VHM0 Spectral significant wave height (Hm0)

Mean wave period (MWP) VTM02 Spectral moments (0,2) wave period (Tm02)

Mean wave direction VMDR Mean wave direction from

Table 6: Terminology used in the pre-operational qualification to cross-reference product data names

Figure 2 shows the locations of the mooring buoys available with their CMEMS unique ID code used in this product quality assessment (01/01/2018-31/12/2019) in the IBI region delivered by CMEMS INSITU-TAC. A more comprehensive description of the in situ observations and the results is in Section IV.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 2: Locations and CMEMS ID code buoys employed to the qualification.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

In the validation framework, one of the top priority issues of the CMEMS IBI-MFC Team is to comprehensively evaluate the quality of the new IBI-WAV operational products provided. Analyzing the quality of the IBI oceanographic forecast system operational performances in terms of reliability and accuracy is mandatory for the CMEMS IBI-MFC to properly inform its end-users and stakeholders about the products confidence level. Furthermore, increasing knowledge on the model solution aids to identify areas where potential improvements in the IBI system can be achieved, making the product evolve towards more advanced versions thanks to reliable upgrades, and encourage interoperability and collaboration between scientific R&D studies and the operational service. Therefore, qualification of each new IBI system releases and validation of their NRT operational products quality constitute a core activity in the CMEMS IBI-MFC.

The accuracy of the present IBI-WAV-NRT product will be operationally assessed by means of the so-called North Atlantic Regional VALidation (NARVAL) tool (Lorente et al., 2018 and 2019; Sotillo et al., 2015). Once put in place, the quality of the latest daily forecast will be monitored every single day through a comparison against both quality-controlled in situ (coastal and deep-water buoys) and remote-sensed observations (satellite-derived estimations). In the first case, three parameters will be analyzed (significant wave height, period and direction), whereas in the second case only the significant wave height will be evaluated. Basic quality indicators will be automatically generated, such as scatter plots, histograms and timeseries comparisons.

Complementarily, the quality of IBI-WAV-NRT product will be also analyzed through NARVAL tool on “delayed mode”: the aforementioned validations will be performed on a monthly, seasonal and annual basis, placing special emphasis on regionalization (i.e., computation of skill metrics for specific subregions within IBI regional domain). In this framework, it is worthwhile mentioning that a variety of skill metrics derived from such comparisons will be regularly computed, stored in netcdf files and delivered to the CMEMS Product Quality Dashboard (PQ-D) where any end-user will be able to freely access such information. Here we provide the web link: http://marine.copernicus.eu/services-portfolio/scientific-quality/

Finally, since IBI-WAV-NRT is dynamically embedded into the GLOBAL-WAV-NRT system, a parent-son model intercomparison will be conducted on a regular basis to infer the consistence of the downscaling approach adopted and quantify the added-value of the nested model solution.

http://marine.copernicus.eu/services-portfolio/scientific-quality/

http://marine.copernicus.eu/services-portfolio/scientific-quality/



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

IV VALIDATION RESULTS

IV.1 Pre-Operational Qualification

As discussed in Section III, a year of hindcast has been carry out in order to evaluate the quality of the IBI_ANALYSIS_FORECAST_WAV_005_005 (period 01/01/2018 – 31/12/2019). Comparisons with observations from satellite remote sensed products and in-situ moorings are shown.

IV.1.1 Qualification with satellite-derived products

Significant wave heights from MFWAM and altimeter SWH data have been compared over the IBI domain. Because data from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters is now being assimilated into the model, an independent altimeter data source is needed. Thus the following diagnostics were performed by comparing the model to the HY-2 satellite altimeter processed by the French Space Agency CNES. Because these results are not directly comparable to those presented in previous issues of this document, we present, where pertinent, results for the model without data assimilation validated using the same HY-2 altimeter data.

Scatter index and bias are computed for the whole domain and also for different regions depending on latitudes. The model bias with altimeter data is shown in Figure 3. The bias appears to be greater in the northern part of the domain where significant wave heights are typically higher.

The scatter index for the combined 2018 and 2019 model runs with data assimilation is shown in Figure 4 and is summarized in the scatter plots of Figure 5, with the color bar representing the density of observations (observations per data point). These scatter plots show the improved model performance when data is assimilated in two ways: (1) the line of best fit (red line) approaches the ideal 1:1 gradient (black line) when data is assimilated, and (2), the spread about the fit is reduced. We can also see that a few outlying points are eliminated when data is assimilated.

The bar plots in Error! Reference source not found. shows, respectively for 2018 and 2019, scatter index by season. The domain is additionally divided into three – southern, central and northern. The differences do not follow much of a pattern: though in general the scatter index is higher in the higher latitudes, this is not always the case, and there is a notable exception in summer 2018, where the scatter index exceeds roughly 13% in the lower latitudes. It also appears as though the differences are less accentuated in 2019 than in 2018.

Bar plots for bias are shown in Figure 7 , for 2018 and 2019 respectively. Biases are overall small with less than 5 cm. We can remark that biases are greater in 2019, as reflected in the table, but with one exception: winter biases are lower in 2019 than in 2018 for all three sub-regions. Biases are also generally highest in spring and summer, but otherwise there is no easily discernible pattern.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 3: Model bias relative to the HY-2 altimeter SWH observations throughout the IBI domain. 2018 and 2019 combined. Data assimilated from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters (L3 CMEMS-TAC-waves).

Figure 4: Scatter index for model against HY-2 altimeter SWH observations throughout the IBI domain. 2018 and 2019 combined. Data assimilated from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters (L3 CMEMS-TAC-waves).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 5: Scatter plot of altimeter SWH data (from the HY-2 satellite) versus model with data assimilated from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters. The red line shows the mean slope between model and data, with the black line representing a perfect fit. 2018 and 2019 combined.

Figure 6: Scatter index by season for 2018 (left) and 2019 (right). Data assimilated from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters. Statistics for June were left out due to errors in the HY-2 altimeter data for that month.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 7: Model biases compared with the HY-2 altimeter SWH observations. Biases are shown per season and per geographic region for 2018 (left) and 2019 (right). Data assimilated into the model are from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters (L3 CMEMS-TAC-waves).

IV.1.2 Qualification with in situ observations

The second part of the IBI wave qualification for the pre-operational period (01/01/2018-31/12/2019) consists in the validation of the parameters significant wave height, mean wave period and mean wave direction from the IBI-WAV product against in-situ observations from mooring buoys available in the IBI region (compiled in the product delivered by the CMEMS IBI INSITU-TAC). Figure 8 shows a map with the location of all the available mooring buoys (42) used to perform the IBI wave qualification.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 8: Location of the coastal (orange dots) and deep waters (green dots) buoys employed to conduct the IBI qualification during the period 01/01/2018-31/12/2019. The IBI service domain (IBISR, red rectangle) has been split into different sub-regions of interest, which are denoted with rectangles of different colours.

Significant wave height comparisons against in-situ data show a good agreement between the modelled and observed sample. Figure 9 reveals the spatial distribution of the statistical metrics obtained for each buoy employed in the present qualification exercise. Every dot represents the value of bias (left) and root mean squared difference (RMSD, right) obtained for the period comprised between 01/01/2018 and 31/12/2019. As it can be observed, such metrics mostly emerged in the ranges [-0.2 m, 0.2 m] and [0 m, 0.3 m], respectively.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 9: Maps of skill metrics derived from the comparison of the significant wave height provided by the IBI-WAV product and the observations from the mooring buoys: bias (left) and root mean squared difference (right) obtained for the pre-operational year (2018/2019).

Likewise Figure 10 shows the distribution of the correlation index (left) and the scatter index (right) for the period, lying predominantly in the ranges [0.96-1] and [0-0.35], respectively.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 10: Maps of skill metrics derived from the comparison of the significant wave height provided by IBI-WAV product and the observations from the mooring buoys: correlation index (left) and scatter index (right) obtained for the pre-operational year (2018/2019).

A summary of the regional statistical results of the significant wave height (SWH) derived from the qualification exercise for each sub-region and every mooring buoy in detail is provided in Table 7, Table 8 and Error! Reference source not found.. Metrics are gathered also for the whole IBI service domaine using only coastal (CO) and Deep-water (DW) mooring buoys in Table 10. Results are slightly better for deep water than for coastal buoys, with the best estimations for NIBSH and WIBSH areas (north-west spanish coast).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

SUB-REGION: ICANA

MOORING BUOY N BIAS RMSD SI LR SLOPE

LR OFFSET CORR

1300130 (DW) 14414 -0.12 0.25 0.14 0.87 0.10 0.94

1300131 (DW) 15508 -0.13 0.22 0.26 0.71 0.11 0.87

TOTAL 29922 -0.12 0.23 0.20 0.79 0.11 0.91

SUB-REGION: CADIZ

MOORING BUOYS N BIAS RMSD SI LR SLOPE

LR OFFSET CORR

6200085 (DW) 17305 -0.01 0.20 0.16 0.97 0.04 0.96

TARIFA (CO) 16146 0.33 0.50 0.78 1.00 0.33 0.74

TOTAL 33451 0.16 0.35 0.47 0.98 0.18 0.85

SUB-REGION: WSMED


LR OFFSET CORR

6100196 (DW) 15880 -0.08 0.29 0.22 0.85 0.11 0.96

6100197 (DW) 17190 -0.06 0.27 0.19 0.92 0.05 0.97

6100198 (DW) 16619 -0.01 0.22 0.21 0.89 0.11 0.96

6100280 (DW) 17471 -0.09 0.23 0.24 0.88 0.02 0.94

6100281 (DW) 17379 -0.09 0.21 0.27 0.87 0.02 0.92

6100417 (DW) 15902 -0.07 0.26 0.26 0.84 0.09 0.92

6100430 (DW) 16768 -0.13 0.18 0.25 0.83 0.06 0.95

6101404 (CO) 16896 -0.03 0.20 0.32 0.92 0.02 0.91

BARCELONA (CO) 16318 -0.14 0.22 0.30 0.90 -0.07 0.94

MELILLA (CO) 14650 0.01 0.19 0.35 1.05 -0.01 0.88

TARRAGONA (CO) 16499 0.01 0.19 0.35 1.05 -0.01 0.88

TOTAL 167280 -0.06 0.23 0.27 0.88 0.05 0.93

SUB-REGION:GIBST


LR OFFSET CORR

6100198 (DW) 16619 -0.01 0.22 0.21 0.89 0.11 0.96

6101404 (CO) 16896 -0.03 0.20 0.32 0.92 0.02 0.91

MELILLA (CO) 14650 0.01 0.19 0.35 1.05 -0.01 0.88

TARIFA (CO) 16146 0.33 0.50 0.78 1.00 0.33 0.74

TOTAL 64311 0.08 0.28 0.42 0.89 0.15 0.89

Table 7: Summary of skill metrics obtained from the comparison of SWH: IBI-WAV products against the coastal (CO) and deep-water (DW) mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the 10 sub-regions used by the IBI MFC to validate regionally the IBI products. N, RMSD, SI, CORR, LR_SLOPE and LR_OFFSET represent, respectively, the number of data analysed, the root mean squared error, the scatter index, the correlation index and the results of the best linear fit (continues in next page).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

SUB-REGION:WIBSH


LR OFFSET CORR

6200083 (DW) 16385 0.02 0.28 0.11 0.93 0.20 0.98

6200084 (DW) 17466 0.01 0.26 0.10 0.93 0.20 0.98

6200191 (DW) 15712 0.04 0.27 0.11 0.93 0.22 0.98

6200192 (DW) 14961 0.02 0.24 0.10 0.93 0.20 0.98

6200199 (CO) 10871 -0.03 0.29 0.11 0.89 0.26 0.98

6201065 (CO) 15466 -0.08 0.32 0.15 0.86 0.20 0.97

TOTAL 90861 0.00 0.28 0.11 0.91 0.21 0.98

SUB-REGION:NIBSH


LR OFFSET CORR

6200024 (DW) 13395 -0.06 0.28 0.14 0.90 0.15 0.98

6200025 (DW) 15291 -0.03 0.25 0.12 0.94 0.10 0.98

6200082 (DW) 16244 -0.05 0.30 0.12 0.92 0.17 0.98

6201030 (DW) 13215 -0.04 0.24 0.11 0.93 0.11 0.98

BILBAO (CO) 14308 -0.18 0.30 0.20 0.90 -0.03 0.96

DONOSTIA (DW) 12493 -0.02 0.26 0.16 0.85 0.22 0.98

TOTAL 84946 -0.06

0.27 0.14 0.91 0.12 0.98

SUB-REGION GOBIS


LR OFFSET CORR

6200024 (DW) 13395 -0.06 0.28 0.14 0.90 0.15 0.98

6200025 (DW) 15291 -0.03 0.25 0.12 0.94 0.10 0.98

6200082 (DW) 16244 -0.05 0.30 0.12 0.92 0.17 0.98

6200083 (DW) 16385 0.02 0.28 0.11 0.93 0.20 0.98

6201030 (DW) 13215 -0.04 0.24 0.11 0.93 0.11 0.98

6201065 (CO) 15466 -0.08 0.32 0.15 0.86 0.20 0.97

BILBAO (CO) 14308 -0.18 0.30 0.20 0.90 -0.03 0.96

DONOSTIA (DW) 12493 -0.02 0.26 0.16 0.85 0.22 0.98

62001 (DW) 16112 0.13 0.31 0.12 0.99 0.14 0.98

62103 (DW) 16023 0.29 0.42 0.29 1.00 0.29 0.96

62107 (DW) 15887 0.06 0.30 0.21 1.03 0.25 0.97

62163 (DW) 15887 0.06 0.30 0.10 0.97 0.16 0.98

62305 (DW) 15925 0.47 0.56 0.66 1.08 0.39 0.93

TOTAL 183721 0.06 0.33 0.19 0.95 0.18 0.97

SUB-REGION ECHAN


LR OFFSET CORR

62103 (DW) 16023 0.29 0.42 0.29 1.00 0.29 0.96

62304 (DW) 13527 0.38 0.47 0.73 1.01 0.37 0.88

62305 (DW) 15925 0.47 0.56 0.66 1.08 0.39 0.93

TOTAL 45475 0.38 0.48 0.56 1.03 0.35 0.92

Table 8: Summary of skill metrics obtained from the comparison of SWH: IBI-WAV products against the coastal (CO) and deep-water (DW) mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the

10 sub-regions used by the IBI MFC to validate regionally the IBI products. N, RMSD, SI, CORR, LR_SLOPE



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

and LR_OFFSET represent, respectively, the number of data analysed, the root mean squared error, the scatter index, the correlation index and the results of the best linear fit (continues in next page).

SUB-REGION IRISH


LR OFFSET CORR

62091 (DW) 3971 -0.03 0.27 0.23 0.87 0.12 0.94

62092 (DW) 10366 0.02 0.32 0.10 0.95 0.19 0.99

62093 (DW) 6166 0.09 0.31 0.12 0.96 0.18 0.97

62094 (DW) 15391 -0.03 0.25 0.13 0.93 0.11 0.98

TOTAL 35894 0.01 0.29 0.15 0.93 0.15 0.97

Table 9: Summary of skill metrics obtained from the comparison of SWH: IBI-WAV products against the coastal (CO) and deep-water (DW) mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the 10 sub-regions used by the IBI MFC to validate regionally the IBI products. N, RMSD, SI, CORR, LR_SLOPE and LR_OFFSET represent, respectively, the number of data analysed, the root mean squared error, the scatter index, the correlation index and the results of the best linear fit.

REGION: IBISR


LR OFFSET CORR

COASTAL (8) 121154 -0.01 0.28 0.32 0.90 0.10 0.91

DEEP (29) 429998 0.02 0.29 0.20 0.92 0.15 0.96

TOTAL (37) 564906 0.01 0.29 0.23 0.92 0.14 0.95

Table 10: Summary of skill metrics obtained from the comparison of SWH: IBI-WAV products against the coastal and deep-water mooring buoys for the period 01/01/2018/-31/12/2019.Metrics averaged for the entire IBI Service Area (IBISR). N, RMSD, SI, CORR, LR_SLOPE and LR_OFFSET represent, respectively, the number of data analysed, the root mean squared error, the scatter index, the correlation index and the results of the best linear fit.

Figure 11,Figure 12 and Figure 13 illustrate a comparative exercise where IBI-WAV accuracy is evaluated by using both a deep-water and a coastal buoy in nearby (subregion WIBSH). More specifically, the results reveal a slight decrease in IBI-VAW performance in coastal areas as the scatter and QQ plots are not as good for 6201065 coastal buoy as it is for the neighbor deep water buoy (buoy code buoy 6200083). In general, the skill metrics gathered in Table 7, Table 8 and Table 9 support this statement on the IBI WAV performance in coastal areas, although coastal Significant Wave Height metrics have improved with the new smoothed bathymetry ETOPO1 and the spatial model resolution of 1/20º.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 11: Comparison of IBI-WAV model product against observational data from 2 buoys located near Langosteira for the period 01/01/2018-31/12/2019. Time series (deep water buoy 6200083, on the top, and on the bottom, the coastal buoy 6201065)

Figure 12: Comparison of IBI-WAV model product against observational data from 2 buoys located near Langosteira for the period 01/01/2018-31/12/2019. Scatter plots, the red (green) line represents the best (perfect) linear fit. Deep water buoy 6200083 on the left, and on the right, the coastal buoy 6201065.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 13: Comparison of IBI-WAV model product against observational data from 2 buoys located in Langosteira for the period 01/01/2018-31/12/2019. Q-Q plots for deep water buoy 6200083 on the left, and on the right, the coastal buoy 6201065.

With regards to the Mean wave period, Figure 14 represents the distribution of the bias and the RMSD which are in the ranges [-1 s, 1 s] and [0 s, 2.5 s], respectively. Figure 15 illustrates the correlation and scatter index obtained for this wave parameter, revealing significantly high values for the former (above 0.9) and values lower than 0.4 for the latter.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 14: Maps of skill metrics derived from the comparison of the Mean Wave Period (MWP) provided by the IBI product and the observations from the mooring buoys: bias (left) and root mean squared difference (right) obtained for the pre-operational year (2018/2019).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 15: Maps of skill metrics derived from the comparison of the Mean Wave Period (MWP) provided by the IBI product and the observations from the mooring buoys: correlation index (left) and scatter index (right)

obtained for the pre-operational year (2018/2019).

A summary of the regional statistical results of the Mean Wave Period derived from the qualification exercise for each sub-region and every mooring buoy in detail is provided in Table 11 and Table 12. Metrics are gathered also for the whole IBI service domaine using only coastal (CO) and Deep-water (DW) mooring buoys in Table 13. The results are appereciably better for deep water buoys than for coastal buoys.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

SUB-REGION: ICANA

MOORING BUOY N BIAS RMSD SI LR SLOPE

LR OFFSET CORR

1300130 (DW) 14414 0.25 0.73 0.13 1.11 -0.37 0.86

1300131 (DW) 15508 0.19 1.02 0.24 0.89 0.68 0.45

TOTAL 29922 0.2 0.88 0.18 1.00 0.16 0.66

SUB-REGION: CADIZ


LR OFFSET CORR

6200085 (DW) 17305 0.22 0.73 0.15 1.09 -0.21 0.89

TARIFA (CO) 16146 0.84 1.68 0.43 1.01 0.80 0.54

TOTAL 33451 0.53 1.21 0.29 1.05 0.30 0.72

SUB-REGION: WSMED


LR OFFSET CORR

6100196 (DW) 15880 -0.36 0.56 0.13 1.01 -0.39 0.91

6100197 (DW) 17190 -0.27 0.56 0.12 0.96 -0.10 0.91

6100198 (DW) 16619 -0.11 0.55 0.13 0.85 0.49 0.81

6100280 (DW) 17471 -0.23 0.58 0.15 0.93 0.05 0.82

6100281 (DW) 17379 -0.05 0.61 0.16 1.05 -0.24 0.80

6100417 (DW) 15902 -0.08 0.61 0.15 0.85 0.52 0.78

6100430 (DW) 16768 -0.38 0.63 0.14 0.95 -0.18 0.86

6101404 (CO) 16896 -0.08 0.95 0.28 0.71 0.92 0.59

BARCELONA (CO) 16318 -0.06 0.66 0.17 0.94 0.16 0.78

MELILLA (CO) 14650 0.40 1.02 0.28 0.89 0.80 0.66

TARRAGONA (CO) 16499 -0.08 0.77 0.20 0.79 0.73 0.70

TOTAL 167280 -0.21 0.59 0.14 0.94 0.02 0.84

SUB-REGION:GIBST


LR OFFSET CORR

6100198 (DW) 16619 -0.11 0.55 0.13 0.85 0.49 0.81

6101404 (CO) 16896 -0.08 0.95 0.28 0.71 0.92 0.59

MELILLA (CO) 14650 0.40 1.02 0.28 0.89 0.80 0.66

TARIFA (CO) 16146 0.84 1.68 0.43 1.01 0.80 0.54

TOTAL 64311 0.26 1.05 0.28 0.86 0.75 0.65

SUB-REGION:WIBSH


LR OFFSET CORR

6200083 (DW) 16385 0.15 0.53 0.08 1.00 0.14 0.95

6200084 (DW) 17466 0.09 0.60 0.09 0.98 0.24 0.94

6200191 (DW) 15712 -0.23 0.57 0.08 0.97 0.00 0.95

6200192 (DW) 14961 -0.18 0.53 0.08 1.00 -0.19 0.96

6200199 (CO) 10871 -0.09 0.63 0.08 0.97 0.15 0.94

6201065 (CO) 15466 0.67 1.08 0.17 1.12 -0.10 0.90

TOTAL 90861 0.07 0.66 0.10 1.01 0.04 0.94

Table 11: Summary of skill metrics obtained from the comparison of MWP: IBI-WAV products against the

coastal (CO) and deep-water (DW) mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the 10 sub-regions used by the IBI MFC to validate regionally the IBI products. N, RMSD, SI, CORR, LR_SLOPE and LR_OFFSET represent, respectively, the number of data analysed, the root mean squared error, the scatter index, the correlation index and the results of the best linear fit (continues in next page).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

SUB-REGION:NIBSH


LR OFFSET CORR

6200024 (DW) 13395 0.21 0.68 0.11 0.96 0.45 0.94

6200025 (DW) 15291 0.21 0.71 0.11 1.05 -0.14 0.93

6200082 (DW) 16244 0.08 0.53 0.08 0.97 0.31 0.95

6201030 (DW) 13215 -0.11 0.56 0.08 0.99 -0.04 0.96

BILBAO (CO) 14308 0.76 1.30 0.23 0.19 -0.31 0.86

DONOSTIA (DW) 12493 -0.39 0.95 0.14 0.94 0.04 0.90

TOTAL 84946 0.13 0.79 0.12 1.02 0.05 0.92

SUB-REGION GOBIS


LR OFFSET CORR

6200024 (DW) 13395 0.21 0.68 0.11 0.96 0.45 0.94

6200025 (DW) 15291 0.21 0.71 0.11 1.05 -0.14 0.93

6200082 (DW) 16244 0.08 0.53 0.08 0.97 0.31 0.95

6200083 (DW) 16385 0.15 0.53 0.08 1.00 0.14 0.95

6201030 (DW) 13215 -0.11 0.56 0.08 0.99 -0.04 0.96

6201065 (CO) 15466 0.67 1.08 0.17 1.12 -0.10 0.90

BILBAO (CO) 14308 0.76 1.30 0.23 0.19 -0.31 0.86

DONOSTIA (DW) 12493 -0.39 0.95 0.14 0.94 0.04 0.90

TOTAL 116797 0.20 0.79 0.12 1.03 0.04 0.92

SUB-REGION IRISH


LR OFFSET CORR

62091 (DW) 3971 -0.49 0.71 0.16 0.93 -0.19 0.85

62092 (DW) 10366 -0.17 0.45 0.06 0.97 0.06 0.96

62093 (DW) 6166 -0.09 0.42 0.06 0.99 -0.05 0.95

62094 (DW) 15391 -0.28 0.53 0.09 0.96 -0.04 0.92

TOTAL 35894 -0.26 0.53 0.09 0.96 -0.04 0.92

Table 12: Summary of skill metrics obtained from the comparison of MWP: IBI-WAV products against the coastal (CO) and deep-water (DW) mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the 10 sub-regions used by the IBI MFC to validate regionally the IBI products. N, RMSD, SI, CORR, LR_SLOPE and LR_OFFSET represent, respectively, the number of data analysed, the root mean squared error, the scatter index, the correlation index and the results of the best linear fit

REGION: IBISR


LR OFFSET CORR

COASTAL (8) 121154 0.29 1.01 0.23 0.95 0.39 0.74

DEEP (23) 335492 -0.08 0.62 0.11 0.97 0.03 0.88

TOTAL (31) 471060 0.01 0.72 0.14 0.97 0.13 0.85

Table 13: Summary of skill metrics obtained from the comparison of MWP: IBI-WAV products against the

coastal and deep-water mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the entire IBI Service Area (IBISR). N, RMSD, SI, CORR, LR_SLOPE and LR_OFFSET represent, respectively, the number of data analysed, the root mean squared error, the scatter index, the correlation index and the results of the best linear fit.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 16 depicts the precision of IBI-WAV during the period 01/01/2018-31/12/2019 analyzed in four different locations, selected according to their relative latitude within the IBISR spatial domain. As it can be observed, consistent metrics are obtained in the entire Atlantic Façade, from south to north: in ICANA sub-region (Buoy 1300130, located in Gran Canaria), WSMED sub-region (Buoy 6100197, located in Mahon), GOBIS sub-region (Buoy 6200083, located in Langosteira) and finally IRISH sub-region (Buoy 62092).

Figure 16: Comparison of IBI-WAV against four buoys for the period 01/01/2018-31/12/2019. Scatter plots and best linear fit (red line) for the Mean Wave Period (MWP). The green line represents the perfect fit.

MW

P

Mo

de

l (s

)

MW

P

Mo

del

(s

)

MW

P

Mo

del

(s

)

MW

P

Mo

del

(s

)

MWP Buoy (s)

MWP Buoy (s)

MWP Buoy (s)

Buoy 6100197

Buoy 6200083 Buoy 62092

MWP Buoy (s)

Buoy 1300130



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

The wave mean direction information was not available for many of the 42 buoys used for the qualification document. For this reason, only 29 buoys have been used to validate with the wave model. For instance, there were no data buoys to validate this parameter in the English Channel.

Figure 17 shows the distribution of the bias (left) and the RMSD (right) for the direction, lying predominantly in the ranges [-6º,10º] and [0º,45º]. Some coastal buoys (Melilla, Tarifa) reveals results outside this range with worse results. It can also be observed how the results are worse in Mediterranean than in Atlantic façade.

Figure 17: Maps of skill metrics derived from the comparison of the Mean Wave Direction (MWD) provided

by the IBI product and the observations from the mooring buoys: bias (left) and root mean squared difference (right) obtained for the pre-operational year (2018/2019).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

A summary of the regional statistical results of the Mean Wave Derived derived from the qualification exercise for each sub-region and every mooring buoy in detail is provided in Table 14 and Table 15 . The best results are in WIBSH sub-region (Galician-Portuguese coast).

SUB-REGION: ICANA MOORING BUOY N BIAS RMSD SI

1300130 (DW) 14414 -8.41 16.48 0.14 1300131 (DW) 15508 7.17 34.21 0.31

TOTAL 29922 -0.62 25.34 0.23 SUB-REGION: CADIZ

MOORING BUOYS N BIAS RMSD SI 6200085 (DW) 17305 2.18 23.18 0.09

TARIFA (CO) 16146 48.50 97.25 0.54 TOTAL 33451 25.34 60.22 0.32

SUB-REGION: WSMED MOORING BUOYS N BIAS RMSD SI

6100196 (DW) 15880 4.32 37.73 0.17 6100197 (DW) 17190 1.01 34.25 0.20 6100198 (DW) 16619 2.67 41.84 0.26 6100280 (DW) 17471 -1.99 37.57 0.21 6100281 (DW) 17379 -2.85 28.36 0.30 6100417 (DW) 15902 -5.80 45.95 0.31 6100430 (DW) 16768 -0.43 36.09 0.24

BARCELONA (CO) 16318 -0.16 34.40 0.25 MELILLA (CO) 14650 -26.18 108.50 0.69

TARRAGONA (CO) 16499 8.94 28.74 0.25 TOTAL 150384 -0.44 38.83 0.24

SUB-REGION:GIBST MOORING BUOYS N BIAS RMSD SI

6100198 (DW) 16619 2.67 41.84 0.26 MELILLA (CO) 14650 -26.18 108.50 0.69 TARIFA (CO) 16146 48.50 97.25 0.54

TOTAL 47415 8.33 82.53 0.50 SUB-REGION:WIBSH

MOORING BUOYS N BIAS RMSD SI 6200083 (DW) 16385 -2.54 13.59 0.05 6200084 (DW) 17466 -1.93 12.25 0.04 6200191 (DW) 15712 1.72 9.46 0.03 6200192 (DW) 14961 1.40 9.69 0.03 6200199 (CO) 10871 8.37 10.54 0.04 6201065 (CO) 15466 -5.77 10.11 0.03

TOTAL 90861 0.21 1094 0.04

Table 14: Summary of skill metrics obtained from the comparison of MWD: IBI-WAV products against the

coastal (CO) and deep-water (DW) mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the 10 sub-regions used by the IBI MFC to validate regionally the IBI products. N, BIAS, RMSD and SI represent, respectively, the number of data analysed, the bias, the root mean squared difference and the scatter index. (continues in next page).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

SUB-REGION:NIBSH

MOORING BUOYS N BIAS RMSD SI

6200024 (DW) 13395 -1.63 16.18 0.05

6200025 (DW) 15291 -5.88 14.73 0.05

6200082 (DW) 16244 -5.26 18.94 0.07

BILBAO (CO) 14308 -2.33 11.82 0.04

DONOSTIA (DW) 12493 -1.95 12.80 0.04

TOTAL 71731 -3.41 14.89 0.05

SUB-REGION GOBIS


6200024 (DW) 13395 -1.63 16.18 0.05

6200025 (DW) 15291 -5.88 14.73 0.05

6200082 (DW) 16244 -5.26 18.94 0.07

6200083 (DW) 16385 -2.54 13.59 0.05

6201065 (CO) 15466 -5.77 10.11 0.03

BILBAO (CO) 14308 -2.33 11.82 0.04

DONOSTIA (DW) 12493 -1.95 12.80 0.04

TOTAL 103582 -3.62 14.02 0.05

SUB-REGION IRISH


62091 (DW) 3971 11.56 33.36 0.21

62092 (DW) 10366 -1.23 15.30 0.06

62093 (DW) 6166 1.70 17.91 0.07

62094 (DW) 15391 3.10 22.05 0.10

TOTAL 35894 3.78 22.15 0.11

Table 15: Summary of skill metrics obtained from the comparison of MWD: IBI-WAV products against the coastal (CO) and deep-water (DW) mooring buoys for the period 01/01/2018/-31/12/2019. Metrics averaged for the 10 sub-regions used by the IBI MFC to validate regionally the IBI products. N, BIAS, RMSD and SI represent, respectively, the number of data analysed, the bias, the root mean squared difference and the scatter index

Some wave direction roses from different mooring stations are shown in

Figure 18 to illustrate the model capability to reproduce the mean wave direction parameter.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 18 : Mean direction wave roses for three different sub-regions: on the top 1300130 buoy , located in Menorca (WSMED region), in the middle, 6100197 buoy, located in Gran Canaria (ICANA region) and on the bottom 6200192 buoy, located on Portuguese coast (WIBSH region).

6200192

1300130

6100197

BUOY

BUOY

BUOY

MODEL

MODEL

MODEL



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

V SYSTEM’S NOTICEABLE EVENTS, OUTAGES OR CHANGES

This section is dedicated to track and describe changes that have occurred in the operational system (including model system version upgrades or changes related to upstream data updates). Specific validation results associated to some dedicated assessment that focus on these changes are also provided.

Currently, the CMEMS IBI-MFC Wave forecast product (IBI_ANALYSIS_FORECAST_WAV_005_005) presents a temporal coverage that goes from the 1st January 2018 to the present. The historic data is composed of best estimates obtained from a hindcast data produced by the Near-Real-Time IBI operational forecast system. The IBI MFC wave forecast system is aimed to be continuously evolved, updating the service based on upgraded operational suites and numerical model applications. Table 16 shows the evolution of the IBI service (in terms of main operational versions and main changes associated to each updated IBI system) along the CMEMS service.

Operational launch

End of operations

Description of novelties associated to the change/event

19/04/2017 21/03/2018 First release of the IBI MFC Wave operational System. The CMEMS-V3 IBI WAV Product (data available from year 2014) generated with this first operational IBI WAV Forecast System release.

22/03/2018 08/07/2019 With the V4 CMEMS release, a minor change in the IBI wave model has been implemented. This change consists in using a minimum water depth of 5 m, adjustment of the dissipation source term and the tail of the wave spectrum for the high frequencies in order to improve the surface stress for the coupling with the ocean model.

09/07/2019 02/12/2019 Wave model 1/20º product introduced in IBI domain. This is a new product replacing the 1/10º MFWAM and is based on MFWAM +ETOPO1 smoothed bathymetry.

03/12/2019 07/07/2020 The IBI NRT WAV system updates the ECMWF wind forcing. Temporal frequency of the forcing enhanced (from the previous 3 hour data to the present hourly winds).

07/07/2020 … Implementation of SWH altimeter data assimilation using optimal interpolation algorithm and off line coupling of current forcing.

Table 16: Historical evolution of the IBI MFC operational forecast system (along the CMEMS Service time). For each system version is provided the period in operations for production of IBI MFC NRT Wave Forecast Products, as well as its main novelties with respect to the previous wave model system.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

A description of the major IBI set-up changes occurred along the CMEMS -Phase II- IBI service is provided bellow. This historical review does not include the latest change implemented in the IBI NRT wave system because this latest upgrade, and its impacts in the solution, is described in detail in the next section.

Qualitative impacts of the atmospheric forcing upgrade: temporal frequency increase; from 3-h to hourly winds (release December 2019)

The IBI WAV NRT system updates its atmospheric forcing in its December 2019 release. The temporal frequency of the ECMWF forecast winds used as forcing is upgraded from the previous 3-h frequency to an hourly one. IBI-WAV-NRT is presently driven using short-range wind forecasts provided by ECMWF with a 1-hourly temporal resolution.

The resulting impact of temporal resolution improvement of the wind compared to the one used in the previous release (3-hourly wind resolution) is illustrated using IBI-WAV model performances for a selected month (March 2019). Figure 19 compares bias (model-observed) of significant wave height generated by the IBI-WAV model forced both with 1-hourly winds (top results) and 3-hourly winds (on the bottom). Both IBI wave solutions are consistent, being quite analogous. As it can be seen in the following figures very small differences are obtained, with a very slight improvement of the new set-up in almost the entire IBI area, but in the English Channel area.

Some timeseries from different mooring stations are shown in Figure 20 to illustrate the local impact of this atmospheric wind frequency change. In general, it is seen that both model solutions are almost identical, but during the storm peaks and their decay. This behaviour can be seen in panels on the top, where observation and model timeseries from moorings located in the Bay of Biscay and the English Channel (where statistical bias, as shown in Figure 19, decreases and increases, respectively when used hourly winds instead the 3-h ones). Finally, bottom panel shows an example from a Mediterranean location, where improvements in terms of model capability to better shaping the storm peaks seems to be higher after upgrading the atmospheric forcing.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 19: SWH bias between in-situ observations (CMEMS moorings buoys) and wave models, the new operational model forced by 1 h-hourly wind (top number, green and in case of worse than 3 h wind bias, in red), and previous version forced by 3 h-hourly wind (bottom numbers, black) for March 2019. Pink boxes for the locations where time series are shown (Figure 20)



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 20: Comparison of IBI-WAV significant wave height for March 2019. The three time series are located in different sub-regions: on the top buoy 62163, located in English Channel (GOBIS region), in the middle, buoy 6200191 (WIBISH region), and on the bottom buoy 61197 (WSMED region), located in Mediterranean sea.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Qualitative impact of model resolution change (release July 2019)

The IBI WAV NRT system duplicates its horizontal resolution (from 1/10º to 1/20º). The joint action of the smoothed bathymetry and this spatial resolution upgraded has led to better Significant Wave Height error metrics. Energy bunching areas that were present in the previous release have been partly alleviated. The trade-off of the higher computational cost is justified with the improved model skill.

Figure 21 shows the improvement in the dissipation of the energy bunching in the Golf of Biscay with the new spatial resolution (1/20º).

Figure 21: Daily maps of Significant Wave Height (SWH) for the previous operational model (left) and the new IBI WAV NRT (right), date 20181101, at lead time +12 h.

Comparing with the previous version, the IBI subregion NIBISH and GOBIS revels better metrics due to the wave model tends to underestimate less on the Catabrian coast with the new ETOPO1 smoothed bathymetry. Figure 22 shows the results of the validation against mooring buoys of both operational models in this area.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 22: SWH bias between in-situ observations (CMEMS buoys) and wave models, the new operational model (top number, blue), and previous version (bottom numbers, black) for the period 16/03/2018-15/03/2019.

Figure 23 illustrates the analogies of both models in deep water (new IBI WAV operational model and previous one), where the qualification with in-situ data is very similar for both wave parameter, significant wave height and wave period. The Q-Q plots exposed are calculated for the IRISH subregion (Buoy 62095) and GOBIS/WIBSH subregion (Buoy 62083) and reveals an underestimation for waves higher than 8 meters.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 23: Comparison IBI WAV model against previous one. QQ plots for the new operational model (IBI WAV model) and previous model (V4) of Significant Wave Height (SWH, left) and Mean Wave Period (MWP, right) for the deep water buoys 62083 and 62095, period 16/03/2018-15/03/2019



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

VI QUALITY CHANGES SINCE PREVIOUS VERSION

This section aims to provide a description of the impacts, in terms of solution and product quality, related to main changes introduced the IBI WAV NRT system. To this aim, comparison between the new model performance and the previous IBI operational one is provided. Qualitative impacts of the data assimilation and current forcing implementation in the IBI-WAV-NRT July 2020 release are shown.

In previous versions, the L3 CMEMS-TAC-wav products were used to qualify the model against satellite-derived product. Due to data from Jason2 and 3, Saral, Cryosat-2 and Sentinel-3 is now being assimilated into the model, the metrics were performed by comparing the model to the HY-2 satellite altimeter processed by the French Space Agency CNES. The resulting impact of the changes in the new system is illustrated using IBI-WAV model performances for the 2018 and 2019 year. Table 17 shows the Estimated Accuracy Numbers (EAN) calculated for the period 01/01/2018-31/12-2019 in the IBI WAV NRT previous version (no DA) and the new one (DA).

IBI Service Domain

Period 2018 no DA 2018 DA 2019 no DA 2019 DA

Bias (m) -0.03 0.02 -0.05 0.03

Scatter Index (%) 12.3 10.9 11.5 10.3

Table 17: IBI EAN related to SWH (year period). IBI-WAV best estimates used in the metric computation. Observational source used as reference: HY-2 satellite-derived product.

The implementation of the assimilation of altimeters SWH in the new IBI system improves significantly the scatter index of SWH by roughly 10% regarding to the validation with independent SWH from the Chinese altimeter mission HY2A. The scatter plots in figures also show that the IBI new wave system induces a slight best slope comparing to the old system. Figure 24 indicates the reduction of scatter index of SWH for ranges of SWH from 1 to 6 meters induced by DA in the new IBI wave system.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 24: Variation of scatter index of SWH (in %) for different range of significant wave height The range of SWH are 0.5-1.5m, 1.5-3m, 3-4.5m, 4.5-6m,>6m. Green color, le previous IBI WAV NRT model version and orange color, the new one.

Figure 25 shows the improved model performance when data is assimilated in two ways: (1) the line of best fit (red line) approaches the ideal 1:1 gradient (black line) when data is assimilated, and (2), the spread about the fit is reduced. We can also see that a few outlying points are eliminated when data is assimilated.

Figure 26 and Figure 27 show quantile-quantile plots of the model against the HY-2 altimeter SWH observations. For most of the wave height range, the data assimilation brings the model closer to the satellite observations, however there are notable exceptions. The very lowest wave heights, below 2 m, tend to be overcorrected and are overestimated when data is assimilated. Meanwhile, for 2018 the very highest waves, above 10 m mostly, are also overcorrected and overestimated, but there are fewer data in this range. This doesn’t appear to be a problem for 2019.



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 25 : Scatter plot of altimeter SWH data (from the HY-2 satellite) versus model with data assimilated

from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters (left, new IBI WAV NRT version) and no data assimilation (right, previous IBI WAV NRT version). The red line shows the mean slope between model and data, with the black line representing a perfect fit. 2018 and 2019 combined.

Figure 26: QQ plots for 2018 of model against the HY-2 altimeter SWH observations. On the left, without data assimilation and on the right, model with data assimilated from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters (L3 CMEMS-TAC-waves).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

Figure 27: QQ plots for 2019 of model against the HY-2 altimeter SWH observations. On the left, without data assimilation and on the right, model with data assimilated from the Jason-2, Jason-3, Saral, Cryosat-2 and Sentinel 3 altimeters (L3 CMEMS-TAC-waves).



Ref:

Date:

Issue:


3 April 2020

4.0

/ 46

VII REFERENCES

Ardhuin, F., et al. (2010), Semi-empirical dissipation source functions for wind-wave models: Part I, Definition, calibration and validation, J. Phys. Oceanogr., 40(9), 1917–1941.

Aouf Lotfi, Cristina Toledano, Arancha Amo, Marcos G. Sotillo. 2019. CMEMS PRODUCT USER MANUAL for Atlantic -Iberian Biscay Irish- Wave Physics Analysis and Forecast Product: IBI_ANALYSIS_FORECAST_WAV_005_005. CMEMS Technical Report (www.marine.copernicus.eu)

Gerling (1992) : Partitioning sequences and arrays of directional ocean wave spectra intocomponents wave systems. J. Atmos. Oceanic Tech.

Janssen et al. (2014), Final report of Work Package I of the FP7 research project “My wave”.

Lorente P., Piedracoba S., Sotillo M.G., Aznar R., Amo-Balandron, A.,Pascual, A., Soto-Navarro J., Alvarez-Fanjul, E. (2016). “Ocean model skill assessment in the NW Mediterranean using multi-sensor data.Journal of Operational Oceanography”. doi: 10.1080/1755876X.2016.1215224

Lorente, P., M.G. Sotillo, L. Aouf, A. Amo-Baladrón, E. Barrera, A. Dalphinet, C. Toledano, R. Rainaud, M. De Alfonso, S. Piedracoba, A. Basañez, J. M. García-Valdecasas, V. Pérez-Muñuzuri, and E. Álvarez-Fanjul. Extreme Wave Height Events in NW Spain: A Combined Multi-Sensor and Model Approach. Remote Sensing, 2018, 10, 1; doi:10.3390/rs10010001.

Sotillo M G, S. Cailleau, P. Lorente, B. Levier, R. Aznar, G. Reffray, A. Amo-Baladrón, J. Chanut, M. Benkiran E. Alvarez-Fanjul (2015): The MyOcean IBI Ocean Forecast and Reanalysis Systems: operational products and roadmap to the future Copernicus Service, Journal of Operational Oceanography, DOI: 10.1080/1755876X.2015.1014663

http://www.marine.copernicus.eu/

Atlantic -Iberian Biscay Irish- IBI Production Centre IBI ......the CMEMS Products User Manual...

Documents

Transcript of Atlantic -Iberian Biscay Irish- IBI Production Centre IBI ......the CMEMS Products User Manual...