The (Mis)understanding of Quality in Climate Services Delivery

Adriaan Perrels, Athanasios Votsis, Reija Ruuhela

Finnish Meteorological Institute (FMI)

EMS Annual Meeting

Dublin 4 – 8 September 2017

Context: EU-MACS European Market for Climate Services

• Towards better matching of supply options and user needs

• Exploring engagement protocols with stakeholders from finance, tourism and urban planning

• Close cooperation with MARCO sister project

Tell us if:

• you are interested in joining the sectoral explorations

• your project could cooperate with EU-MACS

Issues:

• How do CS business models look like by 2020?

• What are key innovations for better uptake?

• What legislation would help?

• User orientation at the core of quality control

Contact: adriaan.perrels@fmi.fiweb-site: http://eu-macs.eu/#

….

Understanding of quality:• C3S – EQC approach

• Metrics on data and data-set properties

• Tractability of data origin and post-processing

• Towards standardized meta data formats

• General definition:• in essence it means adequate fulfilment of user’s requirements

• ‘fit for purpose’?• Performance uncertainty

• Product fit uncertainty

• These uncertainties reduce uptake of CS

• Individual and joint learning can ameliorate this

Quality - concepts

…. Uncertainty of eventual benefits deters use

….

• Closed and open approach in quality assurance (QA)• Closed: internal total control oriented process

• Open: seek assurance by cooperation with user

exploit learning options

• Suitability depends on user profile and product type

• Growth will be especially in open approaches associated with downstream uptake of CS

• Most value added of CS in downstream use• Open QA approaches and exploitation of learning

need to be supported by user data sensitized metrics• … which mostly still need to be developed

Quality - development

….

• User opinions in C3S SECTEUR, e.g.• Notable shares (50% ~ 60%) of dissatisfaction with available

spatial and temporal resolutions

• Usefulness of CS appears to depend very significantly on easy identification and connectivity with (key) variables in the user’s own domain, as well as ease of use

• Survey results in EU-MACS• Evaluation of fitness for purpose jointly with users is still

uncommon (~20% of those that evaluate this)

• Connectivity to user’s issues and linking options to user’s data very important

• extent quality indicators towards user variables

• uncertainty/reliability trade-offs also at the user side; be cautious with imposing purely climate data inspired guidelines

Quality – user expectations

….Resolution impact comparison:

climate & population data

Comparing effects of downscaling datasets for temperature and precipitation

UEA ClimGen Pattern Scaling output annual averages 2031-2040 (ToPDAd D2.1):

From 0.5 degree grid to 25 km grid for selected clusters of grid cells

Spatially weighted averaging applied

….Resolution impact comparison:

climate & population

Source: EU-MACS D1.2 – Annex 6

Whole sample

original downscaled |%Δ|

Precipitation min 24.7 25.6 3.6

(mm) max 178.3 178.1 0.1

median 58.0 58.1 0.2

mean 61.5 61.3 0.3

std. dev. 20.1 19.9 1.0

Temperature min -1.1 -0.5 54.5

(oC) max 19.1 19.1 0.0

median 9.3 9.7 4.3

mean 9.4 10.0 6.4

std. dev. 4.1 4.0 2.4

Also for subareas deviations stay mostly modest (|Δ|<10%)For localized areas especially minimum temperatures may have larger deviations

….Resolution impact comparison:

climate & population

Source: EU-MACS D1.2 – Annex 6

Population:• from EUROSTAT 1km grid to 25 km grid• from EUROSTAT NUTS3 to 25 km grid

1km >> 25 km

orig. upscaled |%Δ|

min 1 0 100

max 52898 5206 90

median 26 30 15

mean 209 81 61

std. dev. 905 193 79

NUTS3 >> 25km

orig. upscaled |%Δ|

min 0 7 0

max 4018 2244 44

median 111 117 5

mean 274 183 33

std. dev. 515 243 53

For vulnerability related analysis this upscaling can already be detrimentalFurthermore, non-grid based divisions (e.g. NUTS3) can offer sometimes better departure for seeking compromise resolution

….Resolution impact comparison:

climate & population

Source: EU-MACS D1.2 – Annex 6

Figure A-1 standardized comparison of deviations at nuts-3 level

0 1 000500 Km

Population density

% dif. for sampled NUTS-3 regions

< -100%

-100% - -50%

-50% - -10%

-10% - 0

0 - 10%

10% - 50%

50% - 91%

This may be the level at which some actors work for small collections of regions

…. Conclusions

Quality assurance (QA) is not only a matter of control, but just as much of

communication

Quality uncertainty of CS concerns both the performance uncertainty (party

covered with traditional QA) and the product fit uncertainty (addressed by

broad scoped QA)

The more a CS involves tailoring, non-climate data, advice and training, or the

more the user lacks expertise in climate and/or risk analysis the more QA should

go beyond the statistical properties and origins of the climate data, and

consider also linking feasibilities with non-climate data and the service delivery

process

Broad scoped QA can include, where appropriate, the review of linking

feasibility with non-climate data; this requires development of new metrics

Social learning both among CS users and CS providers should be promoted in a

systematic way as a part of innovation oriented quality management

EU MACS Consortium

Participant Type of organisation Country

FMI (coordinator)Met-services; climate & adaptationresearch; Finland

HZG-GERICSClimate services & research

Germany

CNR-IRSAHydrological research & consultancy, incl. adaptation

Italy

AcclimatiseClimate services provider

United Kingdom

CMCCClimate research and services

Italy

U_TUMMarket start-up support for innovations

Germany

U_TwenteResearch in innovation mechanisms and policy

Netherlands

JRTechnical & social innovations for climate change issues

Austria

ENoLLPromotion and support of Living Lab applications

Belgium

Thank youhttp://eu-macs.eu/#