1

SDMX The basis for renovating the

ESS Metadata Systems August Götzfried

Head of Unit

EUROSTAT

2

The ESS Vision for the next decade

The Vision for the European Statistical System (ESS) for the next decade –

the basic ideas:

– The ESS vision wants to overcome statistical stove pipes and move

towards more integrated statistical production processes;

– It wants to better integrate the ESS in terms of

IT infrastructure, software, data quality, metadata, methodology etc.

(both in terms of horizontal and vertical integration);

– It aims to stimulate the better use of administrative data in the

statistical data production processes;

– It also calls for better integrated statistical cross-cutting legislation

in covering larger statistical domains.

This vision also impacts the ESS Metadata Standards and the ESS

Metadata Systems.

3

The starting point

Until 2009

The Eurostat and ESS metadata systems were not fully SDMX compliant, i.e.

• Reference (explanatory) metadata structures were used containing statistical concepts not sufficiently harmonised or not clearly explained;

• Different reference metadata structures were used in different statistical domains (e.g. different quality report structures);

In 2009

The SDMX Content-oriented Guidelines were published. These SDMX Content-oriented Guidelines were the basis for upgrading the Eurostat and ESS metadata systems and making them SDMX compliant.

4

Using the SDMX Content-oriented Guidelines

The SDMX Content-oriented Guidelines (SDMX COG) comprise the

following 4 annexes:

– Annex 1: Cross-domain Concepts;

– Annex 2: Cross-domain Code Lists;

– Annex 3: Statistical Subject-Matter Domains;

– Annex 4: Metadata Common Vocabulary;

Two SDMX compliant Metadata Structure Definitions were defined

for reference (explanatory) metadata using a sub-set of the 66

SDMX Cross-domain Concepts (and sub-concepts) listed in annex 1

of the SDMX COG.

5

The Euro SDMX Metadata Structure

The Euro SDMX Metadata Structure (ESMS)

– is the main report structure for reference (explanatory)

metadata;

– is fully SDMX compliant;

– uses 21 concepts (plus additional sub-concepts) taken from

the SDMX cross-domain concepts;

– is a metadata structure definition which is used across all

statistical domains;

– is user oriented (the ESMS files are normally published on

web-pages);

– incorporates the main criteria measuring data quality;

– when fully implemented, will lead to more than 2000

Eurostat and national ESMS files produced and

disseminated.

6

The Euro SDMX Metadata Structure

Example: ESMS – Harmonised indices of consumer prices – Bulgaria

7

The ESS Standard Quality Report Structure

The ESS Standard Quality Report Structure (ESQRS)

– is the main report structure for reference metadata related

to data quality;

– is SDMX compliant (however: some limited quality related

concepts should be included in the SDMX COG in the

future, e.g. imputation rate);

– uses 8 main concepts taken from the SDMX cross-domain

concepts plus more detailed sub-concepts measuring data

quality;

– is a metadata structure definition which is used across all

statistical domains;

– is data producer oriented and therefore sometimes not

publicly released;

– will harmonise quality reporting within the European

Statistical System.

8

The ESS Standard Quality Report Structure

9

Using the ESMS and the ESQRS within the European

Statistical System

The implementation of the ESMS and the ESQRS:

– The two Metadata Structure Definitions are stored in the

Euro SDMX Registry (= main SDMX Registry used within

the ESS);

– The ESMS is fully used at Eurostat and more and more

used when national reference metadata are produced;

– The ESQRS gets now more and more used at Eurostat and

when national quality reporting is done.

Attention

The existing Eurostat and national reference metadata and

quality reports need to be converted to these ESS

standards. This needs to be done for more than 50 statistical

domains.

10

The ESS Metadata Handler as facilitator for implementing the

SDMX compliant metadata standards

The ESS Metadata Handler (ESS MH):

– comprises the IT applications managing the production,

exchange and dissemination of ESS metadata;

– accommodates the ESMS and the ESQRS structures for

production, exchange and dissemination of the national and

Eurostat metadata files;

– allows the on-line production of the national ESMS and

ESQRS files and their exchange within the ESS;

– stimulates the use of the SDMX based ESS metadata

standards within the ESS and the further integration of the

ESS metadata systems.

11

The ESS Metadata Handler as facilitator for implementing the

SDMX compliant metadata standards

Eurostat Website

(ESMS, ESQRS…)

ESS Metadata Handler

National + Eurostat

Metadata Files

(ESMS,

ESQRS…)

ESS Metadata Handler

National Metadata

Files

(ESMS,

ESQRS…)

National Statistical

Institute

PROCESSING PROCESSING and ANALYSIS DISSEMINATION

eDamis

National

dissemination

Eurostat

Metadata

Files

National

Metadata

Files

12

What is next ?

Our further work will concentrate on:

– The full and broad implementation of the ESMS and the

ESQRS in the European Statistical System;

– The further deployment and opening of the ESS Metadata

Handler;

– The definition of additional Metadata Structure Definitions

based on the SDMX COG such as the Euro Process

Metadata Structure (EPMS) using the GSBPM;

– The improvement of the international exchange of the ESS

metadata with other international organisations in order to

reduce the burden on EU Member states;

– The contributions to the updating of the SDMX Content-

oriented Guidelines.