The status of metadata standards and ModernStats

models in SURS

ModernStats World Workshop 2018

SURS – Statistical Office of the Republic of Slovenia Julija Kutin

April, 2018

Statistical Standards and ModernStats models in SURS

1. Statistical Standards: a) SDMX b) ISO/IEC 11179 c) DDI

2. ModernStats models: a) GSBPM b) CSPA

3. Challenges and next steps

1.a. SDMX

1. SDMX and statistical data 2. SDMX and metadata 3. General problems with SDMX:

a) Different internal

rules for dissemination

b) Only for Eurostat c) Not standard tables

for indicators d) Different MSDs

1.b. ISO/IEC 11179 – SDM module

Variables and questions in Excel

Usefulness of SDM module 1. E-questionnaires

a) Questions b) Sub-questions c) Tables – challenge

2. Response validation a) Mandatory questions b) Questions with specific values or rank of values c) Responses in a specific format

3. Build the database 4. Data editing

Local variables

1.c. DDI

1. Mappings ISO/IEC 11179 to DDI 2. For generating questionnaires

2.a. GSBPM

REGISTRY

Analysis of needs and requesrt

Dat

a co

llect

ion

The

sele

ctio

n of

ob

serv

atio

n un

its

Dat

a pr

oces

sing

Dat

a an

alys

is

Use

rs

Survey design and preparation

Dat

a m

anag

emen

t

Big Data

Admin. data

SPRS Address list

Row data Maro

data

Final micro data ARCHIVE

D

ata

diss

emin

atio

n

Met

adat

a di

ss.

Admin. data. Admin. data

Metadata Classifications

Data files Data files Data files Data files

SRKG

SRDAP

PAPI

CATI

eSTAT

Adm.

Editing

DISSEM. DB

ESS MH

eDamis (Eurostat)

SDMX HUB

Si_Stat

Resear-ches

Other IO

Publi-cations

SDMX HUB

Si_Stat

CAPI

SOP

Integration

Imputation

Weighting

Agregation

Doc

umen

tatio

n

Qua

lity

asse

ssm

ent

Eval

uatio

n

Tabulation

Disclosure

SCRP

Manage and document statistical production a) Documentary system for statistical surveys b) Descriptions of processes of the statistical survey c) Guidelines for Quality Assurance (QA)

Usefulness of GSBPM

Documentation of statistical survey 1. Before:

a) Missed documentation b) Not structured documents c) In different places

2. A new system:

a) One place for all surveys b) Documents are structured (GSBPM) c) The information can not be lost d) Unauthorised access is not possible e) Comparability between surveys f) Preparation of documentation is planned part of the survey implementation

System STATDOK is built at 3 levels

I. level

Basic information of the survey

Standardized Excel template

II. level

Description of the phase, sub-processes

7 standardized Word templates

III. level

Implementing documents

Standardized templates and not standardized documents

I. level One sheet for each phase

STATDOK example

II. level 7 templates - one for each sub-process

STATDOK example

1. Methodologist

2. Head of the organizational unit

3. Process / sub-process administrator and leadership

How did we do it? How should it be done? How is it done in other surveys? Do we have data for 19XX? Are they comparable? Where can we find them?

Analysis of documentation Education of co-workers Optimisations

Standardization Similarity Personal data Critical points

…

STATDOK is usefull

2.b. CSPA ESTAT questionnaire generation

1. ESTAT web data collection portal 2. Metadata driven questionnaire generation 3. ISO/IEC 11179 metadata model

a) Concepts, codes/representations, categories, etc., are similar to

DDI concepts.

Generate questionnaire

Manage survey data collection

SDM Meta-data

ISO/IEC 11179

SDM Questionna-ire generator

SDM Questionna-ire creator

Blaise

PDF

XSD

ESTAT ENO reuse context

ENO Questionna-ire generator

Pogues Questionna-ire designer

DDI DDI Xforms

Blaise MD

Orbeon MD

Blaise manual editing Design

questionnaire Web Questio-nnaire Blaise

Web Questio-nnaire

Orbeon

Excel form

Colle-cted data

Oracle

WS

WS

WS

DDI

4. Challenges and next steps 1. Go on with the SDMX standard 2. Meet with DDI deeper 3. Review of internal questionnaire design standard

(complex questionnaires generation) 4. Rebuild SDM module 5. Use ENO DDI compliant SDM as common metadata

repository for questionnaire design & generation (Blaise, Xform, Pdf, etc.)

6. GSBPM will be the basis of our further development 7. The use of CSPA standard should be considered for the specification and development of services

Thank you for your attention!