XÖV and Dat/ML

XML- Basis for standardisation of statistical processes in Germany

Presentation during the Meeting of the Cord Task ForceEurostat

June, 5, 2007 Luxembourg

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 2

XML ?

Strategic Technology► Government Support► Tool Support► Public acceptance► flexible

Needs in Statistical Data Processing► Generic processes (“one application for many surveys“)► Independency from data structure ( „Data of Surveys change very often”)

Use of XML for Standardisation and Improvement of Processes

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 3

XML Status

Tools► Internet data collection (IDEV)► Business data collection (.Core)► Data editing (Pl – Editor)► Tabulation and Analysis (Statspez, SAS)► Dissemination (Genesis)

Standards► Dat/ML (Data markup language)► Tab/ML (Table markup Language)► XMeld, XJustiz … (XÖV)

Project: E-Statistics - Integration

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 4

Statistical Dataflow

Produce transmit receive process analyse

Basic interfaces in statistical data Processing

Enterprises Statistics

E-Dataflow

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 5

XMLDatabase

XML

Collection mgmt.

IDEVSoftware

Internet datacollection

CORE(B2B)

IDEV(Web-forms)

DatML

DatML

businessAppsXBRL

DataEditing

PL-Editor

Tabulation

Statspez

Disse-mination

Genesis

TabMLDatML

Editing database

Tabulationdatabase

Collectiondatabase

DatMLDatML

eStatistics Dataflow

Distributiondatabase

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 6

Services

STATSPEZ-Distributed

DB Registry-DB Editing DB Input-DB Survey DB

SW-Modules

CORE.connect

CORE.reporter

KonVert-System

Java-classesfor Data Editing

Java Framework

XML-Interfaces

DatML/SDF

DatML/EDT

DatML/ASK

DatML/MAP

DatML/RAW

DatML/RES

XÖV

.Base Tools

Data-Edit-Editor

Forms-Editor

SDF-Editor

. . .

eStatistics Components

eStatistics.Integration

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 7

Businesses/Offices

Business DataRepository

SurveyDB[SDF]

Data Supply Specific.[DatML/SDF]

IDE

V /

.C

OR

E

Test Report[DatML/RES]

Raw Data[DatML/RAW]

TransferTicket[http-resp.]

Rec

eivi

ng C

ontr

ols

Registration RegistryDB

Archiving

OutputDB [SET]

Statistical Evaluation &Publication [TabML]

Input Registry

Data Editing[EDT]

InputDB [RAW]

Web forms Paper forms

Data mediaFile upload

Web

Businessapplications

. . . .

?

eDataFlow and Interfaces

Questionnaire Design [ASK]

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 8

PL-EditorSD

F-Editor

Form

s E

dito

rSTATSPEZ

. . .

API

.BASEMetadatabase

.BASEServer

Tools: .BASE-StructureTools: .BASE-Structure

.Base Kernel

.Base API

.Base applications

.Base system

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 9

Distributed Data baseDistributed Data base

Tools: .BASE-NetworkTools: .BASE-Network

.BASE-Distributor

Admin-database

(levels,locality,

user,restrictions)

Admin-Server

Reference-Database

Changes.BASE-Network

.BASE-System

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 10

XML interfaces: Statistics

DatML (Data Markup Language) DatML/RAW : raw data messages DatML/RES : raw data message response format DatML/SDF : survey definition format (for enabling business apps to

automatically and generically create raw data messages) DatML/EDT : data editing and validation DatML/SET : productive data sets and archiving DatML/ASK : for describing web forms DatML/SSP : shared and core components

TabML (Table Markup Language) TabML/LAYOUT : presentation and dissemination purposes TabML/MATRIX : n-dimensional tables and time series, fully metadata-

equipped (non-disclosure, analysis, merging, subsets…)

Data Structure and Web Service

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 11

2. DatML document types …

describe data and metadata in a standardized way according to the task within the statistical data processing

describes a class of XML-based document types for statistical data

facilitates information exchange between all steps of the statistical data processing

1. DatML …

components describe technical, subject-matter and organizational metadata, raw data messages, data edits, data sets, statistical variables, ...

Data Markup Language

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 12

DatML Document Types

DatML/RAW (Raw Data) … for statistical raw data

DatML/RES (Response) … for test protocols

DatML/SDF (Survey Definition Format) … for survey definitions

DatML/SSP (Statistical Specification) … for basic DatML components

DatML/EDT (Data Editing) … for specification of test procedures

DatML/MAP (Data Mapping) … for data mappings

1. Public DatML Formats

2. Internal DatML Formats

… for questionaires DatML/ASK (Questionaires)

… for statistical data sets DatML/SET (Data Set)

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 13

XML interfaces: Government

Priority XKfz - Car registration Population Register XMeld - Population Register

other XPlanung – Government Planning XJustiz - Justice XFinanzen -Tax

Osci communication Infrastructure

XÖV Framework and Project Infrastructure

XÖV Standardisation Part of „Germany Online“ (DOL)

XMl for government Administration

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 14

.base.system

Admin-database

(levels,locality,

user,restrictions)

Admin-Server

Reference-Database

Changes

.base.system

Statistics Intranet

ExternalSurvey

DB

Internet link

Services: Survey Database

Survey DB

Internet

Cord task force 5. 6. 2007 XÖV and Dat/Ml Slide 15

Thank you very much for

your attention

Fritz PfrommerStatistisches Bundesamt, D-65180 WiesbadenPhone: +49 (0)611 / 75-2707E-Mail: Fritz.Pfrommer@destatis.de