Status on the Mapping of Metadata Standards

ISO/IEC 11179, SDMX, and Others

Arofan Gregory, Executive Manager

Open Data Foundation, Inc.

METIS Meeting, Luxembourg,April 9-11, 2008

Overview

• Introduction

• The Mapping Challenge

• Status of the Mappings

• On-Going Work

• Summary

Introduction

• METIS 2006: Several presentations were given addressing some standards mappings– Many side-discussions among attendees

• Standards of note included:– ISO/IEC 11179– Statistical Data and Metadata Exchange (SDMX)– Data Documentation Initiative (DDI)– ISO 19115 (geographic metadata)– Dublin Core– Extensible Business Reporting Language (XBRL)

General Update

• There has been informal work to describe the mappings between these standards among individuals from many organizations

• Progress is generally good, however…– Very labor-intensive– Not mission-critical for most organizations

Initial Approach

• After METIS 2006, a general approach for moving forward was identified:– Map all standards against ISO/IEC 11179, so it acts

as a pivot

• This approach has been largely successful• Not always sufficient for all purposes

– Some standards address issues which ISO/IEC 11179 does not cover

– Pivot approach has been supplemented with some direct mappings

The Mapping Challenge

• There are many ways to map different standards

• As with many technology applications, the criteria for success lies in the answer to the question:– “What am I trying to achieve?”

• Much work has been done in defining use cases which can serve as the basis for producing mappings between standards

High-Level Vision – Standards MappingsFederated Registries (Based on SDMX, ebXML, web services)

Aggregated Data/Metadata

(SDMX)

XBRLBusiness Reports

DDI Microdata

Sets

ISO 19115Geographies

Dublin CoreCitations

Used in

registered

References to source data

Standard classifications

Organizedusing

ISO 11179

Semantic definitions

METS Packaging

Use Cases

• The preceding picture embodies several use cases:– Register microdata sets and their metadata (XBRL,

DDI, etc.)– Query for microdata/metadata– Capture ISO/IEC 11179 metadata for

SDMX/DDI/XBRL objects (concepts, codes, etc.)– Query for ISO/IEC 11179 metadata on an SDMX

object– Link from SDMX aggregates to DDI/XBRL sources– Etc.

What Does This Look Like?

Demo of DDI – SDMX Integration

Starting Point: Mappings

• Five mappings have been identified as a good starting point:– SDMX – ISO/IEC 11179 – DDI – ISO/IEC 11179– XBRL – ISO/IEC 11179– SDMX – DDI– XBRL - SDMX

SDMX – ISO/IEC 11179

• Many false starts – this is a difficult one!• Emphasis is on integration of SDMX

mechanisms and ISO/IEC 11179 metadata– SDMX does not cover the same detail as ISO/IEC

11179, but provides points of contact (eg, code lists and other representations, classifications, concepts)

• This mapping is important to the ISO acceptance of SDMX version 2.0 Technical Specifications– A detailed draft is currently being developed

SDMX and ISO/IEC 11179

Dimension

SDMX Concept

Value Meaning

Value

Domain

Permissible Value

SDMX Structure Definition

Attribute

Measure

SDMX Structure Components

permissible content

takes semantic from a concept

enumerated or non enumerated values

enumerated domains have value meaning

ISO/IEC 11179 Data ElementConcept

ISO/IEC 11179 Concept

link to

ISO/IEC 11179 Data Element

link to

permissible content

SDMX Representation

SDMX Code List

Prototype

• There is an on-going prototype describing the ISO/IEC 11179 model as an SDMX Reference Metadata Structure

• This will greatly simplify the integration of SDMX and ISO/IEC 11179 systems

• The SDMX Registry could serve as a basic ISO/IEC 11179 Registry implementation

DDI – ISO/IEC 11179

• DDI has always been oriented toward the ISO/IEC 11179 model– Concepts, codes/representations, categories, etc. are

similar

• The version 3.0 release of DDI is in final voting stages– It can act as an XML-syntax implementation of

ISO/IEC 11179 for concept banks– This is not mandatory, but it is possible– A detailed draft is being developed

ISO/IEC 11179 class

Facet(from DDI_Base)

facetType : FacetTypefacetValue : StringmeasureUnit : StringisIncluded : Boolean

NonCodedValueDomain(from DDI_Base)

0..*

1

0..*

1

Object Class

Characteristic/Property

Value Meaning

Enumerated Conceptual Domain

Permissable Value

Value Domain

Enumerated Value Domain

Non Enumerated Value Domain

Data Element

Data Element Concept

Concept

description only?

CodedValueDomain(from DDI_Base)

CodeScheme(from Code_Scheme)

0..*

1

0..*

1

CodeItem(from Code_Scheme)

1

1..*

1

1..*

/items

ValueDomain(from DDI_Base)

CategoryItem(from Category_Scheme)

0..*

1

0..*

+codeFor1

VariableRepresentation(from Variable_Scheme)

0..10..1

UniverseScheme

CategoryScheme(from Category_Scheme)

1

1..*

1

1..*

/items

Variable(from Variable_Scheme)

0..10..1

ConceptualVariableScheme

Characteristic

Universe

1..*1..*

/items

DataElementConcept

0..*0..1 0..*

+enumeration

0..1

0..*

1

0..*

+conceptualSemantic

1

1..*1..*

1

+characteristic

1

0..1+universe0..1

ConceptItem

ConceptScheme

1

1..*

1

1..*

items

DDI 3.0 and ISO/IEC 11179

SDMX - DDI

• DDI and SDMX have some degree of overlap– Both describe multi-dimensional data cubes– Both have code lists and concepts, etc.– Both provide XML formats for the data and metadata

• DDI is oriented toward micro-data, SDMX is oriented toward aggregates

• The correspondences based on ISO/IEC 11179 cover much of the mapping (codes, concepts, etc.)

• Higher-level use of concepts and codes/representations in data and metadata structures requires a dedicated mapping

• A detailed draft is being prepared

Prototyping

• An on-going prototype is being developed to derive SDMX data structure definitions from DDI metadata

• This will allow for both aggregates and micro-data documented in DDI to be automatically expressed as SDMX– Micro-data has limited dimensionality

• SDMX tools for data visualization, etc., could directly leverage DDI-documented data sets based on a standard transformation

• Still in early stages

SDMX – XBRL

• SDMX and XBRL have different applications– XBRL is for individual business reports– SDMX is for aggregates

• Both cover multi-dimensional data• SDMX Sponsors and XBRL International have

agreed to define a standard mapping– Work is progressing well, and further meetings are

expected in 2008– To follow progress, see www.sdmx.org

XBRL – ISO/IEC 11179

• Theoretically, an XBRL taxonomy could be described according to the ISO/IEC 11179 model, but…

• No progress has been made in this area

Other Mappings/Notes

• Dublin Core and ISO 19115 can be described in SDMX as a reference metadata structure

• These are both integrated into the DDI 3.0 design– Native Dublin Core XML is supported– Links to ISO 19115-compliant shape files are

supported

• More work is needed here– But not highest priority

On-Going Work

• 4 of 5 mappings should emerge in 2008 as drafts for review:– ISO/IEC 11179 – SDMX– ISO/IEC 11179 – DDI– SDMX – DDI– SDMX – XBRL

• Interesting protoypes/examples are being developed– This includes some prototype tools for performing

cross-walks

Support and Interest

• Generally, support and interest has been very good– Many individuals/organizations have participated

informally: METIS, SDMX Sponsors, XBRL International, ISO committees (Open Forum for Metadata Registries), Open Data Foundation, DDI Alliance, NSOs

– Standards organizations understand the need to provide alignments and mappings, to support their users

Summary

• Overall progress has been good

• METIS can help this work– By providing critical feedback on published

drafts – By providing visibility and credibility to the

results through inclusion in the metadata framework