ESCWA SDMX Workshop Session: Role in the Statistical Lifecycle and Relationship with DDI (Data...

ESCWA SDMX Workshop

Session: Role in the Statistical Lifecycle and Relationship with DDI

(Data Documentation Initiative)

© Metadata Technology

General Statistical Business Process Model and SDMX


General Statistical Process Business Model and SDMX

DDI DDISDMX


General Statistical Process Model and SDMX

DDI DDISDMXSDMX


DDI 3.0 and the Survey Life Cycle

• A survey is not a static process: It dynamically evolved across time and involves many agencies/individuals

• DDI 2.x is about archiving, DDI 3.0 across the entire “life cycle”• 3.0 focus on metadata reuse (minimizes redundancies/discrepancies,

support comparison)• Also supports multilingual, grouping, geography, and others• 3.0 is extensible


DDI 3.0 Modules

• Conceptual Components (concepts, universes)• Data Collection (survey instruments and collection processing)• Logical Products (variables, categories, code lists)• Physical data product (descriptions of file structures)• Physical Instance (instances of data files)• Archiving (information about holding, storage, and organizations)• Comparative (mapping schemes)• Grouping (for comparison, and longitudinal studies, panels, and

series)• Instance (the wrapper)• DDI Profile (describes which DDI 3 elements are used)• Study Unit (describes a single study)


Technical Specifications - Maintainable Schemes (that’s with an ‘e’ not an ‘a’)

• Category Scheme• Code Scheme• Concept Scheme• Control Construct Scheme• GeographicStructureScheme• GeographicLocationScheme• InterviewerInstructionScheme• Question Scheme• NCubeScheme• Organization Scheme• Physical Structure Scheme• Record Layout Scheme• Universe Scheme• Variable Scheme

Packages of reusable metadata maintained by a single agency


DDI “Mind Map”DDI “Mind Map”


DDI – “Mind Map”


Technical Specifications – XML Schemas

• archive• comparative• conceptualcomponent• datacollection• dataset• dcelements• DDIprofile• ddi-xhtml11• ddi-xhtml11-model-1• ddi-xhtml11-modules-1• group• inline_ncube_recordlayout

• instance• logicalproduct• ncube_recordlayout• physicaldataproduct• physicalinstance• proprietary_record_layout (beta)• reusable• simpledc20021212• studyunit• tabular_ncube_recordlayout• xml• set of xml schemas to support

xhtml


Use within data collection

<DDI 3.0>ConceptsUniverseMethodsPurposePeople/Orgs

<DDI 3.0>QuestionsInstrument

<DDI 3.0>Data CollectionData Processing

<DDI 3.0>Funding Revisions

SubmittedProposal

$€ £

Presentations

Archive/RepositoryPublication

+++

+

+

<DDI 3.0>VariablesPhysical Stores

PrincipalInvestigator

Collaborators

Research Staff

Data


DDI and Generic Process Example

Survey/Register

Raw Data SetRaw Data Set

Anonymization, cleaning, Anonymization, cleaning, recoding, etc.recoding, etc.

Micro-Data Set/Micro-Data Set/Public Use FilesPublic Use Files

Tabulation, processing,

Tabulation, processing,

case selection, etc.

case selection, etc.

Aggregation,

Aggregation,

harmonizatio

n

harmonizatio

n

Aggregation, Aggregation, harmonizationharmonization

Aggregate Data SetAggregate Data Set(Lower level)(Lower level)

Aggregate Data SetAggregate Data Set(Higher Level)(Higher Level)

DDIDDI

SDMXSDMX

IndicatorsIndicators


Concept Scheme

Code List, HCL,

SDMX RegistrySDMX DSD and data set

SDMX MSD and metadata set

SDMX

General Statistical Process Model and SDMX

SDMX

SDMX Process Model


Process - Schematic

ProcessProcess

ComputationComputationTransitionTransitionProcess StepProcess Step

Process Artefact

Process Artefact

inputsoutputs

target


Process - UML

1

The process step can reference any identifiable object as input or output.

MaintainableArtefact

Process

Computation

localId : StringsoftwarePackage : StringsoftwareLanguagesoftwareVersion : String

Transition

localId : String

InternationalString

+condition

1

ProcessStep

0..*+step

0..*

0..1

+computation

0..1

0..*+child 0..*+parent

1

+target

10..*+transition

0..*+source

ProcessArtefact

localID : String

0..*

+output

0..* 0..*

+input

0..*

IdentifiableArtefact

11

+artefact

AnnotableArtefact

InternationalString1..*

+description

1..*

ESCWA SDMX Workshop Session: Role in the Statistical Lifecycle and Relationship with DDI (Data...

Documents

Transcript of ESCWA SDMX Workshop Session: Role in the Statistical Lifecycle and Relationship with DDI (Data...