Step by Step Tutorial PX-File_to_SDMX1

7/28/2019 Step by Step Tutorial PX-File_to_SDMX1

1/23

ESSNET on SDMX II

WP2: PC-Axis SDMX Integration

Tutorial of the process using PX-file to

SDMX for CensusHubVersion 0.1

2012-05-21


2/23

Type of Document Project deliverable WP 2; 2d Presentation and information for

other countries

Reference:

Issue: Revision: Status: Draft 2

Created by: Lars Nordbck Date: 2012-05-26

Updated by:

Approved by:

Document Change Record

Issue/Revision

Date Change

Analysis and Design of the Test Client v 0.1 Date Updated: 6/7/2013 2:34:14a6/p6

Page ii of 23Page ii

Prepared by: TC, IB Reviewed by:


3/23

[Title]

Table of contents Page

1 Introduction........................................................................................................................................ 4

1.1 Purpose............................................................................................................................................ 4

1.2 Reference Documents and Standards..............................................................................................4

2 Description of process for CensusHub reporting from PC-Axis files...........................................6

2.1 Workflow........................................................................................................................................... 6

2.2 Interaction with SDMX_RI modules..................................................................................................7

3 Creation of the example.................................................................................................................... 8

4 Preparing the input for the Census HyperCube 6........................................................................... 9

5 Preparing to handle the output of HyperCube 6...........................................................................13

6 Using Census HyperCube 6 in the PC-Axis file based process for Eurostat CensusHub...... ..15

List of figures Page

Figure 2-1: Process for CensusHub reporting from PC-Axis files...................................................6

List of tables Page

Table 1-1: Terms and Abbreviations...................................................................................................5

Analysis and Design of the Test ClientVersion 0.1 Date Updated: 6/7/2013 2:34:14a6/p6

Page 3 of 23Page iii

Prepared by: IB, TC, SL Reviewed by: SLI


4/23

[Title]

1 Introduction

1.1 Purpose

In this paragraph will be described the purpose of the document and how the document is structured

The purpose with this document is to give the participants in the Essnet for SDMX Workshop in Rome

25-27 a documentation of the processes involved when using PC-Axis files as the base for the use of

Eurostat Reference Architecture in the CensusHub project.

1.2 Reference Documents and Standards

Insert all references to documents or convention or standard used in the document. This part can be

divided in:

Standards and Conventions

Documents or standard named in the document

Statistics Sweden documents

PC-Axis file format document:http://www.scb.se/Pages/List____314011.aspx

Attributes in the PX-file; New keywords for attributes on observation values, Petros Likidis

PX-Web Systems Diagram, Mikael Nordberg

PX-Web powerpoint presentation, Mikael Nordberg

Eurostat RI documents

On CIRCA

SDMX - training material

SDMX Self-learning tutorials

Mapping Assistant

Installation Guide of the Mapping Assistant.doc

Mapping Assistant Design and Implementation Guide for Programmers.doc

Mapping Assistant User Manual and Tutorial.doc

Test Client

No instructions found

NSI Web Service

Installing_NSI_Web_Service_(.NET).doc

Analysis and Design of the Test ClientVersion 0.1 Date Updated: 6/7/2013 2:34:14a6/p6

Page 4 of 23Page 4

Prepared by: IB, TC, SL Reviewed by: SLI
http://www.scb.se/Pages/List____314011.aspxhttp://www.scb.se/Pages/List____314011.aspxhttp://circa.europa.eu/Members/irc/dsis/sdmxdevelopment/library?l=/sdmx_training_2010/sdmx_training_sessions&vm=detailed&sb=Titlehttp://circa.europa.eu/Members/irc/dsis/sdmxdevelopment/library?l=/sdmx_training_2010/self-learning_tutorials&vm=detailed&sb=Titlehttp://www.scb.se/Pages/List____314011.aspxhttp://circa.europa.eu/Members/irc/dsis/sdmxdevelopment/library?l=/sdmx_training_2010/sdmx_training_sessions&vm=detailed&sb=Titlehttp://circa.europa.eu/Members/irc/dsis/sdmxdevelopment/library?l=/sdmx_training_2010/self-learning_tutorials&vm=detailed&sb=Title


5/23

Web Client

Analysis of Web application for exposing browsing_dissemination environment of an NSI v0.7.doc

Installation Guide of the Web Application for exposing_browsing dissemination env of NSI (.NET)

v0.8.doc

User Manual of Web application for exposing browsing_dissemination environment of an NSI v0.4.doc

Terms and abbreviations

Terms or abbreviations used in the document

Acronym Definition

SCB Statistics Sweden

RI Eurostat Reference Infrastructure

CIRCA Eurostat website for access to documents and softwareSSD Statistics Sweden Statistical Database on the Website

PX-file PC-Axis file

PX-Edit Software for creating PC-Axis file from different input file formats

Nordic Data

Model

Data model used by the countries using PC-Axis SQL based software

PC-Axis file Open file format based on keywords for metadata and data

Table 1-1: Terms and Abbreviations

Definitions:

Definitions used in the document

5


6/23

2 Description of process for CensusHub reporting from PC-Axis

filesIn this chapter will be described the example providing a description of the whole workflow. This

chapter should be a sort of short User Manual in which is described how to use the example.

2.1 Workflow

In this paragraph should be described the workflow of the example. In a picture should be shown the

modules used of the SMDX-RI and the new modules created for the purpose of the example.

Statistics from a material made of the PC-Axis file with the program PX-Edit version 3.

1. Statistical production produces statistics including attributes on cell level to be included in the PC-

Axis file database.

2. Subsequently, the statistical product load statistics in PX-Edit and enter metadata including

Metadata for Attributes on data cell level.

3. Check of the entered data in the database using PC-Axis SQL or the SSD Web interface.

4. Based on Eurostats description of the statistics, Data Structure Definition (DSD), which is stored in

the local database Mapping Store, a mapping of the SCB material, can be made against what is in the

DSD for the codes, etc. using the program Mapping Assistant. The resulting mapping is then stored in

the Mapping Store.

5. To make sure the material is correct you can use the Test Client that creates an SDMX file of the

statistics.

6. Publishing is executed based on normal copying to the external PC-Axis directory

This routine is not using RSS for notifying because Census data is only posted once.

7a and 7b. Eurostat software downloads then the statistics that dynamically is transformed into SDMX

format.

In this case the Statistical database is outside the inner Firewall and security of the database has to bemonitored.

Figure 2-1: Process for CensusHub reporting from PC-Axis files

6


7/23

Explanation of the staining of components in the image above:Yellow: Applications and databases developed by Eurostat called Eurostat Reference Infrastructure,

Open Source software.

Green: Program developed by Statistics Finland, freeware.

Gray: Existing standard applications and databases for the dissemination of statistics by Statistics

Sweden.

Pink: Programs that are used by the product to produce the file for loading the statistics in the

Statistical Database.

Brown Pink: Product production database.

2.2 Interaction with SDMX_RI modules

In this paragraph must be indicated the modules that interact with the SDMX-RI

No software module is interacting with the SDMX-RI. The interface is the PC-Axis file.

7


8/23

3 Creation of the example

In this chapter will be described the step done in order to realize the example. The example can useone or more modules belonging to the SDMX-RI, some of this modules can be modified and new code

can be added in order to realize the example.

The example is the CensusHub Hypercube number 6.

8


9/23

4 Preparing the input for the Census HyperCube 6

In this chapter a description of the input must be provided

The input in the process is a txt file for Hypercube 6 in the CensusHub project. A subset follows

below:

The Attributes on Cell level are found after the pipe character | in the end of the lines that contains

such an attribute. In this case P,P;,Source: SCB on the fifth data line and NULL|CNI,P on the

ninth data line. On this fifth line there are also marked two textual attributes expressed with the extra

pair of -characters.

This file is put into the PX-Edit software to make it a PC-Axis file.

9


10/23

In PX-Edit the table is exposed as above where you can find the red corners in data line 5 and 9

corresponding to the cell attributes in the txt file above.

Beneath the Table Metadata parts concerning the attributes are found on the three bottom lines.

10


11/23

On the following cell note metadata screen you also can find the attributes as they are picked up in

PX-Edit from the txt-file.

11


12/23

The resulting PC-Axis file looks like this:

Notice the attribute keywords ATTRIBUTE-ID and the cell related keywords ATTRIBUTESincluding the cell position for the expressed attributes.

12


13/23

5 Preparing to handle the output of HyperCube 6

In this chapter a description on how to handle the output must be provided

In PX-Web it is possible to see the tables including the Attributes on cell level, this development is

ongoing.

See http://groups.google.com/group/pcaxis/topics?hl=en

Attributes on cell level in PX - Solution proposal.pdf98K ViewDownload

Attributes in the PX file format.pdf 216K ViewDownload

13
http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+on+cell+level+in+PX+-+Solution+proposal.pdf?hl=en&part=4&view=1http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+on+cell+level+in+PX+-+Solution+proposal.pdf?hl=en&part=4http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+on+cell+level+in+PX+-+Solution+proposal.pdf?hl=en&part=4http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+in+the+PX+file+format.pdf?hl=en&part=5&view=1http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+in+the+PX+file+format.pdf?hl=en&part=5&view=1http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+in+the+PX+file+format.pdf?hl=en&part=5http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+in+the+PX+file+format.pdf?hl=en&part=5http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+on+cell+level+in+PX+-+Solution+proposal.pdf?hl=en&part=4http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+on+cell+level+in+PX+-+Solution+proposal.pdf?hl=en&part=4&view=1http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+on+cell+level+in+PX+-+Solution+proposal.pdf?hl=en&part=4http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+in+the+PX+file+format.pdf?hl=en&part=5&view=1http://groups.google.com/group/pcaxis/attach/0ec2f631fb31ab44/Attributes+in+the+PX+file+format.pdf?hl=en&part=5


14/23

14


15/23

6 Using Census HyperCube 6 in the PC-Axis file based process

for Eurostat CensusHubThis chapter should be a sort of short User Manual in which is described how to use the example.

- Text-file, Census Hyper Cube 6, see chapter 4 above.

- PX-Edit, see chapter 4 above.

- PC-Axis file, see chapter 4 above.

- PX-Web including Attributes on cell level for visual checking, see chapter 4 above.

- Mapping Assistant mapping a PC-Axis file, see below.

- Test Client, see below.

- Web Client, see below.

15


16/23

Mapping Assistant mapping a PX-file

Data Flow

16


17/23

DDB Connections

Data Set

17


18/23

Mapping Set

Header Info

18


19/23

View Mapped data

Test Client

19


20/23

Web Client

Web Client main window

Selecrtions

20


21/23

View Results

View result as a table

21


22/23

22


23/23

View results download in HTML

Step by Step Tutorial PX-File_to_SDMX1

Documents

Transcript of Step by Step Tutorial PX-File_to_SDMX1