S T A T I S T I K A U S T R I A14.4.2010 1

www.statistik.at

Quality Assessment of register-based StatisticsA Quality Framework

Manuela LENK

Directorate Population Statistics

Register based census© STATISTIK AUSTRIA

I n f o r m a t i o n e n

Wir bewegen

S T A T I S T I K A U S T R I A14.4.2010 2

Switch from traditional to register-based census in Austria 2011 Compared to other countries transition time is relatively short Census Test was carried out for 2006

Seven base registers and several comparision registers Registers are combined using unique identifiers (bPK) Gathering experience for the register-based census

Introduction

2

S T A T I S T I K A U S T R I A14.4.2010 3

Quality is influenced by (according to the quality report as requested by Eurostat)

Quality of the administrative data source Existence of an unique key or rather the availability to

link a person Comparision between survey and administrative data Item imputation

Consequence Establishing a quality framework for the register based

census 2011 – cooperation with WU Vienna

Quality Issues in Census Test

3

S T A T I S T I K A U S T R I A14.4.2010 4

Quality assessment of statistics based on administrative data Quality Indicator for

Raw Data – data obtained from administrative sources Each attribute in each register Whole register (data source)

Census 2011 Attributes of the Final Data Pool

Objectives

4

S T A T I S T I K A U S T R I A14.4.2010 5

Assessment of quality is derived from hyperdimensions (HD) for each attribute in each register.

Documentation Pre-Processing External Sources Imputation

Hyperdimensions

5

S T A T I S T I K A U S T R I A14.4.2010 6

Documentation Focus is on processes taking place before the data is transfered to

Statistics Austria Data treatment at the source maintaining the register Reliability of the data source

Quality criteria Relevance of attributes for the source (e.g. legal foundation) Availability for certain reference dates Compatible definitons

Registers are benchmarked using a perfect pseudo register Informations are extracted from the meta database

Hyperdimensions (1)

6

S T A T I S T I K A U S T R I A14.4.2010 7

Pre-Processing Process of Data editing from Raw Data to the Edited Data Analysis of Raw Data

Error corrections Recoding Plausibility checks

Quality criteria Proportion of missing unique keys Proportion of values out of domain Item non-response

Automated generation out of database

Hyperdimensions (2)

7

S T A T I S T I K A U S T R I A14.4.2010 8

External Source Checks accuracy of the data record linkage with survey data (e.g. Labour Force Survey)

Quality criteria Consistency of attribute values

For some attributes no external source for comparision exists, thus it is being replaced by

Expert interviews Rating of accuracy – expert knowledge obtained from working

experience Subjective – should be considered when allocating the weight for

this hyperdimension

Hyperdimensions (3)

8

S T A T I S T I K A U S T R I A

Each hyperdimension receives a weight in accordance with its relevance for the project.

Within these hyperdimensions quality criteria are determined and quality indicators are derived for each hyperdimension (HDD, HDP, HDE).

After weighting these dimensions (vD, vP, vE) we get an aggregated quality indicator q.

14.4.2010 9

Quality Indicators (1)

9

S T A T I S T I K A U S T R I A14.4.2010 10

Finally we are able to derive a quality indicator qij for each attribute in each register

We could derive an aggregated quality indicator for each register too (weighted row sum).

Quality Indicators (2)

10

S T A T I S T I K A U S T R I A14.4.2010 11

Register-based census Quality assessment within the Census Database

Combination of existing registers to a new data base.

Quality assessment of the Final Data Pool Census Database including item imputations.

Application on linked data sets

11

S T A T I S T I K A U S T R I A14.4.2010 12

Unique Attributes attribute exists exactly once in one register and is transfered directly to the CD

qij = qj = qΨj link

Multiple Attributes attribute exists in more than one register. Information from different sources are combined in the CD using certain decision rules – e.g. majority principle (Sex)

Quality indicators for this attribute from different registers are combined to an overall quality indicator qj (e.g Dempster-Shafer Theory)

Additional check using external data source HDE leads to a corrected quality indicator qΨj link

Derived Attributes several attributes are linked to a new attribute Quality indicator qj is generated by using the quality indicators of the

input attributes Additional check using external data source HDE leads to a corrected

quality indicator qΨj link

Assessment of the Census Database (CD)

12

S T A T I S T I K A U S T R I A14.4.2010 13

Census Database including item imputations Hyperdimension Imputation (HDI)

Quality indicators depend on methods of imputation Shall be generated with appropriate valuation

methods The weight for this hyperdimension is approximated by

the proportion of imputation

Quality assessment of the Final Data Pool

13

S T A T I S T I K A U S T R I A

Process-oriented Quality Framework back

14

S T A T I S T I K A U S T R I A

Conclusion

Framework should act as a road map Next milestones are

Establishing check lists (questionnaires) Specify how to derive the quality indicators for each

hyperdimension and how to aggregate them

General approach Application to other projects is possible

15

S T A T I S T I K A U S T R I A14.4.2010 16

Additional Information Quality Framework Fiedler/Schwerer/Berka/Humer/Moser: Quality Assessment for Register

based Statistics in Austria Register-based Census

www.statistik.at

Outlook

16