statistik.at Seite 1
Norbert Rainer
Quality Reporting and Quality Indicators
for Statistical Business Registers
European Conference on Quality in Official Statistics Rome, 8 - 11 July 2008
Special topic session 21
statistik.at Seite 2
Background and motivation
Character of a statistical business register
Some criteria for quality indicators
Example Accuracy
Concluding remarks
Overview
statistik.at Seite 3
Article 6 of the new BR – Regulation ( OJ L61, 5.3.2008, p. 6-16):
Quality standards and reports
(1) MS take all measures to ensure the quality of the BR
(2) MS provide to the Commission (Eurostat) with a report on the quality of the BR
(3) Details of the quality report to be adopted in accordance with the regulatory procedure with scrutiny
(4) MS inform the Commission (Eurostat) on major changes affecting the quality of the BR
Background and motivation (1)
statistik.at Seite 4
Eurostat Business register unit:
-BR annual inquiry: included earlier few pilot questions on quality
-BR Recommendations Manual: Chapter 10 Quality Policy
• describes the quality dimensions (one additional: completeness)
• explains causes of quality defects
• reviews instruments of quality measurement
• outlines quality improvement strategies
• gives examples of quality indicators
Background and motivation (2)
statistik.at Seite 5
Eurostat: development of standard quality indicators
-The standard quality indicators:
Background and motivation (3)
Quality component
Indicator
Relevance R1. User satisfaction index R2. Rate of available statistics
Accuracy
A1. Coefficient of variation A2. Unit response rate (un-weighted/weighted) A3. Item response rate (un-weighted/weighted) A4. Imputation rate and ratio A5. Over-coverage and misclassification rates A6. Geographical under-coverage ratio A7. Average size of revisions
statistik.at Seite 6
Background and motivation (4)
Timeliness and Punctuality
T1. Punctuality of time schedule of effective publication T2. Time lag between the end of reference period and the date of first results T3. Time lag between the end of reference period and the date of the final results
Accessibility and clarity
AC1. Number of publications disseminated and/ or sold AC2. Number of accesses to databases AC3. Rate of completeness of metadata information for released statistics
Comparability
C1. Length of comparable time-series C2. Number of comparable time-series C3. Rate of differences in concepts and measurement from European norms C4. Asymmetries for statistics mirror flows
Coherence CH1. Rate of statistics that satisfies the requirements for the main secondary use
Source: Doc.ESTAT/02/Quality/2005/9/Quality indicators
statistik.at Seite 7
Background and motivation (5)
- The ESS Guidelines for quality reports (draft version 2)
With respect to quality indicators a distinction is made between the following six categories of statistics:
• Sample surveys
• Censuses
• Statistics derived from administrative sources
• Surveys involving data from multiple sources
• Price indices
• Statistical compilationsSource: Doc.ESTAT/DDG-02/Quality/2008/05aa
statistik.at Seite 8
Character of a statistical BR (1)
Sample surveys
Censuses
Statistics derived from administrative sources
Surveys involving data from multiple sources
Price indices
Statistical compilations
not a sample at all and no survey
intention of full coverage, but not a survey
administrative sources are used, but a database
multiple sources may be used, but a database
not any similarities
not any similarities
Categories of statistics Statistical Business register
Statistical registers
statistik.at Seite 9
BR is an instrument for conducting surveys
BR provides links to administrative sources
BR is a statistical database of longitudinal and cross- sectional character
BR is a statistical product to which the quality dimensions
• Relevance
• Accuracy
• Timeliness and punctuality
• Accessibility and clarity
• Comparability and coherence
do also apply
Character of a statistical BR (2)
statistik.at Seite 10
Quality indicators should be based on straightforward
concepts so that international comparability could be achieved
Quality indicators should primarily refer to the quality of the statistical BR, not to the quality of the applied administrative data / register
Quality indicators should be selected and designed with a view to the main uses of the statistical BR
Quality indicators should not be restricted to certain quality measurement instruments
Some criteria for quality indicators (1)
statistik.at Seite 11
A set of few, but significant indicators seems more appropriate than a long list of indicators
However, certain differentiations (especially with a view to accuracy or timeliness) need to be made in order to take into account the “weight” for the overall quality assessment, e.g.:
•kind of units
•size classes
•whether sample unit or not
•level of classification
Some criteria for quality indicators (2)
statistik.at Seite 12
Example: Accuracy
Coverage Measures of under / over coverageMeasures of falsely active unitsMeasures of duplicates
Completeness Measures of missing values of key variables
Deliverability Measures of quality of address data
Sampling Measures of quality of activity codingMeasures of quality of other classification codingMeasures of quality of size class coding
Other processing aspects of Measures of wrong links to administrative data
Measures of wrong unit structures etc.
statistik.at Seite 13
Statistical registers are statistical instruments but also a statistics category of its own
Quality reporting guidelines should take this into account explicitly
Many countries have already produced quality indicators for their BR
Concluding remarks
statistik.at Seite 14
THE END
Thank you for your attention !
Top Related