Assessing the Capacity of Statistical Systems

Development Data Group

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 2

Summary

Overview of the assessment processSome tools and frameworksAssessing organization and managementIndicators of statistical capacity building

Part 1: Overview of the assessment process

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 4

Assessing statistical capacity

The statistical systemInputs• Financial and human resources• Legislative and regulatory framework• Statistical and physical infrastructure

Intermediate processes• Statistical operations and procedures• Organization and management

Outputs • Statistical products and services

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 5

Looking at outputs

Assessing data qualityThe Data Quality Assessment Framework (DQAF)

Data coverage and disseminationComparison with international frameworks and good practiceGeneral Data Dissemination System (GDDS)

Meeting users needsBalance between supply and demandAnticipation of new needs and demands

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 6

Intermediate processes

Reviewing statistical operations and procedures (DQAF and GDDS)

Appropriateness and correspondence with good practiceCommunications with providers and actions to reduce data burden and protect privacyQuality awareness and control

Assessing management and coordinationFinancial management and controlHuman resource managementEffectiveness of logistics

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 7

Inputs

Financial and human resourcesLevels and trends in recurrent and development budgetsNumbers and levels of skills/training

Legislative and regulatory frameworkCompliance with fundamental principles

Statistical infrastructureAdequacy of registers, sampling frames etc,

Physical infrastructureAdequacy of buildings, computers and communications equipment

Part 2: Some tools and frameworks

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 9

Data Quality Assessment Framework

Monitors the quality of economic and social data:

Quality of the statistical productQuality of the statistical agency

Used by IMF for data part of Reports on Standards and Codes (ROSCs)Monitors extent to which observed procedures follow good practice

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 10

Coverage

General DQAF as well as separate frameworks for:

Main economic statistics frameworks:• National accounts; Balance of payments;

Government finance; Money and banking; Consumer price index

Socio-demographic statistics (being prepared by World Bank)• Income poverty (completed); Education;

Health; Population (in preparation)

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 11

Structure

Six dimensions of quality0. Prerequisites of quality1. Integrity2. Methodological soundness3. Accuracy and reliability4. Serviceability5. Accessibility

Hierarchical structureDimensions• Elements

– Indicators– Focal issues and key points

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 12

GDDS

Sets out objectives for data production and dissemination in four “dimensions”:

Data: coverage, periodicity, and timelinessQualityIntegrityAccess by the public

Provides a framework for developmentNational authorities set their own priorities and timing to achieve their objectives

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 13

Participation

Voluntary and involves three actions:1.  Commitment to use the GDDS as a

framework for statistical development2.  Designation of a country coordinator3. Publication of metadata, descriptions

of– • current statistical production and

dissemination practices• plans for short- and longer-term

improvements• need for support including technical

assistance

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 14

Coverage

Economic and financial data – responsible agencies and main data series

Real sectorFiscal sectorFinancial sectorExternal sector

Socio-demographic data – responsible agencies and main data series

PopulationHealthEducationPoverty

Part 3: Assessing the organization and management of statistical agencies

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 16

One approach

Effectiveness of a statistical system is determined by

The products it produces and the services it providesIts functional and organizational structure

Carry out a SWOT analysis ofThe internal organizationThe external environment in which the system operates

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 17

Internal organization

StructureCoordinationHuman resourcesInfrastructureManagement systems

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 18

External environment

Statistical legislation and regulationsBudgetsAccountability and reportingRelationships with usersPublic image

Part 4: Indicators of statistical capacity building

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 20

Assessing capacity

16 quantitative indicatorsResourcesInputsStatistical products

18 qualitative indicatorsEnvironmentCore statistical processesQuality of statistical products

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 21

The quantitative indicators

ResourcesAnnual budget - recurrent and development, locally and externally funded

InputsData sources – censuses, surveys and administrative data

Statistical productsMedia and topics covered

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 22

Using quantitative indicators

Provide rough measure of extent of statistical activitiesUsefulness limited by:

Lack of benchmarksDo not measure efficiency or effectiveness

Need to be interpreted using contextual information provided by qualitative indicators

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 23

Qualitative indicators

Cover a broader view of factors determining capacityBased on DQAF Framework

Six indicators on institutional prerequisitesTwo indicators on data integrityOne indicator on methodological soundnessFour indicators on accuracy and reliability Three indicators on serviceabilityTwo indicators on accessibility

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 24

Coverage

Legal and institutional environmentProfessional and cultural settingMethodological expertiseAdequacy of data sourcesAnalytical and processing capacity and quality controlRelevance of products to users needsEffectiveness of dissemination

Assessing Statistical Capacity and Improving Data Quality for Development Session 3 Slide 25

Measurement and recording

Quantitative indicators use four point assessment scale

Level 1 – largely underdevelopedLevel 2 – developing but with observed deficienciesLevel 3 – moderately well developedLevel 4 – highly developed, in line with good practice