Harmonizing and integrating data

The new architecture for Data Collection Management in Istat

S.Macchia, M.Murgia, M.Perez, C.Romano, R.Torelli Istat - Directorate for Data Collection

2016 Data Collection Workshop Visions on Future Surveying

The Hague. October 3rd – 5th 2016

Towards a new organisation Background The need of transformation and modernization of official statistics has guided Istat towards a re-organization of production process

2 Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

Until 2015 data collection was managed according to a “silo” approach where each "silo" identified a specific field of statistics (census, social statistics, business statistics) and its corresponding production system within a specific Istat Department

Towards a new organization

Since April 2016 a new organization has been set up, based on Business Architecture Model (BAM) BAM covers both statistical activities and strategic organisational tasks and capabilities According to BAM, the production process relies on an integrated organization and centralised corporate cross-cutting support services (technical, scientific and administrative - methodology, information technology, data collection and dissemination, human resources, legal affairs, asset management, accounting )

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

3

Towards a new organisation Aims

Enhance effectiveness/quality, as a result of a standardisation of processes, solutions and tools;

enhance efficiency, as an effect of overcoming stovepipes in conducting processes, so as to facilitate reuse

Enhance productivity

enhance the integrated System of Statistical Registers enriched with single, controlled, and standardised information;

Enhance innovative activities with the saving of resources obtained.

4

Means

• a better exploitation of administrative data and their integration with data survey as well as the use of new and non-traditional data sources

• use of innovative technologies for collection, processing and dissemination of data

• pooling the knowhow

• adoption of standards, common principles and best practices

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

The new organizational structure

Directorate for Data Collection (DC)

…centralizes all functions and activities of data collection phase by

managing every single set of data collection process related to

surveys, administrative data, registers, new and non-traditional data

6 Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

…works in close cooperation with other Istat Directorate/sectors,

collaborating in specific activities, providing and receiving

services from them

Inter-relashionships and functions of DC with other Istat’s sectors

7

Thematic sectors

Administrative and legal sector

Territorial offices

Training sector IT sector

Methodological sector

Conduction

Directorate for Data Collection

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

Establishment of data collection tolls

Organization

AdminD acquisition and integration

Establishment of Data Collection tools • Designing and developing different steps of data collection at central

level, it is the core of the strategy of integration and harmonization of processes.

• It defines data collection modes, designs and develops tools and procedures for the data collection phase, in full compliance with the requirements expressed by the thematic sector;

• For this aim, persons with different expertise have been recruit from other sectors: i) experts of social and business surveys ii) methodologists specialized in the data collection phase and iii) IT experts of data collection software tools.

8 Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

Main tasks: 1. Assessment of statistical (alternative) sources 2. Designing survey and data collection tools 3. Implementing data collection tools 4. Designing training methods and tools 5. Designing infrastructures for data collection

Establishment of Data Collection tools

9

• Identification and assessment of data sources (surveys, administrative sources, registers)

• Designing the process of acquisition of administrative sources in cooperation with DC- Integration/Thematic sectors

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

2. Designing survey and data collection tools

1. Assessment of statistical (alternative) sources

• Survey design in cooperation with Thematic sectors • Choice of data collection mode and analysis of mode effect • Designing and optimization of interviewers networks • Harmonization of concepts, definitions, classifications taking care of

redundancy and response burden • Designing of questionnaire and associated documentation • Defining requirements for outsourcing services • Designing set of monitoring and data quality indicators • Designing and conduction test (pilots surveys, cognitive test, etc.)

Establishment of Data Collection tools

10

• Implementation of computer assisted and paper questionnaires • Definition of metadata and glossary • Building up and maintaining dictionaries and search engines for

automatic/assisted coding

3. Implementation of data collection tools

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

• Design and implementation of training methods and tools for interviewers networks and intermediate bodies (classroom training, interactive training, e-learning, etc )

• Identification of generalized tools and platforms for training

4. Design of training methods and tools

• Designing repository system of data and metadata • Designing management system for data collection flows • Cooperation with IT sector for designing architecture to manage multi-

technique modes

5. Design of infrastructures for data collection

Organization

11

Main tasks: Managing preliminary activities related to administrative/legal aspects

(defining costs, cooperating with administrative and legal sectors for outsourcing services, law references, privacy requirements, etc.);

Managing formal (non thematic) contacts with respondent units (eg. arranging informative campaigns) and non-respondents units (preliminary activities for penalties, etc.) and intermediate bodies involved in data collection processes (eg.municipalities);

Arrangement of training courses for interviewers and/or other intermediate bodies

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

Responsible for organizational aspects during the setting up and start up of DC processes, it takes care of formal contacts with respondents and non respondents units and manages administrative procedures in partnership with legal and administrative sectors

Conduction

12

Main tasks Managing operative aspects of DC, both when in house services are

used (e.g. CAWI technique) and when it is supported by outsourcing services such as CATI (call center) and CAPI technique (interviewers network)

Managing activities related to ‘first level’ contact with respondent units/intermediate bodies, as well as ‘recall’ contact to enhance the response rate

Monitoring the field work and performance of interviewers (if any) through data quality indicators

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

Responsible of field work, managing first level and recall contacts with respondents/intermediate bodies and monitoring the field work

AdminData collection and integration

13

Main tasks: Acquisition of archives by ARCAM system (an interface that guarantees

the security of the data transmission process from public and private institution)

Integration of AdminData (AD) at centralized level (record linkage) into the System of Integrated Microdata (SIM)

Preliminary data editing Dissemination of AD to Istat users; managing the Register of Activities

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

Managing data flows at centralized level, guarantees the integration and internal dissemination of AdminData and of those stored in statistical registers

Final remarks

14 Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

• Istat modernization process mainly focuses on a new System of Integrated Registers (SIR)

• the conceptual framework is based on a better exploitation of Administrative Data and integration with Survey Data;

• in this perspective, DC Directorate plays an important role in terms of harmonization and standardization of statistical processes

• DC is the main driver towards a new culture for the production of statistics

How DC is doing that? Involving human resources with high-level professional skills in specific

field and with different background (social statistics, economic statistics, methodology, information technology, etc)

Promoting a culture which relies on cooperation among sectors inside Istat

………..

Critical issues

• organizational complexity;

• the development and consolidation of an “entrepreneurial spirit” based on supply and demand of cross-cutting services

• cultural change in approaching statistical process

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

15

Harmonizing and integrating data, Directorate for data collection The Hague, Netherlands, 3 – 5 October 2016

16

Monica Perez perez@istat.it

Questions and suggestions are welcome