Developments of the Data Infrastructure in Germany since the end of the 90‘s

PD Dr. Hilmar Schneider, IZA Bonn

ODaF Europe 2009, IZA, Bonn, April 2nd-3rd, 2009

2

Who are the key players in official micro data production in Germany?

• Independent Statistical Offices at the Länder level are collecting data on household and firm level (Micro census, census, income and expenditure survey, cost structure survey, etc.)

• Federal Statistical Office, Wiesbaden (co-ordinates statistical offices at the Länder level, but has no directives; however, many official surveys are based on federal law)

• Federal Labor Agency, Nürnberg (main source: individual data that are relevant for pension claims; individual data related to job search and active labor market policy)

• Federal Reserve Bank of Germany (firm level data and data related to the monetary market)

3

Philosophy of the Federal Statistical Offices

• Providing aggregate figures for relevant issues of the economy and the society

• Data collection according to legal duties

• Relevant information is collected independently in different surveys (Example: Hourly wages can only be computed on the aggregate level)

• Huge waste of information• Heterogeneity on the micro level cannot be exploited for the

identification of causal structures

Consequences

4

The situation in the middle of the 90‘s

• Access to micro data from official statistics was almost impossible• No interaction between official statistics and the research

community• Researchers had to address directly to the administration• Prohibitive cost for access to micro data• Little documentation of available data• Huge concern about the potential of de-anonymization of official

micro data

5

What caused the tipping point at the end of the 90‘s?

• Large-scale research project on anonymization of micro data (Hauser/Müller)

• As a result, the concept of factual anonymization became accepted• Memorandum by Hauser, Wagner, Zimmermann:

Erfolgsbedingungen empirischer Wirtschaftsforschung und empirisch gestützter wirtschafts- und sozialpolitischer Beratung (IZA DP No. 14)

6

The process triggered by the memorandum

• In late 1999, the Federal Minister for Education and Research called a „Commission for the improvement of informational infrastructure between science and statistics“

• In early 2001, the commission published its report („Ways to an improved informational infrastructure“)

• The report contained 35 recommendations, among them Creation of research data centres and data service centres Creation of public use files from official micro data Creation of a Council for Social and Economic Data (RatSWD)

• In late 2001, the Constituing Council for Social and Economic Data was formed

• In 2002, the first research data centres were founded• In late 2004, the Council for Social and Economic Data was

constituted

7

About the RatSWD

• 12 members, 6 of them elected every two years by a scientist community (researchers with a doctoral degree), the other 6 are mandated by research data centres and data service centres

• Funded by the Federal Ministry of Education and Research• Main objectives:

Advising federal governments with regard to further improvement of data infrastructure

Recommendations regarding the establishment and evaluation of research data centres and data service centres

Stimulating and supporting projects that might contribute to the improvement of data infrastructure

Promotion of scientific offspring Improving the interrelations between science and official statistics

• The RatSWD has been called for six years and is currently undergoing an evaluation of its work

8

About the RatSWD

9

The research data centres

• Data Research Centre of the Federal Statistical Office• Data Research Centre of the Statistical Offices of the Länder• Data Research Centre of the Federal Labor Agency at the Institute

for Employment Research• Data Research Centre of the German Pension Insurance• Data Research Centre at the Institute for Educational Progress

10

What are the objectives of Research Data Centres?

• Provision of available data for scientific research• Compliance with data protection rules• Equal treatment of data users• Creation of user friendly data• Independent research• Avoiding privileged access to micro data

11

Structural Problems of Research Data Centres

• Dependent part of a public authority instead of independent institution

• Fuzzy frontier between routine task of authority and special task of RDC

• Declarative disposal with regard to external funds needed• After initial funding by the Federal Ministry of Education and

Research has run out, internal legitimization is becoming dominant

12

Synergy Effects

• KombiFid (establishment data based on surveys carried out by the Federal Statistical Office, and the Deutsche Bundesbank with establishment data of the Federal Labor Agency)

• Merged Biographies (employment status records of the Federal Labor Agency merged with pension records of the German Pension Insurance)

• McTax Panel (tax payer panel merged with data from micro census)

13

The Data Service Centres

• IDSC at IZA • German Micro Data Lab at GESIS

14

What are the objectives of Data Service Centres?

• Provision of comparative data documentation (meta data)• Training and advising data users• Development of adequate concepts of anonymization• Development of prototypical scientific use files• Extension of analytical potential of available data sets• Service for data analysis (remote access)• Support for data users• Agency to international micro data sets• Preparation of virtual data library• Development of links across different data sources• Development of indicators• Independent research with focus on methodology• Many more ...

15

Conflicts between Research Data Centres and Data Service Centres

• Competition between RDCs and DSCs in the field of user service, especially with regard to data documentation and remote data access

• RDSs are under external pressure by service offered through DSCs• RDCs are trying to make DSCs obsolete by adapting services that

were originally provided by DSCs• Danger of losing comprehensive value added (e.g. comparative

documentation of data)• RDCs may lose internal power of legitimation, if competitive

pressure vanishes

16

Concluding remarks

• The creation of RDCs and DSCs has been a success story• Competition between RDCs and DSCs should rather be understood

as a driving force for progress than a redundancy, which absorbes ressources

• Unused potential for the creation of DSCs

IZA – Shaping the Future of Labor

http://www.iza.org