UNSD/ NSCB Regional workshop on data dissemination & communicationManila, Philippines, 20-22 June 2012

Promoting (survey) microdata dissemination policies:

the experience of the IHSN / ADP

François Fonteneau, OECD/PARIS21International Household Survey Network (IHSN)

Accelerated Data Program (ADP)francois.fonteneau@oecd.org

IHSN / ADP: background• Marrakech Action Plan for Statistics (MAPS): creation of IHSN and ADP

• Program funded by World Bank, implemented by a small core team (P21/WB + others); open partnership (ADB, UNSIAP, SPC, DFID , FAO, others)

IHSN: to improve the coordination of intl. organisations’ household survey activities.

ADP: to help data producers in developing countries increasethe use and value of their survey data.

Survey microdata: growing supply, unsatisfied demands

By producers: • “we don’t know how to deal with microdata”

– how to document them?– how to improve transparency on sources?– can we disseminate them in our country?

• If YES, how to disseminate them? • To whom? • What costs?

• “what do others do?”– identification of good practice

By users:• “we didn’t know they had data” Visibility• “they don’t want to share them” Accessibility• “these data are messy” Reliability• “this is not really what I need” Relevance• “how can I assess progress?”

Comparability• “where is the documentation?” Usability• “there is no transparency” Quality, credibility

Survey microdata: growing supply, unsatisfied demands

Different needs, different challengesReports, tables (PDF)

Indicators

On-line interactive tabulation, analysis and

mapping tools

Microdata (n% sample) and detailed metadata

Any web development tool

CensusInfo/DevInfo, etc.

REDATAM/IMIS, SuperStar, Nesstar, etc.

IHSN MMT + NADA

No issue of confidentiality

Disclosure risk managed by the application

Requires data anonymization

What outputs? What tools?

Promoting microdata dissemination policies: a variety of challenges

• Legal apply the law…or change it• Strategic set your priorities (NSDS)• Organizational break silos• Financial allocate minimal budget• Technical steal good practice

• … and cultural (fear of change)=> Advocacy on benefits / risks

For discovering data – You and your users will know about data availability

by providing searchable data catalogsFor making data usable

– Users need to fully understand the data: why, by whom, when, and how they were collected, processed and disseminated

For assessing data quality

=> International metadata standards for microdata: DDI

The importance of documentation

DDI : before and after

(...or worse !)

GSBPM

GSBPM, DDI, SDMX

IHSN Microdata Management Toolkit (DDI metadata editor) NADA: on line searchable survey cataloguing system

IHSN toolkit for documentation

• Anonymized datasets for dissemination– Anonymization techniques/tools:

• Risk measurement / risk reduction / information loss• Methods: recoding, rank swapping, micro-aggregation,

sampling, etc.• Available tool (R package): sdcMicro {guidelines in

preparation}– Different levels of anonymization / different users

controls / different procedures : PUF; SUF• Full dataset in enclave direct control of NSO

Confidentiality of microdata

Confidentiality of microdata

• Only when no confidentiality concerns

• 3 modes for file distribution:– PUF - Direct– SUF - Licensed– Enclave

• Screen users and usage• Progressive approach

From UNECE Task Force

1. It is appropriate for microdata collected for official statistical purposes to be used for statistical analysis to support research as long as confidentiality is protected (i.e. as long as it is not possible to identify data referring to an individual).

2. Microdata should only be made available for statistical purposes.

3. Provision of microdata should be consistent with legal and other necessary arrangements that ensure that confidentiality of the released microdata is protected.

4. The procedures for researcher access to microdata, as well as the uses and users of microdata should be transparent, and publicly available.

UPDATES IN NATIONAL STAT LEGISLATION?

Principles for microdata dissemination

Formulating an access policy and procedures

Principles, guidelines and practices

http://www.ihsn.org/home/index.php?q=focus/dissemination-microdata-files-principles-procedures-and-practices

PHILIPPINESGood practice !

REQUEST ACCESS TO DATA !

INDIA

COLOMBIA

VANUATU

More?

• IHSN: www.ihsn.org• ADP: www.ihsn.org/adp

• francois.fonteneau@oecd.org • odupriez@worldbank.org

THANK YOU !