UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE

---------

WORKSHOP ON POPULATION AND HOUSING CENSUSES

FOR COUNTRIES OF EASTERN EUROPE CAUCUSES AND CENTRAL ASIA

GENEVA, 28-29 SEPTEMBER 2015

---------

UNECE STUDY ON NATIONAL PRACTICES

IN THE 2010 ROUND IN EECCA COUNTRIES----------

PART I

METHODOLOGICAL AND OPERATIONAL ASPECTS

---------

IAN WHITE

METHODOLOGICAL AND OPERATIONAL ASPECTS

• Methodology• Dates of censuses• Technological developments• Outsourcing• Innovations• Field activities• Communications and publicity• Statistical disclosure control• Dissemination• Documentation and metadata• Costs and benefits• Data quality• Challenges and successes

Methodology in the EECCA Region

Traditional census – all 10 countries Full field enumeration, no registers

Elsewhere in the UNECE regionTraditional census – 24 countriesRegister-based census – 9 countries (3 in 2000)Combined methodology – 10 countries (6 in 2000)

Methodology in the EECCA Region

Traditional census – all 10 countries Full field enumeration, no registers

2009 25 February Kazakhstan 24 March Kyrgyzstan 13 April Azerbaijan 14 October Belarus2010 1 October Tajikistan 14 October Russian Federation 2011 12 October Armenia2012 15 December Turkmenistan2014 14 May Moldova 5 November Georgia

No census taken: Ukraine, Uzbekistan

Elsewhere in the UNECE regionTraditional census – 24 countries (30 in 2000)Register-based census – 9 countries (3 in 2000)Combined methodology – 10 countries (6 in 2000)

Technology in the EECCA Region

Technology EECCA Rest of UNECEData collection Internet 0 countries 18 countries (62%)Laptops 0 countries 10 countries (34%)Data processingOMR 4 countries 16 countries (55%)OCR 6 countries 18 countries (62%)Neither 4 countries 9 countries (31%)Geographic Information Systems (GIS)Paper maps 8 countries 23 countries (74%)Vector data 1 country 22 countries (71%)GPS coordinates 1 country 19 countries (61%)Digital boundaries 2 countries 25 countries (81%)Ortho-photography 2 countries 22 countries (71%)Time for preparation of IT infrastructure< I year 1 country 8 countries (21%)1-2 years 2 countries 13 countries (33%)2-3 years 4 countries 8 countries (21%)3-5 years 2 countries 6 countries (15%)> 5 years 0 countries 4 countries (10%)

Outsourcing in the EECCA Region

Outsourced activity EECCA Rest of UNECE

Printing of questionnaires 9 20 (74%)Printing of other field documents/materials 8 20 (74%)Delivery of questionnaires 7 17 (63%)Publicity 6 22 (81%)Collection of questionnaires 5 14 (52%)Mapping enumeration areas 4 13 (48%)Primary data capture/coding 3 17 (63%)Translation of field materials 3 16 (59%)Data storage 3 5 (19%)Imputation 3 3 (11%) Data quality assurance 3 2 (7%)Online response system 0 11 (41%)Recruitment/training of field staff 0 7 (26%)Design/provision of form tracking system 0 7 (26%)

Benefits and challenges of outsourcing

Outsourced activity EECCA Rest of UNECE

Benefit

Saved time 8 15 (56%)Reduced costs 5 14 (52%)Utilized resources/expertise 4 24 (89%)Improved data quality 3 9 (33%)Improved coverage/response 0 4 (15%)Improve public trust 0 6 (22%)

Challenge

Keeping to schedule 6 24 (89%)Managing contracts 3 17 (63%)Keeping to budget 3 15 (56%)Monitoring data quality 2 10 (37%)Meeting user needs 1 7 (26%)Integrating systems 0 13 (48%)Managing negative press and public perceptions 0 6 (22%)

Innovations in the EECCA region

Innovation EECCA Rest of UNECE

Dissemination via the Internet 4 23 (66%)GIS mapping 3 18 (51%)Scanning 3 6 (17%)ICR 3 9 (26%)Automatic coding 2 13 (37%)Edit/imputation 2 14 (40%)Long form/short form 2 2 (6%)Coverage survey 2 3 (9%)Use of registers 0 23 (66%)

Field activities in the EECCA region

,

Field activity EECCA Rest of UNECE

Field staff recruitment Locally only 5 16 (53%) Both nationally/locally 4 10 (33%)

Field staff training Locally only 5 7 (23%) Both nationally/locally 4 19 (63%)

Period of field staff training< I week 4 19 (63%)1 week 1 7 (23%)2 weeks 1 4 (13%) 3 weeks 2 0 -> 3 weeks 1 0 -

Pre-census tests < 1 year before Census 0 8 (27%) 1 year before Census 4 10 (33%)2 years before Census 3 16 (53%)3+ years before Census 1 7 (23%)None 0 1 (3%)

Communications in the EECCA region

Communications EECCA Rest of UNECE

Strategy adopted 8 26 (90%)Strategy not adopted 1 3 (10%)

Consultation on key topics

Questionnaire design 8 20 (77%)User requirements 6 23 (88%) Design, content and dissemination of outputs 5 14 (54%User satisfaction/ public opinion survey 5 4 (15%) Methodology 4 18 (69%)Language and community liaison/outreach 3 14 (54%)Confidentiality and/or statistical disclosure 3 9 (35%)Field operations 2 11 (42%)Data quality 2 6 (23%) Enumerating hard-to count and special groups 1 21 (81%)

Communications in the EECCA region

Key stakeholders EECCA Rest of UNECE

User groups and stakeholders consulted

Central government 8 26 (100%) Local government 8 24 (92%)Academics and/or education service providers 8 24 (92%)Press and the media 7 14 (54% Health service providers 7 6 (23% Other public service/utility providers 7 12 (54%)Ethnic, racial and faith communities 6 15 (58%) Market researchers 4 8 (31%) Business, retailers and other commercial sectors 4 8 (31%)Disability groups 1 13 (50%)Housing and homeless organizations 1 6 (23%)

Communications in the EECCA region

Locations EECCA Rest of UNECE

Locations used in publicity

Regional and local government offices 8 22 (79%)Colleges and universities 8 14 (50%)Schools 6 16 (57%)Banks, post offices, police stations and other public use facilities 5 14 (50%)Stations, airports and seaports 7 7 (25%)Information help points 2 12 (43%)Libraries 4 9 (32%)Places of religious worship 2 8 (28%)Factories and other workplaces 2 3 (11%)Bars, pubs, theatres and places of entertainment 1 2 (7%)

Communications in the EECCA region

Target groups EECCA Rest of UNECE

Groups targeted in publicity

School children and students 7 15 (60%)Particular ethnic/racial/ religious minority groups 5 14 (56%)Recent immigrants 5 9 (36%)Young adult males 5 9 (36%)The elderly 4 12 (48%)Parents of young babies 4 9 (36%)Visually impaired, deaf and other disabled 2 9 (36%)

Statistical disclosure control in the EECCA region

SDC measure EECCA Rest of UNECE

Measures taken to protect confidentiality

Pre-tabular measures only 1 7 (18%)Post-tabular measures only 0 17 (44%)Both methods 5 15 (39%)No measures taken 2 0 -

Type of SDC method

Restricting the number of output categories 4 31 (79%)Applying minimum thresholds for outputs for small areas 4 28 (72%)Modifying the data 2 20 (51%)• Over-imputation 2 0 -•Record swapping 1 5 (13%)•Cell suppression 0 18 (46%)•Rounding 0 9 (23%)• Small cell adjustment 0 2 (5%)

Dissemination in the EECCA region

Dissemination EECCA Rest of UNECE

Main method

Paper/hard/copy publications 6 3 (10%)Static web pages 2 17 (42%)Interactive online databases 1 13 (32%)The EU Census Hub 0 5 (12%)GIS we-based mapping tools 0 1 (2%) CD-ROM/DVD 0 0 -

Secondary methods

CD-ROM/DVD 8 17 (42%)Static web pages 7 22 (55%)Paper/hard/copy publications 5 27 (68%)Interactive online databases 4 17 (42%)IS web-based mapping tools 2 16 (40%)The EU Census Hub 0 28 (70%)

Dissemination in the EECCA region

Dissemination EECCA Rest of UNECE

Dissemination of mircrodata

Microdata disseminated 5 26 (65%)• To all users 3 8 (20%)• To selected users under specific conditions 2 17 (42%)Microdata not disseminated 2 8 (20%)Not decided 2 7 (18%)

Dissemination of evaluation reports

General report only 4 8 (20%)Quality report only 0 13 (32%)Both 2 14 (35%)Neither (or not yet decided) 3 5 (12%)

Dissemination in the EECCA region

Geographic level EECCA Rest of UNECE

Lowest level of geography for which any censusdata is disseminated

Local administrative unit (LAU) level 2 7 14 (36%)Administrative unit larger than LAU 2 2 1 (3%)Smallest geographical unit used in census 0 11 (28%) Census block 0 5 (13%)1 km grid square 0 1 (3%)Other geographical level 0 7 (18%)

Documentation and metadata in the EECCA region

Type of metadata EECCA Rest of UNECE

The census questions 7 29 (72%) Explanatory noted to tables 6 37 (92%)Methodological papers/report 6 35 (88%)Definitions of terms and concepts used 5 37 (92%)Data visualisation (maps, graphs, charts etc) 5 32 (80%)Levels of response 4 22 (55%)Changes to definitions etc since previous census 3 23 (58% User guides 1 16 (40%)Comparisons with other data sources 1 15 (38%)Coverage adjustments 1 12 (30%)Levels of imputation 0 27 (68%)Data dictionary/glossary of terms 0 21 (52%)Commentary on the results 0 13 (32%)Confidence intervals 0 9 (22%)

Costs and benefits in the EECCA region

Median costs Total costs Per capita Per capita (US$ million) (PPP) EECCA countries 10.1 1.53 3.09Rest of UNECE Traditional 10.2 7.55 10.21 Register-based 1.9 0.24 0.18 Combined 18.7 3.94 6.01

Costs and benefits in the EECCA region

Mean percentage of costs EECCA Rest of UNECE (traditional)

Field operation 50.3 53.6

Data processing 9.3 15.0

Printing 9.0 4.4

Mappings and other GIS support

services 7.8 5.1

Project management and administrative

support services 4.6 3.8

Rehearsal/pilots/tests 4.5 2.3

Dissemination and documentation 3.5 4.1

Publicity and promotion 2.0 2.7

Development of online questionnaires 0.2 2.3

All other costs 8.2 6.8

Costs and benefits in the EECCA region

Who benefited EECCA Rest of UNECE Main usersCentral government departments/ organisations 9 33 (85%)Local government authorities/ organisations 9 33 (85%)Own NSI 7 28 (72%)General public 5 12 (31%)Academics and education service providers 4 16 (41%)International organisations 2 14 (36%)

Press and the media 0 6 (15%)Business, marketing, retailers and other commercial sectors 0 4 (10%)Health service providers 0 2 (5%)

Data quality in the EECCA region

Quality dimension EECCA Rest of UNECERelevanceManaged 4 33 (85%)• Published 0 16 (41%)• Not published 4 17 (44%AccuracyManaged 6 38 (97%)• Published 0 22 (56%)• Not published 6 16 (41%)TimelinessManaged 6 37 (95%• Published 0 20 (51%)• Not published 6 17 (44%)AccessibilityManaged 4 35 (90%)• Published 0 18 (46%)• Not published 4 17 (44%)ComparabilityManaged 5 36 (92%)• Published 2 25 (64%)• Not published 3 11 (28%)CoherenceManaged 3 37 (95%)• Published 1 22 (56%)• Not published 2 15 (39%)

Data quality in the EECCA region

Measuring accuracy EECCA Rest of UNECE

Demographic analysisTo measure under-coverage 5 20 (53%)To measure over-coverage 4 19 (50%)To measure variance 7 12 (32%)

Independent post-enumeration coverage surveyUsed to measure under-coverage 1 17 (45%)Used to measure over-coverage 1 16 (42%)Used to measure variance 1 8 (21%)

Other form of coverage surveyTo measure under-coverage 4 4 (11%)To measure over-coverage 3 3 (8%)To measure variance 2 0 -

Comparison with existing surveysTo measure under-coverage 1 15 (39%)To measure over-coverage 1 12 (32%)To measure variance 4 15 (39%)

Data quality in the EECCA region

Measuring accuracy EECCA Rest of UNECE

Comparison with aggregated administrative datasetsTo measure under-coverage 1 16 (42%)To measure over-coverage 1 16 (42%)To measure variance 3 6 (16%)

Comparison with unit record administrative dataTo measure under-coverage 1 17 (45%)To measure over-coverage 1 16 (42%)To measure variance 2 5 (13%)

Analysis of questionnaire return ratesTo measure under-coverage 1 10 (26%)To measure over-coverage 1 8 (21%)To measure variance 1 4 (11%)

Data quality in the EECCA region

Measuring accuracy EECCA Rest of UNECE

Comparison with aggregated administrative datasetsTo measure under-coverage 1 16 (42%)To measure over-coverage 1 16 (42%)To measure variance 3 6 (16%)

Comparison with unit record administrative dataTo measure under-coverage 1 17 (45%)To measure over-coverage 1 16 (42%)To measure variance 2 5 (13%)

Analysis of questionnaire return ratesTo measure under-coverage 1 10 (26%)To measure over-coverage 1 8 (21%)To measure variance 1 4 (11%)

Method used for adjusting figures

Imputation 6 31 (91%)Weighting 0 7 (21%)Other methods 0 7 (21%)

Challenges and successes in the EECCA region

Major challenges EECCA Rest of UNECE

Improving/maintain data quality 8 38 (95%)Insufficient staff resources/expertise 8 26 (65%)Improving data processing 7 35 (88%)Obtaining financial resources 7 30 (75%)Improving data collection 7 27 (68%)Mapping 7 22 (55%)Managing local infrastructures 7 21 (52%)Improving/maintaining response rates 6 28 (70%)Balancing user needs against respondent burden 6 28 (70%)Identifying residential addresses 6 25 (62%)Implementing quality assurance checks 5 38 (95%)Keeping to budget 5 29 (72%) Negative public attitude 5 25 (62%)User acceptance 5 25 (62%) Recruiting sufficient field staff 5 24 (60%)Difficult terrain 5 16 (40%)Keeping to planned timetable 4 34 (85%)

Challenges and successes in the EECCA region

Major successes EECCA Rest of UNECE

Kept within budget 5 28 (70%)Improved/maintained data quality 4 29 (72%)Met deadlines 4 28 (70%)Improved census methodologies 4 27 (68%)Improved census technologies 4 26 (65%)Improved data dissemination 3 23 (58%)Improved/maintained participation/ response rates 2 21 (52%)Improved logistics and coordination 2 20 (50%)Overcoming public resistance 1 12 (30%)Implementing Project Management methods/tools 1 10 (25%)

DISCUSSION