Assessing Information Needs and Survey Alternatives
description
Transcript of Assessing Information Needs and Survey Alternatives
1
Assessing Information Needs and Survey Alternatives
Kathleen Beegle, DECRGPoverty and Inequality CourseModule 1: Multi-topic Household SurveysJanuary 23, 2008
2
The Demand for Data
1. Performance-based management has created pressures on developing countries to improve the quantity and quality of their macro and micro-data:
Is the public sector delivering good services?
Are country policies/poverty reduction strategies reducing poverty?
Is aid supporting poverty reduction?
In the World Bank: “Results-based” CASs, for example
3
The Demand for Data
1. Performance-based management2. Millennium Development Goals (MDGs)
reflects the strong commitment on the part of the international development community and national governments to monitor and evaluate the results of their policies and programs (United Nations, 2000).
4
The Demand for Data
1. Performance-based management2. MDGs
3. HIPC, Poverty Reduction Strategies (PRSPs)
The HPIC (debt reduction initiative for heavily-indebted, low-income countries) requires
a plan for poverty reduction that can be measured and monitored access to concessional lending from the World
Bank and International Monetary Fund (IMF/IDA, 1999a and 1999b).
5
The Demand for Data
Much of the increased demand for data has been focused on household-level data. It is not possible to monitor PRSPs or MDGS
without solid household data Measurement of welfare and other key social
indicators. (Muñoz and Scott, 2005). Decisions on appropriate policies to reach the
MDGs or the targets in the PRSPs need household data.
Although they do not exclusively need/require only household-level data, although they often imply survey data
6
What these demands look like
PRSPMeasure welfare/poverty Identify problems--magnitude, causesAlternative policiesCost/benefitMonitorEvaluate
7
What these demands look like
PRSP MDGs
MDG 1: Eradicate extreme poverty and hungerMDG 2: Achieve universal primary educationMDG 3: Promote gender awareness, empower womenMDG 4: Reduce child mortalityMDG 5: Improve maternal healthMDG 6: Combat HIV/AIDS, malaria and othersMDG 7: Ensure environmental sustainabilityMDG 8: Develop a global partnership for development
8
What these demands look like
PRSP MDGs General Demand
Poverty and Inequality Benefit Incidence Analysis Public services Determinants of observed outcomes Assessment of alternative policies Impact Evaluation Inputs to Program Design
9
Recent Efforts to Increase Data: Quantity and Quality
Partnership in Statistics for Development in the 21st Century (PARIS21) was initiated to “…to act as a catalyst for promoting a culture of evidence-based policymaking and monitoring in all countries, and especially in developing countries.” (PARIS21 web site). 1999
Trust Fund for Statistical Capacity Building (TFSCB), managed by the World Bank, set up in 2000 to help build the capacity of statistical systems/ statistical plans
10
Recent Efforts to Increase Data: Quantity and Quality
World Bank line of credit for Statistical Capacity Building to support countries in the implementation of these statistical master plans.
The Program to Improve Surveys of Living Conditions in Latin America (MECOVI), (co-sponsored by the IADB, ECLAC, WB, regional program to improve quality and data
SPARC- Eastern Carribean Initative to improve surveys, UNDP, CDB, WB IADB, OECS, inter alia. (2004)
LSMS Phase IV: Methodological Research on measurement, field work and technological advances
11
Focus of Presentation
Assessing Information Needs Sources of data Need for multiple sources of data Role of household surveys
12
Assessing Information Needs
Inputs Financial & physical indicators of inputs (monthly)
Internal Outputs Achievement/performance
indicators (annually)
Outcomes Benefits/usage indicators
(annually)
External
Impact Indicators of improvements in living standards (~ 5 years)
13
Data Sources National accounts Current public expenditure statistics Program of Price collection (cons./prod.) Administrative Records (from line
ministries) Qualitative Work Surveys:
Multi-topic w/ welfare focus (LSMS/IS) Monitoring (CWIQ, PS) Income and Expenditure (IES, HBS) Single topic (Labor Force Surveys (LFS) Demographic and Health (DHS)) Enterprise Facilities
14
Surveys and Policy Analysis
Social GoalsSocial Goals
Increase enrollment
Increase female LFP
Lower infant mortality
Gov’t ProgramsGov’t Programs
Conditional Cash Transfers
Day care centers
Public Health Campaign
Households
Individuals
Firms
Surveys: Going Beyond Rates
Understanding secondary school enrollments, 12-18 year olds, Albania 2002
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AveragePercent
• In almost all countries we have a single statistic: mean enrollment at the national level. In this case it is 61%.
•This is interesting for monitoring purposes, but it doesn’t say much about poverty or other factors.
•... A regional disaggregation would be useful
Understanding secondary school enrollments, 12-18 year olds,
Albania 2002
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AveragePercent
• In some countries we have regional breakdowns, with marked contrasts
•The contrast between urban and rural rates emphasizes the disadvantages faced by rural communities.
• Other breakdowns would be useful
Urban
Rural
Understanding secondary school enrollments, 12-18 year olds,
Albania 2002
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AveragePercent
• …possibly, official statistics can add the gender dimension
•…the figures show that, in urban areas, there is no gender differential but a large gap in rural areas.
•But we still don’t know much about who sends their children to school
Urban
RuralMale
Female
Male
Female
Understanding secondary school enrollments, 12-18 year olds,
Albania 2002
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Q1 Q2 Q3 Q4 Q5
Percent
•…With a survey we can show enrollment rates broken down by consumption level--and thus understand an additional dimension
Consumption quintile
Female, urban
Male, urban
Male, rural
Female, rural
Average
19
Gathering information through surveys (household or other)
There is a range of options They can be ordered along two main
dimensions: degree of representativeness subjective/objective dimension
Casestudy
Purposive selection
Quotasampling
Small prob.sample
Large prob.sample
Census
Degree of Representativeness
Direct measurement
Questionnaire(quantitative)
Questionnaire(Qualitative)
Structuredinterview
Openmeetings
Subjective assessments
Conversations
Casestudy
Purposive selection
Quotasampling
Small prob.sample
Large prob.sample
Census
Subjective/Objective Dimension
Direct measurement
Questionnaire(quantitative)
Questionnaire(Qualitative)
Structuredinterview
Openmeetings
Subjective assessments
Conversations
Casestudy
Purposive selection
Quotasampling
Small prob.sample
Large prob.sample
Census
Wonderful World of Surveys
Windscreen Survey
Participant observation
Beneficiary Assessment
Participatory Poverty Assessments Sentinel Site
Surveillance
Household Budget Survey
Census
CWIQ/PSLSMS/ IS
Community Surveys
Direct measurement
Questionnaire(quantitative)
Questionnaire(Qualitative)
Structuredinterview
Openmeetings
Subjective assessments
Conversations
Casestudy
Purposive selection
Quotasampling
Small prob.sample
Large prob.sample
Census
Wonderful World of Surveys: “Statistical Surveys”
Household Budget Survey
Census
CWIQ/PSLSMS/ IS
24
Features of a Statistical Survey
Structured Questionnaire Random/Probability Sample
25
Tradeoffs to Consider When Planning a Survey as Part of a System of Surveys
Overall scope Single vs. Multi-topic Probability vs. Purposive Sampling Sampling vs. Non-Sampling Errors Time vs. Cost Data vs. Capacity Building Surveys over time: repeated cross
sections, panels, rotating
26
Summary
Surveys are one source of information among many (system of information) Consider all the key elements of a National
Poverty Monitoring System
27
Key Elements of a National Poverty Monitoring System
Timely Annual National Accounts Current Public Expenditure Statistics Program of Consumer and Producer Price
Statistics In-depth Welfare Survey (LSMS/IS, IES?) ‘Light’ Annual Monitoring Surveys (CWIQ, PS) Longitudinal Studies Qualitative work on key topics Specific tools for project/program/policy
monitoring and evaluation
28
Summary
Surveys are one source of information among many (system of information)
No one survey can meet all data needs: System of Household Surveys
29
System of Household Surveys Goal: System able to respond to
evolving needs: not produce data X or survey YDetermine data needs before they are
URGENT Identify appropriate instruments, Implement them properly, timely
fashion,Analyze the resulting data
30
Improving the SHS Linking Users and Producers Providing adequate resources Continuous Survey Program
Not necessarily permanent survey Benefits
Avoid loss of capacity Create greater levels of capacity (building on
existing) Economies of scale Policy makers know when data will be available Protects NSO from pressures for ad hoc surveys Ongoing system actually allows more flexibility
and responsiveness
31
Summary
Expanding demand for timely, relevant data
Need to determine the range of data needs to begin to define a system of information
Surveys are one, important, source of information among many
No one survey can meet all data needs: System of Household Surveys
32
References
United Nations (2000). “United Nations Millennium Declaration.” United Nations’ General Assembly, Fifty-fifth Session, New York, New York.
International Monetary Fund and International Development Association (1999a). “Building Poverty Reduction Strategies in Developing Countries.” Report to the Board of Directors, International Monetary Fund and International Development Association, Washington, D.C.
International Monetary Fund and International Development Association (1999b). “Heavily Indebted Poor Countries (HIPC) Initiative: Strengthening the Link between Debt Relief and Poverty Reduction.” International Monetary Fund and International Development Association, Washington, D.C.
Muñoz, Juan and Kinnon Scott (2005). “Household Surveys and the Millennium Development Goals.” Paris21, processed.
33
References
DECRG, (2006) “LSMS IV: Research for Improving Survey Data”, processed
LSMS Web page: http://www.worldbank.org/lsms/
ISLC/ MECOVI web site: http://worldbank.org/lac
then search on MECOVI