Take-Home Messages

DEAN JOLLIFFE & RAKA BANERJEE

Poverty & Inequality Group

Development Research Group

The World Bank

Multi-Topic Household Surveys March 5, 2014

LSMS Team Tools

• LSMS Website: www.worldbank.org/lsms

– Tab: “Survey Design and Analysis Tools”

• LSMS Survey Manual: Margaret E. Grosh and Juan

Munoz. 1996. "A Manual for Planning and Implementing

the LSMS Survey." World Bank Living Standards

Measurement Study, Working Paper No. 126.

• Questionnaires

• Manuals

• Documentation

• Household Surveys: – Designing Household Survey Questionnaires for Developing Countries: Lessons

from 15 years of the Living Standards Measurement Study

• Tracking: – Tracking in Longitudinal Household Surveys

• Climate Change: – Improving Household Survey Instruments for Understanding Agricultural

Household Adaptation to Climate Change: Water Stress and Variability

– Understanding Agricultural Households’ Adaptation to Climate Change and

Implications for Mitigation: Land Management and Investment Options

• Fisheries: – Design and Implementation of Fishery Modules in Integrated Household Surveys

in Developing Countries

• Computer-Assisted Personal Interviewing (CAPI) – Comparative Assessment of CAPI Software Packages

LSMS Publications

ADePT Agriculture

• Software platform for automated economic

analysis, developed to automate and

standardize the production of analytical reports

• Uses micro-level data to produce richly

detailed tables and graphs on specific topics

• LSMS team working with DECCT team to

create ADePT Agriculture, focused on

agricultural indicators

Comparative Living

Standards Project (CLSP)

• Database of harmonized variables from LSMS surveys to facilitate comparative research

• Platform on which users can conduct online analysis; results downloadable in Excel files

• Standard indicators available for basic analyses; option to create new variables by combining harmonized variables

• Analytical capability ranges from frequencies to regression analysis

Methodological Work

LSMS-IV • Finance

• Consumption

• Labor

• Income

• Migration

• Subjective welfare

• Gifting

LSMS-ISA • Land Area

• Soil Fertility

• Crop Production

• Agricultural Labor

• Skill Testing

• Sampling for Nomads

• Livestock Production/

Pastoralists

• Methodological survey experiments to inform questionnaire

and fieldwork design in key domains of data collection

• Hands-on training in-country

• Training courses and materials – Live, web

– Household Survey Clinics

– CMAAE/AERC

• South-South collaboration

• Institutional collaborations within LSMS-ISA – WFP, FAO, IFAD, IFPRI, WFC, ILRI, CGIAR

• Host/participate in workshops

• Present findings at conferences

Training and Dissemination

KEY MESSAGES

Leave Sampling Design

to the Experts

• A survey is only as good as its sample

• Good sampling design is crucial to ensure that: – All key population segments are accounted for

– The survey is representative at the required levels of analysis

– Field work can be implemented properly

– The survey is as precise as possible while maintaining quality data collection

– Account for field constraints • Ex-ante

• Ex-post

Design Data collection

Analysis

Create a Reasonable and

Flexible Schedule

Activity groups 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

1 Logistical planning

2 Questionnaire development

3 Sampling

4 Staffing and training

5 Data entry platform management

6 Data entry

7 Field work

8 Data analysis

9 Documentation

Design Data collection

Analysis

Key Activities

Activity groups 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

1 Logistical planning

2 Questionnaire development

3 Sampling

4 Staffing and training

5 Data entry platform management

6 Data entry

7 Field work

8 Data analysis

9 Documentation

Nominate core team

Field test Training Dissemination

Assess and Allocate Staffing

Needs Appropriately

Key Skill Sets:

• Knowledge of Policy Issues / Local Context

• Questionnaire Design

• Sampling and Statistics

• Data Entry

• Procurement and Logistics

• Training

• Data Checking / Analysis

Key Tasks: Logistics: form local steering committee / data users group, advertising and hiring of staff, training, personnel management, cash flow Technical: sample design, questionnaire design, pilot testing, material preparation, data entry development/customization, documentation/dissemination Supervision: Data quality checks, adherence to TOR, “big picture” issues

Optimal Staff Allocation

Tasks

Logistics

Technical Assistance

Supervision

Resources

Local Team

Consultants /

Experts

Core Team

Example of Possible

Team Structure Local Team

• Collaboration on questionnaire, drafting of training materials,

fieldwork logistics, personnel and data management

Experts

• Sampler

• Data Entry Development

Consultants

• Long term in-country field coordinator to assist local team

• Short term for data cleaning and documentation

Core Team

• Drafting of questionnaire

• Finalization of all materials, administrative support, supervision

Surveys Cannot be Outsourced:

Core Team Must Be Involved

Minimum Set of Core Team Tasks

• Initial Identification / Contracting 3 weeks

• Instrument Design 5 weeks

• Administrative Support 3 weeks

• Document Review 2 weeks

• Review of Pilot Results 2 weeks

• Analysis 4 weeks

Minimum Total 19 weeks

395,300

114,400

313,330

16,500

236,500

147,000

1,223,030

122,303

1,345,333

Money can’t buy happiness … but can help funding a good survey!

Generic Budget:

For a 1 year, 3,200 household LSMS (all-inclusive)

1 Base salaries

2 Travel allowance

3 Materials

4 Printing and copying

5 Consultancy and travel

6 Other

Sub-Total

Contingency (10%)

Total

Keep in Mind when

Budgeting for Surveys …

• It is often difficult to compare costs since reported budgets don’t always include some important costs:

– Core staff time and travel

– Technical assistance (STCs, sampling expert)

• Sometimes budgets include costs not related to data collection itself

– Analysis

– Study tours for NSO staff

• Sometimes budgets include capital equipment which has value after the survey (computers, vehicles)

Consider Implications of

Funding Sources

• Implications of funding – Administration (partner might place constraints on use of

funds)

– Content (partners might want topics detrimental or unfeasible)

– Time (now subject to someone else’s deadlines as well)

– Credibility (partner might help or hurt credibility)

• Sources – Government

– UN Agencies

– Development Banks

– Bilateral Organizations

– Multilateral Organizations

– Philanthropic Organizations

– And more!

Document, document,

document!!

• Basic Information Document - Describe methods

- “Publicize” main stakeholders

- Force to look at/understand data

- Sample information and weights

- Field work procedures

- Industry/occupation codes

- Capacity building/training tool

• Questionnaires

• Manuals

• Other information (i.e. photo aids, etc.)

Archive and Disseminate …and

talk to Diane! Benefits of Archiving

– Save time • Once the system for data

archiving is in place, it can be duplicated for other data collection efforts

– Simplify life • Can focus on the work rather

than data requests

– Preserve data • Assure that the data are available

over the long term

– Increase efficiency • Will always know what is in the

data regardless of staff changes

– Facilitate new discoveries • Researchers can use the data to

do unanticipated studies in the future

Benefits of Dissemination – Fully exploit the data

– Increase impact of analysis

– Transparency increases credibility

– Increase the visibility of work

• Demonstrate the versatility of the organization

• Demonstrate relevance of the work

– Create positive publicity

– Make NSO more competitive for acquiring resources

– Influence changes in programs, policies and practices

Make Data Accessible:

Open Data Policies

• Agreement in advance

• Desirable Elements – Unrestricted access

– Be specific!

– Signed by high authority

– Outlines procedures for getting data

– Well publicized

– Lists courtesies requested of user (optional)

– Sets performance standard for agency (optional)

• Be ready to address concerns over: – Confidentiality

– Institutional turf

– Political sensitivity of data

– Quality of data

In conclusion …

• Sample right, schedule flexibly, staff

adequately, budget smart, document

always, disseminate forcefully

• Plan ahead but always have a fall back

option

For More Information

• Visit the LSMS-ISA website at

www.worldbank.org/lsms-isa

• Visit the LSMS website at

www.worldbank.org/lsms

• Contact the LSMS team at

lsms@worldbank.org