Measuring and analyzing decision-making in

development research:

An Introduction

Workshop Session #1, Part 1

Peterman, de la O Campos and Olney

December 2, 2014: Bioversity, Rome

Empowerment and decision-making: The premise

Women’s empowerment seen as a development goal in itself (increased

gender equity) – as well as a means to achieving other favorable poverty-

related outcomes.

Many program designs and development outcomes have been shown to

depend on women’s ability to negotiate favorable allocations of resources

within the household (Doss 2013).

Thus, women’s empowerment measures are important to measure and

analyze as:

① Program impacts/outcome measures (endline or change over program

period)

② Program moderators/facilitators of program impacts (baseline or initial

conditions/heterogeneous effects).

Why decision-making?

Malhotra and colleagues (2002) review women’s empowerment indicators in 45 studies, which they characterize as “vast and interconnected” – across domains (socio-cultural, economic, familial, legal, political, psychological) at different levels of aggregation (individual, household, community, institutional).

The most commonly used definition of women’s individual empowerment use some version of Kabeer’s (2001) description of “the expansion in people’s ability to make strategic life choices in a context where this ability was previously denied to them.”

Decision-making and autonomy indicators are preferred by many researchers because they represent direct measures of empowerment – rather than indirect (proxy measures: e.g. education, earnings, age).

Standard decision-making questions in

quantitative household surveys

“Who in your household usually has the final say” …

Own health

Own earnings

Children’s health

Children’s education

Small daily household (food) purchases

Large household (asset) purchases

Use of family planning

Collected in the Demographic and Health Surveys and other large multi-topic surveys

Typically asked only to women

Standard response options

Respondent herself

Her partner

Respondent and partner jointly

Respondent and others in the

household jointly

Others in the household

Could also enter one or more

IDs of household members

Variations on the ‘standard module’

Specificity in domains –aimed at programmatic or culturally specific

important areas of agency and decision-making (Burkina Faso):

“Can you make the decision to purchase… toiletries such as soap and

toothpaste? Special foods for your children?”

Asking men or other household members (WEAI)

Interactive/qualitative assessments of decision-making using ranking

tools (Rwanda)

Changes in questionnaire wording/design:

“In an ideal situation, who in your household would make the decision?”

(Ecuador, Yemen).

To what extent do you feel you can make your own personal decisions

regarding [livestock raising] if you want(ed) to? (WEAI 2.0)

Decision-making indicator construction and

analysis

Indicator construction:

Sole decision-making

Sole and joint decision-making

Summation and individual questions versus factor analysis

Treatment of not applicable domains

Gap’s in decision-making (if collecting both men and women)

Analysis:

Multivariate frameworks controlling for household

demographics (impact analysis)

Interaction terms and stratification (heterogeneous effects)

How well are we doing?

Despite a limited number of qualitative studies examining validation

of questions, there is scant quantitative research examining robustness

of quantitative measures both in survey design and analysis of

indicator construction. Many studies still conflate status (static) with

empowerment (process) (Heckert and Fabric 2013).

In many cases, although there is evidence that favorable outcomes are

associated with bargaining power, empirical evidence cannot

rigorously identify causality due to study design and data limitations --

it is therefore difficult to identify specific policies that increase

women’s bargaining power in development settings (Doss 2013).

Review of programming and impacts on women’s

empowerment (van den Bold and colleagues (2013))

Type of intervention Quantitative

evidence

Qualitative

evidence

1 Conditional Cash Transfers (CCTs) Mixed +

2 Unconditional Cash Transfers (UCTs) Mixed More evidence

needed

3 Microfinance Mixed Mixed

4 Agricultural interventions Mixed/More

evidence needed

Mixed/More

evidence needed

“Hence, while many development initiatives seem to target women specifically, or have

women’s empowerment as one of their objectives, no sufficient body of evidence

overwhelmingly points to success in terms of improving women’s empowerment, or

improving nutrition through women’s empowerment (pp. 29)”

Session objectives and organization

Review the measurement of decision-making indicators and

their use in nutrition and agriculture research (Amber)

Case studies:

① Quantitative evaluation in Burkina Faso (Deanna)

② Qualitative evaluation in Rwanda (Ana Paula)

Discuss strengths and weakness of standard quantitative

decision making indicators with application to Ecuador, Uganda

and Yemen (Amber)

Application to ongoing program evaluation work

Works cited

Doss C. 2013. Intrahousehold Bargaining and Resource Allocation in Developing Countries. World Bank Research Observer, 28(1).

Heckert J and MS Fabric. 2013. Improving Data Concerning Women’s Empowerment in Sub-Saharan Africa. Studies in Family Planning, 44(3): 319-344.

Kabeer N. 2001. “Reflections on the measurement of women’s empowerment.” In Discussing Women’s Empowerment – Theory and Practice. Sida Studies No. 3. Novum Grafiska AB: Stockholm.

Malhortra A, Schuler SR and C Boender. 2002. Measuring Women’s Empowerment as a Variable in International Development. Background paper prepared for the World Bank Workshop on Poverty and Gender: New Perspectives.

van den Bold M, Quisumbing A and S Gillespie. 2013. Women’s Empowerment and Nutrition: An Evidence Review. International Food Policy Research Institute (IFPRI) Discussion Paper #01294. Washington DC.

Indicator choice and survey design

experiments from cash and food transfer evaluations in

Ecuador, Uganda and Yemen

Workshop Session #1, Part 2

Peterman, Schwab, Roy, Hidrobo & Gilligan (in progress)

IFPRI – Poverty, Health & Nutrition Division

December 2, 2014: Bioversity, Rome

Review of literature: Transfers and decision-making

Review of programs aimed at empowering women and linkages with

nutrition found mixed and thin evidence base for quantitative

measures of CCTs/UCTs impacts on empowerment – however more

promising evidence from qualitative studies (van den Bold et al. 2013).

Progresa: Qualitatively showed empowerment of women, however

quantitatively only 1 domain out of 5 (spending of own income) (Adato et al. 2000; Handa et al. 2009).

Bolsa Familia: Quantitatively showed impacts on 1 out of 8 domains

(contraceptive use) – concentrated in urban areas (de Brauw et al. 2013).

Zambia Child Grant Program: No quantitative impacts across 9

domains (AIR 2013).

Kenya Hunger Safety Net Program: Mixed quantitative and

qualitative findings (OPM and IDS 2012).

Objectives: Transfers and decision-making in cross-

country transfer evaluations

① Whether relative rankings of decision-making are sensitive to

differences in indicator construction and survey design

experiments.

② Tests correlation between the various decision-making indices

and other proxy (indirect) measures of women’s status

(women’s education and age) or development outcomes

(household dietary diversity and food consumption).

③ Whether the transfer programs had a measurable impact on

decision-making outcomes for women.

Introduction: WFP partnership and cross country study

While the conceptual issues underpinning these issues are well-understood, there is little rigorous evidence on the relative impacts of different modalities

Partnership with the World Food Programme (WFP) to provide new evidence on impacts and cost effectiveness of alternative modalities in Bangladesh, Ecuador, Niger, Yemen and Uganda.

From 2008 to 2011, number of WFP programs using alternative modalities increased 10-fold from 5 projects in 2008 to 51 in 2011 ($208 million in programming).

Target for 2012 was nearly 1/3 of operations in cash, vouchers and “digital foods.”

The Intervention: Context in Ecuador (Hidrobo et al. 2014).

Cash, food, food voucher

transfers

Colombian refugees and poor

Ecuadorians

7 urban centers in the Northern

provinces of Carchi (highland)

and Sucumbíos (lowland)

6 monthly transfers of $40

Targeted towards women (76%)

Conditional on nutrition training

The Intervention: Context in Yemen (Schwab 2013)

Cash and food transfers

Unconditional seasonal safety net

Rural areas of Hajjah and Ibb

Governorates

Bi-monthly transfers for 6

months equal to $25 per month

Targeted towards head of

household

The Intervention: Context in Uganda (Gilligan and Roy 2013)

Cash and food transfers

Targeted to households with child

aged 3 – 5 attending UNICEF

sponsored ECD center

Three rural districts in

Karamojong sub-region (NE)

$12 every 6 weeks, for 12

months

Targeted towards primary

caregiver/mother of child

Overview: Evaluation designs

Ecuador: 80 neighborhoods and 145 clusters in 2 stage randomization.

Neighborhoods randomly assigned to treatment or comparison, thereafter,

clusters within treatment neighborhoods randomized to food, cash or

voucher (N = 2,357 households).

Yemen: 136 food distribution points randomized food or cash. Comparison

household drawn from same areas who were just above the proxy means cut

point for program qualification (N = 3,540 households).

Uganda: 98 ECD centers randomized cash, food or comparison (N = 2980

households).

All countries: Before and after household level surveys, additional

facility/community and biomarkers vary by country. Decision-making

modules to one woman per household, interviewed in private.

Context: Gender and Development

Human Development Index (2012): Ecuador (89), Uganda (161) and

Yemen (160) out of 189 ranked countries.

Gender Inequality Index: Ecuador (83), Uganda (110) and Yemen (148) out

of 148 ranked countries.

Ecuador: Although equitable frameworks (inheritance, asset ownership etc.)

exist, gender-based violence is high (35% lifetime physical partner violence)

and culture of machismo.

Yemen: Entrenched gender discrimination, no legal age at marriage, no law

criminalizing spousal rape, restrictions on women’s movement without male

guardians.

Uganda: Recent progress in legal status of women, however gaps remain in

implementation. Gender-based violence is high (56% lifetime partner

violence) and age at first marriage is 17.9 years (2011 UDHS).

Data: Decision-making indicators

Standard measures: Sole, sole and joint

Underlying threat points: Who makes the decision, or who

would make the decision in the case of a disagreement or

dispute?

Division of tasks/preferences: Who would ideally make the

decision?

Social desirability bias: “There are many women in Uganda who are

able to exert control over decisions in their household and can influence

important aspects of their lives.”

Data: Women’s status and Household-level well-being

Women level:

Age

Education (in years)

Household level:

Dietary Diversity Index (DDI): Number of unique foods consumed in

the last 7 days (1-47)

Value of per capita monthly food consumption (includes food

consumed inside and outside of the household)

Total per capita monthly consumption (food and non-food)

Empirical Specification

0.54

0.72

0.4

0.46

0.61

0.34

0.44 0.47

0.42 0.44 0.4

0.37

0.51

0.67

0.23

0.34

0.56

0.19

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Own work for pay Own health Child's education Child's health Daily food

purchases

Large asset

purchases

Percentage of women's reported sole decision-making

accross domains

Ecuador

Yemen

Uganda

Women report making:

4.5 (out of 9) sole decisions in Ecuador (50%)

2.5 (out of 6) sole decisions in Uganda and Yemen (42%)

Highest: Own health, daily food, purchases

Lowest: Child’s education and large asset purchases

4.47

7.52

6

5.31

9

2.54 3.1

2.43 2.28

6

2.51

4.41

6

0

1

2

3

4

5

6

7

8

9

10

Sole Sole or joint Sole after

disagreement

Ideal decisionmaking Total possible

Comparison of women's decision-making indicators

Ecuador

Yemen

Uganda

Including jointness increases decision-making in all countries (more in Ecuador and

Uganda)

Women report higher decision-making in Ecuador after disagreement, lower in

Yemen

Ideal decision-making is not markedly higher than actual in Ecuador and Yemen

Relative rankings of decision-making using factor analysis

Ecuador (N = 1,174) Sole Sole or joint Ideal

After

disagreement

Sole 1.00

Sole or joint 0.31 1.00

Ideal 0.52 0.47 1.00

After disagreement 0.65 0.36 0.43 1.00

alpha statistic 0.91 0.86 0.89 0.90

Households in Ecuador and Uganda show low correlation between different

constructions of indicators (majority do not exceed 0.50)

Households in Yemen show higher correlations, however differences still exist (0.74 –

0.89)

Relative rankings of households may differ based on how questions are asked

Results: Associations and impacts

Associations with women’s status and household well-being

Age: Positive association

Education (in years): No association

Dietary Diversity Index (DDI): Mixed

Value of per capita monthly food consumption: Positive

association/mixed

Total per capita monthly consumption: Positive association/mixed

Impacts of transfers on decision-making

No impacts of transfer or transfer type in Ecuador or Uganda

Impacts driven by food transfers in Yemen

No differential rankings by social desirability introduction in Uganda

Summary: What do we know, what can we do better?

Phrasing matters: Explore the wording of questions which most reflect local

perceptions of how decisions are made, as well as program goals—particularly in local

languages, which often have limited vocabulary for nuances. Formative research is

particularly helpful for both these points when studying a new context with different

cultural and gender norms.

Ask about the right domains: Specific to the level of influence one might expect the

program to change or depend on for leveraging benefits – specific to context.

Analysis: Pay attention to response options which reflect possible decision-making

arrangements, and think through how indicators will be constructed. We should not

simply assume that sole decision-making is preferable to joint decision-making,

depending on household structure and power dimensions within the household.

Qualitative work: Triangulation of evidence may be necessary to uncover certain

domains of empowerment, not able to be captured with quantitative survey methods.

More research!: We need to continue advancing the frontier of how to most

accurately capture and analyze decision-making and women’s empowerment.

Acknowledgements

In country partners for data collection and survey management: Centro de

Estudios de Población y Desarrollo Social (CEPAR), Yemen Polling Company and

Makerere University.

IFPRI colleagues including John Hoddinott, Nancy Johnson, Amy Margolies,

Hazel Malprit, Vanessa Moreira and Agnes Quisumbing for helpful

discussions at study conception and contributions through work on the larger

food and cash transfer evaluation. Caroline Guiriec for assistance in

administration of the grant.

WFP (Rome, Quito, Kampala and Sana’a) for excellent collaboration and

program implementation.

Funding from the Government of Spain for the impact evaluations and to

the CGIAR Research Program on Agriculture for Nutrition and Health

(ANH) led by IFPRI for the analysis and writing of this paper.

Works cited

American Institutes for Research (AIR) 2013. Zambia’s Child Grant Program: 24-month impact report. Washington, DC: AIR.

de Brauw A, Gilligan DO, Hoddinott J and S Roy. 2013. The Impact of Bolsa Familia on Women’s Decision Making Power. World Development, 59: 487-504.

Gilligan DO and S Roy. 2013. Resources, stimulation and cognition: How transfer programs and preschool shape cognitive development in Uganda. Agricultural & Applied Economics Association’s 2013 AAEA & CAES Joint Annual Meeting: Washington DC.

Handa S, Peterman A, Davis B and M Stampini. 2009. Opening up Pandora’s Box: The effect of Gender Targeting and Conditionality on Household Spending Behavior in Mexico’s ProgresaProgram. World Development 37(6): 1129-1142.

Hidrobo M, Hoddinott J, Peterman A, Margolies A, and V Moreira. 2014. Cash, food, or vouchers? Evidence from a randomized experiment in northern Ecuador. Journal of Development Economics, 107: 144-156.

Oxford Policy Management (OPM) and Institute of Development Studies (IDS). 2012. Kenya Hunger Safety Net Programme: Monitoring and Evaluation Component – Impact Analysis Synthesis Report. Oxford, UK: Oxford Policy Management: Brighton, UK: IDS.

Schwab B. 2013. In the form of bread? A randomized comparison of cash and food transfers in Yemen. Paper presented at the Agricultural & Applied Economics Association’s 2013 AAEA & CAES Joint Annual Meeting: Washington DC.