Causality, RCTs and Impact Evaluations · Et cetera. Peter van der Windt Causality, RCTs and Impact...
Transcript of Causality, RCTs and Impact Evaluations · Et cetera. Peter van der Windt Causality, RCTs and Impact...
Causal Inference and RCTsMeasurement
Practical Things
Causality, RCTs and Impact Evaluations
Peter van der Windt
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
We often want to learn about causality:
Does X cause Y?What causes Y?What is the impact of X?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
We often want to learn about causality:
Does X cause Y?What causes Y?What is the impact of X?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
We often want to learn about causality:
Does X cause Y?
What causes Y?What is the impact of X?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
We often want to learn about causality:
Does X cause Y?What causes Y?
What is the impact of X?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
We often want to learn about causality:
Does X cause Y?What causes Y?What is the impact of X?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty? Endogeneity? (Conflict ↔ Poverty)Education → Salary? Other variable? (E.g. Ability↑ → Education↑ andSalary↑)UN → Security? Selection? E.g. UN selects insecure areas to operate.Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty?
Endogeneity? (Conflict ↔ Poverty)Education → Salary? Other variable? (E.g. Ability↑ → Education↑ andSalary↑)UN → Security? Selection? E.g. UN selects insecure areas to operate.Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty? Endogeneity? (Conflict ↔ Poverty)
Education → Salary? Other variable? (E.g. Ability↑ → Education↑ andSalary↑)UN → Security? Selection? E.g. UN selects insecure areas to operate.Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty? Endogeneity? (Conflict ↔ Poverty)Education → Salary?
Other variable? (E.g. Ability↑ → Education↑ andSalary↑)UN → Security? Selection? E.g. UN selects insecure areas to operate.Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty? Endogeneity? (Conflict ↔ Poverty)Education → Salary? Other variable? (E.g. Ability↑ → Education↑ andSalary↑)
UN → Security? Selection? E.g. UN selects insecure areas to operate.Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty? Endogeneity? (Conflict ↔ Poverty)Education → Salary? Other variable? (E.g. Ability↑ → Education↑ andSalary↑)UN → Security?
Selection? E.g. UN selects insecure areas to operate.Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty? Endogeneity? (Conflict ↔ Poverty)Education → Salary? Other variable? (E.g. Ability↑ → Education↑ andSalary↑)UN → Security? Selection? E.g. UN selects insecure areas to operate.
Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
But this is not that easy because correlation is not causality.
Conflict → Poverty? Endogeneity? (Conflict ↔ Poverty)Education → Salary? Other variable? (E.g. Ability↑ → Education↑ andSalary↑)UN → Security? Selection? E.g. UN selects insecure areas to operate.Et cetera.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison: X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison: X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison: X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison:
X
Cross-sectional comparison: X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison: X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison:
X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison: X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison: X
RCT:
V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
The Fundamental Problem of Causal Inference
An impact evaluation is all about comparison:
What would have happened if there was no project?
Fundamental Problem of Causal Inference
Temporal comparison: X
Cross-sectional comparison: X
RCT: V
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs
Randomized Control Trail (RCT)
Used since decades by medicines
Randomly assign treatment to the units
If you toss a coin 10,000 times: How many times do you have “headsup”?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs
Randomized Control Trail (RCT)
Used since decades by medicines
Randomly assign treatment to the units
If you toss a coin 10,000 times: How many times do you have “headsup”?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs
Randomized Control Trail (RCT)
Used since decades by medicines
Randomly assign treatment to the units
If you toss a coin 10,000 times: How many times do you have “headsup”?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs
Randomized Control Trail (RCT)
Used since decades by medicines
Randomly assign treatment to the units
If you toss a coin 10,000 times: How many times do you have “headsup”?
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs
ALL UNITS
YES Project
= treatment
NO project
= control
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?
Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Power
External validity
Costs
Timing
Ethically correct?
Some units do NOT receive the treatment!?Assignment is random!?
Spillovers
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
Before the project:
4
4
Yes project No project
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
After the project and NO spillovers:
4
4
Yes project No project
3
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
After the project and NO spillovers:
Project Impact = 7 – 4 = 3
4
4
Yes project No project
3
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
After the project and YES spillovers:
4
4
Yes project No project
3 2
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
RCTs and Limitations
After the project and YES spillovers:
Project Impact = 7 – 6 = 1
4
4
Yes project No project
3 2
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Options Different than RCTs
Statistical controls
Matching
Difference-In-Difference
Natural experiment. E.g. Radio
Instrumental variable. E.g. Rainfal
Regression Discontinuity
Process tracing
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Options Different than RCTs
Statistical controls
Matching
Difference-In-Difference
Natural experiment. E.g. Radio
Instrumental variable. E.g. Rainfal
Regression Discontinuity
Process tracing
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Options Different than RCTs
Statistical controls
Matching
Difference-In-Difference
Natural experiment. E.g. Radio
Instrumental variable. E.g. Rainfal
Regression Discontinuity
Process tracing
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Options Different than RCTs
Statistical controls
Matching
Difference-In-Difference
Natural experiment. E.g. Radio
Instrumental variable. E.g. Rainfal
Regression Discontinuity
Process tracing
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Options Different than RCTs
Statistical controls
Matching
Difference-In-Difference
Natural experiment. E.g. Radio
Instrumental variable. E.g. Rainfal
Regression Discontinuity
Process tracing
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Options Different than RCTs
Statistical controls
Matching
Difference-In-Difference
Natural experiment. E.g. Radio
Instrumental variable. E.g. Rainfal
Regression Discontinuity
Process tracing
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Options Different than RCTs
Statistical controls
Matching
Difference-In-Difference
Natural experiment. E.g. Radio
Instrumental variable. E.g. Rainfal
Regression Discontinuity
Process tracing
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Variation in Treatment
YES Sugar NO Sugar
YES Milk 20 0
NO Milk 20 0
Conclusion: Only one treatment part (Sugar) works.
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Variation in Treatment
YES Sugar NO Sugar
YES Milk 20 1
NO Milk 2 0
Conclusion: Complementary between treatment parts.
(I.e. They only work well together.)
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
The Fundamental Problem of Causal InferenceRCTsRCTs and LimitationsOptions Different than RCTsVariation in Treatment
Variation in Treatment
YES Sugar NO Sugar
YES Milk 20 20
NO Milk 20 0
Conclusion: Substitution between treatment parts.
(I.e. It’s either one or the other.)
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)
And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Behavioral Measures
Projects often try to impact behavior:
“Governance”, “Social Cohesion”, “Behavior by soldiers”, Etc.
Problem:
People act strategically (they lie)And they parrot
Solution: Behavioral measure
Make it costly for people to act differently than the truth.
Example: Social Cohesion in Liberia
Example: Good governance in Congo. RAPID & $900
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
Does your husband beat you?
Have you ever been raped?
Problem:
People act strategically (they lie)Safety of the respondent
How to measure such sensitive topics?
List-experimentRandom-answer method
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
Does your husband beat you? Have you ever been raped?
Problem:
People act strategically (they lie)Safety of the respondent
How to measure such sensitive topics?
List-experimentRandom-answer method
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
Does your husband beat you? Have you ever been raped?
Problem:
People act strategically (they lie)Safety of the respondent
How to measure such sensitive topics?
List-experimentRandom-answer method
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
Does your husband beat you? Have you ever been raped?
Problem:
People act strategically (they lie)
Safety of the respondent
How to measure such sensitive topics?
List-experimentRandom-answer method
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
Does your husband beat you? Have you ever been raped?
Problem:
People act strategically (they lie)Safety of the respondent
How to measure such sensitive topics?
List-experimentRandom-answer method
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
Does your husband beat you? Have you ever been raped?
Problem:
People act strategically (they lie)Safety of the respondent
How to measure such sensitive topics?
List-experimentRandom-answer method
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
Does your husband beat you? Have you ever been raped?
Problem:
People act strategically (they lie)Safety of the respondent
How to measure such sensitive topics?
List-experimentRandom-answer method
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
There are two groups:
Group A Group B
I am poor I am poorMarket is too far away Market is too far away
Too dry for good harvest Too dry for good harvestMy husband beats me
To each group ask: “How many of the following are relevant to you?”
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
There are two groups:
Group A Group B
I am poor I am poorMarket is too far away Market is too far away
Too dry for good harvest Too dry for good harvestMy husband beats me
To each group ask: “How many of the following are relevant to you?”
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
There are two groups:
Group A Group B
I am poor I am poorMarket is too far away Market is too far away
Too dry for good harvest Too dry for good harvestMy husband beats me
To each group ask: “How many of the following are relevant to you?”
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Behavioral MeasuresSensitive Questions
Sensitive Questions
A. I am beaten by
my husband
B. I am not beaten
by my husband
Select the answer:
Correct / Incorrect
A
B
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Practical Things
1 Use technology (e.g. PDAs)
2 Training is crucial
3 Pre-analysis plan
4 Use PhDs
5 Academics & policymakers
END
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Practical Things
1 Use technology (e.g. PDAs)
2 Training is crucial
3 Pre-analysis plan
4 Use PhDs
5 Academics & policymakers
END
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Practical Things
1 Use technology (e.g. PDAs)
2 Training is crucial
3 Pre-analysis plan
4 Use PhDs
5 Academics & policymakers
END
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Practical Things
1 Use technology (e.g. PDAs)
2 Training is crucial
3 Pre-analysis plan
4 Use PhDs
5 Academics & policymakers
END
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Practical Things
1 Use technology (e.g. PDAs)
2 Training is crucial
3 Pre-analysis plan
4 Use PhDs
5 Academics & policymakers
END
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Practical Things
1 Use technology (e.g. PDAs)
2 Training is crucial
3 Pre-analysis plan
4 Use PhDs
5 Academics & policymakers
END
Peter van der Windt Causality, RCTs and Impact Evaluations
Causal Inference and RCTsMeasurement
Practical Things
Practical Things
1 Use technology (e.g. PDAs)
2 Training is crucial
3 Pre-analysis plan
4 Use PhDs
5 Academics & policymakers
END
Peter van der Windt Causality, RCTs and Impact Evaluations