Econometric Evaluation of Social Programs Part I: Causal ...jenni.uchicago.edu › econ312 ›...

Introduction Policy

Econometric Evaluation of Social Programs

Part I: Causal Models, Structural Models, and

Econometric Policy Evaluation

James J. Heckman and Edward J. Vytlacil

Econ 312, Spring 2019

Heckman and Vytlacil Econometric Evaluation

Introduction Policy

Introduction

• Evaluating policy is a central problem in economics.

• This requires the economist to construct counterfactuals.

• The existing literature on “causal inference” in statistics is thesource of inspiration for the recent econometric treatmenteffect literature and we examine it in detail.


Introduction Policy

• The literature in statistics on causal inference confuses threedistinct problems that are carefully distinguished in this chapterand in the literature in economics:

(1) Definitions of counterfactuals.

(2) Identification of causal models from idealized data ofpopulation distributions (infinite samples without any samplingvariation). The hypothetical populations may be subject toselection bias, attrition and the like. However, all issues ofsampling variability are irrelevant for this problem.


Introduction Policy

(3) Identification of causal models from actual data, wheresampling variability is an issue. This analysis recognizes thedifference between empirical distributions based on sampleddata and population distributions generating the data.

• Table 1 delineates the three distinct problems.


Introduction Policy

Table 1: Three distinct tasks arising in the analysis of causal models

Task Description Requirements

1 Defining the Set of Hypotheticals A Scientific Theoryor Counterfactuals

2 Identifying Parameters Mathematical Analysis of(Causal or Otherwise) from Point or Set IdentificationHypothetical Population Data

3 Identifying Parameters from Data Estimation andTesting Theory


Introduction Policy

• A model of counterfactuals is more widely accepted the morewidely accepted are its ingredients:

(1) the rules used to derive a model, including whether or not therules of logic and mathematics are followed;

(2) its agreement with other theories; and

(3) its agreement with the evidence.

• Models are of hypothetical worlds obtained byvarying — hypothetically — the factors determining outcomes.


Introduction Policy

• The second problem is one of inference in very large samples.

• Can one recover counterfactuals (or means or distributions ofcounterfactuals) from data that are free of any samplingvariation problems?

• This is the identification problem.

• The third problem is one of inference in practice.

• Can one recover a given model or the desired counterfactualfrom a given set of data?


Introduction Policy

• Some of the controversy surrounding construction ofcounterfactuals and causal models is partly a consequence ofanalysts being unclear about these three distinct problems andoften confusing them.

• Particular methods of estimation (e.g., matching orinstrumental variable estimation) have become associated with“causal inference” and even the definition of certain “causalparameters” because issues of definition, identification, andestimation have been confused in the recent literature.

• The econometric approach to policy evaluation separates theseproblems and emphasizes the conditional nature of causalknowledge.


Introduction Policy

• Human knowledge advances by developing counterfactuals andtheoretical models and testing them against data.

• The models used are inevitably provisional and conditional on apriori assumptions.

• Blind empiricism leads nowhere.

• Economists have economic theory to draw on but recentdevelopments in the econometric treatment effect literatureoften ignore it.


Introduction Policy

• Current widely used “causal models” in epidemiology andstatistics are incomplete guides to interpreting data or forsuggesting estimators for particular problems.

• Rooted in biostatistics, they are motivated by the experimentas an ideal.

• They do not clearly specify the mechanisms determining howhypothetical counterfactuals are realized or how hypotheticalinterventions are implemented except to compare “randomized”with “nonrandomized” interventions.


Introduction Policy

• Because the mechanisms determining outcome selection are notmodeled in the statistical approach, the metaphor of “randomselection” is often adopted.

• Since randomization is used to define the parameters ofinterest, this practice sometimes leads to the confusion thatrandomization is the only way — or at least the best way — toidentify causal parameters from real data.

• In truth, this is not always so, as we demonstrate in thispresentation.


Introduction Policy

• One reason why epidemiological and statistical models areincomplete is that they do not specify the sources ofrandomness generating variability among agents.

• I.e., they do not specify why observationally identical peoplemake different choices and have different outcomes given thesame choice.

• They do not distinguish what is in the agent’s information setfrom what is in the observing statistician’s information set,although the distinction is fundamental in justifying theproperties of any estimator for solving selection and evaluationproblems.


Introduction Policy

• They are also incomplete because they are recursive.

• They do not allow for simultaneity in choices of outcomes oftreatment that are at the heart of game theory and models ofsocial interactions.


Introduction Policy

• The goal of the econometric literature, like the goal of allscience, is to model phenomena at a deeper level, to understandthe causes producing the effects so that one can use empiricalversions of the models to forecast the effects of interventionsnever previously experienced, to calculate a variety of policycounterfactuals, and to use economic theory to guide thechoices of estimators and the interpretation of the evidence.

• These activities require development of a more elaborate theorythan is envisioned in the current literature on causal inferencein epidemiology and statistics.


Introduction Policy

• The recent literature sometimes contrasts structural and causalmodels.

• The contrast is not sharp because the term “structural model”is often not precisely defined.

• There are multiple meanings for this term, which are clarified inthis presentation.


Introduction Policy

• The essential contrast between causal models and expliciteconomic models as currently formulated is in the range ofquestions that they are designed to answer.

• Causal models as formulated in statistics and in theeconometric treatment effect literature are typically black-boxdevices designed to investigate the impact of“treatment” — which are often complex packages ofinterventions — on some observed set of outcomes in a givenenvironment.


Introduction Policy

• Explicit economic models go into the black box to explore themechanism(s) producing the effects.

• In the terminology of Holland (1986), the distinction is betweenunderstanding the “effects of causes” (the goal of thetreatment effect literature) versus understanding the “causes ofeffects” (the goal of the literature building explicit economicmodels).

• By focusing on one narrow black-box question, the treatmenteffect and natural experiment literatures can avoid many of theproblems confronted in the econometrics literature that buildsexplicit economic models.

• This is its great virtue.


Introduction Policy

• At the same time, it produces parameters that are more limitedin application.

• The parameters defined by instruments or “naturalexperiments” are often hard to interpret within any economicmodel.

• Without further assumptions, these parameters do not lendthemselves to extrapolation out of sample or to accurateforecasts of impacts of policies besides the ones beingempirically investigated.


Introduction Policy

• By not being explicit about the contents of the black-box(understanding the causes of effects), it ties its hands in usinginformation about basic behavioral parameters obtained fromother studies as well as economic intuition to supplementavailable information in the data in hand.

• It lacks the ability to provide explanations for estimated“effects” grounded in economics or to conduct welfareeconomics.

• When the components of treatments vary across studies,knowledge does not accumulate across treatment effect studies,whereas it does accumulate across studies estimating commonbehavioral or technological parameters.


Introduction Policy

Economic Policy Evaluation Questions and Criteria ofInterest

• Three broad classes of policy evaluation questions areconsidered in this presentation.

• Policy evaluation question one is:

P1 Evaluating the impact of historical interventions on outcomes,including their impact in terms of welfare.

• By historical, we mean interventions actually experienced anddocumented.


Introduction Policy

• It is useful to distinguish objective or public outcomes from“subjective” outcomes.

• Objective outcomes are intrinsically ex post in nature.

• Subjective outcomes can be ex ante or ex post.

• Thus the outcome of a medical trial produces both a cure rateand the pain and suffering of the patient.

• Ex ante expected pain and suffering may be different from expost pain and suffering.

• Agents may also have ex ante evaluations of the objectiveoutcomes that may differ from their ex post evaluations.


Introduction Policy

• P1 is the problem of internal validity.

• It is the problem of identifying a given treatment parameter ora set of treatment parameters in a given environment.

• The econometric approach emphasizes valuation of theobjective outcome of the trial (e.g., health status) as well assubjective evaluation of outcomes (patient’s welfare), and thelatter may be ex post or ex ante.


Introduction Policy

• Most policy evaluation is designed with an eye toward thefuture and towards informing decisions about new policies andapplication of old policies to new environments:

P2 Forecasting the impacts (constructing counterfactual states) ofinterventions implemented in one environment in otherenvironments, including their impacts in terms of welfare.


Introduction Policy

• Included in these interventions are policies described by genericcharacteristics (e.g., tax or benefit rates) that are applied todifferent groups of people or in different time periods fromthose studied in implementations of the policies on which dataare available.

• This is the problem of external validity.


Introduction Policy

• Finally, the most ambitious problem is forecasting the effect ofa new policy, never previously experienced:

P3 Forecasting the impacts of interventions (constructingcounterfactual states associated with interventions) neverhistorically experienced to various environments, includingtheir impacts in terms of welfare.

• This problem requires that we use past history to forecast theconsequences of new policies.

• It is a fundamental problem in knowledge.


Introduction Policy

• Knight (1921, p. 313) succinctly states the problem:

The existence of a problem in knowledge depends on the futurebeing different from the past, while the possibility of a solutionof the problem depends on the future being like the past.


Introduction Policy

Economic Policy Evaluation Questions and Criteria ofInterest: Notation and Definition of Individual Level

Treatment Effects

• To evaluate is to value and to compare values among possibleoutcomes.

• These are two distinct tasks, which we distinguish in thispresentation.

• We define outcomes corresponding to state (policy, treatment)s for an agent characterized by ω as Y (s, ω), ω ∈ Ω.

• The agent can be a household, a firm, or a country.


Introduction Policy

• One can think of Ω as a universe of agents each characterizedby an element ω.

• The ω encompasses all features of agents that affect Youtcomes.

• Y (·, ·) may be generated from a scientific or economic theory.

• It may be vector valued.

• The Y (s, ω) are outcomes realized after treatments are chosen.

• In advance of treatment, agents will not know the Y (s, ω) butmay make forecasts about them.


Introduction Policy

• Let S be the set of possible treatments with elements denotedby s.

• For simplicity of exposition, we assume that this set is the samefor all ω.

• For each ω, we obtain a collection of possible outcomes givenby Y (s, ω)s∈S .

• The set S may be finite (e.g., there may be J states),countable, or may be defined on the continuum (e.g.,S = [0, 1]), so that there are an uncountable number of states.


Introduction Policy

• For example, if S = 0, 1, there are two treatments, one ofwhich may be a no-treatment state — e.g., Y (0, ω).

• This is the outcome for an agent ω not getting a treatment likea drug, schooling, or access to a new technology, while Y (1, ω)is the outcome in treatment state 1 for agent ω getting thedrug, schooling, or access.


Introduction Policy

Begin material from previous slide presentations(1).


Introduction Policy

• To focus ideas, analyze a prototypical policy evaluationproblem.

• Country can adopt a policy (e.g., democracy).

• Choice Indicator:• D = 1 if it adopts.• D = 0 if not.


Introduction Policy

• Two outcomes (Y0(ω),Y1(ω)), ω ∈ Ω

• Y0(ω) if country does not adopt• Y1(ω) if country adopts

• Causal effect on observed outcomes

• Marshallian ceteris paribus causal effect:

Y1(ω)− Y0(ω)


Introduction Policy

Figure 1: Extended Roy economy for policy adoptionDistribution of gains and treatment parameters

-5 -4 -3 -2 -1 0 1 2 3 4 50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Gain

TT=2.517

AM C=1.5* ATE=0.2

TUT=-0.595

Figure 1. Extended Roy Economy for Policy Adoption Distribution of Gains and Treatment Parameters

*C = Marginal Return

Suppose that a country has to choose whether to implement a policy. Under the policy, the GDP would be Y1.Without the policy,the GDP of the country would be Y0. For sake of simplicity, suppose that

Y1 = μ1 + U1

Y0 = μ0 + U0

where U0 and U1 are unobserved components of the aggregate output. The error terms (U0, U1) are dependent in a general way. Letδ denote the additional GDP due to the policy, i.e. δ = μ1−μ0. We assume δ > 0. Let C denote the cost of implementing the policy.We assume that the cost is a fixed parameter C. We relax this assumption below. The country’s decision can be represented as:

D =

½1 if Y1 − Y0 − C > 00 if Y1 − Y0 − C ≤ 0,

so the country decides to implement the policy (D = 1) if the net gains coming from it are positive. Therefore, we can define theprobabily of adopting the policy in terms of the propensity score

Pr(D = 1) = P (Y1 − Y0 − C > 0)

We assume that (U1, U0) ∼ N (0,Σ), Σ =∙1 −0.5−0.5 1

¸, μ0 = 0.67, δ = 0.2 and C = 1.5.

Gain=Y1 − Y0


Introduction Policy

Figure 1 Legend

Suppose that a country has to choose whether to implement a policy. Under thepolicy, the GDP would be Y1. Without the policy, the GDP of the countrywould be Y0. For the sake of simplicity, suppose that

Y1 = µ1 + U1

Y0 = µ0 + U0

where U0 and U1 are unobserved components of the aggregate output. Theerror terms (U0,U1) are dependent in a general way. Let δ denote the additionalGDP due to the policy, i.e. δ = µ1 − µ0. We assume δ > 0. Let C denote thecost of implementing the policy. We assume that the cost is a fixed parameterC .


Introduction Policy

Figure 1 Legend

We relax this assumption below. The country’s decision can berepresented as:

D =

1 if Y1 − Y0 − C > 00 if Y1 − Y0 − C ≤ 0,

so the country decides to implement the policy (D = 1) if the netgains coming from it are positive. Therefore, we can define theprobability of adopting the policy in terms of the propensity score

Pr(D = 1) = P(Y1 − Y0 − C > 0).

We assume that (U1,U0) ∼ N (0,Σ), Σ =

[1 −0.5−0.5 1

],

µ0 = 0.67, δ = 0.2, and C = 1.5.


Introduction Policy

• More generally, define outcomes corresponding to state (policy,treatment) s for an “agent” characterized by ω as Y (s, ω),ω ∈ Ω = [0, 1], s ∈ S, set of possible treatments.

• The agent can be any economic agent such as a household, afirm, or a country.

• The Y (s, ω) are ex post outcomes realized after treatments arechosen.

• Consider uncertainty and related ex ante and ex postevaluations later on.


Introduction Policy

• The individual treatment effect for agent ω.

Y (s, ω)− Y (s ′, ω) , s 6= s ′, s, s ′ ∈ S , (1)

Individual level causal effect.

• Comparisons can also be made in terms of utilities R (Y (s, ω)).

• R (Y (s, ω) , ω) > R (Y (s ′, ω) , ω) if s is preferred to s ′.

• The difference in subjective outcomes is[R (Y (s, ω) , ω)− R (Y (s ′, ω) , ω)], and is another possibledefinition of a treatment effect. Holding ω fixed holds allfeatures of the person fixed except the treatment assigned, s.


Introduction Policy

• The question,

“What question is the analysis supposed to answer?”

is the big unanswered question in the recent policy evaluationliterature.

• The question is usually unanswered because it is unasked inmuch of the modern treatment effect literature which seeks toestimate “an effect” without telling you which effect or why itis interesting to know it.

• The answer to the question shapes the way we go about policyevaluation analysis.

• A central point in the Cowles research program (Marschak,1949, 1953).


Introduction Policy

• To evaluate is to value and to compare values among possibleoutcomes.

• These are two distinct tasks, which we distinguish.

• We define outcomes corresponding to state (policy, treatment)s for an agent characterized by ω as Y (s, ω), ω ∈ Ω.

• One can think of Ω as a universe of agents each characterizedby an element ω.

• The ω encompasses all features of agents that affect Youtcomes.

• Y (·, ·) may be generated from a scientific or economic theory.


Introduction Policy

• The Y (s, ω) are outcomes realized after treatments are chosen.

• In advance of treatment, agents may not know the Y (s, ω) butmay make forecasts about them.

• These forecasts may influence their decisions to participate inthe program or may influence the agents who make decisionsabout whether or not an individual participates in the program.


Introduction Policy

• Let S be the set of possible treatments with elements denotedby s.

• For simplicity of exposition, we assume that this set is the samefor all ω.

• For each ω, we obtain a collection of possible outcomes givenby Y (s, ω)s∈S .

• The set S may be finite (e.g., there may be J states),countable, or may be defined on the continuum (e.g.,S = [0, 1]) so there are an uncountable number of states.


Introduction Policy

• For example, if S = 0, 1, there are two treatments, one ofwhich may be a no-treatment state (e.g., Y (0, ω) is theoutcome for an agent ω not getting a treatment like a drug,schooling or access to a new technology, while Y (1, ω) is theoutcome in treatment state 1 for agent ω getting the drug,schooling or access).


Introduction Policy

• Each “state” (treatment) may consist of a compound ofsubcomponent states.

• In this case, one can define s itself as a vector (e.g.,s = (s1, s2, . . . , sK ) for K components) corresponding to thedifferent components that comprise treatment.

• Thus a job training program typically consists of a package oftreatments.

• We might be interested in the package of one (or more) of itscomponents.


Introduction Policy

• The outcomes may be time subscripted as well, Yt (s, ω)corresponding to outcomes of treatment measured at differenttimes.

• The index set for t may be the integers, corresponding todiscrete time, or an interval, corresponding to continuous time.

• The Yt (s, ω) are realized or ex post (after treatment)outcomes.


Introduction Policy

• Under this assumption, the individual treatment effect foragent ω comparing objective outcomes of treatment s withobjective outcomes of treatment s ′ is

Y (s, ω)− Y (s ′, ω) , s 6= s ′, (2)

where we pick two elements s, s ′ ∈ S.

• This is also called an individual level causal effect.

• This may be a nondegenerate random variable or a degeneraterandom variable.

• The causal effect is the Marshallian (1890) ceteris paribuschange of outcomes for an agent across states s and s ′.


Introduction Policy

• Economists are interested in the welfare of participants as wellas the objective outcomes (see Heckman and Smith, 1998).

• Although statisticians reason in terms of assignmentmechanisms, economists recognize that agent preferences oftengovern actual choices.

• Comparisons across outcomes can be made in terms of utilities(personal, R (Y (s, ω) , ω), or in terms of planner preferences,RG , or both types of comparisons might be made for the sameoutcome and their agreement or conflict evaluated).

• Write R (Y (s, ω) , ω) as R(s, ω), suppressing the explicitdependence of R on Y (s, ω).

• One can ask if R(s, ω) > R(s ′, ω) or not (is the agent betteroff as a result of treatment s compared to treatment s ′?).


Introduction Policy

• The difference in subjective outcomes is [R(s, ω)− R(s ′, ω)],and is another possible definition of a treatment effect.

• Since the units of R(s, ω) are arbitrary, one could instead recordfor each s and ω an indicator if the outcome in s is greater orless than the outcome in s ′, i.e. R(s, ω) > R(s ′, ω) or not.

• This is also a type of treatment effect.


Introduction Policy

• These definitions of treatment effects embody Marshall’s 1890notion of ceteris paribus comparisons but now in utility space.

• A central feature of the econometric approach to programevaluation is the evaluation of subjective evaluations asperceived by decision makers and not just the objectiveevaluations focused on by statisticians.


Introduction Policy

• The term “treatment” is used in multiple ways in this literatureand this ambiguity is sometimes a source of confusion.

• In its most common usage, a treatment assignment mechanismis a rule τ : Ω→ S which assigns treatment to each ω.

• The consequences of the assignment are the outcomes Y (s, ω),s ∈ S, ω ∈ Ω.

• The collection of these possible assignment rules is T whereτ ∈ T .

• There are two aspects of a policy under this definition.

• The policy selects who gets what.

• More precisely, it selects individuals ω and specifies thetreatment s ∈ S received.


Introduction Policy

• We offer a more nuanced definition of treatment assignmentthat explicitly recognizes the element of choice by agent ω inproducing the treatment assignment rule.

• Treatment can include participation in activities such asschooling, training, adoption of a particular technology, and thelike.

• Participation in treatment is usually a choice made by agents.

• Under a more comprehensive definition of treatment, agents areassigned incentives like taxes, subsidies, endowments andeligibility that affect their choices, but the agent chooses thetreatment selected.


Introduction Policy

• Agent preferences, program delivery systems, aggregateproduction technologies, market structures, and the like mightall affect the choice of treatment.

• The treatment choice mechanism may involve multiple actorsand multiple decisions that result in an assignment of ω to s.

• For example, s can be schooling while Y (s, ω) is earnings givenschooling for agent ω.

• A policy may be a set of payments that encourage schooling, asin the Progressa program in Mexico, and the treatment in thatcase is choice of schooling with its consequences for earnings.


Introduction Policy

• We specify assignment rules a ∈ A which map individuals ωinto constraints (benefits) b ∈ B under different mechanisms.

• In this notation, a constraint assignment mechanism a is a map

a : Ω→ B

defined over the space of agents.

• The constraints may include endowments, eligibility, taxes,subsidies and the like that affect agent choices of treatment.

• The map a defines the rule used to assign b ∈ B.

• Formally, the probability system for the model withoutrandomization is (Ω, σ (Ω) ,F) where Ω is the probabilityspace, σ (Ω) is the σ-algebra associated with Ω and F is themeasure on the space.


Introduction Policy

• When we account for randomization we need to extend Ω toΩ′ = Ω×Ψ, where Ψ is the new probability space induced bythe randomization, and we define a system (Ω′, σ (Ω′) ,F ′).


Introduction Policy

• Some policies may have the same overall effect on theaggregate distribution of b, but may treat given individualsdifferently.

• Under an anonymity postulate, some would judge such policiesas equivalent in terms of the constraints (benefits) offered,even though associated outcomes for individuals and aggregatesmay be different.

• Another definition of equivalent policies is in terms of thedistribution of aggregate outcomes associated with thetreatments.

• In this chapter, we characterize policies at the individual levelrecognizing that sets of A that are characterized by someaggregate distribution over elements of b ∈ B may be whatothers mean by a policy.


Introduction Policy

• Given b ∈ B allocated by constraint assignment mechanisma ∈ A, agents pick treatments.

• We define treatment assignment mechanismτ : Ω×A× B → S as a map taking agent ω ∈ Ω facingconstraints b ∈ B assigned by mechanism a ∈ A into atreatment s ∈ S.

• Note that including B in the domain of definition of τ isredundant since the map a : Ω→ B selects an element b ∈ B.

• We make b explicit to remind the reader that agents aremaking choices under constraints.

• In settings with choice, τ is the choice rule used by agentswhere τ ∈ T , a set of possible choice rules.


Introduction Policy

• It is conventional to assume a unique τ ∈ T is selected by therelevant decision makers, although that is not required in ourdefinition.

• A policy regime p ∈ P is a pair (a, τ) ∈ A× T that mapsagents denoted by ω into elements of s.

• In this notation, P = A× T .


Introduction Policy

• Incorporating choice into the analysis of treatment effects is anessential and distinctive ingredient of the econometric approachto the evaluation of social programs.

• The traditional treatment-control analysis in statistics equatesmechanisms a and τ .

• An assignment in that literature is an assignment to treatment,not an assignment of incentives and eligibility for treatmentwith the agent making treatment choices.

• In this notation, the traditional approach has only oneassignment mechanism and treats noncompliance with it as aproblem rather than as a source of information on agentpreferences, as in the econometric approach.


Introduction Policy

• Thus, under full compliance, a : Ω→ S and a = τ , whereB = S.


Introduction Policy

• Policy invariance is a key assumption for the study of policyevaluation.

• It allows analysts to characterize outcomes without specifyinghow those outcomes are obtained.

• In our notation, policy invariance has two aspects.

• The first aspect is that, for a given b ∈ B (incentive schedule),the mechanism a ∈ A by which ω is assigned a b (e.g. randomassignment, coercion at the point of a gun, etc.) and theincentive b ∈ B are assumed to be irrelevant for the values ofrealized outcomes for each s that is selected.

• Second, for a given s for agent ω, the mechanism τ by which sis assigned to the agent under assignment mechanism a ∈ A isirrelevant for the values assumed by realized outcomes.


Introduction Policy

• Both assumptions define what we mean by policy invariance.


Introduction Policy

• Policy invariance allows us to describe outcomes by Y (s, ω)and ignore features of the policy and choice environment indefining outcomes.

• If we have to account for the effects of incentives andassignment mechanisms on outcomes, we must work withY (s, ω, a, b, τ) instead of Y (s, ω).

• The following policy invariance assumptions justify collapsingthese arguments of Y (·) down: Y (s, ω).


Introduction Policy

• Policy invariance for objective outcomes:

PI-1 (PI-1)

For any two constraint assignment mechanisms a, a′ ∈ A andincentives b, b′ ∈ B, with a(ω) = b and a′(ω) = b′, and for allω ∈ Ω, Y (s, ω, a, b, τ) = Y (s, ω, a′, b′, τ), for alls ∈ Sτ(a,b)(ω) ∩ Sτ(a′,b′)(ω) for assignment rule τ where Sτ(a,b)(ω) isthe image set for τ (a, b). For simplicity we assumeSτ(a,b)(ω) = Sτ(a,b) for all ω ∈ Ω.


Introduction Policy

• This assumption says that for the same treatment s and agentω, different constraint assignment mechanisms a and a′ andassociated constraint assignments b and b′ produce the sameoutcome.


Introduction Policy

• A second invariance assumption invoked in the literature is thatfor a fixed a and b, the outcomes are the same independent ofthe treatment assignment mechanism:

PI-2 (PI-2)

For each constraint assignment a ∈ A, b ∈ B and all ω ∈ Ω,Y (s, ω, a, b, τ) = Y (s, ω, a, b, τ ′) for all τ and τ ′ ∈ T withs ∈ Sτ(a,b) ∩ Sτ ′(a,b), where Sτ(a,b) is the image set of τ for a givenpair (a, b).

• Again, we exclude the possibility of ω-specific image sets Sτ(a,b)and Sτ ′(a,b).


Introduction Policy

• These invariance postulates are best discussed in the context ofspecific economic models.

• These conditions are closely related to the invariance conditionsof Hurwicz (1962).


Introduction Policy

• If treatment effects based on subjective evaluations are alsoconsidered, we need to broaden invariance assumptions (PI-1)and (PI-2) to produce invariance in rewards for certain policiesand assignment mechanisms.

• It would be unreasonable to claim that utilities R (·) do notrespond to incentives.

• Suppose, instead, that we examine subsets of constraintassignment mechanisms a ∈ A that give the same incentives(elements b ∈ B) to agents, but are conferred by differentdelivery systems, a.


Introduction Policy

• For each ω ∈ Ω, define the set of mechanisms delivering thesame incentive or constraint b as Ab (ω):

Ab (ω) = a | a ∈ A, a(ω) = b , ω ∈ Ω.

The set of delivery mechanisms that deliver b may vary amongthe ω.

• Let R (s, ω, a, b, τ) represent the reward to agent ω from atreatment s with incentive b allocated by mechanism a with anassignment to treatment mechanism τ .


Introduction Policy

PI-3 (PI-3)

For any two constraint assignment mechanisms a, a′ ∈ A andincentives b, b′ ∈ B with a(ω) = b and a′(ω) = b′, and for allω ∈ Ω, Y (s, ω, a, b, τ) = Y (s, ω, a′, b′, τ) for alls ∈ Sτ(a,b)(ω) ∩ Sτ(a′,b′)(ω) for assignment rule τ , where Sτ(a,b)(ω)is the image set of τ (a, b) and for simplicity we assume thatSτ(a,b)(ω) = Sτ(a,b) for all ω ∈ Ω. In addition, for any mechanismsa, a′ ∈ Ab (ω), producing the same b ∈ B under the same conditionspostulated in the preceding sentence, and for all ω,R (s, ω, a, b, τ) = R (s, ω, a′, b, τ) .


Introduction Policy

• This assumption says, for example, that utilities are notaffected by randomization or the mechanism of assignment ofconstraints.

• Corresponding to (PI-2) we have a policy invariance assumptionfor the utilities with respect to the mechanism of assignment:


Introduction Policy

PI-4 (PI-4)

For each pair (a, b) and all ω ∈ Ω,

Y (s, ω, a, b, τ) = Y (s, ω, a, b, τ ′)

R (s, ω, a, b, τ) = R (s, ω, a, b, τ ′)

for all τ, τ ′ ∈ T and s ∈ Sτ(a,b) ∩ Sτ ′(a,b).


Introduction Policy

• This assumption rules out general equilibrium effects, socialexternalities in consumption, etc. in both subjective andobjective outcomes.


Introduction Policy

End material from previous slide presentations (1).


Econometric Evaluation of Social Programs Part I: Causal ...jenni.uchicago.edu › econ312 ›...

Documents

Transcript of Econometric Evaluation of Social Programs Part I: Causal ...jenni.uchicago.edu › econ312 ›...