Macroeconomics Review Sheet: Laibson

Matthew Basilico

Spring, 2013

Course Outline:

1. Discrete Time Methods

(a) Bellman Equation, Contraction Mapping Theorem, Blackwell's Sucient Conditions, Nu-merical Methods

i. Applications to growth, search, consumption, asset pricing

2. Continuous Time Methods

(a) Bellman Equation, Brownian Motion, Ito Proccess, Ito's Lemma

i. Application to search, consumption, price-setting, investment, I.O., asset-pricing

Lecture 1

Sequence ProblemWhat is the sequence problem?

Find v(x0) such that

v (x0) = supxt+1∞t=0

∞∑t=0

δtF (xt, xt+1)

subject to xt+1 ∈ Γ(x) with x0 given

Variable denitions

• xt is the state vector at date t

• F (xt, xt+1) is the ow payo at date t

function F is stationary

a.k.a. the ow utility function

• δt is the expontential discount function

1

Discount Variable DenitionsWhat are the denitions of the dierent discount variables?

• δt is the exponential discount function

• δ is referrred to as the exponential discount factor

• ρ is the discount rate

which is the rate of decline of the discount function, so

∗ ρ ≡ − ln δ = −dδt

dt

δt

· so e−ρ = δ

Bellman EquationWhat is the Bellman Equation? What are its components?

Bellman equation expresses the value function as a combination of a ow payo and a discountedcontinuation payo

v(x) = supx+1∈Γ(x)

F (x, x+1) + δv(x+1)

Components:

• Flow payo is F (x, x+1)

• Current value function is v(x). Continuation value function is v(x+1)

• Equation holds for all (feasible) values of x

• v(·) is called the solution to the Bellman Equation

Any old function won't solve it

Policy FunctionWhat is a policy function? What is an optimal policy?

A policy is a mapping from x to the action space (which is equivalent to the choice variables)An optimal policy achieves payo v(x) for all feasible x

Relationship between Bellman and Sequence Problem

A solution to the Bellman will also be a solution to the sequence problem, and vice versa.1. A solution to the Sequence Problem is a solution to the Bellman

2

2. A solution to the Bellman Equation is also a solution to the sequence problem

Notation with Sequence Problem and Bellman example: Optimal Growth with Cobb-Douglas TechnologyHow do we write optimal growth using log utility and Cobb-Douglas technology? How can this betranslated into sequence problem and Bellman Equation notation?

Optimal growth:

supct∞t=0

∞∑t=0

δt ln(ct)

subject to the constraints c, k ≥ 0, kα = c+ k+1, and k0 given

Optimal growth in Sequence Problem notation: [∞- horizon]

v(k0) = supkt+1∞t=0

∞∑t=0

δt ln(kαt − kt+1)

such that kt+1 ∈ [0, kα] ≡ Γ(k) and k0given

Optimal growth in Bellman Equation notation: [2-period]v(k) = sup

k+1∈[0,kα]

ln (kα − k+1) + δv(k+1) ∀k

Methods for Solving the Bellman EquationWhat are the 3 methods for solving the Bellman Equation?

1. Guess a solution

2. Iterate a functional operator analytically (This is really just for illustration)

3. Iterate a functional operator numerically (This is the way iterative methods are used in mostcases)

3

FOC and EnvelopeWhat are the FOC and envelope conditions?

First order condtion (FOC):Dierentiate with respect to choice variable

0 =∂F (x, x+1)

∂x+1+ δv′(x+1)

Envelope Theorem:Dierentiate with respect to state variable

v′(x) =∂F (x, x+1)

∂x

Optimal Stopping using G&CHow do we approach optimal stopping using Guess and Check?

1. Write the Bellman Equation

2. Propose Threshold Rule as Policy Function

3. Use Policy Function to Write Value Function [Bellman Guess]

4. Rene Value Function using continuity of v(·)

5. Find value of threshold x∗ so that Value Function solves Bellman (using indierence)

Expanded:Agent draws an oer x from a uniform distribution with support in the unit interval

1. Write the Bellman Equation

v(x) = max x, δEv(x+1)

2. Propose Threshold Rule as Policy Function

(a) Stationary threshold rule, with threshold x∗:

Accept if x ≥ x∗Reject if x < x∗

3. Use Policy Function to Write Value Function [Bellman Guess]

(a) Stationary threshold rule implies that there exists some constant v such that

v(x) =

x if x ≥ x∗v if x < x∗

4. Rene Value Function using continuity of v(·)

(a) By continuity of the Value Function, it follows that v = x∗

v(x) =

x if x ≥ x∗x∗ if x < x∗

4

5. Find value of threshold x∗ so that Value Function solves Bellman

(a) Use indierence condition

v(x∗) = x∗ = δEv(x+1)

so x∗ = δ

ˆ x=x∗

x=0

x∗f(x)dx+ δ

ˆ x=1

x=x∗xf(x)dx = δ

1

2(x∗)2 + δ

1

2

Giving solution x∗ = δ−1(

1−√

1− δ2)

Lecture 2

Bellman OperatorWhat is the Bellman operator? What are its properties as a functional operator?

Bellman operator B, operating on a function w, is dened(Bw) (x) ≡ sup

x+1∈Γ(x)

F (x, x+1) + δw(x+1) ∀x

Note: The Bellman operator is a contraction mapping that can be used to iterate until convergence toa xed point, a function that is a solution to the Bellman Equation.Having Bw on the LHS frees the RHS continuation value function w from being the same function.Hence if supBw(x) 6= w(x), then keep getting a better (closer) function to the xed point whereBw(x) = w(x).Notation: When you iterate, RHS value function lags (one period). When you are not iterating, itdoesn't lag.

Properties as a functional operator

• Denition is expressed pointwise - for one value of x - but applies to all values of x

• Operator B maps function w to new function Bw

Hence operator B is a functional operator since it maps functions

Properties at solution to Bellman Equation

• If v is a solution to the Bellman Equation, then Bv = v (that is, (Bv)(x) = B(x) ∀x)

• Function v is a xed point of B (B maps v to v)

Contraction MappingWhy does Bnw converge as n→∞? What is a contraction mapping? What is the contraction mappingtheorem?

The reason Bnw converges as n→∞ is that B is a contraction mapping.

Denition: Let (S, d) be a metric space and B : S → S be a function mapping S onto itself. B is acontraction mapping if for some δ ∈ (0, 1), d (Bf, Bg) ≤ δd(f, g) for any two functions f and g.

5

• Intuition: B is a contraction mapping if operating B on any two functions moves them strictlycloser together. (Bf and Bg are strictly closer together than f and g)

• A metric (distance function) is just a way of representing the distance between two functions(e.g. maximum pointwise gap between two functions)

Contraction Mapping Theorem:If (S, d) is a complete metric space and B : S → S is a contraction mapping, then:

1. B has exactly one xed point v ∈ S

2. for any v0 ∈ S, limBnv0 = v

3. Bnv0 has an exponential convergence rate at least as great as − ln δ

Blackwell's TheoremWhat is Blackwell's Theorem? How is it proved?

Blackwell's Sucient Conditions - these are sucienct (but not necessary) for an operator tobe a contraction mappingLet X ⊆ Rl and let C(X) be a space of bounded functions f : X → R, with the sup-metric.Let B : C(X)→ C(X) be an operator satisfying:

1. (Monotonicity) if f, g ∈ C(X) and f(x) ≤ g(x)∀x ∈ X, then (Bf)(x) ≤ (Bg)(x)∀x ∈ X

2. (Discounting) there exists some δ ∈ (0, 1) such that

[B(f + a)] (x) ≤ (Bf)(x) + δa ∀f ∈ C(X), a ≥ 0, x ∈ X

Then B is a contraction with modulus δ.[Note that a is a constant, and f + a is the function generated by adding a constant to the function f ]

Proof:For any f, g ∈ C(X) we have f ≤ g + d(f, g)Properties 1 & 2 of the conditions imply

Bf ≤ B (g + d(f, g)) ≤ Bg + δd(f, g)

Bg ≤ B (f + d(f, g)) ≤ Bf + δd(f, g)

Combining the rst and last terms:

Bf −Bg ≤ δd(f, g)

Bg −Bf ≤ δd(f, g)

|(Bf)(x)− (Bg)(x)| ≤ δd(f, g) ∀x

supx|(Bf)(x)− (Bg)(x)| ≤ δd(f, g)

d(Bf,Bg) ≤ δd(f, g)

6

Checking Blackwell's ConditionsEx 4.1: Check Blackwell conditions for a Bellman operator in a consumption problem

Consumption problem with stochastic asset returns, stocastic labor income, and a liquidity constraint

(Bf)(x) = supc∈[0,x]

u(c) + δEf

(R+1(x− c) + y+1

)∀x

1. Monotonicity:Assume f(x) ≤ g(x)∀x. Suppose c∗f is the optimal policy when the continuation value function is f .

(Bf)(x) = supc∈[0,x]

u(c) + δEf

(R+1(x− c) + y+1

)= u(c∗f ) + δEf

(R+1(x− c∗f ) + y+1

)≤ u(c∗f ) + δEg

(R+1(x− c∗f ) + y+1

)≤ supc∈[0,x]

u(c) + δEg

(R+1(x− c) + y+1

)= (Bg)(x)

[Note (in top line) elimination of sup by using optimal policy c∗f . In bottom line, can use any policyto add in sup]

2. DiscountingAdding a constant (∆) to an optimization problem does not eect optimal choice, so:

[B(f + ∆)] (x) = supc∈[0,x]

u(c) + δE

[f(R+1(x− c) + y+1

)+ ∆

]= supc∈[0,x]

u(c) + δEf

(R+1(x− c) + y+1

)+ δ∆ = (Bf)(x) + δ∆

Iteration Application: Search/StoppingWhat is the general notation? Taking lecture 1 example, can you iterate from an initial guess ofv0(x) = 0 for 2 steps? How about proving convergence?

Notation: Iteration of the Bellman operator B:

vn(x) = (Bnv0)(x) = B(Bn−1v0)(x) = maxx, δE(Bn−1v0)(x)

Let xn ≡ δE(Bn−1v0)(x+1), which is the continuation payo for vn(x).xn is also the cuto threshold associated with vn(x) = (Bnv0)(x)xn is sucient statistic for vn(x) = (Bnv0)(x)

Bellman Operator for the Problem: (Bw)(x) ≡ max x, δEw(x+1). *Note there is no lag here in thew index since we are not iteratingIterating Bellman Operator once on v0(x) = 0:

v1(x) = (Bv0)(x) = max x, δEv0(x+1) = max x, 0 = x

Iterating again on v1(x) = x:

v2(x) =[(B2v0)(x)

]= (Bv1)(x) = max x, δEv1(x+1) = max x, δEx+1

x2 = δEx+1 = δ2

7

v2 =

x2 if x ≤ x2

x if x ≥ x2

Proof at limit:We have:

(Bn−1v0)(x) =

xn−1 if x ≤ xn−1

x if x ≥ xn−1

xn = δE(Bn−1v0)(x) = δ

[ˆ x=xn−1

x=0

xn−1f(x)dx+

ˆ x=1

x=xn−1

xf(x)dx

]=δ

2

(x2n−1 + 1

)We set xn = xn−1 to conrm that this sequence converges to

limn→∞

xn = δ−1(

1−√

1− δ2)

Lecture 3

Classical Consumption Model1. Consumption; 2. Linearization of the Euler Equation; 3. Empirical tests without precautionarysavings eects

Classical Consumption

What is the classical consumption problem, in Sequence Problem and Bellman Equation notation?

1. Sequence Problem Representation

Find v(x) such that

v(x0) = supct∞0

E0

∞∑t=0

δtu(ct)

subject to

a static budget constraint for consumption: ct ∈ ΓC(x)

a dynamic budget constraint for assets: xt+1 ∈ ΓX(xt, ct, Rt+1, yt+1, . . .

)

Variables: x is vector of assets, c is consumption, R is vector of nancial asset returns, y is vector oflabor income

Common Example:The only asset is cash on hand, and consumption is constrained to lie between 0 and x:

ct ∈ ΓC(xt) ≡ [0, xt]; xt+1 ∈ ΓX(xt, ct, Rt+1, yt+1, . . .

)≡ Rt+1(xt − ct) + yt+1; x0 = y0

Assumptions: y is exogenous and iid; u is concave; limc↓0 u′(c) =∞ (so c > 0 as long as x > 0)

8

2. Bellman Equation Representation

[It is more convenient to think about c as the choice variable. The state variable, x, is stochastic, soit is not directly chosen (rather a distribution for xt+1 is chosen at time t)]

v(xt) = supct∈[0,xt]

u(ct) + δEtv(xt+1) ∀x

xt+1 = Rt+1 (xt − ct) + yt+1

x0 = y0

Euler Equation 1: Optimization

How is the Euler equation derived from the Bellman Equation of the consumption problem above usingoptimization?

v(xt) = supct∈[0,xt]

u(ct) + δEtv

(Rt+1(xt − ct) + yt+1

)∀x

1. First Order Condition:

u′(ct) = δEtRt+1v′(xt+1) if 0 < ct < xt (interior)

u′(ct) ≥ δEtRt+1v′(xt+1) if ct = xt (boundary)[

FOCct : 0 = u′(ct) + δEtv′(xt+1)× (−Rt+1) using chain rule

]2. Envelope Theorem

v′(xt) = u′(ct)[Dierentiate with respect to state variable xt : v′(x) = u′(ct)

∂ct∂xt

= u′(ct)]

[Note: xt+1 = Rt+1 (xt − ct) + yt+1 =⇒ ctRt+1 = Rt+1xt − xt+1 + yt+1 =⇒ ct = xt − xt+1+yt+1

Rt+1

]Putting the FOC and Envelope Condition together, and moving the Envelope Condition forward oneperiod, we get:IEuler Equation:

u′(ct) = δEtRt+1u′(ct+1) if 0 < ct < xt (interior)

u′(ct) ≥ δEtRt+1u′(ct+1) if ct = xt (boundary)

Euler Equation 2: Perturbation

How is the Euler equation derived from the Bellman Equation of the consumption problem above usingperturbation?[*Prefers that we know it this way*]

1. General Intuition

• Cost of consuming one dollar less today?

9

Utility loss today = u′(ct)

• Value of saving an extra dollar Discounted, expected utility gain of consuming Rt+1 moredollars tomorrow:

Utility gain tomorrow = δEtRt+1u′(ct+1)

2. Perturbation**Idea: at an optimum, there cannot be a perturbation that improves welfare of the agent.If there is such a feasible pertubation, then we've proven that whatever we wrote down is not anoptimum.

Proof by contradiction: [Assume inequalities, and show they produce contradictions so cannot hold]

1. Suppose u′(ct) < δEtRt+1u′(ct+1) .

[If u′(ct) is less than the discounted value of a dollar saved, then what should I do? Save more,consume less.]

(a) Then we can reduce ct by ε and raise ct+1 by Rt+1 · ε to generate a net utility gain:

0 <[δEtRt+1u

′(ct+1)− u′(ct)]· ε

=⇒ this perturbation raises welfare [Q? - dening welfare here]

[Note the expression is from simply moving u′(ct) to RHS from above and multiplying by ε]

(a) If this perturbation is possible that raises welfare, then can't be true that we started thisanalysis at an optimum.

i. This perturbation is always possible along the equilibrium path.

ii. Hence, if this cannot be an optimum, we've shown that at an optimum, we must have:

u′(ct) ≥ δEtRt+1u′(ct+1)

2. Suppose u′(ct) > δEtRt+1u′(ct+1)

(a) Then we can raise ct by ε and reduce ct+1 by Rt+1 · ε to generate a net utility gain:

i. [If u′(ct) is greater than the value of a dollar saved, then what should I do? Save less,consume more.]

ε ·[u′(ct)− δEtRt+1u

′(ct+1)]> 0

=⇒ this perturbation raises welfare [Q? - dening welfare here]

(b) If this perturbation is possible that raises welfare, then can't be true that we started thisanalysis at an optimum.

i. This perturbation is always possible along the equilibrium path, as long as ct < xt(liquidity constraint not binding).

ii. Hence, if this cannot be an optimum, we've shown that at an optimum, we must have:

u′(ct) ≤ δEtRt+1u′(ct+1) as long as ct < xt

10

It follows that:

u′(ct) = δEtRt+1u′(xt+1) if 0 < ct < xt (interior)

u′(ct) ≥ δEtRt+1u′(xt+1) if ct = xt (boundary)

Linearizing the Euler Equation

How do you linearize the Euler Equation?

Goal: Can we linearize this and make it operational without unrealistic assumptions?Equation tells us what I should expect consumption growth ∆ ln ct+1 to do

1. Assume u is an isoelastic (i.e. CRRA) utility function

u(c) =c1−γ − 1

1− γ

Note: limγ→1cγ−1−1

1−γ =ln c. Important special case.

2. Assume Rt+1 is known at time t

3. Rewrite the Euler Equation [using functional form]:

c−γt = EtδRt+1c−γt+1

4. [Algebra] Divide by c−γt

1 = EtδRt+1c−γt+1c

γt

5. [Algebra] Rearrange, using exp[ln]:

1 = Et exp[ln(δRt+1c

−γt+1c

γt

)]6. [Algebra] Distribute ln:

1 = Et exp [rt+1 − ρ+ (−γ) ln ct+1/ct]

[Note: ln δ = −ρ; lnRt+1 = rt+1]

7. [Algebra] Substitute notation for consumption growth:

1 = Et exp [rt+1 − ρ− γ∆ ln ct+1]

[Note: ln(ct+1/ct) = ln ct+1 − ln ct ≡ ∆ ln ct+1]

8. [Stats] Assume ∆ ln ct+1 is conditionally normally distributed. Apply expectation operator:

1 = exp

[Etrt+1 − ρ− γ∆ ln ct+1 +

1

2γ2Vt∆ ln ct+1

][Note: Eea = eEa+ 1

2V ara where ais a random variable]

[Here we have:− γ∆ ln ct+1 ∼ N (−γ∆ ln ct+1, γ

2Vt∆ ln ct+1)]

11

9. [Algebra] Take ln:

0 = Etrt+1 − ρ− γ∆ ln ct+1 +1

2γ2Vt∆ ln ct+1

10. [Algebra] Divide by γ, rearrange:

(∗) ∆ ln ct+1 =1

γ(Etrt+1 − ρ) +

1

2γVt∆ ln ct+1

Terms of linearized Euler Equation (∗):∆ ln ct+1 = consumption growthVt∆ ln ct+1 =conditional variance in consumption growth conditional on information known at timet.(i.e. conditional on being in year 49, what is the variance in consumption growth between 49 and 50)It is also known as the precautionary savings term as seen in the regression analysis

∆ ln ct+1

Why is ∆ ln ct+1 consumption growth?

ct+1 − ctct

≈ ln

(1 +

ct+1 − ctct

)= ln

(1 +

ct+1

ct− 1

)= ln

(ct+1

ct

)≡ ∆ ln ct+1

[the second step uses the log approximation, ∆x ≈ ln (1 + ∆x)]

Important Consumption Models

What are the two important consumption models?

In 1970s, began recognizing that what is interesting is change in consumption, as opposed to just levelsof consumption. Since then have stayed with this approach.

1. Life Cycle Hypothesis (Modigliani & Brumberg, 1954; Friedman) = Eat the PieProblem

(a) Assumptions

i. Rt = R

ii. δR = 1 [δ < 1;R > 1]

iii. Perfect capital markets, no moral hazard, so future labor income can be exchanged forcurrent capital

(b) Bellman Equation

v(x) = supc≤xu(c) + δEv(x+1) ∀x

x+1 = R(x− c)

x0 = E

∞∑t=0

R−tyt

12

[At date 0, sells all his labor income for a lump sum, discounted at rate r. This sum is hisonly source of wealth now]

[Eat the Pie, except every piece he leaves grows by a factor r]

(c) Euler Equation

i. u′(ct) = δRu′(ct+1) = 1 · u′(ct+1) = u′(ct+1)

ii. =⇒ consumption is constant in all periods

(d) Budget Constraint

∞∑t=0

R−tct = E0

∞∑t=0

R−tyt

[(Discounted sum of his consumption) must be equal to [the expectation at date 0 of (thediscounted sum of his labor income)]]

(e) Subsitute Euler Equation

∞∑t=0

R−tc0 = E0

∞∑t=0

R−tyt

i. To give nal form: [Euler Equation + Budget Constraint][Note:

∑∞t=0R

−t = 11− 1

R

]c0 =

(1− 1

R

)E0

∞∑t=0

R−tyt ∀t

Intuition: Consumption is an annuity. Consumption is equal to interest rate scaling my totalnet worth at date 0(1− 1

R

)is the annuity scaling term; approximately equal to the real net interest rate

2. Certainty Equivalence Model (Hall, 1978)

(a) Assumptions

i. Rt = R

ii. δR = 1 [δ < 1;R > 1]

iii. Can't sell claims to labor income

iv. Quadratic utility : u(c) = αc− β2 c

2

A. Note: This admits negative consumption, and does not imply that limc↓0 u′(c) =∞

B. [Hence it is possible to consume a negative amount]

(b) Bellman Equation

v(x) = supcu(c) + δEv(x+1) ∀x

x+1 = R(x− c) + y+1

x0 = y0

(c) Euler Equation

13

i. Euler becomes: ct = Etct+1 = Etct+nu′(c) = α− βc. u′(ct) = δEtRt+1; u′(ct+1) = Etu

′(ct+1) [Since δR = 1]=⇒ α − βct = Et [α− βct+1] = α − βEtct+1 =⇒ −βct = −βEtct+1 =⇒ ct =Etct+1

ii. So ct = Etct+1 = Etct+n implies consumption is a random walk :

ct+1 = ct + εt+1

iii. So ∆ct+1 can not be predicted by any information available at time t

A. If I know ct, no other economic variable should have predictive power at ct+1; anyother information should be orthogonal

(d) Budget Constraint

∞∑t=0

R−tct ≤∞∑t=0

R−tyt

[Sum from t to ∞ of discounted consumption is ≤the discounted value of stochastic laborincome]

i. Budget Constraint at time t:

∞∑s=0

R−sct+s = xt +

∞∑s=1

R−syt+s

[Discounted sum of remaining consumption = cash on hand (at date t) and the dis-counted sum of remaining labor income]

ii. Now applying expectation: [if it's true on every path, it's true in expectation]

Et

∞∑s=0

R−sct+s = xt + Et

∞∑s=1

R−syt+s

(e) Subsitute Euler Equation [ct = Etct+s]

∞∑s=0

R−sct = xt + Et

∞∑s=1

R−syt+s

i. To give nal form: [Euler Equation + Budget Constraint][Note:

∑∞t=0R

−t = 11− 1

R

]ct =

(1− 1

R

)(xt + Et

∞∑s=1

R−syt+s

)∀t

Intuition: Just like in Modigliani, consumption is equal to interest rate scaling my total networth at date t

i. (but get there just by assuming a quadratic utility function)(1− 1

R

)is the annuity scaling term (approximately equal to the real net interest rate)

(xt + Et∑∞s=1R

−syt+s) is total net worth at date t

14

Empirical Tests of Linearized Euler Equation: Without Precautionary Sav-

ings Eects

Want to prove that consumption growth is unpredictable between t and t+ 1A. Testing Linearized Euler Equation: Without Precautionary Savings Eects

1. Start with: Linearized Euler Equation, in regression form:

∆ ln ct+1 =1

γ(Etrt+1 − ρ) +

1

2γVt∆ ln ct+1 + εt+1

[where εt+1 is orthogonal to any information known at time t.]

2. Assume (counterfactually) precautionary savings is constant

(a) Euler equation reduces to: ∆ ln ct+1 = constant + 1γEtrt+1 + εt+1

[Note: when we replace precautionary term with a constant, we are eectively ignoring its eect(since it is no longer separately identied from the other constant term ρ

γ )]

B. Estimating the Linearized Euler Equation: 2 Goals and Approaches

1. Goal 1: Estimate 1γ

(a) 1γ is EIS, the elasticity of intertemporal substitution [for this model, EIS is the inverse of

the CRRA; ∴ in other models EIS not 1γ ]

(b) Estimate using ∂∆ ln ct+1

∂Etrt+1, with equation ∆ ln ct+1 = constant + 1

γEtrt+1 + εt+1

2. Goal 2: Test the orthagonality restriction Ωt⊥εt+1

(a) This means, test the restriction that information available at time t does not predict con-sumption growth in the following regression [new βXt term, for any variable]:

∆ ln ct+1 = constant +1

γEtrt+1 + βXt + εt+1

(a) For example, does the date t expectation of income growth, Et∆ lnYt+1 predict date t+ 1consumption growth [for Shea, etc test below]?

i. [Model is trying to say that an expectation of rising incomei.e. being hired on jobmarketdoesn't translate into a rise in consumption. α should be 0.]

∆ ln ct+1 = constant +1

γEtrt+1 + αEt∆ lnYt+1 + εt+1

C. Summary of Results:

1. 1γ ∈ [0, 0.2] (Hall, 1988)

(a) EIS is very smallpeople are fairly unresponsive to expected movements in the interest rate;this doesn't seem to be a big driver of consumption growth

i. the parameter that is scaling the expected value of the interest rate is estimated tobe close to 0

15

(b) But estimate may be messed up once you add in liquidity contraints. Someone who is veryliquidity constrained → EIS of 0.

(c) So what exactly are we learning when we see American consumers are not responsive tothe interest rate? [Is it just about liquidity constraints, or does it mean that even in theirabsence, something deep about responsiveness to interest rate]

2. α ∈ [0.1, 0.8] (Campell and Mankiw 1989, Shea 1995)

(a) =⇒ expected income growth (predicted at date t) predicts consupmtion growth at datet+ 1

(b) =⇒ the assumptions (1) the Euler Equation is true, (2) the utility function is CRRA, (3)the linearization is accurate, and (4) Vt∆ ln ct+1 is constant, are jointly rejected

(c) Theories on why does expected income predict consumption growth:

i. Leisure and consumption expenditure are substitutes

ii. Work-based expenses

iii. Households support lots of dependents in mid-life when income is highest

iv. Houesholds are liquidity-constrained and impatient

v. Some consumers use rules of thumb: cit = Yit

vi. Welfare costs of smoothing are second-order

Lecture 4

Precautionary Savings and Liquidity Contraints1. Precautionary Savings Motives; 2. Liquidity Constraints; 3. Application: Numerical solution ofproblem with liquidity constraints; 4. Comparison to eat-the-pie problem

Precautionary MotivesWhy do people save more in response to increased uncertainty?

1. Uncertainty and the Euler Equation

Et∆ ln ct+1 =1

γ(Etrt+1 − ρ) +

γ

2Vt∆ ln ct+1

[where Vt∆ct+1 = Et[∆ ln ct+1 − Et∆ ln ct+1]2

](a) Increase in economic uncertainty raises Vt∆ct+1, raising Et∆ ln ct+1

i. Note: this is not a general result (i.e. doesn't apply to quadratic)

2. Key reason is the convexity of marginal utility

(a) An increase in uncertainty raises the expected value of marginal utility

i. Convexity of marginal utility curve raises marginal utility in expectation in the nextperiod

A. Euler equation (for CRRA): u′(ct) 6= u′(Et(ct+1)) but rather u′(ct) = Etu′(ct+1)

16

B. (Since by Jensen's inequality =⇒ Et(ct+1)ct

> 1)

(b) This increase the motive to save

i. =⇒ referred to as the precautionary savings eect

3. Denition: Precautionary Savings

(a) Precautionary saving is the reduction in consumption due to the fact that future laborincome is uncertain instead of being xed at its mean value.

Liquidity Constraints

Buer Stock ModelsWhat are the two key assumptions of the buer stock models? Qualitatively, what are some of theirpredictions

Since the 1990s, consumption models have emphasized the role of liquidity constraints (Zeldes, Caroll,Deaton).

1. Consumers face a borrowing limit: e.g. ct ≤ xt

(a) This matters whether or not it actually binds in equilibrium (since desire to preserve buerstock will aect marginal utilty as you get close to full-borrowing level)

2. Consumers are impatient: ρ > r

• Predictions

Consumers accumulate a small stock of assets to buer transitory income shocks

Consumption weakly tracks income at high frequencies (even predictable income)

Consumption strongly tracks income at low frequencies (even predictable income)

17

Eat the Pie Problem

A model in which the consumer can securitize her income stream. In this model, labor can be trans-formed into a bond.

1. Assumptions

(a) If consumers have exogenous idiosyncratic labor income risk, then there is no risk premiumand consumers can sell their labor income for

W0 = E0

∞∑t=0

R−tyt

(b) Budget constraint

W+1 = R(W − c)

2. Bellman Equation

v(W ) = supc∈[0,W ]

u(c) + δEv(R(W − c)) ∀x

3. Solve Bellman: Guess Solution Form

v(W ) =

ψW

1−γ

1−γ if γ ∈ [0,∞], γ 6= 1

φ+ ψ lnW if γ = 1

4. Conrm that solution works (problem set)

5. Derive optimal policy rule (problem set)

c = ψ−1γW

ψ−1γ = 1− (δR1−γ)

1γ

Lecture 5

Non-stationary dynamic programming

18

Non-Stationary Dynamic Programming

What type of problems does non-stationary dynamic programming solve? What is dierent about theapproach generally?

Non-Stationary Problems:

• So far we have assumed problem is stationary - that is, value function does not depend on time,only on the state variable

• Now we apply to nite horizon problems

E.g. Born at t = 1, terminate at t = T = 40

Two changes for nite horizon problems: backwards induction

1. We index the value function, since each value function is now date-specic

• vt(x) represents value function for period t

• E.g., vt(x) = Et∑Ts=t δ

s−tu(cs)

2. We don't use an arbitrary starting function for the Bellman operator

• Instead, iterate the Bellman operator on vT (x), where T is the last period in the problem

In most cases, the last period is easy to calculate, (i.e. end of life with no bequestmotive: vT (x) = u(x))

Backward Induction

How does backward induction proceed?

• Since you begin in the nal period, the RHS equation is one period ahead of the LHS sideequation (opposite of previous notation with operator)

Each iteration takes on more step back in time

• Concretely, we have the Bellman Equation

vt−1(x) = supc∈[0,x]

u(c) + δEtvt (R(x− c) + y) ∀x

• *To generate vt−1(x), apply the Bellman Operator

vt−1(x) = (Bvt)(x) = supc∈[0,x]

u(c) + δEtvt (R(x− c) + y) ∀x

(As noted above, generally start at termination date T and work backwards)

• Generally we have:

vT−n(x) = (BnvT )(x)

19

Lecture 6

Hyperbolic Discounting

Context:For β < 1, Ut = u(ct) + β

[δu(ct+1) + δ2u(ct+2) + δ3u(ct+3) + . . .

]Hyperbolic Bellman Equations

Bellman Equation in set up in the following way. Dene three equations where:V is the continuation value functionW is the current value functionC is the consumption function

1. V (x) = U(C(x)) + δE[V (R(x− C(x)) + y)]

2. W (x) = U(C(x)) + βδE[V (R(x− C(x)) + y)]

3. C(x) = arg maxc U(c) + βδE[V (R(x− c) + y)]

Three further identities:

• Identity linking V and W

βV (x) = W (x)− (1− β)U(C(x))

• Envelope Theorem

W ′(x) = U ′(C(x))

• FOC

U ′(C(x)) = RβδE [V ′(R(x− C(x)) + y)]

Hyperbolic Euler Equation

How do we derive an Euler equation for the hyperbolic scenario? What is the intuition behind theequation?

1. We have the FOC

u′(ct) = RβδEt [V ′(xt+1)]

2. Use the Identity linking V and W to subsitute for V ′

= RδEt

[W ′(xt+1)− (1− β)u′(ct+1)

dCt+1

dxt+1

]3. Use envelope theorem

20

(a) [substitute u′(xt+1) for W ′(ct+1)]

= RδEt

[u′(xt+1)− (1− β)u′(ct+1)

dCt+1

dxt+1

]4. Distribute

= REt

[βδ

(dCt+1

dxt+1

)+ δ

(1− dCt+1

dxt+1

)]u′(ct+1) (∗)

Interpretation of Hyperbolic Euler Equation (∗)

• First, note that dCt+1

dxt+1is the marginal propensity to consume (MPC).

• We then see that the main dierence in this Euler equation is that it is a weighted average of

discount factors

= REt

βδ(dCt+1

dxt+1

)︸ ︷︷ ︸short run

+δ

(1− dCt+1

dxt+1

)︸ ︷︷ ︸

long run

u′(ct+1)

• More cash on hand will shift to to the long-run term (less hyperbolic discounting)

Sophisticated vs. Naive Agents

[From Section Notes]

• Sophisticated Agents

The above analysis has assumed sophisticated agents

A sophisticated agent knows that in the future, he will continue to be quasi-hyperbolic

(that is, he knows that he will continue to choose future consumption in such a way thathe discounts by βδ between the current and next period, but by δ between future periods.)

This is reected in the specication of the consumption policy function (as above)

• Naive Agents

Doesn't realize he will discount in a quasi-hyperbolic way in future periods

He wrongly believes that, starting from the next period, he will discount by δ between allperiods.

∗ (Another way of saying: he wrongly believes that he will be willing to commit to thepath that he currently sets)

He will decide today, believing that he will choose from tomorrow forward by the exponen-tial discount function. (e superscript); however, when he reaches the next date, he againreoptimizes:

21

1. Dene

(a) Continuation value function, as given by standard exponential discounting Bellmanequation:

V et+1(x) = maxcet+1∈[0,x]

u(cet+1

)+ δEt+1V

et+2

(R(x− cet+1) + y

)(b) Current value function, as given by the hyperbolic equation:

Wnt (x) = max

cet∈[0,x]

u (cn) + δEtV

et+1 (R(x− cet ) + y)

2. Since he is not able to commit to the exponential continuation function, realized consump-

tion path will be given:

(a) Realized consumption path: cnτ (xτ )Tτ=t

(b) Solution at each time t is characterized by

u′(cet+1) = δREt+1u′ (cet+2

)u′(cnt ) = βδREtV

′e(xt+1) = βδREtu′ (cet+1

)

Lecture 7

Asset Pricing1. Equity Premium Puzzle; 2. Calibration of Risk Aversion; 3. Resolutions to the Equity PremiumPuzzle

Intuition

• Classical economics says that you should take an arbitrarily small bet with any positive re-turn...as long as the payos of the bet are uncorrelated with your consumption path

(punchline)

• Don't want to make a bet where you will give up resources in a low state of consumption (whenthe marginal utility of consumption is higher) for resources in a high state of consumption (wheremarginal utility of consumption is lower) [all because of curvature of the utility function]

• Not variance-based risk, covariance based risk

Asset Pricing Equation

What is the asset pricing equation? How is it derived?

Asset Pricing EquationGeneral Form:

πij = γσic − γσjc

22

where πij = ri − rj (the gap in rate of returns between assets iand j)

σic = Cov(σiεi,∆ ln c)

Risk-Free Asset:Denition: we denote the risk free return: Rft . Specically, we assume that at time t−1, Rft is known.When i =equities and j =risk free asset, we have the simple relationship

πequity,f = γσequity,c

Intuition:

• The equity premium πequity,f , is equal to the amount of risk, σequity,c, multiplied by the price ofrisk, γ.

Asset Notation and Statistics:

1. Allowing for many dierent types of assets: R1t+1, R

2t+1, . . . , R

it+1, . . . , R

It+1

2. Assets have stochastic returns:

Rit+1 = e

(rit+1+σiεit+1− 1

2 [σi]2)where εit+1has unit variance

εit+1 has unit variance =⇒ εit+1 is a normally distributed random variable: εit+1 ∼ N (0, 1)

x ∼ N(µ, σ2) −→Ax+B ∼ N(Aµ+B,A2σ2)

−→ E [exp (Ax+B)] = exp(Aµ+B + 1

2A2σ2)

[Since x ∼ N(µ, σ2)−→ E(exp(x)) = exp(E(x) + 12V ar(x)); so if standard normal:

= exp(µ+ 12σ

2)]

Hence: σiεit+1 + rit+1 − 12 [σi]2 ∼ N(σi ∗ 0 + rit+1 − 1

2 [σi]2, σi2 ∗ 12) = N(rit+1 − 12 [σi]2, σi2)

E [Rti] = E[exp(rit+1 + σiεit+1 − 1

2 [σi]2)]which as we've seen is now exp(a random variable) so

we can use

= exp(mean+ 12V ar) = exp(rit+1 − 1

2 [σi]2 + 12σ

i2) = exp(rit+1)

Finally we have that since for small x: ln(1 + x) ≈ x =⇒ 1 + x ≈ ex

∴ exp(rit) ≈ 1 + rit+1.

Derivation of Asset Pricing Equation: πij = γσic − γσjcHow is the Asset Pricing Equation derived?

Use Euler, Substitute, Take Expectation (Random Variable), Dierence Two Equations, Apply Co-variance Formula

1. Start with the Euler Equation

u′(ct) = Et[δRit+1u

′(ct+1)]

2. Assume u is isoelastic (CRRA):

u(c) =c1−γ − 1

1− γ

23

3. Substitute into the Euler equation

(a) δ = exp(−ρ)

(b) Rit+1 = e

(rit+1+σiεit+1− 1

2 [σi]2)where εit+1has unit variance

(c) u′(c) = c−γ (given CRRA)

c−γ = Et[δRit+1c

−γt+1

]c−γ = Et

[exp

(−ρ+ rit+1 + σiεit+1 −

1

2

(σi)2)

c−γt+1

]

1 = Et

[exp

(−ρ+ rit+1 + σiεit+1 −

1

2

(σi)2)(ct+1

ct

)−γ]

4. Re-write in terms of exp(·), using exp(ln)

1 = Et

[exp

(−ρ+ rit+1 + σiεit+1 −

1

2

(σi)2)

exp

[−γ ln

(ct+1

ct

)]]

1 = Et

[exp

(−ρ+ rit+1 + σiεit+1 −

1

2

(σi)2 − γ∆ ln (ct+1)

)](a) Rearrange, and note

1 = Et

exp

−ρ+ rit+1 −1

2

(σi)2︸ ︷︷ ︸

non-stochastic

+σiεit+1 − γ∆ ln (ct+1)︸ ︷︷ ︸random variable

5. Take Expectation, applying the formula for the expectation of a Random Variable

(a) That is, use: Sx ∼ N(Sµ, S2σ2) −→ E [exp (Sx)] = exp(Sµ+ 1

2S2σ2)

i. For the random variable[σiεit+1 − γ∆ ln (ct+1)

]ii. That is, E

[σiεit+1 − γ∆ ln (ct+1)

]= exp

[−γEt [∆ ln (ct+1)] + 1

2V[σiεit+1 − γ∆ ln (ct+1)

]]

1 =

[exp

(−ρ+ rit+1 −

1

2

(σi)2 − γEt [∆ ln (ct+1)] +

1

2V[σiεit+1 − γ∆ ln (ct+1)

])]6. Take Ln

0 = −ρ+ rit+1 −1

2

(σi)2 − γEt [∆ ln (ct+1)] +

1

2V[σiεit+1 − γ∆ ln (ct+1)

]7. Dierence Euler Equation for assets i and j, and Re-write:

0 = rit+1 − rjt+1 −

1

2

[(σi)2 − (σj)2]+

1

2

[V[σiεit+1 − γ∆ ln (ct+1)

]− V

[σjεjt+1 − γ∆ ln (ct+1)

]]

24

(a) Note that −ρ, γEt [∆ ln (ct+1)] drop out of the equation

(b) Re-write with interest rate dierence on LHS:

rit+1 − rjt+1 =

1

2

[(σi)2 − (σj)2]− 1

2

[V[σiεit+1 − γ∆ ln (ct+1)

]− V

[σjεjt+1 − γ∆ ln (ct+1)

]]8. Apply formula for V (A+B)

(a) [V (A+B) = V (A) + V (B) + 2Cov(A,B)]

i. Also note 2Cov(A,−αB) = −2αCov(A,B)

Note, in our case: V[σiεit+1 − γ∆ ln (ct+1)

]= V

(σiεit+1

)+V (−γ∆ ln (ct+1))+2Cov

(σiεit+1, γ∆ ln (ct+1)

)=(σi)2

+ γ2V∆ ln (ct+1)− 2γσic

rit+1−rjt+1 =

1

2

[(σi)2 − (σj)2 − [(σi)2 + γ2V∆ ln (ct+1)− 2γσic

]+[(σj)2

+ γ2V∆ ln (ct+1)− 2γσjc

]]

Just cancel terms: rit−rjt = 1

2

[(σi)2 − (σi)

2+ (σj)

2 − (σj)2

+ γ2V∆ ln (ct+1)− γ2V∆ ln (ct+1) + 2γσic − 2γσjc

]To get out asset pricing equation:

πij = rit+1 − rjt+1 = γσic − γσjc

Equity Premium Puzzle

What is the equity premium puzzle?

We can rearrange the asset pricing equation for the risk free asset[πequity,f = γσequity,c

]to isolate γ:

γ =πequity,f

σequity,c

Empirical Date: In the US post-war period

• πequity,f ≈ .06

• σequity,c ≈ .0003

γ =.06

.0003= 200

25

Lecture 8

Summary Equations

• Brownian Motion; Discrete to Continuous

∆x ≡ x(t+ ∆t)− x(t) =

+h with prob p−h with prob q = 1− p

E [x(t)− x(0)] = n ((p− q)h) = t

∆t (p− q)h (*)V [x(t)− x(0)] = n

(4pqh2

)= t

∆t4pqh2 (*)

• Calibration

h = σ√

∆t

p = 12

[1 + α

σ

√∆t], ⇒ (p− q) = α

σ

√∆t

E [x(t)− x(0)] = t∆t (p− q)h = t

∆t

(ασ

√∆t)(

σ√

∆t)

= αt︸︷︷︸linear drift

V [x(t)− x(0)] = σ2t︸︷︷︸linear variance

• Weiner

1. ∆z = ε√

∆t where ε ∼ N(0, 1), [also stated ∆z ∼ N(0,∆t)] Vertical movements ∝√

∆t

2. Non overlapping increments of ∆z are independent

• Ito:

dx = a(x, t)dt+ b(x, t)dz; dx = a(x, t)dt︸ ︷︷ ︸drift

+ b(x, t)dz︸ ︷︷ ︸standard deviation

1. Random Walk: dx = αdt+ σdz [with drift α and variance σ2]

2. Geometric Random Walk: dx = αxdt + σxdz [with proportional drift α and proportionalvariance σ2]

• Ito's Lemma: z(t) is Wiener, x(t) is Ito with dx = a(x, t)dt+ b(x, t)dz. Let V = V (x, t), then

dV = ∂V∂t dt+ ∂V

∂x dx+ 12∂2V∂x2 b(x, t)

2dt

=[∂V∂t + ∂V

∂x a(x, t) + 12∂2V∂x2 b(x, t)

2]dt+ ∂V

∂x b(x, t)dz

• Value Function as Ito Process:

Value Function is Ito Process (V ) of an Ito Process (x) or a Weiner Process (z)(need to check this)

dV = a(x, t)dt+ b(x, t)dz =

[∂V

∂t+∂V

∂xa(x, t) +

1

2

∂2V

∂x2b(x, t)2

]︸ ︷︷ ︸

a

dt+∂V

∂xb(x, t)︸ ︷︷ ︸b

dz

• Proof of Ito's Lemma:

dV = ∂V∂t dt+ 1

2∂2V∂t2 (dt)2 + ∂V

∂x dx+ 12∂2V∂x2 (dx)2 + ∂2V

∂x2 dxdt+ h.o.t.`

(dx)2

= b(x, t)2(dz)2 + h.o.t. = b(x, t)2dt + h.o.t.

26

Motivation: Discrete to Continuous Time

Brownian Motion Approximation with discrete intervals in continuous time

Have continuous time world, but use discrete time steps ∆t. Want to see what happens as the steplength goes to zero, that is ∆t→ 0.

Process with jumps at discrete intervals:

• At every ∆t intervals, a process x(t) either goes up or down:

∆x ≡ x(t+ ∆t)− x(t) =

+h with prob p−h with prob q = 1− p

• Expectation of this process:

E(∆x) = ph+ q(−h) = (p− q)h

• Variance of this process

Denition of Variance of a random variable: V (∆x) = E [∆x− E∆x]2

We have: V (∆x) = E [∆x− E∆x]2

= E[(∆x)2

]− [E∆x]

2= 4pqh2

(Since)E[(∆x)2

]= ph2 + q (−h)

2= h2

Analysis of x(t)− x(0) in this process

• Time span of length t implies n = t∆t steps in x(t)− x(0)

=⇒ x(t)− x(0) is a binomial random variable

• General form for probability that x(t)− x(0) is a certain combination of h and −h:Probability that x(t)− x(0) = (k)(h) + (n− k)(−h)

is(nk

)pkqn−k

(where(nk

)= n!

k!(n−k)! )

• Expectation and Variance:

E [x(t)− x(0)] = n ((p− q)h) = t∆t (p− q)h (*)

∗ (n times the individual step ∆x. Since these are iid steps, the sum is n times eachelement in the sum)

V [x(t)− x(0)] = n(4pqh2

)= t

∆t4pqh2 (*)

∗ (Since all steps are iid: when we look at sum of random variables, where randomvariables are all independent, the variance of the sum is the sum of the variances)

We have binomial random variable. As we chop into smaller and smaller time steps, will converge tonormal density.

So far: we've taken the discrete time process, discussed the individual steps ∆x given a time step ∆t,and then we've aggregated across many time steps to an interval of length t, during which time thereare n steps

(n = t

∆t

), and we've observed that this random variable, x(t)−x(0), is a binomial random

27

variable with mean t∆t (p− q)h and variance t

∆t4pqh2. Our next job is to move from discrete number

of steps to smaller and smaller steps.

Calibrate h and p to give us properites we want as we take ∆t→ 0: [Linear drift and variance]

• Let:

h = σ√

∆t (*)

p = 12

[1 + α

σ

√∆t](*)

These Follow:

q = 1− p = 12

[1− α

σ

√∆t]

(p− q) = ασ

√∆t

• Where did this come from? (linking ∆t to h, p, q)We are trying to get to continuous timerandom walk with drift. Random walk has variance that increases linearly with t forcastinghorizon. Would like this property to emerge in continuous time.

Consider the variance (of the aggregate time random variable): t∆t4pqh

2.

∆t is going to zero; p and q are numbers that are around .5 so can think of them as constantsin the limit, 4 is constant, t is the thing we'd like everything to be linear with respect to

∗ So given all this, h must be proportional to√

∆t [then put in scaling parameter σ]

· But then, what must (p−q) be for drift to be linear in tE [x(t)− x(0)] = t

∆t (p− q)h?

(p− q) must also be proportional to√

∆t, which (subject to ane transformations)pins down p, q and (p− q).

∗ No other way to calibrate this for it to make sense as a continuous time random walk

• Expectation and Variance

E [x(t)− x(0)] =t

∆t(p− q)h =

t

∆t

(ασ

√∆t)(

σ√

∆t)

= αt︸︷︷︸linear drift

V [x(t)− x(0)] =t

∆t4pqh2 =

t

∆t4

(1

4

)(1−

(ασ

)2

∆t

)(σ2∆t

)= tσ2

(1−

(ασ

)2

∆t

)−→

σ2t︸︷︷︸linear variance

(as ∆t→ 0)

Intuition as ∆t→ 0

• As take ∆t to 0, mathematically converging to a continuous time random walk.

• At a point t,we have a binomial random variable. It is converging to a normal randomvariable with mean αt and var σ2t.

• [These are the four graphs simulating the process, which appear closer and closer to Brownianmotion as ∆t gets smaller]

28

• Hence this Random Walk, a continuous time stochastic process, is the limit case of a family ofstochastic processes

Properties of Limit Case (Random Walk):

1. Vertical movements proportional to√

∆t (not ∆t)

2. [x(t)− x(0)]→D N(αt, σ2t), since Binomial→D Normal

3. Length of curve during 1 time period > nh = 1∆tσ√

∆t = σ 1√∆t→∞

(a) (Since length will be greater than the number of step sizes, time the length of each stepsize)

(b) Hence length of curve is innity over any nite period of time

4. ∆x∆t = ±σ

√∆t

∆t = ±σ∆t → ±∞

(a) Time derivative, E(dxdt

), doesn't exist

(b) So we can't talk about expectation of the derivative. However, we can talk about thefollowing derivative-like objects (5,6):

5. E(∆x)∆t = (p−q)h

∆t =(ασ√

∆t)(σ√

∆t)∆t = α

(a) So we write E (dx) = αdt

i. That is, in a period ∆t, you expect this process to increase by α

6. V (∆x)∆t =

4( 14 )

(1−(ασ )

2∆t

)σ2∆t

∆t → σ2

(a) So we write V (dx) = σ2dt

• Finally we see what the process is everywhere continuous, nowhere dierentiable

When we let ∆t converge to zero, the limiting process is called a continuous time random walk with(instantaneous) drift α and (instantaneous) variance σ2.We generated this continuous-time stochastic process by building it up as a limit case.We could have also just dened the process directly as opposed to converging to it:

Note that ”x” above becomes ”z” for following :

Wiener Process

Weiner processes: Family of processes described before with zero drift, and unit variance per unit timeDenition

If a continuous time stoachastic process z(t) is a Wiener Process, then any change in z, ∆z, corre-sponding to a time interval ∆t, satises the following conditions:

1. ∆z = ε√

∆t where ε ∼ N(0, 1)

29

(a) [often also stated ∆z ∼ N(0,∆t)]

(b) ε is the gausian piece, and√

∆t is the scaling piece

2. Non overlapping increments of ∆z are independent

(a) A.K.A. If t1 ≤ t2 ≤ t3 ≤ t4 then E [(z(t2)− z(t1)) (z(t4)− z(t3))] = 0

A process tting these two properties is a Weiner process. It is the same process converged to in therst part of the lecture above.

Properties

• A Wiener process is a continuous time random walk with zero drift and unit variance

• z(t) has the Markov property: the current value of the process is a sucient statistic for thedistribution of future values

History doesn't matter. All you need to know to predict in y periods is where you are rightnow.

• z(t) ∼ N(z(0), t) so the variance of z(t) rises linearly with t

(Zero drift case)

Note on terminology :Wiener processes are a subset of Brownian motion (processes), which are a subset of Ito Processes.

Ito Process

Now we generalize. Let drift (which we'll think of as a with arguments x and t, a(x, t)) depend ontime t and state x.[Let lim∆t→0

E∆x∆t equal to a function a with those two arguments]

Similarly, generalize variance.

From Wiener to Ito Process

Let z(t) be a Wiener Process. Let x(t) be another continuous time stocastic process such that

lim∆t→0E∆x∆t = a (x, t) i.e. E (dx) = a (x, t) dt "drift"

lim∆t→0V∆x∆t = b (x, t)

2i.e. V (dx) = b (x, t)

2dt "variance"

Note that the i.e. equations are formed by simply bringing the dt to the RHS.We summarize these properties by writing the expression for an Ito Process:

dx = a(x, t)dt+ b(x, t)dz

• This is not technically an equation, but a way of saying

• Drift and Standard Deviation terms: dx = a(x, t)dt︸ ︷︷ ︸drift

+ b(x, t)dz︸ ︷︷ ︸standard deviation

30

We think of dz as increments of our Brownian motion from Weiner process/limit case ofdiscrete time process.

What is telling us about variance is whatever term is multiplying dz. The process changesas z changes, and what you want to know is how does a little change in zthis backgroundWeiner processcause a change in x. In this case, an instantaneous change in z causes achange in x with scaling factor b.

• Deterministic and Stochastic terms: dx = a(x, t)dt︸ ︷︷ ︸deterministic

+ b(x, t)dz︸ ︷︷ ︸stochastic

Ito Process: 2 Key Examples

• Random Walk

dx = αdt+ σdz

Random Walk with drift α and variance σ2

• Geometric Random Walk

dx = αxdt+ σxdz

Geometric Random Walk with proportional drift α and proportional variance σ2

∗ Drift and variance are now dependent on the state variable x

∗ Can also think of as dxx = αdt+ σdz, so percent change in dx is a random walk

**Note on Geometric Random Walk: If it starts above zero, if will never become negative(or even become zero): even with strongly negative drift, given its proportionality to x theprocess will be driven asymptotically to zero at the lowest.

Laibson: Asset returns are well approximated by Geometric Random Walk

Ito's Lemma

Motivation is to work with functions that take Ito Process as an argument

Ito's Lemma

Basically just a taylor expansion

Theorem: Let z(t) be a Wiener Process. Let x(t) be an Ito Process with dx = a(x, t)dt + b(x, t)dz.Let V = V (x, t), then

dV =∂V

∂tdt+

∂V

∂xdx+

1

2

∂2V

∂x2b(x, t)2dt

31

=

[∂V

∂t+∂V

∂xa(x, t) +

1

2

∂2V

∂x2b(x, t)2

]dt+

∂V

∂xb(x, t)dz

[the second line simply follows from expanding dx = a(x, t)dt+ b(x, t)dz]

Motivation: Work with Value Functions (which will be Ito Processes) that take ItoProcesses as Arguments

• We are going to want to work with functions (value functionsthe solutions of Bellman equations)that are going to take as an argument, an Ito Process

• Motivation is to know what these functions look like, that take x as an argument, where x is anIto Process

More specically, we'd like to know what the Ito Process looks like that characterizes thevalue function V .

∗ Value function V is 2nd Ito Process, which takes an Ito Process (x) as an argument

· Not only want to talk about Ito Process x, but also about Ito Process V

• Since we are usually looking for a solution to a question: i.e. Oil Well

Price of oil is given by x

The Ito Process dx = a(x, t)dt + b(x, t)dz describes the path of x, the price of oil (not theoil well)

∗ e.g. price of oil is characterized by geometric Brownian motion: dx = αxdt+ σxdz

V (x, t) is value of Oil Well, which depends on the price of oil x and time

∗ Want to characterize the value function of the oil well, V (x, t), where x follows an ItoProcess

· Ito Process that characterizes value function over time: dV = a(V, x, t)dt+b(V, x, t)dz

· But we will see that the dependence of a and b on V will drop

We want to write the stochastic process which describes the evolution of V :

dV = a(x, t)dt+ b(x, t)dz

∗ which is called the total dierential of V

∗ Note that the hat terms (a, b) are what we are looking for in this stochastic process,which are dierent than the terms in the Ito Process (a, b)

• Ito's Lemma gives the solution to this problem in the following form:

dV =∂V

∂tdt+

∂V

∂xdx+

1

2

∂2V

∂x2b(x, t)2dt =

[∂V

∂t+∂V

∂xa(x, t) +

1

2

∂2V

∂x2b(x, t)2

]dt+

∂V

∂xb(x, t)dz where:

dV =

[∂V

∂t+∂V

∂xa(x, t) +

1

2

∂2V

∂x2b(x, t)2

]︸ ︷︷ ︸

a

dt+∂V

∂xb(x, t)︸ ︷︷ ︸b

dz

32

Proof of Ito's Lemma

Proof: Using a Taylor Expansion

dV =∂V

∂tdt+

1

2

∂2V

∂t2(dt)2 +

∂V

∂xdx+

1

2

∂2V

∂x2(dx)2 +

∂2V

∂x2dxdt+ h.o.t.

• Any term of order (dt)32 or higher is small relative to terms of order dt.

[This is because (∆t)2 is innitely less important than (∆t)32 , which is innitely less impor-

tant than (∆t)]

• Note that (dz)2

= dt. Since ∆z ∝√

∆t, recalling ∆z ≈ h [there's a Law of Large Numbersmaking this true, this is math PhD land and not something Laibson knows or expects us toknow]

Hence we can eliminate:

∗ (dt)2

= h.o.t.

∗ dxdt = a(x, t)(dt)2 + b(x, t)dzdt = h.o.t.

∗ (dx)2

= b(x, t)2(dz)2 + h.o.t. = b(x, t)2dt + h.o.t.

Combining these results we have:

∂V

∂tdt+

∂V

∂xdx+

1

2

∂2V

∂x2b(x, t)2dt

dz is a Wiener process that depends only on t through random motiondx is a Ito process that depends on x and t through random motionV (x, t) is a value function (and an Ito Process) that depends on a Ito process dx and directly on t

Intuition: Drift in V through Curvature of V :

• Even if we assume a(x, t) = 0 [no drift in Ito Process] and assume ∂V∂t = 0 [holding x xed, V

doesn't depend on t]

• We still have E(dV ) = 12∂2V∂x2 b(x, t)

2dt 6= 0

If V is concave, V is expected to fall due to variation in x [convex → rise]

∗ Ex: V (x) = lnx, hence V ′ = 1x and V ′′ = − 1

x2

· dx = αxdt+ σxdz

· dV =[∂V∂t + ∂V

∂x a(x, t) + 12∂2V∂x2 b(x, t)

2]dt+ ∂V

∂x b(x, t)dz

· =[0 + 1

xαx−1

2x2 (σx)2]dt+ 1

xσxdz

· =[α− 1

2σ2]dt+ σdz

=⇒ Growth in V falls below α due to concavity of V

• Intuition for proof : for Ito Processes, (dx)2behaves like b(x, t)2dt, so the eect of concavity is

of order dt and can not be ignored when calculating the dierential of V

Because of Jensen's inequality, if you are moving on independent variable (x) axis randomly,expected value will go down

33

Lecture SummaryTalked about a continuous time process (Brownian motion, Weiner process or its generalization asan Ito Process) that has these wonderful properties that is kind of like the continuous time analog ofthings in discrete time are thought of as random walks. But it is even more general that this:Has perverse property that it moves innitely far over unit time, but when you look at how it changesover any discrete interval of time, it looks very natural.Also learned how to study functions that take Ito Processes as arguments: key tool in these processesis Ito's Lemma. It uses a 2nd order taylor expansion and throws out higher order terms. So we cancharacterize drift in that function V very simply and powerfully, which we put to use in next lectures.

Lecture 9

Continuous Time Bellman Equation & Boundary Conditions

Continuous Time Bellman Equation

Final Form:

ρV (x, t)dt = maxuw(u, x, t)dt+ E [dV ]

Inserting Ito's Lemma

ρV (x, t)dt = maxu

w(u, x, t)dt+

[∂V

∂t+∂V

∂xa+

1

2

∂2V

∂x2b(x, t)2

]dt

Next 1.5 lectures about solving for V using this equation with Ito's Lemma. First, Merton. Then,stopping problem.

• Intuition

ρV (x, t)dt︸ ︷︷ ︸required rate of return

= maxu

w(u, x, t)dt︸ ︷︷ ︸instantaneous dividend

+ E [dV ]︸ ︷︷ ︸instantaneous capital gain

required rate of return = instantaneous return; instantaneous capital gain = instantaneouschange in the program.

• Derivation

Towards Condensed Bellman

∗ Let w(x, u, t) = instantaneous payo function

· Arguments: x is state variable, u is control variable, t is time

∗ Let x′ = x+∆x and t′ = t+∆t . We'll end up thinking about what happens as ∆t→ 0

· Just write down Bellman equation as we did in discrete time

34

V (x, t) = maxu

w(x, u, t)∆t+ (1 + ρ∆t)

−1EV (x′, t′)

V (x, t) = maxu

w(x, u, t)∆t︸ ︷︷ ︸ow payo ×time step

+ (1 + ρ∆t)−1EV (x′, t′)︸ ︷︷ ︸

discounted value of continuation

∗ 3 simple steps algebra with (1 + ρ∆t) term:

(1 + ρ∆t)V (x, t) = maxu(1 + ρ∆t)w(x, u, t)∆t+ EV (x′, t′)

ρ∆tV (x, t) = maxu(1 + ρ∆t)w(x, u, t)∆t+ EV (x′, t′)− V (x, t)

ρ∆tV (x, t) = maxu

w(x, u, t)∆t+ ρw(x, u, t) (∆t)

2+ EV (x′, t′)− V (x, t)

∗ Let ∆t→ 0 . Terms of order (dt)

2= 0

ρV (x, t)dt = maxuw(x, u, t)dt+ E [dV ] (*)

Substitute using Ito's Lemma

∗ Substitute for E [dV ]

· where

dV =

[∂V

∂t+∂V

∂xa(x, u, t) +

1

2

∂2V

∂x2b(x, u, t)2

]dt+

∂V

∂xb(x, u, t)dz

(since E

[∂V

∂xbdz

]= 0

)

E [dV ] =

[∂V

∂t+∂V

∂xa+

1

2

∂2V

∂x2b2]dt

∗ Substituting this expression into (*) we get

ρV (x, t)dt = maxu

w(u, x, t)dt+

[∂V

∂t+∂V

∂xa+

1

2

∂2V

∂x2b(x, t)2

]dt

· which is a partial dierential equation (in x and t)

• Interpretation

Sequence Problem:

ˆ ∞t

e−ρtw (x(t), u(t), t) dt

Interpretation of Bellman (*) before substuting for Ito's Lemma:

ρV (x, t)dt︸ ︷︷ ︸required rate of return

= maxu

w(x, u, t)︸ ︷︷ ︸instantaneous dividend

+ E [dV ]︸ ︷︷ ︸instantaneous capital gain

35

∗ note that instantaneous capital gain ≡ instantaneous change in the program

∗ And when we plug in Ito's Lemma

∗Instantaneous capital gain =

[∂V

∂t+∂V

∂xa+

1

2

∂2V

∂x2b(x, t)2

]

Application 1: Merton's Consumption

• Merton's Consumption

Consumer has 2 assets

∗ Risk free: return r

∗ Equity: return r + π with proportional variance σ2

Invests share θ in assets, and consumes at rate c

• Ito Process that characterizes dynamics of x, her cash on hand:

dx = [rx+ πxθ − c] dt+ θxσdz

Intuition

∗ Drift term: earn rxdt every period on assets. A fraction θx is getting an additionalequity premium π (πxθdt). Consuming at rate c, so depletes assets at rate cdt.

∗ Standard Deviation term: cost to holding equities is volatility. Volatility scales withthe amount of assets in risky category: θxσdz.

So, a = [rx+ πxθ − c] and b = θxσ

aka equation of motion for the state variable

• Distinguishing Variables

x is a stock (not in the sense of equity, but a quantity)

c is a ow (ow means rate).

∗ Need to make sure units match (usually use annual in economics)

∗ i.e. spending $1000/48hrs is ~$180,000/year

· He can easily consume at a rate that is above his total stock of wealth

∗ c is bounded between 0 and ∞· Not between 0 and x as it was in discrete time

• Bellman Equation

ρV (x, t)dt = maxc,θ

w(u, x, t)dt+

[∂V

∂x[rx+ πxθ − c] +

1

2

∂2V

∂x2(θxσ)2

]dt

Note the maximization over both c and θ, so two FOCs

∗ In world of continuous time, these can be changed instantaneously.

36

· (In principle, in every single instant, could be picking a new consumption level cand asset allocation level θ)

· It will turn out that in CRRA world, he will prefer a constant asset allocation θ,even though he is constantly changing his consumption

V doesn't depend directly on t, so rst term was removed

∗ If someone was nitely lived, then function would depend on t

dt can be removed since notationally redundant

Solving with CRRA utility u(c) = c1−γ

1−γ [Lecture]

• It turns out that in this problem, the value function inherits similar properties to the utilty

function: V (x) = ψ x1−γ

1−γ

• If given utility function is CRRA u(c) = c1−γ

1−γ , then it can be proved that V (x) = ψ x1−γ

1−γ

This is just a scaled version of the utility function (by ψ)

This gives us single solution to Bellman equation

• Solving for Policy Functions [given value function V (x) = ψ x1−γ

1−γ ]

• Assume V (x) = ψ x1−γ

1−γ : nd values for policy functions that solve Bellman Equation

We need to know ψ, θ and optimal consumption policy rule for c

The restriction V (x) = ψ x1−γ

1−γ allows us to do this

• Taking V (x) = ψ x1−γ

1−γ , re-write Bellman plugging in for u(c) = c1−γ

1−γ ,∂V∂x ,

∂2V∂x2

ρψx1−γ

1− γdt = max

c,θ

u(c)dt+

[ψx−γ [(r + θπ)x− c]− γ

2ψx−γ−1(θσx)2

]dt

ρψx1−γ

1− γ= max

c,θ

c1−γ

1− γ+[ψx−γ [(r + θπ)x− c]− γ

2ψx−γ−1(θσx)2

]• Two FOCs, θ and c :

FOCθ : ψx−γπx− 2(γ2

)ψx−γ−1θ(σx)2 = 0

FOCc : u′(c) = c−γ = ψx−γ

• Simplifying

c = ψ−1γ x

θ = πγσ2

Interpretation:

Nice result: Put more into equities the higher the equity premium π; put less into equitieswith higher RRA γ and higher variability of equities σ2

Why is variance of equities replacing familiar covariance term from Asset Pricing lecture?In Merton's world, all the stocasticity is coming from equity returns.

37

• Plugging back in, we get:

ψ−1γ = ρ

γ +(

1− 1γ

)(r + π2

2γσ2

)• Special case: γ = 1 implies c = ρx

So MPC ' .05 (Marginal Propensity to Consume)

But we also get θ = πγσ2 = 0.06

1(0.16)2 = 2.34

Saying to borrow money so have assets of 2.34 in equity [and therefore 1.34 in liabilities]We don't know what's wrong here. This model doesn't match up with what wise souls think youshould do.

• Can we x by increasing γ? Remember no one in classroom had γ ≥ 5

With γ = 5, θ = 0.065(0.16)2 = 0.47

∗ Recommends 47% allocation into equities

Typical retiree:No liabilities; $600,000 of social security (bonds), $200,000 in house, less than $200,000 in 401(k)401(k) usually 60% bonds and 40% stocksEquity allocation is 8% of total assets

Solving with Log utility u(c) = ln(c) [PS5]

• Given that guess for value function is V (x) = ψ ln(x) + φ

• Set up is the same: ρV (x, t)dt = maxc,θ

w(u, x, t)dt+

[∂V∂x [rx+ πxθ − c] + 1

2∂2V∂x2 (θxσ)2

]dt

• When substitute for V (x) and u(c):

ρ [φ+ ψ ln(x)] = maxc,θ

ln(c) + [(r + θπ)x− c]ψx−1 − 1

2ψ(θσx)2ψx−2

• FOCs:

FOCc : c = 1ψx

FOCθ : θ = πσ2

• Plugging back into Bellman

ρφ+ ρψ ln(x) = ψr − ln(ψ)− 1 + ψπ2

2σ2 + ln(x)

• Observation to solve for parameters

We can re-write the above with just constants on RHS: ρψ ln(x)− ln(x) = ψr− ln(ψ)− 1 +ψπ2

2σ2 − ρφ∗ Given that the RHS is constant, this will only hold if ρψ = 1. [Otherwise LHS wouldbe increasing in x, while RHS won't increase, a contradiction]

· Hence, ρψ = 1 and 0 = ψr − ln(ψ)− 1 + ψπ2

2σ2 − ρφ. This means:

The Bellman equation is satised ∀x i

∗ ρψ = 1 and ρφ = ψr − ln(ψ)− 1 + ψπ2

2σ2

38

Giving us

∗ ψ = 1ρ

∗ φ = 1ρ

(rρ + ln(ρ)− 1 + π2

2ρσ2

)• Optimal Policy

Plugging back into FOCs

∗ c = ρx and θ = πσ2

General Continuous Time Problem

Most General Form

• Equation of motion of the state variable

dx = a(x, u, t)dt+ b(x, u, t)dz,

Note dx depends on choice of control u

• Using Ito's Lemma, Continuous Time Bellman Equation

ρV (x, t) = w(x, u∗, t) + ∂V∂t + ∂V

∂x a(x, u∗, t) + 12∂2V∂x2 b(x, u

∗, t)2

u∗ = u(x, t) =optimal value of control variable

Properties: PDE and Boundary Conditions

• Value function is a 2nd order PDE

• V is the 'dependent' variable; x and t are the 'independent' variables

• PDE will have continuum of solutions

Need structure from economics to nd the right solution to this equationthe solution themakes economic sensefrom the innite number of solutions that make mathematical sense.

=⇒ These restrictions are called boundary conditions, which we need to solve PDE

=⇒ Need to use economic logic to derive boundary conditions

Name of the game in these problems is to use economic logic to nd right solution, andeliminate all other solutions

(Any policy generates a solution, we're not looking for any solution that corresponds to anypolicy, but for the solution that corresponds to the optimal policy)

• [In Merton's problem, we exploited 'known' functional form of value function, which imposesboundary conditions]

Two key examples of boundary conditions in economics

1. Terminal Condition

• Suppose problem ends at date T

39

i.e. have already agreed to sell oil well on date T

• Then we know that V (x, T ) = Ω(x, T ) ∀x [Note this is at date T ]

Where Ω(x, t) is (some known, exogenous) termination function. Note that it can bestate dependenti.e. change with the price of oil.

• Can solve problem using techniques analogous to backward induction

• This restriction gives enough structure to pin down solution to the PDE

2. Stationary ∞−horizon problem

• Value function doesn't depend on t =⇒ becomes an ODE

ρV (x) = w(x, u∗) + a(x, u∗)V ′ +1

2b(x, u∗)2V ′′

• Note the use of V ′ since now only one argument for V , can drop the ∂ notation

• We will use Value Matching and Smooth Pasting

Motivating Example for Value Matching and Smooth Pasting:

• Consider the optimal stopping problem:

V (x, t) = maxw(x, t)∆t+ (1 + ρ∆t)−1EV (x′, t′),Ω(x, t)

• where Ω(x, t) is the termination payo in the stopping problem

the solution is characterized by the stopping rule:

∗ if x > x∗(t) continue; if x ≤ x∗(t) stop

∗ Motivation: i.e. irreversibly closing a production facility

Assume that dx = a(x, t)dt+ b(x, t)dz

In continuation region, value function characterized using Ito's Lemma:

∗ ρV (x, t)dt = w(x, t)dt+[∂V∂t + ∂V

∂x a(x, t) + 12∂2V∂x2 b(x, t)

2]dt

Value Matching

• Value Matching:

All continuous problems must have continuous value functions

V (x, t) = Ω(x, t)

for all x, t such that x(t) ≤ x∗(t)

(at boundary and to left of boundary)

40

A discontinuity is not permitted, otherwise one could get a better payo from not followingthe policy

Intuition: an optimal value function has to be continuous at the stopping boundary.

∗ If there was a discontinuity, can't be optimal to quit (if V is above Ω) or to havecontinued so far (if V below Ω)

See graph [remember V on y axis, x on x-axis]

Smooth Pasting

Derivatives with respect to x must be equal at the boundary.Using problem above:

• Smooth Pasting:

Vx (x∗(t), t) = Ωx (x∗(t), t)

Intuition

∗ V and Ω must have same slope at boundary.

· Proof intuition: if slopes are not equal at convex kink, then can show that thepolicy of stopping at the boundary is dominated by the policy of continuing forjust another instant and then stopping. (It doesn't mean the latter is the optimalpolicy, but shows that initial policy of stopping at the boundary is sub-optimal)

∗ If value functions don't smooth paste at x∗(t), then stopping at x∗(t) can't be optimal:

∗ Better to stop an instant earlier or later

· If there is a (convex) kink at the boundary, then the gain from waiting is in√

∆tand the cost from waiting is in ∆t. So there can't be a kink at the boundary.

41

∗ Think of graph [remember V on y axis, x on x-axis]

Will need a 3rd boundary condition from the following lecture

Lecture 10

Third Boundary Condition, Stopping Problem and ODE Strategies

Statement of Stopping Problem

• Assume that a continuous time stochastic process x(t) is an Ito Process,

dx = adt+ bdz

i.e. x is the price of a commodity produced by this rm

• While in operation, rm has ow prot

`w(x) = x

(Or w(x) = x− c if there is a cost of production)

• Assume rm can costlessly (permanently) exit the industry and realize termination payo

Ω = 0

• Intuitively, rm will have a stationary stopping rule

if x > x∗ continue

if x ≤ x∗ stop (exit)

• Draw the Value Function (Ω = 0 in below graph)

42

Characterizing Solution as an ODE

• Overall Value Function

V (x) = maxx∆t+ (1 + ρ∆t)−1EV (x′), 0

Where x′ = x(t+ ∆t), and ρ is interest rate

• In stopping region

V (x) = 0

• In continuation region

General

ρV (x)dt = xdt+ E(dV )

Since: V (x) = x∆t+ (1 + ρ∆t)−1EV (x′)

=⇒ (1 + ρ∆t)V (x) = (1 + ρ∆t)x∆t+ EV (x′)

=⇒ ρ∆tV (x) = (1 + ρ∆t)x∆t+ EV (x′)− V (x).

Let ∆t→ 0 and multiply out. Terms of order (dt)2 = 0.

Substitute for E(dV ) using Ito's Lemma

∗ 2nd Order ODE:

ρV = x+ aV ′ +b2

2V ′′

Interpretation

∗ Value function must satisfy this dierential equation in the continuation region

· We've derived a 2nd order ODE that restricts the form of the value function V inthe continuation region

· 2nd order because highest order derivative is 2nd order, ordinary because onlydepends on one variable (x),

ρV︸︷︷︸required return

= x︸︷︷︸dividend

+

drift in argument︷︸︸︷aV ′ +

drift from jensen's inequality︷ ︸︸ ︷b2

2V ′′︸ ︷︷ ︸

capital gain =how V is instantly drifting

∗ Term b2

2 V′′ is drift resulting from brownian motion'jiggling and jaggling': it is negative

if concave, and positive if convex (as in this case).

• Intuition: x∗ almost always negative. [don't confuse with V (x) = Ω = 0. This is the value youget when stopping; but can decide on some policy of x for when to stop: x∗]

Why? Option value love the ability take the chance at future prots. Even if drift isnegative, will still stay in, since through Brownian motion may end up protable again.

• And now, we have continuum of solutions to ODE, so need restrictions to get to single solution

43

Third Boundary Condition

Boundary Conditions*Three variables we need to pin down: two constants in 2nd order ODE, and free boundary (x∗)

• Value Matching: V (x∗) = 0

• Smooth Pasting: V ′(x∗) = 0

• Third Boundary Condition for large x

As x→∞ the option value of exiting goes to zero

∗ It is so unlikely that prots would become negative (and if it happened, would likelybe very negatively discounted) that value of being able to shut down is close to zero

∗ As x gets very large, value function approaches that of the rm that does not have theoption to shut down (which would mean it would theoretically need to keep running atmajor losses)

Hence V converges to the value associated with the policy of never exiting the industry[equation derived later]

limx→∞

V (x)

1ρ

(x+ a

ρ

) = 1

∗ Denominator is value function of rm that can't shut down

· [Or can think of it as limx→∞ V (x) = 1ρ

(x+ a

ρ

)]

We can also write:

limx→∞

V ′(x) =1

ρ

∗ Intuition: at large x, how does value function change with one more unit of price x?

· Use sequence problem:

· Having one more unit of price produces same exact sequence shifted up by one unitall the way to innity. What is the additional value of a rm with price one unithigher?

· ˆe−ρtx(t)dt︸ ︷︷ ︸

old oil well

−ˆe−ρt(x′)(t)dt︸ ︷︷ ︸new oil well

=

ˆ ∞0

e−ρt · 1 · dt =1

ρ

· So if I raise price by 1, I raise value of rm by 1ρ

Solving 2nd Order ODEs

Denitions

• Goal: Solve for F with independent variable x

• Complete Equation

Generic 2nd order ODE:

44

F ′′(x) +A(x)F ′(x) +B(x)F (x) = C(x)

• Reduced Equation

C(x) is replaced by 0

F ′′(x) +A(x)F ′(x) +B(x)F (x) = 0

Theorem 4.1

• Any solution F (x), of the reduced equation can be expressed as a linear combination of any twolinearly independent solutions of the reduced equation, F1 and F2

F (x) = C1F1(x) + C2F2(x) provided F1, F2linearly independent

Note that two solutions are linearly independent if there do not exist constants A1 and A2

such that

A1F1(x) +A2F2(x) = 0 ∀x

Theorem 4.2

• The general solution of the complete equation is the sum of

any particular solution of the complete equation

and the general solution of the reduced equation

Solving Stopping Problem

1. State Complete and Reduced Equation

• Complete Equation

Dierential equation that characterizes continuation region

ρV = x+ aV ′ +b2

2V ′′

• Reduced equation

0 = −ρV + aV ′ +b2

2V ′′

2. Guess Form of Solution to nd General Solution to Reduced Equation

45

• First challenge is to nd solutions of the reduced equation

• Consider class

erx

• To conrm this is in fact a solution, dierentiate and plug in to nd:

0 = −ρerx + arerx +b2

2r2erx

This implies (dividing through by erx)

0 = −ρ+ ar +b2

2r2

• Apply quadratic formula

r =−a±

√a2 + 2b2ρ

b2

• Let r+ represent the positive root and let r− represent the negative root

• General solution to reduced equation

Any solution to the reduced equation can be expressed

C+er+

+ C−er−

3. Find Particular Solution to Complete Equation

• Want a solution to the complete equation ρV = x+ aV ′ + b2

2 V′′

• Consider specic example: payo function of policy never leave the industry

• The value of this policy is

E

ˆ ∞0

e−ρtx(t)dt

• In solving for value of the policy without the expectation, we'll also need to know E [x(t)]:

E [x(t)] = E

[x(0) +

ˆ t

0

dx(t)

]

= E

[x(0) +

ˆ t

0

[adt+ bdz(t)]

]= E

[x(0) + at+

ˆ t

0

bdz(t)

][´ t

0bdz(t) = 0 as it is the Brownian motion term]. Hence:

E [x(t)] = x(0) + at

• The value of the policy in V (x) = form, without the expectation

46

Derived by integration by parts

ˆudv = uv −

ˆvdu

Taking E´∞

0e−ρtx(t)dt:

E

ˆ ∞0

e−ρtx(t)dt = E

ˆ ∞0

e−ρt [x(0) + at] dt

= E

ˆ ∞0

e−ρt [x(0)] dt+

ˆ ∞0

e−ρt [at] dt =x(0)

ρ+ E

ˆ ∞0

e−ρt [at] dt

Think of dv = e−ρt and u = at

Then v = − 1ρe−ρt and du = adt

So if´udv =

´∞0e−ρt(at)dt then uv −

´vdu = [−at 1

ρe−ρt]∞0 −

´∞0− 1ρe−ρtadt

Note − 1ρe−ρt]∞0 = 1

ρ

= [−at 1ρe−ρt]∞0 − [ 1

ρ2 e−ρta]∞0 = 0 + a 1

ρ2

=1

ρ

(x(0) +

a

ρ

)• Hence our candidate particular solution takes the form

V (x) =1

ρ

(x+

a

ρ

)• Conrm that this is a solution to the complete equation ρV = x+ aV ′ + b2

2 V′′

ρ

[1

ρ

(x+

a

ρ

)]= x+

a

ρ= x+ a

(1

ρ

)+b2

2· 0

4. Put Pieces Together

• General solution to reduced equation

C+er+x + C−er

−x with roots r+, r− =−a±

√a2 + 2b2ρ

b2

• Particular solution to general equation

1

ρ

(x+

a

ρ

)• Hence the general solution to the complete equation is

[Sum of particular solution to complete equation and general solution to reduced equation]

V (x) =1

ρ

(x+

a

ρ

)+ C+er

+x + C−er−x

47

• Now just need to apply boundary conditions

5. Apply Boundary Conditions

• 3 Conditions

Value Matching: V (x∗) = 0

Smooth Pasting: V ′(x∗) = 0

limx→∞ V ′(x) = 1ρ

• This gives us three conditions on the general solution

C+ = 0 [from limx→∞ V ′(x) = 1ρ ]

∗ Which then implies:

V (x∗) = 1ρ

(x∗ + a

ρ

)+ C−er

−x∗ = 0 [Value Matching] (1)

V ′(x∗) = 1ρ + r−C−er

−x∗ = 0 [Smooth Pasting] (2)

• Simplifying from (1) and (2)

Plugging (1) into (2)

∗ (1) implies C−er−x∗ = − 1

ρ

(x∗ + a

ρ

)· Plugging this into (2) we get

∗ 1ρ − r

− 1ρ

(x∗ + a

ρ

)= 0

Simplifying: 1 = r−(x∗ + a

ρ

)=⇒ 1

r− = x∗ + aρ

∗ Gives us x∗ = 1r− −

aρ

Plugging in for r− =−a−√a2+2b2ρ

b2

We get our solution for x∗

x∗ = − b2

a+√a2 + 2b2ρ

− a

ρ< 0 (∗)

The general solution to the complete equation can be expressed as:

• the sum of a particular solution to the complete dierential equation and the general solution tothe reduced equation

Note: this is usually done in search problems by looking for the particular solution of thepolicy never leave. Then you can get a general solution, and then use value matching andsmooth pasting to come up with the threshold value.

48

Lecture 12

Discrete Adjustment - Lumpy Investment

Lumpy Investment Motivation

• Plant level data suggests individual establishments do not smooth adjustment of their capitalstocks.

Instead, investment is lumpy. Ex rms build new plant all at once, not a little bit each year

• Doms & Donne 1993: 1972-1989:largest investment episode, rm i

total investment, rm i. If investment were spread

this ratio would be 118 .

Instead average value was 14 . Hence, on average rms did 25% of investment in 1 year over

18 year period

• We go from convex to ane cost functions:

smooth convex cost functions assume small adjustments are costlessly reversible

use ane functions, where small adjustments are costly to reverse

Lumpy Investment

Capital Adjustment Costs Notation

• Capital adjustment has xed and variable costs

Upward Adjustment Cost: CU + cuI

∗ CU is xed cost, cU is variable cost

Downward Adjustment Cost: CD + cD (−I).

∗ CD + cD |I| is the general case. Here we usually assume cD > 0 and I < 0, so we getCD + cD (−I)

∗ CD is xed cost, cD is variable cost

Firm's Problem

• Firm loses prots if the actual capital stock deviates from target capital stock x∗

Deviations (x− x∗) generate instantaneous negative payo

• Functional form:

− b2

(x− x∗)2= − b

2X2

where X = (x− x∗).

49

• X is an Ito Process between adjustments:

dX = αdt+ σdz

where dz are Brownian increments

Value Function Representation

• Overall Value Function

V (X) = maxE

ˆ ∞τ=t

e−ρ(τ−t)(− b

2X2τ

)dτ −

∞∑n=1

e−ρ(τ(n)−t)A (n)

A (n) =

CU + cUIn if In > 0CD + cD |In| if In < 0

Notation:

τ (n) = date of nth adjustment; A (n) = cost of nth adjustment; In = investment in nthadjustment

∗ Below x∗: Adjust capital at X = U . Adjust to X = u.

∗ Above x∗: Adjust capital at X = D. Adjust to X = d.

• Value Function in all 3 Regions

1. Between Adjustments:

Bellman Equation

ρV (X) = − b2X2 + αV ′ (X) +

1

2σ2V ′′ (X)

∗ Using Ito's Lemma: E [dV ] =[∂V∂t + ∂V

∂x a+ 12∂2V∂x2 b

2]dt

Functional form solved for below [PS6 2.d]

2. Action Region Below

[If X ≤ U ]

V (X) = V (u)− [CU + cU (u−X)]

This implies

V ′ (X) = cU ∀X ≤ U

1. Action Region Above

[If X ≥ D]

V (X) = V (d)− [CD + cD (X − d)]

This implies

V ′ (X) = −cD ∀X ≥ D

50

Boundary Conditions

1. Value Matching

limX↑U

V (X) = V (u)− [CU + cU (u− U)] = V (U) = limX↓U

V (X)

limX↓D

V (X) = V (d)− [CD + cD (D − d)] = V (D) = limX↑D

V (X)

2. Smooth Pasting

limX↑U

V ′ (X) = cU = V ′ (U) = limX↓U

V ′ (X)

limX↓D

V ′ (X) = −cD = V ′ (D) = limX↑D

V ′ (X)

First Order Conditions

• Intuition: When making capital adjustment, willing to move until the marginal value of moving= marginal cost

This gives us two important optimality condtitions:

V ′ (u) = cU

V ′ (d) = −cD

51

Solving for Functional Form of V (X) in Continuation RegionPS6 2.d

Question: Show that V (X) = − b2

(X2

ρ + σ+2αXρ2 + 2α2

ρ3

)is a solution to the continuation region equa-

tion: ρV (X) = − b2X

2 + αV ′ (X) + 12σ

2V ′′ (X).Show that this is the expected present value of the rm's payo stream assuming that adjustmentcosts are innite.

This is guess and check, where we've been provided the guess:

V (X) = − b2

(X2

ρ+σ + 2αX

ρ2+

2α2

ρ3

)Calculate V ′(X) and V ′′(X):

V ′(X) = − b2

(2X

ρ+

2α2

ρ2

), V ′′(X) = − b

ρ

We plug back into the continuous time Bellmann Equation from V (X) = 1ρ

[−b2 X

2 + αV ′(X) + 12σ

2V ′′(X)]:

V (X) =1

ρ

[−b2X2 − αb

2

(2X

ρ+

2α2

ρ2

)− 1

2σ2 b

ρ

]= − b

2ρ

[X2 +

2αX

ρ+σ2

ρ+

2α

ρ2

2]

52

= − b2

[X2

ρ+σ2 + 2αX

ρ2+

2α

ρ3

]So our guessed solution works as a solution to the continuous Bellmann.

We also know that this works as the expected present value of the rm's payo stream, given that thisis the equivalent to the solution to the continuous time Bellman for the continuation region, becausethe innite adjustment costs mean that u = −∞ and d =∞, hence the solution will hold for X ∈ R.

Appendix: Math Reminders

Innite Sum of a Geometric Series with |x| < 1:

∞∑k=0

xk =1

1− x

L'Hopital's Rule

If limx→c

f(x) = limx→c

g(x) = 0 or ±∞

and limx→c

f ′(x)

g′(x)exists

and g′(x) 6= 0

then limx→c

f(x)

g(x)= limx→c

f ′(x)

g′(x)

Probability

Expectation of RV:E(exp(a)) = exp[E[a] + 1

2V ar[a]].When a is a random variable, or anythingx ∼ N(µ, σ2) −→Ax+B ∼ N(Aµ+B,A2σ2)−→ E [exp (Ax+B)] = exp

(Aµ+B + 1

2A2σ2)

[Since x ∼ N(µ, σ2)−→ E(exp(x)) = exp(E(x) + 12V ar(x)); so if standard normal: = exp(µ+ 1

2σ2)]

In case of Asset pricingRti has stochastic returnsWe have: Rti = exp(rti + σiεit − 1

2 [σi]2)where εit has unit variance. Hence ε

it is a normally distributed random variable: εit ∼ N(0, 1)

From above we see that this implies: σiεit + rti − 12 [σi]2 ∼ N(σi ∗ 0 + rti − 1

2 [σi]2, σi2 ∗ 12) = N(rti −12 [σi]2, σi2)

53

Then:E[Rti] = E[exp(rti + σiεit − 1

2 [σi]2)] which as we've seen is now exp(a random variable) so we can use= exp(mean+ 1

2V ar) = exp(rti − 12 [σi]2 + 1

2σi2)

= exp(rti)

Covariance:If A and B are random variables, then

V (A+B) = V (A) + V (B) + 2Cov(A,B)

Log Approximation:For small x:ln(1 + x) ≈ x1 + x ≈ ex

54