Topic 1: Introduction to Intertemporal Optimizationyluo/teaching/econ6012/2013topic1a.pdf ·...

Topic 1: Introduction to Intertemporal Optimization

Yulei Luo

SEF of HKU

September 6, 2013

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 1 / 56

Warm-Up: Intertemporal/Dynamic Optimization

In static optimization, the task is to find a single value for eachcontrol variable, such that the objective function will be maximized orminimized. In contrast, in a dynamic setting, time enters explicitlyand we encounter a dynamic optimization problem. In such aproblem, we need to find the optimal time path of control and stateduring an entire planning period.

Specifically, at any time t, we have to choose the value of somecontrol variable, c(t), which will then affect the value of the statevariable, x(t + 1), via a state transition equation given the currentstate x(t).

Two leading methods to solve DO problems are: optimal control (OC)and dynamic programming (DP). They can be applied in deterministicor stochastic and discrete-time or continuous-time settings.


The Setting of Optimal Control (OC)

In the dynamic discrete-time model, the objective should take theform of summation from t = 0 to t = T :

max{c (t),x (t+1)}

T

∑t=0r (c (t) , x (t) , t) , (1)

subject to

x (t + 1)− x (t) = f (c (t) , x (t) , t) , (2)

x0 = x (0) , x (T + 1) ≥ 0 , (3)

for any t ≥ 0.


(conti.) As in the static optimization case, the Lagrangian for thisintertemporal (dynamic) problem is

L =T

∑t=0{r (ct , xt , t) + λt [f (ct , xt , t) + xt − xt+1]}+ λT+1xT+1,

(4)where λt is the shadow price (i.e., the Lagrange multiplier).

The FOCs for ct are:

∂L∂ct

= r ′c (ct , xt , t) + λt f ′c (ct , xt , t) = 0 for t = 0, 1, · · ·,T . (5)

The FOCs for xt+1 are complicated because each xt appears in twoterms in the Lagrangian function.


(conti.) Consider some relevant terms in the above Lagrangianfunction:

T

∑t=0

λt (xt − xt+1) = (λ0x0 − λT xT+1) +T−1∑t=0

xt+1 (λt+1 − λt ) .

The FOCs for xt+1:

∂L∂xt+1

= r ′x (ct+1, xt+1, t + 1) + λt+1f ′x (ct+1, xt+1, t + 1)

+λt+1 − λt = 0,

for t = 0, · · ·,T − 1.


Theorem (The Maximum Principle)

The necessary FOCs for maximizing (1) subject to (2) are:(i) ∂Lt

∂ct= 0 for all t = 0, · · ·,T .

(ii) xt+1 − xt = f (ct , xt , t) for all t (The state transition equation)(iii) r ′x (ct+1, xt+1, t + 1) + λt+1f ′x (ct+1, xt+1, t + 1) + λt+1 − λt = 0 forall t (The costate (λt ) equation)(iv) xT+1 ≥ 0,λT+1 ≥ 0, xT+1λT+1 = 0 (The transversality condition, orthe complementary-slackness condition)


Example: Optimal Consumption-Saving Model

Suppose that a consumer has the utility function u (ct ) , where ct isconsumption at time t. The consumer’s utility is concave:

u′ > 0, u′′ < 0, (6)

and u (ct ) satisfies the Inada conditions (They ensure thatconsumption will always be interior):

limc→0

u′ (c) = ∞ and limc→∞

u′ (c) = 0. (7)

The consumer is also endowed with a0 = a (0) , and has incomestream derived from holding the asset: yt = rat , where r is theinterest rate. The consumer uses the income to purchase c . Anyincome not consumed is added to the asset holdings as savings:

at+1 = (1+ r) at − ct . (8)


The Model Setup

The consumer’s lifetime utility maximization problem is to

max{ct ,at+1}

T

∑t=0

βtu (ct ) (9)

subject to

at+1 = Rat − ct , t = 0, · · ·,T , (10)

a0 = a (0) , aT+1 ≥ 0, (11)

where β is the consumer’s rate of time preference (β ≤ 1) andR = 1+ r is the gross interest rate.


(conti.) Since limc→0 u′ (c) = ∞, c (t) > 0 for all t ≤ T . Further, toensure that a solution exists, assets are constrained to be nonnegativeafter the terminal period T (i.e., when the consumer dies), aT+1 ≥ 0.Note that this nonnegativity constraint must bind, i.e., aT+1 = 0. Infact, this constraint serves two important purposes:

First, without this constraint aT+1 ≥ 0, the consumer would want toset aT+1 = −∞ and die with outstanding debts. This is clearly notfeasible.Second, the fact that the constraint is binding in the optimal solutionguarantees that resources are not being thrown away after T .


Using Optimal Control to Solve the Model

Setting up the Lagrangian:

L =T

∑t=0

βt {u(ct ) + λt [Rat − ct − at+1]}+ λT+1aT+1 (12)

The FOC for ct is ∂L∂ct= 0:

u′(ct )− λt = 0 for t = 0, 1, · · ·,T . (13)

The FOC for at+1 is ∂L∂at+1

= 0:

−λt + βRλt+1 = 0 for t = 0, · · ·,T − 1. (14)


(conti.) The terminal condition aT+1 ≥ 0 implies that thetransversality condition (the complementary slackness condition) is

λT+1 ≥ 0, aT+1 ≥ 0, aT+1λT+1 = 0, (15)

which means that either the asset holdings (a) must be exhausted onthe terminal date, or the shadow price of capital (λt ) must be 0 onthe terminal date. Since u′ > 0, the marginal value of capital (λ)cannot be 0 and thus the capital stock should optimally be exhaustedby the terminal date T + 1, i.e., aT+1 = 0.


(conti.) Combining (13) with (14) gives

u′(ct ) = βRu′(ct+1), (16)

which is called the Euler equation characterizing optimal consumptiondynamics.

Suppose that the utility function has an isoelastic form

u(ct ) =c1−γt − 11− γ

, (17)

where γ ≥ 0 (Note that when γ = 1, the function reduces to ln ct).We have

c−γt = βRc−γ

t+1 =⇒ct+1ct

= (βR)1/γ . (18)

Therefore, if βR > 1, the optimal consumption will rise over time; ifβR < 1, the optimal consumption will decline over time. WhenβR = 1, ct = ct+1.


(conti.) (18) implies that the elasticity of intertemporal substitution(EIS) is

d(ct+1ct

)/(ct+1ct

)dR/R

=d ln

(ct+1ct

)d lnR

=1γ. (19)

Hence, increasing γ(i.e., reducing 1

γ

)make the agent more unwilling

to postpone consumption (i.e., more unwilling to save). That’s whywe call this type of utility functions the isoelastic utility function.

The optimal problem is pinned down by a given initial condition (a0)and by a terminal condition (aT+1 = 0) . The sets of (T − 1) Eulerequations (18) and constraints (10) then determine the time path ofc as well as a.


Obtaining the Intertemporal Budget Constraint

(conti.) (10) implies that

a1 = Ra0 − c0,a2 = Ra1 − c1,

· · · · ·aT+1 = RaT − cT .

aT+1RT

+( cTRT

+ · · ·+ c1R+ c0

)= Ra0 =⇒

T

∑t=0

ctR t

= Ra0, (20)

where we use the fact that aT+1 = 0.


(conti.) For simplicity, assume that γ = 1. Combining (18) with (20)gives

c0 +1R(βRc0) + · · ·+

1RT

(βTRT c0

)= Ra0 =⇒

c0 =R

1+ β+ · · ·+ βTa0. (21)

Hence, the time path of optimal consumption is

ct =R t+1βt

1+ β+ · · ·+ βTa0 (22)

where t = 0, · · ·,T . Given ct , it is straightforward to determine thetime path of at :

at+1 = Rat − ct = Rat −R t+1βt

1+ β+ · · ·+ βTa0, (23)

where t = 0, · · ·,T and aT+1 = 0.


(conti.) Consider an important special case, βR = 1. In this case,(18) implies that c0 = c1 = · · · = cT ,

c0

(1+ · · ·+ 1

RT

)= Ra0 =⇒

ct =R − 1

1− R−(T+1)a0 . (24)

where t = 0, · · ·,T .Next, the optimal path of at can be derived by solving the followingdifference equation:

at+1 = Rat − ct = Rat −r

1− R−(T+1)a0 =⇒

at =

(1− 1

1− R−(T+1)

)a0R t +

11− R−(T+1)

a0, (25)


Alternative Way to Solve Optimal Consumption

Instead of defining the Lagrange multiplier for each flow budgetconstraint, we may define a Lagrange multiplier for the intertemporalbudget constraint (that is, the lifetime budget constraint):

T

∑t=0

ctR t= Ra0,

where we have used the fact that aT+1 = 0. That is, we may writethe Lagrangian as follows

L =T

∑t=0

βtu(ct ) + λ

(Ra0 −

T

∑t=0

ctR t

),

where λ is the constant Lagrange multiplier for the lifetime budgetconstraint.


(conti.) The FOCs for an optimum are then

βtu′(ct ) = λ1R t, where t = 0, · · ·,T . (26)

Since λ is a constant, the above FOCs implies that the Eulerequations are

u′(ct ) = βRu′(ct+1), where t = 0, · · ·,T − 1,

which is identical to those Euler equations we derived above.


The Infinite Horizon Case

There are some reasons to consider the infinite horizon (T = ∞) case:

People don’t live forever, but they may care about their offspring (abequest motive).Assuming an infinite horizon eliminates the terminal date complications(stop saving, consume their entire value, etc. There is no real worldcounterpart for these actions because the real economy continueforever).Many economic models with a long time horizon tend to show verysimilar results to IH models if the horizon is long enough.IH models are stationary in nature: the remaining time horizon doesn’tchange as we move forward in time.


The Model Setup

The consumer’s lifetime utility maximization problem

maxc (t)

∞

∑t=0

βtu (ct ) (27)

subject to at+1 = Rat − ct , ∀t ≥ 0, given the initial asset holdingsa0 = a (0) .

In addition, we need to impose the following institutional assumption:no Ponzi game condition (nPg):

limt→∞

(at+1R t

)≥ 0, (28)

means that in present value terms, the agent cannot engage inborrowing and lending so that his terminal asset holdings are negative.


(conti.) (28) can help obtain an intertemporal budget constraint:

T

∑t=0

ctR t+aT+1RT

= Ra0.

Applying (28), we have

limT→∞

(T

∑t=0

ctR t+aT+1RT

)= lim

T→∞

T

∑t=0

ctR t+ limT→∞

(aT+1RT

)=

∞

∑t=0

ctR t+ limT→∞

(aT+1RT

)=⇒

∞

∑t=0

ctR t≤ Ra0. (29)


(conti.) In an optimum, (29) must be binding:

∞

∑t=0

ctR t= Ra0 (30a)

because (28) is binding, i.e., limT→∞( at+1R t)= 0 (Otherwise, the

agent can consume the left amount and reach a higher utility level,which contradicts optimum).

Set up the Lagrangian:

L =∞

∑t=0

βtu (ct ) + λ

(Ra0 −

∞

∑t=0

ctR t

)The FOCs w.r.t. ct for any t are

βtu′ (ct )− λ1R t= 0, ∀t ≥ 0, (31)

where λ is the Lagrangian multiplier associated with theintertemporal budget constraint.


(conti.) Note that the FOCs imply the following Euler equationlinking consumption in two consecutive periods

u′ (ct ) = βRu′ (ct+1) , ∀t ≥ 0. (32)

Suppose that the utility function is log, u (ct ) = log ct . (31) impliesthat ct = (βR)

t c0, ∀t ≥ 1. Substituting them into the (30a) gives

∞

∑t=0

(βR)t

R tc0 = Ra0 =⇒

(∞

∑t=0

βt

)c0 = Ra0 =⇒

c0 = R (1− β) a0, (33)

and consumption in periods t ≥ 1 can be recovered:

ct = (βR)t c0 = (βR)

t R (1− β) a0, ∀t ≥ 1. (34)

Similarly, we can determine the optimal time path of asset holdings:

at+1 = Rat − (βR)t R (1− β) a0.


Dynamic Programming (DP)

We use the same infinite-horizon lifetime optimalconsumption-savings model to introduce DP and then provide moregeneral discussions on DP.Consider the maximization problem

Ut =∞

∑s=t

βs−tu (cs )

subject to as+1 = Ras − cs , ∀s ≥ t, given the initial asset holdingsat = a (t). In addition, impose:

limT→∞

(at+T+1R t+T

)≥ 0.

Of course, DP is also applicable with a finite horizon.Assume that there is a function, called the value function, that givesus the maximal constrained value of Ut as a function of the initialwealth at . We write the value function as J (at ), and we assume it isdifferentiable.Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 24 / 56

(conti.) DP rests on a recursive equation involving the value function.That equation, called the Bellman equation, characterizesintertemporally optimizing plans:

A consumption plan optimal from the standpoint of t must maximizeUt+1 subject to the future wealth at+1 produced by today’sconsumption plan ct . If not, the consumer could raise utility bybehaving differently after t.The foregoing property means that an optimizing consumer can behaveas if Ut = u (ct ) + βJ (at+1), where J (at+1) is the constrainedmaximal value of Ut+1.If this is so, ct maximizing lifetime utility is the one maximizingUt = u (ct ) + βJ (at+1) subject to the same constraint.

Formally, the Bellman equation can be written as

J (at ) = maxct{u (ct ) + βJ (at+1)} , (35)

s.t. at+1 = Rat − ct . (36)


(conti.) (35) can be rewritten as:

J (at ) = maxct{u (ct ) + βJ (Rat − ct )} . (37)

The first-order condition is thus

u′ (ct )− βJ ′ (Rat − ct ) = 0. (38)

Note that (38) gives optimal consumption as an implicit function ofcurrent wealth, c = c (a). Substituting it into (37):

J (a) = u (c (a)) + βJ (Ra− c (a)) . (39)

Taking differentiation w.r.t. a gives

J ′ (a) = u′ (c (a)) c ′ (a) + βJ ′ (Ra− c (a))(R − c ′ (a)

), or

J ′ (a) = βRJ ′ (Ra− c (a)) , or J ′ (a) = Ru′ (c) , (40)

which is called the Envelop theorem.


(conti.) Implication: For an optimizing consumer, an increment towealth on any date has the same effect on lifetime utility regardless ofhow to put the wealth: consumption or saving.Combining (38) with (38), we obtain the usual consumption Eulerequation:

u′ (ct ) = βRu′ (ct+1) . (41)

Following the same procedure used above, we can easily solve foroptimal consumption path.Alternatively, we can solve for optimal consumption by using thevalue function. Given the same utility function:

u(ct ) =c1−γt

1− γ, (42)

we first guess the form of the value function and then verify that theguess was right. A natural guess is that the value function takes aparallel form as u(ct )

J (a) = Aa1−γ

1− γ. (43)


(conti.) Substituting this conjectured solution intou′ (ct )− βJ ′ (Rat − ct ) = 0,

c−γt = βA (Rat − ct )−γ , or ct =

R

1+ (βA)1/γat . (44)

Substituting it into the Bellman equation,J (a) = u (c (a)) + βJ (Ra− c (a)), gives

Aa1−γ

1− γ=

c (a)1−γ

1− γ+ βA

(Ra− c (a))1−γ

1− γ, (45)

A =1

β(

β−1/γR1−1/γ − 1)γ , (46)

ct = (βR)1/γ(

β−1/γR1−1/γ − 1)at , (47)

which is consistent with that we obtained using optimal control.Finally, given A, we can obtain J (a).


Dynamic Programming (DP): General Setting

Choose an infinite sequence of controls {ct}∞t=0 to maximize

∞

∑t=0

βt r (xt , ct ) (48)

subject to xt+1 = g (xt , ct ), with x0 given.Assume that r (xt , ct ) is a concave function and the set{(xt+1, xt ) : xt+1 ≤ g (xt , ct )} is convex and compact. DP seeks atime-invariant policy function c = h (x) mapping the state (x) intothe control (c), such that the sequence {ct}∞

t=0 generated by iteratingthe h and g functions starting from x0 solves the original problem.To find h, we need to know the value function that is the optimalvalue of the original problem:

V (x0) = max{ct}∞

t=0

∞

∑t=0

βt r (xt , ct ) (49)

subject to the same constraint.Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 29 / 56

Note that if we knew V (·), c can be obtained by solving for each xthe problem

maxc{r (x , c) + βV (x̃)} (50)

subject to x̃ = g (x , c) with x given and x̃ denotes the state nextperiod.Key point: Have exchanged the original problem of finding {ct}∞

t=0for the problem of finding the optimal V (x) and h (x) that solves thecontinuum of maximum problems (one maximum problem for each x).The task is now to solve for (V (x) , h (x)) linked by the Bellmanequation:

V (x) = maxc{r (x , c) + βV (g (x , c))} . (51)

The maximizer of the right side of it is h (x) that satisfies

V (x) = r (x , h (x)) + βV (g (x , h (x))) , (52)

which is a functional equation to be solved for unknown functions:(V (x) , h (x)). Note that the function equation must hold for everyvalue of x within the permitted domain and the solutions to it arefunctions, V (x) and h (x).Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 30 / 56

Some Properties of the Bellman Equation

In the general setting, we encounter the same problems that we hadin the consumption-savings model:

We need to know the derivatives of the value function, V (x).The value function is unknown.

The envelop theorem of Benveniste and Scheinkman (1978) gives aset of suffi cient conditions under which one can obtain the derivativeof the value function:

State x ∈ X , where X is a convex set with a nonempty interior.Control c ∈ C , where C is a convex set with a nonempty interior.The period objective function, r , is concave and differentiable.The RHS of the state transition equation, g , is concave, differentiable,and invertible in c .


Under the above particular assumptions about r , g , x , and c , there arefour main findings:

1 The Bellman equation, (51), has a unique strictly concave solution.2 This solution can be approached in the limit as j → ∞ by iterations on

Vj+1 (x) = maxc{r (x , c) + βVj (x̃)} (53)

subject to x̃ = g (x , c) with x given, starting from any bounded andcontinuous initial V0.

3 There is a unique and time-invariant optimal policy of the formc = h (x), where h is chosen to maximize the right side of (51).

4 The limiting value function V is differentiable.


Optimal Conditions

Since V is differentiable, the FOC for (51) is

rc (x , c) + βV ′ (g (x , c)) gc (x , c) = 0. (54)

If we also assume that h is differentiable, differentiation of (52) gives

V ′ (x) = rx (x , h (x)) + rc (x , h (x)) h′ (x) (55)

+βV ′ (g [x , h (x)])[gx (x , h (x)) + gc (x , h (x)) h′ (x)

],

which can be reduced to

V ′ (x) = rx (x , h (x)) + βV ′ (g (x , h (x))) gx (x , h (x)) . (56)


Solution Methods

There are three main types of solution methods for solving DP.

1 Guess and verify. It involves guessing and verifying a solution V tothe Bellman equation.

2 Value function iteration. It proceeds by constructing a sequence ofvalue functions and associated policy functions. The sequence iscreated by iterating on the following equation, starting from V0 = 0,and continuing until Vj has converged:

Vj+1 (x) = maxc{r (x , c) + βVj (x̃)} (57)

subject to x̃ = g (x , c) with x given.3 Policy function iteration (or Howard’s improvement algorithm).


Value Function Iteration (optional)

First, consider an initial guess for the value function V0 (x). (It doesnot matter very much and we usually guess it is 0.)

Second, one can then calculate an updated value function V1 (x)using:

V1 (x) = maxc{r (x , c) + βV0 (g (x , c))} , (58)

and doing the maximization numerically over the domain of x . Thismaximization defines approximately the function V1 (x).

Using this new function V1 (x), one can update again and get a newV2 (x):

V2 (x) = maxc{r (x , c) + βV1 (g (x , c))} , (59)

over the domain of x . Repeated this step until the sequence of theapproximate value functions converges to V (x). Note that in therepeated calculations of the value function, one is also calculatingrepeated approximations to the policy function, c = h (x).


Finite-horizon Dynamic Programming

The basic idea of finite-horizon DP is to solve the optimizationproblem period by period — starting from the terminal period, takingthe previous periods’solutions as given, and then working backsequentially to the first period.Having optimized the terminal period (T ), this solution is substitutedinto the period T − 1 problem and then period T − 1 is optimized.Substituting these previous solutions, the solutions for T − 2, T − 3,· · ·, 0 are obtained in sequence in the same way.Suppose that the problem is to maximize:

U (xt ) = r (xt , ct ) + βV (xt+1) , (60)

for t = 0, · · ·,T , subject toxt+1 = f (xt , ct ) , (61)

with x0 given and xT+1 = x . Note that here we assume that thevalue function, V (xt+1), is time invariant and later we can see that itis correct.Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 36 / 56

Consider the problem for period T . First, we maximize

U (xT ) = r (xT , cT ) + βV (xT+1) , (62)

with respect to cT , subject to xT+1 = f (xT , cT ) and taking xT asgiven. Thus we maximize

U (xT ) = r (xT , cT ) + βV (f (xT , cT )) .

The FOC is

∂r (xT , cT )∂cT

+ β∂V (f (xT , cT ))

∂cT= 0,

which means that the solution for cT has the form:

cT = gT (xT ) . (63)


(conti.) Substituting this solution into U (xT ):

U (xT ) = r (xT , cT ) + βV (f (xT , cT ))

= r (f (xT−1, cT−1) , gT (f (xT−1, cT−1)))

+βV (f (f (xT−1, cT−1) , gT (f (xT−1, cT−1))))(64)

, VT (xT−1, cT−1) .

Turning to period T − 1, we maximize

U (xT−1) = r (xT−1, cT−1) + βV (xT ) ,

w.r.t. cT−1, subject to xT = f (xT−1, cT−1) and taking xT−1 asgiven. The FOC is

∂r (xT−1, cT−1)∂cT−1

+ β∂VT (xT−1, cT−1)

∂cT−1= 0, (65)

from which we can write the solution for cT−1 ascT−1 = gT−1 (xT−1) .


Substituting this into U (xT−1) gives

U (xT−1) = r (xT−1, gT−1 (xT−1)) + βV (f (xT−1, gT−1 (xT−1)))(66)

, VT−1 (xT−2, cT−2) .

We can proceed similarly in periods T − 2, · · ·, 0.The general solution for period t has the FOC

∂r (xt , ct )∂ct

+ β∂V (f (xt , ct ))

∂ct= 0, (67)

and optimal control:ct = gt (xt ) . (68)


Application to the Same Consumption-Saving Model

Consider the same consumption-saving problem as above:

V (a0) = max{ct ,at+1}

T

∑t=0

βtu (ct )

s.t.at+1 = Rat − ct , f (at , ct ) , t = 0, · · ·,T ,

given a0 = a (0) and aT+1 = 0, where u (ct ) = ln (ct ).

We first consider the solution for period T . This requires to maximize

U (aT ) = u (cT ) + βV (aT+1) , (69)

with respect to cT , subject to aT+1 = f (aT , cT ), aT+1 = 0, andtaking aT as given. We thus have V (aT ) = u (cT ), i.e., nomaximization is required for the terminal period: cT = RaT , and

U (aT ) = u (RaT ) + βV (0) = ln (RaT ) + βV (0) . (70)


(conti.) The T − 1 problem is to maximize:

U (aT−1) = u (cT−1) + βV (aT )

= u (cT−1) + β [ln (RaT ) + βV (0)] (71)

with respect to cT−1, subject to aT = RaT−1 − cT−1 and takingaT−1 as given. The FOC is:

1cT−1

− β1

RaT−1 − cT−1= 0,

which implies that

cT−1 =R

1+ βaT−1 (72)

and

V (aT−1) = ln(

R1+ β

aT−1

)+ β

[ln(

βR2

1+ βaT−1

)+ βV (0)

].

(73)


(conti.) Similarly, solving the T − 2 problem yields the followingoptimal solution:

cT−2 =R

1+ β+ β2aT−2. (74)

It is straightforward to show that the general solution for periods0, 1, · · ·,T takes the following form:

ct =R

1+ β+ β2 + · · ·+ βT−tat , (75)

which is identical to the corresponding solution we obtained usingoptimal control (i.e., the Lagrange multiplier method). As T → ∞,we have

ct = (1− β)Rat . (76)


From Discrete-time to Continuous-time

We may treat the continuous-time case as a limiting case of thediscrete-time case as the time interval (∆t) goes to 0. (Note that inthe discrete-time models, ∆t = 1.)Specifically, in discrete-time, the flow constraint is

x (t + 1)− x (t) = f (c (t) , x (t) , t) , (77)

which can be rewritten as

x (t + ∆t)− x (t) = f (c (t) , x (t) , t)∆t (78)

if the time interval is ∆t.Dividing both sides by ∆t and letting it goes to 0 gives

x ′t = lim∆t→0

x (t + ∆t)− x (t)∆t

= f (c (t) , x (t) , t) (79)


(conti.) Further, note that when the time interval is ∆t, the objectivefunction can be written as

T /∆t

∑i=0

r (c (i∆t) , x (i∆t) , i∆t)∆t; (80)

as ∆t → 0,

lim∆t→0

T /∆t

∑i=0

r (c (i∆t) , x (i∆t) , i∆t)∆t =∫ T

0r (c (t) , x (t) , t) dt

(81)Hence, the continuous-time version of the dynamic optimizationproblem can be written as

maxc (t)

∫ T

0r (c (t) , x (t) , t) dt,

s.t. x ′t =dx (t)dt

= f (c (t) , x (t) , t) ,

x0 = x (0) , and x (T ) ≥ 0.Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 44 / 56

Solving the Model

The Lagrange function for the above intertemporal problem is

L =∫ T

0

{r (ct , xt , t) + λt

[f (ct , xt , t)− x ′t

]}dt

=∫ T

0

{r (ct , xt , t) + λt f (ct , xt , t) + λ′txt

}dt + λ0x0 − λT xT(82)

where we use the fact that the rule of integration by parts implies

−∫ T

0λtx ′tdt =

∫ T

0λ′txtdt + λ0x0 − λT xT (83)

Now we can regard the integral as the sum and take first derivativesw.r.t. ct and xt :

rc (ct , xt , t) + λt fc (ct , xt , t) = 0 (84)

rx (ct , xt , t) + λt fx (ct , xt , t) + λ′t = 0 (85)


Hamiltonian Function

Similarly, we can define a new function H, a Hamiltonian function,

H = r (ct , xt , t) + λt [f (ct , xt , t)] . (86)

Hence, the FOC for ct , (84), can be rewritten as

∂Ht∂ct

= 0. (87)

Theorem (The Maximum Principle)

The necessary FOCs for maximizing (81) subject to (79) are:(i) Hc , ∂Ht

∂ct= 0 for all t ∈ [0,T ] .

(ii) x ′t = f (ct , xt , t) for all t (The state transition equation)(iii) Hx + λ′t = 0

(rx (ct , xt , t) + λt fx (ct , xt , t) + λ′t = 0

)for all t (The

costate (λt ) equation)(iv) xT ≥ 0,λT ≥ 0, xT λT = 0 (The transversality condition, or thecomplementary-slackness condition)


Continuous-time Version of the Consumption-Saving Model

Consider the following deterministic continuous-time optimalconsumption-saving problem:

maxct

∫ T

0u (ct ) exp (−ρt) dt, (88)

subject toa′t = rat − ct , (89)

where ρ is called the discount rate, r is the instantaneous interestrate, given a0, and aT ≥ 0 (preventing the consumer from dying withdebts).

Here we assume that the utility function u (ct ) satisfies the usualconditions (just like those discussed in the discrete-time case) andct > 0.


The Hamiltonian for this problem is

H = u (ct ) exp (−ρt) + λt (rat − ct ) . (90)

Using the maximum principle,

Hc = 0 =⇒ λt = u′ (ct ) exp (−ρt) , (91)

Ha + λ′t = 0 =⇒ rλt + λ′t = 0, (92)

a′t = rat − ct . (93)

In addition, aT = 0, i.e., it is optimal for the consumer to leave nobequest after T . Note that if at optimum aT > 0, λT > 0, whichcontradicts (91).


(conti.) Using (92), we have

λ′tλt= −r , (94)

which can be easily solved for λt :

λt = λ0 exp (−rt) . (95)

Suppose that u (ct ) =c1−γt −11−γ . Then combining (91) with (95) gives

c−γt = c−γ

0 exp ((ρ− r) t) . (96)


(conti.) Differentiating (92) gives

a′′t = ra′t − c ′t .

Since

c ′t ,dctdt=u′ (ct )u′′ (ct )

(ρ− r) = ctγ(r − ρ) ,

we have

a′′t = ra′t +ctγ(ρ− r)

= ra′t +1γ(ρ− r)

(rat − a′t

)=

[r − 1

γ(ρ− r)

]a′t +

1γr (ρ− r) at , (97)

which is a second-order differential equation (SODE) in terms of at .


(conti.) For r 6= − 1γ (ρ− r), (i.e., no repeated roots for the

corresponding characteristic equation,λ2 −

[r − 1

γ (ρ− r)]

λ− 1γ r (ρ− r) = 0,) this SODE has the

following general solution:

at = A exp (rt) + B exp(− 1

γ(ρ− r) t

), (98)

where A and B can be pinned down by using the initial and terminalconditions:

a0 = A+ B, (99)

aT = A exp (rT ) + B exp(− 1

γ(ρ− r)T

)(= 0) . (100)

From (100), we have A = −B exp([− 1

γ (r − ρ)− r]T).

Substituting it into (99):

B =

[1− exp

([− 1

γ(r − ρ)− r

]T)]a0, (101)

A = − exp([− 1

γ(r − ρ)− r

]T)·[

1− exp([− 1

γ(r − ρ)− r

]T)]a0.


Optimal consumption:

ct , rat − a′t = r[A exp (rt) + B exp

(− 1

γ(ρ− r) t

)]−[Ar exp (rt)− 1

γ(ρ− r)B exp

(− 1

γ(ρ− r) t

)].(102)

Consider a special case in which r = ρ:

c ′t =u′ (ct )u′′ (ct )

(ρ− r) = 0,

i.e., ct = c , independent of time t. In this case, a′t = rat − ctbecomes a′t = rat − c , whose solution is:

at = exp (rt)[a0 −

cr(1− exp (−rt))

]. (103)

At T , we have aT = exp (rT )[a0 − c

r (1− exp (−rT ))]= 0:

c =r

1− exp (−rT )a0. (104)


Continuous-time Bellman Equation (Optional)

Consider the same deterministic continuous-time consumption-savingproblem:

J (a0) = maxct

∫ ∞

0u (ct ) exp (−ρt) dt (105)

subject to a′t = rat − ct , given a0, and aT ≥ 0. Here J (a0) is definedas the value function.

(105) can be rewritten as the following recursive form:

J (a0) = maxc

{∫ ∆t

0u (ct ) exp (−ρt) dt +

∫ ∞

∆tu (ct ) exp (−ρt) dt

}= max

c0{u (c0)∆t + exp (−ρ∆t) J (a∆t )} (106)

for a small interval, ∆t.


(conti.) Here we use the facts:∫ ∆t

0u (ct ) exp (−ρt) dt = u (c0)

∫ ∆t

0exp (−ρt) dt = u (c0)∆t,∫ ∞

∆tu (ct ) exp (−ρt) dt

= exp (−ρ∆t)∫ ∞

0u (ct ) exp (−ρ (t − ∆t)) d (t − ∆t)

, exp (−ρ∆t) J (a∆t ) . (107)

Linearizing J (a∆t ) around a0:

J (a∆t ) ∼= J (a0) + J ′ (a0) (a∆t − a0) .


(conti.) Substituting it back into (106):

J (a0) = maxc0

{u (c0)∆t + (1− ρ∆t)

[J (a0) + J ′ (a0) (a∆t − a0)

]}⇐⇒ ρ∆tJ (a0) = max

c0

{u (c0)∆t + J ′ (a0) (a∆t − a0)−ρ∆tJ ′ (a0) (a∆t − a0)

},(108)

where we use the fact that exp (−ρ∆t) ∼= 1− ρ∆t.Dividing ∆t on both sides:

ρJ (a0) = maxc0

{u (c0) + J ′ (a0)

(lim∆t→0

a∆t−a0∆t

)−ρJ ′ (a0) lim∆t→0 (a∆t − a0)

}= max

c0

{u (c0) + J ′ (a0) a′0

}= max

c0

{u (c0) + J ′ (a0) (ra0 − c0)

}(109)

and taking derivative w.r.t. the control c0: u′ (c0) = J ′ (a0), whichimplies that c0 = h (a0).


(conti.) Substituting c0 = h (a0) into the Bellman equation (109):

ρJ (a0) = u (h (a0)) + J ′ (a0) [ra0 − h (a0)] . (110)

The envelop theorem implies that

ρJ ′ (a0) = u′ (h (a0)) h′ (a0) + J ′′ (a0) [ra0 − h (a0)] (111)

+J ′ (a0)[r − h′ (a0)

]= J ′′ (a0) [ra0 − h (a0)] + J ′ (a0) r , (112)

which means that

(ρ− r) u′ (c0) = u′′ (c0)dc0da0

[ra0 − h (a0)] , (113)

c ′0c0

=(ρ− r) u′ (c0)c0u′′ (c0)

, c0γ(r − ρ) , (114)

where we use the fact that u′′ (c0) dc0da0 = J′′ (a0).


Topic 1: Introduction to Intertemporal Optimizationyluo/teaching/econ6012/2013topic1a.pdf ·...

Documents

Transcript of Topic 1: Introduction to Intertemporal Optimizationyluo/teaching/econ6012/2013topic1a.pdf ·...