On the Differentiability of the Value Function in Dynamic Models of Economics by L. M. Benveniste...

7/29/2019 On the Differentiability of the Value Function in Dynamic Models of Economics by L. M. Benveniste and J. a. Schei

1/6

Econometrica, Vol. 47, No. 3 (May, 1979)

ON THE DIFFERENTIABILITY OF THE VALUE FUNCTION INDYNAMIC MODELS OF ECONOMICSBy L. M. BENVENISTE AND J. A. SCHEINKMAN'

WHEN WORKING WITH dynamic models of economic agents, many authors havemade use of a differentiability assumption on the valuation function. In the beliefthat one should understand the implicit restrictions in the model which thisassumption imposes, we have found a set of assumptions on the underlying modelwhich guarantee that this condition is met for a certain class of problems.The class of problems are the infinite horizon calculus of variations problems:(D) In discrete time, given a technology set T ' R2, find the sequence (x,),t= O, . . ., X which solves

00maximize E u(xt,xt+1, t)t=Osubject to (xt,xt+i) E T for all t and xo fixed.

(C) In continuous time, given a technology set T R n, find the absolutelycontinuous path (x (t)) which solvesmaximizej u(x(t), x(t), t) dt0subject to (x(t), x (t)) e T for all t and x (O)fixed.

Given an initial state xo with an associated solution path y (t, xO)2the valuationfunction (see [2]) is defined as:300(D) V(X0)= Y. u(y(t, xo), y (t + 1, xo), t),t =O

(C) V(xo) = u(y(t, xo), y(t, xo), t) dt.1 J. A. Scheinkman's research was initiated while he was visiting the Group for Applications ofMathematics and Statistics to Economics at the University of California, Berkeley, duringthe summerof 1975. He wishes to thank Gerard Debreu for providing research facilities, and the National ScienceFoundation for financial support. L. M. Benveniste's work was performed under the auspices of theU.S.E.R.D.A. The authors benefited from discussions with W. A. Brock, R. Lucas, A. Mas-Colell, L.W. McKenzie, K. Shell, and also with participants of the M.S.S.B. Seminar in The Structure ofDynamical Systems Arising in Economics and the Social Sciences at the University of Pennsylvania.2We assume that the optimal path exists from the point xo.3 The existence of optimal paths is not necessary to define the valuation function. One can define itas

V(x) = supJ u(x, x, t) dtover all paths x(t) with (x(t), i(t)) E T for all t and x(0) = x.


2/6

728 L. M. BENVENISTE AND J. A. SCHEINKMANWe will work with a specific initial state xo, having the property that thevaluation function is defined and finite in a neighborhood of xo in R', and giveconditions which guarantee that V is differentiable at xo.

1. A MATHEMATICAL RESULTThe following lemma about concave functions is the main tool used in thepaper. The lemma and its proof exist in the literature but we include both forcompleteness.LEMMA 1: Let V be a real valued concave function defined on a convex setD c R . If W is a concave differentiable unction in a neighborhoodN of xo in D

with the property that W(xo) = V(xo) and W(x) - V(x) for all x in N, then V isdifferentiableat xo.PROOF: We will show that there is only one possible subgradient p for theconcave function V at xo. Any subgradient p of V at xo must satisfy

(1) p *(x-xO) > V(x)- V(xO) for all x E D.Since W(x) < V(x) in a neighborhood of xo and W(xo) = V(xo), any vector psatisfying (1) must also satisfy

p (x -xO) V(x) - V(xo) : W(x) - W(xo)for all x in the open set N. By the differentiability of W, this p must be unique, andany concave function with a unique subgradient at xo, must be differentiable at xo(cf. Rockafellar [2, Theorem 25.1, p. 242]). Q.E.D.

2. THE DISCRETE TIME CASEHere we will give sufficient conditions for the differentiability of the valuefunction of problem (D). The following assumptions will be used.ASSUMPTION 1: T is convex and T # P.ASSUMPTION 2: For each t, u (., , t): T -- R is concave and differentiable.ASSUMPTION 3: An optimal solution (not necessarily unique) {y (t, xo)} =1existsfor (D) at xo, and V(x) is well defined for x in some neighborhood of xo.ASSUMPTION 4: (xo, y(1, xo)) E T.Assumptions 1 and 2 are the standard convexity assumptions plus a smoothnesscondition on u, which is clearly needed for the differentiability of V.Assumption 3requires that the value function be well-defined in a neighborhood of xo, and that


3/6

VALUE FUNCTION 729an optimal path exists from x0, while Assumption 4 requires that the optimal pathfrom x0 be initially an interior path.The theorem for discrete time is:

THEOREM 1: Under Assumptions 1, 2, 3, and 4, Vis a differentiable unction atxo with

V'(xo) -u(xo, y(l, xo), 0).axPROOF: By the concavity of u and the convexity of T, V is a concave function.Define the function

ceW(x)= u(x, y(1, xo),0)+ E u(y(t, xo),y(t+1, xo), t).t=l

By Assumptions 3 and 4, W is well defined in an open neighborhood N(xo) of xo.Also since W(x) = u (x, y (1, xo), 0) + c where c is a constant, by Assumption 2, Wis concave and differentiable at xo. By construction, W(x) < V(x) in N(xo).Hence, we can apply Lemma 1 to conclude that V is differentiable at xo withV'(xo) = W'(xo)= -u (xo, y(1, x0), 0). Q.E.D.ax

3. THE CONTINUOUS TIME CASEIn this section, we give sufficient conditions for the value function V associatedwith problem (C) to be differentiable at a point x0.We retain Assumptions 1 and 3, and give new versions of 2 and 4.ASSUMPTION 2': u: T x R -> R is a continuously differentiable function onTxR, and u(., , t): T-*R is concave.ASSUMPTION 4': An optimal solution {y(t, x0)}exists for the initial state x0, andthis solution is interior in the following sense: there exists h > 0, ? > 0, and M > 0such that for 0 S t ? h,

IIYt, xo), y (t, x0)||< Mand if (z, z') E R2n satisfies

jj(y(t, o), y(t, xo))-(z, z')jj-e for some te[0, h]then (z, z') e T; i.e., there exists an e-ball around the path contained in T duringsome initial phase. Also, V(x) is well defined for x in a neighborhood of xo.

In order to simplify notation, we will denote by u1(u2) the vector of partialderivatives of u with respect to the first n coordinates of R 2n (the second ncoordinates, respectively). We now state the main theorem of this section.


4/6

730 L. M. BENVENISTE AND J. A. SCHEINKMANTHEOREM 2: UnderAssumptions 1, 2', 3, and 4', Vis continuously differentiableat xo with

V'(XO) = rnim Uu2(Y (t, xo), y (t, xo), t) dt.PROOF: To prove the above, we again will call upon Lemma 1. But in order to dothis we will need the following claims:CLAIM 1: For lix-xol - e (h/(1 + h))2 = r-,the path v(t, x) originating at x with

v3(t,x) = y(t, xo)+((xo-x)lh)is a feasible path for t e [0, h] and v(h, x) = y (h, xo).PROOF: By definition

11(Yt, XO)'y(t, X0)) _ (v (t, x), v (t, X))112~jjx-xjj x -xo 2 ____X _ X0112+ || 0|=(1h )|X _ X0112 2.

Hence by Assumption 4', the path v(x, t)O t - h is feasible. For the second partof the claim, we haverh rhv(h, x)=x+ z3(t,x) dt=x+ J y(t, xo) dt+xo-x

rh= xO+ y(t, xo) dt = y(h, xo).0CLAIM 2: The function Wh: N, (xo)-- R given by

hWh(X)= u(v(t, x), v3(t, ), t) dt

is well defined and concave.PROOF. To show this, one might observe that if y and z EN, (xo), then v(t, ay +/3z) = av (t, y) + /3v(t, z) where a + ,f = 1 and likewise for the derivatives. Hence, itis concave because u is concave and well-defined by the previous claim.CLAIM 3: Wh is differentiable at xo with

W,h(xo) = j u1(y(t, x0), y(t,x0), t)[1 h-j dtCh 1- J U2(y t, xO),y(t, xO), )- dt.O ~~~~h

PROOF: By the definition of Wh,( h(2) Wh (Xo) =ax| u(v(t,x), v(t,x), t) dt at x = xo.


5/6

VALUE FUNCTION 731By Assumption 4', the set {(v (x, t), v(x, t), t): 0 - t - h, x EN, (xo)}has a compactclosure in T. Hence since u is C', we can interchange the differentiation andintegration in (2), and obtain

h l(3) wh(xo) = -(U (V t,xo), v (t,xo), t) dtax= u1(y(t, xo),V(t, xo), t)[1 -hj dt

h ~~~~~~1JOU2(y(t, xO), y9(t, O), )- dt,o ~~~~hsince v(t, x) = y (t, xo)+ x - xo+ t[(xo- x)/h]. Hence, w (xo) exists. UTo complete the proof, we are now in a position to apply Lemma 1. Forx ENr,(xo), define

Wh (X) = Wh (X) + U(y (xo, t), y(xo, t), t) dt.hThen by definition of V, Wh - V in N, (xo) and Wh xo)= V(xo) with Wh concaveand differentiable. Hence, V'(xo) = W' (xo)= w (xo). Furthermore, taking limitsas h 0, we obtain

V (xO)= limO u1(y(t, x0),(t, x0),t)[1 h] dt

- Uu2(y(t, xO),y (t, xO), t)- dt}= lim -- u2(y(t, xo), y (t, xo), t) dt,h-o h o

since IIul(y(t,xo), y(t, xo), t)II N, 0 S t - h, by Assumptions 2' and 4'. Q.E.D.When the optimal control is piecewise continuous, Assumption 4 will hold forsome h > 0 provided (xo, y (0, xo))E T. Furthermore, in this case we have

1 hV'(xo) = lim - u2(y(t, xo), y (t, xo), t) dt = -U2(Xo, y(0, xO),0).h-0 J0Hence we have proved:COROLLARY 1: UnderAssumptions 1, 2', and 3, if (xo, y (0, xo))e T, and if theoptimal control is a piecewise continuous function of time, then V is C' at xo and

V'(xO) = -u2(xO, y(0, xO), 0).

To conclude, the conditions required for the result seem to be the minimumrequiredfor the general result. It might also be pointed out that these results give a


6/6

732 L. M. BENVENISTE AND J. A. SCHEINKMANgood economic characterization of the shadow prices that arise from the calculusof variations. It follows from Corollary 1, that the shadow prices from the calculusof variations represent marginal valuations of capital. This result has implicationsabout the necessity of the transversality condition for such problems. This iscarried out in a more general framework in Benveniste and Scheinkman [1].

University of RochesterandUniversity of ChicagoManuscriptreceivedSeptember, 1977; final revision receivedMarch, 1978.

REFERENCES[1] BENVENISTE, L., AND J. SCHEINKMAN: "Duality Theory for Dynamic Optimization Models of

Economics," to appear in Journalof EconomicTheory.[2] ROCKAFELLER, R.: Convex Analysis. Princeton, N.J.: Princeton University Press.

On the Differentiability of the Value Function in Dynamic Models of Economics by L. M. Benveniste...

Documents

Transcript of On the Differentiability of the Value Function in Dynamic Models of Economics by L. M. Benveniste...