Research Article A General Setting and Solution of Bellman...
Transcript of Research Article A General Setting and Solution of Bellman...
Research ArticleA General Setting and Solution ofBellman Equation in Monetary Theory
Xiaoli Gan and Wanbo Lu
School of Statistics, Southwestern University of Finance and Economics, Chengdu 610074, China
Correspondence should be addressed to Wanbo Lu; [email protected]
Received 28 September 2014; Revised 22 November 2014; Accepted 22 November 2014; Published 21 December 2014
Academic Editor: Zhihua Zhang
Copyright Β© 2014 X. Gan and W. Lu. This is an open access article distributed under the Creative Commons Attribution License,which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
As an important tool in theoretical economics, Bellman equation is very powerful in solving optimization problems of discrete timeand is frequently used in monetary theory. Because there is not a general method to solve this problem in monetary theory, it ishard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions. In this paper, we discuss the rulesand problems that should be paid attention to when incorporating money into general equilibrium models. A general setting andsolution of Bellman equation in monetary theory are provided. The proposed method is clear, is easy to grasp, is generalized, andalways leads to the correct results.
1. Introduction
In recent years, many economists applied business cycleapproaches to macroeconomic modeling so that monetaryfactors could be modeled into dynamic general equilibriummodels. As an important method of monetary economicsmodeling, infinite horizon representative-agent models pro-vide a close link between theory and practice; its researchframework can guide practice behavior and be tested byactual data. This method can link monetary economics andother popularmodels for studying business cycle phenomenaclosely.There are three basicmonetary economics approachesintroducing money to economic general equilibrium modelsin the infinite horizon representative-agent framework. First,it is assumed that utility could be yielded by money directlyso that the money variable has been incorporated into utilityfunction of the representative-agent models [1]. Second,assuming asset exchanges are costly, transaction costs of someform give rise to amoney demand [2, 3]. Clower [4] considersthat money is used for some types of transactions. Brock [5],McCallum and Goodfriend [6], and Croushore [7] assumethat time and money can be combined to yield transactionservices. Third, money is treated as an asset that can be usedto transfer resources between generations [8].
Dynamic optimization is the main involved issue duringthemodeling process. It is represented and solved by Bellmanequation method, namely, the value function method. Themethod will obtain a forward-looking householdβs path tomaximize lifetime utility through the optimal behavior andfurther relevant conclusions.The setting of Bellman equationis the first and crucial step to solve dynamic programmingproblems. It is hard to grasp the setting and solution ofBellman equation and easy to reach wrong conclusions sincethere is not a general method to set Bellman equation or thesettings of Bellman equation are excessively flexible. Walsh[9] used Bellman equation to set and solve dynamic generalequilibrium models of money. However he does not givea general law of Bellman equation setting. The setting andsolution of the equation in his book are ambiguous and notclear. To the best of the authorsβ knowledge, there have beenno studies of uniform setting of Bellman equation.
In this paper, we provide a set of general setting andsolution methods for Bellman equation with multipliers. Itis very clear and easy to grasp. The most important thing isthat the proposed method can always lead to correct results.We apply our method to monetary general equilibriummodels that are in the framework of the first two basicmonetary economic approaches which incorporate money
Hindawi Publishing CorporationJournal of Applied MathematicsVolume 2014, Article ID 495089, 9 pageshttp://dx.doi.org/10.1155/2014/495089
2 Journal of Applied Mathematics
into economic general equilibrium model and make someextensions. At the same time, we compare our method withother current methods of setting and solution for Bellmanequation to display the clarity and correctness of the proposedmethod. It is assumed that all the relevant assumptions ofapplying Bellman equation are satisfied. This is to ensure thefeasibility of analysis and solution. For the convenience ofdiscussion and suitable length of the paper, wemainly discussthe certainty linear programming problems. The Bellmanequationβs setting and solution of uncertainty problems aresimilar to those with certainty problems essentially.
2. A New Method of Setting and Solution ofBellman Equation
First of all, we provide the theoretical details of the newgeneral method and steps when applying Bellman equationto solve problems. It hasmany advantages.The relationship ofthe results is very clear through the connection ofmultipliers.There are no tedious expansions during the derivations. Itdiffers from the expansion method because it does not needto consider which control variable should be replaced. Thetechnical details about the equivalence of Bellman equationsand dynamic programming problems and the solvability ofset problems can be found in [10]. Actually, it is easy toreach wrong conclusions using other settings and solutionsof Bellman equation with multipliers, or it has to find someparticular skills to get the right results. But our methoddoes not meet the problem. The proposed method can bedescribed as the following five steps.
Step 1. List the expression of target problem.Consider the dynamic programming problem as
π» = maxβ
β
π‘=0
π½π‘
β (π₯1π‘, π₯2π‘, . . . , π₯
ππ‘) (1)
s.t.{{{
{{{
{
π1(π₯1π‘, π₯2π‘, . . . , π₯
ππ‘, π₯π+1,π‘
, . . . , π₯ππ‘) = 0, π β€ π,
.
.
.
ππ(π₯1π‘, π₯2π‘, . . . , π₯
ππ‘, π₯π+1,π‘
, . . . , π₯ππ‘) = 0, π β€ π,
(2)
where the objective function (1) represents the maximumsum of present value of a forward-looking behavioral agentβsevery period objective function β(π₯
1π‘, π₯2π‘, . . . , π₯
ππ‘). In mon-
etary theory, β(β ) usually represents the utility function ofa family and π½ is a subjective rate of discount. Equation(2) is constraints; π
1(β ), . . . , π
π(β ) are dynamic constraint
functions and usually are intertemporal budget constraints.Let control variables {π₯
(π)
1π‘, π₯(π)
2π‘, . . . , π₯
(π)
ππ‘} β {π₯
1π‘, π₯2π‘, . . . , π₯
ππ‘,
π₯π+1,π‘
, . . . , π₯ππ‘}; the remaining variables {π₯
1π‘, π₯2π‘, . . . , π₯
ππ‘,
π₯π+1,π‘
, . . . , π₯ππ‘} β {π₯
(π)
1π‘, π₯(π)
2π‘, . . . , π₯
(π)
ππ‘} = {π₯
(π )
1π‘, π₯(π )
2π‘, . . . , π₯
(π )
ππ‘} are
state variables.
Step 2. Set up Bellman equation with multipliers to expressdynamic optimization problem in Step 1:
π(π₯(π )
1π‘, π₯(π )
2π‘, . . . , π₯
(π )
ππ‘)
= max {β (π₯1π‘, π₯2π‘, . . . , π₯
ππ‘)
+ π½π (π₯(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1)}
+ π1π‘π1(π₯1π‘, . . . , π₯
ππ‘, π₯π+1,π‘
, . . . , π₯ππ‘)
+ β β β + πππ‘
ππ(π₯1π‘, . . . , π₯
ππ‘, π₯π+1,π‘
, . . . , π₯ππ‘) ,
(3)
where π(β ) is the value function and πππ‘is the multiplier of
the πth constraint ππ(β ), π = 1, 2, . . . , π.
Step 3. Compute the partial derivatives of all control variableson the right side of the equation at Step 2 to derive first-orderconditions:
ππ (π₯(π )
1π‘, π₯(π )
2π‘, . . . , π₯
(π )
ππ‘)
ππ₯(π)
ππ‘
=
πβ (π₯1π‘, π₯2π‘, . . . , π₯
ππ‘)
ππ₯(π)
ππ‘
+ π½[
[
ππ (π₯(π )
1,π‘+1, . . . , π₯
(π )
π,π‘+1)
ππ₯(π )
1,π‘+1
β
ππ₯(π )
1,π‘+1
ππ₯(π)
ππ‘
+ β β β +
ππ (π₯(π )
1,π‘+1, . . . , π₯
(π )
π,π‘+1)
ππ₯(π )
π,π‘+1
β
ππ₯(π )
π,π‘+1
ππ₯(π)
ππ‘
]
]
+ π1π‘
ππ1(π₯1π‘, . . . , π₯
ππ‘)
ππ₯(π)
ππ‘
+ πππ‘
πππ(π₯1π‘, . . . , π₯
ππ‘)
ππ₯(π)
ππ‘
= 0,
π = 1, 2, . . . , π.
(4)
During the derivation, it should be taken into accountthat next period state variables can be represented by othercontrol variables according to the constraints, that is, toexpand π₯
(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1. Because the constraints are
dynamic and are usually intertemporal budget constraints,those control variables π₯
(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1at time π‘ +
1 can often be represented as expressions of the previ-ous period control variables. Attention should be paid tocompute the derivatives of control variables hiding inπ(π₯(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1). But there is no need to expand
π₯(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1after themultipliers during the deriva-
tion. Other than that, no particular attention should be paidto other current skills; for example, some control variablesshould be replaced after π₯
(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1represented
by other control variables inπ(π₯(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1) and so
on.
Journal of Applied Mathematics 3
Step 4. By the envelope theorem, take the partial derivativesof control variables at time π‘ on both sides of Bellmanequation to derive the remaining first-order conditions:
ππ (π₯(π )
1π‘, . . . , π₯
(π )
ππ‘)
ππ₯(π )
ππ‘
= π1π‘
πππ(π₯1π‘, . . . , π₯
ππ‘)
ππ₯(π )
ππ‘
+ β β β + πππ‘
πππ(π₯1π‘, . . . , π₯
ππ‘)
ππ₯(π )
ππ‘
,
π = 1, 2, . . . , π.
(5)
Still, it does not need to expand π₯(π )
1,π‘+1, π₯(π )
2,π‘+1, . . . , π₯
(π )
π,π‘+1
after themultipliers during the derivations. Requirements arethe same as Step 3.
Step 5. Obtain the new relevant results about target problemthrough recursion and substitution according to the aboveresults.
We could combine several state variables to one asyou need according to specific economic significance andconstraints when there is no need to describe the relevanteconomic significance of state variables one by one. For exam-ple, π₯(π )1π‘, π₯(π )2π‘
could be combined as π(π₯(π )
1π‘, π₯(π )
2π‘) = π΄
π‘. π΄π‘is a
state variable in value function π(π(π₯(π )
1π‘, π₯(π )
2π‘), π₯(π )
3π‘, . . . , π₯
(π )
ππ‘),
that is, π(π΄π‘, π₯(π )
2π‘, . . . , π₯
(π )
ππ‘). And then we follow the above
five steps to solve specific problem. The rest of state variablescould be operated similarly. But combinations should notbe overlapped over state or combined state variables. Forinstance, suppose that π₯
(π )
1π‘, π₯(π )
2π‘have been combined as
π(π₯(π )
1π‘, π₯(π )
2π‘) = π΄
π‘. It is inflexible to take π₯
(π )
2π‘as an independent
state variable in value functionπ(π΄π‘, π₯(π )
2π‘, . . . , π₯
(π )
ππ‘) or put the
combination of π₯(π )
2π‘, π₯(π )3π‘, π(π₯(π )1π‘
, π₯(π )
2π‘) = π΅
π‘in value function
π(π΄π‘, π΅π‘, . . . , π₯
(π )
ππ‘) as a state variable.
The steps above give a general setting and solution ofBellman equation. These can be summarized as follows: first,set Bellman equation with multipliers of target dynamicoptimization problem under the requirement of no overlapsof state variables; second, extend the late period state variablesin π(β ) on the right side of Bellman equation and there isno need to expand these variables after the multipliers; third,let the derivatives of state variables of time π‘ equal zero andtake the partial derivatives of these variables on both sides ofBellman equation to derive first-order conditions; finally, getmore needed results for analysis from these conditions.
Different from some current settings which allow overlapof state variables in value function, our method does notpermit overlaps. In fact, overlap of state variable is easy toreach wrong conclusions or it has to find some particularskills to get the right results [9]. These methods are hard togeneralize. More details will be seen in the following sections.
Nowwewill take several discrete time dynamic optimiza-tion problems that are under framework of the basic twomethods of incorporating money into general equilibriummodels as examples to show the applications of our setting
and solution method and compare with current popularmethods.
3. Setting and Solution of Bellman Equation inBasic Money-In-Utility Model
3.1. Model and Solution Using Previous Approach. The basicMoney-In-Utility model has few features. The labor-leisureoptions of families are ignored temporarily in utility function.Only family consumption and real money balances areinvolved in utility function; that is, real money balances yieldutility directly. We ignore the uncertainty of currency impactand technological changes temporarily for convenience.
The total present utility value of family life cycle isπ = β
β
π‘=0π½π‘
π’(ππ‘, ππ‘). Usually the per capita version of
intertemporal budget constraint is
ππ‘β‘ π(
ππ‘β1
1 + π) + ππ‘+ (
1 β πΏ
1 + π) ππ‘β1
+(1 + ππ‘β1
) ππ‘β1
+ ππ‘β1
(1 + ππ‘) (1 + π)
= ππ‘+ ππ‘+ ππ‘+ ππ‘,
(6)
where ππ‘, ππ‘, ππ‘, ππ‘, ππ‘, ππ‘β1
, and ππ‘are the per capita
consumption, bonds, stock of capital, money balances, netlump-sum transfer received from the government, nominalinterest rate, and inflation rate, respectively, at time π‘. πΏ is therate of depreciation of physical capital, π is the populationgrowth rate (assumed to be constant), and π(π
π‘β1/(1 +
π)) is the production function assumed to be continuouslydifferentiable and to satisfy the usual Inada conditions (see[11]). Note that control variables can be changed and affectthe total discounted value of utility through consumption π
π‘
in period π‘; state variables cannot be changed. Determinationof state and control variables is crucial to derive the first-orderconditions and other results. The control variables for thisproblem are π
π‘, ππ‘, ππ‘and, π
π‘and the state variables are π
π‘β1,
ππ‘β1
,ππ‘β1
, ππ‘, andπ
π‘, the householdβs initial level of resources.
This problem can be solved by the expansion methodbelow according to Walsh [9]. It is started with setting upvalue function
π (ππ‘) = max {π’ (π
π‘, ππ‘) + π½π (π
π‘+1)} . (7)
Using (6), value function can be expanded as
π (ππ‘)
= max{π’ (ππ‘, ππ‘)
+ π½π(π(ππ‘
1 + π)
+ ππ‘+1
+ (1 β πΏ
1 + π) ππ‘
+(1 + ππ‘) ππ‘+ ππ‘
(1 + ππ‘+1
) (1 + π))} .
(8)
4 Journal of Applied Mathematics
Replace ππ‘as ππ‘β ππ‘β ππ‘β ππ‘; according to (6), (8) can be
written as an expansion equation as follows:
π (ππ‘) = max{π’ (π
π‘, ππ‘)
+ π½π(π(ππ‘β ππ‘β ππ‘β ππ‘
1 + π) + ππ‘+1
+ (1 β πΏ
1 + π) (ππ‘β ππ‘β ππ‘β ππ‘)
+(1 + ππ‘) ππ‘+ ππ‘
(1 + ππ‘+1
) (1 + π))} .
(9)
Compute the partial derivatives of control variables toderive first-order conditions:
ππ (ππ‘)
πππ‘
= π’π(ππ‘, ππ‘) β ππ(ππ‘+1
)
β π½
1 + πβ [π (π
π‘) + 1 β πΏ] = 0,
ππ (ππ‘)
πππ‘
= π½ππ(ππ‘+1
) β 1 + ππ‘
(1 + ππ‘+1
) (1 + π)
β π½ππ(ππ‘+1
) [ππ(ππ‘) + 1 β πΏ
1 + π] = 0,
ππ (ππ‘)
πππ‘
= π’π
(ππ‘, ππ‘) +
π½ππ(ππ‘+1
)
(1 + ππ‘+1
) (1 + π)
β π½ππ(ππ‘+1
) [ππ(ππ‘) + 1 β πΏ
1 + π] = 0,
(10)
and the envelope theorem yields
ππ(ππ‘) = π’π(ππ‘, ππ‘) . (11)
Another expansionmethod will replace ππ‘asππ‘βππ‘βππ‘β
ππ‘; according to (6), (8) can be written as
π (ππ‘) = max{π’ (π
π‘β ππ‘β ππ‘β ππ‘, ππ‘)
+ π½π(π(ππ‘
1 + π) + ππ‘+1
+ (1 β πΏ
1 + π) ππ‘
+(1 + ππ‘) ππ‘+ ππ‘
(1 + ππ‘+1
) (1 + π))} .
(12)
Compute the partial derivatives of control variables toderive first-order conditions as
ππ (ππ‘)
πππ‘
= βπ’π(ππ‘β ππ‘β ππ‘β ππ‘, ππ‘)
+ π½ππ(ππ‘+1
) β 1 + ππ‘
(1 + ππ‘+1
) (1 + π)= 0,
ππ (ππ‘)
πππ‘
= βπ’π(ππ‘β ππ‘β ππ‘β ππ‘, ππ‘) + ππ(ππ‘+1
)
β π½
1 + πβ [ππ(ππ‘) + 1 β πΏ] = 0,
ππ (ππ‘)
πππ‘
= βπ’π(ππ‘β ππ‘β ππ‘β ππ‘, ππ‘)
+ π’π
(ππ‘β ππ‘β ππ‘β ππ‘, ππ‘)
+π½ππ(ππ‘+1
)
(1 + ππ‘+1
) (1 + π)= 0.
(13)
Using the envelope theoremand computing the derivativewith respect to state variable π
π‘, we get
ππ(ππ‘) = π’π(ππ‘β ππ‘β ππ‘β ππ‘, ππ‘) . (14)
3.2. Our Solving Approach. Now, we use our proposed stepsof setting and solution of Bellman equation to solve theabove basic Money-In-Utility problem. First, let the Bellmanequation with multiplier π
π‘be
π (ππ‘) = max {π’ (π
π‘, ππ‘) + π½π (π
π‘+1)}
+ ππ‘(ππ‘β ππ‘β ππ‘β ππ‘β ππ‘) .
(15)
Second, computing the partial derivatives for the controlvariables, we obtain the first-order conditions as
ππ (ππ‘)
πππ‘
= π’π(ππ‘, ππ‘) = ππ‘,
ππ (ππ‘)
πππ‘
= π½ππ(ππ‘+1
) β 1 + ππ‘
(1 + ππ‘+1
) (1 + π)= ππ‘,
ππ (ππ‘)
πππ‘
= π½ππ(ππ‘+1
) β 1
1 + πβ [ππ(ππ‘) + 1 β πΏ] = π
π‘,
ππ (ππ‘)
πππ‘
= π’π
(ππ‘, ππ‘) + π½π
π(ππ‘+1
)
β 1
(1 + ππ‘+1
) (1 + π)= ππ‘.
(16)
Third, using the envelope theorem and computing thederivatives of both sides of Bellman equation with respect tostate variable π
π‘, we get
ππ(ππ‘) = ππ‘. (17)
Journal of Applied Mathematics 5
Finally, based on the above results, it is easy to get
π’π(ππ‘, ππ‘) = ππ‘= π’π
(ππ‘, ππ‘)
+ π½ππ(ππ‘+1
) β 1
(1 + ππ‘+1
) (1 + π).
(18)
This expression indicates that the marginal benefit ofincreased money holdings should be equal to the marginalutility of consumption on period π‘. Similarly, we can easilyobtain the allocation of initial level of resources π
π‘among
consumption, capital, bonds, and money balances, and thesame marginal benefit must be yielded by each use at anoptimum allocation.
These results with multiplier are open-and-shut, and it iseasy to find the economic significance of marginal utility ofconsumption. Comparing with current approach mentionedabove, during the process of solving problem, there is no needto consider which variable should be replaced. There are notedious expanded expressions. Using expansion method, ifwe replace π
π‘, the derived first-order conditions are seen to
be messy and it is not easy to find the relationships of results.If we replace π
π‘, the derived results are a bit more clear, but we
might not think of replacing ππ‘at the beginning. So expansion
methods are not easy to operate, and it is not clear to replacevariable.
4. Setting and Solution of Bellman Equation inShopping-Time Model
4.1. Model and Solution Using Previous Approach. In Shop-ping-Time Model, shopping time is a function of consump-tion and money balances. Because consumption needs shop-ping time, leisure is reduced. Household utility is assumed todepend on consumption and leisure. Consumption can notonly yield utility directly but also decrease utility indirectly.In this section, π is time spent in market employment andππ is time spent shopping. ππ is the function of consumption
and money balances; that is, ππ = π(π,π), ππ
> 0, ππ
β€ 0.Total time available is normalized to equal 1. Growth rate ofpopulation is assumed to be 0 for convenience. Let π be theleisure time. The utility function is π’(π,π, π) = V[π, 1 β π β
π(π, π)]. It donates utility as a function of consumption, laborsupply, and money holdings.
The householdβs intertemporal objective is maximumdiscounted utility subject to resource constraint as
maxβ
β
π=0
π½πV (ππ‘+π
, 1 β ππ‘+π
β π (ππ‘+π
, ππ‘+π
))
= maxβ
β
π=0
π½πV (ππ‘+π
, ππ‘+π
)
s.t. π (ππ‘β1
, ππ‘) + ππ‘+ (1 β πΏ) π
π‘β1
+(1 + ππ‘β1
) ππ‘β1
+ ππ‘β1
1 + ππ‘
= ππ‘+ ππ‘+ ππ‘+ ππ‘.
(19)
Because controllable factors, labor supply, and consump-tion affect labor supply, output is affected not only by state
variable ππ‘β1
but also by these controllable factors. The leftside of the budget constraint is unsuitable to be combined asa state variable π
π‘. Combining the money holdings, bonds,
and transfers to ππ‘= ππ‘+ (((1 + π
π‘β1)ππ‘β1
+ ππ‘β1
)/(1 + ππ‘)), the
householdβs financial assets, value function can be written as
π (ππ‘, ππ‘β1
) = max {V (ππ‘, ππ‘) + π½π (π
π‘+1, ππ‘)} . (20)
According to Walsh [9], after replacing ππ‘+1
, ππ‘, value
function can be expanded as
π (ππ‘, ππ‘β1
)
= max{V [ππ‘, 1 β π
π‘β π (π
π‘, ππ‘)]
+ π½π[ππ‘+1
+(1 + ππ‘) ππ‘+ ππ‘
1 + ππ‘+1
, π (ππ‘β1
, ππ‘)
+ (1 β πΏ) ππ‘β1
+ ππ‘β ππ‘β ππ‘β ππ‘]} .
(21)
Computing the partial derivatives with respect to controlvariables, we get
ππ (ππ‘, ππ‘β1
)
πππ‘
= Vπ[ππ‘, 1 β π
π‘β π (π
π‘, ππ‘)]
β Vπ[ππ‘, 1 β π
π‘β π (π
π‘, ππ‘)] ππ(ππ‘, ππ‘)
β π½ππ(ππ‘+1
, ππ‘) = 0,
ππ (ππ‘, ππ‘β1
)
πππ‘
= βVπ[ππ‘, 1 β π
π‘β π (π
π‘, ππ‘)]
+ π½ππ(ππ‘+1
, ππ‘) β ππ(ππ‘β1
, ππ‘) = 0,
ππ (ππ‘, ππ‘β1
)
πππ‘
= βVπ[ππ‘, 1 β π
π‘β π (π
π‘, ππ‘)] ππ
(ππ‘, ππ‘)
+ π½ππ(ππ‘+1
, ππ‘) β
1
1 + ππ‘+1
β π½ππ(ππ‘+1
, ππ‘) = 0.
(22)
Computing the partial derivatives of both sides of thevalue equation with respect to the state variables π
π‘, ππ‘β1
, weget
ππ(ππ‘, ππ‘β1
) = π½ππ(ππ‘+1
, ππ‘)
ππ(ππ‘, ππ‘β1
) = π½ππ(ππ‘+1
, ππ‘) [ππ(ππ‘β1
, ππ‘) + 1 β πΏ] .
(23)
4.2. Our Solving Approach. Now, we use our proposedmethod to solve the above Shopping-Time Model problem.First, let the Bellman equation with multiplier π
π‘be
π (ππ‘, ππ‘β1
) = max {V (ππ‘, ππ‘) + π½π (π
π‘+1, ππ‘)}
+ ππ‘[ππ‘+ π (π
π‘β1, ππ‘) + (1 β πΏ) π
π‘β1
β ππ‘β ππ‘β ππ‘β ππ‘] ,
(24)
6 Journal of Applied Mathematics
where state variable ππ‘+1
= ππ‘+1
+ (((1 + ππ‘)ππ‘+ππ‘)/(1 +π
π‘+1)).
Then, compute the partial derivatives with respect to controlvariables to derive first-order conditions,
ππ (ππ‘, ππ‘β1
)
πππ‘
= Vπ(ππ‘, ππ‘) β Vπ(ππ‘, ππ‘) ππ(ππ‘, ππ‘) = ππ‘,
ππ (ππ‘, ππ‘β1
)
πππ‘
= π½ππ(ππ‘+1
, ππ‘) β
1 + ππ‘
1 + ππ‘+1
= ππ‘,
ππ (ππ‘, ππ‘β1
)
πππ‘
= π½ππ(ππ‘+1
, ππ‘) = ππ‘,
ππ (ππ‘, ππ‘β1
)
πππ‘
= βVπ(ππ‘, ππ‘) ππ
(ππ‘, ππ‘) +
π½ππ(ππ‘+1
, ππ‘)
1 + ππ‘+1
= ππ‘.
(25)
Using the envelope theorem and computing the deriva-tives with respect to the state variables π
π‘, ππ‘β1
, we get
ππ(ππ‘, ππ‘β1
) = ππ‘,
ππ(ππ‘, ππ‘β1
) = ππ‘[ππ(ππ‘β1
, ππ‘) + 1 β πΏ] .
(26)
Now, we use the above results to compute the opportunitycost of holding money. Since the utility function is
π’ (π, π, π) = V [π, π β π β π (π,π)] , (27)
and based on (6), (25), and (26), we obtain
π’π
(ππ‘, ππ‘, ππ‘)
π’π(ππ‘, ππ‘, ππ‘)
=βVπ(ππ‘, ππ‘) ππ
(ππ‘, ππ‘)
Vπ(ππ‘, ππ‘) β Vπ(ππ‘, ππ‘) ππ(ππ‘, ππ‘)
=ππ‘β ((π½π
π(ππ‘+1
, ππ‘)) / (1 + π
π‘+1))
ππ‘
= 1 β((π½ππ(ππ‘+1
, ππ‘)) / (1 + π
π‘+1))
π½ππ(ππ‘+1
, ππ‘) β ((1 + π
π‘) / (1 + π
π‘+1))
= 1 β1
1 + ππ‘
=ππ‘
1 + ππ‘
.
(28)
Obviously, the expressions of the derived first-orderconditions by previousmethod seem to be tedious andmessy,and it is not so easy to compute the relevant results such as theopportunity cost of holding money. Our proposed methodis comparatively neat and can easily obtain relevant resultscorrectly.
5. Setting and Solution of Bellman Equation inCash-In-Advance Model
5.1. Model and Solution Using Previous Approach. In basicCash-In-Advance model, money is used to purchase goods.Money cannot yield utility itself, but the consumption offuture can yield utility. Svensson [12] assumed that agentsare available for spending only the cash carried over fromthe previous period.This is essentially different fromMoney-In-Utility model. Consider a simple form of discounted
household utility valueββ
π‘=0π½π‘
π’(ππ‘). More complicated utility
function will be discussed in the following sections. TakeCash-In-Advance constraint form as π
π‘β€ (ππ‘β1
/(1 + ππ‘)) + ππ‘
due to Svensson [12]. This means that the agent enters theperiod with money holdings π
π‘β1and receives a lump-sum
transfer ππ‘(in real currency terms) for consumption goods.
Bonds and capital may not be purchased by currency. Ifcapital is assumed to be purchased by money, the Cash-In-Advance constraint will become (π
π‘β1/(1 + π
π‘)) + ππ‘β₯ ππ‘+ ππ‘.
The budget constraint is rewritten in real terms as
π (ππ‘β1
) + ππ‘+ (1 β πΏ) π
π‘β1
+(1 + ππ‘β1
) ππ‘β1
+ ππ‘β1
1 + ππ‘
β₯ ππ‘+ ππ‘+ ππ‘+ ππ‘.
(29)
In monetary theory, constraints are expressed as inequal-ity frequently. This constraint describes that the represen-tative agentβs time π‘ real resources should be more thanor equal to the use of it, that is, purchasing consumption,capital, bonds, and money holdings that are then carried intoperiod π‘ + 1. Because of the assumption of rational agents,the certainty problems we are discussing, and the positiveopportunity cost of money holdings, the constraints becomeequations in equilibriums.
Output is only affected by state variable ππ‘β1
withoutconsidering the effect of labor supply. Following the solutionmethods ofWalsh [9], let the left side of the budget constraintbe a state variableπ
π‘= π(π
π‘β1)+ππ‘+(1βπΏ)π
π‘β1+(((1+π
π‘β1)ππ‘β1
+
ππ‘β1
)/(1 + ππ‘)). It will be very tedious adopting expansion
method with two constraints. Setting Bellman equation withmultipliers will be better. Let state variable π
π‘β1in value
function get the economic significance of money in first-order conditions. However, there is an overlapped settingwith π
π‘,
π (ππ‘, ππ‘β1
) = max {π’ (ππ‘) + π½π (π
π‘+1, ππ‘)}
+ ππ‘(ππ‘β ππ‘β ππ‘β ππ‘β ππ‘)
+ ππ‘(
ππ‘β1
1 + ππ‘
+ ππ‘β ππ‘) .
(30)
5.2. Our Solving Approach. Let ππ‘= π(π
π‘β1)+ππ‘+(1βπΏ)π
π‘β1+
(((1 + ππ‘β1
)ππ‘β1
)/(1 + ππ‘)). There is no overlap with π
π‘in the
value function. The Bellman equation is set as
π (ππ‘, ππ‘β1
) = max {π’ (ππ‘) + π½π (π
π‘+1, ππ‘)}
+ ππ‘(ππ‘+
ππ‘β1
1 + ππ‘
β ππ‘β ππ‘β ππ‘β ππ‘)
+ ππ‘(
ππ‘β1
1 + ππ‘
+ ππ‘β ππ‘) .
(31)
The solution of this problem is similar to the applicationof the proposed method in Sections 3 and 4. We will not giveunnecessary details here.
Note that Walsh [9] puts overlapping state variables ππ‘β1
andππ‘in value function; it is easy to get wrong results in those
Journal of Applied Mathematics 7
problems with Cash-In-Advance constraints although it is noproblem setting π(π
π‘, ππ‘β1
) with no more results requirednow. This will be seen in the following example of Section 6.
6. Setting and Solution of Bellman Equationin Model considering Both Labor Time andCash-In-Advance Constraints
6.1. Model and Solution Using Previous Approach. Assum-ing that money is used to purchase consumption goodsand investments, the Cash-In-Advance constraint becomes(ππ‘β1
/(1 + ππ‘)) + π
π‘β₯ ππ‘+ π₯π‘, where π₯ is investment and
π₯π‘
= ππ‘β (1 β πΏ)π
π‘β1. In considering of the effects of labor
supply on output and leisure on utility, we have ππ‘
= 1 β ππ‘.
The agentβs objective becomes
maxβ
β
π‘=0
π’ (ππ‘, ππ‘)
s.t. π (ππ‘β1
) + ππ‘+ (1 β πΏ) π
π‘β1
+(1 + ππ‘β1
) ππ‘β1
+ ππ‘β1
1 + ππ‘
β₯ ππ‘+ ππ‘+ ππ‘+ ππ‘,
ππ‘β1
1 + ππ‘
+ ππ‘β₯ ππ‘+ π₯π‘.
(32)
Output is affected not only by state variable ππ‘β1
but alsoby controllable factor π
π‘; the left side of the budget constraint
is unsuitable to be combined as a state variable. Thinkingof the setting of the Shopping-Time Model and Cash-In-Advance model above, it is plausible to put overlapped statevariables π
π‘= ππ‘+(((1+ π
π‘β1)ππ‘β1
+ππ‘β1
)/(1+ππ‘)) andπ
π‘β1in
value function following the solution methods of Walsh [9].The Bellman equation will be
π (ππ‘, ππ‘β1
, ππ‘β1
)
= max {π’ (ππ‘, ππ‘) + π½π (π
π‘+1, ππ‘, ππ‘)}
+ ππ‘[ππ‘+ π (π
π‘β1, ππ‘) + (1 β πΏ) π
π‘β1
β ππ‘β ππ‘β ππ‘β ππ‘]
+ ππ‘[ππ‘β ππ‘β ππ‘+ (1 β πΏ) π
π‘β1] .
(33)
The first-order conditions areππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π’π(ππ‘, ππ‘) = ππ‘+ ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π½ππ(ππ‘+1
, ππ‘, ππ‘) β
1 + ππ‘
1 + ππ‘+1
= ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π½ππ(ππ‘+1
, ππ‘, ππ‘) = ππ‘+ ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π½ππ
(ππ‘+1
, ππ‘, ππ‘)
+ π½ππ(ππ‘+1
, ππ‘, ππ‘) β
1
1 + ππ‘+1
= ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= βπ’π(ππ‘, ππ‘) + ππ‘ππ(ππ‘β1
, ππ‘) = 0.
(34)
Using the envelope theorem and computing the deriva-tives with respect to state variables, we get
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= ππ(ππ‘, ππ‘β1
, ππ‘β1
) = ππ‘+ ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘β1
= ππ(ππ‘, ππ‘β1
, ππ‘β1
)
= ππ‘[ππ(ππ‘β1
, ππ‘) + (1 β πΏ)]
+ ππ‘(1 β πΏ) ,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘β1
= ππ
(ππ‘, ππ‘β1
, ππ‘β1
) = (ππ‘+ ππ‘)
1
1 + ππ‘
.
(35)
When the question is to derive the effect of inflation rateon the steady-state capital-labor ratio, that is, the steady-state relationship of ππ π /ππ π and π
π π , assume that the aggregateproduction takes the form π¦
π‘= π(π
π‘β1, ππ‘) = π΄π
πΌ
π‘β1π1βπΌ
π‘.
Using (34) and (35),
π’π(ππ‘, ππ‘)
π’π(ππ‘, ππ‘)
=ππ‘ππ(ππ‘β1
, ππ‘)
ππ‘+ ππ‘
= ([π½ππ
(ππ‘+1
, ππ‘, ππ‘) + π½π
π(ππ‘+1
, ππ‘, ππ‘) β
1
1 + ππ‘+1
]
Γ ππ(ππ‘β1
, ππ‘) ) (π
π‘+ ππ‘)β1
= ([π½ (ππ‘+1
+ ππ‘+1
)1
1 + ππ‘+1
+ π½ (ππ‘+1
+ ππ‘+1
)1
1 + ππ‘+1
]
Γ ππ(ππ‘β1
, ππ‘) ) (π
π‘+ ππ‘)β1
= 2π½1
1 + ππ‘+1
ππ(ππ‘β1
, ππ‘) β
ππ‘+1
+ ππ‘+1
ππ‘+ ππ‘
.
(36)
Thenππ‘
ππ‘+ ππ‘
= 2π½1
1 + ππ‘+1
ππ‘+1
+ ππ‘+1
ππ‘+ ππ‘
. (37)
From (35),
ππ(ππ‘β1
, ππ‘) =
ππ(ππ‘, ππ‘β1
, ππ‘β1
) β (1 β πΏ) (ππ‘+ ππ‘)
ππ‘
= [1
π½β (1 β πΏ)]
ππ‘+ ππ‘
ππ‘
.
(38)
Rewriting the aggregate production as ππ
= πΌπ΄(ππ‘β1
/
ππ‘)πΌβ1, we have
ππ‘
ππ‘+ ππ‘
= (πΌπ΄)β1
(ππ‘β1
ππ‘
)
1βπΌ
[1
π½β (1 β πΏ)] . (39)
8 Journal of Applied Mathematics
Using (37), we obtain
(πΌπ΄)β1
(ππ‘β1
ππ‘
)
1βπΌ
[1
π½β (1 β πΏ)]
= 2π½1
1 + ππ‘+1
ππ‘+1
+ ππ‘+1
ππ‘+ ππ‘
.
(40)
Rewriting this equation, we get
ππ‘β1
ππ‘
= [2π½πΌπ΄
1 + ππ‘+1
(1
π½β 1 + πΏ)
β1
ππ‘+1
+ ππ‘+1
ππ‘+ ππ‘
]
1/(1βπΌ)
. (41)
In steady-state, ππ‘= ππ‘+1
= ππ π , ππ‘= ππ‘+1
= ππ π , ππ‘+1
=
ππ π , ππ‘β1
= ππ π , ππ‘= ππ π , from (41), we have
ππ π
ππ π
= [1 + ππ π
2πΌπ½(
1
π½β 1 + πΏ)]
1/(πΌβ1)
. (42)
This steady-state capital-labor ratio is derived by usingcurrent prevailing methods. However, it is a wrong result. Wewill provide the correct result by using the proposed methodof this paper.
6.2. Our Solving Approach. Output is affected by state vari-able π
π‘β1and control variable π
π‘; it is unsuitable to let the left
side of the budget constraint be combined as a state variable.We should separate π
π‘β1from it. Let state variable π
π‘β1in
value function alone get the economic significance of moneyin first-order conditions. Because overlaps of state variablesare not allowed according to our proposed method, we putππ‘= ππ‘+ (((1 + π
π‘β1)ππ‘β1
)/(1 + ππ‘)) in value function as a state
variable.First, set up value function with multipliers:
π (ππ‘, ππ‘β1
, ππ‘β1
)
= max {π’ (ππ‘, ππ‘) + π½π (π
π‘+1, ππ‘, ππ‘)}
+ ππ‘[ππ‘+
ππ‘β1
1 + ππ‘
+ π (ππ‘β1
, ππ‘)
+ (1 β πΏ) ππ‘β1
β ππ‘β ππ‘β ππ‘β ππ‘]
+ ππ‘[ππ‘β
(1 + ππ‘β1
) ππ‘β1
1 + ππ‘
+ππ‘β1
1 + ππ‘
β ππ‘β ππ‘+ (1 β πΏ) π
π‘β1] .
(43)
Second, compute the derivatives of state variables andderive first-order conditions:
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π’π(ππ‘, ππ‘) = ππ‘+ ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π½ππ(ππ‘+1
, ππ‘, ππ‘) β
1 + ππ‘
1 + ππ‘+1
= ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π½ππ(ππ‘+1
, ππ‘, ππ‘) = ππ‘+ ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= π½ππ
(ππ‘+1
, ππ‘, ππ‘) = ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= βπ’π(ππ‘, ππ‘) + ππ‘ππ(ππ‘β1
, ππ‘) = 0.
(44)
Third, compute the partial derivatives with respect to ππ‘,
ππ‘β1
, and ππ‘β1
and the envelope theorem yields
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘
= ππ(ππ‘, ππ‘β1
, ππ‘β1
) = ππ‘+ ππ‘,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘β1
= ππ(ππ‘, ππ‘β1
, ππ‘β1
)
= ππ‘[ππ(ππ‘β1
, ππ‘) + (1 β πΏ)] + π
π‘(1 β πΏ) ,
ππ (ππ‘, ππ‘β1
, ππ‘β1
)
πππ‘β1
= ππ
(ππ‘, ππ‘β1
, ππ‘β1
) = (ππ‘+ ππ‘)
1
1 + ππ‘
.
(45)
Finally, derive the steady-state capital-labor ratio by theresults above:
ππ π
ππ π
= [1 + ππ π
πΌπ½(
1
π½β 1 + πΏ)]
1/(πΌβ1)
. (46)
The process of deriving this ratio is similar to thederiving process of (42). We will not give unnecessarydetails here. Compare (46) with (42); the result of (42) is(1/2)1/(πΌβ1) times that of (46). Equation (46) is the correct
result. It is the different partial derivatives with respectto ππ‘in (34) and (44) that cause this deviation. In (34),
π½ππ(ππ‘+1
, ππ‘, ππ‘) + π½π
π(ππ‘+1
, ππ‘, ππ‘) β (1/(1 + π
π‘+1)) = π
π‘. In
(44), π½ππ(ππ‘+1
, ππ‘, ππ‘) = π
π‘. Fundamentally speaking, the
reason for the deviation is the overlapped setting of statevariables π
π‘andπ
π‘β1versus the nonoverlapped setting of state
variables ππ‘, ππ‘β1
, andππ‘β1
. One has to use the particular skillsin [9] that is hard to think about and grasp to obtain the rightanswer or get a wrong result adopting the overlapped settingπ(ππ‘, ππ‘β1
, ππ‘β1
). For this reason, the setting of value functionbyWalsh could not be generalized. In this model consideringboth Labor Time and Cash-In-Advance constraints, the basicmethods of incorporating money into general equilibriummodels are generalized, and our proposed method of setting
Journal of Applied Mathematics 9
and solution of Bellman equation demonstrates its clarityand validity and corrects the defect that some other currentmethods of setting and solution are easy to get wrong results.
7. Conclusions
As an important tool in theory economics, Bellman equationis very powerful in solving optimization problems of discretetime and is frequently used in monetary theory. It is hard tograsp the setting and solution of Bellman equation and easy toreach wrong conclusions since there is no general method toset Bellman equation or the settings of Bellman equation areexcessively flexible. In this paper, we provide a set of generalsetting and solution methods for Bellman equation withmultipliers. In the processes of solving monetary problems,comparing with other current methods in classic reference,our proposed method demonstrates its features of clarity,validity, correct results, easy operation, and generalization.Bellman equation is used not only in monetary problemsbut also in almost every dynamic programming problemassociated with discrete time optimization. Our future workis to study the applicability of the proposed method in thispaper in other areas.
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper.
Acknowledgments
Wanbo Luβs research is sponsored by the National ScienceFoundation of China (71101118) and the Program for NewCentury Excellent Talents in University (NCET-13-0961) inChina.
References
[1] M. Sidrauski, βRational choice and patterns of growth in amon-etary economy,β American Economic Review, vol. 57, no. 2, pp.534β544, 1967.
[2] W. Baumol, βThe transactions demand for cash,βQuarterly Jour-nal of Economics, vol. 67, no. 4, pp. 545β556, 1952.
[3] J. Tobin, βThe interest elasticity of the transactions demand forcash,β Review of Economics and Statistics, vol. 38, no. 3, pp. 241β247, 1956.
[4] R. W. Clower, βA reconsideration of the microfoundations ofmonetary theory,β Western Economic Journal, vol. 6, no. 1, pp.1β9, 1967.
[5] W. A. Brock, βMoney and growth: the case of long run perfectforesight,β International Economic Review, vol. 15, pp. 750β777,1974.
[6] B. T. McCallum and M. S. Goodfriend, βDemand for money:theoretical studies,β in The New Palgrave Dictionary of Eco-nomics, pp. 775β781, Palgrave MacMillan, Houndmills, UK,1987.
[7] D. Croushore, βMoney in the utility function: functional equiv-alence to a shopping-time model,β Journal of Macroeconomics,vol. 15, no. 1, pp. 175β182, 1993.
[8] P. A. Samuelson, βAn exact consumption-loan model of interestwith or without the social contrivance of money,β Journal ofPolitical Economy, vol. 66, no. 6, pp. 467β482, 1958.
[9] C. E. Walsh, Monetary Theory and Policy, The MIT Press,Cambridge, Mass, USA, 3rd edition, 2010.
[10] L. Gong,OptimizationMethods in Economics, PekingUniversityPress, Beijing, China, 2000.
[11] D. Romer,AdvancedMacroeconomics, McGraw-Hill, New York,NY, USA, 4th edition, 2011.
[12] L. E. O. Svensson, βMoney and asset prices in a cash-in-advanceeconomy,β Journal of Political Economy, vol. 93, no. 5, pp. 919β944, 1985.
Submit your manuscripts athttp://www.hindawi.com
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttp://www.hindawi.com
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Journal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
CombinatoricsHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
International Journal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttp://www.hindawi.com
Volume 2014 Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Stochastic AnalysisInternational Journal of