CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we...

56
C HAPTER 6 A PPLICATIONS TO P RODUCTION AND I NVENTORY Chapter 6 – p. 1/56

Transcript of CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we...

Page 1: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

CHAPTER 6

APPLICATIONS TO PRODUCTION AND INVENTORY

Chapter 6 – p. 1/56

Page 2: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

PRODUCTION APPLICATIONS

• Hwang, Fan, Erickson (1967)

• Hendricks et al. (1971)

• Bensoussan et al. (1974)

• Thompson-Sethi (1980)

• Stochastic Extension: Sethi-Thompson (1980)

• Kleindorfor et al. (1975)

• Kleindorfor-Lieber(1975)

• Singhal (1981)

Chapter 6 – p. 2/56

Page 3: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

THE PRODUCTION-INVENTORY MODEL

I(t) = the inventory level at timet (state variable),

P (t) = the production rate at timet (control variable),

S(t) = the sales rate at timet (exogenous variable);

assumed to be bounded and differentiable fort ≥ 0,

T = the length of the planning period,

I = the inventory goal level,

I0 = the initial inventory level,

P = the production goal level,

h = the inventory holding cost coefficient;h > 0,

c = the production cost coefficient;c ≥ 0,

ρ = the constant nonnegative discount rate; ρ ≥ 0.

Chapter 6 – p. 3/56

Page 4: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

THE PRODUCTION-INVENTORY MODEL CONT.

The objective function of the model is:

minP≥0

{

J =

∫ T

0e−ρt[

h

2(I − I)2 +

c

2(P − P )2]dt

}

. (1)

The inventory balance equation is:

I(t) = P (t) − S(t), I(0) = I0. (2)

Interpretation: We want to keep the inventory as close aspossible to its goal levelI, and also keep the production rateP as close as possible to its goal levelP . The quadraticterms(h/2)(I − I)2 and(c/2)(P − P )2 impose “penalties"for having eitherI or P not being close to its correspondinggoal level.

Chapter 6 – p. 4/56

Page 5: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE

The current-value Hamiltonian function is

H = λ(P − S) −h

2(I − I)2 −

c

2(P − P )2. (3)

We apply the Pontryagin maximum principle to get

∂H

∂P= λ − c(P − P ) = 0.

From this we obtain thedecision rule

P = P + λ/c. (4)

The optimal control is

P ∗ = max{P + λ/c, 0}. (5)

Chapter 6 – p. 5/56

Page 6: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE CONT.

With the assumptions of a sufficiently largeP and asufficiently smallI0, we haveP ∗ = P + λ/c, and we cansubstitute (4) into (2) to obtain

I = P + λ/c − S, I(0) = I0. (6)

The equation for the adjoint variable is

λ = ρλ −∂H

∂I= ρλ + h(I − I), λ(T ) = 0. (7)

We then use (7) to eliminateλ and (6) to eliminateλ fromthe resulting equation as follows:

I = λ/c − S = ρ(λ/c) + (h/c)(I − I) − S

= ρ(I − P + S) + (h/c)(I − I) − S.Chapter 6 – p. 6/56

Page 7: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE CONT.

We rewrite this as

I − ρI − α2I = −α2I − S − ρ(P − S), (8)

where the constantα is given by

α =√

h/c. (9)

We can now solve (8) by using the standard method

described in Appendix A. The auxiliary equation for (8) is

m2 − ρm − α2 = 0.

Chapter 6 – p. 7/56

Page 8: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE CONT.

It has two real roots

m1 = (ρ −√

ρ2 + 4α2)/2, m2 = (ρ +√

ρ2 + 4α2)/2. (10)

Note thatm1 < 0 andm2 > 0. We can therefore write thegeneral solution to (8) as

I(t) = a1em1t + a2e

m2t + Q(t), I(0) = I0, (11)

whereQ(t) is a special particular integral of (8). To get theother boundary condition, we differentiate (11), substitutethe result into (6), and solve forλ.

Chapter 6 – p. 8/56

Page 9: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE CONT.

We obtain

λ(t) = c(m1a1em1t +m2a2e

m2t + Q+S− P ), λ(T ) = 0. (12)

For ease of expressinga1 anda2, let us define two moreconstants

b1 = I0 − Q(0), (13)

b2 = P − Q(T ) − S(T ). (14)

We now impose the boundary conditions in (11) and (12)and solve fora1 anda2 as follows:

a1 =b2e

m1T − m2b1e(m1+m2)T

m1e2m1T − m2e(m1+m2)T, (15)

a2 =b1m1e

2m1T − b2em1T

m1e2m1T − m2e(m1+m2)T. (16)

Chapter 6 – p. 9/56

Page 10: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE CONT.

If we recall thatm1 is negative andm2 is positive, thenwhenT is sufficiently large so thatem1T ande2m1T arenegligible, we can write

a1 ≈ b1, (17)

a2 ≈b2

m2e−m2T . (18)

We will break each expression into three parts: the first partlabeledStarting Correction is important only whent issmall; the second part labeledTurnpike Expression issignificant for all values oft; and the third part labeledEnding Correction is important only whent is close toT .

Chapter 6 – p. 10/56

Page 11: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE CONT.

Starting Correction Turnpike Expression Ending Correction

I = (b1em1t)+ (Q)+ (

b2

m2em2(t−T )) (19)

P = (m1b1em1t)+ (Q + S)+ (b2e

m2(t−T )) (20)

λ = c(m1b1em1t)+ c(Q + S − P )+ c(b2e

m2(t−T )) (21)

Note that ifb1 = 0, which by (13) meansI0 = Q(0), thenthere is no starting correction. In other words,I0 = Q(0) is astarting inventory that causes the solution to be on theturnpike initially. In the same way, ifb2 = 0, then the endingcorrection vanishes in each of these formulas, and thesolution stays on the turnpike until the end.

Chapter 6 – p. 11/56

Page 12: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

THE INFINITE HORIZON SOLUTION

Note that asT → ∞, the ending correction disappearsbecausea2 defined in (16) becomes0. We now have

λ(t) = c[m1b1em1t + Q + S − P ]. (22)

If S is bounded, thenQ is bounded, and therefore,limt→∞ λ(t) is bounded. Then forρ > 0,

limt→∞

e−ρtλ(t) = 0. (23)

By the sufficiency of the maximum principle conditions(Section 2.4), it can be verified that the limiting solution

I(t) = b1em1t + Q, P (t) = m1b1e

m1t + Q + S (24)

is optimal. IfI(0) = Q(0), the solution is always on theturnpike.

Chapter 6 – p. 12/56

Page 13: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

THE INFINITE HORIZON SOLUTION CONT.

The triple{I, P, λ} = {Q, Q + S, c(Q + S − P )} represents anonstationary turnpike. IfI(0) 6= Q(0), thenb1 6= 0 and theexpressions (24) imply that the path of inventory andproduction only approach but never attain the turnpike.

Note that the solution does not satisfy the sufficiencytransversality condition whenρ = 0. In this case, everysolution gives an infinite value (which is the worst valuesince we are minimizing) for the cost objective function.The limiting solution, asρ → 0, is the Golden Path definedin Section 3.5, and it could be deemed to be the “optimal"infinite horizon solution for the undiscounted case.

Chapter 6 – p. 13/56

Page 14: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

A COMPLETE ANALYSIS OF THE CONSTANT

POSITIVE S CASE WITH INFINITE HORIZON

The solution (11) of the differential equation (8) reduces to

I(t) = a1em1t + a2e

m2t + Q.

Substituting in (8) and recognizing thatS is a constant sothatS = 0, we obtain

Q =ρ

α2(P − S) + I . (25)

Note thata1 anda2 are given in (15) and (16) in terms ofb1

andb2, which from (13), (14), and (25) are

b1 = I0 − I − (ρ/α2)(P − S) andb2 = P − S.

˙I = 0, ˙λ = 0, andP = P + λ/c. (26)

Chapter 6 – p. 14/56

Page 15: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

A COMPLETE ANALYSIS OF THE CONSTANT

POSITIVE S CASE WITH INFINITE HORIZON

CONT.

If I0 = Q, then the optimal solution stays on the turnpike. IfI0 6= Q, then the optimal solution is given by

P (t) = m1b1em1t + S = m1(I0 − Q)em1t + S. (27)

Clearly if I0 ≤ Q, this provides a nonnegative optimalproduction throughout. In caseI0 > Q, note first thatP (t)increases witht and

P (0) = m1(I0 − Q) + S. (28)

Furthermore, ifI0 − Q > −S/m1, we have a negative valuefor P (0) which is infeasible. By (5),P ∗(0) = 0. We can nowdepict this situation in Figure 6.1. The timet shown in thefigure is the time at whichP + λ(t)/c = 0.

Chapter 6 – p. 15/56

Page 16: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.1: OPTIMAL PRODUCTION AND

INVENTORY LEVELS

Chapter 6 – p. 16/56

Page 17: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

A COMPLETE ANALYSIS OF THE CONSTANT

POSITIVE S CASE WITH INFINITE HORIZON

CONT.

It should be obvious that

I(t) = Q −S

m1. (29)

For t < t, we have

I = −S ⇒ I = I0 − St, (30)

λ = ρλ + h(I − I), λ(t) = −cP . (31)

We can substituteI0 − St for I in equation (31) and solve

for λ.

Chapter 6 – p. 17/56

Page 18: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

A COMPLETE ANALYSIS OF THE CONSTANT

POSITIVE S CASE WITH INFINITE HORIZON

CONT.

Note that we can easily obtaint as

I0 − St = Q −S

m1⇒ t =

I0 − Q

S+

1

m1. (32)

For t > t, the solution is given by (19), (20), and (21) witht

replaced by(t − t) andb1 = I(t) − Q = −S/m1.

Chapter 6 – p. 18/56

Page 19: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SPECIAL CASE OF TIME VARYING DEMANDS

Let ρ = 0 andT < ∞. For the first set of examples weassume thatS(t) is a polynomial of degree2p or 2p − 1 sothatS(2p+1) = 0, whereS(k) denotes thekth time derivativeof S with respect tot. That is,

S(t) = C0t2p + C1t

2p−1 + ... + C2p, (33)

where at least one ofC0 andC1 is not zero. Then, it is easyto show that a particular solution of (11) is

Q(t) = I +1

α2S(1) +

1

α4S(3) + ... +

1

α2pS(2p−1). (34)

Chapter 6 – p. 19/56

Page 20: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SPECIAL CASE OF TIME VARYING DEMANDS

CONT.

Case of seasonal demand

We assume thatS(t) is a sinusoidal of form

S(t) = A sin πt + C, (35)

whereA andC are constants. A particular solution of (8)

for thisS is

Q(t) = I +πA

α2 + π2cos πt. (36)

Chapter 6 – p. 20/56

Page 21: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

EXAMPLE 6.1

AssumeP = 30, I = 15, T = 8, ρ = 0, and h = c = 1 sothatα = 1, m1 = −1, andm2 = 1. Assume

S(t) = t(t − 4)(t − 8) + 30 = t3 − 12t2 + 32t + 30.

Solution. It is then easy to show from (34) that

Q(t) = 3t2 − 24t + 53 andQ(t) = 6t − 24.

Then from (19) and (20),

I(t) = (I0 − 53)e−t + Q(t) − 24et−8,

P (t) = −(I0 − 53)e−t + Q(t) + S(t) − 24et−8.

Chapter 6 – p. 21/56

Page 22: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.2: SOLUTION OF EXAMPLE 6.1 WITH

I0 = 10

Chapter 6 – p. 22/56

Page 23: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.3: SOLUTION OF EXAMPLE 6.1 WITH

I0 = 50

Chapter 6 – p. 23/56

Page 24: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.4: SOLUTION OF EXAMPLE 6.1 WITH

I0 = 30

Chapter 6 – p. 24/56

Page 25: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

EXAMPLE 6.2

As another example assume that

S(t) = A + Bt +K

k=1

Ck sin(πDkt + Ek). (37)

Solution. Making use of formulas (34) and (36) we have the

special particular integral

Q(t) = I1 +1

α2B +

K∑

k=1

πCkDk

α2 + (πDk)2cos(πDkt + Ek). (38)

Chapter 6 – p. 25/56

Page 26: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

CONTINUOUS WHEAT TRADING MODEL

T = the horizon time,

x(t) = the cash balance in dollars at timet,

y(t) = the wheat balance in bushels at timet,

v(t) = the rate of purchase of wheat in bushels per unittime; a negative purchase means a sale,

p(t) = the price of wheat in dollars per bushel at timet,

r = the constant positive interest rate earned on the cashbalance,

h(y) = the cost of holdingy bushels per unit time.

Chapter 6 – p. 26/56

Page 27: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

CONTINUOUS WHEAT TRADING MODEL CONT.

The state equations are

x = rx − h(y) − pv, x(0) = x0, (39)

y = v, y(0) = y0, (40)

and the control constraints are

−V2 ≤ v(t) ≤ V1, (41)

whereV1 andV2 are nonnegative constants. The objectivefunction is to

Maximize{ J = x(T ) + p(T )y(T ) } (42)

subject to (39)-(41).

Chapter 6 – p. 27/56

Page 28: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION BY THE MAXIMUM PRINCIPLE

Introduce the adjoint variablesλ1 andλ2 and define theHamiltonian function

H = λ1[rx − h(y) − pv] + λ2v. (43)

The adjoint equations are

λ1 = −λ1r, λ1(T ) = 1, (44)

λ2 = h′(y)λ1, λ2(T ) = p(T ). (45)

It is easy to solve (44) as

λ1(t) = er(T−t), (46)

λ2(t) = p(T ) −

∫ T

t

h′(y(τ))er(T−τ)dτ. (47)

From (43) the optimal control isv∗(t) = bang[−V2, V1;λ2(t) − λ1(t)p(t)]. (48)

Chapter 6 – p. 28/56

Page 29: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

COMPLETE SOLUTION OF A SPECIAL CASE

For this special case we assumeh(y) = 1

2 |y|, r = 0, x(0) = 10, y(0) = 0, V1 = V2 = 1, T = 6,and

p(t) =

{

3 for 0 ≤ t ≤ 3,

4 for 3 < t ≤ 6.(49)

For this case withr = 0, we haveλ1(t) = 1 for all t from(46) so that the TPBVP is

x = −1

2|y| − pv, x(0) = 10, (50)

y = v, y(0) = 0, (51)

λ2(t) =1

2sgn(y), λ2(6) = 4. (52)

Chapter 6 – p. 29/56

Page 30: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.5: THE PRICE TRAJECTORY (6.49)

Chapter 6 – p. 30/56

Page 31: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

COMPLETE SOLUTION OF A SPECIAL CASE

CONT.

The optimal policy (48) reduces to

v∗(t) = bang[−1, 1;λ2(t) − p(t)]. (53)

Sincep(t) is increasing, short-selling is never optimal.Since the storage cost is 1/2 unit per unit time, it never paysto store wheat more than 2 time units. Becausey(0) = 0, wehavev∗(t) = 0 for 0 ≤ t ≤ 1. This obviously must be asingular control. Suppose we start buying wheat att∗ > 1.

From (53) the rate of buying is clearly 1; clearly buying willcontinue at this rate untilt = 3, and no longer. In order notto lose money because of storing wheat, it must be soldwithin 2 time units of its purchase.

Chapter 6 – p. 31/56

Page 32: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

COMPLETE SOLUTION OF A SPECIAL CASE

CONT.

Clearly we should start selling att = 3+ at the maximum

rate of 1, and continue until a last sale timet∗∗.We must

have

3 − t∗ = t∗∗ − 3. (54)

Thus,v∗(t) = 0 in the interval[t∗∗, 6], which is also a

singular control. With this policy,y(t) > 0 for all

t ∈ (t∗, t∗∗).

Chapter 6 – p. 32/56

Page 33: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

COMPLETE SOLUTION OF A SPECIAL CASE

CONT.

From (52),λ2 = 1/2 in the interval(t∗, t∗∗). In order to havea singular control in the interval[t∗∗, 6], we must haveλ2(t) = 4 in that interval. Also, in order to have a singularcontrol in [0, t∗], we must haveλ2(t) = 3 in that interval.Thus,λ2(t

∗∗) − λ2(t∗) = 1, which with λ2 = Y2 allows us to

conclude that

t∗∗ − t∗ = 2, (55)

and thereforet∗ = 2 andt∗∗ = 4. Thus from (52) and (53),

λ2(t) =

3, 0 ≤ t ≤ 2,

2 + t/2, 2 ≤ t ≤ 4,

4, 4 ≤ t ≤ 6.

(56)

Chapter 6 – p. 33/56

Page 34: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.6: ADJOINT VARIABLE, OPTIMAL

POLICY AND INVENTORY IN THE WHEAT

TRADING MODEL

Chapter 6 – p. 34/56

Page 35: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

THE WHEAT TRADING MODEL WITH NO

SHORT-SELLING

We assumeh(y) = y/2, r = 0, x(0) = 10, y(0) = 1,V1 = V2 = 1, T = 3, and

p(t) =

{

−2t + 7 for 0 ≤ t < 2,

t + 1 for 2 ≤ t ≤ 3.(57)

The statement of the problem is:

max{J = x(3) + p(3)y(3) = x(3) + 4y(3)}

subject tox = −1

2y − pv, x(0) = 10,

y = v, y(0) = 1,

v + 1 ≥ 0, 1 − v ≥ 0, y ≥ 0.

(58)

Chapter 6 – p. 35/56

Page 36: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION OF THE WHEAT TRADING PROBLEM

The Hamiltonian is

H = λ1(−y/2 − pv) + λ2v. (59)

The optimal control is

v∗(t) = bang[−1, 1;λ2(t) − λ1(t)p(t)] wheny > 0. (60)

Whenevery = 0, we must imposey = v ≥ 0 in order toinsure that no short-selling occurs. Therefore,

v∗(t) = bang[0, 1;λ2(t) − λ1(t)p(t)] wheny = 0. (61)

Next we form the Lagrangian

L = H + µ1(v + 1) + µ2(1 − v) + ηv. (62)

Chapter 6 – p. 36/56

Page 37: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION OF THE WHEAT TRADING PROBLEM

CONT.

The complementary slackness conditions:

µ1 ≥ 0, µ1(v + 1) = 0, (63)

µ2 ≥ 0, µ2(1 − v) = 0, (64)

η ≥ 0, ηy = 0, η ≤ 0. (65)

Furthermore, the optimal trajectory must satisfy

∂L

∂v= λ2 − pλ1 + µ1 − µ2 + η = 0. (66)

With r = 0, we getλ1 = 1 as before, and

λ2 = −∂L

∂y= 1/2, λ2(3) = 4. (67)

Chapter 6 – p. 37/56

Page 38: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION OF THE WHEAT TRADING PROBLEM

CONT.

From this we see thatλ2 is always increasing exceptpossibly at a jump time. Lett be a time such that there is nojump in the interval(t, 3]. Then,

λ2(t) = t/2 + 5/2 for t ≤ t ≤ 3, (68)

t/2 + 5/2 = −2t + 7 ⇒ t = 1.8. (69)

λ2(t) = t/2 + 5/2 for 1 ≤ t ≤ 3. (70)

Using (66) withλ1 = 1 in the interval(1, 1.8] andv∗ = 0 sothatµ1 = µ2 = 0, we have

λ2 − p + µ1 − µ2 + η = λ2 − p + η = 0,

η(t) = p(t) − λ2(t) = −5/2t + 9/2 and η = −5/2 ≤ 0

for t ∈ (1, 1.8]. (71)Chapter 6 – p. 38/56

Page 39: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION OF THE WHEAT TRADING PROBLEM

CONT.

Sinceht = 0, the jump condition in (4.29) for the

Hamiltonian atτ = 1 reduces to

H[x∗(1), u∗(1−), λ(1−), 1] = H[x∗(1), u∗(1+), λ(1+), 1].

We can rewrite the condition as

λ1(1−)[−y(1)/2 − p(1−)v∗(1−)] + λ2(1

−)v∗(1−) =

λ1(1+)[−y(1)/2 − p(1+)v∗(1+)] + λ2(1

+)v∗(1+).

Sinceλ1(t) = 1 for all t,

−p(1−)v∗(1−)+λ2(1−)v∗(1−) = −p(1+)v∗(1+)+λ2(1

+)v∗(1+).

Chapter 6 – p. 39/56

Page 40: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

SOLUTION OF THE WHEAT TRADING PROBLEM

CONT.

Substituting the values ofp(1−) = p(1+) = 5 from (57),λ2(1

+) = 3 from (70), andv∗(1+) = 0 andv∗(1−) = −1 fromthe above discussion, we obtain

− 5(−1) + λ2(1−)(−1) = −5(0) + 3(0) = 0 ⇒ λ2(1

−) = 5. (72)

λ2(1−) = λ2(1

+) + ζ(1) ⇒ ζ(1) = λ2(1−) − λ2(1

+) = 5 − 3 = 2.

Furthermore, using (72) and (67),

λ2(t) = t/2 + 9/2 for t ∈ [0, 1), (73)

and the optimal control condition (60) holds, justifying oursupposition thatv∗ = −1 in this interval.

Chapter 6 – p. 40/56

Page 41: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.7: ADJOINT TRAJECTORY AND

OPTIMAL POLICY FOR THE WHEAT TRADING

MODEL

Chapter 6 – p. 41/56

Page 42: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

DECISION HORIZONS AND FORECAST

HORIZONS

In some dynamic problems it is possible to show that the

optimal decisions during an initial positive time intervalare

either partially or wholly independent of the data from some

future time onwards. In such cases, a forecast of the future

data needs to be made only as far as that time to make

optimal decisions in the initial time interval. The initialtime

interval is called thedecision horizon and the time up to

which data is required to make the optimal decisions during

the decision horizon is called theforecast horizon.

Chapter 6 – p. 42/56

Page 43: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

HORIZONS FOR THE WHEAT TRADING MODEL

We will demonstrate thatt = 1 is a decision horizon as well

as a weak forecast horizon.

Chapter 6 – p. 43/56

Page 44: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.8: DECISION HORIZON AND OPTIMAL

POLICY FOR THE WHEAT TRADING MODEL

Chapter 6 – p. 44/56

Page 45: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

HORIZONS FOR THE WHEAT TRADING MODEL

WITH WAREHOUSING CONSTRAINT

We modify the example by adding the warehousingconstrainty ≤ 1 or

1 − y ≥ 0, (74)

changing the terminal time toT = 4, and defining the pricetrajectory to be

p(t) =

{

−2t + 7 for t ∈ [0, 2),

t + 1 for t ∈ [2, 4].(75)

The optimal control is defined in three parts as

v∗(t) = bang[−1, 1;λ2(t) − p(t)] when0 < y < 1, (76)

v∗(t) = bang[0, 1;λ2(t) − p(t)] wheny = 0, (77)

v∗(t) = bang[−1, 0;λ2(t) − p(t)] wheny = 1. (78)Chapter 6 – p. 45/56

Page 46: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

HORIZONS FOR THE WHEAT TRADING MODEL

WITH WAREHOUSING CONSTRAINT CONT.

Defining a Lagrange multiplierη1 for the derivative of (74),i.e., for−y = −v ≥ 0, we form the Lagrangian

L = H + µ1(v + 1) + µ2(1 − v) + ηv + η1(−v), (79)

whereµ1, µ2, andη satisfy (63)-(65) andη1 satisfies

η1 ≥ 0, η1(1 − y) = 0, η1 ≤ 0. (80)

Furthermore, the optimal trajectory must satisfy

∂L

∂v= λ2 − p + µ1 − µ2 + η − η1 = 0. (81)

As before,λ1 = 1 andλ2 satisfies

λ2 = 1/2, λ2(4) = p(4) = 5. (82)Chapter 6 – p. 46/56

Page 47: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

HORIZONS FOR THE WHEAT TRADING MODEL

WITH WAREHOUSING CONSTRAINT CONT.

Let t be the time of the last jump of the adjoint function

λ2(t) before the terminal timeT = 4 . Then,

λ2(t) = t/2 + 3 for t ≤ t ≤ 4. (83)

Its graph in Figure 6.9 intersectsp(t) at t = 8/5, and it stays

abovep(t) in the interval[8/5, 4]. The optimal policy in the

absence of the warehousing constraint would be to buy at

the maximum rate. Since such a policy would violate the

constraint, we can conclude thatt > 8/5.

Chapter 6 – p. 47/56

Page 48: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.9: OPTIMAL POLICY AND HORIZONS

FOR THE WHEAT TRADING MODEL WITH

WAREHOUSE CONSTRAINT

Chapter 6 – p. 48/56

Page 49: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

HORIZONS FOR THE WHEAT TRADING MODEL

WITH WAREHOUSING CONSTRAINT CONT.

To find t, we insert a line of slope1/2 above the minimum

price att = 2 in such a way that its two intersections with

p(t) are exactly one time unit apart (the time required to fill

up the warehouse). Thus,

−2(t − 1) + 7 + (1/2)(1) = t + 1 ⇒ t = 17/6.

Then,

λ2(t) =

t/2 + 9/2 for t ∈ [0, 1),

t/2 + 29/12 for t ∈ [1, 17/6),

t/2 + 3 for t ∈ [17/6, 4].

(84)

Chapter 6 – p. 49/56

Page 50: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

VERIFICATION OF THE MAXIMUM PRINCIPLE

To complete the maximum principle, we must deriveexpressions for the Lagrange multipliers in the fourintervals shown in Figure 6.9.Interval [0, 1) : µ2 = η = η1 = 0, µ1 = p − λ2 > 0,

v∗ = −1, 0 < y∗ < 1.

Interval [1, 11/6) : µ1 = µ2 = η1 = 0, η = p − λ2 > 0, η ≤ 0,

v∗ = 0, y∗ = 0.

Interval [11/16, 17/6) : µ1 = η = η1 = 0, µ2 = λ2 − p > 0,

v∗ = 1, 0 < y∗ < 1.Chapter 6 – p. 50/56

Page 51: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

VERIFICATION OF THE MAXIMUM PRINCIPLE

CONT.

Interval [17/6, 4] : µ1 = µ2 = η = 0, η1 = λ2 − p > 0, η1 ≤ 0,

v∗ = 0, y∗ = 1.

Time t = 1 is a decision horizon, andt = 17/6 is a strong

forecast horizon.

Chapter 6 – p. 51/56

Page 52: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

EXAMPLE 6.3

Assume the price trajectory to be

p(t) =

−2t + 7 for t ∈ [0, 2),

t + 1 for t ∈ [2, 17/6),

25t/7 − 44/7 for t ∈ [17/6, 4],

which is sketched in Figure 6.10. Note that the price

trajectory up to time17/6 is the same as before, and the

price after time17/6 goes above the extension of the price

shield in Figure 6.9.

Chapter 6 – p. 52/56

Page 53: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.10: OPTIMAL POLICY AND HORIZONS

FOR EXAMPLE 6.3

Chapter 6 – p. 53/56

Page 54: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

EXAMPLE 6.4

Assume the price trajectory to be

p(t) =

−2t + 7 for t ∈ [0, 2),

t + 1 for t ∈ [2, 17/6),

t/2 + 21/4 for t ∈ [17/6, 4],

which is sketched in Figure 6.11.

Note that if we had solved the problem withT = 1, theny∗(1) = 0; and if we had solved the problem withT = 17/6,theny∗(1) = 0 andy∗(17/6) = 1. The latter problem has thesmallestT such that bothy∗ = 0 andy∗ = 1 occur fort > 0,

given the price trajectory. This is one of the ways that time17/6 can be found to be a forecast horizon along with thedecision horizon at time1.

Chapter 6 – p. 54/56

Page 55: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

FIGURE 6.11: OPTIMAL POLICY AND HORIZONS

FOR EXAMPLE 6.4

Chapter 6 – p. 55/56

Page 56: CHAPTER 6sethi/Prosper/Control-Tex-Ch6_031309.pdf · SOLUTION BY THE MAXIMUM PRINCIPLE CONT. If we recall that m1 is negative and m2 is positive, then when T is sufficiently large

ADDITIONAL REFERENCES

There are other ways to find strong forecast horizons; see

Pekelman (1974, 1975, 1979), Lundin and Morton (1975),

Morton (1978), Kleindorfer and Lieber (1979), Sethi and

Chand (1979, 1981), Chand and Sethi (1982, 1983, 1990),

Bensoussan, Crouhy, and Proth (1983), Teng, Thompson,

and Sethi (1984), Bean and Smith (1984), Bes and Sethi

(1988), Bhaskaran and Sethi (1987, 1988), Chand, Sethi,

and Proth (1990), Bylka and Sethi (1992), Bylka, Sethi, and

Sorger (1992), and Chand, Sethi, and Sorger (1992),

Dawande, Gavirneni, Naranpanawe, and Sethi (2005).

Chapter 6 – p. 56/56