MIT2 852S10 Control
Transcript of MIT2 852S10 Control
-
8/22/2019 MIT2 852S10 Control
1/108
MIT2.852
ManufacturingSystemsAnalysis
Lectures1921
Scheduling:Real-TimeControlofManufacturingSystems
StanleyB.GershwinSpring,2007
Copyrightc2007
Stanley
B.
Gershwin.
-
8/22/2019 MIT2 852S10 Control
2/108
DefinitionsEventsmaybecontrollable ornot,andpredictable
ornot.controllable uncontrollable
predictable
loadinga
part
lunch
unpredictable ??? machinefailure
Copyright2007StanleyB.Gershwin.c 2
-
8/22/2019 MIT2 852S10 Control
3/108
DefinitionsSchedulingistheselectionoftimesforfuturecontrollableevents.
Ideally,schedulingsystemsshoulddealwithallcontrollableevents,andnotjustproduction.Thatis,theyshouldselecttimesforoperations,
set-upchanges,preventivemaintenance,etc.Theyshouldatleastbeaware ofset-upchanges,
preventivemaintenance,etc.whentheyselecttimesforoperations.
Copyright2007StanleyB.Gershwin.c 3
-
8/22/2019 MIT2 852S10 Control
4/108
DefinitionsBecauseofrecurringrandomevents,schedulingis
anon-goingprocess,andnotaone-timecalculation.
Scheduling,orshopfloorcontrol,isthebottomofthescheduling/planninghierarchy.Ittranslatesplansintoevents.
Copyright c 42007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
5/108
IssuesinFactoryControlProblemsaredynamic;currentdecisionsinfluence
futurebehaviorandrequirements.Therearelargenumbersofparameters,time-varying
quantities,andpossibledecisions.Sometime-varyingquantitiesarestochastic.Somerelevantinformation(MTTR,MTTF,amountof
inventoryavailable,etc.) isnotknown.Somepossiblecontrolpoliciesareunstable.Copyright c 52007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
6/108
Dynamic ExampleProgramming Problem
DiscreteTime,DiscreteState,DeterministicF6
8
A1
B L5
6
22
G5
1064M
9
2D 7
72
4 N
594E
5 I 86
136 2C
H
J
1
4Z
K 3 O
Problem: findtheleastexpensivepathfromAtoZ.Copyright c 62007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
7/108
Dynamic ExampleProgramming Problem
Letg(i,j)bethecostoftraversingthelinkfromitoj.Leti(t)bethetthnodeonapathfromAtoZ.Thenthepathcostis
Tg(i(t1), i(t))
t=1whereT isthenumberofnodesonthepath,i(0)=A,andi(T) =Z.T isnotspecified;itispartofthesolution.
Copyright c 72007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
8/108
Dynamic ExampleProgramming SolutionApossibleapproachwouldbetoenumerateallpossiblepaths
(possiblesolutions).However,therecanbealotofpossiblesolutions.
Dynamicprogrammingreducesthenumberofpossiblesolutionsthatmustbeconsidered.Goodnews: itoftengreatly reducesthenumberofpossiblesolutions.Badnews: itoftendoesnotreduceitenoughtogiveanexactoptimalsolutionpractically(ie,withlimitedtimeandmemory).Thisisthecurseofdimensionality.
Goodnews: wecanlearnsomethingbycharacterizingtheoptimalsolution,andthatsometimeshelpsingettingananalyticaloptimalsolutionoranapproximation.
Goodnews: ittellsussomethingaboutstochasticproblems.Copyright2007StanleyB.Gershwin.c 8
-
8/22/2019 MIT2 852S10 Control
9/108
Dynamic ExampleProgramming Solution
InsteadofsolvingtheproblemonlyforAastheinitialpoint,wesolveitforall possibleinitialpoints.Foreverynodei,defineJ(i)tobetheoptimalcosttogo fromNodeitoNodeZ(thecostoftheoptimalpathfromitoZ).Wecanwrite
TJ(i) = g(i(t1), i(t))
t=1wherei(0)=i;i(T) =Z;(i(t1), i(t))isalinkforeveryt.Copyright c 92007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
10/108
Dynamic ExampleProgramming Solution
ThenJ(i)satisfiesJ(Z) = 0
and,iftheoptimalpathfromitoZ traverseslink(i,j),J(i) =g(i,j) +J(j).
i j
Z
Copyright2007StanleyB.Gershwin.c 10
-
8/22/2019 MIT2 852S10 Control
11/108
Dynamic ExampleProgramming Solution
SupposethatseverallinksgooutofNodei.
4j
3j
2j
5j
1j
6j
i
Z
Supposethatforeachnodej forwhichalinkexistsfromitoj,theoptimalpathandoptimalcostJ(j)fromj toZ isknown.Copyright2007StanleyB.Gershwin.c 11
-
8/22/2019 MIT2 852S10 Control
12/108
Dynamic ExampleProgramming Solution
ThentheoptimalpathfromitoZ istheonethatminimizesthesumofthecostsfromitojandfromj toZ.Thatis,
J(i)=min [g(i,j) +J(j)]j
wheretheminimizationisperformedoveralljsuchthatalinkfromitojexists.ThisistheBellmanequation.This
is
a
recursion
or
recursive
equation
because
J()
appears
onbothsides,althoughwithdifferentarguments.J(i)canbecalculatedfromthisifJ(j)isknownforeverynodejsuchthat(i,j)isalink.Copyright c 122007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
13/108
Dynamic ExampleProgramming SolutionBellmansPrincipleofOptimality: ifiandjarenodesonanoptimalpathfromAtoZ,thentheportionofthatpathfromAtoZbetweeniandj isanoptimalpathfromitoj.
A
j
i
Copyright2007StanleyB.Gershwin.c 13
-
8/22/2019 MIT2 852S10 Control
14/108
Dynamic ExampleProgramming Solution
Example: AssumethatwehavedeterminedthatJ(O) = 6andJ(J)=11.TocalculateJ(K),
g(K,O) +J(O)J(K)= ming(K,J) +J(J)
3 + 6= min = 9.9+11
Copyright2007StanleyB.Gershwin.c 14
-
8/22/2019 MIT2 852S10 Control
15/108
Dynamic ExampleProgramming Solution
Algorithm1.SetJ(Z) = 0.2.Findsomenodeisuchthat
J(i)hasnotyetbeenfound,andforeachnodej inwhichlink(i,j)exists,J(j)isalready
calculated.AssignJ(i)accordingto
J(i)=min [g(i,j) +J(j)]j3.RepeatStep2untilallnodes,includingA,havecosts
calculated.Copyright c 152007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
16/108
K
Dynamic ExampleProgramming Solution
F
A
B11 L
5
6HC
14
D
EI
O
J
9
G
M
N48
14
13
11
17
12
136
11
Z
Copyright2007StanleyB.Gershwin.c 16
-
8/22/2019 MIT2 852S10 Control
17/108
Dynamic ExampleProgramming SolutionTheimportantfeaturesofadynamicprogrammingproblemarethestate(i);thedecision (togotojafteri);
theobjectivefunction T
t=1g(i(t1), i(t))thecost-to-gofunction(J(i));theone-steprecursionequationthatdeterminesJ(i)
(J(i)=minj[g(i,j) +J(j)]); thatthesolutionisdeterminedforeveryi, notjustAandnotjustnodesontheoptimalpath;
thatJ(i)dependsonthenodestobevisitedafteri, notthosebetweenAandi.Theonlythingthatmattersisthepresentstateandthefuture;
thatJ(i)isobtainedbyworkingbackwards.Copyright2007StanleyB.Gershwin.c 17
-
8/22/2019 MIT2 852S10 Control
18/108
Dynamic ExampleProgramming Solution
Thisproblemwasdiscretetime,discretestate,deterministic.
Otherversions:discretetime,discretestate,stochasticcontinuoustime,discretestate,deterministiccontinuoustime,discretestate,stochasticcontinuoustime,mixedstate,deterministiccontinuoustime,mixedstate,stochastic
instochasticsystems,weoptimizetheexpected cost.Copyright c 182007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
19/108
ProgrammingDiscretetime,discretestate
StochasticDynamic
Supposeg(i,j)isarandomvariable;orifyouareatiandyouchoosej,youactuallygotokwith
probabilityp(i,j,k).Thenthecostofasequenceofchoicesisrandom.Theobjectivefunctionis
TE g(i(t1), i(t))
t=1
andwecandefineJ(i) =Emin[g(i,j) +J(j)]
jCopyright2007StanleyB.Gershwin.c 19
-
8/22/2019 MIT2 852S10 Control
20/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Context: Theplanning/schedulinghierarchyLongterm: factorydesign,capitalexpansion,etc.Mediumterm:demandplanning,staffing,etc.Shortterm:
responsetoshorttermeventspartreleaseanddispatchIn
this
problem,
we
deal
with
the
response
to
short
term
events.
Thefactoryandthedemandaregiventous;wemustcalculateshorttermproductionrates;theseratesarethetargetsthatreleaseanddispatchmustachieve.
Copyright c 202007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
21/108
Dynamic
ProgrammingContinuousTime,MixedState
StochasticExamplex1
x2
d1
d2
u (t)1
u (t)2 Type 2
Type 1
r, p
Perfectlyflexiblemachine,twoparttypes.i timeunitsrequiredtomakeTypeiparts,i= 1,2.
Exponentialfailuresandrepairswithratespandr.Constantdemandratesd1,d2.Instantaneousproductionratesui(t), i= 1,2control
variables.Downstreamsurplusesxi(t).Copyright c 212007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
22/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Objective: Minimizethedifferencebetweencumulativeproductionandcumulativedemand.Thesurplussatisfiesxi(t) =Pi(t)Di(t)
t
CumulativeProduction
and Demand production P (t)
surplus x (t)
i
i
idemand D (t) = d ti
Copyright c2007StanleyB.Gershwin. 22
-
8/22/2019 MIT2 852S10 Control
23/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Feasibility:Fortheproblemtobefeasible,itmustbepossibletomake
approximatelydiT TypeipartsinalongtimeperiodoflengthT, i= 1,2.(Whyapproximately?)
ThetimerequiredtomakediT partsisidiT.Duringthisperiod,thetotaluptimeofthemachineie,the
timeavailableforproductionisapproximatelyr/(r+p)T.Therefore,wemusthave1d1T +2d2Tr/(r+p)T,or
2
ridi
r+pi=1
Copyright2007StanleyB.Gershwin.c 23
-
8/22/2019 MIT2 852S10 Control
24/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Ifthisconditionisnotsatisfied,thedemandcannotbemet.Whatwillhappentothesurplus?Thefeasibilityconditionisalsowritten
2di r
i=1 i r+pwherei =1/i.Iftherewereonlyoneparttype,thiswouldber
dr+p
Lookfamiliar?Copyright2007StanleyB.Gershwin.c 24
-
8/22/2019 MIT2 852S10 Control
25/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Thesurplussatisfiesxi(t) =Pi(t)Di(t)
where
tPi(t) = u
i(s)ds; D
i(t) =d
it
0Therefore
dxi(t)=ui(t)di
dt
Copyright2007StanleyB.Gershwin.c 25
-
8/22/2019 MIT2 852S10 Control
26/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Todefinetheobjectivemoreprecisely,lettherebeafunctiong(x1, x2)suchthatgisconvexg(0,0)=0 lim g(x1, x2) =; lim g(x1, x2) =.
x1 x1 lim g(x1, x2) =; lim g(x1, x2) =.x2 x2
Copyright2007StanleyB.Gershwin.c 26
-
8/22/2019 MIT2 852S10 Control
27/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Examples:g(x1, x2) =A1x12 +A2x22g(x1, x2) =A1|x1|+A2|x2|g(x1, x2) =g1(x1) +g2(x2)where
+ gi(xi) =g(i+)xi +g(i)xi , xi+ =max(xi,0),xi =min(xi,0),
g(i+) >0, g(i) >0.
Copyright c 272007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
28/108
DynamicProgramming
Objective:
ContinuousTime,MixedStateStochasticExample
Tmin
E g(x1(t), x2(t))dt
0
x1
g(x ,x )1 2
x2
Copyright2007StanleyB.Gershwin.c 28
-
8/22/2019 MIT2 852S10 Control
29/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Constraints:u1(t)0; u2(t)0
Short-termcapacity:Ifthemachineisdownattimet,
u1(t) =u2(t) = 0
Copyright2007StanleyB.Gershwin.c 29
-
8/22/2019 MIT2 852S10 Control
30/108
ProgrammingContinuousTime,MixedState
StochasticExampleAssumethemachineisupforashortperiod[t,t+t].Lett
Dynamic
besmallenoughsothatui isconstant;thatisui(s) =ui(t), s[t,t+t]
Themachinemakesui(t)tpartsoftypei.ThetimerequiredtomakethatnumberofTypeipartsisiui(t)t.Therefore
iui(t)tt 1/i
oriui(t)1
1/1
2
u2
0
iCopyright2007StanleyB.Gershwin.c 30
-
8/22/2019 MIT2 852S10 Control
31/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamic
Machinestatedynamics: Define(t)tobetherepairstateofthemachineattimet.(t) = 1meansthemachineisup;(t) = 0meansthemachineisdown.
prob((t+t) = 0|(t)=1)=pt+o(t)prob((t+t) =1|(t)=0)=rt+o(t)
Theconstraintsmaybewritteniui(t)(t); ui(t)0
iCopyright2007StanleyB.Gershwin.c 31
-
8/22/2019 MIT2 852S10 Control
32/108
ProgrammingContinuousTime,MixedState
StochasticExampleDynamicDynamicprogrammingproblemformulation: T
minE g(x1(t), x2(t))dtsubjectto: 0
dxi(t)=ui(t)di
dtprob((t+t) = 0|(t)=1)=pt+o(t)prob((t+t) =1|(t)=0)=rt+o(t)
iui(t)(t); ui(t)0i
x(0), (0)specified
Copyright2007StanleyB.Gershwin.c 32
-
8/22/2019 MIT2 852S10 Control
33/108
Dynamic ElementsofaDPProblemProgrammingstate: xalltheinformationthatisavailabletodeterminethe
futureevolutionofthesystem.control: utheactionstakenbythedecision-maker.objectivefunction: J thequantitythatmustbeminimized;dynamics: theevolutionofthestateasafunctionofthecontrol
variablesandrandomevents.constraints: thelimitationsonthesetofallowablecontrolsinitialconditions: thevaluesofthestatevariablesatthestart
ofthetimeintervaloverwhichtheproblemisdescribed.Therearealsosometimesterminalconditions suchasinthenetworkexample.
Copyright c 332007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
34/108
Dynamic ElementsofaDPSolutionProgrammingcontrolpolicy: u(x(t), t). Astationary ortime-invariant policyisoftheformu(x(t)).
valuefunction: (alsocalledthecost-to-go function)thevalueJ(x,t)oftheobjectivefunctionwhentheoptimalcontrolpolicyisappliedstartingattimet,whentheinitialstateisx(t) =x.
Copyright c 342007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
35/108
Bellmans Continuousx,tEquation Deterministic
T
Problem: min g(x(t), u(t))dt+F(x(T))u(t),0tT 0
suchthatdx(t)
=f(x(t), u(t), t)dt
x(0)specified
h(x(t), u(t))0
xRn, uRm, fRn, hRk,andgandF arescalars.Data: T,x(0),andthefunctionsf,g,h,andF.Copyright c 352007StanleyB.Gershwin.
-
8/22/2019 MIT2 852S10 Control
36/108
Bellmans Continuousx,tEquation DeterministicThecost-to-gofunctionis
TJ(x,t)=min g(x(s), u(s))ds+F(x(T))
t
T
J(x(0),0)=min g(x(s), u(s))ds+F(x(T))0 t1 T
= min g(x(t), u(t))dt+ g(x(t), u(t))dt+F(x(T)) .u(t), 0 t1
0tT
Copyright2007StanleyB.Gershwin.c 36
C i
-
8/22/2019 MIT2 852S10 Control
37/108
Bellmans Continuousx,tEquation Deterministic
t1 T= min g(x(t), u(t))dt+ min g(x(t), u(t))dt+F(x(T))
u(t),
0 u(t), t1
0tt1 t1tT t1 = min g(x(t), u(t))dt+J(x(t1), t1) .
u(t), 00tt1
Copyright2007StanleyB.Gershwin.c 37
C ti
-
8/22/2019 MIT2 852S10 Control
38/108
Bellmans Continuousx,tEquation Deterministic
where T
J(x(t1), t1)= min g(x(t), u(t))dt+F(x(T))u(t),t1tT t1
suchthatdx(t)
=f(x(t), u(t), t)dt
x(t1)specifiedh(x(t), u(t))0
Copyright2007StanleyB.Gershwin.c 38
C ti t
-
8/22/2019 MIT2 852S10 Control
39/108
Bellmans Continuousx,tEquation Deterministic
Breakup[t1, T]into[t1, t1 +t][t1 +t,T] :t1+tJ(x(t1), t1)=min g(x(t), u(t))dt
u(t1) t1+J(x(t
1 +t), t
1 +t)}
wheretissmallenoughsothatwecanapproximatex(t)andu(t)withconstantx(t1)andu(t1),duringtheinterval.Then,approximately,J(x(t1), t1)=min g(x(t1), u(t1))t+J(x(t1 +t), t1 +t)
u(t1)
Copyright2007StanleyB.Gershwin.c 39
C ti t
-
8/22/2019 MIT2 852S10 Control
40/108
Bellmans Continuousx,tEquation Deterministic
Or,J(x(t1), t1)=min g(x(t1), u(t1))t+J(x(t1), t1)+
u(t1)J J
(x(t1), t1)(x(t1 +t)x(t1)) + (x(t1), t1)tx tNotethat
dxx(t1 +t) =x(t1) + t=x(t1) +f(x(t1), u(t1), t1)tdt
Copyright2007StanleyB.Gershwin.c 40
Continuous t
-
8/22/2019 MIT2 852S10 Control
41/108
Bellmans Continuousx,tEquation Deterministic
ThereforeJ(x,t1) =J(x,t1)
J J+min g(x,u)t+ (x,t1)f(x,u,t1)t+ (x,t1)t
u x twherex=x(t1);u=u(t1) =u(x(t1), t1).Then(droppingthetsubscript)
J J (x,t)=min g(x,u) + (x,t)f(x,u,t)t u x
Copyright2007StanleyB.Gershwin.c 41
Continuous x t
-
8/22/2019 MIT2 852S10 Control
42/108
Bellmans Continuousx,tEquation DeterministicThisistheBellmanequation. Itisthecounterpartoftherecursionequationforthenetworkexample.IfwehadaguessofJ(x,t)(forallxandt)wecouldconfirmitbyperformingtheminimization.
IfweknewJ(x,t)forallxandt,wecoulddetermineubyperformingtheminimization.Ucouldthenbewritten
Ju=U x, , t .
xThiswouldbeafeedbacklaw.
TheBellmanequationisusuallyimpossibletosolveanalyticallyornumerically.Therearesomeimportantspecialcasesthatcanbesolvedanalytically.Copyright2007StanleyB.Gershwin.c 42
Continuous x t
-
8/22/2019 MIT2 852S10 Control
43/108
Bellmans Continuousx,tEquation ExampleBang-BangControl
min |x|dt0
subjecttodx
=udtx(0)specified1u1
Copyright2007StanleyB.Gershwin.c 43
B Continuous x t
-
8/22/2019 MIT2 852S10 Control
44/108
Bellmans Continuousx,tEquation ExampleTheBellmanequationis
J J (x,t)= min |x|+ (x,t)u .t xu,
1u1J(x,t) =J(x)isasolutionbecausethetimehorizonisinfiniteandtdoesnotappearexplicitlyintheproblemdata(ie,g(x) =|x|isnotafunctionoft.Therefore
dJ0= min |x|+ (x)u .
dxu,1u1
J(0)=0becauseifx(0)=0wecanchooseu(t) =0forallt.Thenx(t) = 0foralltandtheintegralis0.Thereisnopossiblechoiceofu(t)thatwillmaketheintegrallessthan0,sothisistheminimum.Copyright2007StanleyB.Gershwin.c 44
B ll Continuous x t
-
8/22/2019 MIT2 852S10 Control
45/108
Bellmans Continuousx,tEquation Example
Theminimumisachievedwhen
u=
Why?
1 if dJ(x)>0dx
1 if dJ(x)
-
8/22/2019 MIT2 852S10 Control
46/108
Bellmans Continuousx,tEquation Example
ConsiderthesetofxwheredJ/dx(x)0andu=1,dJ
(x) =|x|dx
Copyright2007StanleyB.Gershwin.c 46
B ll Continuous x t
-
8/22/2019 MIT2 852S10 Control
47/108
Bellmans Continuousx,tEquation Example
Tocompletethesolution,wemustdeterminewheredJ/dx>0,0forallx 0because|x|>0sotheintegralof|x(t)|mustbepositive.=SinceJ(x)> J(0)forallx 0,wemusthave=
dJ(x)0dx
Copyright2007StanleyB.Gershwin.c 47
B ll Continuous x, t
-
8/22/2019 MIT2 852S10 Control
48/108
Bellmans Continuousx,tEquation Example
ThereforedJ
(x)>=xdxso
1J = x22
and
1 if x 0
Copyright2007StanleyB.Gershwin.c 48
Continuous x, t,Discrete Bellmans
-
8/22/2019 MIT2 852S10 Control
49/108
Continuousx,t,DiscreteStochastic
T
BellmansEquationJ(x(0), (0),0)=minE g(x(t), u(t))dt+F(x(T))
u 0
suchthatdx(t)
=f(x,,u,t)dt
prob [(t+t) = = =ijtforalli,j,ii|(t) j] =jx(0), (0)specified
h(x(t), (t), u(t))0Copyright2007StanleyB.Gershwin.c 49
Bellmans Continuous x, t,Discrete
-
8/22/2019 MIT2 852S10 Control
50/108
BellmansEquation
Continuousx,t,DiscreteStochastic
GettingtheBellmanequationinthiscaseismorecomplicatedbecausechangesbylargeamountswhenitchanges.LetH()besomefunctionof.Weneedtocalculate
EH((t
+
t))
=
E
{H((t
+
t))
|(t)}
=
H(j)prob {(t+t) =j |(t)}
j
Copyright2007StanleyB.Gershwin.c 50
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
51/108
Bellman sEquation
Co t uous , , sc ete
Stochastic
= H(j)j(t)t+H((t))1 j(t)t+o(t)j j=(t)=(t)
= H(j)j(t)t+H((t)) 1 +(t)(t)t +o(t)j=(t)
E{H((t+t))|(t)}=H((t))+
H(j)j(t)t+o(t)
jWeusethisinthederivationoftheBellmanequation.
Copyright2007StanleyB.Gershwin.c 51
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
52/108
Bellman sEquation
, ,
Stochastic
TJ(x(t), (t), t)= min E g(x(s), u(s))ds+F(x(T))
u(s), tts
-
8/22/2019 MIT2 852S10 Control
53/108
Bellman sEquation
t+t= min E g(x(s), u(s))ds
, ,
Stochastic
u(s),
t0st+t
T
+
min
E g(x(s), u(s))ds+
F
(x(T
))
u(s), t+t
t+tsT
Copyright2007StanleyB.Gershwin.c 53
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
54/108
Bellman sEquation Stochastic
= minu(s), E
t+tt g(x(s),u(s))ds
tst+t
+J(x(t+t), (t+t), t+t)
Next,
we
expand
the
second
term
in
a
Taylor
series
about
x(t).
Weleave(t+t)alone,fornow.
Copyright2007StanleyB.Gershwin.c 54
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
55/108
Bellman sEquation Stochastic
J(x(t), (t), t) =minE g(x(t), u(t))t+J(x(t), (t+t), t) +u(t)
J J(x(t), (t+t), t)x(t) + (x(t), (t+t), t)t +o(t)
x twhere
x(t) =x(t+t)x(t) =f(x(t), (t), u(t), t)t+o(t)
Copyright c 552007StanleyB.Gershwin.
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
56/108
Bellman sEquation Stochastic
UsingtheexpansionofEH((t+t)),J(x(t), (t), t)=min g(x(t), u(t))t
u(t)
+J(x(t), (t), t) + J(x(t),j,t)j(t)tj
J J
+ (x(t), (t), t)x(t) + (x(t), (t), t)t +o(t)x t
Wecancleanupnotationbyreplacingx(t)withx,(t)with,andu(t)withu.Copyright2007StanleyB.Gershwin.c 56
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
57/108
Bellman sEquation Stochastic
J(x,,t)=minu
g(x,u)t+J(x,,t)+
jJ(x,j,t)jt
J
J
+ (x,,t)x+ (x,,t)t +o(t)x t WecansubtractJ(x,,t)frombothsidesandusetheexpressionforxtoget...
Copyright2007StanleyB.Gershwin.c 57
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
58/108
Bellman sEquation Stochastic
0=min g(x,u)t+ J(x,j,t)jtu j
J J +
x(x,
,
t)f(x,
,
u,
t)t
+
t(x,,t)t+o(t)
or,
Copyright2007StanleyB.Gershwin.c 58
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
59/108
Bellman sEquation Stochastic
J
(x,
,
t) =
J(x,
j,
t)j+
t j
Jmin
g(x,
u) +
(x,
,
t)f(x,
,
u,
t)
u x
Badnews: usuallyimpossibletosolve;Goodnews: insight.
Copyright c 592007StanleyB.Gershwin.
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
60/108
Bellman sEquation Stochastic
Anapproximation:whenT islargeandf isnotafunctionoft,typicaltrajectorieslooklikethis:
x
t
Copyright2007StanleyB.Gershwin.c 60
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
61/108
e a s
Equation StochasticThatis,inthelongrun,xapproachesasteady-stateprobabilitydistribution.
Let
J
be
the
expected
value
of
g(x,
u),
where
u
is
theoptimalcontrol.Supposewestartedtheproblemwithx(0)arandomvariablewhoseprobabilitydistributionisthesteady-statedistribution.Then,forlargeT,
T
EJ =minuE 0 g(x(t), u(t))dt+F(x(T))JT
Copyright2007StanleyB.Gershwin.c 61
Bellmans Continuousx,t,Discrete
-
8/22/2019 MIT2 852S10 Control
62/108
Equation StochasticForx(0)and(0)specified
J(x(0), (0),0)JT +W(x(0), (0))or,moregenerally,forx(t) =xand(t) =specified,
J(x,,t)J(Tt) +W(x,)
Copyright2007StanleyB.Gershwin.c 62
FlexibleManufacturing
-
8/22/2019 MIT2 852S10 Control
63/108
ManufacturingSystemControlSinglemachine,multipleparttypes.x,u,dareN-dimensionalvectors.
TminE g(x(t))dt
subjectto: 0dxi(t)
=ui(t)di, i= 1,...,Ndt
prob((t+t) = 0|(t)=1)=pt+o(t)prob((t+t) =1|(t)=0)=rt+o(t)
iui(t)(t); ui(t)0i
x(0), (0)specified
Copyright2007StanleyB.Gershwin.c 63
FlexibleManufacturing
-
8/22/2019 MIT2 852S10 Control
64/108
ManufacturingSystemControl
Define() ={u| iiui}.Then,for= 0,1,J (x,,t) = J(x,j,t)j+t
j
Jmin g(x) + (x,,t)(ud)u() x
Copyright2007StanleyB.Gershwin.c 64
FlexibleManufacturing
-
8/22/2019 MIT2 852S10 Control
65/108
ManufacturingSystemControl
ApproximatingJ withJ(Tt) +W(x,)gives:J = (J(Tt) +W(x,j))j+
jW
min g(x) + (x,,t)(ud)u() xRecallthat
j = 0...j
Copyright2007StanleyB.Gershwin.c 65
FlexibleManufacturing
-
8/22/2019 MIT2 852S10 Control
66/108
ManufacturingSystemControl
soJ = W(x,j)j+
jW
min g(x) + (x,,t)(ud)u() xfor= 0,1
Copyright2007StanleyB.Gershwin.c 66
FlexibleManufacturing
-
8/22/2019 MIT2 852S10 Control
67/108
ManufacturingSystemControl
Thisisactuallytwoequations,onefor= 0,onefor= 1.WJ =g(x) +W(x,1)rW(x,0)r (x,0)d,x
for= 0,WJ =g(x) +W(x,0)pW(x,1)p+ min (x,1)(ud)
u(1) xfor= 1.
Copyright2007StanleyB.Gershwin.c 67
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
68/108
ManufacturingSystemControl Technically,notflexible!
Now,xanduarescalars,and(1)=[0,1/]=[0, ]
dWJ =g(x) +W(x,1)rW(x,0)r (x,0)d,
dxfor= 0,dW
J =g(x) +W(x,0)pW(x,1)p+ min (x,1)(ud)0u dxfor= 1.
Copyright2007StanleyB.Gershwin.c 68
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
69/108
ManufacturingSystemControl
Seebook,Sections2.6.2and9.3;seeProbabilityslides#91120.
When= 0, u= 0.When= 1,ifdW 0, u= 0.dx
Copyright c 692007StanleyB.Gershwin.
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
70/108
ManufacturingSystemControl
W(x,)hasbeenshowntobeconvexinx. IftheminimumofW
(x,
1)
occurs
at
x
=
Z
and
W
(x,
1)
is
differentiable
for
all
x,
thendW 0x > Zdx
Therefore,ifx < Z, u=,ifx=Z, uunspecified,ifx > Z, u= 0.Copyright c 702007StanleyB.Gershwin.
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
71/108
Manufacturing
SystemControlSurplus,orinventory/backlog:Productionpolicy: ChooseZ(thehedgingpoint)Then,if= 1,
ifx < Z, u=,ifx=Z, u=d,ifx > Z, u=0;
if= 0, u
= 0.
HowdowechooseZ?
dx(t)=u(t)d
dtCumulative
Production and Demand production
d t + Z
hedging point Z
surplus x(t)
demand dt
t
Copyright2007StanleyB.Gershwin.c 71
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
72/108
g
SystemControl DeterminationofZ
ZJ =Eg(x) =g(Z)P(Z,1)+ g(x) [f(x,0)+f(x,1)]dx
inwhichP andf formthesteady-stateprobabilitydistributionofx.WechooseZ tominimizeJ. P andf aregivenby
f(x,0)=Aebx
f(x,1)
=
A
debx
d
P(Z,1)=ApdebZ
Copyright2007StanleyB.Gershwin.c 72
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
73/108
g
SystemControl DeterminationofZwhere
r p
b=
d d
andAischosensothat
Z[f(x,0)+f(x,1)]dx+P(Z,1)=1
Aftersomemanipulation,A= bp(d) ebZ
db(
d) +
pand
db(d)P(Z,1)=
db(d) +pCopyright2007StanleyB.Gershwin.c 73
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
74/108
g
SystemControl DeterminationofZSinceg(x) =g+x+ +gx ,
ifZ0,then ZJ =gZP(Z,1) gx[f(x,0)+f(x,1)]dx;
ifZ >0,
0
J =g+ZP(Z,1) gx[f(x,0)+f(x,1)]dx
Z+ g+x[f(x,0)+f(x,1)]dx.
0
Copyright2007StanleyB.Gershwin.c 74
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
75/108
g
SystemControl DeterminationofZTominimizeJ:
ln Kb(1+g)
ifg+Kb(g+ +g)
-
8/22/2019 MIT2 852S10 Control
76/108
g
SystemControl DeterminationofZThatis,wechooseZsuchthat
ebZ =min 1, Kb g+ +gg+or
bZ 1 g+e =max 1,Kb g+ +g
Copyright2007StanleyB.Gershwin.c 76
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
77/108
SystemControl DeterminationofZ
0prob(x0) = (f(x,0)+f(x,1))dx
0d=A 1 + ebxdx
d d 1 =A 1 + =A
d b b(d)bp(d) bZ = e
db(d) +p b(d)p bZ= e
db(d) +pCopyright2007StanleyB.Gershwin.c 77
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
78/108
SystemControl DeterminationofZOr,
prob(x0)=
pmax 1,
1 g+db(d) +p Kb g+ +g
Itcanbeshownthatp
Kb= p+bd(d)Therefore
prob(x0)=Kbmax 1, 1 g+Kb g+ +gp g+
=max ,p+bd(d) g+ +g
Copyright2007StanleyB.Gershwin.c 78
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
79/108
SystemControl DeterminationofZThatis,if
p>
g+,thenZ = 0andp+bd(d) g+ +g
prob(x0)= p ;p+bd(d)
if p < g+ ,thenZ >0andp+bd(d) g+ +g
prob(x0)= g+ .g+ +gThislooksalotlikethesolutionofthenewsboyproblem.Copyright2007StanleyB.Gershwin.c 79
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
80/108
Zvs.dSystemControlBasevalues:g+ = 1,g =10d=.7,= 1.,r=.09,p=.01.
10090807060
Z 50403020100
d0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Copyright2007StanleyB.Gershwin.c 80
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
81/108
SystemControl Zvs.g+Basevalues:g+ = 1,g =10d=.7,= 1.,r=.09,p=.01.
70605040
0 0.5 1 1.5 2 2.5 3g+
Z3020
100
3.5
Copyright2007StanleyB.Gershwin.c 81
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
82/108
SystemControl Zvs.gBasevalues:g+ = 1,g =10d=.7,= 1.,r=.09,p=.01.
14
12
10
8
Z
6
4
2
00 1 2 3 4 5 6 7 8 9 10 11
g
Copyright2007StanleyB.Gershwin.c 82
Flexible Single-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
83/108
SystemControl Zvs.pBasevalues:g+ = 1,g =10d=.7,= 1.,r=.09,p=.01.
140012001000800
Z600400
2000
0.005 0.01 0.015 0.02 0.025 0.03 0.035p0 0.04
Copyright2007StanleyB.Gershwin.c 83
Flexible Two-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
84/108
SystemControlx1
x2
d1
d2
u (t)1
u (t)2 Type 2
Type 1
r, p
u11/1
1/
2
u2
0
Capacityset(1)whenmachineisup.Copyright2007StanleyB.Gershwin.c 84
Flexible Two-part-typecaseManufacturing
-
8/22/2019 MIT2 852S10 Control
85/108
SystemControl
Wemustfindu(x,)tosatisfyW
min (x,,t) u
u() xPartialsolutionofLP:IfW/x1 >0andW/x2 >0,u1 =u2 = 0.IfW/x1
-
8/22/2019 MIT2 852S10 Control
86/108
1
0
2
SystemControlCase:ExactsolutionifZ = (Z1, Z2) = 0x
2
x1
1 2u = u = 0
dx
dt
2
2u = 0
u = 01
1u =
1
2
u =
u1
1/
1/2
u2
0
u1
1/1
1/2
u2
1/1
1/
u2
0 u1
Copyright2007StanleyB.Gershwin.c 86
Flexible Two-part-typecaseManufacturing
S C l
-
8/22/2019 MIT2 852S10 Control
87/108
1
2
0
SystemControlCase:ApproximatesolutionifZ >0x
2
1 2u = u = 0
dx
dt
2
2u = 0
u = 01
1
u =1
2
u =
u1
1/
1/2
u2
0
1/1
1/
u2
0
u1
1/1
1/2
u2
x1
u1
Copyright2007StanleyB.Gershwin.c 87
Flexible Two-part-typecaseS t C t l
Manufacturing
-
8/22/2019 MIT2 852S10 Control
88/108
45
SystemControlTwoparts,multiplemachineswithoutbuffers:
e12
4
61e
e
34e
23e
3
12
x2
Z
6x
1
556e
6
1 2
3
4
5
u2
e34
12e
u1
Ie23
e45
e56
d
61e
Copyright2007StanleyB.Gershwin.c 88
Flexible Two-part-typecaseManufacturingS t C t l
-
8/22/2019 MIT2 852S10 Control
89/108
SystemControlProposedapproximatesolutionformultiple-part,
singlemachinesystem:Rankordertheparttypes,andbringthemtotheirhedgingpointsinthatorder.
Copyright2007StanleyB.Gershwin.c 89
Flexible Single-part-typecaseS t C t l
ManufacturingS rpl s and tokens
-
8/22/2019 MIT2 852S10 Control
90/108
SystemControl SurplusandtokensOperatingMachineMaccordingtothehedgingpointpolicyisequivalenttooperatingthisassemblysystem
according
to
a
finite
B
bufferpolicy.D
M
S
FG
Copyright2007StanleyB.Gershwin.c 90
Flexible Single-part-typecaseS t C t l
ManufacturingSurplus and tokens
-
8/22/2019 MIT2 852S10 Control
91/108
SystemControl SurplusandtokensDisademandgenerator.
Whenever a demand arrives, DsendsatokentoB.
S isasynchronizationmachine. S is perfectly reliable and in
finitelyfast.
M
D
S
FG
B
F Gisafinitefinishedgoodsbuffer.
Bis
an
infinite
backlog
buffer.
Copyright c 912007StanleyB.Gershwin.
Flexible Single-part-typecaseManufacturingSystem Control Material/token policies
-
8/22/2019 MIT2 852S10 Control
92/108
SystemControl Material/tokenpoliciesOperator
AnoperationcannottakeMachineplace
unless
there
is
a
Part Part tokenavailable.Operation
Consumable Waste
TokensauthorizeToken Token
production.Thesepoliciescanoftenbeimplementedeither withfinite
bufferspace,orafinitenumberoftokens.Mixturesarealsopossible.
Bufferspacecouldbeshelfspace,orfloorspaceindicatedwithpaintortape.
Copyright c 922007StanleyB.Gershwin.
Multi-stage Proposedpolicysystems
-
8/22/2019 MIT2 852S10 Control
93/108
systems
TocontrolM B M B M
1 1 2 2 3
addaninformationflowsystem:B
1M B M
2 2 3M
1
S S2 3
D
S1
BB1
SB2
BB2
SB3
BB3
SB1
Copyright2007StanleyB.Gershwin.c 93
Multi-stage Proposedpolicysystems
-
8/22/2019 MIT2 852S10 Control
94/108
systemsB
1M B M
2 2 3M
1
S S2 3
D
S1
BB1
SB2
BB2
SB3
BB3
SB1
Bi arematerial buffersandarefinite.SBi aresurplus buffersandarefinite.BBi arebacklog buffersandareinfinite.ThesizesofBi andSBi arecontrolparameters.Problem: predictingtheperformanceofthissystem.Copyright c 942007StanleyB.Gershwin.
Multi-stagesystems
ThreeViewsofScheduling
-
8/22/2019 MIT2 852S10 Control
95/108
systemsThreekindsofschedulingpolicies,whicharesometimes
exactly
the
same.
Surplus-based: makedecisionsbasedonhow
muchproductionexceeddemand.Time-based: makedecisionsbasedonhowearlyor
lateaproductis.Token-based: makedecisionsbasedonpresence
orabsenceoftokens.Copyright c 952007StanleyB.Gershwin.
Multi-stage ObjectiveofSchedulingsystems
Surplusand
time
-
8/22/2019 MIT2 852S10 Control
96/108
Surplus and time
and Demand
earliness
production P(t)
demand D(t)
surplus/backlog x(t)
Objectiveistokeepcumulativeproductionclosetocumulativedemand.
CumulativeProduction
Surplus-basedpolicieslookatverticaldifferencesbetweenthegraphs.
Time-basedpolicieslookatthehorizontal
t
differences.Copyright2007StanleyB.Gershwin.c 96
Multi-stage Otherpoliciessystems
CONWIPkanban
and
hybrid
-
8/22/2019 MIT2 852S10 Control
97/108
CONWIP, kanban, and hybrid
CONWIP: finitepopulation,infinitebufferskanban: infinitepopulation,finitebuffershybrid: finitepopulation,finitebuffers
Copyright c 972007StanleyB.Gershwin.
Multi-stage Otherpoliciessystems
CONWIP,kanban,
and
hybrid
-
8/22/2019 MIT2 852S10 Control
98/108
CONWIP, kanban, and hybrid
CONWIPSupply Demand
Token flow
Demandislessthancapacity.How
does
the
number
of
tokens
affect
performance
(production
rate,inventory)?
Copyright c 982007StanleyB.Gershwin.
Multi-stage Otherpoliciessystems
CONWIP,kanban,
and
hybrid
-
8/22/2019 MIT2 852S10 Control
99/108
0.85
0.855
0.86
P
CONWIP, kanban, and hybrid
0.835
0.84
0.845
0.865
0.87
0.875
0 20 40 60 80 100 120
0
5
10
20
25
30
AverageBufferLevel
n1n2n3
15
0 20 40 60
Population Population
80 100 120
cCopyright2007StanleyB.Gershwin. 99
Multi-stage Otherpoliciessystems
Basestock
-
8/22/2019 MIT2 852S10 Control
100/108
Basestock
Demand
Copyright2007StanleyB.Gershwin.c 100
Multi-stage Otherpoliciessystems
FIFO
-
8/22/2019 MIT2 852S10 Control
101/108
First-In,FirstOut.Simpleconceptually,butyouhavetokeeptrackofarrivaltimes.Leavesoutmuchimportantinformation:duedate,valueofpart,currentsurplus/backlog
state,etc.
Copyright2007StanleyB.Gershwin.c 101
Multi-stage Otherpoliciessystems
EDD
-
8/22/2019 MIT2 852S10 Control
102/108
Earliestduedate.Easytoimplement.Doesnotconsiderworkremainingontheitem,value
oftheitem,etc..
Copyright2007StanleyB.Gershwin.c 102
Multi-stage Otherpoliciessystems
SRPT
-
8/22/2019 MIT2 852S10 Control
103/108
ShortestRemainingProcessingTimeWheneverthereisachoiceofparts,loadtheonewithleastremainingworkbeforeitisfinished.Variations: includewaitingtimewiththeworktime.
Useexpectedtimeifitisrandom.
Copyright2007StanleyB.Gershwin.c 103
Multi-stage Otherpoliciessystems
Criticalratio
-
8/22/2019 MIT2 852S10 Control
104/108
Widelyused,butmanyvariations.Oneversion:Processingtimeremaininguntilcompletion
DefineCR= Duedate- CurrenttimeChoosethejobwiththehighestratio(provideditispositive).Ifajobislate,theratiowillbenegative,orthedenominator
willbezero,andthatjobshouldbegivenhighestpriority.Ifthereismorethanonelatejob,schedulethelatejobsin
SRPTorder.
Copyright2007StanleyB.Gershwin.c 104
Multi-stage Otherpoliciessystems
LeastSlack
-
8/22/2019 MIT2 852S10 Control
105/108
Thispolicyconsidersapartsduedate.Defineslack =duedate- remainingworktimeWhenthereisachoice,selectthepartwiththeleast
slack.Variationsinvolvedifferentwaysofestimating
remainingtime.
Copyright c 1052007StanleyB.Gershwin.
Multi-stage Otherpoliciessystems
Drum-Buffer-Rope
-
8/22/2019 MIT2 852S10 Control
106/108
DuetoEliGoldratt.
Basedon
the
idea
that
every
system
has
a
bottleneck.
Drum: thecommonproductionratethatthesystemoperates
at,whichistherateofflowofthebottleneck.
Buffer:
DBRestablishes
a
CONWIP
policy
between
the
entranceofthesystemandthebottleneck.ThebufferistheCONWIPpopulation.
Rope: thelimitonthedifferenceinproductionbetweendifferentstagesinthesystem.
But:Whatifbottleneckisnotwell-defined?Copyright c 1062007StanleyB.Gershwin.
Conclusions
-
8/22/2019 MIT2 852S10 Control
107/108
Manypoliciesandapproaches.Nosimplestatementtellingwhichisbetter.Policiesarenotallwell-definedintheliteratureorinpractice.Myopinion:
Thisisbecausepoliciesarenotderived fromfirstprinciples.Instead,theyaretestedandcompared.Currently,wehavelittleintuitiontoguidepolicydevelopment
andchoice.
Copyright2007StanleyB.Gershwin.c 107
MIT OpenCourseWarehttp://ocw.mit.edu
2.852 Manufacturing Systems Analysis
Spring 2010
http://ocw.mit.edu/http://ocw.mit.edu/ -
8/22/2019 MIT2 852S10 Control
108/108
Spring 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
http://ocw.mit.edu/termshttp://ocw.mit.edu/terms