The Maximum Principle of Optimal Control:A History of Ingenious Idea and Missed Opportunities
Hans Josef Pesch 1, Michael Plail 2
1 University of Bayreuth, Germany2 Steinebach, Wörthsee, Germany
Optimization Day, University of Southern Australia,Adelaide, Australia,
January 29, 2011
Outline
• Carathéodory‘s Royal Road of the Calculus of Variations
• Hestenes‘ secret report and first formulation
• Bellman‘s and Isaacs‘ regrets
Hans Josef Pesch, Roland Bulirsch: The Maximum Principle, Bellman‘s Equation, and Carathéodory‘s WorkJ. of Optimization Theory and Applications, Vol. 80, No. 2, Feb. 1994
Hans Josef Pesch, Michael Plail:The Maximum Principle of Optimal Control: A History of Ingenious Idea and Missed OpportunitiesControl and Cybernetics, Vol. 38, No. 4A, 973-995, 2009.
• Pontryagin and his students: adoration and embitterment
Missed Opportunities to the Maximum Principle of Optimal Control
Carathéodory‘s Royal Road in the Calculus of Variations
Relationship between
and
allows the reduction of
to
Hilbert‘s Independence Theorem
Hamilton-Jacobi Equations
Problems of the Calculus of Variations
Problems of Finite Optimization
Carathéodory‘s Royal Road in the Calculus of Variations
subject to implicit differential equations
Search for -curves that extremize
Lagrangian problems: precursors of optimal control
for line elements of curves
with
DOF: n - pcontrols
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 1: Definition: extremal (minimal or maximal)
Different from today‘s terminology: weak extremum / minimum / maximum
Stage 2: Proof of necessary Legendre-Clebsch condition
or
in today‘s terminology for minimization
has a positive definit Hessian
for fixed
closer neighborhood
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 3: Caratheodory‘s equivalent variational problems
Let
then
independent of
Let
Then: integration along two curves yields
Thus
andand therefore any line element where
can be passed by one and only one minimal curve
Carathéodory‘s Royal Road in the Calculus of Variations
adding a null Lagrangian
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 4: Caratheodory‘s existence result for a special problem
with
for all and all with
then the solutions of are extremals of
If there exists
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 5: Caratheodory‘s sufficient condition
If there exists
for which there hold
and
for sufficiently small , then the solutions of
yield
Hence we have to determine the functions
such that
(as function of ) possesses a minimum for
with value
(Carathéodory, 1935)
That is the so-called Bellman EquationNo imbedding or extremal fields on Carathéodory‘s Royal Road
or
C‘s fundamental equations:
Carathéodory‘s Royal Road in the Calculus of Variations
Substituting the fundamental equations and replacing by yields
Hence we obtain the necessary condition of Weierstraß
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 6: Caratheodory‘s formulation of Weierstraß‘ condition
Similarly
Introducing the Lagrange function
the fundamental equations take the form
(Carathéodory: 1926)
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 7: Lagrangian variational problems
Exit to the Maximum Principle?
Introducing canonical variables
and solving these equation for yields
with
Defining the Hamiltonian in canonical coordinates
the Weierstraß necessary condition takes the form
and
Recall Caratheodory‘s Hamiltonian
Carathéodory‘s closed approach to optimal control (from 1935)
Today‘s Hamiltonian
degree of freedom: control
degree of freedom: control?
Exit to the Maximum Principle from C‘s Royal Road
call them controls
canonical equations
Exit to the Maximum Principle from C‘s Royal Road
With the maximizing Hamiltonian for
and the costate
we obtain as long as
By means of the Euler-Lagrange equation
and because of
Furthermore
Hence, must have a maximum with respect to along a curve
From here it is still a big step to
Missed Carathéodory the exit?
Exit to the Maximum Principle from C‘s Royal Road
1904
1932
Constantin Carathéodory (1873 - 1950)
• Born in Berlin to Greek parents, grew up in Brussels (father was the Ottoman ambassador) to Belgium • The Carathéodory family was well-respected in Constantinople (many important governmental positions)
• Formal schooling at a private school in Vanderstock (1881-83); travelling with is father to Berlin, Italian Riviera; grammar school in Brussels (1985); high school Athénée Royal d'Ixelles, graduation in 1891 • Twice winning of a prize as the best mathematics student in Belgium• Trelingual (Greek, French, German), later: English, Italian, Turkish, and the ancient languages
• École Militaire de Belgique (1891-95), École d'Application (1893-1896): military engineer
• War between Turkey and Greece (break out 1897); British colonial service: construction of the Assiut dam (until 1900); Studied mathematics: Jordan's Cours d'Analyse a.o.; Measurements of Cheops pyramid (published in 1901)
Constantin Carathéodory (1873 - 1950)
Constantin Carathéodory (1873 - 1950)
• Graduate studies at the University of Göttingen (1902-04) (supervision of Hermann Minkowski: dissertation in 1904 (Oct.) on Diskontinuierliche Lösungen der Variationsrechnung• In March 1905: venia legendi (Felix Klein)
• Various lecturing positions in Hannover, Breslau, Göttingen and Berlin (1909-20)• Prussian Academy of Sciences (1919, together with Albert Einstein)
• Plan for the creation of a new University in Greece (Ionian University) (1919, not realized due to the War in Asia Minor in 1922); the present day University of the Aegean claims to be the continuation• University of Smyrna (Izmir), invited by the Greek Prime Minister (1920); (major part in establishing the institution, ends in 1922 due to war• University on Athens (until 1924)• University of Munich (1924-38/50); Bavarian Academy of Sciences (1925)
• C. played a remarkable opposing role together with the Munich „Dreigestirn“ (triumvirate) (Perron, Tietze) within the Bavarian Academy of Science during the Nazi terror in Germany
Magnus Rudolph Hestenes (1906 – May 31, 1991)
Thus, has a maximum value
with respect to along
a minimizing curve .
Research Memorandum RM-100,
Rand Corporation, 1950
I became interested in control theory in 1948.
At that time I formulated the general control
problem of Bolza …, and observed the maximum
principle … is equivalent to the conditions of
Euler-Lagrange and Weierstrass …
It turns out that I had formulated what is now
known as the general optimal control problem.
The Maximum Principle (first formulation, controls, 1950)
Missed opportunity
Richard Ernest Bellman (Aug. 26, 1920 – March 19, 1984)
Rufus Philip Isaacs (1914 – 1981)
The Maximum Principle (Bellman‘s & Isaacs‘ Equation, 1951+)
Isaacs in 1973 about his Tenet of Transition of 1951
Once I felt that here was the heart of the subject …..
Later I felt that it … was a mere truism.
Thus in (my book) Differential Games
it is mentioned only by title. This I regret.
I had no idea, that Pontryagin‘s principle
and Bellman‘s maximal principle
(a special case of the tenet, appearing a little later
in the Rand seminars) would enjoy such
a widespread citation.
Missed opportunities
Lev Semenovich Pontryagin (Лев Семёнович Понтрягин) (Sept. 3, 1908 – May 3. 1988)
The Maximum Principle (1956)
This fact is a special case
of the following general principle
which we call maximum principle
Doklady Akademii Nauk SSSR,
Vol. 10, 1956
The Maximum Principle (1956)
Vladimir G. Boltyanski Revaz V. Gamkrelidze
proved the Maximum Principle
Boltyanski in 1991 about the Maximum Principle of 1956
By the way, the first statement of the maximum principle was given
by Gamkrelidze, who has established (generalizing the famous
Legendre Theorem) a sufficient condition for a sort of weak
optimality problem. Then, Pontryagin proposed to name
Gamkrelidze‘s condition Maximum Principle. … Finally, I understood
that the maximum principle is not a sufficient, but only a necessary
condition of optimality.
Pontryagin was the Chairman of our department at the Steklov
Mathematical Institute, and he could insist on his interests.
So, I had to use the title Pontryagin‘s Maximum Principle
in my paper. This is why all investigators in region of mathematics
and engineering know the main optimization criterium as the
Pontryagin‘s Maximum Principle.
Gamkrelidze in 2008 about Pontryagin
My life was a series of missed opportunities, but one opportunity, I have not missed, to have met Pontryagin.*
* at the Banach Center Conference on 50 Years of Optimal Control in Bedlewo, Poland, on September 15, 2008
Plail, M.: Die Entwicklung der optimalen Steuerungen. Vandenhoeck & Ruprecht, Göttingen, Germany, 1998
Carathéodory‘s words:
Constantin Carathéodory (Κωνσταντίνος Καραθεοδωρή)* Sept. 13, 1873 in Berlin; † Feb. 2, 1950, Munich
I will be glad if I have succeeded in impressing the idea that it is not only pleasant and entertainingto read at times the works of the old mathematicialauthors, but that this may occasionally be of usefor the actual advancement of science.
Besides this there is a great lesson we can derive from the facts which I have just referred to. We haveseen that even under conditions which seem mostfavorable very important results can be discardedfor a long time and whirled away from the main streamwhich is carrying the vessel science. …
If their ideas are too far in advance of their time, andif the general public is not prepared to accept them, these ideas may sleep for centuries on the shelvesof our libraries … awaiting the arrival of the prince charming who will take them home. (C.C. 1937)
Thank you for your attention!
Both papers and a third forthcoming onecan be downloaded from
www.ingmath.uni-bayreuth.de/
Email: [email protected]
Top Related