THOUGHTS ON SOMEOPTIMAL CONTROL

PROBLEMSIn memory of

U. HELMKE & R. KALMANPaul A. Fuhrmann

Ben-Gurion University of the Negev

Sde Boker, MARCH 18-25, 2017 – p. 1/45

Things you see from here,you don’t see from there.

Sde Boker, MARCH 18-25, 2017 – p. 2/45

WHY THINK ABOUTOPTIMAL CONTROL NOW?

• Better late than never.• Finding the right resolution in which to

approach system theory is our goal. If theresolution is too low, the theory may not bepractically applicable. On the other hand, ifthe resolution is too high, we may see the treesbut miss the forest. Abstract vs. concrete.

• Gain better understanding of the subject byclarifying the relations between differentaspects of control theory, e.g., robuststabilization, model reduction, optimalregulator and estimation problems. This hasthe potential of leading to a grand unificationof control theory.

Sde Boker, MARCH 18-25, 2017 – p. 3/45

CONT.• Open the possibility of extending methods to

other settings as: classes of infinitedimensional systems, special classes of systems(positive real, bounded real).

• Extend optimal control theory to deal withcomplexity, that is, to networks of systems,using local optimality results for the nodes aswell as the interconnection data.

Sde Boker, MARCH 18-25, 2017 – p. 4/45

BASIC IDEAS FORREACHING A TARGET

• Use coprime factorizations, functional models.• Shift/translation realizations.• Construct the reachability mapR : U −→ X .• Derive representation ofKerR.• Reduce the reachability map to

R : U/KerR −→ X .

• Assuming reachability, this is a moduleisomorphism.

• Invert R, using doubly coprime factorizations.• For a discrete time system, this leads to time

optimal controllers.

Sde Boker, MARCH 18-25, 2017 – p. 5/45

WHAT HAPPENS WHEN ALL SPACES HAVEHILBERT SPACE STRUCTURE?

DUE TO THE AVAILABILITY OFORTHOGONAL DECOMPOSITION,OPTIMIZATION PROBLEMS ARE GREATLYSIMPLIFIED.

Sde Boker, MARCH 18-25, 2017 – p. 6/45

FROM ALGEBRATO ANALYSISAlthough the technicalities of polynomial modelbased system theory for discrete time linearsystems over an arbitrary field are vastly differentfrom the Hardy space based theory for someclasses of continuous-time systems there are strongalgebraic similarities. These, with the help ofheavy analytic tools, can be used to extend thealgebraic approach to a wide variety of optimalcontrol and estimation problems for several classesof, not necessarily rational, analytic functions.

Some of the ideas and results presented owe muchto a cooperation with Raimund Ober in the early1990s and a long term one with Uwe Helmke.

Sde Boker, MARCH 18-25, 2017 – p. 7/45

SOME REFERENCESA. Beurling,“On two problems concerning lineartransformations in Hilbert space”, Acta Math., 81,(1949), 239-255.

P.D. Lax, ”Translation invariant subspaces” ActaMath., 101, (1959), 163-178.

L. Carleson, “Interpolation by bounded analyticfunctions and the corona problem”,Ann. of Math.,76, (1962), 547-559.

D. Sarason, “Generalized interpolation inH∞”,Trans. Amer. Math. Soc. 127, 179-203.

P.A. Fuhrmann, “On the corona problem and itsapplication to spectral problems in Hilbert space”,Trans. Amer. Math. Soc., 132, (1968), 55-67.

Sde Boker, MARCH 18-25, 2017 – p. 8/45

MORE REFERENCESP.A. Fuhrmann and R. Ober, “A functionalapproach to LQG balancing, model reduction androbust control”, Int. J. Contr. (1993a), 57, 627-741.

P.A. Fuhrmann and R. Ober, “On coprimefactorizations”, T. Ando volume, BirkhauserVerlag, 1993.

P.A. Fuhrmann, “A duality theory for robustcontrol and model reduction”, Lin. Alg. Appl.,(1994), vols. 203-204, 471-578.

Sde Boker, MARCH 18-25, 2017 – p. 9/45

SOME ANALOGIESAlgebra Analysis

L2(−∞,∞;Cm)

F((z−1))m L2(0,∞;Cm)⊕ L2(−∞, 0;Cm))

= F[z]m ⊕ z−1F[[z−1]]m = H2

− ⊕H2+, whereH2

± = FL2(0,∞

F[z] H∞±

F[z]m = XD ⊕DF[z]m H2+ = H(Mr)⊕MrH

2+

F[z]-module structure H∞± functional calculus

Submodules Shift/Translationinvariant subspaces

D(z)F[z]m S(z)H2+

D(z) nonsingular S(z) inner

Sde Boker, MARCH 18-25, 2017 – p. 10/45

Algebra Analysis

Polynomial Models Shifts/TranslationsHG : F[z]m −→ z−1

F[[z−1]]p HG : H2+ −→ H2

−

Matrix fractions DSS FactorizationsShift realization Translation realizationF[z]-homomorphisms Intertwining maps, CLTPolynomial Bezout equation Carleson Corona ThmF[z]-unimodular embedding H∞-unimodular embeddingInversion Inversion

Sde Boker, MARCH 18-25, 2017 – p. 11/45

REACHABILITY MAPAND OPEN LOOP CONTROL(A,B) ∈ F

n×n × Fn×m a reachable pair. Our aim is

to compute all open loop control sequences thatsteer the system from the origin to the stateξ ∈ F

n.Define the reachability mapR(A,B) : F[z]

m −→ Fn:

R(A,B)

∑si=0 uiz

i :=∑s

i=0AiBui, u(z) =

∑si=0 uiz

i ∈ F[

R(A,B)u := πzI−ABu, u(z) ∈ F[z]m

KerR(A,B) = DF[z]m

R := R(A,B)|XD ≃ F[z]m/KerR(A,B)

u∗(z) = R−1ξ

u(z) = u∗(z) +D(z)g(z)

Set of steering controllers is an affine space.Sde Boker, MARCH 18-25, 2017 – p. 12/45

DCF & INVERSION OF R(A,B) ∈ F

n×n × Fn×m a reachable pair and let

(zI − A)−1B = N(z)D(z)−1 be coprimefactorizations, withD(z) ∈ F[z]m×m, N(z) ∈ F[z]n×m.(

Θ(z) Ξ(z)

−B zI − A

)(

D(z) −Ξ(z)

N(z) Θ(z)

)

=

(

I 0

0 I

)

(

D(z) −Ξ(z)

N(z) Θ(z)

)(

Θ(z) Ξ(z)

−B zI − A

)

=

(

I 0

0 I

)

Ξ(z)(zI − A) = D(z)Ξ(z)

u(z) = R−1ξ = πDΞξ.

Sde Boker, MARCH 18-25, 2017 – p. 13/45

THE SHIFT REALIZATION

G(z) = V (z)T (z)−1U(z) +W (z) = D + C(zI −A)−1B

A = ST

Bξ = πTUξ,

Cf = (V T−1f)−1

D = G(∞).

Realization is reachable⇔ T (z) andU(z) leftcoprime.Realization is observable⇔ T (z) and V (z) rightcoprime.Hautus and Kalman criteria.

Sde Boker, MARCH 18-25, 2017 – p. 14/45

INVERSION FOR HIGH OR-DER SYSTEMS(T (z), U(z)) ∈ F[z]q×q × F[z]q×m a reachable pair,i.e., left coprime, and letT (z)−1U(z) = U(z)T (z)−1

be coprime factorizations, withT (z) ∈ F[z]m×m, U(z) ∈ F[z]q×m.(

Y (z) X(z)

−U(z) T (z)

)(

T (z) −X(z)

U(z) Y (z)

)

=

(

I 0

0 I

)

X(z)T (z) = T (z)X(z)

u(z) = R−1ξ = πTXξ.

Sde Boker, MARCH 18-25, 2017 – p. 15/45

PROGRAM OUTLINE• Impulse response/transfer functions asH∞-homomorphisms. The reducedreachability and observability maps. Hankeloperators.

• Strictly noncyclic (Close to rational) functions,Douglas-Shapiro-Shields factorizations.Hankels with ”large” kernels, ”small” image.Beurling-Lax representations in terms of innerfunctions.

• Model operators, intertwining maps and thecommutant lifting theorem. Inversion.Intertwining maps and reduced Hankeloperators.

Sde Boker, MARCH 18-25, 2017 – p. 16/45

OUTLINE (Contd.)• Shift/translation realizations.• Kernel and image of Hankel operators and

their Beurling-Lax representations.• Inner functions and their state space

representations. Homogeneous Riccati(Lyapunov) equation.

• Normalized coprime factorizations and theirunimodular embeddings. ¶

• Characteristic functions.• Optimal control.

Sde Boker, MARCH 18-25, 2017 – p. 17/45

OUTLINE (Contd.)• Optimal control and model reduction of some

networks of linear systems, from local toglobal, by interpolation.

• Weak controllability, rational (AAK)approximation and suboptimal control.

Sde Boker, MARCH 18-25, 2017 – p. 18/45

INNER FUNCTIONSSTATE SPACE FORMULASM+ ∈ H∞

+ & M ∗+ = M−1

+ .(A,B) is reachable andA is stable. HomogeneousRiccati equation.

XA + A∗X = −XBB∗X

M =

(

A B

−B∗X I

)

M−1 =

(

A+BB∗X B

B∗X I

)

Sde Boker, MARCH 18-25, 2017 – p. 19/45

STABLE SYSTEMSAny strictly proper, rational function G(s) has aminimal realization of the formG(s) = C(sI − A)−1B

(A,B) reachable,(C,A) observable.AssumingG+(s) ∈ H∞

+ , thenA is stable.

Hankel operatorG+(s) ∈ H∞

+

HG+: H2

− −→ H2+

HG+f = P+G+f

Abstract realization:X = H2

− ⊖ KerHG+= KerHG+

⊥ and defining thereachability map RI : H

2− −→ KerHG+

⊥ andobservability mapOI : KerHG+

⊥ −→ H2+ by

Sde Boker, MARCH 18-25, 2017 – p. 20/45

ABSTRACT REALIZATIONX = H2

− ⊖ KerHG+= KerHG+

⊥

RI : H2− −→ KerHG+

⊥

RIu = PKerHG+⊥u, u(s) ∈ H2

−

OI : KerHG+⊥ −→ H2

+

OIh = HG+h, h(s) ∈ KerHG+

⊥

HG+= OIRI

Sde Boker, MARCH 18-25, 2017 – p. 21/45

NORMALIZED COPRIME(DSS) FACTORIZATIONS

JS =

(

I 0

0 0

)

JS −NRCF, G ∈ H∞− , G = NSM

−1S

(

M ∗S N∗

S

)

(

I 0

0 0

)(

MS

NS

)

= M ∗SMS = I

JS −NLCF, G ∈ H∞− , G = M

−1S NS

(

MS NS

)

(

I 0

0 0

)(

M∗S

N∗S

)

= MSM∗S = I

Sde Boker, MARCH 18-25, 2017 – p. 22/45

DSS FACTORIZATIONG+(s) = C(sI −A)−1B, A stableG+(s) = H(s)D(s)−1 = D(s)−1H(s)

= (H(s)E(s)−1)(E(s)D(s)−1)

= Mr(s)N∗r (s) = Nl(s)

∗Ml(s) (DSS)

Ml(s) =

(

A B

C0 I

)

&Ml(s)−1 = Ml(s)

∗ ⇒

(

A− BC0 B

−C0 I

)

≃

(

A∗ −C∗0

B∗ I

)

KerHG+= H−(M

∗l ) = M ∗

l H2−

⊥ ⇒ C0 = −B∗X+

XA+ A∗X = −XBB∗XImHG+

= H+(Mr)

Sde Boker, MARCH 18-25, 2017 – p. 23/45

DOUBLY COPRIMEFACTORIZATIONS(

Vℓ(s) Uℓ(s)

−Nℓ(s) Mℓ(s)

)(

Mr(s) −Ur(s)

Nr(s) Vr(s)

)

=

(

I 0

0 I

)

(

A X−1+ C∗ B

CZ+X+ I 0

−B∗X+ 0 I

)

,

(

A −Z+C∗ Z+X+B

C I 0

−B∗Z−1+ 0 I

)

A∗X +XA = −XBB∗X

AZ + ZA∗ = −ZC∗CZ.

X+, Z+ > 0

Sde Boker, MARCH 18-25, 2017 – p. 24/45

INTERNALSTABILIZATION

G

K

-

y2 e2

y1e1 - -

-

The feedback configuration(G,K) is calledinternally stable if

I G

K I

−1

=

(I −GK)−1 −(I −GK)−1G

−K(I −GK)−1 (I −KG)−1

∈ H∞+ .

Sde Boker, MARCH 18-25, 2017 – p. 25/45

INTERNALSTABILIZATION

G(s) = M−1ℓ Nℓ = NrM

−1r

K(s) = V −1ℓ Uℓ = UrV

−1r

The following statements are equivalent:

(Vℓ(s)Mr(s) + Uℓ(s)Nr(s)−1 ∈ H∞

+

(Mℓ(s)Vr(s) +Nℓ(s)Ur(s)−1 ∈ H∞

+

Vℓ(s)Mr(s) + Uℓ(s)Nr(s) = I,

Mℓ(s)Vr(s) +Nℓ(s)Ur(s) = I.

Sde Boker, MARCH 18-25, 2017 – p. 26/45

DCF and YOULA-KUCERAPARAMETRIZATION(

Vℓ(s) Uℓ(s)

−Nℓ(s) Mℓ(s)

)(

Mr(s) −Ur(s)

Nr(s) Vr(s)

)

=

(

I 0

0 I

)

K = UV −1 = V−1U

K = (U0 −MrQ)(V0 +NrQ)−1

= (V 0 −QN l)−1(U 0 +QM l)

Sde Boker, MARCH 18-25, 2017 – p. 27/45

THE S-CONTROLLERAND S-CHARACTERISTICG+(s) = N∗

ℓ (s)Mℓ(s) = Mr(s)N∗r (s)

There exists a unique stabilizing controller

K = V −1ℓ Uℓ = UrV

−1r

for which RS, the S-characteristic ofG+, definedby,

RS(s) := Mℓ(s)U∗ℓ (s) = U ∗

r (s)Mr(s)

is in H∞+ and strictly proper.

KerHRS= M ∗

rH2−

ImHRS= H+(Mℓ)

Sde Boker, MARCH 18-25, 2017 – p. 28/45

HANKEL OPERATORSAND INTERTWINING MAPS

H+(Mℓ)

H−(M∗ℓ )

H−(M∗r )

H+(Mr)

W W−1Mℓ Mr

HG+

HRS

-

?

6

whereW : H+(Mℓ) −→ H+(Mr) andW−1 : H+(Mr) −→ H+(Mℓ) are given by

Wg = P+N∗ℓ g, g(s) ∈ H+(Mℓ)

W−1f = P+U∗r f, f(s) ∈ H+(Mr).

Sde Boker, MARCH 18-25, 2017 – p. 29/45

S-CHARACTERISTICSTATE SPACE REALIZATION

RS = ξS(G+) =

(

A Z+C∗

B∗X+ 0

)

Ξ−1A∗ + AΞ−1 = −Z+C∗CZ+

Θ−1A+A∗Θ−1 = −X+BB∗X+.

Ξ+ = Z−1+ , Θ+ = X−1

+

ξS(RS) =

(

A Θ+X+B

CZ+Ξ+ 0

)

=

(

A B

C 0

)

= G+(s).

Sde Boker, MARCH 18-25, 2017 – p. 30/45

DCFSTATE SPACE REALIZATION(

Mr(s) −Ur(s)

Nr(s) Vr(s)

)

=

−(A∗ +X+BB∗) −X+Z+C∗ X+Z+X+B

CX−1+ I 0

−B∗Z−1+ X−1

+ 0 I

=

−(A∗ + C∗CZ+) −C∗ X+B

CZ+ I 0

−B∗ 0 I

,

Sde Boker, MARCH 18-25, 2017 – p. 31/45

OPTIMAL STEERINGTO TARGET

x = Ax+ Bu, A STABLE

0 = limt→−∞ x(t)

x(0) = ξ ∈ Cn.

R−(A,B) : L

2(−∞, 0;Cm) −→ Cn

R−(A,B)u =

∫ 0

−∞ e−AτBu(τ)dτ,

R−,∗(A,B) : C

n −→ L2(−∞, 0;Cm)

R−,∗(A,B)ξ = B∗e−A∗τξ, τ ≤ 0,

KerR−(A,B) = u(t) ∈ L2(−∞, 0;Cm)|

∫ 0

−∞ e−AτBu(τ)dτ = 0

Sde Boker, MARCH 18-25, 2017 – p. 32/45

REACHABILITY GRAMIANOPTIMAL CONTROLLER

W = R−(A,B)R

−,∗(A,B) =

∫ 0

−∞ e−AτBB∗e−A∗τdτ,

AW +WA∗ = −BB∗.

u∗(t) = B∗e−A∗tW−1ξ = B∗W−1x∗(t)

x∗(t) = We−A∗tW−1ξ.

OPTIMAL CONTROLLER IS A STATE

FEEDBACK CONTROLLER

Sde Boker, MARCH 18-25, 2017 – p. 33/45

NO STABILITYASSUMPTION

G(s) =

(

A B

C 0

)

minimal realization.

Sde Boker, MARCH 18-25, 2017 – p. 34/45

NORMALIZED (LQ)COPRIME FACTORIZATION

JL =

(

I 0

0 I

)

JL −NRCF, G = NLM−1L

(

M ∗L N∗

L

)

(

I 0

0 I

)(

ML

NL

)

= I

JL −NLCF, G = M−1L NL

(

ML NL

)

(

I 0

0 I

)(

M∗L

N∗L

)

= I

Sde Boker, MARCH 18-25, 2017 – p. 35/45

NORMALIZEDCOPRIME FACTORIZATION

G = NLM−1L = M

−1L NL

(

V U

−NL ML

)(

ML −U

NL V

)

=

(

I 0

0 I

)

(

ML

NL

)

=

A− BB∗X B

−B∗X I

C 0

.

A∗X +XA+ C∗C −XBB∗X = 0

(

−NL ML

)

=

(

A− ZC∗C B ZC∗

−C 0 I

)

AZ + ZA∗ + BB∗ − ZC∗CZ = 0

Sde Boker, MARCH 18-25, 2017 – p. 36/45

THE LQG CONTROLLER

R∗L = M ∗

LUL +N∗LVL ∈ H∞

−

(

UL

VL

)

=

(

ML

NL

)

R∗ +

(

−N∗L

M∗L

)

R∗L = ULM

∗L + V LN

∗L ∈ H∞

−

(

V L −UL

)

= R∗(

−NL ML

)

+(

M ∗L N∗

L

)

Sde Boker, MARCH 18-25, 2017 – p. 37/45

L-CONTROLLERSTATE SPACE EQs(

UL

VL

)

=

A− BB∗X ZC∗

−B∗X 0

C I

(

V L −UL

)

=

(

A− ZC∗C B ZC∗

−B∗X I 0

)

K = ULV−1L =

(

A− BB∗X − ZC∗C ZC∗

−B∗X 0

)

Sde Boker, MARCH 18-25, 2017 – p. 38/45

THE L-CHARACTERISTIC

R∗L = ΦKS

∗K = S∗

IΦI

KerHR∗L= SKH

2+

⊥

ImHR∗L= S∗

IH2−

⊥

Sde Boker, MARCH 18-25, 2017 – p. 39/45

TWO MATRIXCOMPLETIONS

G = NLM−1L = M

−1L NL

(

−J1J2

)

=

(

−N∗L

M∗L

)

SI

(

K1 K2

)

= SK

(

M ∗ N∗)

(

V L UL

−NL ML

)

,

(

ML −UL

NL VL

)

UNIMODULAR

ΩK =

(

K1 K2

−NL ML

)

,ΩI =

(

ML −J1NL J2

)

INNER

Sde Boker, MARCH 18-25, 2017 – p. 40/45

THE KEY DIAGRAMSKH

2+

⊥

ΩKH2+

⊥ Ω∗IH

2−

⊥

S∗IH

2−

⊥

? ?

HHHHHHHHHHHHHHHj

-

-*

ZK Y ∗I

HR∗

H(−N

∗

M∗

)

( −N M )

H(M∗ N∗)H(

−N∗

M∗

)

ZKf = PΩKH2+⊥

UL

VL

f

YI

h1

h2

= PS∗

IH2

−⊥(M∗ N∗)

h1

h2

YK

g1

g2

= PSKH2+⊥

(

−N M

)

g1

g2

Sde Boker, MARCH 18-25, 2017 – p. 41/45

HANKEL TOINTERTWINING MAP

SKH2+

⊥

ΩKH2+

⊥ Ω∗IH

2−

⊥ ΩIH2+

⊥

S∗IH

2−

⊥ SIH2+

⊥

? ? ?

HHHHHHHHHHHHj

- -

- -

*

ZK Y ∗I

HR∗ SI

H

−N∗

M∗

(

−N M)

ΩI

H(M∗ N∗)H

−N∗

M∗

Sde Boker, MARCH 18-25, 2017 – p. 42/45

SKH2+

⊥

ΩKH2+

⊥ ΩIH2+

⊥

SIH2+

⊥

? ?

@@@@@

@@@@@

@@@@@@

@@R

-

-

PΩKH2+

⊥

(

UL

VL

)

×

PSIH2+

⊥

(

K1 K2

)

×

PSIH2+

⊥ΦI×

−PΩIH2+

⊥

(

0

I

)

×

PSIH2+

⊥ΦI

PΩIH2+

⊥

(

−N M

0 0

)

×

PΩIH2+

⊥

(

I

0

)

×

Sde Boker, MARCH 18-25, 2017 – p. 43/45

OPTIMIZATIONPROBLEMS I

SKH2+

⊥

ΩKH2+

⊥ ΩIH2+

⊥

SIH2+

⊥

? ?

@@@@@@

@@@@@

@@@R

-

-

21)

12 =

1

(1− σ21)

12

Q∈H∞

+‖

UL

VL

+

M

N

Q‖∞

Qi∈H∞

+‖

UL

VL

+

M J1

N J2

Q1

Q2

‖∞

Q∈H∞

+‖

R∗ +Q

I

µ1 = minQ∈H∞

+‖ΦI + SIQ‖∞ = minQ∈H∞

+‖R∗ +Q‖∞ = minQ∈H∞

+‖ΦK +QSK‖∞

µ1

(1+µ21)12

µ1

(1 + µ21)

12

1

(1 + µ21)

12

= minQi∈H∞

+

‖

K∗1

K∗2

+

Q1

Q2

‖∞

= minQi∈H∞

+‖

M

N

+

Q1

Q2

SI‖∞

µ1

1 + µ21

= σ1(1− σ21)

12 = min

Qij∈H∞

+

‖

−N M

0 0

+

−N M

K1 K2

Q11 Q12

Q21 Q22

‖∞

= minQij∈H∞

+‖

−N∗

M∗

(

−N M

)

+

Q11 Q12

Q21 Q22

‖∞Sde Boker, MARCH 18-25, 2017 – p. 44/45

OPTIMIZATIONPROBLEMS II

SKH2+

⊥

ΩKH2+

⊥ ΩIH2+

⊥

SIH2+

⊥

6 6

@@

@@

@@

@@

@@

@@

@@I

µ2n)

− 12 = (1− σ2

n)12

minQi∈H∞

+‖(

−N M

)

+ SK

(

Q1 Q2

)

‖∞

minQi∈H∞

+‖(

J∗1 J∗

2

)

+(

Q1 Q2

)

‖∞

1

µn

= minQ∈H∞

+

‖US + SKQ‖∞ = minQ∈H∞

+

‖R∗S +Q‖∞ = min

Q∈H∞

+

‖S∗KUS +Q‖∞

(1 + µ2n)

12

µn

(1 + µ2n)

12

µn

(1 + µ2n)

12 =

minQi∈H∞

+‖(

ΦI I

)

+ SI

(

Q1 Q2

)

R‖∞ =

minQi∈H∞

+‖(

R∗S S∗

I

)

+(

Q1 Q2

)

‖∞

1 + µ2n

µn

=1

σn(1− σ2n)

= minQij∈H∞

+‖

UL

VL

(

I US

)

+

M J1

N J2

Q11 Q12

Q21 Q22

‖∞

( )

Sde Boker, MARCH 18-25, 2017 – p. 45/45