Spectral Properties of Linear-Quadratic Regulators

Robert Stengel Optimal Control and Estimation MAE 546

Princeton University, 2018

Copyright 2018 by Robert Stengel. All rights reserved. For educational use only.http://www.princeton.edu/~stengel/MAE546.html

http://www.princeton.edu/~stengel/OptConEst.html

!  Stability margins of single-input/single-output (SISO) systems

!  Characterizations of frequency response!  Loop transfer function!  Return difference function!  Kalman inequality!  Stability margins of scalar linear-

quadratic regulators

1

Return Difference Function and Closed-Loop Roots

Single-Input/Single-Output Control Systems

2

SISO Transfer Function and Return Difference Function

A s( ) : Open - Loop Transfer Function

1+ A s( )⎡⎣ ⎤⎦ : Return Difference Function

•  Block diagram algebra

Δy s( ) = A s( ) ΔyC s( )− Δy s( )⎡⎣ ⎤⎦1+A s( )⎡⎣ ⎤⎦Δy s( ) = A s( )ΔyC s( )

•  Unit feedback control law

Δy s( )ΔyC s( ) =

A s( )1+ A s( ) : Closed - Loop Transfer Function

3

Return Difference Function and Root Locus

A s( ) = kn s( )d s( )

1+ A s( ) = 1+kn s( )d s( ) = 0 defines locus of roots

d s( ) + kn s( ) = 0 defines locus of closed-loop roots

4

Return Difference Example

A s( ) = kn s( )d s( ) =

k s − z( )s2 + 2ζω ns +ω n

2 =1.25 s + 40( )

s2 + 2 0.3( ) 7( )s + 7( )2

1+ A s( ) = 1+ k s − z( )s2 + 2ζω ns +ω n

2 = 1+1.25 s + 40( )

s2 + 2 0.3( ) 7( )s + 7( )2= 0

s2 + 2 0.3( ) 7( )s + 7( )2⎡⎣ ⎤⎦ +1.25 s + 40( ) = 05

Closed-Loop Transfer Function Example

A s( )1+ A s( ) =

1.25 s + 40( )s2 + 2 0.3( ) 7( )s + 7( )2

⎡

⎣⎢⎢

⎤

⎦⎥⎥

1+ 1.25 s + 40( )s2 + 2 0.3( ) 7( )s + 7( )2

⎡

⎣⎢⎢

⎤

⎦⎥⎥

A s( )1+A s( ) =

1.25 s + 40( )s 2 + 2 0.3( ) 7( )s + 7( )2⎡⎣ ⎤⎦ +1.25 s + 40( )

6

=1.25 s + 40( )

s 2 + 2 0.3( ) 7( ) +1.25⎡⎣ ⎤⎦s + 7( )2 +1.25 40( )⎡⎣ ⎤⎦

=kn s( )

d s( ) + kn s( )

Open-Loop Frequency Response: Bode Plot

A jω( ) = K jω − z( )jω( )2 + 2ζωn jω +ωn

2=

1.25 jω + 40( )jω( )2 + 2 0.3( ) 7( ) jω + 7( )2

•  Gain Margin–  Referenced to 0 dB line–  Evaluated where phase angle =

–180°•  Phase Margin

–  Referenced to –180°–  Evaluated where amplitude ratio

= 0 dB

Two plots• 20 log A jω( ) vs. logω

• A jω( )⎡⎣ ⎤⎦ vs. logω

7

Open-Loop Frequency Response: Nyquist Plot

•  Single plot; input frequency not shown explicitly

•  Gain and Phase Margins referenced to (–1) point

•  GM and PM represented as length and angle

•  Dotted lines are M-Circles denoting the amplitude of closed-loop frequency response (dB)

Only positive frequencies need be considered

Re A jω( )( ) vs. Imag A jω( )( )

8

20 logΔy jω( )ΔyC jω( ) = 20 log

A jω( )1+ A jω( )

Algebraic Riccati Equation in the Frequency Domain

9

Linear-Quadratic Control

J = 12

ΔxT (t) ΔuT (t)⎡⎣

⎤⎦Q 00 R

⎡

⎣⎢

⎤

⎦⎥

Δx(t)Δu(t)

⎡

⎣⎢⎢

⎤

⎦⎥⎥dt

0

∞

∫

= 12

ΔxT (t)QΔx(t)+ Δu(t)RΔuT (t)⎡⎣ ⎤⎦dt0

∞

∫

Δx(t) = FΔx(t) +GΔu(t)

Quadratic cost function for infinite final time

•  Linear, time-invariant dynamic system

•  Constant-gain optimal control law

0 = −Q − FTP − PF + PGR−1GTPQ = −FTP − PF + CTRC

Algebraic Riccati equation

10

Δu t( ) = −R−1GTPΔx t( ) = −CΔx t( )

C ! R−1GTP

Frequency Characteristics of the Algebraic Riccati Equation

Add and subtract sP such that

P sIn − F( ) + −sIn − FT( )P +CTRC = Q

P −F( ) + −FT( )P +CTRC = Q

State Characteristic

Matrix

AdjointCharacteristic

Matrix

11

Frequency Characteristics of the Algebraic Riccati Equation

GT −sIn − FT( )−1PG +GTP sIn − F( )−1G +GT −sIn − F

T( )−1CTRC sIn − F( )−1G=GT −sIn − F

T( )−1Q sIn − F( )−1G

Pre-multiply each term by

Post-multiply each term by

P sIn − F( ) + −sIn − FT( )P +CTRC = Q

sIn − F( )−1G

GT −sIn − FT( )−1

12

Frequency Characteristics of the Algebraic Riccati Equation

Substitute on left using the control gain matrix

Add R to both sides

GT −sIn − FT( )−1CTR +RC sIn − F( )−1G +GT −sIn − F

T( )−1CTRC sIn − F( )−1G=GT −sIn − F

T( )−1Q sIn − F( )−1G

R +GT −sIn − FT( )−1CTR +RC sIn − F( )−1G +GT −sIn − F

T( )−1CTRC sIn − F( )−1G= R +GT −sIn − F

T( )−1Q sIn − F( )−1G

C = R−1GTPGTP = RC

13

Frequency Characteristics of the Algebraic Riccati Equation

R +GT −sIn − FT( )−1CTR +RC sIn − F( )−1G +GT −sIn − F

T( )−1CTRC sIn − F( )−1G= R +GT −sIn − F

T( )−1Q sIn − F( )−1GThe left side can be factored as*

Im +GT −sIn −FT( )−1

CT⎡⎣⎢

⎤⎦⎥R Im +C sIn −F( )−1

G⎡⎣⎢

⎤⎦⎥

= R +GT −sIn −FT( )−1

Q sIn −F( )−1G

* Verify by multiplying the factored form14

= R + GT −sIn −FT( )

−1HT⎡

⎣⎢⎤⎦⎥

H sIn −F( )−1G⎡

⎣⎢⎤⎦⎥

Modal Expression of Algebraic Riccati EquationDefine the loop transfer function matrix

A s( ) ! C sIn − F( )−1G

15

Modal Expression of Algebraic Riccati EquationRecall the cost function transfer matrix

Laplace transform of algebraic Riccati equation becomes

Y1 s( ) ! Y s( ) = H sIn − F( )−1G, where Q = HTH

Im + A −s( )⎡⎣ ⎤⎦T R Im + A s( )⎡⎣ ⎤⎦ = R + YT −s( )Y s( )

16

Algebraic Riccati Equation

•  Cost function transfer matrix, Y(s)–  Reflects control-induced state

variations in the cost function–  Governs closed-loop modal properties

as R becomes small–  Does not depend on R or P

Im + A −s( )⎡⎣ ⎤⎦T R Im + A s( )⎡⎣ ⎤⎦ = R + YT −s( )Y s( )

Y s( ) = H sIn − F( )−1G

Δui = −C siIn − F( )−1GΔui= −A si( )Δui , i = 1,n

•  Loop transfer function matrix, A(s)–  Defines the modal control vector

when s = si

17

LQ Regulator Portrayed as a Unit-Feedback System

A s( ) = C sIn − F( )−1G

Im + A s( ) = Im +C sIn − F( )−1G : Return Difference Matrix

18

Determinant of Return Difference Matrix Defines Closed-Loop Eigenvalues

Im +A s( ) = Im +C sIn − F( )−1G

= Im +CAdj sIn − F( )G

sIn − F= Im +

CAdj sIn − F( )GΔOL s( )

ΔCL s( ) = ΔOL s( ) Im +A s( )

= ΔOL s( ) Im +CAdj sIn − F( )G

ΔOL s( )

=ΔOL s( )ΔOL s( ) ΔOL s( )Im + CAdj sIn − F( )G = 0

Closed-Loop Characteristic Equation

19

Stability Margins and Robustness of Scalar

LQ Regulators

20

Scalar Case

Im + A −s( )⎡⎣ ⎤⎦T R Im + A s( )⎡⎣ ⎤⎦ = R + YT −s( )Y s( )

Multivariable algebraic Riccati equation

Algebraic Riccati equation with scalar control

1+ A −s( )⎡⎣ ⎤⎦r 1+ A s( )⎡⎣ ⎤⎦ = r +YT −s( )Y s( )

whereA s( ) = C sIn − F( )−1G 1×1( )dim C( ) = 1× n( )dim F( ) = n × n( )dim G( ) = n ×1( )

Y s( ) = H sIn − F( )−1Gdim Y s( )⎡⎣ ⎤⎦ = p ×1( )dim H( ) = p × n( )

21

Scalar Case

1+ A − jω( )⎡⎣ ⎤⎦r 1+ A jω( )⎡⎣ ⎤⎦ = r +YT − jω( )Y jω( )

or

1+ A − jω( )⎡⎣ ⎤⎦ 1+ A jω( )⎡⎣ ⎤⎦ = 1+Y T − jω( )Y jω( )

r

1+ A − jω( )⎡⎣ ⎤⎦ 1+ A jω( )⎡⎣ ⎤⎦ = 1+ c ω( )⎡⎣ ⎤⎦ − jd ω( ){ } 1+ c ω( )⎡⎣ ⎤⎦ + jd ω( ){ }= 1+ c ω( )⎡⎣ ⎤⎦

2+d 2 ω( ){ } = 1+ A jω( ) 2 (absolute value)

Let s = jω

A jω( ) is a complex variable

22

Kalman Inequality

In frequency domain, cost transfer function becomes

Consequently, the return difference function magnitude is greater than one

Yi jω( ) = li ω( ) + jmi ω( )⎡⎣ ⎤⎦, i = 1, p

1+Y T − jω( )Y jω( )

r= 1+

li2 ω( ) + mi

2 ω( )⎡⎣ ⎤⎦ri=1

p

∑

1+ A jω( ) ≥ 1 Kalman Inequality

23

Nyquist Plot Showing Consequences of Kalman Inequality

24

1+ A jω( ) stays outside of unit circle centered on (–1,0)

Uncertain Gain and Phase Modifications to the LQ

Feedback Loop

How large an uncertainty in loop gain or phase angle can be tolerated by the LQ regulator?

25

UncertainElement

LQ Gain Margin Revealed by

Kalman Inequality

!  Stability is preserved if!  No encirclements of the –1 point,

or!  Number of counterclockwise

encirclements of the –1 point equals the number of unstable open-loop roots

!  Nominal LQR satisfies the criteria!  Loop gain change expands or

shrinks entire Nyquist plot26

Nyquist Stability Criteria

Gain Change Expands or Shrinks Entire Plot

!  Closed-Loop LQ system is stable until –1 point is reached, and # of encirclements changes

kU ! Uncertain gain

Aoptimal jω( ) = CLQ jωIn − F( )−1G

Anon−optimal jω( ) = kUAoptimal jω( )= kUCLQ jωIn − F( )−1G

Anon−optimal jω( ) = kU Aoptimal jω( )

27

Scalar LQ Regulator Gain Margin

Increased gain margin = InfinityDecreased gain margin = 50% 28

LQ Phase Margin Revealed by Kalman

Inequality!  Stability is preserved if

!  No encirclements of the –1 point!  Number of counterclockwise encirclements of the –1

point equals the number of unstable open-loop roots

Therefore, Phase Margin of LQ regulator ≥ 60°

Return Difference Function, 1 + A jω( ), is excluded from a unit circle centered at –1,0( )

A jω( ) = 1 intersects a unit circle centered at the origin

Intersection of the unit circles occurs where the phase angle of A jω( ) = –180o ± 60o( )

29

LQ Regulator Preserves Stability with Phase Uncertainties of At Least –60°

!  Phase-angle change rotates entire Nyquist plot

!  Closed-Loop LQ system is stable until –1 point is reached

ϕU ! Uncertain phase angle, deg

Aoptimal jω( ) = CLQ jωIn − F( )−1G

Anon−optimal jω( ) = e jϕU Aoptimal jω( )= e jϕUCLQ jωIn − F( )−1G

ϕnon−optimal =ϕoptimal +ϕU

30

Reduced-Gain-/Phase-Margin Tradeoff

Reduced loop gain decreases allowable phase lag while retaining closed-loop stability (see OCE, Fig. 6.5-5)

31

Control Design for Increased Gain Margin

!  Obtain lowest possible LQ control gain matrix, C, by choosing large R!  Gain margin is 1/2 of these gains!  Speed of response (e.g., bandwidth) may be too slow

!  Increase gains to restore desired bandwidth!  Control system is sub-optimal but has higher

gain margin than LQ system designed for same bandwidth

32

Control Design for Increased Gain Margin

R ! ρ 2Ro

FTP+ PF +Q− PGR−1GTP = 0Copt = R

−1GTP

Csub−opt = Ro−1GTP = ρ2Copt

Ksub−opt =12ρ2

,∞⎛⎝⎜

⎞⎠⎟

!  High R, low-gain optimal controller

!  Increased gain to restore bandwidth

!  Increased gain margin for high-bandwidth controller

33

Example: Control Design for Increased Robustness

(Ray, Stengel, 1991)

!  Three controllers!  a) Q = diag(1 1 1 0) and R = 1!  b) R = 1000!  c) Case (b) with gains multiplied by 5

Open-loop longitudinal eigenvaluesλ1−4 = −0.1± 0.057 j, − 5.15, 3.35

34

Loop Transfer Function Frequency Response with

Elevator Control H jω( ) = C jωI − F( )−1G

Q = 1 0 1 0⎡⎣ ⎤⎦, R = 1 or 1000

35

Loop Transfer Function Nyquist Plots with Elevator Control

H jω( ) = C jωI − F( )−1G

36

Next Time: Singular Value Analysis of

LQ Systems

37

Supplemental Material

38

Root Loci for Three Cases Transmission zeros

z1,2 = 0 −1.2⎡⎣ ⎤⎦Case aCase b

Case c

!  Closed-loop system roots!  Originate at stable images of open-loop poles!  2 roots to transmission zeros!  2 roots to –∞, multiple Butterworth spacing

39

Open-Loop Frequency Response: Nichols Chart

!  Single plot!  Gain and Phase

Margins shown directly

20 log10 A jω( )⎡⎣ ⎤⎦ vs. A jω( )⎡⎣ ⎤⎦

40

Loop Transfer Function Nichols Charts with Elevator Control

H jω( ) = C jωI − F( )−1G

41

Probability of Instability Describes Robustness to Parameter Uncertainty

(Ray, Stengel, 1991)!  Distribution of closed-loop roots with

!  Gaussian uncertainty in 10 parameters!  Uniform uncertainty in velocity and air density

!  25,000 Monte Carlo evaluations

Stochastic Root Locus

!  Probability of instability!  a) Pr = 0.072!  b) Pr = 0.021!  c) Pr = 0.0076

42

Effect of Time Delay

43

Time Delay Example: DC Motor Control

Control command delayed by τ sec

44

Effect of Time Delay on Step Response

With no delay

Phase lag due to time delay reduces closed-loop stability

45

As input frequency increases, phase angle eventually exceeds –180°

AR e− jτω( ) = 1φ e− jτω( ) = −τω

Bode Plot of Pure Time Delay

46

Effect of Pure Time Delay on LQ Regulator Loop Transfer Function

Laplace transform of time-delayed signal:

L u t −τ( )⎡⎣ ⎤⎦ = e−τ sL u t( )⎡⎣ ⎤⎦ = e

−τ su s( )τU ! Uncertain time delay, sec

Aoptimal jω( ) = CLQ jωIn − F( )−1GAnon−optimal jω( ) = e–τU jωAoptimal jω( )

= e–τU jωCLQ jωIn − F( )−1G47

Effect of Pure Time Delay on LQ Regulator Loop Transfer Function

Crossover frequency, ω cross , is frequency for which

Aoptimal jω cross( ) = CLQ jω crossIn − F( )−1G = 1

Time delay that produces 60° phase lag

τU =60°ω cross

π180°

⎛⎝⎜

⎞⎠⎟=

π3ω cross

, sec

48