Validating Aircraft Models in the Gap Metric

Andrei Dorobantu, Gary Balas, and Tryphon Georgiou

Aerospace Engineering & Mechanics

Electrical and Computer Engineering

University of Minnesota

Gloverfest24 September 2013

Validating Aircraft Models in the Gap Metric

Andrei Dorobantu, Gary Balas, and Tryphon Georgiou

Aerospace Engineering & Mechanics

Electrical and Computer Engineering

University of Minnesota

floG v steer24 September 2013

Validating Aircraft Models in the Gap Metric

Andrei Dorobantu, Gary Balas, and Tryphon Georgiou

Aerospace Engineering & Mechanics

Electrical and Computer Engineering

University of Minnesota

lovers Get f24 September 2013

Validating Aircraft Models: Mind the Gap

4/28

Low-Cost, Safety-Critical Systems

(a) Intelligent vehicles (b) Medical devices

(c) Unmanned aircraft systems (UAS)Sources: Wikipedia, AeroVironment, Insitu

5/28

Low-Cost, Safety-Critical Systems

(c) Unmanned aircraft systems (UAS)Sources: Wikipedia, AeroVironment, Insitu

6/28

Low-Cost UAS Challenges

UAS vs. traditional aircraft

1 Limited budget

2 Restricted physical space

3 Restricted weight

4 Short development time

Source: Y. C. Yeh, Triple-triple redundant 777 primary flight computer, 1996

7/28

Efficient and Affordable Models

Significant cost savings can be achieved if Model Based Design isdone properly; however, accurate models are required.

• “There are known knowns. These are things we know that we know.There are known unknowns. That is to say, there are things that weknow we don’t know. But there are also unknown unknowns. Thereare things we don’t know we don’t know.” Donald Rumsfeld

Consider the F-35 aircraft, designed extensively based onmathematical models, which is facing challenges:

• “Test pilots say some of the F-35’s delays can be traced to theconcept that computer simulation and modeling would recognizemany of the F-35 design problems right away, instead of theold-fashioned method of testing: in the air [1].”

Cost effective flight testing provides opportunities to verifyassumptions, refine models.1. W.J. Hennigan and Ralph Vartabedian, ”F-35 fighter jet struggles to take off,” L.A. Times, 6/12/2013.

8/28

Low-Cost UAS Test Platform

9/28

System Identification & Model Validation

1 Established techniques• Time domain• Frequency domain• Software available

2 Opportunity:• Engineering insight

for low-cost systems

1 Emerging techniques• LMI-based optimization• Statistics/residuals• Ad-hoc application

2 Opportunity:• Rigorous tools tied to

robustness requirements

10/28

Problem Formulation

Main Idea• How do aerospace engineers validate models using flight data?

• Bring mathematical rigor to the process

• Ultimate goal:

• Reduce development cost• Fast, rigorous tools to validate models and control systems

• Theil’s Inequality Coefficient• Originally proposed as a tool for

economic forecasting

• Adopted for system analysis by theaerospace community

TIC(y1, y2) =

√1n

∑ni=1(y1i−y2i )2√

1n

∑ni=1(y1i )

2+√

1n

∑ni=1(y2i )

2Henri Theil

Source: Intl. Statistics Institute

11/28

Theil’s Inequality Coefficient

Aerospace engineering perspective:

1 Ranges from 0 to 1

2 TIC = 0→ perfect model

3 TIC = 1→ no correlation

4 Lower TIC → better model

TIC(y1, y2) =

√1n

∑ni=1(y1i−y2i )2√

1n

∑ni=1(y1i )

2+√

1n

∑ni=1(y2i )

2

12/28

Theil’s Inequality Coefficient

TIC(y1, y2) =

√1n

∑ni=1(y1i−y2i )2√

1n

∑ni=1(y1i )

2+√

1n

∑ni=1(y2i )

2

• TICroll = 0.07

• TICpitch = 0.12

• TICyaw = 0.26

13/28

ONE IDEA

Theil’s Inequality Coefficient ≈ Gap Metric

14/28

Arriving at the Gap Metric

Modified TIC:

TICsys(y1, y2, u) =

∣∣∣∣∣∣∣∣∣∣∣∣ uy1

− uy2

∣∣∣∣∣∣∣∣∣∣∣∣2∣∣∣∣∣∣

∣∣∣∣∣∣ uy1

∣∣∣∣∣∣∣∣∣∣∣∣2

+

∣∣∣∣∣∣∣∣∣∣∣∣ uy2

∣∣∣∣∣∣∣∣∣∣∣∣2

Gap Metric:

~δ(P1, P2) = sup||u1||2≤1

infu2

∣∣∣∣∣∣∣∣∣∣∣∣u1y1

−u2y2

∣∣∣∣∣∣∣∣∣∣∣∣2∣∣∣∣∣∣

∣∣∣∣∣∣u1y1

∣∣∣∣∣∣∣∣∣∣∣∣2

(a) TIC diagram (b) Gap diagram

15/28

The Gap Metric: Aligning Input/Outputs

Aerospace engineers already do this:

0 1 2 3 4 5 6 7 80

5

10

15

20

Orig

inal

Res

pons

e

0 1 2 3 4 5 6 7 80

5

10

15

20

Time [s]

Alig

ned

Res

pons

e

• TIC = 0.077

• TIC = 0.047

16/28

The Gap Metric: An Evolution

1 Time domain gap metric:

~δ(P1, P2) = sup||u1||2≤1

infu2

∣∣∣∣∣∣∣∣∣∣∣∣u1y1

−u2y2

∣∣∣∣∣∣∣∣∣∣∣∣2∣∣∣∣∣∣

∣∣∣∣∣∣u1y1

∣∣∣∣∣∣∣∣∣∣∣∣2

2 Frequency domain gap metric:

~δ(P1, P2) = infQ∈H∞

|| G1 −G2Q ||∞

• For linear systems P1 and P2

• G1 and G2 are graph operators, represent

input–output signals

3 ν-gap metric:

δν(P1, P2) =∣∣∣∣∣∣ (1 + P2P

∗2 )− 1

2 (P2 − P1) (1 + P1P∗1 )− 1

2

∣∣∣∣∣∣∞

• Applicable to frequency responses

G. Zames & A. El-Sakkary

T. Georgiou & M. Smith

G. Vinnicombe (Source: IEEE)

17/28

The Gap Metric and Robustness

1 Gap metric born in the robust control framework:

• Guaranteed stability margins for closed-loop systems

• Coprime factor uncertainty

• P2(s) ∈ P∆(s)def= P1(s) (1 + δ1)/(1− δ2)

• |δi| ≤ ε ≤ 1

2 Related to classical robustness margins:

• Gain margin = 20 log101+ε1−ε

• Phase margin = 2 arcsin ε

• Disk margin = 2 ε1−ε2

K. Glover, G. Vinnicombe, and G. PapageorgiouSource: IEEE

18/28

Simple Example

Model validation of simple second-order systems:

• Quantify model similarity using the gap metric

P1(s) =18.75s+ 225

s2 + 9s+ 225P2(s) =

18.75s+ 225

s2 + 7.22s+ 246.5

100

101

102

−10

0

10

Mag

nitu

de [d

B]

100

101

102

−100

−50

0

Frequeny [rad/s]

Pha

se [d

eg]

P1

P2

(a) Frequency responses

0 0.5 1 1.5 20

0.5

1

1.5

2

Res

pons

e

Time [s]

(b) Step responses

19/28

Simple Example

P1(s) =18.75s+ 225

s2 + 9s+ 225P2(s) =

18.75s+ 225

s2 + 7.22s+ 246.5

0 0.5 1 1.5−0.4

−0.2

0

0.2

0.4

Real

Imag

inar

y

PM 1/GM

0.1 rad/s

100 rad/s

15 rad/s

Perturbationε−disk

Gap Metric = 0.14 Simultaneous Gain/Phase Margin = 2.48 dB, 16.2 deg

20/28

Simple Example

P1(s) =18.75s+ 225

s2 + 9s+ 225P2(s) =

18.75s+ 225

s2 + 7.22s+ 246.5

160 170 180 190 200−5

−4

−3

−2

−1

0

1

2

3

Gai

n [d

B]

Phase [deg]

Worst−case

Perturbationε−disk (in Nichols Plane)Exclusion Region

(a) Nichols Diagram

0 1 2 3

−2.5

−2

−1.5

−1

−0.5

0

0.5

Real

Imag

inar

y

P1

P2

(b) Nyquist Diagram

Gap Metric = 0.14 Simultaneous Gain/Phase Margin = 2.48 dB, 16.2 deg

21/28

UAS Application

Real flight data and models:• 6 flight tests

• Chirp-style inputs

0 2 4 6 8 10 12

−10

0

10

Aile

ron

Inpu

t [de

g]

0 2 4 6 8 10 12

−100

0

100

Rol

l Rat

e [d

eg/s

]

0 2 4 6 8 10 12−50

0

50

Yaw

Rat

e [d

eg/s

]

0 2 4 6 8 10 12−5

0

5

Y−

Acc

el. [

m/s

2 ]

Time [s]

22/28

UAS Application

100

101

5

10

15

20

Mag

nitu

de [d

B]

100

101

−90

0

90

180

Pha

se [d

eg]

P1

P2

P2,raw

100

101

0.8

0.85

0.9

0.95

1

Coh

eren

ce [0

−1]

Frequeny [rad/s]

(a) Aileron to roll rate

100

101

5

10

15

20

Mag

nitu

de [d

B]

100

101

−90

0

90

180

Pha

se [d

eg]

P1

P2

P2,raw

100

101

0.8

0.85

0.9

0.95

1

Coh

eren

ce [0

−1]

Frequeny [rad/s]

(b) Elevator to pitch rate

100

101

5

10

15

20

Mag

nitu

de [d

B]

100

101

−90

0

90

180

Pha

se [d

eg]

P1

P2

P2,raw

100

101

0.8

0.85

0.9

0.95

1

Coh

eren

ce [0

−1]

Frequeny [rad/s]

(c) Rudder to yaw rate

23/28

UAS Application: δail → p

24/28

UAS Application: Model Validation

0.5 1 1.5−0.6

−0.3

0

0.3

0.6

Real

Imag

inar

y

Averaged dataRaw data

(a) Aileron to roll rate

0.5 1 1.5−0.6

−0.3

0

0.3

0.6

RealIm

agin

ary

Averaged dataRaw data

(b) Elevator to pitch rate

0.5 1 1.5−0.6

−0.3

0

0.3

0.6

Real

Imag

inar

y

Averaged dataRaw data

(c) Rudder to yaw rate

Roll Rate Pitch Rate Yaw RateRaw Avg. Raw Avg. Raw Avg.

Model Quality Gap Metric ε 0.20 0.14 0.14 0.10 0.27 0.19

ControllerRequirements

Gain Mar. [dB] 3.58 2.47 2.45 1.80 4.70 3.29Phase Mar. [deg] 23.43 16.21 16.10 11.82 30.73 21.29Disk Mar. 0.42 0.28 0.29 0.21 0.57 0.39

25/28

UAS Application: Model Validation (Zoom)

0.5 1 1.5−0.6

−0.3

0

0.3

0.6

Real

Imag

inar

y

Averaged dataRaw data

26/28

UAS Application: Time Domain Verification

27/28

Summary

28/28