Max Planck Institute Magdeburg
Model Order Reduction for Nonlinear Systems
Lihong Feng
MAX-PLANCK-INSTITUT
DYNAMIK KOMPLEXER
TECHNISCHER SYSTEME
MAGDEBURG
Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory
Max Planck Institute Magdeburg 2
Overview
• Linearization MOR.
• Qudratic MOR.
• Bilinearization MOR.
• Variational analysis MOR.
• Trajectory piece-wise linear MOR.
• Proper orthogonal decomposition (POD).
• References.
Max Planck Institute Magdeburg 3
Linearization MOR
Original large ODE
)()(
)()(/
tLXty
tBuXfdtCdX
)(Xf
AXXf )(~
Linearization: approximate
by a linear function )(Xf
)()(~
1
)(])(,[/ 00
tLXty
tuXDXfBAXdtCdX f
)()(
))(()(2
1)()()(
00
00000
XXDXf
XXXHXXXXDXfXf
f
f
T
f
Taylor series expansion:
)()(~
)()()(/ 00
tLXty
tBuXXDXfdtCdX f
.,1,0,])[(,~
},,,{
1
0
1
21
irCACsMBAr
rMrMrMrizationorthogonalV
i
i
j
,...1,0,)( 1 irCAM i
i
B~
Max Planck Institute Magdeburg 4
Example
1 2 3 N-3 N-2 N-1 N
C C C C C C Ci(t)
Y. Chen, MIT thesis 1999.
)()(
),(
0
0
1
)(
)()(
)()(
)()(
1
11
3221
211
tLXty
tu
xxg
xxgxxg
xxgxxg
xxgxg
dt
dX
nn
kkkk
1)( 40 xexg x
xxgg
xg
xggxg
41)0(')0(
!2
)0('')0(')0()( 2
)()(
),(
0
0
1
4141
418241
418241
4282
1
11
321
21
tLXty
tu
xx
xxx
xxx
xx
dt
dX
nn
kkk
Max Planck Institute Magdeburg 5
Example
0 2 4 6 8 100
0.005
0.01
0.015
0.02
0.025
time, u(t)=0, t<3, u(t)=1, t>=3;
voltage a
t node 1
+ q=10 __ full simulation
Max Planck Institute Magdeburg 6
Example
0 2 4 6 8 100
0.005
0.01
0.015
0.02
0.025
time, u(t)=0, t<3, u(t)=1, t>=3;
voltage a
t node 1
+ q=10
-- q=20
__full simulation
Max Planck Institute Magdeburg 7
Example
0 2 4 6 8 100
0.005
0.01
0.015
0.02
0.025
time, u(t)=0, t<3, u(t)=1, t>=3;
voltage a
t node 1
+ q=10
-- q=20
...linear, no reduction
__ full simulation
Max Planck Institute Magdeburg 8
Quadratic MOR
)(Xf )(Xg
)(Xf
)(Xg
nqRZVZX q ,,
)()(ˆ
)(~
/
tLVZty
tuBVWVZVZVAVZVdtCXdZV TTTTTT
Approximate by a quadratic polynomial
)()(
)()(/
tLXty
tBuXfdtCdX
Taylor series expansion:
)()(~)(
~/
tLXty
tuBWXXAXdtCdX T
))(()(2
1)()(
))(()(2
1)()()(
00000
00000
XXXHXXXXDXf
XXXHXXXXDXfXf
f
T
f
f
T
f
.,1,0,])[(,~
)(
},,,{
1
0
1
0
21
irCACsMBACsr
rMrMrMrizationorthogonalV
i
i
j
Max Planck Institute Magdeburg 9
Example
1 2 3 N-3 N-2 N-1 N
C C C C C C Ci(t)
Y. Chen, MIT thesis 1999.
)()(
),(
0
0
1
)(
)()(
)()(
)()(
1
11
3221
211
tLXty
tu
xxg
xxgxxg
xxgxxg
xxgxg
dt
dX
nn
kkkk
1)( 40 xexg x
22'
2'
80041!2
)0('')0()0(
!2
)0('')0()0()(
xxxg
xgg
xg
xggxg
)()(
),(
0
0
1
)(800
)(800)(800
)(800)(800
)(800800
4141
418241
418241
4282
2
1
2
1
2
1
2
32
2
21
2
21
2
1
1
11
321
21
tLXty
tu
xx
xxxx
xxxx
xxx
xx
xxx
xxx
xx
dt
dX
nn
kkkk
nn
kkk
Max Planck Institute Magdeburg 10
Example
1 2 3 N-3 N-2 N-1 N
C C C C C C Ci(t)
Y. Chen, MIT thesis 1999.
)()(
),(
0
0
1
)(800
)(800)(800
)(800)(800
)(800800
4141
418241
418241
4282
2
1
2
1
2
1
2
32
2
21
2
21
2
1
1
11
321
21
tLXty
tu
xx
xxxx
xxxx
xxx
xx
xxx
xxx
xx
dt
dX
nn
kkkk
nn
kkk
)()(~)(
~/
tLXty
tuBWXXAXdtCdX T
W
is a tensor, it has n matrices, the ith matrix
corresponds to the ith element of the
nonlinear vector.
Max Planck Institute Magdeburg 11
Example
1 2 3 N-3 N-2 N-1 N
C C C C C C Ci(t)
Y. Chen, MIT thesis 1999.
0000
0000
00800800
008001600
1
nnRW
W
1
1
000000
0000
08008000
08000800
000800800
0000
i
i
i
RW nni
1,,1 iii
80080000
800800
0000
00
0000
nnn RW
XWX
XWX
XWX
WXX
nT
iT
T
T
1
VZWVZ
VZWVZ
VZWVZ
WVZVZ
nTT
iTT
TT
TT
1
Max Planck Institute Magdeburg 12
Example
0 2 4 6 8 100
0.005
0.01
0.015
0.02
0.025
time, u(t)=0, t<3, u(t)=1, t>=3;
voltage a
t node 1
-- quadratic MOR, q=10
…linear, no reduction
__ full simulation
Max Planck Institute Magdeburg 13
Example
0 2 4 6 8 100
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
time, u(t)=0, t<3, u(t)=1, t>=3;
voltage a
t node 1
-- quadratic MOR, q=10
…quadratic MOR, q=20
__ full simulation
Max Planck Institute Magdeburg 14
Example
0 2 4 6 8 100
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
time, u(t)=0, t<3, u(t)=1, t>=3;
voltage a
t node 1
-- quadratic MOR, q=10
…quadratic, no reduction
__ full simulation
Max Planck Institute Magdeburg 15
Bilinearization MOR
)(Xf )(Xg
)(Xf
)(XgnqRZVZX q ,,
)()(ˆ
)()(/
tLVZty
tuBVtVZuNVVZAVdtdZ TTT
Approximate by quadratic polynomial , but written into Kronecker
product
)()(
)()(/
tLXty
tBuXfdtdX
Taylor series expansion:
XXAXAXf
XXXHXXXXDXf
XXXHXXXXDXfXf
f
T
f
f
T
f
210
00000
00000
)(
))(()(2
1)()(
))(()(2
1)()()(
)()(
)()(/
tXLty
tuBtuXNXAdtdX
2, nNRX N
Max Planck Institute Magdeburg 16
XX
XX
0
BB 0LL
11
21
0 AIIA
AAA
0
00
BIIBN
)()(
)()(/
tXLty
tuBtuXNXAdtdX
)()(
)()(/
tLXty
tBuXfdtdX
Carleman bilinearization
Carleman bilinearization:
[1] W.J. Rugh, Nolinear System Theory, The John Hopkins University Press, Boltimore, 1981.
[2] S. Sastry, Nonlinear Systems: Analysis, Stability and Control, Springer, New York, 1999.
Bilinearization MOR
BaBa
BaBa
BA
mnm
n
1
111
Kronecker product
Max Planck Institute Magdeburg 17
)()(
)()(/
tXLty
tuBtuXNXAdtdX
How to compute V?
1
)()(n
n tyty
nnn
t
n
reg
nn dtdtttuttttuttthty 1210
21
)( )()(),,,()(
BeNeNeLttthtAtAtAT
n
reg
nnn 11),,,( 21
)(
Volterra series expression of bilinear system According to the theory in [Rugh 1981], the output response of the bilinear system can be expressed into Volterra series,
BAAsINAAsINAAsINAAsIL
BAIsNAIsNAIsNAIsLsssh
nn
Tn
nn
T
n
reg
n
111
1
111
2
111
1
111
1
1
1
2
1
1
1
21
)(
)()()()()1(
)()()()(),,,(
Laplace transform (drop for simplicity):
Bilinearization MOR
Max Planck Institute Magdeburg 18
)()(
)()(/
tXLty
tuBtuXNXAdtdX
How to compute V?
BAAsINAAsINAAsINAAsIL
BAIsNAIsNAIsNAIsLsssh
nn
Tn
nnn
reg
n
111
1
111
2
111
1
111
1
1
1
2
1
1
1
21
)(
)()()()()1(
)()()()(),,,(
Laplace transform:
i
n
i
nn sAsAIAsI 111)(
1 1
1
1
1
21
)(
1
111)1(),,,(n
nnn
l l
lllll
n
n
n
reg
n BANNALAsssssh
BANNALAllmllln
nnn 11)1(),,( 1
Multimoments:
Bilinearization MOR
Max Planck Institute Magdeburg 19
)()(
)()(/
tXLty
tuBtuXNXAdtdX
How to compute V?
1 1
1
1
1
21
)(
1
111)1(),,,(n
nnn
l l
lllll
n
n
n
reg
n BANNALAsssssh
BANNALAllmllln
nnn 11)1(),,( 1
Multimoments:
},,{},{}{ 1
1
111
1 BABAspanbAAKVrangeq
q
},,{},{}{ 11
1
,1
1
1
11
j
q
jjjqj NVANVANVANVAAKVrange j
j
},,{}{ 1 JVVcolspanVrange
)()(ˆ
)()(/
tLVZty
tuBVtVZuNVVZAVdtdZ TTT
Reduced model:
Bilinearization MOR
Max Planck Institute Magdeburg 20
Example
0 2 4 6 8 100
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
time, u(t)=0, t<3, u(t)=1, t>=3;
voltage a
t node 1
original system
quadratic MOR, q=20
quadratic, no reduction
bilinearization, q=20
V for bilinearization MOR: },{}{ 21 VVcolspanVrange
})(,,){(}{ 191
1 BABAspanVrange
)}1(:,){(}{ 1
1
2 NVAspanVrange
V for quadratic MOR:
},,{}{ 201 BABAspanVrange
Max Planck Institute Magdeburg 21
Variational analysis MOR
XXXAXXAXAXf
XXXHXXXXDXfXf f
T
f
3210
0000
)(
))(()(2
1)()()(
Taylor series expansion:
)()(
)()(/
tLXty
tBuXfdtdX
)()(
)(~~/ 21
tLXty
tuBXXAXAdtdX
)()(
)(~~/ 321
tLXty
tuBXXXAXXAXAdtdX
)()()(),( 3
3
2
2
1 tXtXatXtX
Variational analysis:
)()(
)(~~/ 21
tLXty
tuBXXAXAdtdX
)()(
)(~~/ 321
tLXty
tuBXXXAXXAXAdtdX
or
or
Original system:
Max Planck Institute Magdeburg 22
Variational analysis MOR
)()()()( 3
3
2
2
1 tXtXatXtX
Variational analysis [11]:
)()(
)(~~/ 321
tLXty
tuBXXXAXXAXAdtdX
)()(
)(~~)]()()[(
)]()[(
)(/)(
3
3
2
2
13
3
2
2
13
3
2
2
13
3
3
2
2
13
3
2
2
12
3
3
2
2
113
3
2
2
1
tLXty
tuBXXXXXXXXXA
XXXXXXA
XXXAdtXXXd
)(~~)(/)(: 111 tuBtXAdttdX
)()(/)(: 112212
2 XXAtXAdttdX
)()()(/)(: 111312212313
3 XXXAXXXXAtXAdttdX
0)( if ,0)( :Assume tutX
Max Planck Institute Magdeburg 23
Variational analysis MOR
)()()()( 3
3
2
2
1 tXtXatXtX
Variational analysis:
)()(
)(~~/ 321
tLXty
tuBXXXAXXAXAdtdX
)(~~)(/)(: 111 tuBtXAdttdX
)()(/)(: 112212
2 XXAtXAdttdX
)()()(/)(: 111312212313
3 XXXAXXXXAtXAdttdX
}~
,,~
{ 1
1
1
11111 BABAspanVZVXq
},,{ 212
1
122222 AAAAspanVZVX
q
]},[,],,[{ 32132
1
133332 AAAAAAspanVZVX
q
33
3
22
2
11
3
3
2
2
1
3
3
2
2
1 )()()()(
ZVZVZV
XXX
tXtXatXtX
},,{)( 321 VVVspantX
Max Planck Institute Magdeburg 24
Variational analysis MOR
Original system:
)()(
)()(/
tLXty
tBuXfdtdX
)()(
)(~~/ 321
tLXty
tuBXXXAXXAXAdtdX
},,{)( 321 VVVspantX
Compute V: },,{)( 321 VVVspanVrange
Reduced model: )()(ˆ
)(~~/ 321
tLVZty
tuBVVZVZVZAVVZVZAVVZAVdtdZ TTTT
VZtX )(
Max Planck Institute Magdeburg 25
Example
V for bilinearization MOR:
},{}{ 21 VVcolspanVrange
},,{}{ 191
1 BABAspanVrange
)}1(:,{}{ 1
2 NVAspanVrange
V for Variational analysis MOR:
},{}{
)}6:1(:,{}{
}{
},,{}{
21
0
2
1
12
2
0
2
41
11
VVspanVrange
VAspanVrange
AorthV
BABAspanVrange
0 2 4 6 8 100
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
time, u(t)=0, t<3; u(t)=1, t>=3
voltage a
t node 1
original exact output response
Bilinearization MOR, q=20, relative error=0.0078
Variational analysis MOR, q=9, relative error=0.0073
Max Planck Institute Magdeburg 26
Example
V for [Phillips 2000] method:
V for our method [Feng 2014]:
},{}{
)]}6:1(:,[{}{
}{
},,{}{
21
0
2
1
12
2
0
2
41
11
VVspanVrange
VAspanVrange
AorthV
BABAspanVrange
},{}{
)}6:1(:,{}{
)(
},,{}{
21
0
22
112
10
2
41
11
VVspanVrange
VspanVrange
VVAAV
BABAspanVrange
0 2 4 6 8 100
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
time, u(t)=0, t<0.3; u(t)=1, t>=0.3
voltage a
t node 1
original
our method, q=9, relative error=0.0073
Phillips 2003 method, q=9, relative error=0.0076
Max Planck Institute Magdeburg 27
Trajectory piece-wise linear MOR
)(Xf
)(1 Xg
)(2 Xg
)(4 Xg
)(3 Xg
0X1X
2X
)()(
)()(/
tLXty
tBuXfdtdX
Original system:
)()(
,)(~/1
0
tLXty
BuXgwdtdXs
i
ii
1,,1,0),()()( siXXAXfXg iiii
iXk
j
jkjkix
XfaaA
)(),(
Max Planck Institute Magdeburg 28
Trajectory piece-wise linear MOR
)()(
)()(/
tLXty
tBuXfdtdX
Original system:
)()(
,)(/1
0
tLXty
BuXgdtdXs
i
i
))((~~
1,,1,0),()(~)()(~)(
iiiiii
iiiiii
XAXfwXAw
siXXAXwXfXwXg
)()(
),()~~(/ 0
1
0
tLXty
tBuwBXAwdtdX i
s
i
ii
ii AXXfBBBB )(],,[~
00
How to compute V?
},,{}{
1,,1,0}~
,,~
{}{
11
1
s
i
q
iiii
VVspanVrange
siBABAspanVrange i
Max Planck Institute Magdeburg 29
Trajectory piece-wise linear MOR
)()(
)()(/
tLXty
tBuXfdtdX
Original system:
)()(ˆ
),(~~(/ 0
1
0
tLVZty
tBuVwBVVZAVwdtdZ T
i
Ts
i
i
T
i
Trajectory piece-wise linear system:
Reduced model:
)()(
),()~~(/ 0
1
0
tLXty
tBuwBXAwdtdX i
s
i
ii
Max Planck Institute Magdeburg 30
Proper orthogonal decomposition (POD)
POD and SVD
SVD:
nmTT RD
YVUVUY
:
00
0or
s.t. ,),,( and ),,(exist there,matrix any For 11
nn
n
mm
m
nm RvvVRuuURY
ly.respective and
of columns first theincluding matrices thebe and Let ).,,( Here, 1
VU
dVUdiagD dd
d
It is obvious,
d
i
d
i
iRji
d
i
iRiji
m
k
kj
d
ki
d
i
iij
Td
d
i
iij
TddTdd
i
iij
Td
ij
Td
i
d
ij
Tdd
n
uyuuuyuYUuYU
uVDUUuVDVDuy
VDUyyY
mm
1 1111
111
1
.,,))((
))()(())(())((
)(),,(
Y can be represented in terms of d linearly independent columns of . dU
Max Planck Institute Magdeburg 31
Proper orthogonal decomposition (POD)
Definition
.,1 ,~,~ s.t. minarg}{1
~,,~11
ljiuuu ijRji
n
j
jRuu
l
ii mml
.rank of basis POD called are }{ vectors the},,1{For 1 ludl l
ii
: of columns theingapproximatin
ions,approximat rank all among optimal, is }{ basis POD The 1
Y
lu l
ii
2
1||~~,|| Here, mm RiRij
l
ijj uuyy
Max Planck Institute Magdeburg 32
Model Order Reduction using POD
)())(()(
ROM get the to Use.3
)~,,~( ,~~
of from rank of vectorsPOD Get the 2.
),(
snapshots get the tosystemnonlinear original theSolve 1.
1
1
tBuVtVzfVdt
tdzEVV
V
uuVVUX
XSVDq
xxX
TTT
q
T
tt m
Algorithm MOR using POD
How to deal with ?
An effective way is to approximate the nonlinear function by projecting it onto
a subspace that approximates the space generated by the nonlinear function,
and with dimension
))(( tVzf
.nl
),,( ),()( 1 luuUtUctf
Max Planck Institute Magdeburg 33
DEIM
).()()()(
that,so
)()()()()(
thenr,nonsingula is Suppose
,],,[
matrix aconsider ,particularIn
).()(
system inedoverdeterm thefrom rows heddistinguis select we,)( determine To
1
1
1
tfPUPUtUctf
tfPUPtctUcPtfP
UP
ReeP
tUctf
mtc
TT
TTTT
T
mn
l
?,,1, indices especify th tohow and compute toHow liU i
Compute :
U
).,,( 3.
)( : toApply 2.
)).(,),((matrix a into ))(( of snapshots eCollect th 1.
1
1
F
l
F
TFF
tt
uuU
VUFFSVD
xfxfFtxfm
Max Planck Institute Magdeburg 34
DEIM
Using DEIM to decide the indices:
Algorithm Discrete Empirical Interpolation Method (DEIM)
for end .8
], [ ], [ .7
||max]|,[| .6
.5
),,( where,for )( Solve 4.
do to2for .3
][ ],[ ],[ .2
|| max]|,.[|1
],[ :Output
Ffor basis POD :Input
T
11
11
11
1
1
1
i
F
i
l
F
i
i
F
i
TT
F
F
lT
l
l
i
F
i
iePPuUU
r
Uur
uPUP
li
ePuU
u
R
u
Max Planck Institute Magdeburg 35
DEIM
Come back to :
))(( tVzfV T
)).(()())(( 1 tVzfPUPUtVzf TT
. oft independen still is ROM solving during ))(( ofn computatio then ),( of entries
few aon dependsonly entry each but above, as evaluatedisely componentwnot is ))(( If
. oft independen is ROM solving during ))(( ofn Computatio
))(()())((
that so
))(())((
then))),((,)),((())(( If
1
11
ntVzfVtx
ftxf
ntVzfV
tVzPfUPUVtVzfV
tVzPftVzfP
txftxftxf
T
i
T
TTTT
TT
nn
can be precomputed before solving the ROM
Max Planck Institute Magdeburg 36
References
Quadratic MOR:
[1] Yong Chen, “Model order reduction for nonlinear systems”, MIT master thesis, 1999.
Bilinearization MOR:
[2] J. R. Phillips, “Projection Frameworks for Model Reduction of Weakly Nonlinear
Systems”, Proc. IEEE/ACM DAC, 184-189, 2000.
[3] Z. Bai, and D. Skoogh, “A projection method for model reduction of bilinear dynamical systems”, Linear Algebra Appl., 415: 406-425, 2006.
[4] L. Feng and P. Benner, A Note on Projection Techniques for Model Order Reduction of
Bilinear Systems, AIP Conf. Proc. 936: 208-211, 2007.
Max Planck Institute Magdeburg Model Order Reduction of Parametric Systems 37 30.06.2014
Variational analysis MOR:
[5] J. R. Phillips, “Automated Extraction of Nonlinear Circuit Macromodels ”, Proc. of IEEE
Custom Itegrated Circuits Conference, 451-454, 2000.
[6] L. Feng, X. Zeng, Ch. Chiang, D. Zhou, Q, Fang “Direct nonlinear order reduction with
variational analysis”, Proc. of Design, Automation and Test in Europe Conference and
Exhibition, 2:1316-1321, 2004.
Trajectory piece-wise linear (polynomial) MOR:
[7] M. Rewienski and J. White, “ A Trajectory Piecewise-Linear Approach to Model Order
Reduction and Fast Simulation of Nonlinear Circuits and Micromachined Devices”, IEEE
Trans. Comput.-Aided Des. Integr. Circuits Syst., 22(2):155-170, 2003.
[8] M. Rewienski and J. White, “ Model order reduction for nonlinear dynamical systems based on trajectory piecewise-linear approximations ”, Linear Algebra Appl., 415(2-3):426-454,
2006.
References
Max Planck Institute Magdeburg Model Order Reduction of Parametric Systems 38 30.06.2014
References
POD:
[9] S. Volkwein, “Model reduction using proper orthogonal decomposition ”, Lecture notes, June 7,
2010.
[10] S. Schaturantabut and D. C. Sorensen, “Nonlinear model reduction via discrete empirical
interpolation”, SIAM J. Sci. Comput. 32(5): 2737-2764, 2010.
A very good book for nonlinear system:
[11] W. J. Rugh, “Nonlinear system theory, the Volterra/Wiener Approach”, The Johns Hopkins
University Press, 1981.
Top Related