Post on 08-Apr-2016
description
Model Reduction – 2013
Class 8
Department of Electrical EngineeringEindhoven University of Technology
Siep Weiland
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 1 / 39
Outline
1 Function approximationspectral decompositionsproper orthogonal decompositions
2 Projection frameworklinear systemsnon-linear systemsinfinite dimensional systems
3 Reduction of spatial-temporal systemsa wave propagation examplehow to choose basis functions?how to derive coefficients?results on wave propagation (Fourier)results on wave propagation (POD)
4 Summary
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 2 / 39
Function approximation
Function approximation
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 3 / 39
Function approximation spectral decompositions
spectral decompositions
Image RGB decomposition
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 4 / 39
Function approximation spectral decompositions
Decompositions of light and sound
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 5 / 39
Function approximation spectral decompositions
Function approximation
Given function w ∈ W, find function wr ∈ Wr so as to minimize‖w − wr‖.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 6 / 39
Function approximation spectral decompositions
Function approximation
Given function w ∈ W, find function wr ∈ Wr so as to minimize‖w − wr‖.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5originalpolynomial degree 5
• Wr polynomials of degree ≤ r
• wr (x) = a0 + a1x + . . .+ arxr , r = 5
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 6 / 39
Function approximation spectral decompositions
Function approximation
Given function w ∈ W, find function wr ∈ Wr so as to minimize‖w − wr‖.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5originalpolynomial degree 10
• Wr polynomials of degree ≤ r
• wr (x) = a0 + a1x + . . .+ arxr , r = 10
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 6 / 39
Function approximation spectral decompositions
Function approximation
Given function w ∈ W, find function wr ∈ Wr so as to minimize‖w − wr‖.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5originalpolynomial degree 15
• Wr polynomials of degree ≤ r
• wr (x) = a0 + a1x + . . .+ arxr , r = 15
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 6 / 39
Function approximation spectral decompositions
Function approximation
Given function w ∈ W, find function wr ∈ Wr so as to minimize‖w − wr‖.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5originalFourier degree 5
• Wr trigonometric functions of frequency ≤ r2π/|X|• wr (x) = a0 + a1 cos(πx) + . . .+ ar cos(rπx), r = 5
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 6 / 39
Function approximation spectral decompositions
Function approximation
Given function w ∈ W, find function wr ∈ Wr so as to minimize‖w − wr‖.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5originalFourier degree 10
• Wr trigonometric functions of frequency ≤ r2π/|X|• wr (x) = a0 + a1 cos(πx) + . . .+ ar cos(rπx), r = 10
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 6 / 39
Function approximation spectral decompositions
Function approximation
Given function w ∈ W, find function wr ∈ Wr so as to minimize‖w − wr‖.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5originalFourier degree 15
• Wr trigonometric functions of frequency ≤ r2π/|X|• wr (x) = a0 + a1 cos(πx) + . . .+ ar cos(rπx), r = 15
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 6 / 39
Function approximation spectral decompositions
Solution by projection
Projection framework
Equip function space W with inner product 〈·, ·〉
with
• Norm ‖w‖2 = 〈w ,w〉 induced by inner product
• Orthogonal projection Π :W →Wr onto Wr
Projection theorem
w∗r in Wr minimizes ‖w − wr‖ if and only if
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 7 / 39
Function approximation spectral decompositions
Solution by projection
Projection framework
Equip function space W with inner product 〈·, ·〉
with
• Norm ‖w‖2 = 〈w ,w〉 induced by inner product
• Orthogonal projection Π :W →Wr onto Wr
Projection theorem
w∗r in Wr minimizes ‖w − wr‖ if and only if
w∗r = Πw
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 7 / 39
Function approximation spectral decompositions
Solution by projection
Projection framework
Equip function space W with inner product 〈·, ·〉
with
• Norm ‖w‖2 = 〈w ,w〉 induced by inner product
• Orthogonal projection Π :W →Wr onto Wr
Projection theorem
w∗r in Wr minimizes ‖w − wr‖ if and only if
w − w∗r ⊥ Wr
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 7 / 39
Function approximation spectral decompositions
Solution by projection
Projection framework
Equip function space W with inner product 〈·, ·〉
with
• Norm ‖w‖2 = 〈w ,w〉 induced by inner product
• Orthogonal projection Π :W →Wr onto Wr
Projection theorem
w∗r in Wr minimizes ‖w − wr‖ if and only if
〈w − w∗r , v〉 = 0 for all v ∈ Wr
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 7 / 39
Function approximation spectral decompositions
Solution by projection
Projection framework
Equip function space W with inner product 〈·, ·〉
with
• Norm ‖w‖2 = 〈w ,w〉 induced by inner product
• Orthogonal projection Π :W →Wr onto Wr
Projection theorem
w∗r in Wr minimizes ‖w − wr‖ if and only if
w∗r = U(U>U)−1U>w with im(U) =Wr
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 7 / 39
Function approximation spectral decompositions
Solution by spectral decompositions
Write w ∈ W as
w(x) =∑
k akϕk(x)
• ϕk basis functions in W that are orthonormal in the sense that
〈ϕk , ϕ`〉 =
{1 if k = `
0 otherwise
• ak = 〈w , ϕk〉 are (Fourier) coefficients
If Wr = span(ϕ1, . . . , ϕr ) then
w∗r (x) =∑r
k=1 akϕk(x)
minimizes ‖w − wr‖ over all wr ∈ Wr .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 8 / 39
Function approximation spectral decompositions
Spectral decompositions - example
Joseph Fourier (1768-1830)
• W = L2([0, 1]) with innerproduct
〈f , g〉 =
∫ 1
0f (x)gH(x) dx
• Fourier basis
ϕk(x) = e ikπx
• Optimal approximation of w
w∗r (x) =r∑
k=0
〈w , ϕk〉ϕk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 9 / 39
Function approximation spectral decompositions
Spectral decompositions - example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2originalFourier degree 4
r = 4
• W = L2([0, 1]) with innerproduct
〈f , g〉 =
∫ 1
0f (x)gH(x) dx
• Fourier basis
ϕk(x) = e ikπx
• Optimal approximation of w
w∗r (x) =r∑
k=0
〈w , ϕk〉ϕk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 9 / 39
Function approximation spectral decompositions
Spectral decompositions - example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2originalFourier degree 8
r = 8
• W = L2([0, 1]) with innerproduct
〈f , g〉 =
∫ 1
0f (x)gH(x) dx
• Fourier basis
ϕk(x) = e ikπx
• Optimal approximation of w
w∗r (x) =r∑
k=0
〈w , ϕk〉ϕk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 9 / 39
Function approximation spectral decompositions
Spectral decompositions - example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2originalFourier degree 20
r = 20
• W = L2([0, 1]) with innerproduct
〈f , g〉 =
∫ 1
0f (x)gH(x) dx
• Fourier basis
ϕk(x) = e ikπx
• Optimal approximation of w
w∗r (x) =r∑
k=0
〈w , ϕk〉ϕk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 9 / 39
Function approximation spectral decompositions
Spectral decompositions - example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2originalFourier degree 100
r = 100
• W = L2([0, 1]) with innerproduct
〈f , g〉 =
∫ 1
0f (x)gH(x) dx
• Fourier basis
ϕk(x) = e ikπx
• Optimal approximation of w
w∗r (x) =r∑
k=0
〈w , ϕk〉ϕk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 9 / 39
Function approximation spectral decompositions
Spectral decompositions - example
Josiah Wilard Gibbs (1839-1903)
• W = L2([0, 1]) with innerproduct
〈f , g〉 =
∫ 1
0f (x)gH(x) dx
• Fourier basis
ϕk(x) = e ikπx
• Optimal approximation of w
w∗r (x) =r∑
k=0
〈w , ϕk〉ϕk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 9 / 39
Function approximation spectral decompositions
Spectral decompositions - example
Gives strong convergence
limr→∞
∫ 1
0|w(x)− wr (x)|2 dx = 0
but possibly
limr→∞
|w(x)− wr (x)| 6= 0
for some x .(no pointwise convergence).
• W = L2([0, 1]) with innerproduct
〈f , g〉 =
∫ 1
0f (x)gH(x) dx
• Fourier basis
ϕk(x) = e ikπx
• Optimal approximation of w
w∗r (x) =r∑
k=0
〈w , ϕk〉ϕk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 9 / 39
Function approximation spectral decompositions
The POD basis problem
Given a collection of M functions w1, w2, . . . , wM in W
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−4
−3
−2
−1
0
1
2
3
4
5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
Find r dimensional subspace Wr = span(ϕ1, . . . , ϕr ) such that the averageerror
1
M
M∑j=1
‖wj − Πwj‖2
is minimal.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 10 / 39
Function approximation proper orthogonal decompositions
The POD basis problem
The POD basis problem
Given data w1, . . . ,wM ∈ W, find orthonormal basis {ϕk , k = 1, 2, . . .} ofW such that the error
J(ϕ1, . . . , ϕr ) =M∑j=1
‖wj −r∑
k=1
〈wj , ϕk〉ϕk‖2
is minimal for all truncation levels r .
• Basis will be data dependent
• Needs inner product 〈f , g〉 on W.
• Orthonormal means
〈ϕk , ϕ`〉 =
{1 if k = `
0 otherwise
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 11 / 39
Function approximation proper orthogonal decompositions
Solution of POD basis problem
Theorem
Suppose that W = RN . An orthonormal basis {ϕk , k = 1, . . . ,N} of W isa POD basis if and only if
WW>ϕk = λkϕk
where λ1 ≥ · · · ≥ λn, n = rank(W ) and
W =(w1 · · · wM
)∈ RN×M .
Moreover, in that case
J(ϕ1, . . . , ϕr ) =∑k>r
λk
POD basis can be obtained from eigenvalue decomposition!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 12 / 39
Function approximation proper orthogonal decompositions
Proof
For any orthonormal basis we have
J(ϕ1, . . . , ϕr ) =M∑j=1
‖∑k>r
〈wj , ϕk〉ϕk‖2 =
=∑k>r
M∑j=1
〈wj , ϕk〉2 =∑k>r
ϕ>k WW>ϕk
(if): If WW>ϕk = λkϕk then J =∑
k>r λk is minimal for all r sinceλ1 ≥ · · · ≥ λn.
(only if): If {ϕk}Nk=1 is a POD basis and {ψ`}n`=1 eigenvectors of WW>, then
J =∑k>r
ϕ>k WW>ϕk =∑k>r
n∑`=1
λ` 〈ϕk , ψ`〉
is minimal for all r only if 〈ϕk , ψ`〉 = δk,`. But then ϕk is ev of WW>.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 13 / 39
Function approximation proper orthogonal decompositions
Computation of POD basis by SVD
Need to solve WW>ϕk = λkϕk
• Suppose n = rank(W ). Compute singular value decomposition
W = UΣV>
where• U =
(u1 · · · uN
)∈ RN×N , unitary
• V =(v1 · · · vM
)∈ RM×M unitary
•
Σ =
(Σ 00 0
); Σ = diag(σ1, . . . , σn)
• Then WW>uk = σ2kuk and λk = σ2
k .
ϕk = uk is POD basis
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 14 / 39
Function approximation proper orthogonal decompositions
Computation of POD basis by EVD
• data matrix W wide (N ≤ M)Compute eigenvalue decomposition
WW>uk = λkuk , ‖uk‖ = 1, k = 1 . . . , n
• data matrix W tall (N ≥ M)Compute eigenvalue decomposition
W>Wvk = λkvk , ‖vk‖ = 1, k = 1 . . . , n
Set uk = 1λ2kWvk
ϕk = uk is POD basis
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 15 / 39
Projection framework linear systems
Projection framework
Complex model
• Model {w = Aw + Bu
y = Cw + Du
• Variable projection
w ≈ Uwr
• Vector field projection
imV ⊥ [w − Aw − Bu]
Reduced order model
• Combined:
w ≈ Uwr
V>w = V>Aw + V>Bu
• Reduced order model{V>Uwr = Arwr + Bru
y = Crwr + Dru
whereAr = V>AU, Br = V>B,Cr = CU.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 16 / 39
Projection framework linear systems
Projection framework
Complex model
• Model {w = Aw + Bu
y = Cw + Du
• Variable projection
w ≈ Uwr
• Vector field projection
imV ⊥ [w − Aw − Bu]
Reduced order model
• Combined:
w ≈ Uwr
V>w = V>Aw + V>Bu
• Reduced order model{V>Uwr = Arwr + Bru
y = Crwr + Dru
whereAr = V>AU, Br = V>B,Cr = CU.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 16 / 39
Projection framework linear systems
Projection matrices U and V
Ar= V>
A U
U and V projection matrices that project on imU and imV
U>U = Ir , V>V = Ir
Reduction methods differ in selection of U and V .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 17 / 39
Projection framework linear systems
Projection matrices U and V
Ar= V>
A U
U and V projection matrices that project on imU and imV
U>U = Ir , V>V = Ir
Reduction methods differ in selection of U and V .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 17 / 39
Projection framework linear systems
How to select U and V ?
• Use gramiansbalancing type of algorithms
• Use eigenvectors of Amodal truncations/time scale separations
• Use U = VGalerkin projections
• Use time series data• measurement w(t), t = 1, . . . ,M• use SVD to pick relevant directions
• Use frequency domain data• measurement W (ω), ω1, . . . ωM
• use SVD to pick relevant directions
• Use Krylov spacesnot discussed in this course. . .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 18 / 39
Projection framework non-linear systems
Projection framework for non-linear systems
Complex nonlinear model
• Model{w(t) = f (w(t), u(t))
y(t) = g(w(t), u(t))
• Variable projection
w ≈ Uwr
• Vector field projection
imV ⊥ [w − f (w , u)]
Reduced order model
• Combined:
w ≈ Uwr
V>w = V>f (w , u)
• Reduced order model{V>Uwr = fr (wr , u)
y = gr (wr , u)
wherefr (wr , u) = V>f (Uwr , u),gr (wr , u) = g(Uwr , u).
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 19 / 39
Projection framework non-linear systems
Projection framework for non-linear systems
Complex nonlinear model
• Model{w(t) = f (w(t), u(t))
y(t) = g(w(t), u(t))
• Variable projection
w ≈ Uwr
• Vector field projection
imV ⊥ [w − f (w , u)]
Reduced order model
• Combined:
w ≈ Uwr
V>w = V>f (w , u)
• Reduced order model{V>Uwr = fr (wr , u)
y = gr (wr , u)
wherefr (wr , u) = V>f (Uwr , u),gr (wr , u) = g(Uwr , u).
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 19 / 39
Projection framework infinite dimensional systems
Projection framework for distributed systems
PDE or distributed system
• Model{∂w∂t = F (∂w∂x , . . . , )
y(x , t) = g(w(x , t), u(x , t))
• Variable projection
w(x , t) ≈ Uwr (x , t)
more difficult to define !!
• Vector field projection
imV ⊥[∂w
∂t− F (
∂w
∂x, . . . , )
]
Reduced order model
• Combined:
w(x , t) ≈ Uwr (x , t)
V>∂w
∂t= V>F (
∂w
∂x, . . . , )
• Reduced order model⟨v , ∂wr
∂t
⟩=⟨v ,F (∂wr
∂x , . . . , )⟩
wr = 〈u,w〉y(x , t) = g(wr , u)
where v ∈ imV , u ∈ imU and〈·, ·〉 is some inner product (seebelow)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 20 / 39
Projection framework infinite dimensional systems
Projection framework for distributed systems
PDE or distributed system
• Model{∂w∂t = F (∂w∂x , . . . , )
y(x , t) = g(w(x , t), u(x , t))
• Variable projection
w(x , t) ≈ Uwr (x , t)
more difficult to define !!
• Vector field projection
imV ⊥[∂w
∂t− F (
∂w
∂x, . . . , )
]
Reduced order model
• Combined:
w(x , t) ≈ Uwr (x , t)
V>∂w
∂t= V>F (
∂w
∂x, . . . , )
• Reduced order model⟨v , ∂wr
∂t
⟩=⟨v ,F (∂wr
∂x , . . . , )⟩
wr = 〈u,w〉y(x , t) = g(wr , u)
where v ∈ imV , u ∈ imU and〈·, ·〉 is some inner product (seebelow)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 20 / 39
Reduction of spatial-temporal systems a wave propagation example
Reduction of spatial-temporal systems
Example: wave propagation equationConsider solutions w(x , t) of partial differential equation
∂2w
∂t2− κ2∂
2w
∂x2= 0
where• space: x ∈ X = [0, 1];• time: t ∈ T = [0, 1]• initial and boundary conditions
w(x , 0) = w0(x), w(0, t) = 0, w(1, t) = 0,∂w
∂t(x , 0) = w1(x)
We wish to make a spectral decomposition of solution:
w(x , t) =∑∞
k=1 ak(t)φk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 21 / 39
Reduction of spatial-temporal systems a wave propagation example
Reduction of spatial-temporal systems
Example: wave propagation equationConsider solutions w(x , t) of partial differential equation
∂2w
∂t2− κ2∂
2w
∂x2= 0
where• space: x ∈ X = [0, 1];• time: t ∈ T = [0, 1]• initial and boundary conditions
w(x , 0) = w0(x), w(0, t) = 0, w(1, t) = 0,∂w
∂t(x , 0) = w1(x)
We wish to make a spectral decomposition of solution:
w(x , t) =∑∞
k=1 ak(t)φk(x)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 21 / 39
Reduction of spatial-temporal systems a wave propagation example
Wave propagation in a string
(click to animate)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25
Solution wave equation for κ = 3.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 22 / 39
Reduction of spatial-temporal systems a wave propagation example
Three questions
What basis functions ϕk(x) should we take ??
How do we obtain the coefficients ak(t) ??
How do we project to get a simple reduced order model ??
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 23 / 39
Reduction of spatial-temporal systems how to choose basis functions?
What basis functions ϕk should we take ??
To talk about
• normalized basis functions: we need structure of a normed vectorspace
• orthogonal basis functions: we need structure of an inner product.
For every t, the solution w(·, t) therefore needs to belong to an innerproduct space1: (
W, 〈·, ·〉)
that describes the relevant waves.
First need a relevant inner product – but what’s relevant here?
1Better even: a separable Hilbert space.Class 8 (TUE) Model Reduction – 2013 Siep Weiland 24 / 39
Reduction of spatial-temporal systems how to choose basis functions?
What basis functions ϕk should we take ??
To talk about
• normalized basis functions: we need structure of a normed vectorspace
• orthogonal basis functions: we need structure of an inner product.
For every t, the solution w(·, t) therefore needs to belong to an innerproduct space1: (
W, 〈·, ·〉)
that describes the relevant waves.
First need a relevant inner product – but what’s relevant here?
1Better even: a separable Hilbert space.Class 8 (TUE) Model Reduction – 2013 Siep Weiland 24 / 39
Reduction of spatial-temporal systems how to choose basis functions?
Choice of inner product
Possible inner product choices (1)
• Square integrable functions
W = L2([0, 1],R) = {f : [0, 1]→ R | ‖f ‖22 :=
∫ 1
0f 2(x)dx <∞}
with inner product
〈f , g〉 :=
∫ 1
0f (x)g(x)dx
• Basis This set has an orthonormal basis of Fourier modes:
1,√
2 cos(kπx),√
2 sin(kπx)
which implies for every f ∈ W the classical Fourier decomposition:
f (x) =∞∑k=1
ak√
2 cos(kπx) +∞∑`=1
b`√
2 sin(`πx) + cm
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 25 / 39
Reduction of spatial-temporal systems how to choose basis functions?
Choice of inner product
Possible inner product choices (1)
• Square integrable functions
W = L2([0, 1],R) = {f : [0, 1]→ R | ‖f ‖22 :=
∫ 1
0f 2(x)dx <∞}
with inner product
〈f , g〉 :=
∫ 1
0f (x)g(x)dx
• Basis This set has an orthonormal basis of Fourier modes:
1,√
2 cos(kπx),√
2 sin(kπx)
which implies for every f ∈ W the classical Fourier decomposition:
f (x) =∞∑k=1
ak√
2 cos(kπx) +∞∑`=1
b`√
2 sin(`πx) + cm
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 25 / 39
Reduction of spatial-temporal systems how to choose basis functions?
Choice of inner product
Possible inner product choices (2)• Square integrable functions
W = {f ∈ L2([0, 1],R) | f (0) = 0 = f (1)}
with same inner product.(This takes the initial conditions of the wave equation into account!)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 26 / 39
Reduction of spatial-temporal systems how to choose basis functions?
Choice of inner product
Possible inner product choices (2)• Square integrable functions
W = {f ∈ L2([0, 1],R) | f (0) = 0 = f (1)}
with same inner product.(This takes the initial conditions of the wave equation into account!)
• Basis: Has orthonormal basis of Fourier modes:
ϕk(x) =√
2 sin(kπx); x ∈ [0, 1]; k = 1, 2, · · ·
and implies the classical Fourier decomposition:
f (x) =∞∑k=1
ak√
2 sin(kπx)
with ak = 〈f , ϕk〉 =∫ 1
0
√2 sin(kπx)f (x)dx .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 26 / 39
Reduction of spatial-temporal systems how to choose basis functions?
Choice of inner product
Possible inner product choices (2)• Square integrable functions
W = {f ∈ L2([0, 1],R) | f (0) = 0 = f (1)}
with same inner product.(This takes the initial conditions of the wave equation into account!)
• Basis: Has orthonormal basis of Legendre polynomials:
ϕk(x) =8k
2 · k!
dk
dxk(x2 − x)k , k = 1, 2, · · ·
and implies a Legendre decomposition (not used in this course):
f (x) =∞∑k=1
akϕk(x)
with ak = 〈f , ϕk〉 =∫ 1
0 f (x)ϕk(x)dx .Class 8 (TUE) Model Reduction – 2013 Siep Weiland 26 / 39
Reduction of spatial-temporal systems how to choose basis functions?
Choice of inner product
Possible inner product choices (3)
• Discretization firstDiscretize X = [0, 1] in N disjoint intervals and assume that w(x , t) isonly interesting at samples xk = k∆x , ∆x = 1/N. Then
W = RN
with its usual (standard) inner product
〈f , g〉 = f >g .
• POD basisConsider M time samples w(x , t1), . . . , w(x , tM) of measured dataand store them in an N ×M snapshot matrix
Wsnap =
w(x1, t1) · · · w(x1, tM)...
...w(xN , t1) · · · w(xN , tM)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 27 / 39
Reduction of spatial-temporal systems how to choose basis functions?
Choice of inner product
Possible inner product choices (3)
• Discretization firstDiscretize X = [0, 1] in N disjoint intervals and assume that w(x , t) isonly interesting at samples xk = k∆x , ∆x = 1/N. Then
W = RN
with its usual (standard) inner product
〈f , g〉 = f >g .
• POD basisConsider M time samples w(x , t1), . . . , w(x , tM) of measured dataand store them in an N ×M snapshot matrix
Wsnap =
w(x1, t1) · · · w(x1, tM)...
...w(xN , t1) · · · w(xN , tM)
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 27 / 39
Reduction of spatial-temporal systems how to choose basis functions?
A data dependent basis
Now let Wsnap = UΣV> be an SVD of Wsnap with
U =(ϕ1 · · · ϕN
); ϕk ∈ RN .
Then vectors {ϕk , k = 1, . . . ,N} define an orthonormal basis of RN .
This is a POD basis!
Thus the time-averaged error
M∑`=1
‖w(x , t`)− wr (x , t`)‖2 =M∑`=1
N∑k=r+1
a2k(t`)
is minimal for all truncation levels r
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 28 / 39
Reduction of spatial-temporal systems how to choose basis functions?
A data dependent basis
Now let Wsnap = UΣV> be an SVD of Wsnap with
U =(ϕ1 · · · ϕN
); ϕk ∈ RN .
Then vectors {ϕk , k = 1, . . . ,N} define an orthonormal basis of RN .
This is a POD basis!
Thus the time-averaged error
M∑`=1
‖w(x , t`)− wr (x , t`)‖2 =M∑`=1
N∑k=r+1
a2k(t`)
is minimal for all truncation levels r
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 28 / 39
Reduction of spatial-temporal systems how to choose basis functions?
A data dependent basis
Now let Wsnap = UΣV> be an SVD of Wsnap with
U =(ϕ1 · · · ϕN
); ϕk ∈ RN .
Then vectors {ϕk , k = 1, . . . ,N} define an orthonormal basis of RN .
This is a POD basis!
Thus the time-averaged error
M∑`=1
‖w(x , t`)− wr (x , t`)‖2 =M∑`=1
N∑k=r+1
a2k(t`)
is minimal for all truncation levels r
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 28 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Next question: Get coefficients ak(t)
Once an orthonormal basis {ϕk(x), k = 1, 2, . . .} has been decided upon,the coefficients ak(t) in
w(x , t) =∞∑k=1
ak(t)ϕk(x)
satisfy
ak(t) = 〈w(x , t), ϕk(x)〉 =
∫ 1
0w(x , t)ϕk(x)dx
Nice, but useless as we do not know w(x , t). . .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 29 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Alternative: Substitute expansion in PDE ∂2w∂t2 = κ2 ∂2w
∂x2 . This yields:
∞∑k=1
ak(t)ϕk(x) = κ2∞∑k=1
ak(t)ϕk(x)
Then project both sides on ϕn, n = 1, 2, . . ., to infer that⟨ ∞∑k=1
ak(t)ϕk(x), ϕn(x)
⟩=
⟨κ2∞∑k=1
ak(t)ϕk(x), ϕn(x)
⟩
Using the orthonormality of the inner product, this yields:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 30 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Alternative: Substitute expansion in PDE ∂2w∂t2 = κ2 ∂2w
∂x2 . This yields:
∞∑k=1
ak(t)ϕk(x) = κ2∞∑k=1
ak(t)ϕk(x)
Then project both sides on ϕn, n = 1, 2, . . ., to infer that⟨ ∞∑k=1
ak(t)ϕk(x), ϕn(x)
⟩=
⟨κ2∞∑k=1
ak(t)ϕk(x), ϕn(x)
⟩
Using the orthonormality of the inner product, this yields:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 30 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Now have a close look at:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
• This is an ordinary differential equation in the time-varyingcoefficients an
• So we went from a PDE to an ODE!
• To solve it, we need to compute ϕk and 〈ϕk(x), ϕn(x)〉 for all k andall n.
not something to look forward to. . .
• We will be interested in a finite number of an’s only.
• To compute these, the infinite sum is replaced by a finite one too.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 31 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Now have a close look at:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
• This is an ordinary differential equation in the time-varyingcoefficients an
• So we went from a PDE to an ODE!
• To solve it, we need to compute ϕk and 〈ϕk(x), ϕn(x)〉 for all k andall n.
not something to look forward to. . .
• We will be interested in a finite number of an’s only.
• To compute these, the infinite sum is replaced by a finite one too.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 31 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Now have a close look at:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
• This is an ordinary differential equation in the time-varyingcoefficients an
• So we went from a PDE to an ODE!
• To solve it, we need to compute ϕk and 〈ϕk(x), ϕn(x)〉 for all k andall n.
not something to look forward to. . .
• We will be interested in a finite number of an’s only.
• To compute these, the infinite sum is replaced by a finite one too.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 31 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Now have a close look at:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
• This is an ordinary differential equation in the time-varyingcoefficients an
• So we went from a PDE to an ODE!
• To solve it, we need to compute ϕk and 〈ϕk(x), ϕn(x)〉 for all k andall n.
not something to look forward to. . .
• We will be interested in a finite number of an’s only.
• To compute these, the infinite sum is replaced by a finite one too.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 31 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Now have a close look at:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
• This is an ordinary differential equation in the time-varyingcoefficients an
• So we went from a PDE to an ODE!
• To solve it, we need to compute ϕk and 〈ϕk(x), ϕn(x)〉 for all k andall n.
not something to look forward to. . .
• We will be interested in a finite number of an’s only.
• To compute these, the infinite sum is replaced by a finite one too.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 31 / 39
Reduction of spatial-temporal systems how to derive coefficients?
How do we obtain the coefficients ak(t) ??
Now have a close look at:
an(t) = κ2∑∞
k=1 ak(t) 〈ϕk(x), ϕn(x)〉, n = 1, 2, . . .
• This is an ordinary differential equation in the time-varyingcoefficients an
• So we went from a PDE to an ODE!
• To solve it, we need to compute ϕk and 〈ϕk(x), ϕn(x)〉 for all k andall n.
not something to look forward to. . .
• We will be interested in a finite number of an’s only.
• To compute these, the infinite sum is replaced by a finite one too.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 31 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
Let’s try the Fourier basis!
• Fourier basis from “Possible choice 2”: ϕk(x) =√
2 sin(kπx).
• Then, ϕk(x) = −(k2π2)ϕk(x).
• Then, using orthonormality, the inner products simplify to
〈ϕk(x), ϕn(x)〉 =
{0 if k 6= `
−n2π2 if k = `
• Then, the coefficients satisfy a set of decoupled ODE’s:
an(t) = −κ2n2π2an(t); n = 1, 2, 3, · · ·
And we can explicitly solve these!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 32 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
Let’s try the Fourier basis!
• Fourier basis from “Possible choice 2”: ϕk(x) =√
2 sin(kπx).
• Then, ϕk(x) = −(k2π2)ϕk(x).
• Then, using orthonormality, the inner products simplify to
〈ϕk(x), ϕn(x)〉 =
{0 if k 6= `
−n2π2 if k = `
• Then, the coefficients satisfy a set of decoupled ODE’s:
an(t) = −κ2n2π2an(t); n = 1, 2, 3, · · ·
And we can explicitly solve these!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 32 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
Let’s try the Fourier basis!
• Fourier basis from “Possible choice 2”: ϕk(x) =√
2 sin(kπx).
• Then, ϕk(x) = −(k2π2)ϕk(x).
• Then, using orthonormality, the inner products simplify to
〈ϕk(x), ϕn(x)〉 =
{0 if k 6= `
−n2π2 if k = `
• Then, the coefficients satisfy a set of decoupled ODE’s:
an(t) = −κ2n2π2an(t); n = 1, 2, 3, · · ·
And we can explicitly solve these!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 32 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
Let’s try the Fourier basis!
• Fourier basis from “Possible choice 2”: ϕk(x) =√
2 sin(kπx).
• Then, ϕk(x) = −(k2π2)ϕk(x).
• Then, using orthonormality, the inner products simplify to
〈ϕk(x), ϕn(x)〉 =
{0 if k 6= `
−n2π2 if k = `
• Then, the coefficients satisfy a set of decoupled ODE’s:
an(t) = −κ2n2π2an(t); n = 1, 2, 3, · · ·
And we can explicitly solve these!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 32 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
Let’s try the Fourier basis!
• Fourier basis from “Possible choice 2”: ϕk(x) =√
2 sin(kπx).
• Then, ϕk(x) = −(k2π2)ϕk(x).
• Then, using orthonormality, the inner products simplify to
〈ϕk(x), ϕn(x)〉 =
{0 if k 6= `
−n2π2 if k = `
• Then, the coefficients satisfy a set of decoupled ODE’s:
an(t) = −κ2n2π2an(t); n = 1, 2, 3, · · ·
And we can explicitly solve these!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 32 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
Indeed:an(t) = −κ2n2π2an(t); n = 1, 2, 3, · · ·
is a second order ordinary diff. eqn. which has as its solution
an(t) = An cos(ωnt) + Bn sin(ωnt); ωn = κnπ
where constants An, Bn follow from initial conditions:
An = an(0) =
∫ 1
0w0(x)ϕn(x) dx =
√2
∫ 1
0w0(x) sin(nπx) dx
Bn =1
ωnan(0) =
√2
ωn
∫ 1
0w1(x) sin(nπx) dx
This gives the complete solution
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 33 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
For equation lovers, the explicit expression for the solution is:
w(x , t) =∞∑k=1
ak(t)ϕk(x) =
∞∑k=1
[√2
∫ 1
0w0(x) sin(kπx) dx cos(ωkt)+
+
√2
ωk
∫ 1
0w1(x) sin(kπx) dx
]√
2 sin(kπx)
which satisfies the boundary conditions
w(0, t) = w(1, t) = 0; t ≥ 0
(by construction of the basis) and
w(x , 0) = w0(x);∂w
∂t(x , 0) = w1(x).
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 34 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
How do we get a simple reduced order model ??
The truncated expansion
wr (x , t) =∑r
k=1 ak(t)φk(x)
is the approximate solution of order r , and requires construction of ak(t)for 1 ≤ k ≤ r only.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 35 / 39
Reduction of spatial-temporal systems results on wave propagation (Fourier)
Results on wave propagation (Fourier)
(click to animate) Simulation with ϕk(x) =√
2 sin(kπx):
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25 original degree=100
approximation degree=5
Solutions wave equation
position
Solution wave propagation with κ = 3, blue: exact solution, red: r = 5thorder Fourier approximation
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 36 / 39
Reduction of spatial-temporal systems results on wave propagation (POD)
Results on wave propagation (POD)
Let’s try the POD basis!
• POD basis from “possible choice 3”: ϕk(x) on discretized grid.
• Hence, ϕk(x) needs to be approximated numerically on the grid by
ϕk,xx(xj) :=ϕk(xj−1)− 2ϕk(xj) + ϕk(xj+1)
(δx)2δx grid size
• Compute inner products 〈ϕk,xx , ϕn〉, 1 ≤ k ≤ r
• Solve coupled ODE’s:
an(t) = κ2r∑
k=1
ak(t) 〈ϕk,xx , ϕn〉 , n = 1, 2, . . . , r .
• This is of the form a = Aa with a = col(a1, . . . , an).
Again, we can explicitly solve this (in Matlab)!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 37 / 39
Reduction of spatial-temporal systems results on wave propagation (POD)
Results on wave propagation (POD)
Let’s try the POD basis!
• POD basis from “possible choice 3”: ϕk(x) on discretized grid.
• Hence, ϕk(x) needs to be approximated numerically on the grid by
ϕk,xx(xj) :=ϕk(xj−1)− 2ϕk(xj) + ϕk(xj+1)
(δx)2δx grid size
• Compute inner products 〈ϕk,xx , ϕn〉, 1 ≤ k ≤ r
• Solve coupled ODE’s:
an(t) = κ2r∑
k=1
ak(t) 〈ϕk,xx , ϕn〉 , n = 1, 2, . . . , r .
• This is of the form a = Aa with a = col(a1, . . . , an).
Again, we can explicitly solve this (in Matlab)!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 37 / 39
Reduction of spatial-temporal systems results on wave propagation (POD)
Results on wave propagation (POD)
Let’s try the POD basis!
• POD basis from “possible choice 3”: ϕk(x) on discretized grid.
• Hence, ϕk(x) needs to be approximated numerically on the grid by
ϕk,xx(xj) :=ϕk(xj−1)− 2ϕk(xj) + ϕk(xj+1)
(δx)2δx grid size
• Compute inner products 〈ϕk,xx , ϕn〉, 1 ≤ k ≤ r
• Solve coupled ODE’s:
an(t) = κ2r∑
k=1
ak(t) 〈ϕk,xx , ϕn〉 , n = 1, 2, . . . , r .
• This is of the form a = Aa with a = col(a1, . . . , an).
Again, we can explicitly solve this (in Matlab)!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 37 / 39
Reduction of spatial-temporal systems results on wave propagation (POD)
Results on wave propagation (POD)
Let’s try the POD basis!
• POD basis from “possible choice 3”: ϕk(x) on discretized grid.
• Hence, ϕk(x) needs to be approximated numerically on the grid by
ϕk,xx(xj) :=ϕk(xj−1)− 2ϕk(xj) + ϕk(xj+1)
(δx)2δx grid size
• Compute inner products 〈ϕk,xx , ϕn〉, 1 ≤ k ≤ r
• Solve coupled ODE’s:
an(t) = κ2r∑
k=1
ak(t) 〈ϕk,xx , ϕn〉 , n = 1, 2, . . . , r .
• This is of the form a = Aa with a = col(a1, . . . , an).
Again, we can explicitly solve this (in Matlab)!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 37 / 39
Reduction of spatial-temporal systems results on wave propagation (POD)
Results on wave propagation (POD)
Let’s try the POD basis!
• POD basis from “possible choice 3”: ϕk(x) on discretized grid.
• Hence, ϕk(x) needs to be approximated numerically on the grid by
ϕk,xx(xj) :=ϕk(xj−1)− 2ϕk(xj) + ϕk(xj+1)
(δx)2δx grid size
• Compute inner products 〈ϕk,xx , ϕn〉, 1 ≤ k ≤ r
• Solve coupled ODE’s:
an(t) = κ2r∑
k=1
ak(t) 〈ϕk,xx , ϕn〉 , n = 1, 2, . . . , r .
• This is of the form a = Aa with a = col(a1, . . . , an).
Again, we can explicitly solve this (in Matlab)!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 37 / 39
Reduction of spatial-temporal systems results on wave propagation (POD)
Results on wave propagation (POD)
Let’s try the POD basis!
• POD basis from “possible choice 3”: ϕk(x) on discretized grid.
• Hence, ϕk(x) needs to be approximated numerically on the grid by
ϕk,xx(xj) :=ϕk(xj−1)− 2ϕk(xj) + ϕk(xj+1)
(δx)2δx grid size
• Compute inner products 〈ϕk,xx , ϕn〉, 1 ≤ k ≤ r
• Solve coupled ODE’s:
an(t) = κ2r∑
k=1
ak(t) 〈ϕk,xx , ϕn〉 , n = 1, 2, . . . , r .
• This is of the form a = Aa with a = col(a1, . . . , an).
Again, we can explicitly solve this (in Matlab)!
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 37 / 39
Reduction of spatial-temporal systems results on wave propagation (POD)
Results on wave propagation (POD)
Simulations: (click to animate)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25original degree=100
Fourier approx. degree=5
POD approx. degree=5
Solutions wave equation
position
Solution wave equation for κ = 3, red: 5th order Fourier approximation,green: 5th order POD approximation.
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 38 / 39
Summary
Summary
• Most model reduction techniques require projection of the statevariable and state equations to define low order models
• This idea generalizes to nonlinear systems and systems described byPDE’s
• For systems described by PDE’s we consider spectral expansions
w(x , t) =∞∑k=1
ak(t)ϕk(x)
and their truncations as (approximate) solutions.• Functions ϕk are selected to form an orthonormal basis of some
Hilbert space/inner product space.• Many choices are possible, including data-based functions
(POD-method)• Once basis functions are fixed, coefficients ak satisfy ODE’s that we
can solve.• Low order POD approximations work well on wave propagation
example.
Previous class Next class
Class 8 (TUE) Model Reduction – 2013 Siep Weiland 39 / 39