Nonlinear Model Order Reduction using POD/DEIM for Optimal...

Introduction POD-DEIM algorithm Optimal Control Burgers' equation Outlook Literature

Nonlinear Model Order Reduction usingPOD/DEIM for Optimal Control of

Burgers' Equation

Manuel M. Baumann

July 15, 2013


Outline

1 What is Model Order Reduction (MOR) ?

2 Model Order Reduction using POD-DEIMProper Orthogonal Decomposition (POD)Discrete Empirical Interpolation Method (DEIM)Application: MOR for Burgers' equation

3 PDE-constrained OptimizationSecond-order optimization algorithmFirst-order methods: BFGS and SPG

4 Optimal Control for the reduced-order Burgers' equation

5 Summary and future research

1 / 34


Nonlinear dynamical systems

Consider the nonlinear dynamical system

y(t) = Ay(t) + F(t, y(t)), y(t) ∈ RN

y(0) = y0(1)

arises in many applications, e.g. mechanical systems, �uiddynamics, neuron modeling, ...

the matrix A represents the linear dynamical behavior and thefunction F represents nonlinear dynamics

often large dimension of (1) leads to huge computational work

2 / 34


The idea of model order reduction

Approximate the state via

y(t) ≈ U`y(t), U` ∈ RN×`, y ∈ R`,

where the matrix U` has orthonormal columns,the so-called principal components of y, and`� N.

Galerkin projection of the original full-order system leads to areduced system of ` equations:

UT`

[U` ˙y − AU`y − F(t,U`y)

]= 0

⇒ ˙y = UT` AU`︸︷︷︸=:A

y + UT` F(t,U`y)

3 / 34


Two questions are left...

Considering the reduced model

˙y(t) = Ay(t) + UT` F(t,U`y(t)), y(t) ∈ R`

two questions are left:

1 How to obtain the matrix U` of principal components ?

2 Note that U`y(t) ∈ RN is still large. How do we evaluateF(t,U`y(t)) e�ciently ?

4 / 34


Proper Orthogonal Decomposition (POD)

The Proper Orthogonal Decomposition (POD)

During the numerical simulation, build up the snapshot matrix

Y := [y(t1), ..., y(tns )] ∈ RN×ns ,

with ns being the number of snapshots.

Perform a Singular Value Decomposition (SVD)

Y = UΣV T

and let U` := U(:,1:l) consist of those left singular vectors of Ythat correspond to the ` largest singular values in Σ.

5 / 34


Discrete Empirical Interpolation Method (DEIM)

The Discrete Empirical Interpolation Method (DEIM)

Consider the nonlinearity

N := UT`︸︷︷︸

`×N

F(t,U`y(t))︸︷︷︸N×1

The approximation

F ≈W c, W ∈ RN×m, c ∈ Rm

is over-determined. Therefore, �nd projector P such that:

PTF = (PTW )c ⇒ F ≈W c = W (PTW )−1PTF⇒ N ≈ UT

` W (PTW )︸︷︷︸m×m

−1PTF(t,U`~y(t))

6 / 34



The Discrete Empirical Interpolation Method (DEIM)

Consider the nonlinearity

N := UT`︸︷︷︸

`×N

F(t,U`y(t))︸︷︷︸N×1

The approximation

F ≈W c, W ∈ RN×m, c ∈ Rm

is over-determined. Therefore, �nd projector P such that:

PTF = (PTW )c ⇒ F ≈W c = W (PTW )−1PTF⇒ N ≈ UT

` W (PTW )︸︷︷︸m×m

−1PTF(t,U`~y(t))

7 / 34



Algorithm 1 The DEIM algorithm [Chaturantabut, Sorensen, 2010]

1: INPUT: {wi}mi=1 ⊂ RN linear independent2: OUTPUT: ~℘ = [℘1, ..., ℘m]T ∈ Rm, P ∈ RN×m

3: [|ρ|, ℘1] = max{|w1|}4: W = [w1],P = [e℘1 ], ~℘ = [℘1]5: for i = 2 to m do

6: Solve (PTW )c = PTwi for c7: r = wi −W c

8: [|ρ|, ℘i ] = max{|r|}

9: W ← [W wi ],P ← [P e℘i ], ~℘←[~℘℘i

]10: end for

8 / 34



The product PTF is a selection of entries

Let m = 3. Suppose the DEIM-algorithm has chosen indices℘1, ..., ℘m such that:

Assuming that F(·) acts pointwise, we obtain:

N ≈ UT` W (PTW )−1PTF(t,U`~y(t))

= UT` W (PTW )−1︸︷︷︸

`×m

F(t,PTU`~y(t))︸︷︷︸m×1

9 / 34


Application: MOR for Burgers' equation

The nonlinear 1D Burgers' model

yt +

(1

2y2 − νyx

)x = f , (x , t) ∈ (0, L)× (0,T ),

y(t, 0) = y(t, L) = 0, t ∈ (0,T ),

y(0, x) = y0(x), x ∈ (0, L).

1 FEM-discretization in space leads to:

M y(t) = −12By2(t)− νCy(t) + f, t > 0

y(0) = y0

2 Time integration via implicit Euler + Newton's method

10 / 34



POD-DEIM for Burgers' equation

Suppose, Φ` is an M-orthogonal POD basis.

The POD reduced Burgers' equation

=I`︷︸︸︷ΦT

` MΦ` ~y(t) = −1

2ΦT

` B(Φ`~y(t))2 − νΦT

` CΦ`~y(t)

⇒ ~y(t) = −1

2B`(Φ`~y(t))2 − νC`~y(t)

Next, obtain W via a truncated SVD of [y2(t1), ..., y2(tns )] and apply DEIM to the

columns of W .

The POD-DEIM reduced Burgers' equation

~y(t) = −1

2B(F~y(t))2 − νC~y(t),

with B = ΦT

` BW (PTW )−1 ∈ R`×m, F = PTΦ` ∈ Rm×`, and C = C` ∈ R`×`.

11 / 34



0

0.5

1

0

0.5

10

1

t

ℓ = 3,m = 13

x

yℓ(t,x

)

12 / 34



0

0.5

1

0

0.5

10

1

t

ℓ = 5,m = 13

x

yℓ(t,x

)

13 / 34



0

0.5

1

0

0.5

10

1

t

ℓ = 7,m = 13

x

yℓ(t,x

)

14 / 34



0

0.5

1

0

0.5

10

1

t

ℓ = 9,m = 13

x

yℓ(t,x

)

15 / 34



0

0.5

1

0

0.5

10

1

t

ℓ = 11,m = 13

x

yℓ(t,x

)

16 / 34



Computational Speedup [1]

Conclusion: High accuracy of the POD-DEIM reduced model.

But is it also faster?

POD POD−DEIM0

0.5

1

1.5

2

Spe

edup

ν = 0.01, N = 80

POD POD−DEIM0

1

2

3

4

5

Spe

edup

ν = 0.001, N = 200

POD POD−DEIM0

10

20

30

40

50

60

Spe

edup

ν = 0.0001, N = 800

Spatial discretization of the full model depends on viscosityparameter ν

choose `,m such that relative L2-error in O(10−4)

17 / 34




For a �xed ν = 0.01, we could show the independence of thePOD-DEIM reduced model of the full-order dimension N.

80 500 1.000 1.500 2.000 2.500 3.0000.02

0.04

0.06

0.08

0.1

0.12

0.14

N

t PD

Eso

l [s]

ν = 0.01

PODPOD−DEIM

80 500 1.000 1.500 2.000 2.500 3.0000

200

400

600

800

1000

N

SP(2

)

ν = 0.01

PODPOD−DEIM

Computation time for solving the POD-DEIM reducedBurgers' equation is almost constant (right)

POD-DEIM almost 4 times faster than pure POD (left)

18 / 34


PDE-constrained optimization

Minimize

minuJ (y(u), u),

where y is the solution to a nonlinear, possibly time-dependentpartial di�erential equation,

c(y , u) = 0.

J is called objective function,

in order to evaluate J , we need to solve c(y , u) = 0 for y(u),

solve with algorithms for unconstrained minimization problems.

19 / 34


Second-order optimization algorithm

A Newton-type optimization algorithm

Minimize J (y(u), u) in u using information of the �rst and secondderivative.

20 / 34



Gradient computation via adjoints

Consider the Lagrangian function

L(y , u, λ) = J (y , u) + λT c(y , u)

and impose the zero-gradient condition ∇yL(y , u, λ) = 0.

We derive the adjoint equation:

cy (y(u), u)Tλ = −∇yJ (y(u), u)

Algorithm 2 Computing ∇J (u) via adjoints [Heinkenschloss, 2008]

1: For a given control u, solve c(y , u) = 0 for the state y(u)2: Solve the adjoint equation cy (y(u), u)Tλ = −∇yJ (y(u), u) for λ(u)

3: Compute ∇J (u) = ∇uJ (y(u), u) + cu(y(u), u)Tλ(u)

21 / 34



Gradient computation via adjoints

Consider the Lagrangian function

L(y , u, λ) = J (y , u) + λT c(y , u)

and impose the zero-gradient condition ∇yL(y , u, λ) = 0.

We derive the adjoint equation:

cy (y(u), u)Tλ = −∇yJ (y(u), u)

Algorithm 3 Computing ∇J (u) via adjoints [Heinkenschloss, 2008]

1: For a given control u, solve c(y , u) = 0 for the state y(u)2: Solve the adjoint equation cy (y(u), u)Tλ = −∇yJ (y(u), u) for λ(u)

3: Compute ∇J (u) = ∇uJ (y(u), u) + cu(y(u), u)Tλ(u)

21 / 34


First-order methods: BFGS and SPG

First-order optimization algorithms

Instead of solving

∇2J (yk , uk)sk = −∇J (yk , uk),

�rst-order methods approximate the Hessian via Hk and solve

Hksk = −∇J (yk , uk).

We have used Matlab implementations of the BFGS and theSPG method,

Evaluation of J and gradient computation as seen before,

SPG easily allows to include bounds on the control, i.e.ulower ≤ u(t, x) ≤ uupper which is used in many applications

22 / 34


Optimal Control problem for Burgers' equation

Minimize

minu

1

2

∫ T

0

∫ L

0

[y(t, x)− z(t, x)]2 + ωu2(t, x) dx dt,

where y is a solution to the nonlinear Burgers' equation

yt +

(1

2y2 − νyx

)x

= f + u, (x , t) ∈ (0, L)× (0,T ),

y(t, 0) = y(t, L) = 0, t ∈ (0,T ),

y(0, x) = y0(x), x ∈ (0, L).

u is the control that determines y

z is the desired state

23 / 34


Control goal

We want to control the solution of Burgers' equation in such a waythat it stays in the desired state z(t, ·) = y0,∀t:

0

0.5

1

00.5

10

0.5

1

1.5

t

uncontrolled state

x

y(t,x

)

0

0.5

1

00.5

10

0.5

1

1.5

t

desired state

x

z(t,x

)

Figure: Uncontrolled (u ≡ 0) and desired state for ν = 0.01.

24 / 34


Numerical treatment

1 Discretize the objective function and Burgers' equation in timeand space

2 Apply adjoints in order to compute gradient and Hessian

3 Apply �rst-order or second-order optimization algorithm

4 Explore the usage of a POD-DEIM reduced model

25 / 34


Numerical tests

Newton-type method for the full-order Burgers' model:

0

1

0

1−0,5

1

1.5

t

x

y(t,x

)

k = 0 (uncontrolled)

0

1

0

1−0,5

1

1,5

t

xy(

t,x)

k = 1

0

1

0

1−0,5

1

1,5

t

x

y(t,x

)

k = 2

0

1

0

1−0,5

1

1,5

t

x

y(t,x

)

k = 3

0

1

0

1−0.5

1

1.5

t

x

y(t,x

)

k = 4

0

1

0

1−0.5

1

1.5

t

x

y(t,x

)

k = 5

26 / 34


The corresponding optimal control at each iteration:

0

1

0

1−6

0

7

u(t,x

)

x

t

k = 0 (initial)

0

1

0

1−6

0

7

t

x

u(t,x

)

k = 1

0

1

0

1−6

0

7

t

x

u(t,x

)

k = 2

0

1

0

1−6

0

7

t

x

u(t,x

)

k = 3

0

1

0

1−6

0

7

t

x

u(t,x

)

k = 4

0

1

0

1−6

0

7

t

x

u(t,x

)

k = 5

27 / 34


We propose the following algorithm for POD-DEIM reducedoptimal control :

28 / 34


Final state and control of the POD-DEIM reduced optimal controlproblem:

0

1

0

1−0.5

0

1

t

ν = 0.01

x

Φℓy(t)

` = m = 7

0

1

0

1−0.5

0

1

t

ν = 0.01

x

Φℓy(t)

` = m = 15

0

1

0

1−9

0

5

t

ν = 0.01

x

u(t,x

)

` = m = 7

0

1

0

1−9

0

5

t

ν = 0.01

x

u(t,x

)

` = m = 15

29 / 34



Reduced optimal control using the Newton-type method:

0

20

40

60

80

100

Spe

edup

PODPOD−DEIM

ν = 0.01 ν = 0.001 ν = 0.0001

101.3

79.0

3.7 4.41.35 1.9

at �nal state: relative L2-error in O(10−2)

comparable value of the objective function at convergence

use same stopping criteria for full-order and reduced model

30 / 34



Some other results.

For ν = 0.0001, low-dimensional controlleads to a speedup of ∼ 20 for all threeoptimization methods.

0

1

01

−30

0

15

t

discrete control

u(t,x

)

x

SPG allows a bounded control−2 ≤ u(t, x) ≤ 2. For ν = 0.0001, weobtained a speedup of 3.6 for POD and8.8 for POD-DEIM.

0

1

01

−2

0

2

t

bounded control

u(t,x

)

x

31 / 34


Concluding Remarks

What I learnt:

The accuracy of the reduced Burgers' model is of the sameorder when POD is extended by DEIM.

Optimal Control of Burgers' equation using POD-DEIM leadsto a speedup of ∼ 100 for small ν.

For the reduced model, all derivatives need to be computed interms of the reduced variable. This can be quite hard inpractice.

32 / 34


Future Research

What I still want to learn:

Use the POD basis Φ` also for dimension reduction of thecontrol, i.e.

u(t) ≈ Φ`~u(t) =∑i=1

ϕi ui (t)

Extend Burgers' model to 2D/3D

More sophisticated choice of reduced dimensions ` and m

33 / 34


This Master project was supervised byMarielba Rojas and Martin van Gijzen.

Thank you for your attention!

Are there any questions or remarks?

https://github.com/ManuelMBaumann/MasterThesis

34 / 34

https://github.com/ManuelMBaumann/MasterThesis


Further information can be found in...

S. Chaturantabut and D. SorensenNonlinear Model Reduction via Discrete Empirical Interpolation.SIAM Journal of Scienti�c Computing, 2010.

M. HeinkenschlossNumerical solution of implicitly constrained optimization problems.Technical report, Department of Computational and AppliedMathematics, Rice University, 2008.

K. Kunisch and S. VolkweinControl of the Burgers Equation by a Reduced-Order Approach

Using Proper Orthogonal Decomposition.Journal of Optimization Theory and Applications, 1999.

Nonlinear Model Order Reduction using POD/DEIM for Optimal...

Documents

Transcript of Nonlinear Model Order Reduction using POD/DEIM for Optimal...