(Ref: Introduction to Model Order Reduction,Wil Schilders ...inavon/pubs/LectureSC.pdf · "On Lines...

Lectures on Model Reduction (Ref: Introduction to Model Order Reduction,Wil Schilders in Model Order Reduction:Theory, Research Aspects and Applications in computational science and engineering. Springer,2008)

• Method of snapshots and POD algorithm Motivation The ever increasing demand for realistic simulations of complex

products places a heavy burden on the shoulders of mathematicians

and, more generally, researchers working in the area of

Realistic simulations imply that the errors of the virtual models

should be small, and that different aspects of the product must be

taken into account.

The former implies that care must be taken in the numerical

treatment and that, for example, a relatively fine adaptively

determined mesh is necessary in the simulations.

The latter explains the trend in coupled simulations, for example

combined mechanical and thermal behaviour, or combined

mechanical and electromagnetic behaviour.

An important factor in enabling the complex simulations carried

out today is the increase in computational power. Computers and

chips are getting faster, Moore’s law predicting that the speed will

double every 18 months (see Figure 2).

This increase in computational power appears to go hand-in-

hand with developments in numerical algorithms.

Iterative solution techniques for linear systems are mainly

responsible for this speed-up in algorithms, as is shown in

Figure 3.

Important contributions in this area are the conjugate

gradient method , preconditioned conjugate gradient

methods (ICCG, biCGstab) and multigrid methods.

Dynamical Systems

To place model reduction in a mathematical context, we

need to realize that many models developed in

computational science consist of a system of partial and/or

ordinary differential equations, supplemented with boundary

conditions.

Important examples

• The Navier-Stokes equations in computational fluid

dynamics (CFD), and

• the Maxwell equations in electromagnetics (EM).

When partial differential equations

are used to describe the behaviour, one often encounters

the situation that the independent variables are space and

time.

Thus, after (semi-) discretising in space, a system of

ordinary differential equations is obtained in time.

Therefore, we limit the discussion to ODE’s and consider

the following explicit finite-dimensional dynamical system

(following book of Antoulas):

Here, u is the input of the system, y the output, and x the so-called

state variable.

The dynamical system can thus be viewed as an input-output

system, as displayed in Figure 7.

The complexity of the system is characterized by the number of its

state variables, i.e. the dimension n of the state space vector x.

It should be noted that similar dynamical systems can also be

defined in terms of differential algebraic equations,

in which case the first set of equations in (1) is replaced by

Model order reduction can now be viewed as the task of

reducing the dimension of the state space vector, while

preserving the character of the input‐output relations.

In other words, we should find a dynamical system of the form

where the dimension of x is much smaller than n.

In order to provide a good approximation of the original input-

ou s should be satisfied: tput system, a number of condition

• the approximation error is small,

• preservation of properties of the original system, such as

stability and passivity

• the reduction procedure should be computationally efficient.

A special case is encountered if the functions f and g are

linear, in which case the system reads

Here, the matrices A,B,C,D can be time-dependent, in which case

we have a linear time-varying (LTV) system, or time-independent,

in which case we speak about a linear time-invariant (LTI) system.

For linear dynamical systems, model order reduction is equivalent

to reducing the matrix A, but retaining the number of columns of B

and C.

Approximation by Projection

Although there are ways of approximating input-output systems,

there is a unifying feature of the approximation methods that is

worthwhile discussing briefly: projection.

Methods based on this concept truncate the solution of the original

system in an appropriate basis.

To illustrate the concept, consider a basis transformation T that

maps the original n-dimensional state space vector x into a vector

that we denote by

where x is k-dimensional.

The basis transformation T can then be written as

and its inverse as

is an oblique projection along the kernel of W ∗ onto the

k-dimensional subspace that is spanned by the columns of the

matrix V .

If we substitute the projection into the dynamical system (1), the

first part of the set of equations obtained is

Note that this is an exact expression.

The approximation occurs when we would delete the terms

involving x , in which case we obtain the reduced system

For this to produce a good approximation to the original system,

the neglected term T1 x must be sufficiently small. This has

implications for the choice of the projection Π. In the following

sections, various ways of constructing this projection are discussed

POD model reduction methods application to

geosciences and 4-D VAR data assimilation

1. Introduction Interest in reduced cost of implementation of 4-D VAR data

assimilation in the geosciences motivated research efforts aimed

towards reducing dimension of control space without significantly

compromising quality of the final solution.

IntroductionProper Orthogonal Decomposition

POD Model Reduction of Large ScaleGeophysical Models

Ionel M. Navon

Department of Scientific ComputingFlorida State University

Tallahassee, Florida

I.M. Navon


Outline

1 IntroductionIntroductionMotivation

2 Proper Orthogonal DecompositionProper Orthogonal DecompositionConstruction of POD basisPOD and SnapshotsPOD and SVD

I.M. Navon


IntroductionMotivation

Introduction

• Karl Pearson(1901) paper:"On Lines and Planes of Closest Fit to A System of Points" seems to beat origin of a general approach to dimensionality reduction. Alsodeveloped independently by Hotelling(1933).

• Idea of principal component analysis is to reduce dimensionality of adata set in which there are a large number of interrelated variables,while retaining as much as possible of the variation present in the dataset.

• The reduction is achieved by transforming to a new set of variables-theprincipal components which are uncorrelated, and which are ordered.So that first, few retain most of the variation present in data set.

I.M. Navon


IntroductionMotivation

Motivation

• In many fields of science and engineering such as fluid dynamics,geophysical fluid dynamics, large-scale systems used to be simulated,optimized and controlled. Since they are solved using discretizations ofnonlinear PDEs, they yield high dimensional problems.

• The number of degrees of freedom for simulation kept pace withincreasing computing power.

• In particular optimal design, control PDE constrained optimization, theproblems become too large to be tackled with standard techniques.

• Hence need for model reduction techniques to reduce computationalcost and storage requirements.

I.M. Navon


Proper Orthogonal DecompositionConstruction of POD basisPOD and SnapshotsPOD and SVD

Proper Orthogonal Decomposition

• It is a projection method where dynamical system is projected onto asubspace of original phase space.

• In combination with Galerkin projection provides powerful tool toderive surrogate models for high dimensional dynamical systems.

• Requires only standard matrix computation despite its application tononlinear problems.

• Solves: Find a subspace approximating a given set of data in an optimalleast squares sense.

• Uses data points given by sampling from experiments or by trajectoriesof the physical system extracted from simulations of the full nonlinearmodel.

I.M. Navon



Construction of POD basis

Consider dynamical system described by PDEs. We restrict ourselves tofinite dimensions and start with vector space V and a given data in V.SetV = Rn and set of sampled data y = {y1(t), ..., ym(t)} given by trajectories

yi(t) ∈ Rn, i = 1, . . . , t ∈ [0, T]

Use principal component analysis of data to find subspace Vd ⊂ Vapproximating data in some least squares sense, i.e. we seek orthogonalprojection:

∏d : V −→ Vd of fixed rank d that minimizes total least-squares

distance

‖y−∏

d

y‖2 =m∑

i=1

∫ T

0‖yi(t)−

∏

d

yi(t)‖2 dt

I.M. Navon




Solution of this problem relies on introduction of the correlationmatrixK ∈ Rn×n defined by

K =m∑

i=1

∫ T

0yi(t)yi(t)∗ dt

where star(∗) stands for transpose of vector or matrix.K is by definition symmetric positive definite matrix with real non-negativeeigenvalues λ1 ≥ λ2 ≥ · · ·λn ≥ 0.Let uj denote corresponding eigenvectors

Kuj = λuj, j = 1, . . . , n

I.M. Navon




Due to the special structure of K we can choose them in fact as anorthogonal basis of V.Thus for POD an optimal subspace Vd of dimension d representing the datais given by

Vd = span{u1, u2, . . . , ud}

Vectors uj are called POD modes.

I.M. Navon




TheoremLet K be correlation matrix of data defined above and letλ1 ≥ λ2 ≥ · · ·λn ≥ 0 be the ordered eigenvalues of K. Then it holds

minVd‖y−

∏

d

y‖ =m∑

j=m−d+1

λj

where min is taken over all subspaces Vd of dimension d.Further, the optimal orthogonal projection

∏

d

: V −→ Vd

with∏

d

∏∗d = I is given by

∏

d

=d∑

j=1

uju∗j

I.M. Navon




Each data vector yi(t) ∈ V can be written as

yi(t) =m∑

j=1

yij(t)uj

where yij =< yi(t), uj >.It then holds that

∏

d

yi(t) =d∑

j=1

uju∗j (n∑

l=1

yil(t)ul) =d∑

j=1

yij(t)uj

since < ui, uj >= δij

I.M. Navon



Choosing the dimension

In terms of a dynamical system we look at eigenvalues of K. Largeeigenvalues correspond to main characteristics of the system, while smalleigenvalues provide only small perturbations of overall dynamics.We aim atchoosing d small enough while the relative information content

I(d) =

∑dj=1 λj∑nj=1 λj

i.e.

d = argmin{I(d) : I(d) ≥ p100

}

I.M. Navon



Choosing the dimension

• POD modes are not constructed to be the modes approximating thedynamics generating the given data set.

Consider new techniques

• POD coupled with balanced-truncation• Dual weighted (goal-oriented POD)• POD works naturally with finite-element discretization

I.M. Navon



POD and Snapshots

In practical applications we have dimensions of 106 -1010. Calculation ofPOD modes requires solution of a full matrix K ∈ Rn×n, i.e. maybeinfeasible.Sirovich(1987) proposed the method of snapshots.Instead of solving eigensystem for K ∈ Rn×n one considers only a matrix inRm×m, where m is number of snapshots considered. Snapshots areconstructed from trajectories of dynamical system by evaluating them atdiscrete time instances t1, t2, . . . , tm ∈ [0, T]. We here consider a newcorrelation matrix K defined by

K =m∑

i=1

y(ti)y(ti)∗

I.M. Navon



POD and Snapshots

How many snapshots should one choose?Consider matrix y = (y(t1), . . . , y(tm)) ∈ Rn×n i.e. columns of the snapshots,then y∗y ∈ Rm×m and solving eigenvalue problem

y∗yvj = λvj, j = 1, . . . , m, vj ∈ Rm

Choosing orthogonal basis of eigenvectorsv1, . . . , vm POD modes are givenby

uj =1√λj

yvj, j = 1, . . . , m

I.M. Navon



POD and SVD

Approximating POD basis should contain as much information as possible.We can write the problem of approximating the snapshot vectors yi by asingular vector u as constrained optimization problem

maxm∑

i=1

| < yj, u > |2s.t.|u| = 1

Using Lagrangian formalism a necessary condition for this problem is givenby the eigenvalue problem

yy∗u = σ2u

Singular value analysis yields that u1 solves this eigenvalue problem withfunctional value σ2

1 .

I.M. Navon



POD and SVD

Iterate this procedure and derive that ui, i = 1, . . . , d solves

maxm∑

j=1

| < yi, u > |2 s.t.|u| = 1 and < ui, uj >= 0, j = 1, . . . , i−1 >

and the value of the functional is given by σ2i . By construction it is clear that

for every d ≤ m the approximation of the columns y = (y1, y2, . . . , ym) bythe first d singular vectors {ui}d

i=1 is optimal in the least − squares senseamong all rank d approximations of the columns of y.

I.M. Navon



POD and SVD

This allows practical determination of a POD basis.If m ≤ n holds, one cancompute m eigenvalues vi corresponding to the largest eigenvalues ofy∗y ∈ Rn×n. These relate to POD basis as follows:

ui =1σi

yvi, i = 1, . . . , d

I.M. Navon

Recent advances

1. A dual weighted trust-region adaptive POD 4D-Var applied to a

Finite-Element shallow-water Equations Model

• We consider a limited-area finite-element discretization of the shallow-water

equations model. Our purpose in this paper is to solve an inverse problem

for the above model controlling its initial conditions in presence of

observations being assimilated in a time interval (window of assimilation).

• We then attempt to obtain a reduced-order model (ROM) of above inverse

problem, based on proper orthogonal decomposition (POD), referred to as

POD 4-D Var.

• Different approaches of POD implementation of the reduced inverse

problem are compared, including a dual-weighed method for snapshot

selection coupled with a trust-region POD approach.

• Numerical results obtained point to an improved accuracy in all metrics

tested when dual-weighing choice of snapshots is combined with POD

adaptivity of the trust-region type. Results of ad-hoc adaptivity of the POD

4-D Var turn out to yield less accurate

results than trust-region POD when compared with high-fidelity model. Directions

of future research are finally outlined.

2. A POD reduced order unstructured mesh ocean modelling method for

moderate Reynolds number flows

• Herein a new approach to enhance the accuracy of a novel Proper

Orthogonal Decomposition (POD) model applied to moderate Reynolds

number flows (of the type typically encountered in ocean models) is

presented.

• This approach develops the POD model of Fang et al. [Fang, F., Pain, C.C.,

Navon, I.M., Piggott, M.D., Gorman, G.J., Allison, P., Goddard, A.J.H.,

2008. Reduced-order modelling of an adaptive mesh ocean model.

International Journal for Numerical Methods in Fluids] used in conjunction

with the Imperial College Ocean Model (ICOM), an adaptive, non-

hydrostatic finite element model.

• Both the velocity and vorticity results of the POD reduced order model

(ROM) exhibit an overall good agreement with those obtained from the full

model.

• The accuracy of the POD-Galerkin model with the use of adaptive meshes is

first evaluated using the Munk gyre flow test case with Reynolds numbers

ranging between 400 and 2000.

• POD models using the L2 norm become oscillatory when the Reynolds

number exceeds Re = 400.

• This is because the low order truncation of the POD basis inhibits generally

all the transfers between the large and the small (unresolved) scales of the

fluid flow.

• Accuracy is improved by using the H1 POD projector in preference to the

L2 POD projector. The POD bases are constructed by incorporating

gradients as well as function values in the H1 Sobolev norm.

• The accuracy of numerical results is further enhanced by increasing the

number of snapshots and POD bases.

• Error estimation was used to assess the effect of truncation (involved in

the POD-Galerkin approach) when adaptive meshes are used in conjunction with

POD/ROM.

• The RMSE of velocity results between the full model and POD-Galerkin

model is reduced by as much as 50% by using the H1 norm and increasing

the number of snapshots and POD bases.

3.A POD reduced-order 4D-Var adaptive mesh ocean modelling

Approach

• A novel proper orthogonal decomposition (POD) inverse model, developed

for an adaptive mesh ocean model (the Imperial College Ocean Model,

ICOM), is presented here.

• The new POD model is validated using the Munk gyre flow test case, where

it inverts for initial conditions.

• The optimized velocity fields exhibit overall good agreement with those

generated by the full model. The correlation between the inverted and the

true velocity is 80–98% over the majority of the domain.

• Error estimation was used to judge the quality of reduced-order adaptive

mesh models.

• The cost function is reduced by 20% of its original value, and further by

70% after the POD bases are updated.

• In this study, the reduced adjoint model is derived directly from the

discretized reduced forward model.

• The whole optimization procedure is undertaken completely in reduced

space. The computational cost for the four-dimensional variational (4D-Var)

data assimilation is significantly reduced (here a decrease of 70% in the test

case) by decreasing the dimensional size of the control space, in both the

forward and adjoint models.

• Computational efficiency is further enhanced since both the reduced forward

and adjoint models are constructed by a series of time-independent sub-

matrices. The reduced forward and adjoint models can be run repeatedly

with negligible computational costs.

• An adaptive POD 4D-Var is employed to update the POD bases as

minimization advances and loses control, thus adaptive updating of the POD

bases is necessary.

• Previously developed numerical approaches Fang et al. (Int. J. Numer. Meth.

Fluids 2008) are employed to accurately represent the geostrophic balance

and improve the efficiency of the POD simulation.

4. A POD goal-oriented error measure for mesh optimisation

• A novel approach for designing an error measure to guide an adaptive

meshing algorithm using a POD adjoint or goal-based method, is presented

here. In this work the multi-field error measure was applied to each field

type at each time level between two points in time where mesh adaptivity is

applied.

• The aim is to obtain a new mesh that can resolve all the fields at all time

levels, to an optimal (with respect to the functional) level of accuracy.

• These goal based methods solve both forward and adjoint equations to form

the overall error norms or metric tensors.

• Once metric tensors are obtained (the focus of this work) the tetrahedral

elements are then optimised so that they have unit element lengths and good

quality aspect ratios when measured with respect to the metric tensors.

Forward and adjoint evolutionary equations are solved to obtain forward and

adjoint solutions which together with their equation residuals are used to

form the error norms.

• This is the first attempt to apply the POD approach to form the overall error

indicators for optimizing meshes which can resolve all the fields during the

simulation period, to an optimal (with respect to a prescribed functional)

level of accuracy. The POD approach facilitates efficient integration

backwards in time and yields the sensitivity analyses necessary for the goal

based error estimates.

• The accuracy of both the primal and adjoint reduced models are thus

optimised (through the use of anisotropic mesh adaptivity) (for example,

here, the RMSE of velocity results between the POD results and the true

ones is reduced by half in the case of the inversion for initial conditions).

• The introduction of the POD approach in the error measure ensures that the

additional overhead associated with the use of finite elements coupled to a

mesh adaptivity algorithm is less than the overhead incurred from the use of

a uniform grid, composed of elements of the finest resolution required.

• Importantly, the goal functional for optimising meshes is consistent with that

for 4-D Var data assimilation.

• The error indicators developed here can also be used to implement a dual-

weighted POD method for order reduction and 4D-Var data assimilation in

future work.

3.2.1 Improved POD Bases

The POD provides the optimal basis for interpolating the input collection, but they are not

necessarily the best basis on which to build a dynamical system. Searching

for new bases for model reduction is an active research area. The primary emphasis of this

research will be to seek bases that are i.) well suited to ocean flows, e.g. are able to identify

dynamically relevant structures and ii.) have advantages when used within an optimization

setting, e.g. are robust to perturbations in model parameters and can be efficiently updated

when the flow evolves such that it is not well represented by the basis. The following are

potentially fruitful research directions.

Principal Interaction Pattern (PIP) Analysis The goal is to find a limited number

of structures in seemingly very complicated physical scenarios. These vary according to

nonlinear equations determining the interaction between the different structures. By minimizing

a suitably chosen error function, calculated by comparing a PIP model with observed

or synthetic data sets, both the structures and their interaction coefficients are determined

simultaneously. This may serve as a useful tool for identifying basic structures where POD

encounters difficulties in 3D.

Principal Interval Decomposition (PID) The principal interval decomposition

provides a natural complement to many reduced-basis techniques. The PID is a natural

way to select time intervals on which to perform basis selection. Although it can provide a

mechanism to build reduced-order models with specified error estimates, it usually does not

do this efficiently. Our interest in this approach lies in the promise of finding structures that

are dynamically important but only persist over short periods of time. Averaging methods

such as the POD will miss these structures since they do not persist for a long period of

time. This may lead to models that are better suited to predict complex phenomena, such

as the behavior of the system near bifurcation points.

3.2.2 Closure Modeling ( Iliescu, Borggaard)

• From the earliest stages of POD, it was recognized that a simple Galerkin

truncation will generally produce inaccurate results, even if the modes

retained capture most of the system's energy .

• Thus, closure modeling (i.e., modeling the effect of the discarded modes on

the modes retained in the system) has always played a central role in POD

reduced-order modeling (POD ROM) strategies.

• The closure modeling has developed into two main directions(that often

overlap):

• (i) improving the numerical stability and

(ii) improving the physical accuracy.

• The research proposed here is concerned with the latter. Although there are

several closure modeling strategies (e.g., the novel approach of Noack et al.

that uses a finite-time thermodynamics

formalism), our approach is different in that it addresses explicitly the following

well-known drawbacks of the POD Galerkin truncation:

• The proposed closure modeling strategies target: (i) a wide range of

Reynolds numbers, and (ii) time intervals longer than those over which

snapshots were collected. Both features are essential for the reduced-order

modeling data assimilation strategies relevant to realistic ocean flows.

Proposed Research:

• We will develop POD closure models based on ideas from two state-of the-

art LES modeling approaches and compare them with state-of-the-art closure

modeling strategies, such as those developed by Noack's group .

• POD Eddy Viscosity (POD-EV) models:

• Our approach builds on the pioneering work

in (see also) and employs the concept of energy cascade. That is, the effect

of the discarded POD modes is modeled by adding a dissipative term to the ROM,

which is equivalent to increasing the effective viscosity in the POD-ROM

The validity of the energy cascade concept in a POD setting was confirmed in a

recent numerical study .

This study, together with the long list of recent reports in which variations of the

simple mixing length eddy viscosity model were successfully used clearly

indicate that turbulence modeling ideas could be used in the development of

closure models for POD.

(Ref: Introduction to Model Order Reduction,Wil Schilders ...inavon/pubs/LectureSC.pdf · "On Lines...

Documents

Transcript of (Ref: Introduction to Model Order Reduction,Wil Schilders ...inavon/pubs/LectureSC.pdf · "On Lines...