2014.10.dartmouth

Adjoin Chaos”Least Squares Shadowing” sensitivity analysis of chaotic high fidelity

simulations

Qiqi WangAssistant Professor of Aero & Astro, MIT

We gratefully acknowledge:

U Mass Dartmouth, Oct 29, 2014

2

Use of Computational Simulations

AnalysisSimulation on manually picked geometry and condition

DesignBeyond single simulation, towards optimization and parametric study

3




Low fidelity simulationPotential flow solver,RANS, URANS

High fidelity SimulationLarge Eddy Simulation (LES), Detached Eddy Simulation (DES), Unsteady Multi-physics Simulations

4




Low fidelity simulationPotential flow solver,RANS, URANS THE FRONTIER


THE FRONTIER

ESTABLISHED

5




Low fidelity simulationPotential flow solver,RANS, URANS THE FRONTIER


THE FRONTIER

ESTABLISHED

• Reduce time and resources used in physical testing• 1970s – Boeing 757, 77 wings tested in wind tunnel.

• 1990s – Boeing 777, 11 wings tested in wind tunnel.

• Simulation-based design limited by the fidelity of simulation• Unsteady effects (noise, vibration)

• Complex turbulent flows (separation, mixing)

• 2000s – Boeing 787, 11 wings tested in wind tunnel.

• High fidelity design can lead to faster, better next generation aerospace vehicles.

Why Design using High Fidelity Simulations?

7

From top500.org

10 years

1000 times

In 10 years, you will think about a 100,000-core computer approximately the way you think about a 100-core computer today

Total GFLOPS, top500

Fastest supercomputer

500th supercomputer

The hardware will get there

How about the software and the math behind?

8

What’s different in high fidelity simulations?

Optimal Control

Jet Engine Combustor

(Stanford / DOE)

Launch abort

system

(NASA)

Maneuvering F18(Airforce)

Landing gear noise

Design optimization

Rotorcraft wake

(NASA)

• Many high fidelity simulations in important aerospace applications are unsteady and chaotic.

• Design in Chaos: High fidelity, unsteady simulation and design.

Towards Optimization with LES

Towards Optimization:Parameterized LES of turbine blade – baseline

Towards Optimization:Parameterized LES of turbine blade – Optimized

A chaotic optimization

12

min𝑠

1

𝑇 0

𝑇

𝐽 𝑢 𝑑𝑡 𝑠. 𝑡.𝑑𝑢

𝑑𝑡= 𝑓(𝑢, 𝑠)

• Cost function is averaged over long T• f is chaotic

Time

Dra

g C

oef

fici

en

t

Lift

Co

effi

cie

nt

What is chaos? Why does it matter?

13Cylinder in Re = 500 flow, Wang and Gao 2013

How to perform sensitivity analysis and adjoint? Divergence of sensitivity, a.k.a. the “butterfly effect”, poses an interesting mathematical question.

Chaos: solutions diverge under small perturbations in the governing equation

14

What is chaos? Why does it matter?

15Time

Dra

g C

oef

fici

en

t

Lift

Co

effi

cie

nt

Chaos is not periodic. Predicting time averaged quantity with small sampling error requires very long simulation.

What is Optimization in Chaos?

16

min𝑠

1

𝑇 0

𝑇



• The cost function is averaged over a long T• f governs chaotic dynamics

Time

Dra

g C

oef

fici

en

t

Lift

Co

effi

cie

nt

But how do I know?

This could be YOURobjective function ina chaotic simulation

This could be YOURobjective function ina periodic simulation

A prime indicator: Divergence Of Derivative

An objective function produced by a

chaotic simulation

Time averaged objective in chaos and its derivative

Derivatives computed with traditional adjoint method

19

Divergence Of Derivative

Lea et al. Sensitivity analysis of the climate of a chaotic system,Tellus A 2000

Lorenz systemOutput z: rate of heat transferInput ρ: temperature difference

Time averaged output

Input Input

Derivative of time averaged output

20

Divergent solution of linearized equation inchoatic 2D cylinder wake

Credit: T. Barth, ASA Ames.

Re=3900

Re=10000

21

Divergent of Derivative in chaotic vortex shedding over stalled airfoil

FUN3D, NASA Langley

Re=10000 Magnitude of the adjoint gradient

22

Divergent of Derivative in 3D turbulent cylinder wake

Adjoint solution for

a circular cylinder

Re=500. z-vorticity

CDP, Stanford CTR

Code NASA Ames (Barth)

CTR (Stanford) NASA Langley (FUN3D)

Numericalscheme

Density-based Discontinuous Galerkin

Pressure-based finite volume

Density-based finite volume

Flow field 2D cylinder wake

3D wakeJet in crossflow

Stalled airfoil wake

Solution to both linearized and adjoint Navier-Stokes equations diverge exponentially as flow develops

unsteady, aperiodic (a.k.a. chaotic) behavior

23

24

Derivative does not commute with time average

Lea et al. Sensitivity analysis of the climate of a chaotic system,Tellus A 2000

Lorenz systemOutput z: rate of heat transferInput ρ: temperature difference

Time averaged output

Input Input

Derivative of time averaged output

Small perturbation leads to large difference in solution of initial value problems

25

Input ParameterInput Parameter

Time averaged outputTime dependent output

Tim

e

In a chaotic problem with fixed initial condition,

• Small perturbation leads to large difference in solution

• Linearized (and adjoint) equations diverge

• Sensitivity computation fails even for well-defined time average quantities.

• A barrier to optimization, inference and uncertainty quantification using high fidelity simulations.

26

In a chaotic problem with fixed initial condition,

• Small perturbation leads to large difference in solution

• Linearized (and adjoint) equations diverge

• Sensitivity computation fails even for well-defined time average quantities.

• A barrier to optimization, inference and uncertainty quantification using high fidelity simulations.

Initial value problem of chaos is ill-conditioned.

27

Is there a solution? Yes.


min𝑠

1

𝑇 0

𝑇



Where f governs chaotic dynamics

𝑠. 𝑡. 𝑢 0 = 𝑢0

So that the solution is unique defined?


min𝑠

1

𝑇 0

𝑇



Where f governs chaotic dynamics

𝑠. 𝑡. 𝑢 0 = 𝑢0

So that the solution is unique defined?

Least Squares Shadowing (LSS): uniquely define the solution without incurring the “butterfly effect”.

10.1016/j.jcp.2014.03.002

http://arxiv.org/ct?url=http://dx.doi.org/10.1016/j.jcp.2014.03.002&v=dd182b8a

What is least squares shadowing?

• Initial Value Problem:

31

Solutions to similar initial value problems

32

• u(t) (ρ=30) and ur(t) (ρ=28) are very different after some time!

What is least squares shadowing?

• Initial Value Problem:

• Least Squares Problem:

33

34

Time Transformation

• Time transformation is required to keep the trajectories close in phase space for infinite time

Without Time Transformation With Time Transformation

• Least squares problem with time transformation:

• Assume Ergodicity: No initial condition for u!

• “Integral phase condition” for finding homoclinic cycles in dynamical systems

• Does using least squares instead of an initial condition remove the ill-conditioning of chaos?

Least Squares Problem

Reference solution

35

36

• u(t) (ρ=30) and ur(t) (ρ=28) are very different after some time!

Solutions to similar initial value problems

Solutions to similar Least Squares Shadowing Problems

37

• u(τ) (ρ=30) shadows ur(t) (ρ=28)!

38

Least squares shadowing

Condition number O(1)

Initial value problem

Condition number O(eλT)

Input ParameterInput Parameter

Tim

e

Tim

e

Time dependent output Time dependent output

New approach: Least Squares Shadowing

• Linearize the initial value problem

• Linearize the regularized problem without initial condition

• Compute sensitivity of time averaged quantity from linearized solution

• Wang, Hu and Blonigan. Sensitivity computation of chaotic limit cycle oscillations. Journal of Computational Physics. 2014. arXiv:1204.0159

39

where is solution to the linearized initial value problem

regularized problem without initial condition

“Least Squares Shadowing”

40

“Least Squares Shadowing” Works

Computed derivative of the output to the input (Initial value problem)

41

Computed derivative of the output to the input (Least squares problem)

• Linearize the regularized problem without initial condition

• Compute sensitivity of time averaged quantity from linearized solution

An objective function produced by a

chaotic simulation

Time averaged objective in chaos and its derivative

Derivatives computed with traditional adjoint method

Derivatives computed with Least Squares Shadowing

43

Shadowing Lemma

Consider a system governed by:

For any δ>0 there exists ε>0, such that for every “ε-pseudo-solution” u satisfying ǁdu/dt−f(u)ǁ<ε, there

exists a true solution u satisfying du/dτ−f(u)=0 under a time transformation τ(t), such that

ǁu(τ)−u(t)ǁ<δ, |1−dτ/dt|<δ

Pilyugin SY. Shadowing in dynamical systems. Volume 1706, Lecture Notes in Mathematics, Springer: Berlin, 1999.

Collection of 100 Lorenz System

trajectories

Each color represents a shadowing trajectory at a different parameter value.

44

Parallel space-time multigrid enables least squares sensitivity computation for isotropic homogeneous turbulent flows

45

Taylor micro-scale Reynolds number = 33

How to compute the Least Squares Shadowing Solution

46

Algorithm I: Shooting

• Search for initial condition, u(τ(T0))

Algorithm II: Full trajectory Design

• Search for entire trajectory, u(τ(t)), T0≤t ≤T1

Algorithm III: Multiple shooting

• Search for trajectory checkpoints, u(τ(t)), t = t0, t1,…,tK

• Solve for Lagrange multipliers, w:

• Operator is 2nd order and coercive

• 2nd order Boundary Value Problem in time.

• Solved with Multigrid in space and time.

Full Trajectory Design

47

• Guess u(t),w(t) at each checkpoint, solve in segment:

• Use mismatch in u(τ)-ur(t) and w(t) at checkpoints to update guess:

Algorithm III, Multiple shooting

48

0 2 4 6 8 10-5

0

5

10

15

t

u-u

r

1 Iteration



49

0 2 4 6 8 10-5

0

5

10

15

t

u-u

r

1 Iteration

Algorithm III, Checkpoint Design



50

0 2 4 6 8 10-5

0

5

10

15

t

u-u

r

0 Iterations


0 2 4 6 8 10-0.5

0

0.5

1

1.5

2

t

u-u

r



51

2 Iterations


0 2 4 6 8 100

0.5

1

1.5

2

t

u-u

r• Guess u(t),w(t) at each checkpoint, solve in segment:


52

10 Iterations


Lower memory, easier implementation

53

• Solve a n(2K-1) by n(2K-1) symmetric linear system

• Solver can be built using existing components:• Non-linear solver for .

• Forward (tangent) solver for .

• Adjoint solver for Lagrange multipliers .

• Vector dot product (i.e. ).

Optimization in Chaos• Traditional sensitivity analysis fails for chaotic and

turbulent fluid flows.

• Least Squared Shadowing has shown great promise for these flows, especially using multiple shooting algorithm.

• Currently working on Least Squares Shadowing for wall bounded turbulent flow with complex geometries.

• Least Squares Shadowing will enable design and uncertainty quantification of chaotic and turbulent fluid flows.

We gratefully acknowledge:

2014.10.dartmouth

Engineering

Transcript of 2014.10.dartmouth