Design Optimization Utilizing Gradient/Hessian Enhanced Surrogate Model

Design Optimization UtilizingDesign Optimization UtilizingGradient/Hessian Enhanced Surrogate ModelGradient/Hessian Enhanced Surrogate Model

Dept. of Mechanical Engineering,University of Wyoming, USA

Wataru YAMAZAKI,Markus P. RUMPFKEIL,Dimitri J. MAVRIPLIS

28th, June, 2010,28th AIAA Applied Aerodynamics Conference

Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ.Wataru YAMAZAKI, Univ. of Wyoming-2-

Outline

*Background- Efficient CFD Gradient/Hessian calculations- Surrogate Model Enhanced by Gradient/Hessian- Uncertainty Analysis

*Objectives

*Surrogate Model Approaches- Kriging- Direct and Indirect Gradient-enhanced Kriging- Gradient/Hessian-enhanced Kriging Approaches

*Results & Discussion- Analytical Function Fitting- Aerodynamic Data Modeling- 2D Airfoil Drag Minimization- Uncertainty Analysis at Optimal Airfoil

*Conclusions


Background~ Efficient CFD Hessian Calculation

M.P. Rumpfkeil and D.J. Mavriplis, AIAA-2010-1268“Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method”

An efficient CFD Hessian calculation methodby Adjoint method and Automatic Differentiation (AD)

For steady flowi. Solutions for grid deformation / flow residual equations

ii. Adjoint solutions for flow / grid deformation equations

iii. Ndv linear solutions each for dx/dDj and dw/dDj

iv. Ndv(Ndv+1)/2 cheap evaluations for each Hessian component

0, DxDs 0,, DwDxDR

0

TT

F

ww

R0

T

TT

F

x

R

xx

s

sR jkT

jkT

jkkj

FdDdD

Fd

2


Background~ Efficient CFD Hessian Calculation

Grid Deformation

Flow Residual

Flow AdjointMesh Adjoint

dx/dD1

dw/dD1

dx/dD2

dw/dD2

dx/dDNdv

dw/dDNdv

......

Gradient and Hessian

An efficient CFD Hessian calculation methodby Adjoint method and Automatic Differentiation (AD)



Background~ Approximate CFD Hessian

For steady flow, a special form of objective function

K

kkkk FFwF

1

2target

2

1

2target2target

2

1.. DDDLLL CCwCCwFge

K

k j

k

T

i

kk

K

k ji

kkkk

K

k j

k

T

i

kk

ji

dD

dF

dD

dFw

dDdD

FdFFw

dD

dF

dD

dFw

dDdD

Fd

1

1

2target

1

2

Last approximation is accurate only nearly optimum Approximate Hessian only requires the first-order derivatives


0


Background~ Uncertainty Analysis

Uncertainty due to manufacturing tolerances in-service wear-and-tear etc

Analysis of mean/variance/PDF ofobjective function w.r.t. fluctuation of design variables

Full Monte-Carlo Simulation Thousands/Millions exact function calls Accurate and easy, but computationally expensive

Moment Method Taylor series expansion by grad/Hessian at the center No information about PDF

Inexpensive Monte-Carlo Simulation Thousands/Millions surrogate model function calls Much lower computational cost


Objectives

The efficient adjoint gradient/Hessian calculation methodswill be effective…

for more efficient global design optimizationwith G/H-enhanced surrogate model approach

for more accurate and cheaper uncertainty analysisby inexpensive Monte-Carlo simulation

with G/H-enhanced surrogate model

Development of gradient/Hessian-enhanced surrogate models Application to design optimization and uncertainty analysis


Kriging, Gradient-enhanced Kriging

Kriging model approach - originally in geological statistics

Two gradient-enhanced Kriging (cokriging or GEK)

Direct CokrigingGradient information is included in the formulation(correlation between func-grad and grad-grad)

Indirect CokrigingSame formulation as original KrigingAdditional samples are created by using the gradient infoKriging model by both real and additional pts

xx

xxx

xx

i

i

yyy T

add

iadd2D example

: Real Sample Point: Additional Sample Point


Gradient/Hessian-enhanced Kriging

Indirect Approach

xHxGx

xxx

x

TTadd

iadd

iyy

2

1

2D example: Real Sample Point: Additional Sample Point

Arrangements to Use Full Hessian / Diagonal Terms

Major parameters : distance between real / additional pts number of additional pts per real pt

Worse matrix conditioning with smaller distancelarger number of additional pts

Severe tradeoffs for these parameters


Gradient/Hessian-enhanced KrigingDirect ApproachConsider a random process model estimating a function valueby a linear combination of function/gradient/Hessian components

n

iii

n

iii

n

iii yyywy

1

''

1

'

1

ˆ x

Minimizing Mean-Squared-Error (MSE) between exact/estimated function

2

1

''

1

'

1

ˆ YyyywEyMSEn

iii

n

iii

n

iii x

with an unbiasedness constraint

11

n

iiw

Solving by using the Lagrange multiplier approach

iforJJJ

w

JwyMSEJ

iii

n

ii

01ˆ

1 x


Gradient/Hessian-enhanced KrigingDirect Approach

Introducing correlation function for covariance termsCorrelation is estimated by distance between two pts with radial basis function

kjji

k

ji

xRxZZCov

RZZCov

ji

ji

xx

xx

xx

xx

,,

,,

2

2

1~0

rx

F

FR

T

Unknown parameters are determined by the following system of equations

,,,,, 111 wT x

Final form of the gradient/Hessian-enhanced direct Kriging approach is

FYRr xx 1ˆ Ty

samplesgivenatninformatioexactyyy

dataobservedbetweennscorrelatio

dataobservedandbetweennscorrelatio

termconstantmean

T

TT

,,,,, ''1

'11

111

Y

R

xr

YRFFRF

x


Gradient/Hessian-enhanced KrigingDirect Approach

nnnnnn

Nn

Nn

Nn

Nn

Nn

Nn

Ndvn

Ndvn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Ndvn

Nn

Ndvn

Nn

dvdv

nn

dv

n

dvdv

nn

dv

nnnn

dv

n

dv

nn

dvdv

nn

dv

n

dv

nn

dv

n

dv

nn

dv

n

dv

n

dv

nn

nnn

dv

nnn

nnn

n

dv

n

n

xx

R

xx

R

xx

R

xx

R

x

R

x

R

xx

R

xx

R

xx

R

xx

R

x

R

x

Rxx

R

xx

R

x

R

xx

R

x

R

x

R

xx

R

xx

R

xx

R

x

R

x

R

x

Rx

R

x

R

x

R

x

RRR

x

R

x

R

x

R

x

RRR

22

,4

211

2

,4

2

,3

11

2

,3

2

,2

2

,2

11

22

,4

11

211

2

,4

11

2

,3

11

11

2

,3

11

2

,2

11

2

,2

2

,3

11

2

,3

2

,2

11

,2

,,

11

2

,3

11

11

2

,3

11

,2

11

2

,2

11

,

11

,

2

,2

11

2

,2

,

11

,,,

2

,2

11

2

,2

,

11

,,,

111

111111111

111

111111111

11

1

111111

111

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

R

Correlations between F-F, F-G, G-G, F-H, G-H and H-H Up to 4th order derivatives of correlation function Automatic Differentiation by TAPENADE No sensitive parameter Better matrix conditioning than indirect approach

FYRr xx 1ˆ Ty


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI

Exact Function Sample Points Kriging RMSE EI

Infill Sampling Criteria for Optimization How to find promising location on surrogate model ? Maximization of Expected Improvement (EI) value Potential of being smaller than current minimum (optimal) Consider both estimated function and uncertainty (RMSE)

s

yys

s

yyyyEI minmin

min

xxxx

00s

EI,

y

EI

Results & DiscussionResults & Discussion


2D Rastrigin Function Fitting

2122

21 2cos2cos1020 xxxxy x

80 samples by Latin Hypercube SamplingDirect Kriging approach

Exact Rastrigin Function Function-based KrigingGradient-enhancedGradient/Hessian-enhanced


5D Rosenbrock Function Fitting

F: Function-based KrigingFG: Gradient-enhancedFGHd: G/diag. Hess-enhancedFGH: G/full Hess-enhanced

RMSE .vs. Number of sample points Superiority in direct Kriging approaches

thanks to exact enforcement of derivative information better conditioning of correlation matrix


Validation on Rosenbrock Func.

Minimization of 20D Rosenbrock 30 initial sample points by LHS EI-based infill sampling criteria Faster convergence in

G/H-enhanced direct approach

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E+02

1.E+03

1.E+04

0 100 200 300

Number of Sample Points

Obj

ecti

ve F

unct

ion

F

Direct_FG

Direct_FGH

Indirect_FGH

1

1

221

2 1001dvN

iiii xxxy x

Uncertainty analysis on 2D Rosenbrock 5 sample points for surrogate model

(No sample point on the center location) Superior performance in

G/H-enhanced Inexpensive MC (IMC)

CDFs of Full-MC and IMCOptimization History

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50 60 70 80 90

Function ValueC

DF

Full-MC

IMC_F

IMC_FG

IMC_FGH


Aerodynamic Data Modeling Unstructured mesh CFD Steady inviscid flow, NACA0012 20,000 triangle elements Mach Number [0.5, 1.5] Angle of Attack[deg] [0.0, 5.0] 21x21=441 validation data

Exact Hypersurface of Lift Coefficient Exact Hypersurface of Drag Coefficient


Aerodynamic Data Modeling

Adjoint gradient is helpful to construct accurate surrogate model CFD Hessian is not helpful due to noisy design space

Function-based KrigingGradient-enhancedExact

Cl

Cd


2D Airfoil Shape Optimization

Unstructured mesh CFD Steady inviscid flow, M=0.755 NACA0012, 16 DVs for Hicks-Henne function Objective function of inverse design form

Exact / Approximate CFD Hessian available Computational time of F : 2 min,

FG : 4 min, FGHapprox

. : 6 min, FGHexact : 36 min (4 min in parallel)

Geometrical constraint for sectional area

22

2target2target

000.02

100675.0

2

12

1

dl

dddlll

CC

CCwCCwF

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

x

H(x

)



Start from 16 initial sample points which only have function info Gradient/Hessian evaluations only for new optimal designs

Faster convergence in derivative-enhanced surrogate model Best design in gradient/exact Hessian-enhanced model



Towards supercritical airfoils Shock reduction on upper surface

NACA0012 (Baseline) Optimal by G/exact H-enhanced model


2D Airfoil Shape Uncertainty Analysis

Geometrical uncertainty analysis at optimal airfoil Center = optimal obtained by Grad/exact H model

Comparison between2nd order Moment Method (MM2)

using gradient/Hessian at the centerInexpensive Monte-Carlo (IMC1)

using final surrogate model obtained in optimizationInexpensive Monte-Carlo (IMC2)

using different G/H-enhanced model by 11 samplesFull Non-Linear Monte-Carlo (NLMC)

using 3,000 CFD function calls

optimal (center) ±0.1 airfoil



Mean of objective w.r.t. standard deviation of all design variables IMC showed good agreement with NLMC at smaller st. devi. Necessity of additional sampling criteria for total model accuracy ? Promising IMC with much cheaper computational cost

MM2 using derivative at the centerIMC1 using G/H surrogate model obtained in optimizationIMC2 using different G/H model by 11 samples (for st.devi.=0.01)NLMC using 3,000 CFD function calls


Concluding Remarks / Future Works

Development of gradient/Hessian-enhanced Kriging models Application to design optimization and uncertainty analysis

Direct Kriging approach is superior to indirect approach More accurate fitting on exact function Faster convergence towards global optimal design Promising inexpensive Monte-Carlo simulation at much lower cost

Application to higher-dimensional / complicated design problem Robust design with inexpensive Monte-Carlo simulation Gradient/Hessian vector product-enhanced approach

Thank you for your attention !!

Appendix


Moment Method

Taylor series expansion by grad/Hessian at the center No information about PDF

1st order Moment Method

2nd order Moment Method

dv

i

c

N

iD

iMM

cMM

dD

dF

F

1

2

21

1

x

x

dv dv

ji

c

dv

i

c

N

i

N

jDD

jiMMMM

N

iD

iMMMM

dDdD

Fd

dD

Fd

1 1

22

21

22

1

22

2

12

2

1

2

1

x

x


Gradient/Hessian-enhanced KrigingImplementation Details

Correlation function of a RBF

Estimation of hyper parameters by maximizing likelihood function with GA

Correlation matrix inversion by Cholesky decomposition

Search of new sample point location by maximizing Expected Improvement (EI) value with GA

else

hforhhhhscf

0

13183513

1,

226


Infill Sampling Criteria for Optimization How to find promising location on surrogate model ? Expected Improvement (EI) value Potential of being smaller than current minimum (optimal) Consider both estimated function and uncertainty (RMSE)

s

yys

s

yyyyEI minmin

min

xxxx

00s

EI,

y

EI

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


EI-based criteria have good balancebetween global/local searching


5D Rosenbrock Function Fitting

# of pieces of information = sum of # of F/G/H net components To scatter samples is better than concentration at limited samples Approximated computational time factor

G/H-enhanced surrogate model provides better performancewith efficient Gradient/Hessian calculation methods

FGHFGFhasiifTTTF i

N

ii

sample

//,3/2/1,1


1D Step Function Fitting

-0.5

0.0

0.5

1.0

1.5

0.0 0.2 0.4 0.6 0.8 1.0

Design Variable

Fun

ctio

n V

alue Exact

Samples

F

FG

FGH

Much better fit by G/H-enhanced direct Kriging


Minimization of 20D Rosenbrock Func.

Minimization of 20 dimensional Rosenbrock function No computational cost for Func/Grad/Hess evaluation Expensive for construction - likelihood function maximization

- inversion of correlation matrix Parallel computation for the likelihood maximization problem

1.E-02

1.E-01

1.E+00

1.E+01

1.E+02

1.E+03

1.E+04

0 3000 6000 9000 12000 15000

Computational Time [sec]

Obj

ecti

ve F

unct

ion

F

FG

FGH


Uncertainty Analysis

Uncertainty analysis at (1.0,1.0) on 2D Rosenbrock 5 sample points for surrogate model approaches

(No sample point on the center location) 2nd order Moment Method (MM2) by G/H at the center Superior results in G/H-enhanced Inexpensive MC (IMC)

St. Devi. = 0.15 means the possibility within -0.15<dx<0.15 is about 70%

-10

10

30

50

70

90

110

130

150

0.0 0.1 0.2 0.3 0.4 0.5

Standard Deviation of DVs

Mea

n of

Fun

ctio

n

Full-MC

MM2

IMC_F

IMC_FG

IMC_FGH

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50 60 70 80 90

Function Value

CD

F

CDF at St. Devi.=0.15


Aerodynamic Data Modeling

0.208

0.209

0.210

0.211

0.212

1.390 1.395 1.400 1.405 1.410Mach Number

CL

CFD Data

Linear by Adj_Grad

Quadratic by Adj_G/H

0.1080

0.1082

0.1084

0.1086

0.1088

0.1090

1.390 1.395 1.400 1.405 1.410

Mach Number

CD

CFD Data

Linear by Adj_Grad

Quadratic by Adj_G/H

Cl Cd

NACA0012 M=1.4 AoA=3.5[deg] Noisy in Mach number direction



Cumulative Density Function at St. Devi. of 0.01 Quadratic model only by using gradient/Hessian at optimal Additional sampling criteria to increase total model accuracy

Design Optimization Utilizing Gradient/Hessian Enhanced Surrogate Model

Documents

Transcript of Design Optimization Utilizing Gradient/Hessian Enhanced Surrogate Model