WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and...
-
Upload
sabina-roberts -
Category
Documents
-
view
229 -
download
7
Transcript of WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and...
![Page 1: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/1.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Engineering OptimizationConcepts and Applications
Fred van Keulen
Matthijs Langelaar
CLA H21.1
![Page 2: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/2.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Contents
● Optimization problem checking and simplification
● Model simplification
![Page 3: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/3.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Model simplification
● Basic idea:
Expensivemodel
Optimizer
Cheapmodel
Optimizer
● Motivation:
– Replacement of expensive function, evaluated many times
– Interaction between different disciplines
– Estimation of derivatives
– Noise
![Page 4: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/4.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Model simplification (2)
● Drawback: loss of accuracy
● Different ranges: local, mid-range, global
● Synonyms:
– Approximation models
– Metamodels
– Surrogate models
– Compact models
– Reduced order models
Extractinformation
Constructapproximation
Procedure:
![Page 5: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/5.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Model simplification (3)
● Information extraction: linked to techniques from physical experiments: “plan of experiments” / DoE
● Many approaches! Covered here:
– Taylor series expansions
– Exact fitting
– Least squares fitting (response surface techniques)
– Kriging
– Reduced basis methods
– Briefly: neural nets, genetic programming, simplified physical models
● Crucial: purpose, range and level of detail
![Page 6: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/6.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Taylor series expansions
● Approximation based on local information:
21 )(''!2
1)('
!1
1)()( hxfhxfxfhxf
0
)( )(!
1
n
nn hxfn
N
n
nn hxfn0
)( )(!
1Truncation error!
● Use of derivative information!
● Valid in neighbourhood of x
)()(!
1
0
)( NN
n
nn hohxfn
![Page 7: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/7.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Taylor approximation example
1st order2nd order3rd order4th order5th order20th order
3
)5/cos(
51
52/
xx
ef
x
FunctionApproximation(x = 20)
x
![Page 8: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/8.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Exact fitting (interpolation)
● # datapoints = # fitting parameters
● Every datapoint reproduced exactly
● Example:
xaaf 10
2
1
1
0
2
1
1
1
f
f
a
a
x
x
x1 x2
f2
f1
![Page 9: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/9.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Exact fitting (2)
● Easy for intrinsically linear functions:
● No smoothing / filtering / noise reduction
● Danger of oscillations with high-order polynomials
n
iii fafafafaf
1221100
● Often used: polynomials, generalized polynomials:
1 2 1 2log( ) log log logm nf a bx x f a b m x n x
![Page 10: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/10.jpg)
WB1440 Engineering Optimization – Concepts and Applications
9th orderpolynomial
Oscillations
● Referred to as “Runge phenomenon”
● In practice: use order 6 or less
2
1
1 25x5th order9th order
![Page 11: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/11.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Least squares fitting
● Less fitting parameters than datapoints
● Smoothing / filtering behaviour
● “Best fit”? Minimize sum of deviations:
N
iii xfxf
1
|)(~
)(|mina
● “Best fit”? Minimize sum of squared deviations:
N
iii xfxf
1
2)(
~)(min
a
x
f
f~
![Page 12: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/12.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Least squares fitting (2)
● Choose fitting function linear in parameters ai :
)()()()()(~
221100 xfaxfaxfaxfaxf mm
NmNmNNN
m
m
m
Na
a
a
a
xfxfxfxf
xfxfxfxf
xfxfxfxf
xfxfxfxf
xf
xf
xf
xf
2
1
0
2
1
0
210
2222120
1121110
0020100
2
1
0
)()()()(
)()()()(
)()()()(
)()()()(
)(~
)(~
)(~
)(~
εMaf ~
● Short notation:
![Page 13: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/13.jpg)
WB1440 Engineering Optimization – Concepts and Applications
LS fitting (3)
● Minimize sum of squared errors:
(Optimization problem!)
MafMafεεa
~~
minTTL
MaMfMMafM0a
TTTL2
~2
~2
fMMMafMMaM~~ 1 TTTT
![Page 14: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/14.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Polynomial LS fitting
● Polynomial of degree m:
mmxaxaxaxaxf 2
21
10
0)(~
mmm
mmm
m
m
m
ma
a
a
a
xxx
xxx
xxx
xxx
xf
xf
xf
xf
2
1
0
2
1
0
2
22
22
12
11
02
00
2
1
0
1
1
1
1
)(~
)(~
)(~
)(~
mii
ii
ii
i
mm
imi
mi
mi
miiii
miiii
miii
xf
xf
xf
f
a
a
a
a
xxxx
xxxx
xxxx
xxxN
~
~
~
~
22
1
0
221
2432
132
2
![Page 15: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/15.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Polynomial LS example
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-0.2
0
0.2
0.4
0.6
0.8
1
1.2
samples
quadratic 6th degree
![Page 16: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/16.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Multidimensional LS fitting
● Polynomial in multiple dimensions:
ii fa
xyayxayaxa
xyayaxa
yaxaayxf
29
28
37
36
52
42
3
210),(~
● Number of coefficients ai for quadratic polynomial in Rn:
Curse of dimensionality!
)2)(1(2
1 nnm
![Page 17: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/17.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Fractional factorial design
Response surface
● Generate datapoints through sampling:
– Generate design points through Design of Experiments
– Evaluate responses
● Fit analytical model
● Check accuracy
2n full factorial designx1
x2
x3
![Page 18: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/18.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Latin Hypercube Sampling (LHS)
● Popular method: LHS
● Based on idea of Latin square:
● Properties:
– Space-filling
– Any number of design points
– Intended for box-like domains
– Matlab: lhsdesign0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
![Page 19: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/19.jpg)
WB1440 Engineering Optimization – Concepts and Applications
(LS) Fit quality indicators● Accuracy? More / fewer terms?
● Examine the residuals
– Small
– Random! xi
Okay: >0.6
Okay: >>1
2
2
2
~1ff
Ri
i
1
~
2
2
mN
mffF
i
i
● Statistical quality indicators:
– R2 correlation measure:
– F-ratio (signal to noise):
![Page 20: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/20.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Nonlinear LS
● Linear LS: intrinsically linear functions (linear in ai):
paTxaxaxaxf 22
11
00)(
paTx xaxaeaxf 210 log)(
● Nonlinear LS: more complicated functions of ai:
● More difficult to fit! (Nonlinear optimization problem)
● Matlab: lsqnonlin
1)(
221
0
xaxa
xaxf
![Page 21: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/21.jpg)
WB1440 Engineering Optimization – Concepts and Applications
LS pitfalls
● Scattered data:
● Wrong choice of
basis functions:
x
f
x
f
![Page 22: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/22.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Kriging● Named after D.C. Krige, mining engineer, 1951
● Statistical approach: correlation between neighbouring points
– Interpolation by weighted sum:
– Weights depend on distance
– Certain spatial correlationfunction is assumed(usually Gaussian)
N
iiii yxxxy
1
),()(
![Page 23: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/23.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Kriging properties
● Kriging interpolation is “most likely” in some sense (based on assumptions of the method)
● Interpolation: no smoothing / filtering
● Many variations exist!
● Advantage: no need to assume form of interpolation function
● Fitting process more elaborate than LS procedure
![Page 24: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/24.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Kriging example
● Results depend strongly on statistical assumptions and
method used:
Dataset z(x,y) Kriging interpolation
![Page 25: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/25.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Reduced order model
● Idea: describing system in reduced basis:
– Example: structural dynamics
fwKwMfKuuM~~~
● Select small number of “modes” to build basis
– Example: eigenmodes
![Page 26: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/26.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Reduced order model (2)
● Reduced basis:
Bwu kωωωB 21
Nk
iiiw
1
ωu
Bwu
fBf
KBBK
MBBM
T
T
T
~
~
~
fKBwwMB
fKuuM
● Reduced system equations:N1 Nk k1
fBKBwBwMBB TTT
kN NkNN
N1kN
![Page 27: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/27.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Reduced order models● Many approaches!
– Selection of type and number of basis vectors
– Dealing with nonlinearity / multiple disciplines
● Active research topic
● No interpolation / fitting, but approximate modeling
![Page 28: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/28.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Aerodynamic model
Example:Aircraft model
Structural model
Mass model
![Page 30: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/30.jpg)
WB1440 Engineering Optimization – Concepts and Applications
(input)
output
Neural nets
To determine internal neuron parameters, neural nets must be trained on data.
x f(x)
![Page 31: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/31.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Neural net features● Versatile, can capture complex behavior
● Filtering, smoothing
● Many variations possible
– Network
– Number of neurons, layers
– Transfer functions
● Many training steps might be required (nonlinear optimization)
● Matlab: see e.g. nndtoc
![Page 32: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/32.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Genetic programming
● Building mathematical functions using
evolution-like approach
● Approach good fit by crossover and
mutation of expressions
^2
+
/
x2
x3
x1
2
32
1
x
x
x
![Page 33: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/33.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Genetic programming
● LS fitting with population of analytic expressions
● Selection / evolution rules
● Features:
– Can capture very complex
behavior
– Danger of artifacts /
overfitting
– Quite expensive procedure
![Page 34: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/34.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Simplified physical models
● Goal: capture trends from underlying physics through
simpler model:
– Lumped / Analytic / Coarse
● Parameters fitted to “high-fidelity” data
Simplified model
Correctionfunction
x f(x)
● Refinement: correction function, parameter functions ...
![Page 35: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/35.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Model simplification summary
Many different approaches:
– Local: Taylor series (needs derivatives)
– Interpolation (exact fit):
(Polynomial) fitting
Kriging
– Fitting: LS
– Approximate modeling: reduced order / simplified models
– Other: genetic programming, neural nets, etc
![Page 36: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/36.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Response surfaces in optimization● Popular approach for
computationally expen-sive problems:
1. DoE, generate samples (expensive) in part of domain
2. Build response surface (cheap)
3. Perform optimization on response surface (cheap)
4. Update domain of interest, and repeat
Expensivemodel
Optimizer
Cheapmodel
Optimizer
● Additional advantage: smoothens noisy responses
● Easy to combine with parallel computing
![Page 37: WB1440 Engineering Optimization – Concepts and Applications Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1.](https://reader034.fdocuments.us/reader034/viewer/2022051416/56649ed25503460f94be2068/html5/thumbnails/37.jpg)
WB1440 Engineering Optimization – Concepts and Applications
Example: Multi-point
Approximation Method
Trust region
Design domain
Response surface
Sub-optimal point
Optimum
(Expensive) simulation