The Functional Innovation Methodology - Information Technology
Response surface methodology for functional data …roche/slides_RSM.pdfResponse surface methodology...
Transcript of Response surface methodology for functional data …roche/slides_RSM.pdfResponse surface methodology...
Response surface methodology for functional data withapplication to nuclear safety
Angelina Rochejoint work with Michel Marques (CEA Cadarache)
47e Journées de Statistique1-5 juin 2015
1 / 19
Outline
1 Motivation
2 Response surface methodology
3 Extension to the functional setting
4 Application to nuclear safety
2 / 19
Motivation
Outline
1 Motivation
2 Response surface methodology
3 Extension to the functional setting
4 Application to nuclear safety
3 / 19
Motivation
Motivation : application to nuclear safety
Behaviour of a nuclear reactor vessel in case of loss of coolant accident.
4 / 19
Motivation
Motivation : application to nuclear safety
Thermo-mecanical code (CEA) : simulate the behaviour of the vessel.
Input:Evolution of the temperature in the vessel t 7→ T(t);Evolution of the pressure t 7→ P(t);Evolution of heat transfer t 7→ H(t).
Output: margin factor Y ∈ R.
Aim: maximise the margin factor Y .
4 / 19
Response surface methodology
Outline
1 Motivation
2 Response surface methodology
3 Extension to the functional setting
4 Application to nuclear safety
5 / 19
Response surface methodology
Response surface methodologyBrief history
Box and Wilson (1950): optimal conditions for chemical experimentation→widely used in industry.
Sacks et. al (1989): Extension to numerical experiments
↪→ Bates et. al (1996): conception of electrical circuit;
↪→ Lee and Hajela (1996): conception of rotor blades...
Recent advances: Facer and Müller (2003), Khuri and Mukhopadhyay (2010),Georgiou, Stylianou and Aggarwal (2014).
6 / 19
Response surface methodology
Methodology
Goal: minimisation of (x1, . . . , xd) ∈ Rd 7→ m(x1, . . . , xd) ∈ R, unknown.
Additional experiments possible but with a cost, output observed with errory = m(x1, ..., xd) + ε.
Idea1 Definition of a departure point (x0,1, ..., x0,d) ∈ Rd and of a surrogate model
Order 1 (steepest ascent): y = b0 +∑d
j=1 bjxj + ε;
Order 2 (optimisation model): y =∑d
j=1 bjxj +∑
1≤i<j≤d bi,j +∑d
j=1 bi,ix2j + ε;
...
2 Realisation of new experiments around (x0,1, ..., x0,d) ∈ Rd ↪→ choice of Design ofExperiments (DoE).
3 Least-squares fit of the model coefficient and optimisation : definition of a new point(x1,1, ..., x1,d).
7 / 19
Extension to the functional setting
Outline
1 Motivation
2 Response surface methodology
3 Extension to the functional setting
4 Application to nuclear safety
8 / 19
Extension to the functional setting
Problems raised by the functional context
First and second-order models can be defined easily but... How to define functional design of experiments ?One possible answer: combine dimension reduction with classicalfinite-dimensional design of experiments
(x(i)0 = (x(i)
0,1, . . . , x(i)0,d) ∈ Rd, i = 1, . . . , n0) d-dimensional design of experiments;
{ϕ1, . . . , ϕd} orthonormal family of H
x(i)0 = x0 +
d∑j=1
x(i)0,jϕj,
−→ functional design of experiments.... How can we define the directions {ϕ1, . . . , ϕd} ?Possible basis of approximation
Fixed basis: Fourier, B-splines, wavelets,...If a training sample exists: data driven basis
PCA basis;PLS basis Wold (1975), Preda and Saporta (2005), Delaigle and Hall (2012): allows totake into account the interaction between x and y.
Random directions...
9 / 19
Extension to the functional setting
Problems raised by the functional context
First and second-order models can be defined easily but... How to define functional design of experiments ?One possible answer: combine dimension reduction with classicalfinite-dimensional design of experiments
(x(i)0 = (x(i)
0,1, . . . , x(i)0,d) ∈ Rd, i = 1, . . . , n0) d-dimensional design of experiments;
{ϕ1, . . . , ϕd} orthonormal family of H
x(i)0 = x0 +
d∑j=1
x(i)0,jϕj,
−→ functional design of experiments.... How can we define the directions {ϕ1, . . . , ϕd} ?Possible basis of approximation
Fixed basis: Fourier, B-splines, wavelets,...If a training sample exists: data driven basis
PCA basis;PLS basis Wold (1975), Preda and Saporta (2005), Delaigle and Hall (2012): allows totake into account the interaction between x and y.
Random directions...
9 / 19
Extension to the functional setting
Problems raised by the functional context
First and second-order models can be defined easily but... How to define functional design of experiments ?One possible answer: combine dimension reduction with classicalfinite-dimensional design of experiments
(x(i)0 = (x(i)
0,1, . . . , x(i)0,d) ∈ Rd, i = 1, . . . , n0) d-dimensional design of experiments;
{ϕ1, . . . , ϕd} orthonormal family of H
x(i)0 = x0 +
d∑j=1
x(i)0,jϕj,
−→ functional design of experiments.... How can we define the directions {ϕ1, . . . , ϕd} ?Possible basis of approximation
Fixed basis: Fourier, B-splines, wavelets,...If a training sample exists: data driven basis
PCA basis;PLS basis Wold (1975), Preda and Saporta (2005), Delaigle and Hall (2012): allows totake into account the interaction between x and y.
Random directions...
9 / 19
Extension to the functional setting
Example of functional design of experimentsFactorial 2d design in H = L2([0, 1])
d = 2, 16 curves d = 4, 32 curves d = 8, 280 curves
Fourier
0.0 0.2 0.4 0.6 0.8 1.0
−2−1
01
2
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−4−2
02
4
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−50
5
t
x i(t)
PCA1
0.0 0.2 0.4 0.6 0.8 1.0
−3−2
−10
12
3
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0−6
−4−2
02
46
t
x i(t)0.0 0.2 0.4 0.6 0.8 1.0
−10−5
05
10
t
x i(t)
PLS2
0.0 0.2 0.4 0.6 0.8 1.0
−2−1
01
2
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−6−4
−20
24
6
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−10−5
05
10
t
x i(t)
X brownian motion, Y = ‖X − f‖2 + ε, f (t) = cos(4πt) + 3 sin(πt) + 10, ε ∼ N (0, 0.01)
1calculated from (Xi)500i=1
2calculated from (Xi, Yi)500i=1
10 / 19
Extension to the functional setting
Example of functional design of experimentsCentral Composite Designs in H = L2([0, 1])
d = 2, 4 curves d = 4, 16 curves d = 8, 256 curves
Fourier
0.0 0.2 0.4 0.6 0.8 1.0
−2−1
01
2
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−4−2
02
4
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−50
5
t
x i(t)
PCA3
0.0 0.2 0.4 0.6 0.8 1.0
−3−2
−10
12
3
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0−6
−4−2
02
46
t
x i(t)0.0 0.2 0.4 0.6 0.8 1.0
−10−5
05
10
t
x i(t)
PLS4
0.0 0.2 0.4 0.6 0.8 1.0
−2−1
01
2
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−6−4
−20
24
6
t
x i(t)
0.0 0.2 0.4 0.6 0.8 1.0
−10−5
05
10
t
x i(t)
X brownian motion, Y = ‖X − f‖2 + ε, f (t) = cos(4πt) + 3 sin(πt) + 10, ε ∼ N (0, 0.01)
3calculated from (Xi)500i=1
4calculated from (Xi, Yi)500i=1
11 / 19
Extension to the functional setting
Properties of the functional DoE
For multivariate inputs, all polynomial models can be written in a matrix form
Y = Xβ + ε. (1)
The design properties are based on the matrix X.
Orthogonality: XtX is diagonal.
Rotatability : Var(y(x)) depends only on ‖x‖.
D-optimality criterion: det(XtX) is maximal....
With our definition of functional DoE, the model can also be written in theform (1), with the same design matrix X as the multivariate DoE.
12 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
−10
−50
510
1520
t
x(t)
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Initial point
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
−10
−50
510
1520
t
x(t)
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Initial pointMinimal point f (t) (target)
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
−10
−50
510
1520
t
x(t)
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Initial pointMinimal point f (t) (target)28 factorial design5
5directions : PLS basis calculated from (Xi,m(Xi) + εi)500i=1 (Xi brownian motion)
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
−10
−50
510
1520
t
x(t)
Least-squares fit of a firstorder model → estima-tion of direction of steep-est descent
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Initial pointMinimal point f (t) (target)
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
−10
−50
510
1520
t
x(t)
Observed response on descent path:
0.0 0.2 0.4 0.6 0.8
01
03
05
0
α
ob
se
rve
d r
esp
on
se
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Initial pointMinimal point f (t) (target)Points of the descent direction
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
−10
−50
510
1520
t
x(t)
Observed response on descent path:
0.0 0.2 0.4 0.6 0.8
01
03
05
0
α
ob
se
rve
d r
esp
on
se
Minimal point of the descent direction
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Minimal point of the descent directionMinimal point f (t) (target)Points of the descent direction
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
05
1015
2025
t
x(t)
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Minimal point of the descent directionMinimal point f (t) (target)
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
05
1015
2025
t
x(t)
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Minimal point of the descent directionMinimal point f (t) (target)
} Central Composite Design5
5directions : PLS basis calculated from (Xi,m(Xi) + εi)500i=1 (Xi brownian motion, d = 8)
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
05
1015
2025
t
x(t)
Least-squares fit of a sec-ond order model → esti-mation of stationary point
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Minimal point of the descent directionMinimal point f (t) (target)
} Central Composite Design5
5directions : PLS basis calculated from (Xi,m(Xi) + εi)500i=1 (Xi brownian motion, d = 8)
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
05
1015
2025
t
x(t)
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Minimal point of the descent directionMinimal point f (t) (target)Stationary point (estimation of the minimalpoint)
13 / 19
Extension to the functional setting
MethodologyAdaptation to a functional context
Goal: minimisation of x 7→ m(x), unknown.
0.0 0.2 0.4 0.6 0.8 1.0
46
810
1214
16
t
x(t)
0
12
0 100 200 300 400 500
02
04
0
number of experiments
va
lue
of
the
re
sp
on
se 0
1 2
Example:m(x) = ‖x− f‖2 with
f (t) = cos(4πt)+3 sin(πt)+10;
ε ∼ N (0, 10).
Legend:Step pointsMinimal point f (t) (target)
13 / 19
Extension to the functional setting
Choice of dimension d : 2d factorial design
Monte-Carlo study of response improvement m(x(0)0 )−m(x(1)
0 )
m(x(0)0 )
after the first descent step.
m(x) = ‖x− f‖2
f (t) = cos(4πt) + 3 sin(πt) + 10
2 3 4 5 6 7 8 9 10 11
98.5
99.0
99.5
100.
0
PLS
f (t) = cos(8.5πt) ln(4t2 + 10)
2 3 4 5 6 7 8 9 10 11
020
4060
8010
0
PLS
14 / 19
Extension to the functional setting
Choice of dimension d : 2d factorial design
Monte-Carlo study of response improvement m(x(0)0 )−m(x(1)
0 )
m(x(0)0 )
after the first descent step.
m(x) = ‖x− f‖2
f (t) = cos(4πt) + 3 sin(πt) + 10
2 3 4 5 6 7 8 9 10 11
98.5
99.0
99.5
100.
0
PLS
f (t) = cos(8.5πt) ln(4t2 + 10)
2 3 4 5 6 7 8 9 10 11
020
4060
8010
0
PLS
Exponential increase of the size of design with the dimension.
14 / 19
Extension to the functional setting
Choice of dimension d : fractional factorial design
Monte-Carlo study of response improvement m(x(0)0 )−m(x(1)
0 )
m(x(0)0 )
after the first descent step.
m(x) = ‖x− f‖2
f (t) = cos(4πt) + 3 sin(πt) + 10
24 2(5−1) 2(6−2) 2(7−3) 2(8−4)
99.2
99.4
99.6
99.8
100.0
factorial design
perce
ntage
of im
prove
ment
f (t) = cos(8.5πt) ln(4t2 + 10)
24 2(5−1) 2(6−2) 2(7−3) 2(8−4)
020
4060
factorial design
perce
ntage
of im
prove
ment
Generation of factorial designs : minimal aberration design (package FrF2, Grömping, 2014).
15 / 19
Application to nuclear safety
Outline
1 Motivation
2 Response surface methodology
3 Extension to the functional setting
4 Application to nuclear safety
16 / 19
Application to nuclear safety
Design of Experiments
Directions : PLS basis of training samples (Ti,Yi)1≤i≤n, (Pi,Yi)1≤i≤n,(Hi,Yi)1≤i≤n, i = 1, ..., n, n = 195.
Limited number of possible additional experiments (about 200) : we fix heren0 = 128 = 27.
Combination of multivariate designs :Temperature 210−5 factorial design;
Pressure 24−2 factorial design;
Heat transfer 23−2 factorial design.
Temperature Pressure Heat transfer
0 1000 2000 3000 4000 5000
5010
020
0
t(s)
Tem
pera
ture
Te(
t)
0 1000 2000 3000 4000 5000
5.0e
+06
1.5e
+07
t(s)
Pre
ssio
n P
(t)
0 1000 2000 3000 4000 5000
050
000
1000
0015
0000
t(s)Tr
ansf
ert t
herm
ique
H(t
)
17 / 19
Application to nuclear safety
Estimation of gradient
First-order surrogate model : Yi = 〈βT ,Ti〉+ 〈βP,Pi〉+ 〈βH,Hi〉+ εi
Least-squares estimate of βP, βT , βH .
Direction of steepest ascent : Ti = T0 + α0βT , Pi = P0 + α0βP andHi = H0 + α0βH .
0 1000 2000 3000 4000 5000
5010
015
020
025
030
0
temps
x0Te
100 200 300 400 500
1.45
1.50
1.55
1.60
α0
Val
ue o
f the
res
pons
e
estimated steepest ascent dir.estimation of the optimum
18 / 19
Conclusion and perspectives
Conclusion
Adaptation of response surface methodology to context where the input are curves.
Definition of functional DoE by combining multivariate DoE and dimensionreduction techniques.
Application to simulated data and to a thermo-mechanical code of the CEA.
PerspectivesSelection of dimension/coefficients :
Theoretical study of the minimisation procedure;
Sensitivity analysis.
Constrained DoE.
More complex surrogate models.
More information: Roche, A (2015). Response surface methodology forfunctional data with application to nuclear safety, prépublication MAP5 2015-11.
Thank you for your attention !
19 / 19