Asymptotic Property of Semiparametric ... -...
Transcript of Asymptotic Property of Semiparametric ... -...
Applied Mathematical Sciences, Vol. 9, 2015, no. 50, 2477 - 2491
HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2015.52104
Asymptotic Property of Semiparametric
Bootstrapping Kriging Variance in
Deterministic Simulation
Elmanani Simamora
Department of Mathematics
State University of Medan North Sumatera, Indonesia
Subanar
Department of Mathematics
Gadjah Mada University Yogyakarta, Indonesia
Sri Haryatmi Kartiko
Department of Mathematics
Gadjah Mada University Yogyakarta, Indonesia
Copyright © 2015 Elmanani Simamora, Subanar and Sri Haryatmi Kartiko. This is an open
access article distributed under the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original work is
properly cited.
Abstract
Plug-in kriging variance underestimates true kriging variance. This
underestimator happens because kriging plug-in predictor ignores the randomness
of errors or uncertainty of outputs in the locations of observed data. The correct
kriging variance is proposed using semiparametric bootstrapping procedure. The
simulation result for increasing observed I/O data location shows three properties,
which are: (i) the values of generic estimation of kriging variance, semiparametric
bootstrapping kriging variance, is always bigger than plug-in kriging variance, (ii)
the decline of the estimation values of both estimators tends to be zero, and (iii)
conditional number of correlation matrix increases, enabling matrix in ill
condition. One of the causes of ill condition is rounding error in expensive
computation, decreasing the accuracy of the estimation, even causing loss of the
Corresponding author: Elmanani Simamora, Department of Mathematics, State University of Medan, North Sumatera, Indonesia. E-mail: [email protected]
2478 Elmanani Simamora et al.
solution of kriging equation system. With the assumption that computation aspect
is ignored, ill condition, it can be analytically shown that the asymptotic property
of both estimators, i.e: plug-in kriging variance and generic estimator, are
consistent to zero.
Keywords: Kriging, Variance, Bootstrapping, Semiparametric, Asymptotic
1. Introduction
In kriging model, model parameters are usually not known for certain, but
they are estimated based on observed I/O data (sample) instead. For example,
model parameters are stated in vector [ , , ]T , while ˆ ˆ ˆˆ[ , , ]T states
the estimators. , and notations state parameters of regresion, process
variance, and correlation, respectively. Plugging-in ˆ into kriging predictor
produces estimators of Emperical Best Linear Unbiased Predictor (EBLUP)
kriging variance, it is stated that plug-in kriging variance then underestimates true
kriging variance. Hertog [1] show this analytically in conditional expectation
framework. We bring forward this evidence again in the next part by using true
kriging variance description in simpler expectation.
This underestimator is caused by kriging prediction which is exact
interpolation in deterministic simulation, which ignores randomness of errors or
uncertainty of outputs in the locations of observed data. Several researchers in
deterministic simulation have corrected this underestimation using parametric
bootstrapping to consider output uncertainty in the locations of observed data, e.g.
[1], [2], [3]. For general discussion on bootstrapping, they refer to [4]. Their
purposes are to give generic estimators of kriging variance, typically kriging
variance is unknown, closer to the truth and gives false statement for formulation
of plug-in kriging variance. Unfortunately, their generic estimators are very
different from plug-in kriging variance when process variance estimation, ̂ , is
very large.
We, Simamora et al. [5], [6], propose a new procedure to generate output
uncertainty in the locations of observed data using semiparametric bootstrapping
in deterministic simulation. For general discussion on bootstrapping, we refer to
[7], [8], [9]. We, Simamora [6], have comparatively studied parametric and
semiparametric bootstrapping procedures. Essentially, we show that
semiparametric bootstrapping procedure is better than parametric. The aspect
measured in that study is the performances of both bootstrappings in (i) estimation
values of both generic estimators, and (ii) coverage probability and length of
confidence interval estimations.
This paper is a continuation of Simamora et al. [5], [6] which will be
presented in two sections, namely simulation and analytic sections. The
simulation section will consider the properties of simulation results if observed
I/O data increases. Meanwhile, the properties which will be considered are (i) values of generic estimation, semiparametric bootstrapping kriging variance, vs plug-
Asymptotic property of semiparametric bootstrapping kriging variance 2479
in kriging variance, (ii) trend of estimation values of both estimators, and (iii)
conditional number of correlation matrix. Normally, increasing size of observed
data causes conditional number of correlation matrix increases, enabling matrix to
be in ill condition. One of the causes of ill condition is rounding error in
expensive computation, so estimation accuracy decreases and even there is loss of
solution of kriging equation system. Lophaven [10], [11] offers ill condition
correlation matrix regulation so that conditional number can be suppressed when
observed I/O data is quite large. Meanwhile, Zimmermann [12] gives regulation
of maximum likelihood estimator limits of correlation function parameters if the
conditional numbers are leading to infinite. For analytic section, we assume
computational aspects are not considered and conducted it more natural.
The size of observed I/O data leading toward infinite will show that the
property of both plug-in and generic kriging variance estimators are consistent to
zero. The size of bootstrap sample leading toward infinite can reduce estimation
uncertainty or in other words will yield ideal estimation of ideal bootstrap, (see [4],
p. 50-53).
This paper is arranged as follows Section 1 provides research background.
Section 2 summarizes kriging model (including notations and terms), deriving the
formula of true kriging variance which is Best Linear Unbiased Predictor (BLUP)
and plug-in kriging variance. Section 3 introduces generic estimator algorithm of
kriging variance by [6]. Section 4 presents simulation result. Section 5 presents
several propositions for analytic studies. Lastly, we draw conclusion.
2. Kriging Models
Nowadays, kriging method is widely used in various fields, such as
environment, hydrology, geostatistic, engineering, research operation, and
economy. The term krigeage (kriging) was coined by Matheron (1963) to respect
the name of the inventor, Prof D.G Krige, a mining engineer from South Africa,
(see [13], p.50).
Sacks et al. (1989) apply kriging on deterministic simulation model where
kriging is an exact interpolation which does not have any observation error
(nugget effect) in observed I/O data. Several symbols and terminologies in this
section are taken from [10], [14]. Output (response) ( )y x as realization of
stochastic process Y :
0
( ) ( ) ( )p
i i
i
Y x g x Z x
, (1)
where x is input variable with d -dimension. For example
0( ) [g ( ) ( )]T
pg x x g x
expresses selected function and 0[ ]T
p is regression parameter.
Modelling (1) is the sum of ( )T g x as linear regression model and ( )Z x as
stochastic process. Assume that ( )Z x has E[ ( )] 0Z x and 2E[ ( ) ( )]=Z x Z x .
2480 Elmanani Simamora et al.
x and t covariances, two different input variables are notated as 2( , ) ( , )C x t R x t with ( , )R x t as the correlation between x and t . Lophaven
[10] provides several models of correlation function. In this paper, we choose
Gaussian correlation function model
2
1
, , exp i i
i
d
iR x t x t
. (2)
Expansion of n -experiment location, 1[ ]T
n nX x x as observed input
data produces ( 1) 1[g( ) ( )]T
n p nG x g x and , 1[ ( , , )]n
n n i j i jR x x R which state
function matrix and correlation matrix, respectively. For example 0 nx X , an
untried point, so correlation vector of every ( )iZ x in nX with 0( )Z x is
expressed as
0 1 0 0( ) [ ( , , ) ( , , )]T
nr x R x x R x x .
Kriging prediction in 0x is stated as
0
ˆ( ) ( )T T
Xy x Y G Z , (3)
where 1[ ] n T is weight vector,
1[ ( ) ( )]T
X nY y x y x is observed output
data and 1[ ( ) ( )]T
nZ Z x Z x as error vector in n -experiment location. Kriging
prediction error is stated as 0 0ˆ( ) ( )y x y x , with
0 0 0() )( ) (Ty x g x Z x . Kriging
predictors are called BLUP if they minimize Mean Squared Error Prediction
(MSPE),
2
0 0ˆmin [( ( ) ( )) ]E y x y x
2
0min 1 2 ( )T T r x
R (4)
under one similarity constraint condition
0 0ˆ[ ( ) ( )] 0E y x y x
0( ) 0T g xG . (5)
For optimization problem (4) with one similarity constraint (5), we introduce
lagrangian function
0
2
01 2 ( ) (, ) )( T T T Tr x G g x R , (6)
where states lagrange multiplier vector. For example, * solution of (4) and * which are suitable Lagrange multiplier vector based on first order necessary
condition of optimization problem, (see [15]) produces
Asymptotic property of semiparametric bootstrapping kriging variance 2481
* * 2
0
** ( )( , ) 2 0Gr x R
*
0( ) 0T g xG .
This last form is generally called kriging equation system in [16]. If it is rewritten
as a matrix, it will be
*
0*
02
( )
( )2
T
r xG
g xG
R
0. (7)
Solution of (7) is
1 1*
1
02 0) ( ) (( )2
T TG G x g xG r
R R
*
* 1
0 2( )
2
Gr x
R . (8)
Substitute (8) to (3), so kriging predictor can be derived as
* 1
0 0 0ˆ( ) ( ) ( )T T T
X Xy x Y g x r x Y G R . (9)
MSPE of kriging prediction 0ˆ( )y x is
* * *
0 0
1 1 1
2
2
0 0
ˆMSPE ( ) 1 2 ( )
1 ( ) ( ) ( ) ,
T T
T T T
y x r x
G G r x r x
R
R R (10)
where 1
0 0( ) ( )T r x gG x R .
Based on selection of correlation function model (2) kriging predictor uses
maximum likelihood method to estimate , where likelihood function of model
parameter is
2 1
2
1 1ln ln| |
2 2 2
T
XX
nL Y G R Y G
R . (11)
MLE method is used to discover optimum estimators of correlation function
model parameter. Maximum likelihood estimators (MLEs), ˆ ˆ ˆˆ[ , , ]T , of
log-likehood of (11) produces rather complicated calculation. We use MATLAB
DACE Toolbox from [10].
Plugging-in MLE ˆ into kriging predictor (9) produces
1
0 Plug-in 0 0 0ˆ ˆ ˆˆˆ ˆ ˆ( ) ( , ) ( ) ( )T T
Xy x y x g x r x Y G R , (12)
while plugging-in MLE ˆ into (10)
0 0 Plug-inPlug-in
1 1 1
0
2
0
ˆ ˆMSPE ( ) MSPE ( ), m
ˆ ˆˆ ˆ ˆ ˆ1
ˆ
ˆ ( ) ( ) ( ) ,
n
n
T T T
y x y x
G G r x r x
R R (13)
makes MSPE (12), which is called plug-in kriging variance, underestimated.
2482 Elmanani Simamora et al.
Proposition 1
For example 0ˆMSPE ( )y x
is true kriging variance at untried point 0x
which is
generally unknown. Plugging-in ˆ into kriging predictor produces
underestimated or biased 0ˆMSPE ( )y x estimators.
Proof
Note 2
0 0 0ˆ ˆMSPE( ( )) E ( ( ) ( )y x y x y x , modifying a part of the equation will
produce 2
0 0 0ˆ ˆMSPE( ( )) E ( ( ) ( )y x y x y x
2
0 Plug-in 0 Plug-in 0 Plug-in 0 P
2
0 0 lu 0- n0 g iˆ ˆ ˆ ˆ( ) ) 2 ( ) ) ( ) )ˆ ˆE (( ( ) (( ( ) ( ( ) ( (( )))y x y x y x yy x y x y xx y x
2
0 Plug-in 0 Plug-in 0 Plug-in 0 Pl
2
0 0 u0 0g-inˆ ˆE[( ( ) (ˆ ˆ ˆ ˆ( ) ] 2E[ ( )( ) ( )] (]E[ ( ) E[ ( ) )]y x y x y xy x x y x y xy xy ,
because kriging predictor is unbiased, so 0 0)( ] 0, )E[ (yy xx , producing
2
0 Plug-in 0 Plug-in
2
0 0ˆ ˆ( ) ] EˆE[( ( ) ( )][ ( )y xy x y yx x
2
0 Plug-in 0 0Plug-in Pl0 ug-inˆ ˆ ˆ( ) ] MSPE ( ) MSPE ( )ˆE[( ( ) y x y x yx xy . (Q.E.D).
3. Algorithm of Generic Estimator of Kriging Variance
In this paper, we adopt directly-semiparametric bootstrapping algorithm
from [6] without modification because this study is a continuation of [5], [6].
Directly-Semiparametric Bootstrapping Algorithm
1. Based on observed I/O data, determine MLE ˆ ˆ ˆˆ[ , , ]T to estimate the
distribution of Y(x).
2. Based on step 1, determine 2 ˆˆ ˆ T
n n R L L , nL is lower than triangular
matrix sized n n based on Cholesky decomposition.
3. Determine 1
1[ ]T
n X nU Y u u L called decorrelation transformation.
4. Determine 1[ ]T
nU u u , where
n
jj i
i i
uu u
n
.
5. Sample *
1
b
nU of step 4, amounting to 1,2, ,b B .
6. Determine 1 2
ˆ ˆˆˆ ˆ
n T
c
c
, where 2
0ˆˆ ( )ˆ rc x , to get 1nL with Cholesky
decomposition.
7. Sample * * *
0 1 1[ , ( )]b b T b
X n nY y x U L .
8. By using bootstrap sample *b
XY match a kriging model.
9. Based on step 8, determine kriging prediction in 0x , say *
0ˆ ( )by x .
10. Based on B bootstrap sample, calculate generic estimator based on formula
Asymptotic property of semiparametric bootstrapping kriging variance 2483
* * * 2
0 SPB 0 01
1ˆ ˆMSPE( ( )) m ( ( ) ( ))
Bb b b
by x y x y x
B . (14)
4. Simulation Results
Simulation-optimization requires relatively long considerable computation
time. This is because kriging predictors calculate MLEs numerically using
complicated constrained maximization algorithm. Considering this, we only plot
two experimental designs for observed I/O data, where data size increases in every
input with one, two and three dimensions. All input point experimental designs
follow [6], but with increased size of input experimental design.
In this simulation, the accuracy of kriging system solution (7) is influenced
by conditional number of correlation matrix R̂ . The magnitude of conditional
number, , is due to: errors in matrix elements, long algorithms, rounding (see
[17]). If is relatively big, kriging system problem (7) produces inaccurate
solution or even a loss in the solution. One of the considerations in this paper is the
amount of cumulated rounding, called rounding error, in expensive simulation
which enables correlation matrix R̂ in ill condition.
The natures observed in this simulation, the increase of the size of observed
I/O data, are (i) the values of generic estimation vs plug-in kriging variance, and (ii)
trend of estimation values of both estimators, (iii) conditional number of correlation
matrix.
4.1. Input Points with One Dimension
The experimental design used by [6] for input point with one dimension is
sampling strategy with the same distance as input space [0,10]. While selection of
multi-modal test function is taken from [1],
4 3 2( ) 0.0579 1.11 6.845 14.007 2f x x x x x ,
with initial value 0 1 and correlation parameter space (0,20] . The sizes of
locations of experimental design of observed I/O data are 4n and 9n with
untried point 0 1.1115x .
For bootstrap sample size 10000B , the simulation in Figure 1 shows that
(i) the value of generic estimation, SPBm , is always bigger than plug-in kriging
variance, Plug-inm , (ii) the decline of estimation values of both estimators tends to be
zero, and (iii) observed I/O data 4n produces 1 , while 9n produces
41.7926 .
2484 Elmanani Simamora et al.
Figure 1. Plots of generic estimators vs plug-in kriging variance where inputs
have one dimension and bootstrap sample size 10000B . (a) Observed I/O data
4n , (b) Observed I/O data 9n .
4.2. Input Points with Two Dimensions
Similar with experimental design for inputs with one dimension, we use
unmodified observed I/O data from [6]. Locations of experimental design of
observed I/O data are sized 25n and 41n with untried point 0 [-4.5 4.5]x .
With the input space is 2
1 2( , ) [ 5 10] [0 15]i i
x x R with response vector XY
Branin test function
2 2
1 2 2 1 1 12
5 5 1( , ) ( 6) 10(1 )cos 10
4 8i i i i i iy x x x x x x
with initial values 0 [2 2] , 1 1[10 10 ]lob and [10 10]upb .
For the same size of bootstrap sample as inputs with one dimension, Figure 2
shows the same behavior as inputs with one dimension. However, for observed I/O
data 25n , 41n produces 41.0972 10 and 12 7.992 10 respectively.
Figure 2. Plots of generic estimators vs plug-in kriging variance where inputs
have two dimensions and bootstrap sample sizes 10000B . (a) Observed I/O
data 25n , (b) Observed I/O data 41n .
0 2000 4000 6000 8000 100000
10
20
30
40
50
60
70
80
B
MSPE
(a)
mSPB
mPlug-in
0 2000 4000 6000 8000 100000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
B
MSPE
(b)
mSPB
mPlug-in
0 2000 4000 6000 8000 100000
100
200
300
400
500
600
700
800
900
1000
B
MSPE
(a)
0 2000 4000 6000 8000 100000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
B
MSPE
(b)
mSPB
mPlug-in
mSPB
mPlug-in
Asymptotic property of semiparametric bootstrapping kriging variance 2485
4.3. Input Points with Three Dimensions
Following the experimental locations of [6], for inputs with three dimensions,
the sizes of observed I/O data are 32n and 200n with untried point
0 [0.5 0.5 0.5]x . Multi-modal test function using Hartman 3 has 4 minimum
local and one minimum global 4 3
2
1 1
( ) exp( ( ) )i ij j ij
i j
f x K x H
, where (1.0,1.2,3.0,3.2)T
3.0 10 30
0.1 10 35
3.0 10 30
0.1 10 36
K
, 4
3689 1170 2673
4699 4387 747010
1091 8732 5547
381 5743 8828
H
where
3
0[0,1] [0,1] [0,1] ; [111]; ; [10 10 101 1 1 1 1 ]1lx R upbob e e e .
The treatment is the same as inputs with one and two dimensions for many
bootstrap sample, figure 3 shows the same behavior as inputs with one and two
dimensions. However, for observed I/O data 32n , 200n produces 5 1.506 10 , 15 5.1071 10 , respectively.
Figure 3. Plots of generic estimators vs plug-in kriging variance where inputs
have three dimensions and bootstrap sample size 10000B . (a) Observed I/O
data 32n , (b) Observed I/O data 200n .
5. Analytic Study
In this section, we assume that computation aspect is not considered and is
conducted in more natural way. This means that elements in correlation matrix R̂ ,
symmetric and positive definite (SPD), are assumed to have no rounding, so
inverse R̂ is always present. If this is the case, kriging system problem (7)
0 2000 4000 6000 8000 100004
5
6
7
8
9
10
11
12
13
14x 10
-3
B
MSPE
(a)
0 2000 4000 6000 8000 100000
0.5
1
1.5
2
2.5x 10
-3
B
MSPE
(b)
mSPB
mPlug-in
mSPB
mPlug-in
2486 Elmanani Simamora et al.
always has a solution when the size of I/O data is infinite. The main purpose is to
prove that the asymptotic property of plug-in kriging variance and generic
estimator are consistent to zero. In proving this, there are several tiered
propositions (hierarchy) which will be derived.
5.1. Asymptotic Property of Plug-in Kriging Variance
To prove asymptotic property of plug-in kriging variance is consistent to
zero, tiered propositions are made.
Proposition 2.
For example 1[ ]T
n nX x x is n-design location in input space , where d
i nx X and 0
n
nx X is an untried point in input space , so
1
0ˆ ˆm )i (l n n
n
nir ex
R
Where ie canonical vector of identity matrix I and i I a set on indices.
Proof
For n design location, there is i I so that 0
nx X . This causes
0( )nr x one of i-th column of ˆ
R , because 1ˆ ˆ
R R I so
1
0ˆ ˆ ( ) ,0, ,0,1,0, ,0i ,l m
T
n in
n
nr x e
R . (Q.E.D).
Proposition 3.
Due to proposition 2 it can be expressed that
1. 1
0 0ˆ ˆlim ( ( ) (m )i ) 0l n n
n nn
T
nnr xG g x
R .
2. 0 Plug-inˆlim limˆ mMSPE ( ), 0
n n
ny x
.
Proof
1. Based on proposition 2, 1
0ˆ ˆm )i (l n n
n
nir ex
R is produced, creating
1
0 0
1
0 0
0
0 0
ˆ ˆlim ( ( ) ( ))
ˆ ˆlim ( ) ( )
( )
( ) (
li
) 0. (Q.E.D)
m T
n
T
T
i
n n
n n nn
n n
n nn
n
n n
r x g x
r x g x
G
g x
g x g
G
e
x
G
R
R
2. Due to proposition 3.1 is obtained
Plug-in0ˆMSPE ( ), ˆlim lim m
n
n
ny x
Asymptotic property of semiparametric bootstrapping kriging variance 2487
1 1 1
0
2
0ˆ ˆˆ ˆli ˆm 1 ( ) ( ) ( )T T T n n
n n n n n nn
G G r x r x
R R
1 1 1
0 0
2 ˆ ˆˆ ˆ1 lim ( ( ) ) lim ( ( ))ˆ ( )T T T n n
n n n n n nn n
G G r x r x
R R
1
0 0
2 ˆˆ ˆ1 lim ( ( ) ( )ˆ )T n n
n n nn
r x r x
R
1
0 0
2 ˆˆ ˆ1 lim ( ( )).limˆ ( ( )) .T n n
n n nn n
r x r x
R
Because 0
1ˆl m ( )i n
n
nn ir x e
R and 0 :,
ˆˆlim ( )T n
n in
r x R
, it is obtained
0 :,Plug-in
2
2
2
ˆli ˆˆMm
ˆ
ˆ 0. (Q.E.D
SPE ( ) 1
ˆ1
1 ).1
n
n
i i
ii
y x R e
R
Based on the final proposition, it is shown that asymptotic property of plug-in
kriging variance are consistent to zero.
5.2. Asymptotic Property of Generic Estimator
In the same way as the above, tiered propositions are made to prove that
asymptotic property of generic estimator is consistent to zero.
Proposition 4.
If * * * *
0 1 1[ , ( )]n
n T b
X n nY y x U L is a multivariate sample of quasibootstrap, where
*
1
b
nL is obtained from the decomposition of Cholesky matrix
* **
1 * 2 *
ˆ ˆˆˆ ˆ( ) ( )
b bb
n b T b
c
c
,
it produces
1. * * *
0
1
ˆlim ( )b n b b
ij jn
j
y x l u
.
2. * * * * *
0 ( 1)( 1) 1
1
lim ( ) limb n b b b b
ij j n n nn n
j
y x l u l u
for a ni I .
Proof
1. If *
n
b
XY is an observed response of b-th bootstrap, then kriging predictor in
untried 0
nx is
* ** ** 1 *
0 0 0ˆ ˆˆˆ) ( ) ( ( )) (ˆ )( ) (
n
b T b Tb n n b b
n
n b
n Xg r Gy x x x Y R .
2488 Elmanani Simamora et al.
For n design location then
* * * 1 *
* * * 1 *
* * *
* *
0 0 0
1 *
*
0 0
*
0 0
ˆ ˆˆˆ) ( ) ( ( )) ( )
ˆ ˆˆˆ) ( ) ( ( )) ( )
ˆ ˆˆˆ) ( ) ( ( )) ( ) ,
ˆlim ( ) lim (
( lim
( lim .lim
n
n
n
b T b T b b
n n
b T b T b b
n n
b T b T b
b n n n b
Xn n
n n b
Xn
n n b
Xn n
b
n n
g r G
g r
y x x x Y
x x Y
x x Y
G
g r G
R
R
R
because * * 1
0ˆˆ( ( )li )m ) (n
n
b T b T
n n ir ex
R then
* ** *
* *
* *
0 0
*
0
*
0
*
0 0
* *
*
ˆlim ( ) ( lim
( lim
( lim
( lim
ˆ ˆ) ( ) .
ˆ ˆ) ( ) .
ˆ ˆ) ( )
ˆ ˆ) ( ) (
lim lim
) ( )
n
n
n
n
b n n b
Xn n
n b
Xn
n
b T T b
i
b T T b
i
b T T T b
i i
b T T b T
i
T T
i i
b
Xn
n b n
Xn
b
Xn n
g e G
g
y x x Y
x Y
x Y
x Y x
Y
e G
g e e G
g e g
e e
L** * *
1
. (Q.E.D).b b b
n ij j
b
j
nU l u
2. Note * *11 1
* ** *
* * 221 22
1 1*
0
** * *
1( 1)1 ( 1)2 ( 1)( 1)
0 0
0
( )
n
b b
b bb bX b b
n nb n
bb b b
nn n n n
l u
Y ul lU
y x
ul l l
L ,
produces
*
1 1* * * * *
0 ( 1)1 ( 1)( 1) ( 1)
1*
1
( )
b
nb n b b b b
n n n n j j
jb
n
u
y x l l l u
u
.
Based on the proof description of proposition 2, it can be concluded that * *
( 1)1 ( 1)lim b b
n n nn
l l
is equal to one of i-th lines of *lim b
nn
L , producing
* * * * *
0 ( 1)( 1) 1
1
lim ( ) limb n b b b b
ij j n n nn n
j
y x l u l u
. (Q.E.D).
Proposition 5.
If
*
1 1* * * * *
0 ( 1)1 ( 1)( 1) ( 1)
1*
1
( )
b
nb n b b b b
n n n n j j
jb
n
u
y x l l l u
u
then * *
( 1)( 1) 1lim 0b b
n n nn
l u
.
Asymptotic property of semiparametric bootstrapping kriging variance 2489
Proof
Note equation to find main diagonal element of matrix 1nL using
12 1/2
1
( )j
jj jj jj
k
l A l
,
for a matrix A. By using the same provision as *
1ˆ b
n then
1/2
* 2 * * * 2
( 1)( 1) ( 1)( 1) ( 1)
1
ˆ( ) ( )n
b b b b
n n n n n k
k
l R l
.
Because *
( 1)( 1)ˆ 1b
n nR and * 2 2 *
( 1)
1
ˆlim ( ) ( )n
b b
n kn
k
l
then produce
1/2
* * 2 * * 2 *
( 1)( 1) 1 ( 1) 1
1
ˆlim lim ( ) ) 0n
b b b b b
n n n n k nn n
k
l u l u
. (Q.E.D)
Proposition 6.
If n design location and 0B it will meet the following condition
* * * 2
0 0 01
1ˆ ˆlim MSPE( ( )) lim ( ( ) ( )) 0
Bn b n b n
bn nB B
y x y x y xB
.
Proof
Note
* * * 2
0 0 01
* * 2
0 01
1ˆ ˆlim MSPE( ( )) lim ( ( ) ( ))
1ˆlim lim( ( ) ( )) .
Bn b n b n
bn nB B
B b n b n
bB n
y x y x y xB
y x y xB
Due to propositions 4 is obtained
22* * * *
0 0 0 0
2
* * * * * *
( 1)( 1) 1
1 1
2* *
( 1)( 1) 1
ˆ ˆlim ( ) ( ) lim ( ) lim ( )
lim
lim ,
b n b n b n b n
n n n
b b b b b b
ij j ij j n n nn
j j
b b
n n nn
y x y x y x y x
l u l u l u
l u
and using propositions 5 is finally obtained *
0ˆlim MSPE( ( )) 0n
nB
y x
. (Q.E.D).
Based on the last proposition, it is shows that asymptotic property of generic
estimator of kriging variance, semiparametric bootstrapping kriging variance, are
consistent to zero.
2490 Elmanani Simamora et al.
6. Conclusion
From the results of expensive simulation-simulation optimization-where
observed I/O data increased, it can be concluded that (i) the values of generic
estimation of kriging variance are always bigger than plug-in kriging variance, (ii)
the decrease of estimation values of both estimators tend to be zero, and (iii)
conditional number of correlation matrix increased. This conditional number is
influenced by the amount of rounding in the solution of optimization and the
influence of input dimension. This can be seen in the simulation results for inputs
with more than one dimension, showing that conditional numbers are relatively
bigger.
With the assumption that computation aspect is ignored, correlation matrix
can be in ill condition. Analytically, it is shown that the asymptotic property of
both estimators using tiered propositions (hierarchy) is consistent to zero.
References
[1] D. Den Hertog, J.P. Kleijnen, and A. Y. D. Siem, The correct Kriging
variance estimated by bootstrapping, Journal of the Operational Research Society,
57.4 (2006), 400-409. http://dx.doi.org/10.1057/palgrave.jors.2601997
[2] J.P. Kleijnen and Ehsan Mehdad, Conditional Simulation for Efficient Global
Optimization, Proceedings of the 2013 Winter Simulation Conference: Simulation:
Making Decisions in a Complex World. IEEE Press (2013), 969-979.
http://dx.doi.org/10.1109/wsc.2013.6721487
[3] J.P. Kleijnen, Simulation-Optimization via Kriging and Bootstrapping: A
Survey, Journal of Simulation, 2014, 8.4 241-250.
http://dx.doi.org/10.1057/jos.2014.4
[4] Efron, Bradley, and Robert J. Tibshirani, An Introduction to the Bootstrap,
Chapman & Hall, New York, 1993.
[5] Elmanani Simamora, Subanar and S.H. Kartiko, The Procedure of Kriging
Variance Estimation Based on Semi parametric Bootstrapping in Deterministic
Simulation. International Journal of Applied Mathematics and Statistics™, 2014,
52.7: 99-110.
[6] Elmanani Simamora, Subanar and S.H. Kartiko, A Comparative Study of
Parametric and Semiparametric Bootstrapping in Deterministic Simulation.
Accepted-International Journal of Applied Mathematics and Statistics™, 2015.
[7] Solow, A. R., Bootstrapping Correlated Data. Mathematical Geology,17(7),
1985, 769-775. http://dx.doi.org/10.1007/bf01031616
Asymptotic property of semiparametric bootstrapping kriging variance 2491
[8] Schelin, L., & Sjöstedt-de Luna, S., Kriging Prediction Intervals Based on
Semiparametric Bootstrap, Mathematical Geosciences, 42(8), 2010, 985-1000.
http://dx.doi.org/10.1007/s11004-010-9302-9
[9] Iranpanah, N., Mansourian, A., Tashayo, B., & Haghighi, F. Spatial
Semi-parametric Bootstrap Method for Analysis of Kriging Predictor of Random
Field. Procedia Environmental Sciences 3 (2011), 81-86.
http://dx.doi.org/10.1016/j.proenv.2011.02.015
[10] Lophaven, S. N., H. B. Nielsen, and J. Sondergaard, DACE: a MATLAB
Kriging toolbox, version 2.0. Lyngby, Denmark: IMM Technical University of
Denmark, 2002.
[11] Lophaven, S. N., Nielsen, H. B., & J. Søndergaard, Aspects of the Matlab
Toolbox DACE. Informatics and Mathematical Modelling, Technical University
of Denmark, DTU, 2002.
[12] Zimmermann, R., Asymptotic Behavior of the Likelihood Function of
Covariance Matrices of Spatial Gaussian Processes, Journal of Applied
Mathematics 2010, 1-17. http://dx.doi.org/10.1155/2010/494070
[13] Forrester, A., A. S´obester, and A. Keane, Engineering Design via Surrogate
Modelling: A Practical Guide, John Wiley & Sons, 2008.
http://dx.doi.org/10.1002/9780470770801
[14] Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P., Design and Analysis
of Computer Experiments, Statistical Science, 1989, 409-423.
http://dx.doi.org/10.1214/ss/1177012413
[15] Nocedal. J, S.J. Wright, Numerical Optimization, Springer, New York, USA,
1999. http://dx.doi.org/10.1007/b98874
[16] Noel A. C. Cressie, Statistics for Spatial Data, Vol. 900, New York: Wiley,
1993.
[17] Ipsen, Ilse CF, Numerical matrix analysis: Linear systems and least squares,
Siam, 2009. http://dx.doi.org/10.1137/1.9780898717686
Received: February 17, 2015; Published: March 26, 2015