Nonparametric Methods II 1 Henry Horng-Shing Lu Institute of Statistics National Chiao Tung...
-
Upload
vincent-shanon-murphy -
Category
Documents
-
view
214 -
download
0
Transcript of Nonparametric Methods II 1 Henry Horng-Shing Lu Institute of Statistics National Chiao Tung...
Nonparametric Methods II
1
Henry Horng-Shing LuInstitute of Statistics
National Chiao Tung [email protected]
http://tigpbp.iis.sinica.edu.tw/courses.htm
PART 3: Statistical Inference by Bootstrap Methods References Pros and Cons Bootstrap Confidence Intervals Bootstrap Tests
2
References Efron, B. (1979). "Bootstrap Methods:
Another Look at the Jackknife". The Annals of Statistics 7 (1): 1–26.
Efron, B.; Tibshirani, R. (1993). An Introduction to the Bootstrap. Chapman & Hall/CRC.
Chernick, M. R. (1999). Bootstrap Methods, A practitioner's guide. Wiley Series in Probability and Statistics.
3
Pros (1) In statistics, bootstrapping is a modern,
computer-intensive, general purpose approach to statistical inference, falling within a broader class of re-sampling methods.
http://en.wikipedia.org/wiki/Bootstrapping_(statistics)
4
Pros (2) The advantage of bootstrapping over
analytical method is its great simplicity - it is straightforward to apply the bootstrap to derive estimates of standard errors and confidence intervals for complex estimators of complex parameters of the distribution, such as percentile points, proportions, odds ratio, and correlation coefficients.
http://en.wikipedia.org/wiki/Bootstrapping_(statistics)
5
Cons The disadvantage of bootstrapping is that
while (under some conditions) it is asymptotically consistent, it does not provide general finite sample guarantees, and has a tendency to be overly optimistic.
http://en.wikipedia.org/wiki/Bootstrapping_(statistics)
6
How many bootstrap samples is enough? As a general guideline, 1000 samples is
often enough for a first look. However, if the results really matter, as many samples as is reasonable given available computing power and time should be used.
http://en.wikipedia.org/wiki/Bootstrapping_(statistics)
7
Bootstrap Confidence Intervals1. A Simple Method2. Transformation Methods
2.1. The Percentile Method 2.2. The BC Percentile Method 2.3. The BCa Percentile Method 2.4. The ABC Method (See the book: An
Introduction to the Bootstrap.)
8
1. A Simple Method Methodology Flowchart R codes C codes
9
Normal Distribution
10
2 21 2
2
/2 /2
1
/2 /2
, , ..., ~ ( , ), is known.
ˆˆ ~ ( , ), ~ (0, 1).
/ˆ
( ) 1/
where 1
ˆ ˆ( / / ) 1
iid
n
LCL UCL
X X X N
X N Z Nn n
P z zn
Z
P z n z n
Asymptotic CI for The MLE More generally,
i.i.d.Let is MLE, then
http://en.wikipedia.org/wiki/Pivotal_quantity
11
1 2, , , nX X X F x
ˆ
0,1ˆ. .
nX N
s e
ˆ
ˆ ˆ
ˆ1 as
ˆ ˆ 1 as
P Z Z n
P Z Z n
Bootstrap Confidence Intervals When n is not large, we can construct more
precise confidence intervals by bootstrap methods for many statistics including the MLE and others.
12
Simple Methods (1) Theorem in Gill (1989):
Under regular conditions,
Want
13
*1
ˆ ,
ˆ ˆ , , .
on
on n
n F d F B F
n X X d F B F
1P LCL UCL
Simple Methods (2)Note that
14
* * **
12 2
* * *
12 2
* *
12 2
ˆ ˆ ˆ ˆ ˆ ˆ1
ˆ ˆ ˆ ˆ ˆ
ˆ ˆ ˆ ˆ 2 2
P
P
P
P LCL UCL
An Example by The Simple Method (1)
Resampling with replacement from
Repeat times,we can get .
15
11 2 101
11 2 101 51
1, , , ~ , median
2
1ˆ, 2
iid
n
X X X N F
X X X F X
1 101, ,X X
* * *1 2 101
* * 1 *51
1ˆ2n
X X X
F X
1000B
* * *1 2 1000
ˆ ˆ ˆ
An Example by The Simple Method (2)
is an approximate confidence interval for .
16
*(1) *
(1000)*(25) *
(975)
95%
* * ** 25 975
* * ** 25 975
* * * *25 975 975 25
* *975 25
ˆ ˆ ˆ 1
ˆ ˆ ˆ ˆ ˆ ˆ
ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ2 2
ˆ ˆ ˆ ˆ2 , 2
P
P
P P
LCL UCL
100 1 %
Flowchart of The Simple Method
1717
*2x
*Bx
*(2)
1 2ˆ ( , , ..., ) ( )ndata x x x s x x
*1x
resample B times
*(1)
1 2[( 1) / 2], [( 1)(1 / 2)]v B v B
2 1
* *( ) ( )
ˆ ˆ ˆ ˆ2 , 2v vLCL UCL
*( )ˆB*
(2)
get resample statistics and then sort them * *b bs x
confidence interval 100 1 %
The Simple Method by R (1)
18
The Simple Method by R (2) Example
19
The Simple Method by C (1)
20
resample B times:
* *ˆ ( )b bmean x
*bx
ˆ ( ) ( )s x mean x
The Simple Method by C (2)
21
calculate v1, v2 and interval
The Simple Method by C (3)
22
The Simple Method by C (4)
23
2. Transformation Methods 2.1. The Percentile Method 2.2. The BC Percentile Method 2.3. The BCa Percentile Method
24
2.1. The Percentile Method Methodology Flowchart R codes C codes
25
The Percentile Method (1) The interval between the 2.5% and 97.5%
percentiles of the bootstrap distribution of a statistic is a 95% bootstrap percentile confidence interval for the corresponding parameter. Use this method when the bootstrap estimate of bias is small.
http://bcs.whfreeman.com/ips5e/content/cat_080/pdf/moore14.pdf
26
The Percentile Method (2) Suppose
Then
Assume that there exists an unbiased and (monotonly) increasing function such that .
27
ˆ ~Y H
~H Y U
1 1~ ~ 0,1H Y U N
g
ˆ 0,1g g N
The Percentile Method (3) If , then
28
ˆ 0,1g g N *ˆ ˆ 0,1g g N
1
**
* 1 ** 1 1
11
1 *1 1 1
ˆ ˆ 1
ˆ ˆ ˆ ˆ and
ˆ
ˆ Notice: for 0,1
ˆ ˆ ˆ and
B
B
P g g z
P g g z
P g g z
P g g z z z N
P g g z
The Percentile Method (4) Similarly,
and
Summary of the percentile method:
29
*
1 1ˆ 1
BP
* *
1 /2 1 1 /2ˆ ˆ 1
B BP
*
1
*
1 1
* *
1 /2 1 1 /2
ˆ 1
ˆ 1
ˆ ˆ 1
B
B
B B
P
P
P
Flowchart of The Percentile Method
3030
*2x
*Bx
*(2)
1 2ˆ ( , , ..., ) ( )ndata x x x s x x
*1x
resample B times
*(1)
1 2[( 1) / 2], [( 1)(1 / 2)]v B v B
*( )ˆB*
(2)
get resample statistics and then sort them * *b bs x
confidence interval 100 1 %
1 2
* *( ) ( )ˆ ˆ,v vLCL UCL
The Percentile Method by R (1)
31
The Percentile Method by R (2) Example
32
The Percentile Method by C (1)
33
calculate v1, v2 and interval
The Percentile Method by C (2)
34
The Percentile Method by C (3)
35
2.2. The BC Percentile Method Methodology Flowchart R code
36
The BC Percentile Method Stands for the bias-corrected percentile
method. This is a special case of the BCa percentile method which will be explained more later.
37
Flowchart of The BC Percentile Method
3838
*2x
*Bx
*(2)
1 2ˆ ( , , ..., ) ( )ndata x x x s x x
*1x
resample B times
*(1)
1 0 1 2 02 , 2v z z v z z
*( )ˆB*
(2)
get resample statistics and then sort them * *b bs x
confidence interval 100 1 %
1 2
* *
1 1ˆ ˆ,B v B v
LCL UCL
0estimate z
1( ) z
1 *0
1
1 ˆ ˆestimate by 1B
bb
zB
The BC Percentile Method by R (1)
39
The BC Percentile Method by R (2) Example
40
2.3. The BCa Percentile Method Methodology Flowchart R code C code
41
The BCa Percentile Method (1) The bootstrap bias-corrected
accelerated (BCa) interval is a modification of the percentile method that adjusts the percentiles to correct for bias and skewness.
http://bcs.whfreeman.com/ips5e/content/cat_080/pdf/moore14.pdf
42
The BCa Percentile Method (2)
43
1 1
1 2
*
** 0
* 1 ** 0 *
0
01
0
1 *0
1 1
ˆ ˆ1
ˆ1
ˆ ˆ ˆ ˆ1
ˆ1
1
ˆ
1
ˆ ˆ ˆ1
ˆ ˆB
g gP U z z
a g
P g g a g z z P
g gP U z z
a g
g z zP g
a z z
P g g a g z z P
*
The BCa Percentile Method (3) Similarly,
and
and
and44
2
*
1 1ˆ 1
BP
1 2
* *
1 1 1 1ˆ ˆ 1 2
B BP
11 1? 1 P z
1
0
00
ˆˆ ˆ1
1
g z zg a g z z
a z z
0
1 00
11
z zP Z z
a z z
1
00
01
z zz z
a z z
The BCa Percentile Method (4) Similarly,
and
45
1 0
2 01 0
11
z zP Z z
a z z
0
* ** *
*
* 0 0 0
?
ˆ ˆ ˆ ˆ
ˆ ˆ ˆ ˆ
ˆ ˆ1 1
z
P P g g
g g g gP z z z
a g a g
1 *0 *
ˆ ˆz P 1 *0
1
1 ˆ ˆˆ 1B
bb
zB
The BCa Percentile Method (5)
where
and
46
3
13/2
2
1
?
ˆ ˆ
ˆˆ ˆ6
n
ii
Jackn
ii
a
a
1, 1ˆ , , , ,n i i ni F X X X
1
1ˆ ˆn
iin
Flowchart of The BCa Percentile Method
4747
*2x
*Bx
*(2)
1 2ˆ ( , , ..., ) ( )ndata x x x s x x
*1x
resample B times
*(1) *
( )ˆB*
(2)
get resample statistics and then sort them * *b bs x
confidence interval 100 1 %
0estimate ,z a
1( ) z
1 *0
1
1 ˆ ˆestimate by 1 and by JackknifeB
bb
z aB
/ 2 0 1 / 2 01 0 2 0
/ 2 0 1 / 2 0
1 ( ), 1 ( )1 ( ) 1 ( )
z z z zz z
a z z a z z
1 2
* *
1 1 1 1ˆ ˆ,B B
LCL UCL
The BCa Percentile Method by R (1)
48
Step 1: Install the library of bootstrap in R.
Step 2: If you want to check BCa, type “?bcanon”.
The BCa Percentile Method by R (2)
49
The BCa Percentile Method by R (3)
50
The BCa Percentile Method by R (4) Example
51
The BCa Percentile Method by C (1)
52
The BCa Percentile Method by C (2)
53
The BCa Percentile Method by C (3)
54
The BCa Percentile Method by C (4)
55
Exercises Write your own programs similar to those
examples presented in this talk. Write programs for those examples
mentioned at the reference web pages. Write programs for the other examples that
you know. Prove those theoretical statements in this
talk.
56