Week9 Annotated
description
Transcript of Week9 Annotated
ACTL2002/ACTL5101 Probability and Statistics: Week 8
ACTL2002/ACTL5101 Probability and Statistics
c© Katja Ignatieva
School of Risk and Actuarial StudiesAustralian School of Business
University of New South Wales
Week 8Probability: Week 1 Week 2 Week 3 Week 4
Estimation: Week 5 Week 6 Review
Hypothesis testing: Week 7 Week 8
Linear regression: Week 10 Week 11 Week 12
Video lectures: Week 1 VL Week 2 VL Week 3 VL Week 4 VL Week 5 VL
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Last eight weeks
Introduction to probability;
Moments: (non)-central moments, mean, variance (standarddeviation), skewness & kurtosis;
Special univariate (parametric) distributions (discrete &continue);
Joint distributions;
Convergence; with applications LLN & CLT;
Estimators (MME, MLE, and Bayesian);
Evaluation of estimators;
Interval estimation;
Hypothesis testing.2301/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
This week
Special hypothesis tests:
- Fisher exact test;
- Contingency tests:
One sample Binomial.
One sample Multinomial;
r -Sample Multinomial.
- Application contingency tests:
Goodness of fit test;
Test for homogeneity;
Test for independence.
- Non-parametric tests:
Sign test.
Binomial test (test on quantiles);
- Goodness of fit tests.2302/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Introduction
Last two weeks we have seen hypothesis testing;
One part of it was the test statistic;
To do so, we assumed a distribution for X .
However, there are cases where we don’t know the distributionof X .
Use non-parametric methods: e.g., Fisher exact test, Wilcoxonpaired sample signed rank test.
The non-parametric methods do not rely on the estimation ofthe parameters describing the distribution.
2303/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Hypergeometric distribution
Take n draws from a sample without replacement;
Let the total number of objects be N(≥ n);
Have two possible different outcomes (e.g. “blue ball” and“red ball”), one of them (say blue balls) has M(≤ N) objects.
Let X be the random variable of blue balls selected.
This provides us the Hypergeometric distribution:X ∼ HYP(n,M,N) with n = 1, 2, . . . ,N, M = 0, 1, . . . ,N.The probability mass function of a Hypergeometricdistribution is given by:
pX (x) =
(Mx
)·(N−Mn−x
)(Nn
) .
2304/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Hypergeometric distribution
0 1 2 3 4 50
0.1
0.2
0.3
0.4
x
probab
ility ma
ss fun
ction Hypergeometric(5,4,10) p.m.f.
0 1 2 3 4 50
0.5
1
xcumula
tive de
nsity f
unction Hypergeometric(5,4,10) c.d.f.
2305/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Example
Example: Male bank supervisors were used as the subjects forthe following experiment. They had to make decisions onitems in an in-basket. One experiment was to decide whetherto promote or not.
By random selection, 24 of the supervisors examined a filelabeled as being that of a male employee and 24 examined afile labeled as being that of a female employee. Apart fromthe label, the files were identical. The results were:
Number Male Female
Promote x =21 14 M =35Hold file 3 10 13
n =24 24 N =48
Note: Given margins X ≤ 24, more on slide 2309.2306/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Example
It appears there is evidence of sex bias. But could this haveoccurred by chance? That is, even if there were no bias andthe supervisors were oblivious to gender, discrepancies likethose observed could occur with fairly large probability bychance alone.
Let us represent counts as (2× 2 contingency table):
N11 N12 n1·N21 N22 n2·n·1 n·2 n··
Question: How to test relationship between two randomvariables?
Solution: One way: Chi-squared test, approximate for large n.
Alternative: Fisher’s exact test.2307/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Fisher’s exact test of independence
The Fisher exact test is given by:
Test (H1 can be two-sided or one-sided):
H0 :N11
n·1=
N12
n·2v.s. H1 :
N11
n·16= N12
n·2, for a given α.
H0 implies no sex bias, i.e., differences due to randomization.
Margins (n1·, n2·, n·1, n·2, and n··) are known constants.
Test statistic: We have N11 is hypergeometric with p.m.f.:
pN11(n11) =
(n1·n11
)(n2·n21
)(n··n·1
) ,
i.e., N = n··, n = n·1, M = n1·, x = n11, n2· = N −M, and n21 = n− x .2308/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Fisher’s exact test
Fisher’s exact test
Determine rejection region:One sided test: C = min|n1· − n·2|, |n2· − n·1|, . . . , ul orC = ll , . . . ,minn1·, n·1. Note, the min is due toconstraint on marginals.
We have for the one-sided test that (see slide 2306)
ul = argminx
∑xy=min|n1·−n·2|,|n2·−n·1| pN11(y) ≤ α
, or
ll = argmaxx
∑minn1·,n·1x pN11(y) ≤ α
.
Two sided test: C =min|n1·−n·2|, |n2·−n·1|, . . . , ul∪ll , . . . ,minn1·, n·1,where ll and ul are given using α/2.
Rejection region for a two-sided test with α =0.05C = 11, . . . , 14, 21, . . . , 24. Observed value of falls in thisregion, thus reject H0.
pN11(11) = 1.29 · 10−5, pN11(12) = 3.36 · 10−4, pN11(13) = 0.0035,pN11(14) = 0.021. Thus Pr(N11 ≤ 14) = Pr(N11 ≥ 21) = 0.0245.2309/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Binomial
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Binomial
First, consider c = 2, then O1 ∼ Bin(n,E1/n). We have:
T =(O1 − E1)2
E1 · (1− E1/n)∗=
(O1 − E1)2
E1+
(O1 − E1)2
n − E1
=(O1 − E1)2
E1+
((n − O1)− (n − E1))2
n − E1
∗∗=
(O1 − E1)2
E1+
(O2 − E2)2
E2
** O2 ≡ n − O1 and E2 ≡ n − E1; * where:
1
E1(1− E1/n)=
n
E1 · (n − E1)=
n−E1 + E1
E1 · (n − E1)
=n − E1
E1 · (n − E1)+
E1
E1 · (n − E1)=
1
E1+
1
n − E1.
Note that√T equals o1−n·p√
n·p·(1−p)hence is approximately
standard normal distributed. Thus, T is approximately χ2(1)distributed ⇒ appropriate test statistic.
2310/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Multinomial
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Multinomial
One sample MultinomialWe specify c cell probabilities of one variable: One sampleMultinomial.
This gives us a c by one contingency table.
We have n outcomes A1, . . . ,Ac , with specified probabilitiesPr(A1), . . . ,Pr(Ac).
Example: a group of individuals is asked whether they preferto invest their super in:
1) riskfree asset;2) partly risky asset, partly riskfree asset;3) risky asset.
The null hypothesis is:
H0 :p1 = p2 = p3 = 1/3
v.s. H1 :p1 6= p2 or p1 6= p3 or p2 6= p3.2311/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Multinomial
χ2 densities
0 2 4 6 8 100
0.5
1
xprob
abilit
y den
sity f
uncti
on χ2(ν) p.d.f.
ν = 1ν = 3ν = 10
0 2 4 6 8 100
0.5
1
xcum
ulativ
e de
nsity
func
tion χ2(ν) c.d.f.
2312/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Multinomial
One sample MultinomialWe perform an experiment with 90 individuals, and find thefollowing:
# times selected Expected Observed (Oi − Ei )2/Ei
1) Riskless 23 30 23 49/302) Combination 36 30 36 36/303) Risky 31 30 31 1/30
Total 90 90 90 86/30
Note: 31 = 90− 23− 36.
Do the data present sufficient evidence to warrant rejection ofthe hypothesis of no preference?
Test statistic: T =c∑
i=1
(Oi−Ei )2
Ei, reject for large values of T .
T is chi-squared distributed with c−1 degrees of freedom(last cell is determined by total and values of other cells).2313/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Multinomial
Example: (Chi-squared) Goodness of fit testUse one sample multinomial case.
Adopt this test for a complete specified distribution, i.e.,H0 : X ∼ FX (x) v.s. H1 : X has another distribution.
Divide the sample space in c cells, A1, . . . ,Ac and letEj = n · Pj , where Pj = Pr(X ∈ Aj), and X ∼ FX (x).
We have Oj ∼ Bin(n,Pj)→ N(Ej ,Ej · (1− Pj)) ≈ N(Ej ,Ej).
Hence, we have a multinominal test:
T =∑c
j=1(Oj−Ej )
2
Ej∼ χ2(df ), with df = c − 1 degrees of
freedom.
Note: MLE for k parameters: df = c − 1−k.
General principal for selecting cells: as many (⇒ Pj → 1),such that (np =)Ej≥ 5, i.e., highest degree of freedom, butchi-squared approximation is fairly accurate.
2314/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Multinomial
Example: bombs. The following data come from an article byR.D. Clarke in the 1946 Journal of the Institute of Actuaries. Itdescribes the pattern of damage by German V-2 bombs during theSecond World War.
No. of flying bombs Actual no. of
per square squares Expected (Oi−Ei )2
Ei
0 229 Pr(X = 0) · n =228 0.00951 211 Pr(X = 1) · n =211 0.00052 93 Pr(X = 2) · n =98 0.26983 35 Pr(X = 3) · n =30 0.7004 7 Pr(X = 4) · n =7 0.0004
5 and over (7 bombs) 1 Pr(X ≥ 5) · n =2 0.1917
Total 576 576 1.17Let Xi |H0 ∼ Poi(λ) be the number of bombs on square i .
λ = x =∑n
i=1 xin = 0·229+1·211+...+7·4+1·7
576 = 537576 .
n = 576 number of squares; Pr(Xi = j) = e−λ·λjj! .
2315/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
One sample Multinomial
Example: Goodness of fit test: bombs
Clarke noted: ‘During the flying bomb attack on London, frequentassertions were made that the points of impact of the bombstended to be grouped in clusters. . . ’, rather than displaying arandom distribution over the metropolis. It was as if certain areaswere specifically targeted, or as if bombs tended to fall together forsome mechanical reason. Both these explanations appearedunlikely. To determine whether clustering really took place, ‘. . . 144square kilometres of south London [were] . . . divided into 576squares of 1/4 square kilometre each, and a count was made of thenumber of squares containing 0, 1, 2, 3, . . . , etc. flying bombs.Over the period considered the total number of bombs within thearea involved was 537. The expected numbers of squarescorresponding to the actual number yielded by the count were thencalculated from the Poisson formula. . . .’
2316/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
r -Sample Multinomial
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
r -Sample Multinomial
r -Sample Multinomial
Suppose we have now r independent samples of a Multinomialpopulation (gives a c by r contingency table).
For j = 1, . . . , r define c outcomes A1j , . . . ,Acj ;
Denote, under the null pi |j = Eij /∑c
i=1 Eij , we have:∑ci=1 pi |j = 1.
Test H0 : pi |j = p0i |j , we have Eij = nip
0i |j .
Test statistic:
T =r∑
j=1
c∑i=1
(Oij − Eij)2
Eij∼ χ2(df )
with df = (c−1) · (r−1) degrees of freedom (recall: sum ofchi-squared r.v.)
2317/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Example homogeneity: We consider three different healthinsurers with the same number of insured and the number oftreatments of six different group (times 1000). Is there a differencebetween the insureds of the different insurers?
Treatment Insurer A Insurer B Insurer C Total
Pharmaceutic 147 186 101 434Intensive care 25 26 11 62Podiatry 32 39 15 86Ambulance 94 105 37 236Hospital 59 74 28 161Chiropractic 18 10 10 38
total 375 440 202 1017
Test whether all cell probabilities are equal.Note: r = 3 independent samples; c = 6 treatments.18 = 375− 147− . . .− 59 and 37 = 236− 94− 105. (table withexpected: see slide 2320)2318/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Example: Goodness of fit: homogeneity
We test:
H0 : πi1 = πi2 = . . . = πiJ , for i = 1, . . . , r ,i.e., H0 : πi1 = πi2 = πi3, for i = 1, . . . , 6.
We can use Pearson’s chi-square statistic:
T =r∑
i=1
c∑j=1
(Oij − Eij)2
Eij∼ χ2(df )
with degrees of freedom given by:
df = (c−1) · (r−1).
2319/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Example: Goodness of fit: homogeneity
Expected Insurer A Insurer B Insurer C Total πMLi
Pharmaceutic 160 188 86 434 0.427Intensive care 23 27 12 62 0.061Podiatry 32 37 17 86 0.085Ambulance 87 102 47 236 0.232Hospital 59 70 32 161 0.158Chiropractic 14 16 8 38 0.037
Total 375 440 202 1017πMLj = 375
1017440
1017202
1017 1
We have t =∑r
j=1
∑ci=1
(Eij−Oij )2
Eij= 12.27, χ2
0.95(10) = 18.5, thus
we can not reject H0. Table with observed: see slide: 2318.where 59 = n · 0.158 · 375
1017 ; 12 = n · 0.061 · 2021017 and 0.037 = 38
1017 .2320/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Example: Chi-square test of independence
Example: A death benefit insurer is interested in whether theprobability of reaching the age of 80 is dependent on themarital status.Question: why this of interest?
Married Not married Total
LE > 80 550 61 611LE ≤ 80 681 144 825
Total 1 231 205 1 436
“If I were to take a sample of 1 436 people cross-classified intotwo categories which were in fact unrelated in the populationfrom which the sample was drawn, I might find associations asstrong or stronger than those observed in this table. Whyshould I believe that there is any real association in yourtable?”
2321/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Example: Chi-square test of independence
The joint distribution of the counts nij , where i = 1, . . . , r andj = 1, . . . , c is Multinomial with cell probabilities denoted byπij .
Let:
πi · =c∑
j=1
πij and π·j =r∑
i=1
πij
denote the marginal probabilities that an observation will fallin the i th row and j th column, respectively.
If the row and column classifications are independent of eachother
πij = πi ·π·j .2322/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Example: Chi-square test of independence
We thus test:
H0 : πij = πi ·π·j , for i = 1, . . . , r , j = 1, . . . , cv.s. H1 : at least one πij 6= πi ·π·j .
You can show, under H0 (independence), the MLE of πij is:
πij = πi ·π·j =ni ·n·n·jn.
Under the alternative, the MLE is πij = nij/n.These estimates can be used to form a LRT or anasymptotically equivalent Pearson’s χ2 test:
r∑i=1
c∑j=1
(Oij − Eij)2
Eij∼ χ2(df ),
where Eij = n · πij =ni·n·jn and Oij = n · πij = nij .
2323/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Examples
Example: Chi-square test of independence
with degrees of freedom:
df = r · c − 1− (r − 1)︸ ︷︷ ︸# par πi
− (c − 1)︸ ︷︷ ︸# par πj
= (r − 1) · (c − 1).
We would reject the null hypothesis of independence andconclude that there is a relationship between marital statusand the probability of reaching age 80.
The chi-square statistic used here to test independence isidentical in form and degrees of freedom to that used earlierto test homogeneity/goodness of fit. The hypotheses aredifferent and the sampling schemes are different.
2324/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Exercises
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Exercises
Exercise: Are storm claims independent
Two insurers, SC and RS, have provided some data abouttheir storm claims to an institute.
Unfortunately you are not bale to obtain more detailed datathan the following:
C = 0 C < 50 50 < C < 100 100 < C < 500 C > 500 total# claims SC 1214 434 155 168 29 2000# claims RS 1430 320 113 126 11 2000Total 2644 754 268 294 40 4000
Question: Are the claims of SC and RS dependent?
2325/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Exercises
Solution: Are storm claims independent
Solution: T =∑ (Ei−Oi )
2
Ei∼ χ2(4), χ2
0.95(4) = 9.49
χ20.99(4) = 13.28. We have t = 55.56, thus reject H0.
C = 0 C < 50 50 < C < 100 100 < C < 500 C > 500 TotalE# claims SC 1322 377 134 147 20 2000E# claims RS 1322 377 134 147 20 2000proportion 2644
4000754
4000268
4000294
400040
4000 4000(Oi − Ei ) SC 108 −57 −21 −21 −9(Oi − Ei ) RS −108 57 21 21 9(Oi−Ei )
2
EiSC 8.82 8.62 3.29 3 4.05 27.78
(Oi−Ei )2
EiRS 8.82 8.62 3.29 3 4.05 27.78
Question: Insurer SC currently uses a Pareto(α = 2, λ = 100)(with F (x) = 1− (1 + x/λ)−α) for the distribution of theclaim size when an insured issues a claim. Is that the correctdistribution?
2326/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Contingency tables
Exercises
Solution: Are storm claims Pareto(2,100) distributedSolution:
C = 0 C < 50 50 < C < 100 100 < C < 500 C > 500 total# claims SC 1214 434 155 168 29 2000F (X ) 5/9 3/4 35/36 1E# claims SC 436.67 152.83 174.67 21.83 786(Oi − Ei ) −2.67 2.167 −6.67 7.167(Oi−Ei )
2
Ei0.016 0.0307 0.2544 2.35 2.65
Note: Discard C = 0, because only interested in when aclaim is made.
T =∑ (Ei−Oi )
2
Ei∼ χ2(3), χ2
0.95(3) = 7.81 χ20.99(3) = 11.34.
We have t = 2.65, thus accept H0.
Note: if parameters where estimated: df = 3− 2 = 1,χ2
0.95(1) = 3.84 χ20.99(1) = 6.6. Still accept H0.
2327/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Two sample test
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Two sample test
One sample sign test
Consider a random variable X ∼ F (x), let m be the median.We observe xi , . . . , xn.
Test: H0 : m = m0 v.s. H1 : m < m0.
Let ti = 1xi−m0<0 =
1, if xi < m0;0, else.
.
Under H0, each ti is a Bernoulli experiment, with p = 0.5.
Test statistic: T =∑n
i=1 ti ∼ Bin(n, p).
Reject H0 if B(t; n, 0.5) ≤ α (the c.d.f. of a Bin(n, p) in t)Example: test whether mean claim size is $24.5 thousand,sample of 20 observations:
10.8 12.7 13.9 18.1 19.4 21.3 23.5 24.0 24.6 25.025.4 27.7 30.1 30.6 32.3 33.3 34.7 38.8 40.3 55.5
Solution: B(t = 8; 20, 0.5) = 0.2517, thus cannot reject H0.2328/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Two sample test
Two sample sign testWe have paired data (xi , yi ). Are X and Y independent?
Test: H0 : Pr(X < Y ) = 1/2 v.s. H1 : Pr(X − Y < 0) < 1/2.
Let ti = 1xi−yi<0 =
1, if xi < yi ;0, else.
.
Under H0, each ti is a Bernoulli experiment, with p = 0.5.
Test statistic: T =∑n
i=1 ti ∼ Bin(n, p).
Reject H0 if B(t; n, 0.5) ≤ αExample: A MVI offers contracts in fifteen regions, elevenhad an increase in mean claim size and four had a decrease.Did the claim size increase?Solution: p-value: B(t = 4; 15, 0.5) = 0.0592, at α = 6% itis significant that the mean claim size had increased.
2329/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Two sample test
Wilcoxon paired sample signed-rank test: We have paireddata (xi , yi ). Do X and Y have the same distribution?
Let di = xi − yi denote the difference of the pairs.Rank absolute differences: u1 = min|di |, . . . , un = max|di |.Test H0 : FX = FY v.s. H1 : FX 6= FY .
Let T+ (T−) the sum of the positive (negative) ranks.
Let Vi = 1ui>0, under H0 we have Vi ∼ Ber(1/2) then
E [T+] = E [∑n
i=1 i · Vi ] =∑n
i=1 i · E [Vi ]∗= n(n + 1)/4, *
using E[Vi ] = p = 1/2 and∑n
i=1 i = n(n+1)2 and
Var (T+) = Var (∑n
i=1 i · Vi ) =∑n
i=1 i2 · Var (Vi )
∗∗=∑n
i=1 i2/4
∗∗∗= n(n + 1)(2n + 1)/24. ** using Var(Vi ) =
p(1− p) = 1/4 and *** using∑n
i=1 i2 = n(n+1)(2n+1)
6 .
Test statistic: T = (T+ − E [T+])/√
Var (T+) ∼ N(0, 1) orS = (T− − E [T−])/
√Var (T−) ∼ N(0, 1), using CLT;
and in case of a two-sided test: max(T ,S).2330/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Two sample test
Example: Wilcoxon paired sample signed-rank test
Test whether the distribution of the number of deaths in caraccidents with a female driver is lower than with a maledriver. Data 7 months:
Month 1 2 3 4 5 6 7
Females 103 118 164 103 146 150 137Males 112 139 111 186 207 154 131di 9 21 -53 83 61 4 -6i =Rank 3 4 5 7 6 1 2
E [T+] = n · (n + 1)/4 = 14, with n = 7 months andVar (T+) = n · (n + 1) · (2n + 1)/24 = 35.
t = (t+ − E [T+])/√
Var (T+) = (21− 14)/√
35.
Φ(
21−14√35
)= 0.12. Cannot reject H0 at 5% confidence level.
2331/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Test on quantiles
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Test on quantiles
Test on quantiles
If we have an empirical (simulated) distribution, how todetermine the CI of a quantile of the population distributionFX (x)?
Test H0 : xq = x(i) v.s. H1 : xq > x(i) at α, for a quantile q.
Let t be the number of observations in the sample that aresmaller than the qth quantile xq. Then we have:T ∼ Bin(n, q).
Reject H0 if B(t; n, q) ≤ α.
Thus: Pr(xq < x(t)) = 1− α, where t is
argmaxt
∑tj=0
(nj
)· pj0 · (1− p0)n−j ≤ 1− α
.
2332/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Test on quantiles
Example: Test on quantiles
We are interested in the reserves for a mortgage life insurance(with a large pool).
The contract implies that you pay a fixed premium for thenext 40 years.
When you die before the end of the contract the insurer paysthe principal.
Contract has an expected profit in the first years and anexpected loss in the last years.
An insurer should hold enough capital in order to pay theexpected loss in the future.
2333/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Test on quantiles
Example: Test on quantiles
Reserve equals the 97.5% quantile.
Value does depend on evolution of mortality: unknown & noclosed form (special) distribution.
Use N = 1000 simulations.
Ordered simulations: x(975) = 23.05, x(982) = 23.81, andx(983) = 24.03.
T ∼ Bin(1000, 0.975), with Pr(T ≤ 983) = 0.964 andPr(T ≤ 982) = 0.942.
Hence, we can reject that the 97.5% quantile is larger than24.03 with probability 95%.
2334/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Exercises
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Exercises
Exercise: find the mean of GHI and BHIWe have the average claim size of health insurers GHI andBHI over time. The average claim sizes are:
Year 2011 2010 2009 2008 2007
GHI 13.73 1.13 5.35 3.28 9.31BHI 16.04 4.92 3.51 18.65 4.11
Year 2006 2005 2004 2003 2002
GHI 2.67 6.42 1.50 6.10 3.75BHI 4.26 3.46 5.48 2.19 9.35
Assume that the average claim size of health insurance has asymmetric distribution.
Question: Find the price, which is set equal to the 75%quantile of the mean claim size for both GHI and BHI.
Solution: T ∼ Bin(10, 0.5). Hence Pr(T ≤ 6) = 0.8281.Thus P[GHI ] = 5.35 and P[BHI ] = 4.92.2335/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Exercises
Exercise: Capital requirement GHIA consultancy firm has made some simulations for GHI for theaverage claim size next year.
Provided are 10,000 simulations of the average claim size.
A subset of their (ordered) simulations are:
Simulation 9950 9951 9952 9953 9954 9955 9956 9957 9958
GHI 21.68 21.84 21.94 21.97 22.04 22.07 22.11 22.12 22.13
Simulation 9959 9960 9961 9962 9963 9964 9965 9966 9967
GHI 22.26 22.49 22.86 22.86 22.96 23.56 23.64 23.86 24.02
Question: What is the required reserve for next year (the99.5% quantile) with probability 90% (one sided).
Solution: T ∼ Bin(10000, 0.995) ≈ N(9950, 9950 · 0.005 = 49.75).
Pr(
X−µσ < z0.95
)= 95%⇒ x(9950+1.96·
√49.75) = x(9963.8) = 23.56.
Recommendation: more simulations required!2336/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Exercises
Exercise: Are GHI and BHI independentGHI has made lots of profits the past decade, whereas BHIhas made some losses over the last decade.GHI is in the process of taking over BHI. This was initiated bythe board of directors, who thought that the claim sizes ofBHI and GHI are independent.If so, this might reduce their price, leading to more attractivepolicies and thus more insured.Question: The BoD has asked you to verify whether they arecorrect? They are convinced at a 90% significance.Solution: When testing H0 :BHI and GHI are independentv.s. dependent one obtains: T ∼ Bin(10, 0.5). HencePr(T ≤ 7) = 0.9453⇒ C = 7, . . . , 10. We observe T = 6,thus cannot reject H0, hence independent.Question: Comment on the test performed.Solution: This assumes H0, but α = 0.1, does not imply β = 0.1!
2337/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Non-parametric tests
Exercises
Exercise: Is GHI “better” than BHIGHI is still in the process of taking over BHI.
The consultant argues that the claim distribution of BHI isdifferent than GHI.
Question: Can you verify wether the consultant is correct?
Solution: Year 2011 2010 2009 2008 2007
di -2.31 -3.79 1.84 -15.37 5.2i 3 5 2 10 8
Year 2006 2005 2004 2003 2002
di -1.59 2.96 -3.98 3.91 -5.6i 1 4 7 6 9
Hence T− = 3 + 5 + 10 + 1 + 7 + 9 = 35Φ((35− 27.5)/
√96.25) = Φ(0.76) = 0.72.
Conclusion: Do not pay the bill!2338/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Tests already seen
Last week: Quantile-Quantile test (bootstrap test version ofCW test).
Likelihood ratio test (bootstrap test (week 8) & approximatetest (week 7)):
T (X ) = −2 log (Λ) (= −2 (log (f0(x))− log (fA(x))))
has -under the null hypothesis- asymptotically a chi-squareddistribution with degrees of freedom dim (Ω)− dim (Ω0).
This week: Pearson chi-squared test.
Other test: AD, CV, KS, and K: based on empirical c.d.f. andhypothesized c.d.f. (note: quantile function is uniformlydistributed, see exercise 3a week 6).
2339/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Anderson-Darling & Cramer-von Mises test OPTIONALTest H0 : X ∼ F (X ; θ) v.s. H1 : X F (X ; θ). Test statistic:
T =n ·∫ ∞−∞
(Fn(x)− F (x))2 · w(x)dF (x)
where w(x) =
1, if T = CV , Cramer-von Mises test;
(F (x) · (1− F (x)))−1, if T = AD, Anderson-Darling test.
AD =n∑
i=1
1− 2i
n·(log(Fn(x(i))) + log(1− Fn(x(n+1−i)))
)− n
CM =1
12n+
n∑i=1
(Fn(x(i))−
i − 0.5
n
)2
, (discrete sample).
When parameters are not estimated, we have that Fn(x) =U(i), i.e., distribution of T does not depend on FX (x).
If parameters are estimated, the distribution does not dependon the parameter values, but will depend on form of FX (x).
2340/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Figure Anderson-Darling & Cramer-von Mises test
0 0.4 1.6 30
0.3297
0.5
0.7981
1
Exp(1) c.d.f.e.c.d.f.
2341/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Critical values Anderson-Darling & Cramer-von Mises test
H0 Statistic α = 0.1 0.05 0.025 0.01
F (x) CM 0.347 0.461 0.581 0.743
EXP(λ) (1 + 0.16/n) · CM 0.177 0.224 0.273 0.337
N(µ, σ2) (1 + 0.5/n) · CM 0.104 0.126 0.148 0.178
WEI(θ, β)(1 + 0.2/
√n)· CM 0.102 0.124 0.146 0.175
F (x) AD 1.933 2.492 3.070 3.857
EXP(λ) (1 + 0.6/n) · AD 1.078 1.341 1.606 1.957
N(µ, σ2)(1 + 0.75/n − 2.25/n2
)· AD 0.631 0.752 0.873 1.035
WEI(θ, β)(1 + 0.2/
√n)· AD 0.637 0.757 0.877 1.038
Anderson and Darling (1954), Pettitt and Stephens (1976),Stephens (1977), D’Agostino (1986), Stephens (1986), Pearson &Hartley (1972)CV test for Poisson: Spinelli and Stephens (1997)2342/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Kolmogorov-Smirnoff/Kuiper test OPTIONALThe KS test statistic is based on the maximum differencebetween the sample CDF and the hypothesized CDF.H0 : X ∼ FX (x) v.s. H1 : X FX (x). Let:
D+ =maxi
i
n− F (x(i))
D− =max
i
F (x(i))− i − 1
n
D︸︷︷︸
KS-statistic
= maxD+,D−
V︸︷︷︸
Kuiper-statistic
= D+ + D−
Distribution of statistics does not depend on FX (x).If parameters are estimated, the distribution does not dependon the parameter values, but will depend on the form ofFX (x).Note: approximate test, but good for small n.
2343/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Figure Kolmogorov-Smirnoff/Kuiper test
D− = 0.117
D+ = 0.072
Exp(1/3.7) c.d.f.e.c.d.f.
2344/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Critical values Kolmogorov-Smirnoff/Kuiper testH0 Statistic α = 0.1 0.05 0.025 0.01
F (x)(√
n + 0.12 + 0.11/√n)· D 1.224 1.358 1.480 1.628
EXP(λ)(√
n + 0.26 + 0.5/√n)· (D − 0.2/n) 0.995 1.094 1.184 1.298
N(µ, σ2)(√
n − 0.1 + 0.85/√n)· D 0.819 0.895 0.955 1.035
WEI(θ, β)√n · D 0.803 0.874 0.939 1.007
F (x)(√
n + 0.155 + 0.24/√n)· V 1.620 1.747 1.862 2.001
EXP(λ)(√
n + 0.24 + 0.35/√n)· (V − 0.2/n) 0.527 1.655 1.774 1.910
N(µ, σ2)(√
n + 0.05 + 0.82/√n)· V 1.386 1.489 1.585 1.693
WEI(θ, β)√n · V 1.372 1.477 1.557 1.671
Chandra et al. (1981): uses extreme value distribution, which isrelated to Weibull distributionCritical values see: Stephens (1974, 1977, 1979), Aho et al.(1983), Dufour and Maag (1978), Koziol (1989), and Bain andEngelhardt (1983).2345/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
An overview
Comparison1. Plot ecdf and cdf in a graph, is it a good fit?
2. Compare the characteristics of the distributions (mean,variance, skewness, kurtosis).
3. Plot QQ-plot ⇒ is it a good fit.
1-3 Graphical methods give an idea of the fit (good/bad,tail/around median etc), interpret it in your setting!
Chi-squared test: easy to adjust for estimated number ofparameters, approximate test.
KS/Kuiper test: good power, hard to work with; moresensitive near center of the distribution;
AD/CM test: good power, hard to work with. CM/AD testplaces higher weight on the tails.2346/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
Exercises
Hypothesis testing
Fisher’s exact testFisher’s exact test
Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises
Non-parametric testsTwo sample testTest on quantilesExercises
Goodness of fit testsAn overviewExercises
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
Exercises
Exercise: CM OPTIONALConsider the sample from slide 2341 with only twoobservations. Do the following tests at a level of significanceof 5%.
Question: Perform the CM test if the the parameter was notestimated.
Solution: StatisticCM = 1/24 + (0.3297− 0.25)2 + (0.7981− 0.75)2 = 0.050,critical value: 0.461, Thus do not reject H0.
Question: Perform the CM test if the the parameter wasestimated.
Solution:CM = 1/24 + (0.3297− 0.25)2 + (0.7981− 0.75)2 = 0.050;statistic (1 + 0.08) · 0.050 = 0.054 critical value: 0.224, Thusdo not reject H0.2347/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
Exercises
Exercise: AD OPTIONALConsider the sample from slide 2341 with only twoobservations. Do the following tests at a level of significanceof 5%.
Question: Perform the AD test if the the parameter was notestimated.
Solution: Statistic AD = −1/2 · (log(0.3297) + log(1−0.7981)) +−3/2 · (log(0.7981) + log(1− 0.3297))− 2 = 0.293,critical value: 2.492, Thus do not reject H0.
Question: Perform the AD test if the the parameter wasestimated.
Solution: AD = −1/2 · (log(0.3297) + log(1− 0.7981)) +−3/2 · (log(0.7981) + log(1− 0.3297))− 2 = 0.293; statistic(1 + 0.3) · 0.293 = 0.381 critical value: 1.341, Thus do notreject H0.2348/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
Exercises
Exercise: KS/Kuiper OPTIONAL
Consider the sample from slide 2341 with only twoobservations. Perform the KS and Kuiper tests at a level ofsignificance of 5%.
We have:
i F (x(i)) in
i−1n
in − F (x(i)) F (x(i))− i−1
n
1 0.3297 1/2 0/2 0.1703 0.32972 0.7981 2/2 1/2 0.2019 0.2981
D+ = 0.2019 D− = 0.3297
Question: Perform the KS test if the parameter was notestimated.
Solution: D = 0.3297; statistic:(√
2 + 0.12 + 0.11/√
2) · 0.3297 = 0.531 critical value: 1.358,thus do not reject H0.
2349/2350
ACTL2002/ACTL5101 Probability and Statistics: Week 8
Goodness of fit tests
Exercises
Exercise: KS/Kuiper (cont.) OPTIONALQuestion: Perform KS test if the parameter was estimated.
Solution: D = 0.3297; statistic:(√
2 + 0.26 + 0.5/√
2) · (0.3297− 0.1) = 0.466 critical value:1.094, thus do not reject H0.
Question: Perform the Kuiper test if the parameter was notestimated.
Solution: V = 0.5316; statistic:(√
2 + 0.115 + 0.24/√
2) · 0.5316 = 0.903 critical value:1.747, thus do not reject H0.
Question: Perform Kuiper test if the parameter wasestimated.
Solution: V = 0.5316; statistic:(√
2 + 0.24 + 0.35/√
2) · (0.5316− 0.1) = 0.821 critical value:1.655, thus do not reject H0.2350/2350