Confidence Region Hypothesis Testing

8/13/2019 Confidence Region Hypothesis Testing

1/8

Confidence region Hypothesis testing

Posterior mean

Confidence level data Y parameter

Find a region C(Y) of the parameter based on posterior belongs to this region with prob 1-

Natural confidence region Highest posterior density (HPD)

C(Y) = : P(|Y) > K- chose K st. P(C(Y)|Y) = 1-

Optimal HPD gives the region with smallest volume for given

Bayesian depends on prior best one can do if prior is correct inference is relative easy conceptually if

one can compute posterior HPD is well-defined.

Frequentist: confidence region should cover true parameter with probability 1- (on larger) regardless

of prior. Harder problem

Normal known variance and unknown mean

Y ~ N(,2) ~N(0,

2)

Posterior: ~N(Y, 2)

Y= 2Y/(

2+

2)

-2=

-2

-2 when inf

hat

2 2

Y


2/8

C= *YK, Y+ K+

Kis /2 quantile of Cis classical confidence interval

Full normal with unkown mean and unknown variance in 1-D

Y= {Yi} Yi ~N(,2) =,2) both and 2 are unknown

Likelihood: exponential family form

Sufficient stat: Ybar =( 1/n)SumYi

SY= Sum (YiYbar)2

P(Y|) = ProductP(Yi|)

= Product 1/*sqrt(2)+exp-(Yi)2/2

2}

1/nexp{-[sum(YiYbar)2+ n(Ybar)2+/22}

1/nexp{-[Sy +nYbar

2-2nYbar n

2+/2

2

[SY, nYbar2, 2nYbar, n]T[-1/22, -1/22, -1/22, -1/22]

Conjugate prior four parameter , , ,

P() (2)- exp{- *(-)2 +/22}

Posterior P(,2|Y) (2

2)^- exp* (-)

2 +/2

2}1/

nexp{-[SY+ n(Ybar)

2+/2

2}

(22)^- -n/2exp* (-)2 -SY- n(Ybar)2+/22}

Y= ( nYbar)/+n

=*,+ =* ,2]

Jeffreys prior

P(,2) ~1/

3

A more common prior (non-informative) is

P(,2) ~1/

2

Posterior

P(,2|Y) (

2)^- 1-n/2 exp{[ SY

2- n(Ybar)

2+/2

2}


3/8

Interested in confident interval for look at the marginal posterior

P(|Y) = integrationP(,2|Y)d 2

[ SY2- n(Ybar)

2]^(-n/2)

n-1distribution

Bayesian confidence interval (credible interval)

C= [Ybart/2,n-1SY/sqrt(n(n-1)), Ybar + t/2,n-1SY/sqrt(n(n-1))]

Bayesian t-confidence region whether =0 or not

Hypothesis testing

Deciding about assumption of given observation

Null hypothesia Ho: 0

Vs alternative hypothesis H1: 1

Example: 0 = 0-

1 = R\{0}

Binary outcome

1: accet Null hypothesis

0 reject Null hypothesis

Possible outcomes

Decision\ Truth H0 is true H0 is false

Accept H0 Right decision Type II error

Reject H0 Type I error Right decision

Generally want probability of type I error small, say


4/8

(,d) = 0 if d=I(0)

a0 if 0& d=0 (type I error)

a1 if 0& d=1 (type II error)

d{0,1} a0>=0, a1>=0

Bayes optimal estimator

1 if P(0|Y) >a1/(a0+a1)

0 otherwise

Normal-normal

: known variance unknown mean

~ N(0, 2) Y~N(,2)

Posterior ~N(Y, 2)

Y= 2Y/( 2+ 2)

0 = :


5/8

Alternative: try to alleviate the provlem

Bayes factor (assuming prior of M0 = prior of M1)

B10= P(Y|M1)/P(Y|M0) = P(Y|1)/ P(Y|0)= integral 1P(Y|)P()/P(1)d-/integral0

P(Y|)P()/P(0)d)

B10=[ P(1|Y)/P(1)]/ *P(0|Y) /P(0)]

Likelihood ratio test

LR10= sup 1P(Y|)/sup 0P(Y|)

N withproper prior

LR10~ B10

Bayes factor can handle point null hypothesis

Cannot handle improper prior because it cannot be renormalized.

One should avoid improper prior for hypothesis testing and model selection

How to balance the two models have to be careful

Jeffreys scale

Bayes factor

Rule of thumb interpretation for hypothesis testing (analogous to p-value)

Log10B10 (0, 0.5) evidence against H0 is poor

(0.5, 1) substantial

(1, 2) decisive

>=2 significant level about 0.01

Analogous to p-value


6/8

Improper prior: P() 1

Normalization not well defined

P(Y|M1) /P(Y|M0) = integral exp{-(Y-)2/2d--,/exp-Y

2/2}=sqrt(2pi)exp{y

2/2}

Proper prior

~ N(0, 2) (later let )

P(Y|M1) /P(Y|M0) =[1/sqrt(1+ 2)+exp2y2/2(1+ 2)- 0

Bartletts paradox

Bayesian Computation

Monte Carlo Method

Given prior P()

Likelihood P(Y|)

Want to fin posterior P(|Y) P()P(Y|)

Computation associated with posterior integration of functions of parameter with respect to posterior

(mean, variance)

Sample form the posterior (monte Carlo)

Mode finding or approximate posterior by simple distribution

Interior integration

E~P(.|y)h() = h()P(|y)d

Generally let f() = P(|y) be density

Want to calculate

J = integral h()f()d

h(): function such as mean, variance

Monte Carlo integration draw sample *1, , n+ from f()

Approximate J by

J hmbar = 1/m h(i)


7/8

If i are independent, by law of large numbers m hmbarJ. in probability

Variance

E(hmbarJ)2= 1/mE(h() J)

2= 1/m( h()-J)

2f()d

Posterior simulation methods

Independent sample techniques

Inverse transformation sampling

important sampling

Rejection sampling

Adv: convergence easy to understand

Disadvantage: hard to deal with complex high dimensional problem

Dependent sampling

MCMC(Markov-chain Monte Carlo)

Techniques

Gibbs Sampler

Metropolis/Metropolis-Hastings

Easy to deal with complex problems

Convergence harder to determine

Inverse transformation sampling

Assume has pdf f()

Cdf F(a) that invertible

Want to sample 1. . . m~f()

Algorithm j= 1,,m

Generate random number uj~U(0,1)

Let j be the number such that F(j) = uj (i.e. j= F-1(uj))

Claim j~ f()

Proof cdf of jis P(j


8/8

Example: exponential

Pdf: f() = e-(>=0)

Cdf : F() = 1- e-(>=0)

Inverse: F-1(u) = -ln(1-u)/

Algorithm

uj ~ U(0, 1)

j=( -1/)ln(1-u)

Example: Weibull Distribution

Pdf: f() = (/)-1exp{-(/)}

Cdf: F() = 1 - exp{-(/)}

F-1(u) = -ln(1-u)]1/

Sampling algorithm

Generate ujU(0,1)

j-ln(1-u)]1/

Confidence Region Hypothesis Testing

Documents

Transcript of Confidence Region Hypothesis Testing