1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate...

93
1 Inferences about a Mean Inferences about a Mean Vector Vector Shyh-Kang Jeng Shyh-Kang Jeng Department of Electrical Engineeri Department of Electrical Engineeri ng/ ng/ Graduate Institute of Communicatio Graduate Institute of Communicatio n/ n/ Graduate Institute of Networking a Graduate Institute of Networking a nd Multimedia nd Multimedia

Transcript of 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate...

Page 1: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

11

Inferences about a Mean VectorInferences about a Mean Vector

Shyh-Kang JengShyh-Kang JengDepartment of Electrical Engineering/Department of Electrical Engineering/Graduate Institute of Communication/Graduate Institute of Communication/

Graduate Institute of Networking and MultiGraduate Institute of Networking and Multimediamedia

Page 2: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

22

InferenceInference

Reaching valid conclusions Reaching valid conclusions concerning a population on the basis concerning a population on the basis of information from a sampleof information from a sample

Page 3: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

33

Plausibility of Plausibility of 00 as a Value for a as a Value for a

Normal Population MeanNormal Population Mean

n

ii

n

ii

n

XXn

sXn

Xns

Xt

XXX

H

H

1

22

1

0

21

01

00

1

1,

1,

/

:statistics test eAppropriat

population

normal a from sample Random:,,,

: hypothesis ealternativ sided)-(Two

: hypothesis Null

Page 4: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

44

Student’s Student’s tt-distribution-distribution

2

12

2

1)

2(

)2

1(

)(

/

f

f

tf

f

f

tf

f

Zt

Page 5: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

55

Student’s Student’s tt-distribution-distribution

Page 6: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

66

Test of HypothesisTest of Hypothesis

)2/()()(

,i.e.

)2/(/

if level cesignificanat

offavor in Reject

210

120

2

10

10

n

n

txsxnt

tns

xt

HH

Page 7: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

77

Confidence IntervalConfidence Interval

n

stx

n

stx

n

stx

%-α

tns

xH

nn

n

n

)2/()2/(

)2/(

interval confidence )1(100 in the lies

)2/(/

or levelat reject not Do

101

1

0

10

0

Page 8: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

88

Plausibility of Plausibility of 00 as a Multivariate as a Multivariate

Normal Population MeanNormal Population Mean

n

jjj

n

jj

n

nn

n

nT

T

H

H

11

01

0

0

1

02

2

21

1

0

'1

1,

1

'

'

:statistics sHotelling'

population

normal a from sample Random:,,,

: hypothesis ealternativ sided)-(Two

: hypothesis Null

XXXXSXX

μXSμX

μXS

μX

XXX

μμ

μμ

0

0

Page 9: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

99

TT22 as an as an FF-Distribution-Distribution

pnpF

pn

pnT

,

2 1:

Page 10: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1010

FF-Distribution-Distribution

21

21

11

,

2/

2

1

122

2

1

21

21

222

121

2122

21

:

122

2)(

0,/

/

lyrespective, and d.f. with t,independen:,

ff

ff

ff

F

fFf

F

f

fff

ff

Ff

Ff

fF

ff

Page 11: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1111

FF-Distribution-Distribution

Page 12: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1212

Nature of Nature of TT22-Distribution-Distribution

pnppp

pnpp

np

n

jjj

Fpn

pnF

p

p

NWn

N

T

Xnn

XXXX

XnT

,1 d.f.,

1,

1

21,

0

1

10

2

)(

)1(

1 d.f.

d.f. calculus,by

),0()(1

1)',0(

vectorrandom

normal temultivaria

d.f.

matrix

randomWishart

' vectorrandom

normal temultivaria

1

'

'

Page 13: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1313

Test of HypothesisTest of Hypothesis

)()1(

)()'(

if level cesignificanat

offavor in Reject

,

12

10

pnpFpn

pn

nT

HH

00 μxSμx

Page 14: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1414

Example 5.1 Evaluating Example 5.1 Evaluating TT22

1,223,22

2

1

0

4)23(

2)13(:

9

7

56

98

27/49/1

9/13/156983

27/49/1

9/13/1

93

34,

6

8

5

9,

38

610

96

FFT

T

S

SxμX

Page 15: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1515

Example 5.2 Testing aExample 5.2 Testing aMean VectorMean Vector

level %10 at the Reject 18.874.9

18.8)1.0(17

319)10.0(

)(

)1( : valueCritical

74.9,

402.0002.0258.0

002.0006.0022.0

258.0022.0586.0

628.3640.5810.1

640.5788.199010.10

810.1010.10879.2

,

965.9

400.45

640.4

normalitycheck ,20 0.10. level aat Test

10504':,10504':

02

17,3,

21

10

HT

FFpn

pn

T

n

HH

pnp

S

Sx

μμ

Page 16: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1616

Invariance of Invariance of TT22-Statistic-Statistic

0

10

01

0

0,1

0,2

00,

1

)'(

'')'(

'

,)(

''1

1,

singular-non : ,

μxSμx

μxCCSCCμx

μySμy

dCμμdCμYμ

CSCyyyySdxCy

CdCXY

YyY

YY

y

n

n

nT

E

n

n

jjj

Page 17: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1717

TT22-Statistic from -Statistic from Likelihood Ratio TestLikelihood Ratio Test

),(max

),(max ratio Likelihood

'2

1exp

2

1

),(

1ˆ,'

ˆ2

1),(max

,

0

10

102/2/

0

11

2/2/2/,

Σμ

Σμ

μxΣμxΣ

Σμ

xxμxxxxΣ

ΣΣμ

Σμ

Σ

Σμ

L

L

L

nn

eL

n

jjjnnp

n

jj

n

jjj

npnnp

Page 18: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1818

TT22-Statistic from -Statistic from Likelihood Ratio TestLikelihood Ratio Test

n

jjj

npnnp

n

jjj

n

jjj

n

jjj

n

eL

1000

2/2/

02/

0

100

1

100

1

10

10

'1ˆ

ˆ2

1),(max

'tr

'tr'

μxμxΣ

ΣΣμ

μxμxΣ

μxμxΣμxΣμx

Σ

Page 19: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

1919

Result 4.10Result 4.10

Σ

B

b

ebe

b

pp

pp

bppb

bb

2/1for only holding

equality with , definite positive allfor

211

scalar positive :

matrix definite positive symmetric:

)(

2/)tr( 1

Page 20: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2020

Likelihood Ratio TestLikelihood Ratio Test

c

H

HH

n

n

jjj

n

jjj

n

n

2/

100

1

2/

0

0

0100

0/2

'

'

ˆ

ˆ if

level at the Reject

:against :

of test ratio Likelihood

lambda Wilks':ˆ/ˆ

μxμx

xxxx

Σ

Σ

μμμμ

ΣΣ

Page 21: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2121

Result 5.1Result 5.1

12/2

0100

2

21

11

because : vs.:

of test likelihood the toequivalent is test

),( from sample random:,,,

n

T

HH

T

N

n

pn

μμμμ

ΣμXXX

Page 22: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2222

Proof of Result 5.1Proof of Result 5.1

121

11212211211

22121122

2221

1211

0

01

|

|

1|

|'

AAAAAAAAAAA

AA

AA

μx

μxxxxx

A

n

nn

jjj

Page 23: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2323

Proof of Result 5.1Proof of Result 5.1

12

0

/2

2

0

10000

1

0

1

10

1

100

11

ˆ

ˆ

11ˆˆ

'''

''1'

'')1(

n

T

n

Tnn

n

n

n

n

n

jjj

n

jjj

n

jjj

n

jjj

n

jjj

Σ

Σ

ΣΣ

μxμxμxμxxxxx

μxxxxxμxxxxx

μxμxxxxx

Page 24: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2424

Computing Computing TT22 from Determinants from Determinants

)1(

'

')1(

)1(ˆ

ˆ)1(

1

100

02

n

n

nn

T

n

jjj

n

jjj

xxxx

μxμx

Σ

Σ

Page 25: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2525

General Likelihood Ratio MethodGeneral Likelihood Ratio Method

cL

L

HH

H

L

)(max

)(max

if : offavor in Rejects

:

sample randomby function likelihood:)(

,parameters populationunknown :

0

010

00

θ

θ

Θθ

Θθ

θ

Θθθ

Θθ

Θθ

Page 26: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2626

Result 5.2Result 5.2

00

2

ofdimension ofdimension -

where variable,random aely approximat is

)(max

)(maxln2ln2

large, is size samplewhen

0

0

ΘΘ

θ

θ

Θθ

Θθ

L

LΛ-

n

Page 27: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2727

100(1100(1--))%% Confidence Region Confidence Region

level cesignificanat

offavor in :reject not test will

thefor which all of consistingRegion

1 truecover the will)(

where)R(:region confidence )%-100(1

array databy

determined valueslikely ofregion :)(

,parameters populationunknown :

1

00

0

H

H

RP

R

θθ

θ

θX

X

X

θX

Θθθ

Page 28: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2828

100(1100(1--))%% Confidence Region Confidence Region

)()1(

'

such that allby determined ellipsoid The

)(

of interval The

,1

21

12

pnp

n

Fpn

npn

txsxn

μxSμx

μ

:Case Normal teMultivaria

:Case Normal Univariate

Page 29: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

2929

Axes of the Confidence EllipsoidAxes of the Confidence Ellipsoid

iii

ipnpi Fpnn

np

eSe

e

x

where

)()(

)1(

are axes the,center at the beginning

,

Page 30: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3030

Example 5.3 :Example 5.3 :Microwave Oven RadiationMicrowave Oven Radiation

704.0710.0,002.0

710.0704.0,026.0

228.200391.163

391.163018.203

0146.00117.0

0117.00144.0,

603.0

564.0

opendoor with radiation measured

closeddoor with radiation measured

'22

'11

1

42

41

e

e

S

Sx

x

x

Page 31: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3131

Example 5.3 :Example 5.3 :95% Confidence Region95% Confidence Region

0.05 level cesignifican at the 589.0

562.0: offavor in

rejected benot would589.0

562.0: test By this

region. confidence 95% in the is

62.630.1

589.0603.0

562.0564.0

228.200391.163

391.163018.203589.0603.0562.0564.042

589.0562.0'

62.6)05.0(40

)41(2

603.0

564.0

228.200391.163

391.163018.203603.0564.042

1

0

40,2

2

121

μ

μ

μ

μ

H

H

F

Page 32: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3232

Example 5.3 : Example 5.3 : 95% Confidence Ellipse for 95% Confidence Ellipse for

018.0

)23.3()40(42

)41(2002.0)(

)(

)1(

064.0

)23.3()40(42

)41(2026.0)(

)(

)1(

:axesminor -semi andmajor -semi

603.0564.0'

:center

,2

,1

pnp

pnp

Fpnn

np

Fpnn

np

x

Page 33: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3333

Example 5.3 : Example 5.3 : 95% Confidence Ellipse for 95% Confidence Ellipse for

Page 34: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3434

Simultaneous Simultaneous Confidence StatementsConfidence Statements

Sometimes we need confidence Sometimes we need confidence statements about the individual statements about the individual component meanscomponent means

All if the separate confidence All if the separate confidence statements should hold statements should hold simultaneously with a specified high simultaneously with a specified high probabilityprobability

Page 35: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3535

Concept of Simultaneously Concept of Simultaneously Confidence StatementsConfidence Statements

Page 36: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3636

Confidence Interval of Linear Confidence Interval of Linear Combination of VariablesCombination of Variables

nt

nt

tt

n

ns

zt

sz

NZ

ZN

nn

n

z

Z

z

zZ

p

Saaxaμa

Saaxa

Saa

μaxa

Saaxa

ΣaaμaΣaaμa

XaΣμX

')(''

')('

)(

'

)''(

/

','

)','(:,','

'),,(:

11

21

2

2

Page 37: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3737

Maximum Maximum tt22 Value for All Value for All aa

μxS

a

μxSμx

Saa

μxa

Saa

μxa

Saa

μxa

aa

1

21

22

22

toalproportion for occurs maximum

'

'

'max

'

'max

'

'maxmax

Tn

nn

nt

aa

Page 38: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3838

Maximization LemmaMaximization Lemma

dBd

Bxx

dx

Bxx

dBdBxxdx

dBx

dBdBxx

dx

dB

x

12

12

1

12

0

''

'

0'

)')('('

:Proof

0for when attained maximum

''

'max

orgiven vect matrix, definite positive

cc

Page 39: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

3939

Result 5.3: Result 5.3: TT22 Interval Interval

1least at y probabilit with 'contain will

')(

)1(

and ')(

)1(

by determined

interval) ( interval the, allfor usly Simultaneo

),( from sample random:,,,

,

,

2

21

μa

Saaxa

Saaxa

a

ΣμXXX

pnp

pnp

pn

Fpnn

np'

Fpnn

np'

T

N

Page 40: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4040

Comparison of Comparison of tt- and - and TT22-Intervals-Intervals

Page 41: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4141

Simultaneous Simultaneous TT22-Intervals-Intervals

n

sF

pn

npx

n

sF

pn

npx

n

sF

pn

npx

n

sF

pn

npx

n

sF

pn

npx

n

sF

pn

npx

pppnppp

pppnpp

pnppnp

pnppnp

)()(

)1()(

)(

)1(

)()(

)1()(

)(

)1(

)()(

)1()(

)(

)1(

,,

22,22

22,2

11,11

11,1

Page 42: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4242

Example 5.4: Shadows of the Example 5.4: Shadows of the Confidence EllipsoidConfidence Ellipsoid

Page 43: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4343

Example 5.5Example 5.5

11.2337.2325.217

37.2305.12651.600

25.21751.60034.5691

,

13.25

69.54

59.526

87

sciencefor score CQT:

for verbal score CQT:

history and science socialfor score CLEP:

3

2

1

Sx

n

X

X

X

Page 44: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4444

Example 5.5Example 5.5

32

233322,32

32

321

1

,,

for interval confidence 95%least -atan is 3.1229.56 i.e.,

2)05.0(

)(

)1(

are interval confidence its of points end ,for ]1,1,0['

61.2665.23,16.5822.51,88.54930.50387

34.569129.859.526

87

34.569129.859.526

29.8)05.0(387

)187(3)(

)1(

n

sssF

pn

npxx

FFpn

np

pnp

pnppnp

a

Page 45: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4545

Example 5.5: Confidence Ellipses Example 5.5: Confidence Ellipses for Pairs of Meansfor Pairs of Means

Page 46: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4646

One-at-a-Time IntervalsOne-at-a-Time Intervals

n

stx

n

stx

n

stx

n

stx

n

stx

n

stx

ppnpp

ppnp

nn

nn

)2/()2/(

)2/()2/(

)2/()2/(

11

22122

2212

11111

1111

Page 47: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4747

Bonferroni InequalityBonferroni Inequality

m

m

ii

m

ii

ii

ii

ii

CPCP

CPCP

miCP

aC

21

11

'

1

true11false 1

false oneleast at 1 true all

,,2,1,1true

about statement confidence:

Page 48: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4848

Bonferroni Method of Bonferroni Method of Multiple ComparisonsMultiple Comparisons

n

s

ptx

n

s

ptx

n

s

ptx

n

s

ptx

mmm

in

s

mtxP

pmm

ppnpp

ppnp

nn

m

iii

ni

i

2

2

2

2

11

all, contains 2

,/

11

11111

1111

terms

1

Page 49: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

4949

Example 5.6Example 5.6

646.0560.0or 42

0146.0327.2603.00125.0

607.0521.0or 42

0144.0327.2564.00125.0

327.2)2

025.0(

025.02/05.0,2

2

22412

1

11411

41

n

stx

n

stx

t

p i

Page 50: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5050

Example 5.6Example 5.6

Page 51: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5151

(Length of Bonferroni Interval )/(Length of Bonferroni Interval )/(Length of (Length of TT22-Interval)-Interval)

Page 52: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5252

Limit Distribution of Limit Distribution of Statistical DistanceStatistical Distance

n-p

n

n

n-p

n

pnn

N

p

p

p

largefor

ely approximat:)()'(

large is

y whenprobabilithigh with toclose

largefor

ely approximat :)()'(

size sample largefor )1

,(nearly :

21

21

μXSμX

ΣS

μXΣμX

ΣμX

Page 53: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5353

Result 5.4Result 5.4

20

10

0100

21

'

if ,ely approximat cesignifican of level aat

,: offavor in rejected is :

large

covariance definite positive and mean with

population a from sample random:,,,

p

n

n

HH

pn

μxSμx

μμμμ

Σμ

XXX

Page 54: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5454

Result 5.5Result 5.5

1ely approximat

yprobabilit with ,every for ,'contain will

')('

large

covariance definite positive and mean with

population a from sample random:,,,

2

21

aμa

Saax

Σμ

XXX

na

pn

p

n

Page 55: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5555

Result 5.5Result 5.5

)-(1 confidence with ,contain

)(

, pairs allfor

)( )(

)( )(

statements confidence ussimultaneo )%-100(1

2

1

22

11211

1121

ki

pkk

ii

kkik

ikiikkii

ki

ppppp

pppp

pp

x

x

ss

ssxxn

n

sx

n

sx

n

sx

n

sx

Page 56: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5656

Example 5.7: Musical Aptitude Example 5.7: Musical Aptitude Profile for 96 Finish StudentsProfile for 96 Finish Students

Page 57: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5757

Example 5.7: Simultaneous 90% Example 5.7: Simultaneous 90% Confidence LimitsConfidence Limits

plausiblenot are componentsmeter tempo,melody,

22222331342731

studentsAmerican of Profile

13.2427.21

39.2361.20,93.2427.22

01.3639.32,75.3605.34

67.2853.24,14.3006.26

02.12)10.0(,)10.0(

'0

7

65

43

21

27

27

n

sx ii

i

Page 58: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5858

One-at-a-Time and Bonferroni ConfOne-at-a-Time and Bonferroni Confidence Intervalsidence Intervals

n

s

pzx

n

s

pzx

n

szx

n

szx

iiii

iii

iiii

iii

22

intervals confidence Bonferroni

22

intervals confidence time-a-at-One

Page 59: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

5959

Large-Sample 95% Intervals for Large-Sample 95% Intervals for Example 5.7Example 5.7

Page 60: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6060

95% Intervals for Example 5.795% Intervals for Example 5.7

Page 61: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6161

Control ChartControl Chart

Represents collected data to Represents collected data to evaluate the capabilities and stability evaluate the capabilities and stability of the processof the process

Identify occurrences of special Identify occurrences of special causes of variation that come from causes of variation that come from outside of the usual processoutside of the usual process

Page 62: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6262

Example 5.8: Overtime Hours for Example 5.8: Overtime Hours for a Police Departmenta Police Department

Page 63: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6363

Example 5.8 Example 5.8 Univariate Control ChartUnivariate Control Chart

Page 64: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6464

Monitoring a Sample for StabilityMonitoring a Sample for Stability

ondistributi

square-chi a as )()'( eApproximat

oft independennot isbut normal, is

1)(

0)(

),(

as ddistributetly independen:,,,

1

21

XXSXX

SXX

ΣXXCov

XX

Σμ

XXX

jj

j

j

j

p

n

n

n

E

N

Page 65: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6565

Example 5.9: Example 5.9: 99% Ellipse Format Chart99% Ellipse Format Chart

Page 66: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6666

Example 5.9: -Chart for Example 5.9: -Chart for XX22X

Page 67: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6767

Example 5.10: Example 5.10: TT22 Chart Chart for for XX11 and and XX22

Page 68: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6868

Example 5.11: Robotic WeldersExample 5.11: Robotic Welders

bleeach variaon nsobservatio

successivefor n correlatio serial eappreciabl No

reasonable is assumption Normal

(cfm) flow Gas (inert):

(in/min) speed Feed:

(amps)Current :

(volts) Voltage:

4

3

2

1

X

X

X

X

Page 69: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

6969

Example 5.11: Example 5.11: TT22 Chart Chart

Page 70: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7070

Example 5.11: 99% Quality Control Example 5.11: 99% Quality Control Ellipse for ln(Gas flow) and voltageEllipse for ln(Gas flow) and voltage

Page 71: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7171

Example 5.11: -Chart for Example 5.11: -Chart for ln(Gas flow)ln(Gas flow)

X

Page 72: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7272

Control Regions for Future Control Regions for Future Individual ObservationsIndividual Observations

Set for future observations from Set for future observations from collected data when process is stablecollected data when process is stable

Forecast or prediction regionForecast or prediction region– in which a future observation is in which a future observation is

expected to lieexpected to lie

Page 73: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7373

Result 5.6Result 5.6

pnp

pn

Fpn

pnn

nT

N

,

12

21

)1( as ddistribute is

'1

ondistributi same the

fromn observatio future:

),( astly independen:,,,

XXSXX

X

ΣμXXX

Page 74: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7474

Proof of Result 5.6Proof of Result 5.6

pnp

npp

Fpn

pn

n

n

WNn

n

n

nn

E

,1

1,

)1(:'

1

)(:),,0(:1

1

1)()()(

0)(

XXSXX

ΣSΣXX

Σ

ΣΣXCovXCovXXCov

XX

Page 75: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7575

Result 4.8Result 4.8

n

jj

n

jj

nn

n

j

n

jjjjpnn

jp

n

b

c

bbb

ccNccc

N

1

2

1

2

122112

1 1

222111

j

21

)()'(

)'()(

matrix covariancewith

normaljoint are and

)(,:

),(:

tindependenmutually :,,,

ΣΣcb

ΣcbΣ

VXXXV

ΣμXXXV

ΣμX

XXX

Page 76: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7676

Example 5.12 Control EllipseExample 5.12 Control Ellipse

Page 77: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7777

TT22-Chart for Future Observations-Chart for Future Observations

)05.0()(

)1(UCL

0 LCL

order in time

'1

Plot

,

12

pnpFpn

pn

n

nT

xxSxx

Page 78: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7878

Control Chart Based on Control Chart Based on Subsample MeansSubsample Means

ΣΣ

XX

XXXXX

XX

ΣXX

XXX

Σμ

nm

n

mn

n

n

n

n

n

nnnnn

nm

nN

nj

mN

j

njjj

j

pj

n

jjj

p

)1(1111

Cov1

Cov1

1

1111)

11(Cov

)Cov(

))1(

,0(:

1, at timemean subsample :

timesame at the sampled be units 1),,( :Process

2

2

12

2

111

1

Page 79: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

7979

Control Chart Based on Control Chart Based on Subsample MeansSubsample Means

1,

12

21

1

)(

:'1

),0(:1

)(:1

pnnmp

jj

pj

p,nm-nn

Fpnnm

pnnmn

nmT

Nn

nm

Wn

XXSXX

ΣXX

SSSS

Page 80: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8080

Control Regions for Future SubsamControl Regions for Future Subsample Observationsple Observations

1,1

1

1

1

)1(

)(:'

1

1Cov

1Cov

11Cov)Cov()Cov(

))1(

,(:

1,mean subsample future :

timesame at the sampled be units 1),,( :Process

pnnmp

n

pj

n

jj

p

Fpnnm

pnnm

n

nmnm

n

n

nn

nm

nN

n

mN

XXSXX

ΣXX

XXXXX

Σ0XX

XXX

Σμ

Page 81: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8181

Control Chart Based on Control Chart Based on Subsample MeansSubsample Means

1,

12

21

1

)(

:'1

),0(:1

)(:1

pnnmp

jj

pj

p,nm-nn

Fpnnm

pnnmn

nmT

Nn

nm

Wn

XXSXX

ΣXX

SSSS

Page 82: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8282

EM AlgorithmEM AlgorithmPrediction stepPrediction step– Given some estimate of the unknown Given some estimate of the unknown

parameters, predict the contribution of parameters, predict the contribution of the missing observations to the the missing observations to the sufficient statistics sufficient statistics

Estimation stepEstimation step– Use the predicted statistics to compute Use the predicted statistics to compute

a revised estimate of the parametersa revised estimate of the parameters

Cycle from one step to the otherCycle from one step to the other

Page 83: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8383

Complete-Data Sufficient StatisticsComplete-Data Sufficient Statistics

')1(1

'2

11

XXSXXT

XXT

nn

n

n

jjj

n

jj

Page 84: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8484

Prediction Step for Prediction Step for Multivariate Normal DistributionMultivariate Normal Distribution

'~~~,~;|'

'~~~~~~

~,~;|'

~~~~~,~;|~

~,~ estimatesGiven

of components available:

of components missing:

)2()1()2()2()1(~

)'2()1(

)1()1(21

1221211

)2()1()1(~

)'1()1(

)2()2(12212

)1()2()1()1(

)2(

)1(

jjjjjjj

jj

jjjjj

jjjj

jj

jj

E

E

E

xxΣμxXXxx

xxΣΣΣΣ

ΣμxXXxx

μxΣΣμΣμxXx

Σμ

xx

xx

Page 85: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8585

Result 4.6Result 4.6

211

221211

221

2211

221

22

2221

1211

2

1

2

1

covariance

and )(mean with normal

is given ofon distributi lconditiona

0,

|

|

,),,(:

ΣΣΣΣ

μxΣΣμ

xXX

Σ

ΣΣ

ΣΣ

Σ

μ

μ

μΣμ

X

X

X

pN

Page 86: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8686

Estimation Step for Estimation Step for Multivariate Normal DistributionMultivariate Normal Distribution

'~~~

~

~~

estimates likelihood

maximum revised theCompute

2

1

μμT

Σ

n

n

Page 87: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8787

Example 5.13: EM AlgorithmExample 5.13: EM Algorithm

1~,4

3~,4

1~,2

5~,2

1~

2

1

4

66656766~

4~,1~,6~,

5

215

627

30

1323123322

2222

11

321

X

Page 88: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8888

Example 5.13: Prediction StepExample 5.13: Prediction Step

18.170~

99.32~~~~~

73.5'~~~~~~

~|

~

~|

~

~~|~

~~|~

~~|~

~

~

~

~

~

~

~

131211

~

131211

21121

1221211

~211

3132121

2212111

2221

1211

332313

232212

131211

)2(

)1(

3

2

1

xxxxxx

xx

xxx

ΣΣΣ

ΣΣ

ΣΣ

ΣΣ

Σ

μ

μ

Page 89: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

8989

Example 5.13: Prediction StepExample 5.13: Prediction Step

~

4342

41~242

~

4241

~

4241

~241

3431

22122

1

~

42

41

2221

1211

332313

232212

131211

)2(

)1(

3

2

1

5.6

0.32,

97.127.8

27.806.41

3.1

4.6~~~~

~

~|

~

~|

~

~|~~

~|~~

~|~~

~

~

~

~

~

~

~

xx

x

xxx

xxx

xx

x

ΣΣ

ΣΣ

ΣΣ

Σ

μ

μ

Page 90: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

9090

Example 5.13: Prediction StepExample 5.13: Prediction Step

00.7450.2018.101

50.2097.627.27

18.10127.2705.148~

00.16

30.4

13.24~

~~~

2

43332313

42322212

41312111

1

T

T

xxxx

xxxx

xxxx

Page 91: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

9191

Example 5.13: Estimation StepExample 5.13: Estimation Step

50.283.017.1

83.059.033.0

17.133.061.0

'~~~1~

00.4

08.1

03.6~1~

2

1

μμTΣ

n

n

Page 92: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

9292

Time Dependence in ObservationsTime Dependence in Observations

)05.0()1(1

yprobabilit coverage actual

)05.0('such that all

interval confidence 95% nominal

1,

model AR(1) :

212

21

1

pp

p

ttt

P

n

μXSμXμ

εμXΦμX

Page 93: 1 Inferences about a Mean Vector Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking.

9393

Coverage Probability of Coverage Probability of the 95% Confidence Ellipsoidthe 95% Confidence Ellipsoid