Post on 26-Mar-2015
Agenda of Week VII
Review of Week VI
Multiple regression
Canonical correlation
Review of Week VI
Inference on MV Mean Vector Two populations
Inference on MV Mean Vector Multi-populations: MANOVA
Multiple Regression
Basic assumptions Linearity
X is an n by K matrix with rank K
Regression
Homoscedasticity
y X
[ | ] 0E X
2[ ] [ ' | ]Var E X I
Multiple Regression
Hypotheses
0 : 0 . : 0aH vs H
0 : 0 . : 0i a iH vs H
By ANOVA
When the null hypothesis is rejected,
For all i.
Multiple Regression
' ( ) '( )
' ' ' ' ' '
' 2 ' ' '
Min y X y X
y y X y y X X X
y y X y X X
1
'2 ' 2 ' 0
( ' ) '
X y X X
X X X y
OLS model
OLS solution
Multiple RegressionANOVA table
Source Sum of squares d.f. Mean square F
Regression 2' 'b X y ny k- 1 2' '
1
b X y ny
k
Residual 'e e n- k 'e e
n k
Total 2'y y ny n- 1 2'
1
y y ny
n
2' '1
'
b X y nyk
e en k
Coefficient of
determination
22
'1
'
e eR
y y ny
Multiple Regression
2 2 1'. [ ] ( ' )
e es Est V s X X
n k
2
kk kkt
s S
t-statistics (d.f. = n-k)
Where Skk is the kth diagonal element of (X’X)-1
Standard error of the regression
Multiple Regression
2
2
kkt s S
Standard error of coefficient
Canonical Correlation
Correlation structure between a group of independent variables and a group of dependent variables A kind of multiple regression with more than two
dependent variables Example: Physical size group vs. Exercise group How much explained the variation of a group of
variables by other group of variables?
Canonical Correlation
Basic model
1
.
.i
p
X
X X
X
Group of independent variables
1
.
.i
q
Y
Y Y
Y
Group of dependent variables
( ) ( ), ( ) ( )
( , ) ( )XX XX YY YY
XY XY
Cov X S Cov Y S
Cov X Y S
Covariance matrixes
Canonical Correlation
Basic model
1 1
1 1
' ......
' ......
i i ip p
i i iq q
V X X X
W Y Y Y
LC of two groups of variables
' '( , ) ( ( , ) )
' ' ' 'XY XY
XX YY XX YY
SV W r V W
S S
Correlation between V and W
Canonical Correlation
Correlation testing
0 : 0 : 0xy a xyH vs H Hypotheses
22
1
3
1, 12 3
2 2
2 2
1
1
( 1)2
4( )1.5 ,2 5
p qp q ms
Wilks r
pqF
pqms
p qp qm n sp q
Correlation
between V and W
Canonical Correlation
,( , ) ( , )
. .
( ) ( ) 0,
( ) ( ) 1
Max V W or r V W
s t
E V E W
V V V W
Objective: To find α and β maximizing ρ or r.
Canonical Correlation
1 1' '2 2
1 1 1 1,XX YYV e S X W f S Y
1 1' '2 21 1' , 'XX YYe S f S
1st canonical variable
kth canonical variable
1 1' '2 2,k k XX k k YYV e S X W f S Y
Canonical Correlation
1 1 1( , )r r V W
1st canonical correlation coefficient
kth canonical coefficient coefficient
( , )k k kr r V W
Canonical Correlation
1 112 2
1 112 2
,
k
XX XY YY YX XX
k
YY YX XX XY YY
e eigen vector of kth eigen value of
S S S S S
f eigen vector of kth eigen value of
S S S S S
Canonical Correlation
( , ) , ( , )XX X YY Yr V X AS D r W Y BS D
Canonical loading matrix
Canonical cross loading matrix
( , ) , ( , )XY Y YX Xr V Y AS D r W X BS D