Linear Models Of Regression: Bias-Variance Decomposition ...
No Intercept Regression and Analysis of Variance.
14
No Intercept Regression and Analysis of Variance
-
Upload
gwendoline-owen -
Category
Documents
-
view
213 -
download
0
Transcript of No Intercept Regression and Analysis of Variance.
No Intercept Regression and Analysis of Variance
Example Data Set
Y X
5 20
6 23
7 27
8 33
8 31
9 35
10 43
5 19
6 25
7 29
8 31
Estimate two models
• Model with y-interceptY = a + b * X
Regression Statistics
Multiple R 0.984
R Square 0.969
Adjusted R Square 0.965
Standard Error 0.300
Observations 11
• Model no y-interceptY = b * X
Regression Statistics
Multiple R 0.999
R Square 0.998
Adjusted R Square 0.898
Standard Error 0.333
Observations 11
Observations
• The model with a y-intercept is more complex than the model with no y-intercept.
• One would expect then that the R2 of the model would decline when the y-intercept is removed. BUT, the R2 actually increases.
• If the explanatory power of the model, R2, increases, then the error of the model, Standard Error, should decrease. But, the Standard Error actually increases.
Analysis of Variance Tablemodel with y-intercept
ANOVA
df SS MS F Significance F
Regression 1 24.83 24.83 276.72 0.000000
Residual 9 0.81 0.09
Total 10 25.64
Analysis of Variance Tablemodel no y-intercept
ANOVA
df SS MS F Significance F
Regression 1 591.89 591.89 5338.74 0.000000
Residual 10 1.11 0.11
Total 11 593.00
Comparison of Sum of Squares
Y-intercept df SS
Regression 1 24.83
Residual 9 0.81
Total 10 25.64
No y-intercept df SS
Regression 1 591.89
Residual 10 1.11
Total 11 593.00
Revision of Sum of Squaresfor no-intercept model
No y-intercept df SS
Regression 1SST – SSE
25.64 – 1.11 = 24.53
Residual 9 1.11
Total 10 25.64
These are from the model with a y-intercept.
This is re-calculated.
Comparison of Revised Sum of Squares
Y-intercept df SS
Regression 1 24.83
Residual 9 0.81
Total 10 25.64
No y-intercept df SS
Regression 1 24.53
Residual 9 1.11
Total 10 25.64
Revised Statistics
• Model no y-interceptY = b * X
Regression Statistics
Multiple R 0.999
R Square 0.998
Adjusted R Square 0.898
Standard Error 0.333
Observations 11
SSR / SST= 24.53 / 25.64= 0.957
√ SSE / d.o.f.
= √ 1.11 / 9= 0.351
Comparison of Revised Statistics
• Model with y-interceptY = a + b * X
Regression Statistics
Multiple R 0.984
R Square 0.969
Adjusted R Square 0.965
Standard Error 0.300
Observations 11
• Model no y-interceptY = b * X
Regression Statistics
Multiple R 0.999
R Square 0.957
Adjusted R Square 0.898
Standard Error 0.351
Observations 11
Revised Observations
• The model with a y-intercept is more complex than the model with no y-intercept.
• One would expect then that the R2 of the model would decline when the y-intercept is removed. BUT, the R2 actually increases.
• If the explanatory power of the model, R2, increases, then the error of the model, Standard Error, should decrease. But, the Standard Error actually increases.
Analysis of Variance Tablemodel no y-intercept
REVISED
ANOVA
df SS MS F Significance F
Regression 1 24.53 24.53 199.43 0.000000
Residual 9 1.11 0.123
Total 10 25.64
SS / df
MSR/ MSE
FDIST (199.43, 1,9)
Summary
ANOVA
df SS MS F Significance F
Regression 1 Y Y Y Y
Residual X 1.11 Y
Total X X
When comparing a model with a y-intercept to the same model without a y-intercept
1. Revise the ANOVA table for the no-intercept model with values from the y-intercept model (X).
2. Recalculate necessary items (Y), and the R2 and the Standard Error.
