Christopher Dougherty EC220 - Introduction to econometrics (chapter 3) Slideshow: multicollinearity...
-
Upload
alisha-hames -
Category
Documents
-
view
232 -
download
5
Transcript of Christopher Dougherty EC220 - Introduction to econometrics (chapter 3) Slideshow: multicollinearity...
Christopher Dougherty
EC220 - Introduction to econometrics (chapter 3)Slideshow: multicollinearity
Original citation:
Dougherty, C. (2012) EC220 - Introduction to econometrics (chapter 3). [Teaching Resource]
© 2012 The Author
This version available at: http://learningresources.lse.ac.uk/129/
Available in LSE Learning Resources Online: May 2012
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms. http://creativecommons.org/licenses/by-sa/3.0/
http://learningresources.lse.ac.uk/
X2 X3 Y
10 19 51
11 21 56
12 23 61
13 25 66
14 27 71
15 29 76
MULTICOLLINEARITY
3232 XXY
12 23 XX
1
Suppose that Y = 2 + 3X2 + X3 and that X3 = 2X2 – 1. There is no disturbance term in the equation for Y, but that is not important. Suppose that we have the six observations shown.
MULTICOLLINEARITY
2
The three variables are plotted as line graphs above. Looking at the data, it is impossible to tell whether the changes in Y are caused by changes in X2, by changes in X3, or jointly by changes in both X2 and X3.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Change from previous observation
X2 X3 Y X2 X3 Y
10 19 51 1 2 5
11 21 56 1 2 5
12 23 61 1 2 5
13 25 66 1 2 5
14 27 71 1 2 5
15 29 76 1 2 5
MULTICOLLINEARITY
3
3232 XXY
12 23 XX
Numerically, Y increases by 5 in each observation. X2 changes by 1.
MULTICOLLINEARITY
4
Hence the true relationship could have been Y = 1 + 5X2.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 1 + 5X2 ?
MULTICOLLINEARITY
3232 XXY
12 23 XX
5
However, it can also be seen that X3 increases by 2 in each observation.
Change from previous observation
X2 X3 Y X2 X3 Y
10 19 51 1 2 5
11 21 56 1 2 5
12 23 61 1 2 5
13 25 66 1 2 5
14 27 71 1 2 5
15 29 76 1 2 5
MULTICOLLINEARITY
6
Hence the true relationship could have been Y = 3.5 +2.5X3.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 3.5 + 2.5X3 ?
MULTICOLLINEARITY
7
These two possibilities are special cases of Y = 3.5 – 2.5p + 5pX2 + 2.5(1 – p)X3, which would fit the relationship for any value of p.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 3.5 – 2.5p + 5pX2 + 2.5(1 – p)X3
MULTICOLLINEARITY
8
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 3.5 – 2.5p + 5pX2 + 2.5(1 – p)X3
There is no way that regression analysis, or any other technique, could determine the true relationship from this infinite set of possibilities, given the sample data.
MULTICOLLINEARITY
uXXY 33221 23 XX
9
What would happen if you tried to run a regression when there is an exact linear relationship among the explanatory variables?
MULTICOLLINEARITY
uXXY 33221 23 XX
10
We will investigate, using the model with two explanatory variables shown above. [Note: A disturbance term has now been included in the true model, but it makes no difference to the analysis.]
MULTICOLLINEARITY
uXXY 33221 23 XX
11
23322 XXYYXX iii
2
33222
332
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
The expression for the multiple regression coefficient b2 is shown above. We will substitute for X3 using its relationship with X2.
MULTICOLLINEARITY
uXXY 33221 23 XX
12
23322 XXYYXX iii
2
33222
332
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
222
2
222
2222
222
233 ][][
XX
XXXX
XXXX
i
ii
ii
First, we will replace the terms highlighted.
MULTICOLLINEARITY
uXXY 33221 23 XX
13
We have made the replacement.
222
222 XXYYXX iii
2
33222
2222
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
222
2
222
2222
222
233 ][][
XX
XXXX
XXXX
i
ii
ii
MULTICOLLINEARITY
uXXY 33221 23 XX
14
222
222 XXYYXX iii
2
33222
2222
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
222
2222
22223322 ][][
XX
XXXX
XXXXXXXX
i
ii
iiii
Next, the terms highlighted now.
MULTICOLLINEARITY
uXXY 33221 23 XX
15
222
222 XXYYXX iii
00
2222
222
2222
22233
2
XXXXXX
XXYYXXb
iii
iii
222
2222
22223322 ][][
XX
XXXX
XXXXXXXX
i
ii
iiii
We have made the replacement.
MULTICOLLINEARITY
uXXY 33221 23 XX
16
222
222 XXYYXX iii
00
2222
222
2222
22233
2
XXXXXX
XXYYXXb
iii
iii
YYXX
YYXX
YYXXYYXX
ii
ii
iiii
22
22
2233 ][][
Finally this term.
00
2222
222
2222
22222
2
XXXXXX
XXYYXXb
iii
iii
MULTICOLLINEARITY
uXXY 33221 23 XX
17
222
222 XXYYXX iii
YYXX
YYXX
YYXXYYXX
ii
ii
iiii
22
22
2233 ][][
Again, we have made the replacement.
MULTICOLLINEARITY
uXXY 33221 23 XX
18
222
222 XXYYXX iii
00
2222
222
2222
22222
2
XXXXXX
XXYYXXb
iii
iii
It turns out that the numerator and the denominator are both equal to zero. The regression coefficient is not defined.
MULTICOLLINEARITY
uXXY 33221 23 XX
19
222
222 XXYYXX iii
00
2222
222
2222
22222
2
XXXXXX
XXYYXXb
iii
iii
It is unusual for there to be an exact relationship among the explanatory variables in a regression. When this occurs, it s typically because there is a logical error in the specification.
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
20
However, it often happens that there is an approximate relationship. For example, when relating earnings to schooling and work experience, it if often reasonable to suppose that the effect of work experience is subject to diminishing returns.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
21
A standard way of allowing for this is to include EXPSQ, the square of EXP, in the specification. According to the hypothesis of diminishing returns, 4 should be negative.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
22
We fit this specification using Data Set 21. The schooling component of the regression results is not much affected by the inclusion of the EXPSQ term. The coefficient of S indicates that an extra year of schooling increases hourly earnings by $2.75.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 2, 537) = 67.54 Model | 22513.6473 2 11256.8237 Prob > F = 0.0000 Residual | 89496.5838 537 166.660305 R-squared = 0.2010-------------+------------------------------ Adj R-squared = 0.1980 Total | 112010.231 539 207.811189 Root MSE = 12.91
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
23
In the specification without EXPSQ it was 2.68, not much different.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
24
uEXPSQEXPSEARNINGS 4321
The standard error, 0.23 in the specification without EXPSQ, is also little changed and the coefficient remains highly significant.
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
25
uEXPSQEXPSEARNINGS 4321
By contrast, the inclusion of the new term has had a dramatic effect on the coefficient of EXP. Now it is negative, which makes little sense, and insignificant.
MULTICOLLINEARITY
26
uEXPSQEXPSEARNINGS 4321
Previously it had been positive and highly significant.
. reg EARNINGS S EXP
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 2, 537) = 67.54 Model | 22513.6473 2 11256.8237 Prob > F = 0.0000 Residual | 89496.5838 537 166.660305 R-squared = 0.2010-------------+------------------------------ Adj R-squared = 0.1980 Total | 112010.231 539 207.811189 Root MSE = 12.91
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
27
uEXPSQEXPSEARNINGS 4321
The coefficient of EXPSQ is also strange. It is positive, suggesting increasing returns to experience. However, it is not significant.
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
28
uEXPSQEXPSEARNINGS 4321
The reason for these problems is that EXPSQ is highly correlated with EXP. This makes it difficult to discriminate between the individual effects of EXP and EXPSQ, and the regression estimates tend to be erratic.
. cor EXP EXPSQ(obs=540)
| EXP EXPSQ------+------------------ EXP | 1.0000EXPSQ | 0.9812 1.0000
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
29
The high correlation causes the standard error of EXP to be larger than it would have been if EXP and EXPSQ had been less highly correlated, warning us that the point estimate is unreliable.
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
30
When high correlations among the explanatory variables lead to erratic point estimates of the coefficients, large standard errors and unsatisfactorily low t statistics, the regression is said to said to be suffering from multicollinearity.
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
31
Note that the coefficients remain unbiased and the standard errors remain valid.
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
32
Multicollinearity may also be caused by an approximate linear relationship among the explanatory variables. When there are only 2, an approximate linear relationship means there will be a high correlation, but this is not always the case when there are more than 2.
Copyright Christopher Dougherty 2011.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section 3.4 of C. Dougherty,
Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oup.com/uk/orc/bin/9780199567089/.
Individuals studying econometrics on their own and who feel that they might
benefit from participation in a formal course should consider the London School
of Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
20 Elements of Econometrics
www.londoninternational.ac.uk/lse.
11.07.25