Post on 07-Aug-2021
Chapter 3.Two-Variable Regression Model:The Problem of Estimation
Ordinary Least Squares Method (OLS)
Recall that, PRF: Yi = β1 + β2 Xi + ui
Thus, since PRF is not directly observable, it is estimated by SRF; that is,
iii uXY ˆˆˆ21 ++= ββ
And,
iii uYY ˆˆ +=
On Error Term More
iii YYu ˆˆ −=
iii uYY ˆˆ +=If
Then,
And,iii XYu 21
ˆˆˆ ββ −−=
On error term moreWe need to choose SRF in such a way that, error terms should be as
small as possible,
That is,
The sum of residuals which is represented by
( )∑ ∑ −= iii YYu ˆˆ
Should be as SMALL as possible
On Error Terms moreTherefore, the essential solution is to find a criterion in
order to minimize error disturbances in SRF.
All of the errors are to be as closer as possible to the
central line of SRF
Then, Least Squares Criterion Comes as a Solution
Least Squares Criterion is based on:
( )∑ ∑ −=22 ˆˆ iii YYu
( )∑ −−=2
21ˆˆ
ii XY ββ
Thus,
( )∑ = 212 ˆ,ˆˆ ββfui
Example to Least Squares Criterion
The first Model is Better?Why?
Sum of squares of Error disturbances of the second model is lower
Regression Equation
iii uXY ˆˆˆ21 ++= ββ
( )( )( )
( )
∑∑∑
∑∑ ∑
∑ ∑ ∑
=
−
−−=
−
−=
2i
2
222
x
ˆ
i
i
i
ii
ii
iiii
x
y
XX
YYXX
XXn
YXYXnβ
( )X
XXn
YXXYX
ii
iiiii
2
22
2
1
ˆ-Y
ˆ
β
β
=
−
−=
∑ ∑∑ ∑ ∑ ∑
Sample mean of YSample mean of X
The Classical Linear Regression Model (CLRM): The Assumptions Underlying The Method of Least Squares
The inferences about the true β1 and β2 are important because the estimated values of them are needed to be closer and closer to population values.
Therefore CLRM, which is the cornerstone of most econometric theory, makes 10 assumptions.
Assumptions of CLRM:Assumption 1. Linear Regression Model
The regression model is linear in the parameters, that is:
Yi = β1 + β2 Xi + ui
Assumption 2. X values are fixed in repeated sampling.More technically, X is assumed to be non-stochastic
X: 80$ income level → Y: 60$ weekly consumption of a familyX: 80$ income level → Y: 75$ weekly consumption of another family
Assumption 2 is known as: Conditional Regression Analysis, that is, conditional on the given values of the regressor(s) X.
Assumption 3. Zero Mean value of disturbance ui
( ) 0/ =ii XuE
Assumption 4. Homoscedasticity or Equal Variance of ui
( ) ( )[ ]( )
cefor varian stands var
3 Assumption of because /
//var
2
2
2
where
XuE
XuEuEXu
ii
iiiii
σ=
=
−=
Homoscedasticity vs Heteroscedasticity
( ) 2/var σ=ii Xu
Assumption 5. No Autocorrelation between the disturbances
Autocorrelation
If :
PRF: Yt = β1 + β2Xt + ut
And if ut and ut-1 are correlated, then Yt depends not only Xt, but also on ut-1.
Autocorrelation in Graphs
Assumption 6. Zero Covariance between ui and Xi.
Assumption 7.
Assumption 8.
Assumption 9.
Assumption 10. There is No Perfect Multicollinearity
That is, there is no perfect linear relationship among the explanatory variables.
tnnt uXXXY ++++= ββββ .....22110
High correlation among independent variables causes multicollinearity which also causes standard errors to be high, hypotheses to be inefficient (low t values), etc...
Properties of the Least-Squares Estimators: The Gauss-Markov Theorem
Gauss-Markov Theorem is the least squares approach of Gauss (1821) with the minimum variance approach of Markov (1900).
Standard error of estimate is simply the standard deviation of the Y values about the estimated regression line and is often used as a summary measure of the “goodness of fit” of the estimated regression line.
BLUE (Best Linear Unbiased Estimator)
1. An estimator is linear, that is, a linear function of a random variable, such as the dependent variable Y in the regression model.
2. An estimator is unbiased, that is, its average or expected value, E(β2), is equal to the true value, β2.
3. An estimator has minimum variance in the class of all such linear unbiased estimators; an unbiased estimator with the least variance is known as an efficient estimator.
Therefore, in the regression context it can be proved that the OLS estimators are BLUE which also sets the base of Gauss-Markov Theorem.
The Coefficient of Determination, r2: A Measure of “Goodness of Fit”
The coefficient of determination, r2 (two-variable case) or R2 (multiple regression) is a summary measure that tells how well the sample regression line fits the data.
The Ballentine View of R2
See Peter Kennedy, “Ballentine: A Graphical Aid for Econometrics”, Australian Economics Papers, Vol 20, 1981, 414-416. The name Ballentine is derived from the emblem of the well-known Ballantine beer with its circles.
Coefficient of Determination, r2
TSS = ESS + RSS
where;TSS = total sum of squaresESS = explained sum of squaresRSS = residual sum of squares
( )( ) ( )∑
∑∑∑
−+
−
−=
+=
2
2
2
2i ˆY
1
YY
u
YY
Y
TSSRSS
TSSESS
i
i
i
If TSS = ESS + RSS, then:
On r2 more:
R2 indicates the explained part of the regression model, therefore,
TSSESSr =2
And,
( )( ) TSS
ESS
YY
YYr
i
i=
−
−=∑∑
2
2
2ˆ
Alternatively,
( )
TSSRSSr
YY
ur
i
i
−=
−−=∑∑
1
ˆ1
2
2
22
Coefficient of Determination
Coefficient Of Determination
HW # 1:
Problem 3.20 (Chapter 3)
Consumer Prices and Money Supply in Japan
1982 to 2001