Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.....
-
date post
21-Dec-2015 -
Category
Documents
-
view
221 -
download
2
Transcript of Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.....
![Page 1: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/1.jpg)
Multivariate Regression Model
y = x1 + x2 + x3 +… +
The OLS estimates b0,b1 ,b2 , b3 .. …. are sample statistics used to estimate respectively
y is the DEPENDENT variableEach of the xj is an INDEPENDENT variable
![Page 2: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/2.jpg)
Conditions:
Each explanatory variable Xj is assumed
(1A) to be deterministic or non-random
(1B) : to come from a ‘fixed’ population
(1C) : to have a variance V(xj) which is not ‘too large’The above assumptions are best suited to a situation of a controlledexperiment
![Page 3: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/3.jpg)
Assumptions concerning the random term
(IIA) E(i ) = 0 for all i
(IIB) Var(i) = constant for all i
(IIC) Covariance (k) = for any iand k (IID) Each of the i has a normal
distribution
![Page 4: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/4.jpg)
Properties of b0 , b1 , b2 , b3
1. Each of these statistics is a linear functions of the Y values.
2. Therefore, they all have normal distributions
3. Each is an unbiased estimator.
That is, E(bk) =
![Page 5: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/5.jpg)
4. Each bk is the most efficient estimator of all unbiased estimators.
![Page 6: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/6.jpg)
Best
Linear
Unbiased
Estimator of the respective parameter
Thus, each of b0 , b1 , b2 ….is
![Page 7: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/7.jpg)
Conclusion
Each estimator bi has a normal distribution with mean = and variance = bi
2 where bi2 is
unknown.
![Page 8: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/8.jpg)
Income (£ per week) of an individual is regressed on a constant, education (in years), age (in years) and wealth inheritance (in £), using EViews.
Number of observations is 20 and the regression output is given below:
![Page 9: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/9.jpg)
Variable Coefficient Std.Error t-Stats Prob. C -1001.87 520.71 -1.92 0.0654
AGE 8.85 5.45 1.62 0.1168
EDUCATION 95.17 38.54 2.46 0.0252
WEALTH 1.51 0.46 3.26 0.0031
![Page 10: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/10.jpg)
SignificanceLevel (
The Maximum Type 1 Error= SignificanceLevel
![Page 11: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/11.jpg)
The smaller the p-value the more significant is the test
p-value
![Page 12: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/12.jpg)
The proposed regression model is:
Income = ß0 + ß1(Age) +ß2(Education)
+ ß3(Wealth Inheritance)
… . . (A)
We are proposing that Income is the variable dependent on three independent variables: Age, Education and Wealth.
![Page 13: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/13.jpg)
0 is a constant.
It measures the effect of other deterministic factors on Income not included in the model.
1 , 2, 3 measure the effect of a marginal
change in Age, Education and Wealth,
respectively.
![Page 14: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/14.jpg)
However, we recognise that there may be other random factors affecting the dependent variable Income.
So we add a random variable to the model which now becomes:
Income = ß0 + ß1(Age) +ß2(Education)
+ ß3(Wealth Inherited) + … . . (B)
![Page 15: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/15.jpg)
We use the least squares technique to estimate the model B.
Therefore, our estimation of the proposedmodel B is Ye = -1001.87 + 8.85*AGE + 95.17*EDUCATION + 1.51*WEALTH INHERITANCE
Here Ye is the estimated value of income
![Page 16: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/16.jpg)
-1001.87 is the estimate of ß0, 8.85 is
the estimate of ß1,; 95.17 is the estimate
of ß2 and 1.51 is the estimate of ß3
The least-squares estimates of the ß-values are denoted by b-values. Thus, b1 is the estimate of ß1 and b2 is the
estimate of ß2 . In our case, b1 = 8.85
and b2 = 95.17.
![Page 17: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/17.jpg)
We next make the following assumptions on the specification of model B so that the least-squares method produces ‘good’ estimators.
![Page 18: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/18.jpg)
i. is normally distributed with mean 0 and an unknown variance 2
. In the context of the model B, can be thought
of as a luck factor which can be good (positive values) or bad (negative values),
If the positive and negative values cancel out on average, we can say that mean value is 0.
![Page 19: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/19.jpg)
The values are uncorrelated across the population
(Whether or not you are lucky does not influence my being lucky/unlucky)
i. The values have the same variance (2)
across it. (Every individual is exposed to the same extent/chance of good or bad luck)
![Page 20: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/20.jpg)
The values are uncorrelated with the independent variables Age, Education and Wealth Inheritance.
(For example, an old person is as likely to be lucky as a young one;
or a university graduate is as likely to be unlucky as someone with no A-levels).
![Page 21: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/21.jpg)
We now test (at 10% significance) the following hypothesis:
Education has a positive effect on income Step 1: Set up the hypotheses
H0 : ß2 = 0 (Education has no effect)
H1 : ß2 > 0(Education has a positive effect)
one-tailed test
![Page 22: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/22.jpg)
Step 2: Select statistic
The estimator b2 is the test-statistic
Step3 : Identify the distributionof b2
![Page 23: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/23.jpg)
Best Linear in the dependent variable incomeUnbiased Estimator of 2
Assumptions i-iii above imply that b2 is
![Page 24: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/24.jpg)
Since b2 is unbiased, E(b2) = 2
b2 has a normal distribution because it
is linear in Income
Thus, b2~ N(2, 22) where
is unknown.
![Page 25: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/25.jpg)
Therefore, the test statistic ist (b2- 2) / (standard error of b2)
has a Student’s t-distribution with 20-4 = 16 d.o.f.
Step 4: Construct test statistic We use the standard error of b2
because we do not know what is
![Page 26: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/26.jpg)
EViews therefore gives us a t-statistic regarding education of 2.46907
As 2 = 0 under the null hypothesis (H0)
t = b2 / (standard error of b2)
The corresponding probability value is 0.0252.
![Page 27: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/27.jpg)
Select fx /TDIST. For X, enter 2.469607,
the t-Statistic value. The degree of freedomis 16. EViews calculates two-tail probabilitySo number of tails is 2. You now get the 2-tail probability of 0.025165 from Excel.
Since we are performing a one-tail test, take half the probability value, or 0.0126 .
![Page 28: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/28.jpg)
Step 5: Compare with critical value tC
tC = 1.336757for a one-tailed test with significance level () = 0.1 and d.o.f. = 16
tC = 1.336757 < 2.469607
![Page 29: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/29.jpg)
Step 6 : Draw conclusion
The test is significant. Reject H0 at 10% and at 5% (1.745884 < 2.469607) butnot at 1% (2.583492 > 2.469607)
Step 7: Interpret result
The data supports (with at least 98% accuracy) the hypothesis that EDUCATION is an important explanatory variable affecting income.
![Page 30: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/30.jpg)
The probability of a type 1 error is nothing but the area to the right of t-statistic, or 0.0126.
In rejecting H0, we are prone to make a
Type 1 Error.
![Page 31: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/31.jpg)
Example 2: Use output 2 to test the hypothesis (at 5% significance) that weightgain is proportional to foodvalue.
H0 : a = 0 (proportionality) H1 : a 0 (non-proportionality)
The estimator a is the test-statistic
Step 1:
Step 2:
The Model :: y = x + and add the assumptions (Lec17)
![Page 32: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/32.jpg)
Conditions:
The explanatory variable X is assumed
(1A) to be deterministic or non-random
(1B) : to come from a ‘fixed’ population
(1C) : to have a variance V(x) which is not ‘too large’The above assumptions are best suited to a situation of a controlledexperiment
![Page 33: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/33.jpg)
Assumptions concerning the random term
(IIA) E(i ) = 0 for all i
(IIB) Var(i) = constant for all i
(IIC) Covariance (j) = for any iand j (IID) Each of the i has a normal
distribution
![Page 34: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/34.jpg)
Step 3:Thus, a~ N(, ) where is unknown.
Step 4:Therefore, the test statistict (a- ) / (standard error of a) has a Student’s t-distribution with 10-2 = 8 d.o.f.
![Page 35: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/35.jpg)
The p-value is 0.0169 < 0.05
Foodvalue is not the only variable that affects weightgain
Step 6: Draw conclusion
Step 5: Compare with critical value tC
tC = -2.31 > -3.005262
tC = -2.31 for a two-tailed test with significance level () = 0.05 and d.o.f.= 8
The test is significant. Reject H0 at 5%
Step 7: Interpret
![Page 36: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/36.jpg)
Use output 3 to test (at 5% significance) the following hypothesis: Exercise has a negative effect on weight
gain The proposed regression model is:
Weightgain
= ß0 + ß1(Foodvalue) +ß2(Exercise) +
Example 3:
![Page 37: Multivariate Regression Model y = x1 + x2 + x3 +… + The OLS estimates b 0,b 1,b 2, b 3.. …. are sample statistics used to estimate](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d6a5503460f94a48140/html5/thumbnails/37.jpg)
Step 1: Set up the hypotheses
H0 : ß2 = 0 (Exercise has no effect)
H1 : ß2 < 0(Exercise has a negative effect)