Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter...
-
Upload
cameron-fox -
Category
Documents
-
view
214 -
download
0
Transcript of Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter...
![Page 1: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/1.jpg)
Linear Regression Basics IIIViolating Assumptions
Fin250f: Lecture 7.2
Spring 2010
Brooks, chapter 4(skim)
4.1-2, 4.4, 4.5, 4.7, 4.9-13
![Page 2: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/2.jpg)
Outline
Violating assumptionsParameter stabilityModel building
![Page 3: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/3.jpg)
OLS Assumptions
Error variancesError correlationsError normalityFunctional forms and linearityOmitting variablesAdding irrelevant variables
![Page 4: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/4.jpg)
Error Variance
var(ut )=σ2
var(ut) =σ t2
OLS unbiased, consistent, but inefficient
Weighting observations by noise (ARCH/GARCH)
![Page 5: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/5.jpg)
Error VarianceWhich is a bigger error?
yt
yt
*
*
*
***
Y
X
![Page 6: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/6.jpg)
Error Correlations
E(utut+ j ) ≠0
Patterns in residualsPlot residuals/residual diagnosticsFurther modeling necessary
If you can forecast u(t+1), need to work harder
![Page 7: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/7.jpg)
Error Normality
Skewness and kurtosis in residualsTesting
Plots Bera-Jarque test
How can this impact results?
![Page 8: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/8.jpg)
Bera-Jarque Test for Normality
b1 =E(u3)σ 3 ,b2 =
E(u4 )σ 4
W =Tb12
6+(b2 −3)2
24⎛
⎝⎜⎞
⎠⎟
W : χ 2 (2)
![Page 9: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/9.jpg)
Nonnormal Errors: Impact
For some theory: No In practice can be big problem Many extreme data points Forecasting models work hard to fit these
extreme outliers Some solutions:
Drop data points Robust forecast objectives (absolute errors)
![Page 10: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/10.jpg)
Functional Forms
Y=a+bXActual function is nonlinearSeveral types of diagnostics
Higher order (squared) terms (RESET) Think about specific nonlinear models
Neural networks Threshold models
Tricky: More later
![Page 11: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/11.jpg)
yt =a+β1xt,1 +β2xt,2 +ut
Omitting Variables
Leave out x(2)
If it is correlated with x(1) this is a problem.
Beta(1) will be biased and inconsistent.
Forecast will not be optimal
![Page 12: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/12.jpg)
Irrelevant Variables
Overfitting/data snooping Model fits to noise
Impacts standard errors for coefficientsCoefficients still consistent and
unbiased
![Page 13: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/13.jpg)
Parameter Stability
Known break point Chow test Predictive failure test
Unknown break Quant likelihood ratio test Recursive least squares
![Page 14: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/14.jpg)
Chow Test
yt =a+ βxt + ut
yt1 =a1 +β1xt
1 + ut1
yt2 =a2 +β 2xt
2 + ut2
RSS= u∑ t
2
RSS1 = (∑ ut1)2
RSS2 = (∑ ut2 )2
![Page 15: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/15.jpg)
Chow Test
RSS−(RSS1 + RSS2 )RSS1 + RSS2
(T −2k)k
k=number of regressors
2k = number of regressors unrestricted
Test statistic F(k,T-2k)
![Page 16: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/16.jpg)
Predictive Failureyt =a+ βxt + ut
yt1 =a1 +β1xt
1 + ut1, Large subsample 1,T1
RSS= ut2
t=1
T
∑
RSS1 = (t=1
T1
∑ ut1)2
T2 =T −T1
![Page 17: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/17.jpg)
Predictive Failure
(RSS−RSS1) /T2
RSS1 / (T1 −k)=
Expected squared error at endExpected squared error before end
F(T2 ,T1 −k)
![Page 18: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/18.jpg)
Unknown Breaks
Search for breakLook for maximum Chow levelDistribution is tricky
Monte-carlo/bootstrap
![Page 19: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/19.jpg)
Recursive/rolling estimation
Recursive Estimate (1,T1) move T1 to full sample T See if parameters converge
Rolling Roll bands (t-T,t) through data Watch parameters move through time
We’ll use some of these
![Page 20: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/20.jpg)
Pure Out of Sample Tests
Estimate parameters over (1,T1)Get errors over (T1+1,T)
![Page 21: Linear Regression Basics III Violating Assumptions Fin250f: Lecture 7.2 Spring 2010 Brooks, chapter 4(skim) 4.1-2, 4.4, 4.5, 4.7, 4.9-13.](https://reader036.fdocuments.us/reader036/viewer/2022062519/5697bfbf1a28abf838ca36ad/html5/thumbnails/21.jpg)
Model Construction
General -> specific Less financial theory More statistics Problems: large unwieldy models
Simple -> general More theory at the start Problems: can leave out important stuff