Multiple Regression
description
Transcript of Multiple Regression
![Page 1: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/1.jpg)
Multiple Regression
![Page 2: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/2.jpg)
Multiple Regression
• Multiple regression extends linear regression to allow for 2 or more independent variables.
• There is still only one dependent (criterion) variable.• We can think of the independent variables as ‘predictors’
of the dependent variable.• The main complication in multiple regression arises
when the predictors are not statistically independent.
![Page 3: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/3.jpg)
Example 1: Predicting Income
Age
Hours Worked
MultipleRegression Income
![Page 4: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/4.jpg)
Example 2: Predicting Final Exam Grades
Assignments
Midterm
MultipleRegression Final
![Page 5: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/5.jpg)
Coefficient of Multiple Determination
• The proportion of variance explained by all of the independent variables together is called the coefficient of multiple determination (R2).
• R is called the multiple correlation coefficient.• R measures the correlation between the predictions and
the actual values of the dependent variable.
• The correlation riY of predictor i with the criterion (dependent variable) Y is called the validity of predictor i.
![Page 6: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/6.jpg)
Uncorrelated Predictors
21 Yr 2
2Yr
Total variance
Variance explained by assignments Variance explained by midterm
2 2 2 2 21 2=Total proportion of variance explained = Y Y Y YR r r
![Page 7: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/7.jpg)
Uncorrelated Predictors• Recall the regression formula for a single predictor:
• If the predictors were not correlated, we could easily generalize this formula:
Y Xz rz
1 1 2 2Y Y Yz r z r z
![Page 8: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/8.jpg)
Example 1. Predicting Income
Correlations
1 .040* .229**.012 .000
3975 3975 3975.040* 1 .187**
.012 .000
3975 3975 3975
.229** .187** 1
.000 .0003975 3975 3975
Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)
N
Pearson CorrelationSig. (2-tailed)N
AGE
HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week
TOTAL INCOME
AGE
HOURSWORKEDFOR PAY
OR INSELF-
EMPLOYMENT - inReference Week
TOTALINCOME
Correlation is significant at the 0.05 level (2-tailed).*.
Correlation is significant at the 0.01 level (2-tailed).**.
![Page 9: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/9.jpg)
Correlated Predictors
21 Yr 2
2Yr
Total variance
Variance explained by assignments Variance explained by midterm
2 2 21 2=Total proportion of variance explained < Y YR r r
![Page 10: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/10.jpg)
Correlated Predictors
• Due to the correlation in the predictors, the optimal regression weights must be reduced:
1 1 2 2Yz z z
1 2 12 2 1 121 22 2
12 12
where
and 1 1
Y Y Y Yr r r r r rr r
1 2 beta weights (standardized partial re
andgres
are callesion coeffi
d thc
s)
eient
2 22 1 2 1 2 12
1 1 2 2 212
21
Y Y Y YY Y
r r r r rR r rr
![Page 11: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/11.jpg)
Raw-Score Formulas
0 1 1 2 2Y B B X B X
1 2
1 1 2 2
0 1 1 2 2
where
and
and
Y Y
X X
s sB Bs s
B Y B X B X
![Page 12: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/12.jpg)
Example 1. Predicting Income
Correlations
1 .040* .229**.012 .000
3975 3975 3975.040* 1 .187**
.012 .000
3975 3975 3975
.229** .187** 1
.000 .0003975 3975 3975
Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)
N
Pearson CorrelationSig. (2-tailed)N
AGE
HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week
TOTAL INCOME
AGE
HOURSWORKEDFOR PAY
OR INSELF-
EMPLOYMENT - inReference Week
TOTALINCOME
Correlation is significant at the 0.05 level (2-tailed).*.
Correlation is significant at the 0.01 level (2-tailed).**.
1 1 2 2Yz z z
1 2 12 2 1 121 22 2
12 12
where
and 1 1
Y Y Y Yr r r r r rr r
2 22 1 2 1 2 12
1 1 2 2 212
21
Y Y Y YY Y
r r r r rR r rr
![Page 13: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/13.jpg)
Example 1. Predicting Income
020
4060
80
0
20
40
60
800
1
2
3
4
5
6
7
x 104
Age (years)Hours worked per week (hours)
Ann
ual I
ncom
e (C
AD
)
![Page 14: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/14.jpg)
Degrees of freedom
1 wheresample sizenumber of predictors
df n knk
![Page 15: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/15.jpg)
Semipartial (Part) Correlations
• The semipartial correlations measure the correlation between each predictor and the criterion when all other predictors are held fixed.
• In this way, the effects of correlations between predictors are eliminated.
• In general, the semipartial correlations are smaller than the validities.
![Page 16: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/16.jpg)
Calculating Semipartial Correlations
• One way to calculate the semipartial correlation for a predictor (say Predictor 1) is to partial out the effects of all other predictors on Predictor 1and then calculate the correlation between the residual of Predictor 1 and the criterion.
• For example, we could partial out the effects of age on hours worked, and then measure the correlation between income and the residual hours worked.
![Page 17: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/17.jpg)
Calculating Semipartial Correlations
• A more straightforward method:
1 2 12(1.2) 2
121Y Y
Yr r rr
r
(1.2)where is the semipartial correlation between Predictor 1 and Yr Y
i.e., the correlation between and Predictor 1 after partialling out the effects of Predictor 2 on Predictor 1.
Y
![Page 18: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/18.jpg)
Example 2: Predicting Final Exam Grades
Assignments
Midterm
MultipleRegression Final
![Page 19: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/19.jpg)
Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)
Correlations
1 .356 .127.233 .680
13 13 13.356 1 .615*.233 .025
13 13 13.127 .615* 1.680 .025
13 13 13
Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)N
Assignments
Midterm
Final
Assignments Midterm Final
Correlation is significant at the 0.05 level (2-tailed).*.
212 120.356 0.127r r 2
1 120.127 0.016Yr r 22 20.615 0.378Y Yr r
![Page 20: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/20.jpg)
Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)
212 120.356 0.127r r 2
1 120.127 0.016Yr r 22 20.615 0.378Y Yr r
1 1 2 2Yz z z
1 2 12 2 1 121 22 2
12 12
where
and 1 1
Y Y Y Yr r r r r rr r
2 22 1 2 1 2 12
1 1 2 2 212
21
Y Y Y YY Y
r r r r rR r rr
![Page 21: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/21.jpg)
Example 2. Predicting Final Exam Grades
0 1 1 2 2Y B B X B X
1 2
1 1 2 2
0 1 1 2 2
where
and
and
Y Y
X X
s sB Bs s
B Y B X B X
![Page 22: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/22.jpg)
Example 2. Predicting Final Exam Grades
7080
90100
2040
6080
0
50
100
150
Assignment grade (%)Midterm grade (%)
Fina
l gra
de (%
)
![Page 23: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/23.jpg)
SPSS Output
![Page 24: Multiple Regression](https://reader036.fdocuments.us/reader036/viewer/2022081520/568164a6550346895dd69605/html5/thumbnails/24.jpg)
Example 3. 2006-07 6130 Grades
• Try doing the calculations on this dataset for practice.