Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)
description
Transcript of Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)
![Page 1: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/1.jpg)
Regression Concept & Examples,Latent Variables,
&Partial Least Squares (PLS)
1
![Page 2: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/2.jpg)
Simple Regression Model• Make prediction about the starting salary of a current college
graduate• Data set of starting salaries of recent college graduates
2
Data Set Compute Average Salary
How certain are of this prediction?There is variability in the data.
![Page 3: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/3.jpg)
Compute Total Variation
Simple Regression Model
• The smaller the amount of total variation the more accurate (certain) will be our prediction.
• Use total variation as an index of uncertainty about our prediction
3
![Page 4: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/4.jpg)
Simple Regression Model• How “explain” the variability - Perhaps it depends on
the student’s GPA
4
Salary GPA
![Page 5: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/5.jpg)
• Find a linear relationship between GPA and starting salary• As GPA increases/decreases starting salary increases/decreases
5
Simple Regression Model
![Page 6: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/6.jpg)
• Least Squares Method to find regression model– Choose a and b in regression model (equation) so that it minimizes the sum
of the squared deviations – actual Y value minus predicted Y value (Y-hat)
6
Simple Regression Model
![Page 7: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/7.jpg)
• How good is the model?
7
Simple Regression Model
u-hat is a “residual” value The sum of all u-hats is zeroThe sum of all u-hats squared is the total variance not explained by the model“unexplained variance” is 7,425,926
a= 4,779 & b = 5,370 A computer program computed these values
![Page 8: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/8.jpg)
Simple Regression Model
8Total Variation = 23,000,000
![Page 9: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/9.jpg)
9
Simple Regression Model
Total Unexplained Variation = 7,425,726
![Page 10: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/10.jpg)
Simple Regression Model
• Relative Goodness of Fit– Summarize the improvement in prediction using regression model
• Compute R2 – coefficient of determination
10
Regression Model (equation) a better predictor than guessing the average salaryThe GPA is a more accurate predictor of starting salary than guessing the averageR2 is the “performance measure“ for the model.
Predicted Starting Salary = 4,779 + 5,370 * GPA
![Page 11: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/11.jpg)
Detailed Regression Example
11
![Page 12: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/12.jpg)
Data Set
12
Obs # Salary GPA Months Work1 20000 2.8 482 24500 3.4 243 23000 3.2 244 25000 3.8 245 20000 3.2 486 22500 3.4 367 27500 4.0 208 19000 2.6 489 24000 3.2 36
10 28500 3.8 12
![Page 13: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/13.jpg)
Scatter Plot - GPA vs Salary
13
![Page 14: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/14.jpg)
Scatter Plot - Work vs Salary
14
![Page 15: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/15.jpg)
Pearson Correlation Coefficients-1 <= r <= 1
15
Salary GPAMonths Work
Salary 1 GPA 0.898007 1 Months Work -0.93927 -0.82993 1
![Page 16: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/16.jpg)
Three Regressions
• Salary = f(GPA)• Salary = f(Work)• Salary = f(GPA, Work)• Interpret Excel Output
16
![Page 17: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/17.jpg)
Interpreting Results
• Regression Statistics– Multiple R, – R2, – R2
adj
– Standard Error Sy
• Statistical Significance– t-test– p-value– F test
17
![Page 18: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/18.jpg)
Regression Statistics Table
18
• Multiple R– R = square root of R2
• R2
– Coefficient of Determination
• R2adj
– used if more than one x variable
• Standard Error Sy
– This is the sample estimate of the standard deviation of the error (actual – predicted)
![Page 19: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/19.jpg)
ANOVA Table
19
• Table 1 gives the F statistic • Tests the claim – there is no significant relationship between your
all of your independent and dependent variables
• The significance F value is a p-value• should reject the claim: – Of NO significant relationship between your independent and dependent variables if p<
– Generally = 0.05
![Page 20: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/20.jpg)
Regression Coefficients Table
20
• Coefficients Column gives – b0 , b1, ,b2 , … , bn values for the regression equation.
– The b0 is the intercept
– b1value is next to your independent variable x1
– b2 is next to your independent variable x2.
– b3 is next to your independent variable x3
![Page 21: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/21.jpg)
Regression Coefficients Table
• p values for individual t tests each independent variables
• t test - tests the claim that there is no relationship between the independent variable (in the corresponding row) and your dependent variable.
• Should reject the claim• Of NO significant relationship between your independent variable (in
the corresponding row) and dependent variable if p<.
21
![Page 22: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/22.jpg)
Salary = f(GPA)
22
Regression Statistics f(GPA)Multiple R 0.898006642R Square 0.806415929Adjusted R Square 0.78221792Standard Error 1479.019946Observations 10
ANOVA
df SS MS F Significance FRegression 1 72900000 72900000 33.32571 0.00041792Residual 8 17500000 2187500 Total 9 90400000
CoefficientsStandard Error t Stat P-value Lower 95% Upper 95%
Intercept 1928.571429 3748.677 0.514467 0.620833 -6715.89326 10573.04GPA 6428.571429 1113.589 5.772843 0.000418 3860.63173 8996.511
![Page 23: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/23.jpg)
Salary = f(Work)
23
Regression Statistics f(Work) Multiple R 0.939265177R Square 0.882219073Adjusted R Square 0.867496457Standard Error 1153.657002Observations 10
ANOVA df SS MS F Significance F
Regression 1 79752604.17 79752604 59.92271 5.52993E-05Residual 8 10647395.83 1330924 Total 9 90400000
CoefficientsStandard Error t Stat P-value Lower 95% Upper 95%
Intercept 30691.66667 1010.136344 30.38369 1.49E-09 28362.28808 33021.0453Months Work -227.864583 29.43615619 -7.74098 5.53E-05
-295.7444812 -159.98469
![Page 24: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/24.jpg)
Salary = f(GPA, Work)
24
Regression Statistics f(GPA,Work)Multiple R 0.962978985R Square 0.927328525Adjusted R Square 0.906565246Standard Error 968.7621974Observations 10
ANOVA df SS MS F Significance F
Regression 2 83830499 41915249 44.66195 0.00010346Residual 7 6569501 938500.2 Total 9 90400000
CoefficientsStandard Error t Stat P-value Lower 95% Upper 95%
Intercept 19135.92896 5608.184 3.412144 0.011255 5874.682112 32397.176GPA 2725.409836 1307.468 2.084495 0.075582 -366.2602983 5817.08Months Work -151.2124317 44.30826 -3.41274 0.011246 -255.9848174
-46.440046
![Page 25: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/25.jpg)
Compare Three “Models”
25
Regression Statistics f(GPA,Work)Multiple R 0.962978985R Square 0.927328525Adjusted R Square 0.906565246Standard Error 968.7621974Observations 10
Regression Statistics f(Work) Multiple R 0.939265177R Square 0.882219073Adjusted R Square 0.867496457Standard Error 1153.657002Observations 10
Regression Statistics f(GPA)Multiple R 0.898006642R Square 0.806415929Adjusted R Square 0.78221792Standard Error 1479.019946Observations 10
![Page 26: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/26.jpg)
Latent Variables(Theoretical Entities)
26
![Page 27: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/27.jpg)
Latent Variables• Latent Variables– Explanatory Variables that are not directly
measured– Identified by “Exploratory Factor Analysis”– Confirmed by “Confirmatory Factor Analysis”
• Statistical Methods for Latent Variables– Principles Components Analysis– PLS– SEM
27
![Page 28: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/28.jpg)
Example: Confirmatory Factor Analysis
Intention to Use Travelocity Website
28
![Page 29: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/29.jpg)
Research Instrument
29
![Page 30: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/30.jpg)
30
![Page 31: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/31.jpg)
31
![Page 32: Regression Concept & Examples, Latent Variables, & Partial Least Squares (PLS)](https://reader035.fdocuments.us/reader035/viewer/2022062321/56813509550346895d9c5a49/html5/thumbnails/32.jpg)
32