Chapter 12 Simple Linear Regression n Simple Linear Regression Model n Least Squares Method n...

Chapter 12Chapter 12 Simple Linear Regression Simple Linear Regression

Simple Linear Regression ModelSimple Linear Regression Model Least Squares MethodLeast Squares Method Coefficient of DeterminationCoefficient of Determination Model AssumptionsModel Assumptions Testing for SignificanceTesting for Significance

Simple Linear Regression ModelSimple Linear Regression Model

yy = = 00 + + 11xx + +

where:where:

00 and and 11 are called are called parameters of the modelparameters of the model,,

is a random variable called theis a random variable called the error term error term..

The The simple linear regression modelsimple linear regression model is: is:

The equation that describes how The equation that describes how yy is related to is related to xx and and an error term is called the an error term is called the regression modelregression model..

Simple Linear Regression EquationSimple Linear Regression Equation

The The simple linear regression equationsimple linear regression equation is: is:

• EE((yy) is the expected value of ) is the expected value of yy for a given for a given xx value. value.• 11 is the slope of the regression line. is the slope of the regression line.• 00 is the is the yy intercept of the regression line. intercept of the regression line.• Graph of the regression equation is a straight line.Graph of the regression equation is a straight line.

EE((yy) = ) = 00 + + 11xx


Positive Linear RelationshipPositive Linear Relationship

EE((yy))EE((yy))

xxxx

Slope Slope 11

is positiveis positive

Regression lineRegression line

InterceptIntercept00


Negative Linear RelationshipNegative Linear Relationship

EE((yy))EE((yy))

xxxx

Slope Slope 11

is negativeis negative

Regression lineRegression lineInterceptIntercept00


No RelationshipNo Relationship

EE((yy))EE((yy))

xxxx

Slope Slope 11

is 0is 0

Regression lineRegression lineInterceptIntercept

00

Estimated Simple Linear Regression Estimated Simple Linear Regression EquationEquation

The The estimated simple linear regression estimated simple linear regression equationequation

0 1y b b x 0 1y b b x

• is the estimated value of is the estimated value of yy for a given for a given xx value. value.yy• bb11 is the slope of the line. is the slope of the line.• bb00 is the is the yy intercept of the line. intercept of the line.

• The graph is called the estimated regression line.The graph is called the estimated regression line.

Estimation ProcessEstimation Process

Regression ModelRegression Modelyy = = 00 + + 11xx + +

Regression EquationRegression EquationEE((yy) = ) = 00 + + 11xx

Unknown ParametersUnknown Parameters00, , 11

Sample Data:Sample Data:x yx y

xx11 y y11

. .. . . .. . xxnn yynn

bb00 and and bb11

provide estimates ofprovide estimates of00 and and 11

EstimatedEstimatedRegression EquationRegression Equation

Sample StatisticsSample Statistics

bb00, , bb11

0 1y b b x 0 1y b b x

Least Squares MethodLeast Squares Method

Least Squares CriterionLeast Squares Criterion

min (y yi i )2min (y yi i )2

where:where:

yyii = = observedobserved value of the dependent variable value of the dependent variable

for the for the iith observationth observation^yyii = = estimatedestimated value of the dependent variable value of the dependent variable

for the for the iith observationth observation


Slope for the Estimated Slope for the Estimated Regression EquationRegression Equation

n

xx

n

yxyx

bi

i

iiii

2

2

1

yy-Intercept for the Estimated Regression -Intercept for the Estimated Regression EquationEquation


0 1b y b x 0 1b y b x

where:where:xxii = value of independent variable for = value of independent variable for iithth observationobservation

nn = total number of observations = total number of observations

__yy = mean value for dependent variable = mean value for dependent variable

__xx = mean value for independent variable = mean value for independent variable

yyii = value of dependent variable for = value of dependent variable for iithth observationobservation

Reed Auto periodically hasReed Auto periodically has

a special week-long sale. a special week-long sale.

As part of the advertisingAs part of the advertising

campaign Reed runs one orcampaign Reed runs one or

more television commercialsmore television commercials

during the weekend preceding the sale. Data from aduring the weekend preceding the sale. Data from a

sample of 5 previous sales are shown on the next sample of 5 previous sales are shown on the next slide.slide.

Simple Linear RegressionSimple Linear Regression

Example: Reed Auto SalesExample: Reed Auto Sales

Simple Linear RegressionSimple Linear Regression

Example: Reed Auto SalesExample: Reed Auto Sales

Number ofNumber of TV AdsTV Ads

Number ofNumber ofCars SoldCars Sold

1133221133

14142424181817172727

Estimated Regression EquationEstimated Regression Equation

ˆ 10 5y x ˆ 10 5y x

1 2

( )( ) 205

( ) 4i i

i

x x y yb

x x

1 2

( )( ) 205

( ) 4i i

i

x x y yb

x x

0 1 20 5(2) 10b y b x 0 1 20 5(2) 10b y b x

Slope for the Estimated Regression EquationSlope for the Estimated Regression Equation

yy-Intercept for the Estimated Regression Equation-Intercept for the Estimated Regression Equation

Estimated Regression EquationEstimated Regression Equation

Scatter Diagram and Trend LineScatter Diagram and Trend Line

y = 5x + 10

0

5

10

15

20

25

30

0 1 2 3 4TV Ads

Ca

rs S

old

Coefficient of DeterminationCoefficient of Determination

Relationship Among SST, SSR, SSERelationship Among SST, SSR, SSE

where:where: SST = total sum of squaresSST = total sum of squares SSR = sum of squares due to regressionSSR = sum of squares due to regression SSE = sum of squares due to errorSSE = sum of squares due to error

SST = SSR + SST = SSR + SSE SSE

2( )iy y 2( )iy y 2ˆ( )iy y 2ˆ( )iy y 2ˆ( )i iy y 2ˆ( )i iy y

The The coefficient of determinationcoefficient of determination is: is:


where:where:

SSR = sum of squares due to regressionSSR = sum of squares due to regression

SST = total sum of squaresSST = total sum of squares

rr22 = SSR/SST = SSR/SST


rr22 = SSR/SST = 100/114 = .8772 = SSR/SST = 100/114 = .8772

The regression relationship is very strong; 88%The regression relationship is very strong; 88%of the variability in the number of cars sold can beof the variability in the number of cars sold can beexplained by the linear relationship between theexplained by the linear relationship between thenumber of TV ads and the number of cars sold.number of TV ads and the number of cars sold.

Sample Correlation CoefficientSample Correlation Coefficient

21 ) of(sign rbrxy 21 ) of(sign rbrxy

ionDeterminat oft Coefficien ) of(sign 1brxy ionDeterminat oft Coefficien ) of(sign 1brxy

where:where:

bb11 = the slope of the estimated regression = the slope of the estimated regression

equationequation xbby 10ˆ xbby 10ˆ

21 ) of(sign rbrxy 21 ) of(sign rbrxy

The sign of The sign of bb11 in the equation in the equation is “+”. is “+”.ˆ 10 5y x ˆ 10 5y x

=+ .8772xyr =+ .8772xyr

Sample Correlation CoefficientSample Correlation Coefficient

rrxyxy = +.9366 = +.9366

Assumptions About the Error Term Assumptions About the Error Term

1. The error 1. The error is a random variable with mean of zero. is a random variable with mean of zero.1. The error 1. The error is a random variable with mean of zero. is a random variable with mean of zero.

2. The variance of 2. The variance of , denoted by , denoted by 22, is the same for, is the same for all values of the independent variable.all values of the independent variable.2. The variance of 2. The variance of , denoted by , denoted by 22, is the same for, is the same for all values of the independent variable.all values of the independent variable.

3. The values of 3. The values of are independent. are independent.3. The values of 3. The values of are independent. are independent.

4. The error 4. The error is a normally distributed random is a normally distributed random variable.variable.4. The error 4. The error is a normally distributed random is a normally distributed random variable.variable.

Testing for SignificanceTesting for Significance

To test for a significant regression relationship, weTo test for a significant regression relationship, we must conduct a hypothesis test to determine whethermust conduct a hypothesis test to determine whether the value of the value of 11 is zero. is zero.

To test for a significant regression relationship, weTo test for a significant regression relationship, we must conduct a hypothesis test to determine whethermust conduct a hypothesis test to determine whether the value of the value of 11 is zero. is zero.

Two tests are commonly used:Two tests are commonly used: Two tests are commonly used:Two tests are commonly used:

tt Test Test andand FF Test Test

Both the Both the tt test and test and FF test require an estimate of test require an estimate of 22,, the variance of the variance of in the regression model. in the regression model. Both the Both the tt test and test and FF test require an estimate of test require an estimate of 22,, the variance of the variance of in the regression model. in the regression model.

An Estimate of An Estimate of


210

2 )()ˆ(SSE iiii xbbyyy 210

2 )()ˆ(SSE iiii xbbyyy

where:where:

ss 22 = MSE = SSE/( = MSE = SSE/(n n 2) 2)

The mean square error (MSE) provides the estimateThe mean square error (MSE) provides the estimate

of of 22, and the notation , and the notation ss22 is also used. is also used.


An Estimate of An Estimate of

2

SSEMSE

n

s2

SSEMSE

n

s

• To estimate To estimate we take the square root of we take the square root of 22..

• The resulting The resulting ss is called the is called the standard error ofstandard error of the estimatethe estimate..

Chapter 12 Simple Linear Regression n Simple Linear Regression Model n Least Squares Method n...

Documents

Transcript of Chapter 12 Simple Linear Regression n Simple Linear Regression Model n Least Squares Method n...