7172017 httpnumericalmethodsengusfedu 1

Linear Regression

Major All Engineering Majors

Authors Autar Kaw Luke Snyder

httpnumericalmethodsengusfeduTransforming Numerical Methods Education for STEM

Undergraduates

Linear Regression

httpnumericalmethodsengusfedu

httpnumericalmethodsengusfedu 3

What is RegressionWhat is regression Given n data points

best fit )(xfy = to the data

Residual at each point is

)(xfy =

Figure Basic model for regression

)()()( 2211 nn yxyxyx

)( iii xfyE minus=

y

x

)( 11 yx

)( nn yx)( ii yx

)( iii xfyE minus=

httpnumericalmethodsengusfedu 4

Linear Regression-Criterion1Given n data points best fit xaay 10 += to the data

Does minimizingsum=

n

iiE

1work as a criterion

x

xaay 10 +=)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

)()()( 2211 nn yxyxyx

httpnumericalmethodsengusfedu 5

Example for Criterion1

x y

20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion1

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

httpnumericalmethodsengusfedu 4

Linear Regression-Criterion1Given n data points best fit xaay 10 += to the data

Does minimizingsum=

n

iiE

1work as a criterion

x

xaay 10 +=)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

)()()( 2211 nn yxyxyx

httpnumericalmethodsengusfedu 5

Example for Criterion1

x y

20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion1

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

httpnumericalmethodsengusfedu 5

Example for Criterion1

x y

20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion1

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet2

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet3

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

httpnumericalmethodsengusfedu 6

Linear Regression-Criteria1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x minus 4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Sheet3

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

httpnumericalmethodsengusfedu 7

Linear Regression-Criterion1

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

04

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet3

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

httpnumericalmethodsengusfedu 8

Linear Regression ndash Criterion 1

04

1=sum

=iiE for both regression models of y=4x-4 and y=6

The sum of the residuals is minimized in this case it is zero but the regression model is not unique

Hence the criterion of minimizing the sum of the residuals is a bad criterion

0

2

4

6

8

10

0 1 2 3 4

y

x

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet3

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

httpnumericalmethodsengusfedu 9

Linear Regression-Criterion1

04

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y=4x-4 and y vs x data

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

Sheet3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

httpnumericalmethodsengusfedu 10

Linear Regression-Criterion2

x

)( 11 yx

)( 22 yx)( 33 yx

)( nn yx

)( ii yx

iii xaayE 10 minusminus=

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

Will minimizing ||1sum=

n

iiE work any better

xaay 10 +=

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

httpnumericalmethodsengusfedu 11

Example for Criterion2

x y20 40

30 60

20 60

30 80

Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2

Figure Data points for y vs x data

Table Data Points

0

2

4

6

8

10

0 1 2 3 4

y

x

Minimizesum=

n

iiE

1||

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

Chart1

x

y

4

6

6

8

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

2
3
2
3

Sheet1

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

	x		y
	2		4
	3		6
	2		6
	3		8

Sheet1

x

y

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet2

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet3

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

httpnumericalmethodsengusfedu 12

Linear Regression-Criterion2

4||4

1=sum

=iiE

x y ypredicted E = y - ypredicted

20 40 40 00

30 60 80 -20

20 60 40 20

30 80 80 00

Table Residuals at each point for regression model y=4x minus 4

Figure Regression curve y= y=4x minus 4 and y vs xdata

0

2

4

6

8

10

0 1 2 3 4

y

x

Using y=4x minus 4 as the regression curve

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Chart2

x

y

4

16

6

24

6

32

8

4

48

56

64

72

8

88

96

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

2		14
3		16
2		18
3		2
	22
	24
	26
	28
	3
	32
	34

Sheet1

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6

Sheet1

x

y

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Sheet2

y =4x - 4

y = 6

y

x

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Sheet3

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

httpnumericalmethodsengusfedu 13

Linear Regression-Criterion2

x y ypredicted E = y - ypredicted

20 40 60 -20

30 60 60 00

20 60 60 00

30 80 60 20

4||4

1=sum

=iiE

0

2

4

6

8

10

0 1 2 3 4

y

x

Table Residuals at each point for regression model y=6

Figure Regression curve y=6 and y vs x data

Using y=6 as a regression curve

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Chart1

x

y

4

6

6

6

6

6

8

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

2		02
3		04
2		06
3		08
	1
	12
	14
	16
	18
	2
	22
	24
	26
	28
	3
	32
	34
	36
	38

Sheet1

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

	x		y
	2		4
	3		6
	2		6
	3		8
		y=4x-4		y = 6
	0			6
	02			6
	04			6
	06			6
	08			6
	1			6
	12			6
	14		16		6
	16		24		6
	18		32		6
	2		4		6
	22		48		6
	24		56		6
	26		64		6
	28		72		6
	3		8		6
	32		88		6
	34		96		6
	36			6
	38			6

Sheet1

x

y

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

Sheet2

y = 6

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

Sheet3

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

httpnumericalmethodsengusfedu 14

Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6

The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique

Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion

44

1=sum

=iiE

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

httpnumericalmethodsengusfedu 15

Least Squares CriterionThe least squares criterion minimizes the sum of the square of the

residuals in the model and also produces a unique line

( )2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayES

x

11 yx

22 yx

33 yx

nnyx

iiyx

y

Figure Linear regression of y vs x data showing residuals at a typical point xi

xaay 10 +=

iii xaayE 10 minusminus=

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

httpnumericalmethodsengusfedu16

Finding Constants of Linear Model( )

2

110

1

2 sum minusminus=sum===

n

iii

n

iir xaayESMinimize the sum of the square of the residuals

To find

( )( ) 0121

100

=minusminusminusminus=partpart sum

=

n

iii

r xaayaS

( )( ) 021

101

=minusminusminusminus=partpart sum

=

n

iiii

r xxaayaS

giving

i

n

iii

n

ii

n

ixyxaxa sumsumsum

===

=+1

2

11

10

0a and 1a we minimize with respect to 1a 0aandrS

sumsumsum===

=+n

iii

n

i

n

iyxaa

111

10

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

httpnumericalmethodsengusfedu17

Finding Constants of Linear Model0aSolving for

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

and

2

11

2

1111

2

0

minus

minus=

sumsum

sumsumsumsum

==

====

n

ii

n

ii

n

iii

n

ii

n

ii

n

ii

xxn

yxxyxa

1aand directly yields

xaya 10 minus=

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

httpnumericalmethodsengusfedu18

Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below

Angle θ Torque T

Radians N-m

0698132 0188224

0959931 0209138

1134464 0230052

1570796 0250965

1919862 0313707

Table Torque vs Angle for a torsional spring

Find the constants for the model given byθ21 kkT +=

Figure Data points for Torque vs Angle data

01

02

03

04

05 1 15 2

θ (radians)

Torq

ue (N

-m)

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

Chart2

θ (radians)

Torque (N-m)

0188224

0209138

0230052

0250965

0313707

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

0698132
0959931
1134464
1570796
1919862

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965
	1919862		0313707

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

Sheet2

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

Sheet3

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu19

Example 1 cont

1a

The following table shows the summations needed for the calculations of the constants in the regression model

θ 2θ θTRadians N-m Radians2 N-m-Radians

0698132 0188224 0487388 0131405

0959931 0209138 0921468 0200758

1134464 0230052 12870 0260986

1570796 0250965 24674 0394215

1919862 0313707 36859 0602274

62831 11921 88491 15896

Table Tabulation of data for calculation of important

sum=

=5

1i

5=nUsing equations described for

25

1

5

1

2

5

1

5

1

5

12

minus

minus=

sumsum

sumsumsum

==

===

ii

ii

ii

ii

iii

n

TTnk

θθ

θθ

( ) ( )( )( ) ( )228316849185

1921128316589615

minus

minus=

21060919 minustimes= N-mrad

summations

0aTand with

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu20

Example 1 cont

n

TT i

isum==

5

1_

Use the average torque and average angle to calculate 1k

_

2

_

1 θkTk minus=

ni

isum==

5

1_

θθ

519211

=

11038422 minustimes=

528316

=

25661=

Using

)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu21

Example 1 Results

Figure Linear regression of Torque versus Angle data

Using linear regression a trend line is found from the data

Can you find the energy in the spring if it is twisted from 0 to 180 degrees

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu22

Linear Regression (special case)

Given

best fit

to the data

)( )()( 2211 nn yxyx yx

xay 1=

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu23

Linear Regression (special case cont)

Is this correct

2

11

2

1111

minus

minus=

sumsum

sumsumsum

==

===

n

ii

n

ii

n

ii

n

ii

n

iii

xxn

yxyxna

xay 1=

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu24

x

11 yx

ii xax 1

nnyx

iiyx

iii xay 1minus=ε

y

Linear Regression (special case cont)

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu25

Linear Regression (special case cont)

iii xay 1minus=ε

sum=

=n

iirS

1

2ε

( )2

11sum

=

minus=n

iii xay

Residual at each data point

Sum of square of residuals

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu26

Linear Regression (special case cont)

Differentiate with respect to

gives

( )( )sum=

minusminus=n

iiii

r xxaydadS

11

1

2

( )sum=

+minus=n

iiii xaxy

1

2122

01

=dadSr

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

1a

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu27

Linear Regression (special case cont)

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

( )sum=

+minus=n

iiii

r xaxydadS

1

21

1

22

021

22

1

2

gt= sum=

n

ii

r xda

Sd

sum

sum

=

== n

ii

n

iii

x

yxa

1

2

11

Does this value of a1 correspond to a local minima or local maxima

Yes it corresponds to a local minima

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu28

Linear Regression (special case cont)

Is this local minima of an absolute minimum of rS rS

1a

rS

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

httpnumericalmethodsengusfedu29

Example 2

Strain Stress

() (MPa)

0 0

0183 306

036 612

05324 917

0702 1223

0867 1529

10244 1835

11774 2140

1329 2446

1479 2752

15 2767

156 2896

To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data

E using the regression modelεσ E= and the sum of the square of the

00E+00

10E+09

20E+09

30E+09

0 0005 001 0015 002

Strain ε (mm)

Stre

ss σ

(Pa)

residuals

Figure Data points for Stress vs Strain data

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

Chart3

Strain ε (mm)

Stress σ (Pa)

0

306000000

612000000

917000000

1223000000

1529000000

1835000000

2140000000

2446000000

2752000000

2767000000

2896000000

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

0
000183
00036
0005324
000702
000867
0010244
0011774
001329
001479
0015
00156

Sheet1

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

	0698132		0188224
	0959931		0209138				θ
	1134464		0230052
	1570796		0250965				0		0			0		0
	1919862		0313707				0183		306			000183		306000000
					036		612			00036		612000000
					05324		917			0005324		917000000
					0702		1223			000702		1223000000
		σε				0867		1529			000867		1529000000
		ε				10244		1835			0010244		1835000000
					11774		2140			0011774		2140000000
					1329		2446			001329		2446000000
	Є					1479		2752			001479		2752000000
					15		2767			0015		2767000000
					156		2896			00156		2896000000

Sheet1

θ (radians)

Torque (N-m)

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

Sheet2

Strain ε (mm)

Stress σ (Pa)

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

Sheet3

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

httpnumericalmethodsengusfedu30

Example 2 cont

i ε σ ε 2 εσ

1 00000 00000 00000 00000

2 18300times10minus3 30600times108 33489times10minus6 55998times105

3 36000times10minus3 61200times108 12960times10minus5 22032times106

4 53240times10minus3 91700times108 28345times10minus5 48821times106

5 70200times10minus3 12230times109 49280times10minus5 85855times106

6 86700times10minus3 15290times109 75169times10minus5 13256times107

7 10244times10minus2 18350times109 10494times10minus4 18798times107

8 11774times10minus2 21400times109 13863times10minus4 25196times107

9 13290times10minus2 24460times109 17662times10minus4 32507times107

10 14790times10minus2 27520times109 21874times10minus4 40702times107

11 15000times10minus2 27670times109 22500times10minus4 41505times107

12 15600times10minus2 28960times109 24336times10minus4 45178times107

12764times10minus3 23337times108

Table Summation data for regression model

sum=

12

1i

sum=

minustimes=12

1

32 1027641i

iε

sum=

times=12

1

81033372i

iiεσ

sum

sum

=

== 12

1

2

12

1

ii

iii

Eε

εσ

3

8

10276411033372

minustimestimes

=

GPa84182=

sum

sum

=

== n

ii

n

iii

E

1

2

1

ε

εσ

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

httpnumericalmethodsengusfedu31

Example 2 Resultsεσ 91084182 times=The equation

Figure Linear regression for stress vs strain data

describes the data

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures primers textbook chapters multiple-choice tests worksheets in MATLAB MATHEMATICA MathCad and MAPLE blogs related physical problems please visit

httpnumericalmethodsengusfedutopicslinear_regressionhtml

THE END

httpnumericalmethodsengusfedu