1

Demand Forecasting

• Four Fundamental Approaches • Time Series

– General Concepts

– Evaluating Forecasts – How ‗good‘ is it? – Forecasting Methods (Stationary)

• Cumulative Mean • Naive Forecast • Moving Average • Exponential Smoothing

– Forecasting Methods (Trends & Seasonality) • OLS Regression • Holt‘s Method • Exponential Method for Seasonal Data • Winter‘s Model Other Models

Demand Forecasting

• Forecasting is difficult – especially for the future • Forecasts are always wrong • The less aggregated, the lower the accuracy • The longer the time horizon, the lower the accuracy • The past is usually a pretty good place to start • Everything exhibits seasonality of some sort • A good forecast is not just a number – it should include a

range, description of distribution, etc. • Any analytical method should be supplemented by

external information • A forecast for one function in a company might not be

useful to another function (Sales to Mkt to Mfg to Trans)

2

Four Fundamental Approaches

Subjective

– Judgmental

• Sales force surveys

• Delphi techniques

• Jury of experts

– Experimental

• Customer surveys

• Focus group sessions

• Test Marketing

Objective

– Causal / Relational

• Econometric Models

• Leading Indicators

• Input-Output Models

– Time Series

• ―Black Box‖ Approach

• Uses past to predict

the future

Time Series Concepts

1. Time Series – Regular & recurring basis to forecast

2. Stationarity – Values hover around a mean

3. Trend- Persistent movement in one direction

4. Seasonality – Movement periodic to calendar

5. Cycle – Periodic movement not tied to calendar

6. Pattern + Noise – Predictable and random components of a

Time Series forecast

7. Generating Process –Equation that creates TS

8. Accuracy and Bias – Closeness to actual vs Persistent

tendency to over or under predict

9. Fit versus Forecast – Tradeoff between accuracy to past

forecast to usefulness of predictability

10. Forecast Optimality – Error is equal to the random noise

3

Evaluating Forecasts

• Visual Review

• Errors

• Errors Measure

• MPE and MAPE

• Tracking Signal

Demand Forecasting

• Generate the large number of short-term, SKU

level, locally dis-aggregated demand forecasts

required for production, logistics, and sales to

operate successfully. „

• Focus on: ŠŠŠŠ Š – Forecasting product demand

– Mature products (not new product releases)

– Short time horizon (weeks, months, quarters, year)

– Use of models to assist in the forecast

– Cases where demand of items is independent

4

Historical Data

0

50

100

150

200

250

300

350

400

0 10 20 30 40 50

Forecasting Terminology

Initialization ExPost

Forecast Forecast

Historical Data

―We are now looking at a future from here, and the future we were looking at in February now includes some of our past, and we can incorporate the past into our forecast. 1993, the first half, which is now the past and was the future when we issued our first forecast, is now over‖

Laura D‘Andrea Tyson, Head of the President‘s Council of Economic Advisors, quoted in November of 1993 in the Chicago Tribune, explaining why the Administration reduced its projections of economic growth to 2 percent from the 3.1percent it predicted in

February.

Forecasting Terminology

5

Forecasting Problem

• Suppose your fraternity/sorority house

consumed the following number of cases of

beer for the last 6 weekends: 8, 5, 7, 3, 6, 9

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7

Week

Cases

• How many cases

do you think your

fraternity / sorority

will consume this

weekend?

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7 8

Week

Ca

se

s

Forecasting:

Simple Moving Average Method

• Using a three period moving average, we would get the following forecast:

63

963F

6

0 1

2 3

4 5 6

7 8

9 10

0 1 2 3 4 5 6 7 8

Week

Ca

se

s

Forecasting:

Simple Moving Average Method

• What if we used a two period moving average?

5.72

96F

• The number of periods used in the moving average forecast affects the ―responsiveness‖ of the forecasting method:

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7 8

Week

Cas

es

Forecasting:

Simple Moving Average Method

2 Periods

3 Periods

1 Period

7

t A(t) F(t)

1 8

2 5

3 7

4 3 6.67

5 6 5

6 9 5.33

7 6

8 6

9 6

10 6

Forecasting Terminology

• Applying this terminology to our problem

using the Moving Average forecast:

Initialization

ExPost

Forecast

Forecast

Model

Evaluation

• Rather than equal weights, it might make sense to use weights which favor more recent consumption values.

• With the Weighted Moving Average, we have to select weights that are individually greater than zero and less than 1, and as a group sum to 1:

• Valid Weights: (.5, .3, .2) , (.6,.3,.1), (1/2, 1/3, 1/6)

• Invalid Weights: (.5, .2, .1), (.6, -.1, .5), (.5,.4,.3,.2)

Forecasting:

Weighted Moving Average Method

8

Forecasting:

Weighted Moving Average Method

• A Weighted Moving Average forecast with

weights of (1/6, 1/3, 1/2), is performed as follows:

• How do you make the Weighted Moving

Average forecast more responsive?

79)2

1(6)

3

1(3)

6

1(F

• Exponential Smoothing is designed to give the

benefits of the Weighted Moving Average forecast

without the cumbersome problem of specifying

weights. In Exponential Smoothing, there is only

one parameter ():

= smoothing constant (between 0 and 1)

Forecasting:

Exponential Smoothing

)t(F)1()t(A)1t(F

9

Initialization:

•

• )2(F)1()2(A)3(F

)1(A)2(F

Forecasting:

Exponential Smoothing

)2(F)1()2(A)3(F

2/)]2(A)1(A[)2(F

t A(t) F(t)

1 8

2 5 6.5

3 7 5.9

4 3 6.34

5 6 5

6 9 5.4

7 6.84

8 6.84

9 6.84

10 6.84

Forecasting:

Exponential Smoothing

• Using = 0.4,

Initialization

ExPost

Forecast

Forecast

10

Forecasting:

Exponential Smoothing

Period Weight 0.1 0.3 0.5 0.7 0.9

1 (1 – )6 0.05314 0.03529 0.00781 0.00051 0.00000

2 (1 – )5 0.05905 0.05042 0.01563 0.00170 0.00001

3 (1 – )4 0.06561 0.07203 0.03125 0.00567 0.00009

4 (1 – )3 0.0729 0.1029 0.0625 0.0189 0.0009

5 (1 – )2 0.081 0.147 0.125 0.063 0.009

6 (1 – ) 0.09 0.21 0.25 0.21 0.09

7 0.1 0.3 0.5 0.7 0.9

Forecasting:

Exponential Smoothing

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7

Period

Weig

ht

= 0.1

= 0.3

= 0.5

= 0.7

= 0.9

11

Outliers (eloping point)

Exp Exp

t A(t) = 0.3 = 0.7

1 8

2 5 6.50 6.50

3 6 6.05 5.45

4 3 6.04 5.84

5 4 5.12 3.85

6 15 4.79 3.96

7 7.85 11.69

8 7.85 11.69

9 7.85 11.69

10 7.85 11.69

0

2

4

6

8

10

12

14

16

1 2 3 4 5 6 7 8 9 10

Outlier

Data with Trends

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7

12

0

1

2

3

4

5

6

7

8

9

10

1 2 3 4 5 6 7

A(t)

Data with Trends

= 0.3

= 0.5

= 0.7

= 0.9

Forecasting:

Simple Linear Regression Model

Simple linear regression

can be used to forecast

data with trends

D is the regressed forecast value or dependent variable in the model, a is the intercept value of the regression line, and b is the slope of the regression line.

a

bIaD

D

0 1 2 3 4 5 I

b

13

0

2

4

6

8

10

12

0 1 2 3 4 5 6 7

Forecasting:

Simple Linear Regression Model

In linear regression, the

squared errors are minimized

Error

Forecasting:

Simple Linear Regression Model

n

1i

n

1i

2

i

2

i

n

1i

n

1i

n

1i iiii

)I()I(n

)D)(I()DI(nb

bIaD

n

1i

n

1i ii )I(n

bD

n

1a

14

0

50

100

150

200

250

0 2 4 6 8 10 12 14 16

Limitations in

Linear Regression Model

As with the simple moving average model, all data points

count equally with simple linear regression.

Forecasting:

Holt‘s Trend Model

• To forecast data with trends, we can use an exponential smoothing model with trend, frequently known as Holt‘s model:

L(t) = A(t) + (1- ) F(t)

T(t) = [L(t) - L(t-1) ] + (1- ) T(t-1)

F(t+1) = L(t) + T(t)

• We could use linear regression to initialize

the model

15

Holt‘s Trend Model:

Initialization

t A(t)

1 32

2 38

3 50

4 61

5 52

6 63

7 72

8 53

9 99

10 92

11 121

12 153

13 183

14 179

15 224

First, we‘ll initialize the model: x y x2 xy

1 32 1 32

2 38 4 76

3 50 9 150

4 61 16 244

Sum 30 502

Average 2.5 45.25

5.20)(9.9)(2.5-45.3=xb-y=a

9.95

5.49

)5.2(430

.5)4(45.25)(2-502=

)xn(-x

)x)(yn(-xy=b

222

L(4) = 20.5+4(9.9)=60.1

T(4) = 9.9

Holt‘s Trend Model:

Updating

t A(t) L(t) T(t) F(t)

1 32

2 38

3 50

4 61 60.1 9.9

5 7052

L(t) = A(t) + (1- ) F(t)

0.3

0.4

L(5) = 0.3 (52) + 0.7 (70)=64.6

T(t) = [L(t) - L(t-1) ] + (1- ) T(t-1) T(5) = 0.4 [64.6 – 60.1] + 0.6 (9.9) = 7.74

F(t+1) = L(t) + T(t) F(6) = 64.6 + 7.74 = 72.34

64.6 7.74 72.34 6

16

t A(t) L(t) T(t) F(t)

1 32

2 38

3 50

4 61 60.1 9.9

5 52 64.60 7.74 70

6 72.34

Holt‘s Trend Model:

Updating

63

0.3

0.4

L(6) = 0.3 (63) + 0.7 (72.34)=69.54

T(6) = 0.4 [69.54 – 64.60] + 0.6 (7.74) = 6.62

F(7) = 69.54 + 6.62 = 76.16

69.54 6.62

76.16 7 72

Holt‘s Model Results

t A(t) L(t) T(t) F(t)

1 32

2 38

3 50

4 61 60.1 9.9

5 52 64.60 7.74 70

6 63 69.54 6.62 72.34

7 72 74.91 6.12 76.16

8 53 72.62 2.76 81.03

9 99 82.46 5.59 75.38

10 92 89.24 6.06 88.06

11 121 103.01 9.15 95.30

12 153 124.41 14.05 112.16

13 183 151.82 19.39 138.46

14 179 173.55 20.33 171.22

15 224 202.92 23.94 193.88

16 226.86

17 250.80

18 274.74

19 298.68

Initialization

ExPost

Forecast

Forecast

17

Regression

0

50

100

150

200

250

300

350

0 5 10 15 20

Initialization ExPost Forecast Forecast

Holt‘s Model Results

Forecasting:

Seasonal Model (No Trend)

0

5

10

15

20

25

30

35

40

45

50

Spr

ing

2003

Sum

mer

200

3

Fall 2

003

Win

ter 2

003

Spr

ing

2004

Sum

mer

200

4

Fall 2

004

Win

ter 2

004

Spr

ing

2005

Sum

mer

200

5

Fall 2

005

Win

ter 2

005

A(t)

2003 Spring 16

Summer 27

Fall 39

Winter 22

2004 Spring 16

Summer 26

Fall 43

Winter 23

2005 Spring 14

Summer 29

Fall 41

Winter 22

18

L(t) = A(t) / S(t-p) + (1- ) L(t-1)

S(t) = g [A(t) / L(t)] + (1- g) S(t-p)

Seasonal Model Formulas

p is the number of periods in a season

Quarterly data: p = 4

Monthly data: p = 12

F(t+1) = L(t) * S(t+1-p)

Seasonal Model Initialization

S(5) = 0.60

S(6) = 1.00

S(7) = 1.55

S(8) = 0.85

L(8) = 26.5

Quarter

Average

16.0

26.5

41.0

22.5

Seasonal

Factor

S(t)

0.60

1.00

1.55

0.85

Average Sales

per Quarter = 26.5

A(t)

2003 Spring 16

Summer 27

Fall 39

Winter 22

2004 Spring 16

Summer 26

Fall 43

Winter 23

19

Seasonal Model Forecasting

g 0.3

0.4

26.71 1.03 25.18

26.62 1.55 41.32

25.18 0.59 16.00 2005 Spring 14

Summer 29

Fall 41

Winter 22 26.34 0.84 22.60

2006 Spring

Summer

Fall

Winter

15.53

27.02

40.69

22.25

A(t) L(t)

Seasonal

Factor

S(t) F(t)

2004 Spring 16 0.60

Summer 26 1.00

Fall 43 1.55

Winter 23 26.50 0.85

0

5

10

15

20

25

30

35

40

45

50

0 2 4 6 8 10 12 14 16

Seasonal Model Forecasting

20

Forecasting: Winter‘s Model for Data

with Trend and Seasonal Components

L(t) = A(t) / S(t-p) + (1- )[L(t-1)+T(t-1)]

T(t) = [L(t) - L(t-1)] + (1- ) T(t-1)

S(t) = g [A(t) / L(t)] + (1- g) S(t-p)

F(t+1) = [L(t) + T(t)] S(t+1-p)

Seasonal-Trend Model

Decomposition

• To initialize Winter‘s Model, we will use

Decomposition Forecasting, which itself

can be used to make forecasts.

21

41

Decomposition Forecasting

• There are two ways to decompose forecast

data with trend and seasonal components:

– Use regression to get the trend, use the trend

line to get seasonal factors

– Use averaging to get seasonal factors, ―de-

seasonalize‖ the data, then use regression to

get the trend.

Decomposition Forecasting

• The following data contains trend and seasonal components:

0

50

100

150

200

250

0 2 4 6 8 10

Period Quarter Sales

1 Spring 90

2 Summer 157

3 Fall 123

4 Winter 93

5 Spring 128

6 Summer 211

7 Fall 163

8 Winter 122

22

Decomposition Forecasting

• The seasonal factors are obtained by the same method used for the Seasonal Model forecast:

Period Quarter Sales

1 Spring 90

2 Summer 157

3 Fall 123

4 Winter 93

5 Spring 128

6 Summer 211

7 Fall 163

8 Winter 122

Average = 135.9

Average to 1

Qtr. Ave.

109

184

143

107.5

Seas.

Factor

0.80

1.35

1.05

0.79

1.00

Decomposition Forecasting

• With the seasonal factors, the data can be de-seasonalized by dividing the data by the seasonal factors:

Deseasonalized Data

100.0

110.0

120.0

130.0

140.0

150.0

160.0

170.0

0 2 4 6 8 10

Sales

Seas.

Factor

Deseas.

Data

90 0.80 112.2

157 1.35 115.9

123 1.05 116.9

93 0.79 117.5

128 0.80 159.6

211 1.35 155.8

163 1.05 154.9

122 0.79 154.2

Regression on the De-seasonalized data will give the trend

23

Decomposition Forecasting

Regression Results

Period

(X)

Deseas.

Sales (Y) X2 XY

1 112.2 1 112.2

2 115.9 4 231.8

3 116.9 9 350.7

4 117.5 16 470

5 159.6 25 798

6 155.8 36 934.8

7 154.9 49 1084.3

8 154.2 64 1233.6

SUM 204 5215.4

Average 4.5 135.9

2.101)=5(7.71)(4.-135.9=xm-y=b

7.71=42

324

)5.4(8204

.5)8(135.9)(4-5215.4=

)xn(-x

)x)(yn(-xy=m

222

Decomposition Forecast

• Regression on the de-seasonalized data

produces the following results:

– Slope (m) = 7.71

– Intercept (b) = 101.2

• Forecasts can be performed using the

following equation

– [mx + b](seasonal factor)

24

0

50

100

150

200

250

300

1 2 3 4 5 6 7 8 9 10 11 12

Decomposition Forecasting

Period Quarter Sales Forecast

1 Spring 90 87.4

2 Summer 157 157.9

3 Fall 123 130.8

4 Winter 93 104.5

5 Spring 128 112.1

6 Summer 211 199.7

7 Fall 163 163.3

8 Winter 122 128.8

9 Spring 136.8

10 Summer 241.4

11 Fall 195.7

12 Winter 153.2

Winter‘s Model Initialization

• We can use the decomposition forecast to define the following Winter‘s Model parameters:

L(n) = b + m (n)

T(n) = m

S(j) = S(j-p)

L(8) = 101.2 + 8 (7.71) = 162.88

T(8) = 7.71

S(5) = 0.80

S(6) = 1.35

S(7) = 1.05

S(8) = 0.79

So from our previous model, we have

25

Winter‘s Model Example

176.41 10.04 0.81 136.47

197.85 14.60 1.39 251.71

215.00 15.62 1.06 223.07

9 Spring 152

10 Summer 303

11 Fall 232

12 Winter 171 226.37 13.92 0.78 182.19

13 Spring

14 Summer

15 Fall

16 Winter

195.19

352.41

283.09

220.87

= 0.3 = 0.4 g = 0.2

Period Quarter Sales L(t) T(t) S(t) F(t)

1 Spring 90

2 Summer 157

3 Fall 123

4 Winter 93

5 Spring 128 0.8

6 Summer 211 1.35

7 Fall 162 1.05

8 Winter 122 162.88 7.71 0.79

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Winter‘s Model Example

26

Evaluating Forecasts

―Trust, but Verify‖ Ronald W. Reagan

• Computer software gives us the ability to mess up more data on a greater scale more efficiently

• While software like SAP can automatically select models and model parameters for a set of data, and usually does so correctly, when the data is important, a human should review the model results

• One of the best tools is the human eye

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Visual Review

• How would you evaluate this forecast?

27

0

50

100

150

200

250

300

350

400

0 10 20 30 40 50

Forecast Evaluation

Initialization ExPost

Forecast Forecast

Where Forecast is Evaluated

Do not include

initialization data

in evaluation

0

50

100

150

200

250

300

350

400

20 25 30 35 40

Errors

All error measures compare the forecast model to

the actual data for the ExPost Forecast region

28

Errors Measure

All error measures are based on the comparison of forecast values to actual values in the ExPost Forecast region—do not include data from initialization.

t F(t) A(t)

Error

F(t) - A(t) | F(t) - A(t) |

25 95 91 4 4

26 125 137 -12 12

27 197 193 4 4

28 227 199 28 28

29 230 278 -48 48

30 274 344 -70 70

31 274 291 -17 17

32 255 250 5 5

33 244 171 73 73

34 211 152 59 59

35 114 111 3 3

36 85 127 -42 42

Sum -13 365

Bias and MAD

t F(t) A(t)

Error

F(t) - A(t) | F(t) - A(t) |

25 95 91 4 4

26 125 137 -12 12

35 114 111 3 3

36 85 127 -42 42

Sum -13 365

08.112

13

n

)t(A)t(FBias

42.3012

365

n

)t(A)t(FMAD

29

• Bias tells us whether we have a tendency to over- or under-forecast. If our forecasts are ―in the middle‖ of the data, then the errors should be equally positive and negative, and should sum to 0.

• MAD (Mean Absolute Deviation) is the average error, ignoring whether the error is positive or negative.

• Errors are bad, and the closer to zero an error is, the better the forecast is likely to be.

• Error measures tell how well the method worked in the ExPost forecast region. How well the forecast will work in the future is uncertain.

Bias and MAD

Absolute vs. Relative Measures

• Forecasts were made for two sets of data. Which forecast was better?

Data Set 1

Bias = 18.72

MAD = 43.99

Data Set 2

Bias = 182

MAD = 912.5

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4 5 6 7 8 9 10 11 12

Data Set 1 Data Set 2

0

5000

10000

15000

20000

25000

30000

1 2 3 4 5 6 7 8 9 10 11 12

30

MPE and MAPE

• When the numbers in a data set are larger in

magnitude, then the error measures are likely to

be large as well, even though the fit might not be

as ―good‖.

• Mean Percentage Error (MPE) and Mean

Absolute Percentage Error (MAPE) are relative

forms of the Bias and MAD, respectively.

• MPE and MAPE can be used to compare

forecasts for different sets of data.

MPE and MAPE

• Mean Percentage Error (MPE)

n

)t(A

)t(A)t(F

MPE

n

)t(A

)t(A)t(F

MAPE

• Mean Absolute Percentage Error (MAPE)

31

MPE and MAPE

Data Set 1

148.012

774.1MAPE

037.012

446.0MPE

t A(t) F(t)

F(t) - A(t)

A(t)

| F(t) - A(t) |

A(t)

9 177 125.4 -0.292 0.292

10 275 338.85 0.232 0.232

11 363 493.2 0.359 0.359

12 91 89.1 -0.021 0.021

13 194 176 -0.093 0.093

14 376 463.05 0.232 0.232

15 659 658.8 0.000 0.000

16 146 116.7 -0.201 0.201

17 219 226.6 0.035 0.035

18 514 587.25 0.143 0.143

19 875 824.4 -0.058 0.058

20 130 144.3 0.110 0.110

0.446 1.774

MPE and MAPE

Data Set 2

066.012

792.0MAPE

010.012

121.0MPE

t A(t) F(t)

F(t) - A(t)

A(t)

| F(t) - A(t) |

A(t)

9 6332 5973 -0.057 0.057

10 12994 15147 0.166 0.166

11 21325 20844 -0.023 0.023

12 3527 3582 0.016 0.016

13 7283 6765 -0.071 0.071

14 14963 17091 0.142 0.142

15 24325 23436 -0.037 0.037

16 4054 4014 -0.010 0.010

17 8173 7557 -0.075 0.075

18 16804 19035 0.133 0.133

19 27458 26028 -0.052 0.052

20 4496 4446 -0.011 0.011

0.121 0.792

32

MPE and MAPE

Data Set 2

066.012

792.0MAPE

010.012

121.0MPE

148.012

774.1MAPE

037.012

446.0MPE

Data Set 1

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4 5 6 7 8 9 10 11 12

0

5000

10000

15000

20000

25000

30000

1 2 3 4 5 6 7 8 9 10 11 12

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Tracking Signal

• What‘s happened in this situation? How could we detect this in an automatic forecasting environment?

33

Tracking Signal

• The tracking signal can be calculated after each actual

sales value is recorded. The tracking signal is

calculated as:

n

)t(A)t(F

)t(A)t(F

)t(MAD

RSFE)t(TS

• The tracking signal is a relative measure, like MPE

and MAPE, so it can be compared to a set value

(typically 4 or 5) to identify when forecasting

parameters and/or models need to be changed.

Tracking Signal

t A(t) F(t) F(t) - A(t) RSFE | F(t) - A(t) | S | F(t) - A(t) | MAD TS

1 15.1

2 16.8 15.9

3 11.4 14.6 3.2 3.2 3.2 3.2 3.20 1.00

4 18.7 15.8 -2.9 0.3 2.9 6.1 3.05 0.10

5 11.8 14.6 2.8 3.1 2.8 8.9 2.97 1.04

6 17.2 15.4 -1.8 1.3 1.8 10.7 2.68 0.49

7 12.9 14.6 1.7 3.0 1.7 12.4 2.48 1.21

8 22.9 17.1 -5.8 -2.8 5.8 18.2 3.03 -0.92

9 24.0 19.2 -4.8 -7.6 4.8 23 3.29 -2.31

10 32.6 23.2 -9.4 -17.0 9.4 32.4 4.05 -4.20

11 38.5 27.8 -10.7 -27.7 10.7 43.1 4.79 -5.78

12 36.6 30.4 -6.2 -33.9 6.2 49.3 4.93 -6.88

13 40.6 33.5 -7.1 -41.0 7.1 56.4 5.13 -8.00

14 51.0 38.7 -12.3 -53.3 12.3 68.7 5.73 -9.31

15 51.9 42.7 -9.2 -62.5 9.2 77.9 5.99 -10.43

18.7 18.7

34

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Tracking Signal

TS = -5.78