Real-Time Electricity Pricing - Northwestern...

Real-Time Electricity Pricing

Xi Chen, Jonathan Hosking and Soumyadip Ghosh

IBM Watson Research Center / Northwestern UniversityYorktown Heights, NY, USA

X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 1 / 37

Agenda

1 IntroductionProject BackgroundMotivationData Summary

2 Modeling and EstimationGeneral FrameworkA Single HouseholdRTP “Average” Household14 RTP Households

3 Conclusions and Future Work

4 Acknowledgment

5 Questions


Olympic Peninsula ProjectProject Background

Why

To investigate distributed real-time control of the grid and to test impacts of time-varyingrates

Who and What

Residential, commercial and municipal test sites, and several distributed generator werecoordinated to manage the constrained feeder electrical distribution

Where

In and near Sequim and Port Angeles, Washington, on the Olympic Peninsula

When

From 4/1/2006 through 3/31/2007


Transactive Control

Figure: Control of Imposed Distribution Constraint Using Transactive Control, courtesy of PNNL.


Pricing Contract Programs

The Residential Customers

112 households are divided into four contract groups:1 Fixed price: Price remains constant.a

2 TOU/CPP: Critical peak rate > On-peak rate > off-peak rate; during each period, priceremains the same.

3 RTP: Price varies every fifteen minutes; customers set automatic response to pricechanges; they can override their pre-set response settings at any time

4 Control: No program account and no bills. Customers received $150 in appreciation oftheir participation.

aThe unit of price is $/kWh

In our study, we focus on analyzing the relationship between real-time price andelectricity usage of the RTP group.


Data Summary: A Quick Overview

Problems with data on the residential customers

1 Randomly missing data entries (especially for RTP group)

2 Unequal lengths of the RTP households’ data files3 Failed to keep a record on changes of households’ mode settings4 No demographics (PII) of the households available5 Potential preselection bias in the experimental design6 Possible data recording errors7 Incomplete weather data a

aAccording to the weather expert Dr.Lloyd Treinish at IBM, the oceanic climate occurs in most parts of PacificNorthwest including the areas of our interest, which leads to dramatic weather changes throughout a typical year.




1 Randomly missing data entries (especially for RTP group)2 Unequal lengths of the RTP households’ data files

3 Failed to keep a record on changes of households’ mode settings4 No demographics (PII) of the households available5 Potential preselection bias in the experimental design6 Possible data recording errors7 Incomplete weather data a





1 Randomly missing data entries (especially for RTP group)2 Unequal lengths of the RTP households’ data files3 Failed to keep a record on changes of households’ mode settings

4 No demographics (PII) of the households available5 Potential preselection bias in the experimental design6 Possible data recording errors7 Incomplete weather data a





1 Randomly missing data entries (especially for RTP group)2 Unequal lengths of the RTP households’ data files3 Failed to keep a record on changes of households’ mode settings4 No demographics (PII) of the households available

5 Potential preselection bias in the experimental design6 Possible data recording errors7 Incomplete weather data a





1 Randomly missing data entries (especially for RTP group)2 Unequal lengths of the RTP households’ data files3 Failed to keep a record on changes of households’ mode settings4 No demographics (PII) of the households available5 Potential preselection bias in the experimental design

6 Possible data recording errors7 Incomplete weather data a





1 Randomly missing data entries (especially for RTP group)2 Unequal lengths of the RTP households’ data files3 Failed to keep a record on changes of households’ mode settings4 No demographics (PII) of the households available5 Potential preselection bias in the experimental design6 Possible data recording errors

7 Incomplete weather data a





1 Randomly missing data entries (especially for RTP group)2 Unequal lengths of the RTP households’ data files3 Failed to keep a record on changes of households’ mode settings4 No demographics (PII) of the households available5 Potential preselection bias in the experimental design6 Possible data recording errors7 Incomplete weather data a



Data Summary: Customer ID 671

month/year missing usage=0 price<0 total5/2006 4 0 19 236/2006 19 50 60 1297/2006 249 0 4 2538/2006 0 4 0 49/2006 324 0 0 324

10/2006 1 2 0 311/2006 0 0 0 012/2006 5 6 0 111/2007 3 0 0 32/2007 0 0 0 03/2007 7 0 33 40

Percentage 1.90% 0.19% 0.36% 2.46%


Data Summary: 14 Households in the RTP Group

month/year missing usage=0 price<0 total5/2006 56 32 266 3546/2006 266 78 840 11847/2006 3486 68 56 36108/2006 0 177 0 1779/2006 4536 17 0 4553

10/2006 14 27 0 4111/2006 0 46 0 4612/2006 70 161 0 2311/2007 42 57 0 992/2007 0 26 0 263/2007 98 23 462 583

Percentage 1.90% 0.16% 0.36% 2.42%


Time-Series Regression Analysis—Basic Picture

Figure: Time series of electricity demands and price in 15-min time interval


CustID 671: Late Spring, Early Summer

Figure: Late Spring: 5/1/2006—6/30/2006 with No spike in price


CustID 671: Summer

Figure: Summer: 8/1/2006—8/31/2006 with No spike in price


CustID 671: Fall

Figure: Fall: 10/1/2006—10/31/2006


CustID 671: Winter

Figure: Winter: 11/1/2006—2/28/2007


Time-Series Regression Models: Why?

Rationale Behind Time-Series Regression Models

1 We are interested in capturing the effect of real-time price on electricity demand, nomatter how slight it might be⇒ regression models

2 Some factors that impact electricity demand are not available to us⇒ omitted variablebias

3 It is desirable to capture the dynamics in electricity demand given the volatilityobserved in electricity usage of any household⇒ including lagged terms of dependentvariable into predictor matrix


Time-Series Regression Models: How?Some Methods and Concepts in Econometrics

Before fitting . . .

Time series involved in this regression: stationary or not?

Spurious regression

A linear combination of I(1) variables is stationary, then the variables are calledcointegrated⇒ a long-run relationship between the I(1) variables. a We want to learnabout the true causal relationship, so need to do a formal unit root test in each series

a“Unit root”: Yt = θYt−1 + εt , where θ = 1⇒ Nonstationarity or random walk, or I(1) variable.

After fitting . . .

Stationarity and unit root problem in residuals← Augmented Dickey–Fuller test;

Autocorrelations in residuals← Box-Pierce and Ljung-Box tests


CustID 671: Time–Series Regression ModelsWe consider time-series regression models for four seasons:

log[q(t)] = β0 + βi0 · 1{t is the i-th time interval of a weekday}

+f (log[q(t − k)]) + h (log[p(t − `)])

+βTmp · log[T(t − 3)] + βRH · log[H(t − 3)] + ε(t) (1)

where k = 1, 2, 3, 4, 94, 95, 96, 97, 98 and ` = 0, 1, 3, 4, 73, 84 (terms vary for four seasons);f(·) and h(·) are linear combinations of back-shifted log[q(t)] and log[p(t)] respectively.

How did we choose the lagged terms in Equation (1)?

Vector AutoRegression (VAR)y1ty2ty3ty4t

=

A11(B) A12(B) A13(B) A14(B)A21(B) A22(B) A23(B) A24(B)A31(B) A32(B) A33(B) A34(B)A41(B) A42(B) A43(B) A44(B)

y1ty2ty3ty4t

+

C1C2C3C4

+

ε1ε2ε3ε4

(2)

where y1t = log[q(t)], y2t = log[p(t)], y3t = log[T(t)] and y4t = log[H(t)]. Aij(B), i, j = 1, 2, 3 and 4take the form

∑pk=1 aijk Bk , where B is the lag (backshift) operator defined by Bk yt = yt−k and p = 96.

Through VAR, we found that...

Compared to log[q(t)], log[p(t)] can be better predicted given the historical electricity demands and itsown lagged terms.


CustID 671: Time–Series Regression ModelsWe consider time-series regression models for four seasons:


+f (log[q(t − k)]) + h (log[p(t − `)])

+βTmp · log[T(t − 3)] + βRH · log[H(t − 3)] + ε(t) (1)

where k = 1, 2, 3, 4, 94, 95, 96, 97, 98 and ` = 0, 1, 3, 4, 73, 84 (terms vary for four seasons);f(·) and h(·) are linear combinations of back-shifted log[q(t)] and log[p(t)] respectively.How did we choose the lagged terms in Equation (1)?

Vector AutoRegression (VAR)y1ty2ty3ty4t

=

A11(B) A12(B) A13(B) A14(B)A21(B) A22(B) A23(B) A24(B)A31(B) A32(B) A33(B) A34(B)A41(B) A42(B) A43(B) A44(B)

y1ty2ty3ty4t

+

C1C2C3C4

+

ε1ε2ε3ε4

(2)

where y1t = log[q(t)], y2t = log[p(t)], y3t = log[T(t)] and y4t = log[H(t)]. Aij(B), i, j = 1, 2, 3 and 4take the form

∑pk=1 aijk Bk , where B is the lag (backshift) operator defined by Bk yt = yt−k and p = 96.

Through VAR, we found that...

Compared to log[q(t)], log[p(t)] can be better predicted given the historical electricity demands and itsown lagged terms.


CustID 671: Regression and Tests ResultsWe use the Newey-West adjusted heteroscedastic-serial consistent least-squaresregression method to fit models specified by Equation (1).

statistics spring summer fall winterR̄

20.526 0.489 0.568 0.712

Tests

For time-series regression, we need to do tests the stationarity and independence of the regressionresiduals!

p-valuesTest spring summer fall winter

Box-Pierce (p=1) 0.5330 0.8797 0.9929 0.8317Box-Pierce (p=96) 0.1656 0.1656 0.3883 0.0064

Dickey-Fuller <0.01 (p = 16) <0.01 (p = 14) <0.01 (p = 13) <0.01 (p = 22)

Table: Cust ID 671— TS Regression results and tests on residuals

More on the tests

Box-Pierce Test: H0: There is no autocorrelation exists;Dickey-Fuller Test: H0 : Unit root does exist in the underlying process.


CustID 671: Price Elasticities and Effects of Weather FactorsA glance at own price elasticity of energy demand ( d q(t)

q(t) /d p(t)

p(t) ) at time t :

Season β̂ for log[p(t)] β̂Tmp β̂RH

Spring(0.0147)

−0.0524(0.2016)

0.0875(0.0055)

0.0931

Summer(0.1348)

−0.0595(0.008)

0.4769(0.003)

0.2136

Fall(0.2432)

−0.0161(0.3724)

0.0689(0.9777)

−0.0012

Winter(0.0170)

−0.0133(<0.0001)

−0.1494(0.2588)

−0.0409

Table: Price elasticity estimates and coefficient estimates for temperature and RH for four seasons

Summary

1 Better fit for winter season than non-winter season, especially when there are spikes in price.2 Weather effects:

I Temperature : an important factor in determining the electricity demand, especially forsummer and winter;

I Relative humidity can be an important factor for summer, and it also has a slight impact forspring.

3 Price plays a role in determining the electricity demand in winter and spring; its impacts seemminor when compared to the weather factors in magnitude.



q(t) /d p(t)

p(t) ) at time t :


Spring(0.0147)

−0.0524(0.2016)

0.0875(0.0055)

0.0931

Summer(0.1348)

−0.0595(0.008)

0.4769(0.003)

0.2136

Fall(0.2432)

−0.0161(0.3724)

0.0689(0.9777)

−0.0012

Winter(0.0170)

−0.0133(<0.0001)

−0.1494(0.2588)

−0.0409


Summary

1 Better fit for winter season than non-winter season, especially when there are spikes in price.

2 Weather effects:I Temperature : an important factor in determining the electricity demand, especially for

summer and winter;I Relative humidity can be an important factor for summer, and it also has a slight impact for

spring.3 Price plays a role in determining the electricity demand in winter and spring; its impacts seem

minor when compared to the weather factors in magnitude.



q(t) /d p(t)

p(t) ) at time t :


Spring(0.0147)

−0.0524(0.2016)

0.0875(0.0055)

0.0931

Summer(0.1348)

−0.0595(0.008)

0.4769(0.003)

0.2136

Fall(0.2432)

−0.0161(0.3724)

0.0689(0.9777)

−0.0012

Winter(0.0170)

−0.0133(<0.0001)

−0.1494(0.2588)

−0.0409


Summary

1 Better fit for winter season than non-winter season, especially when there are spikes in price.2 Weather effects:

I Temperature : an important factor in determining the electricity demand, especially forsummer and winter;

I Relative humidity can be an important factor for summer, and it also has a slight impact forspring.

3 Price plays a role in determining the electricity demand in winter and spring; its impacts seemminor when compared to the weather factors in magnitude.


CustID 671: Factors that Impact Electricity Demand with Price Threshold

Use the price threshold p(t − 2) > $0.2 /kWh, we obtain the following for the winter season:

Variable β̂ for log[p(t)] β̂Tmp β̂RH

log[p(t)] when p(t − 2) > $0.2/kWh(0.0119)

−0.0166(0.0112)

−0.0609(0.6978)

−0.0075

log[p(t)] when p(t − 2) ≤ $0.2/kWh(0.0001)

−0.0154(<0.0001)

−0.0668(0.5885)

−0.0039

Table: Price elasticity estimates and coefficient estimates for temperature and RH for winter seasonwith price threshold

We observe that ...

1 When price is above $0.2/kWh, CustID 671 is more price sensitive and cares lessabout cold weather.

2 When price is below $0.2/kWh, CustID 671 is less price sensitive and more lowtemperature concerned.

3 Price does have a significant impact on the electricity demand in winter; however,temperature seems to play a dominating role.


CustID 671: Time-of-Day Model (Winter)Fit separate time-series regression models for the 96 time-of-day points, for winter andnon-winter seasons.

Figure: R̄2 for 96 fitted time-of-day regression models (Winter)


CustID 671: Time-of-Day Model (Non-Winter)

Figure: R̄2 for 96 fitted time-of-day regression models (Non-winter)


CustID 671: Time-of-Day Model— Own Price Elasticities

Time-of-Day LB β̂ UB01:15 -0.1346 -0.0686 -0.002506:45 -0.1651 -0.0875 -0.009909:45 -0.3587 -0.2119 -0.065110:00 -0.5219 -0.3199 -0.117810:15 -0.4177 -0.2482 -0.078716:45 -0.1948 -0.1101 -0.025518:15 0.0264 0.1237 0.221123:45 0.0102 0.0673 0.1244

Table: Cust ID 671— 95% C.I.’s for significant TODprice elasticities (winter) that do not include 0

Time-of-Day LB β̂ UB01:45 -0.0632 -0.0332 -0.003206:45 -0.1227 -0.0649 -0.007208:30 0.0705 0.1672 0.263811:15 0.0047 0.0809 0.157014:30 -0.1937 -0.1106 -0.027517:15 -0.1573 -0.0824 -0.007520:00 -0.1504 -0.0902 -0.0300

Table: Cust ID 671— 95% C.I.’s for significant TODprice elasticities (non-winter) that do not include 0

Observation

1 CustID 671 seems more flexible in shifting its electricity demand to non-peak hours,compared to peak-hours;

2 CustID 671 has relatively inelastic electricity demand in midnight hours than in othernon-peak hours.


CustID 671: TOD Model—Temperature Effect

Figure: Cust ID 671— 95% C.I.’s forsignificant temperature coefficient(Non-winter)that do not include 0

Figure: Cust ID 671— 95% C.I.’s for significanttemperature coefficient (Winter) that do not include0


CustID 671: Time-of-Day Models

Summary on TOD models

1 Poor fitting performance for some time-of-day points, especially when price does notvary a lot whereas electricity demand does⇒ singularity of predictor matrix.

2 More difficult to find better regression models for non-winter than for winter3 Design incentive policy for utilities to explore the differences between price elasticities

of on-peak and off-peak hours4 Make inferences about individual household’s electricity demand fingerprint (such as

occupancy of the house and family’s work schedules) and further exploit them





2 More difficult to find better regression models for non-winter than for winter

3 Design incentive policy for utilities to explore the differences between price elasticitiesof on-peak and off-peak hours

4 Make inferences about individual household’s electricity demand fingerprint (such asoccupancy of the house and family’s work schedules) and further exploit them






of on-peak and off-peak hours

4 Make inferences about individual household’s electricity demand fingerprint (such asoccupancy of the house and family’s work schedules) and further exploit them






of on-peak and off-peak hours4 Make inferences about individual household’s electricity demand fingerprint (such as

occupancy of the house and family’s work schedules) and further exploit them


RTP “Average” Household: Regression and Tests ResultsFit time-series regression models for four seasons: 1


+f (log[q(t − k)]) + h (log[p(t − `)])

+βTmp log[T(t − 3)] + βRH log[H(t − 3)] + ε(t) (3)

where basically k = 1, 2, 3, 4, 85, 86, 90, 94, 96 and ` = 0, 1, 2, 3, 4 (terms vary for fourseasons)

Better regression fitting performance!

statistics spring summer fall winterR̄2 0.748 0.689 0.803 0.878

Stronger autocorrelation exists in the regression residuals!


Box-Pierce (p=1) 0.587 0.326 0.5962 0.7686Box-Pierce (p=96) 0.027 0.116 0.327 1.921 × 10−5

Dickey-Fuller <0.01(p=16) <0.01(p=14) <0.01 (p=14) <0.01 (p=22)

Table: RTP “Average”— TS Regression results and tests on residuals

1Newey-West adjusted heteroscedastic-serial consistent Least-squares Regression


RTP “Average” Household: Regression and Tests ResultsFit time-series regression models for four seasons: 1


+f (log[q(t − k)]) + h (log[p(t − `)])

+βTmp log[T(t − 3)] + βRH log[H(t − 3)] + ε(t) (3)

where basically k = 1, 2, 3, 4, 85, 86, 90, 94, 96 and ` = 0, 1, 2, 3, 4 (terms vary for fourseasons)Better regression fitting performance!

statistics spring summer fall winterR̄2 0.748 0.689 0.803 0.878

Stronger autocorrelation exists in the regression residuals!


Box-Pierce (p=1) 0.587 0.326 0.5962 0.7686Box-Pierce (p=96) 0.027 0.116 0.327 1.921 × 10−5

Dickey-Fuller <0.01(p=16) <0.01(p=14) <0.01 (p=14) <0.01 (p=22)

Table: RTP “Average”— TS Regression results and tests on residuals

1Newey-West adjusted heteroscedastic-serial consistent Least-squares Regression


RTP “Average” Household: Price Elasticities and Effects of Weather Factors

A glance at own price elasticity of energy usage ( d q(t)q(t) /

d p(t)p(t) ) at time t :


Spring(0.1055)

−0.0339(0.0539)

−0.0942(0.5163)

−0.0100

Summer(0.1980)

−0.0808(0.0039)

0.2181(0.0662)

0.0584

Fall(0.4058)

−0.0060(<0.0001)

−0.2446(0.2015)

0.0215

Winter(0.0101)

−0.0074(<0.0001)

−0.1396(0.3139)

−0.0138


With price threshold p(t − 2) > $0.2 /kWh, we obtain the following estimates for the winter season:

Variable β̂ for log[p(t)] β̂Tmp β̂RH

log[p(t)] when p(t − 2) > $0.2/kWh(0.0240)

−0.0160(0.0058)

−0.0684(0.6466)

−0.0179

log[p(t)] when p(t − 2) ≤ $0.2/kWh(0.0022)

−0.0094(<0.0001)

−0.0692(0.3054)

−0.0142

Table: Price elasticity estimates and coefficient estimates for temperature and RH for winter with pricethreshold


RTP “Average” Household: Time-of-Day Model (Winter)Fit separate time-series regression models for the 96 time-of-day points, for winter andnon-winter seasons.

Figure: R̄2 for the 96 time-of-day models (Winter)


RTP “Average” Household: TOD Model (Non-Winter)

Figure: R̄2 for the 96 time-of-day models (Non-winter)


RTP “Average” Household: TOD Model — Elasticities

Time-of-Day LB β̂ UB00:15 -0.4320 -0.2184 -0.004900:30 -0.3598 -0.2032 -0.046701:00 -0.3408 -0.1875 -0.034301:15 -0.3601 -0.1810 -0.001901:30 -0.1596 -0.0855 -0.011303:30 -0.2923 -0.2015 -0.110707:45 0.0048 0.0285 0.052208:15 -0.0512 -0.0283 -0.005411:45 0.0257 0.0752 0.124812:15 -0.1393 -0.0763 -0.013214:30 -0.1301 -0.0818 -0.033516:00 0.0118 0.0696 0.127317:45 -0.0716 -0.0375 -0.003523:15 -0.2474 -0.1281 -0.0087

Table: RTP AVG— 95% C.I.’s for significant priceelasticities (winter)

Time-of-Day LB β̂ UB02:15 0.0008 0.0156 0.030502:30 0.0012 0.0158 0.030305:00 -0.0282 -0.0149 -0.001606:15 -0.0727 -0.0453 -0.017906:30 -0.0341 -0.0189 -0.003607:00 -0.0401 -0.0208 -0.001609:00 -0.0432 -0.0218 -0.000409:30 0.0170 0.0360 0.055013:00 0.0134 0.0387 0.064114:45 -0.0614 -0.0397 -0.018119:00 0.0119 0.0372 0.062620:15 -0.0399 -0.0246 -0.009420:45 0.0115 0.0309 0.0502

Table: RTP AVG— 95% C.I.’s for significant priceelasticities (non-winter)


RTP “Average” Household: TOD Model—Temperature Effect

Figure: RTP AVG — 95% C.I.’s for significanttemperature coefficient (Winter)

Figure: RTP AVG — 95% C.I.’s for significanttemperature coefficient(Non-winter)


Models for 14 RTP Households

Modeling Perspective

Choice to make: Separate vs. Pooled Cross Sectional ModelsHow differently do the 14 households behave from one another?


14 RTP Households: Separate ModelsFit a common model for each single household use “lmList” R.

Figure: 95% C.I.’s of coefficients for the regressions of log[q(t)] on lagged log[q(t)], log[p(t)] andweather variables.


14 RTP Households: Separate Models

Figure: Fitted vs. True for regressions of log[q(t)] on lagged log[q(t)], log[p(t)] and weather variables.


14 RTP Households: Observations and Conclusions

1 Substantial variation in the coefficients’ 95% C.I.’s, especially true for log[q(t − 1)] andlog[q(t − 96)] (both of them are statistically significant).

2 Test of poolability. We use “pooltest” from the package “plm” in R. 2 ⇒We reject thehypothesis that a pooled model estimation is appropriate. Therefore, we do notconsider using pooled fixed effects model on the 14 households.

3 Mixed (or Random) Effects Model: only data of 14 RTP households available; No otherexogenous variables available so far except for price and weather factors; The randomeffects of our interest can be highly correlated with other regressors⇒ prohibit us fromusing random effects model.

2It compares a model obtained for the full sample (with different intercepts for different households) and a modelbased on the estimation of a separate equation for each household


Summary and Future Work

Conclusions

We used time-series regression models to obtain the price elasticities of electricitydemand for the households in the RTP group;

The regression fitting performance is better for predicting demand in winter; For thepurpose of reasonably accurate forecasting, it’s better to use ARIMA model forelectricity demand and let the data speak for itself.

The demand is relatively more price elastic in winter compared to non-winter, this isespecially true when price spikes are present;

Temperature plays an important role in determining electricity demand; according toour results, people are more sensitive to changes in temperature than changes inprice;

The responsiveness that individual RTP household showed to changes in price differfrom one another; their daily electricity usage patterns also differ from each other to agreat extent.

Future work

Fit different regression TOD models for on-peak and peak-hours

Analyze the interactions between temperature and price signals

Study the TOU/CPP group


Acknowledgment

We are grateful to Dr. Pu Huang, Dr. Ramesh Natarajan @ IBM and Prof. JeremyStaum @ Northwestern University for their helpful comments and discussion


Questions and Comments?


Real-Time Electricity Pricing - Northwestern...

Documents

Transcript of Real-Time Electricity Pricing - Northwestern...