1 1

The Physical Market

AMSI Workshop, April 2007

2 2

Overview

The NEM is the physical market for electricity in Australia.

This seminar aims to:

Describe the NEM with a view to its influence on the derivative markets

Expose some useful mathematical modelling techniques for certain aspects of the NEM

3 3

Structure of the NEM

~

~

~

Pool

Retail supplier

Retail supplier

Generators

P,Q,T

$400

$350

4 4

Market Rules

• Gross market• 5-minute intervals, 30 minute clearing• National market with 6 regions• Bid-based dispatch• System marginal price applies to all

participants• Demand always met with VoLL cap• Ancillary services also on-market

The need for the secondary derivative

marketRisk profile due to

deviation from a continuous marketInter-regional basis risk,

Accumulation of interregional funds

Analysis of ‘behaviour’ and gaming on physical

market

Models of volatile price processAnalysis 8

Optimisation algorithms for minimum cost dispatch under

constraints

5 5

Regional Structure

6 6

The Energy Market

7 7

Spot market for Energy

5 minute Price ($/MWh) + dispatch (MW)

Minimum load

Ramp rates

HV link constraints

Network losses30 minute Price ($/MWh) + cleared MW

4-second target data for frequency control

8 8

Losses

• Within a region:

• Payment to a generator

= MLF Pool Price MWh sent out

• System generation at the terminals

• Across regions, losses taken into account

• Equivalent volume “at the node”

= MLF MWh sent out

9 9

Models of the NEM

• Structural models– Prophet, iPool, Prosym

• Parsimonious, hybrid models– Inhouse, academic research

• Risk, econometric models– Lacima, Accurisk

10 10

Inputs to models

• Availability

• Demand

• Transmission

• Behaviour

• Regulatory

11 11

Availability

• Reliability models of power stations

• “Complex systems” of elements – With/without redundancy

• Failure described with Poisson process

• Improved with time-varying hazard intensity

12 12

13 13

Availability model implementationFailure time is exponential distribution: PDF

1. Model failure times directly: CDF

Let R ~ U(0,1) then

X = F-1(R) = -log(1-R)/2. Model failure times directly:

Pr(failure in [0, 0+dt])

xexf )(

x xedssfxF

01)()(

)(1 2dtOdte dt

14 14

‘Behaviour’

• Generator bidding and commitment• More market rules:

– 10 bid price and volume bands– ‘Rebids’ permitted by shifting volume

• Portfolio optimisation subject to:– Market rules– Contract cash flows– Longer-term objectives– Response of competitors

15 15

Demand

• A relatively exogenous variable

• Limited elasticity to price

• Driven by:– Industrial processes– Human-related influences– Random residuals

16 16

Market data: modelling / visualising

• A regular 30-minute market

• Data matrices in daily resolution

Matlab

17 17

Time Series Plots

18 18

Intensity Plots (imagesc)

19 19

Overlay plots (plot)

20 20

PCA

• The overlay visualisation provides motivation for – Principal Component Analysis– Singular Value Decomposition

• “Daily Demand shape”

= “Average Shape” + “Perturbation”

21 21

Modelling Demand

• Demand = f (i,d,s,T,H)– i = interval number (1-48)– d = day type (M,T,W,.., S,PH)– s = season– T = prevailing temperature– H = prevailing humidity

• Explanatory variables: – 48 8 4

22 22

Principal Components Analysis

• Let v1, …, vn Rm

• Establish the correlation matrix of the vectors, M Rmm

• Determine the eigenvectors of the correlation matrix, e1, e2, .., em

• These are the principal components

• Determine the eigenvalues of the correlation matrix 1, 2, …, m

23 23

Properties of the components

• Being a correlation matrix, j are real and positive

– {ei} are orthogonal

– That is principal components are independent– Eigenvalues of the correlation matrix sum to

the dimension m

• Explanation provided by component j (R2) is j / m

24 24

Intuitive explanation

• Principal components are vector subspaces explaining most variation in the data

25 25

26 26

Models using the PCs

• Model a process with say three components.• Fit each observation on the components:

Min { || vj - α1j e1 + α2

j v2 + α3j v3 || : [α3,α3 ,α3] }

• Then observation j is explained:vj = α1

j e1 + α2j v2 + α3

j v3 + rj

• Obtain sequence: A1, A2, A3, …, An

• Where Aj =

3

2

1

27 27

Models using the PCs

• Then v1, v2, v3, …, vn (each in Rn) is simplified to A1, A2, …, An (each in R3).

• Now regress on the explanatory variables for a linear model:

A = f (d,s,T,H)

• “Katestone” demand modelling software

28 28

Singular Value Decomposition

• Identical to the PCA, but from a different historical development

• Data in columns: D R48 365

• Find unitary matrices U and V and diagonal matrix S: D = U S VT

• U R48 48, S R48 365, V R365 365

• Elements of S are positive decreasing in magnitude

29 29

SVD approximations

00000000

00000000

00000000

00000000

• D = U S VT

u1, u2, .., um

S1, S2, … Sm

30 30

SVD approximations

000000000

000000000

000000000

000000001

4321

S

uuuu

0000000011uS

nVuSVuSVuS 11~

1121~

1111~

1 000000

An optimal weighting of a single shape u1.

Include two components: get an linear combination of u1 and u2.

31 31

2004 demand

32 32

33 33

34 34

35 35

Applications in demand models

• Step 1: Principal Components

• Step 2: Explanatory power:

• Step 3: Best fit to components:

• Step 4: Regressed on weather variables:

36 36

Alternative application of PCA

• Say a demand model exists already

• Perform PCA decomposition of residuals:

• Describe perturbations with components– Dj = D*j + 1e1 + 2e2 + 3e3 + r

– Where j ~ N(0,σj2)

• Then future demand is simulated by perturbations of the modelled demand

37 37

Weather-dependence of demand

38 38

Models for Pool Price

39 39

Pool Price and PCA

• Data structure is similar to demand

• Apply the same PCA methods

• Apply fits to price rather than log price– Note on Excel’s exponential fitting!

40 40

41 41

Principal Components of Price

• Step 1: Principal Components

• Step 2: Explanatory power:

• Step 3: Best fit to components:

42 42

Historical sampling

• Stochastic model

• Want a pool price process with behaviour representative of historical observations

• Perform historical sampling

• Extend by biasing subject to known variables

43 43

44 44

Drawing a path

• Draw a half-hourly succession of U(0,1) random variables

• Lookup the resulting pool price

• Problem: more autocorrelation in the real world

45 45

Inducing autocorrelation

• Require prices to be more autocorrelated than independent random draws

• Need successive U(0,1) variables to be correlated

• Need to generate very long sequences of autocorrelated random variables

46 46

Tricks:• To generate U(0,1) variable:

=RAND()

• To generate N(0,1) variable =NORMSINV(RAND())

• To generate two -correlated N(0,1) variables:– X ~N(0,1), T~N(0,1)– Y = X + (1- 2)T– Then (X,Y) =

47 47

For a long sequence• X1, X2, X3, … XN ~ N(0,1) iid• Y1 = X1

• Y2 = Y1 + (1- 2)X2

• Y3 = Y2 + (1- 2)X3

= [X1 + (1- 2)X2] + (1- 2)X3

• So Y = B*X• To speed the process:• Y=F-1F(B*X)=F-1(F(B)F(X))• Y1, Y2, Y3, …, YN ~ N(0,1) autocorrelated• P = NORMSDIST(X)

48 48

Demand-driven price model

• Pool price = f(demand, bids, transmission)

49 49

• We know process for weather,

• Know weather impact on demand,

• Know demand and price relationship

50 50

Ito’s lemma

• Mechanism for describing a new random process which is derived from another

• Primary process:

dS = a(t,S) dt + b(t,S) dW• Secondary process: V(t,S)

dtS

VbdS

S

Vdtt

VdV

2

22

2

1

51 51

Price from Demandy = 5.1894e0.0002x

0

5

10

15

20

25

30

35

40

3500 4000 4500 5000 5500 6000 6500 7000

0

10

20

30

40

50

60

70

1 14 27 40 53 66 79 92 105 118 131 144 157 170 183 196

Actual PriceDemandPrice(demand) model

Valid with a time-varying bid stack: PCA again?

52 52

Standard Price Model

• Derive a stochastic differential equation without reference to explanatory variables

• MRJD – to be introduced for option pricing

53 53

Transmission and Prices

54 54

Regional Structure

55 55

Intraregional Constraints

56 56

Negative prices?

57 57

Interregional Constraints

58 58

Price Separation

1000 MW flowing South (-10%)

QLD generators paid $18

NSW customers paying $1200

59 59

Dispatch Process

60 60

Black Hole Money

• NEMDE:– “What is the next cheapest dispatch to provide the

next MW of electricity?”

• When constrained:– QLD – isolated region with large generation supply– NSW – shortage of supply

• Inter Regional Settlement Residues• Electricity flows up the price gradient, so value

can only accumulate to NEMMCO.• Representative of a true discontinuous process

61 61

Models of Flows and Price Spreads

• Require stochastic model of:– Flow level– Price in region 1– Price in region 2

• Under the physical models of:– Flow limits– Losses