Richard O’Neillrichard.oneill@ferc.govChief Economic Advisor

Federal Energy Regulation Commission

NAS Resource Allocation: Economics and EquityThe Aspen InstituteQueenstown, MDMarch 20, 2002

This does not necessarily reflect the view of the Commission.

Richard P. O’NeillFederal Energy Regulatory Commission

Benjamin F. HobbsThe Johns Hopkins University and Energieonderzoek Centrum

Nederland

Paul M. SotkiewiczUniversity of Florida

William StewartCollege of William and Mary

Michael RothkopfRutgers University

Udi HelmanFederal Energy Regulatory Commission and The Johns Hopkins

University

menugin $4

scotch $6

hot dog $3

burger $5 valuegin $5

scotch $9

hot dog $3

burger $5

$5

You must order from the menu

Adam Smith on network regulation

“The tolls for the maintenance of a high road, cannot with any safety be made the property of private persons. ... It is proper, therefore, that the tolls for the maintenance of such work should be put under the management of commissioners or trustees.” [Wealth of Nations , Book V, Chap 1, p. 684]

If Adam Smith is not enough, why intervene?

To provide for reliabilityTo ensure revenue adequacy (no subsidies)To facilitate entry

generation, transmission, consumptionTo mitigate market powerTo provide competitive price signalsTo protect property rights

How to intervene?

nondiscriminatory lower transactions costsmore not less optionscontrol market powerhighly efficient “just” prices

will prices be too low?Are low prices bad?are market prices

unconstitutional?All power corrupts, but we need the electricity.

Electric markets are Electric markets are incomplete and incomplete and

complexcomplex incomplete if not pricing all desired products asymmetric markets: vertical demand curve bidding nonconvexities

Supply: start-up and no-load demand: for y continuous hours at < $x intertemporal dependencies

reactive (imaginary/orthogonal) power socialize transient stability/ voltage socialize generation and load characteristics can decentralized markets handle this?

Basics of market designBasics of market design

Contracts (not compacts) Marginal/incremental cost bidding

Start-up and min run

Trading rules Financially sound Market clearing prices

Incentives For doing “good” things For not doing “bad” things(socal gas, no withholding) With collars for political reasons

Information

Auction Design Principles "Everything should be made as simple as

possible ... but not simpler." EinsteinDon'ts

Create gaming opportunities in the name of simplicity

Foreclose marginal (incremental) cost bidsAssume away non-convexitiesincrease riskAllow bids that are not firm offersfavor large players

Do'sAllow marginal cost biddingmarket clearing price (with scarcity rents) make the process internally consistentCreate property rights and simple alternativesAllow self scheduling

=22/7

= 4[1 -1/3 + 1/5 – 1/7 + …]

differences in auction vs. differences in auction vs. COS based regulationCOS based regulation

Issue Auction COSestimating short run marginal costs yes yesestimating capacity yes yeshold-up problem yes yesestimating return on equity no yesestimating depreciation no yesestimating units of service no yescost allocation no yesestimating proper discounts no yesmeasuring withholding yes yesfree-rider problem yes no

market design objectives market design objectives max bid efficiency within constraintsmax bid efficiency within constraints

All RTO (centralized) markets are optional self-scheduling and voluntary market bids low transactions costs; no new risks allow marginal costs bidding(multipart bids) minimize incentives for market power abuse don’t favor large participants/portfolio max arbitrage/min averaging simultaneous market clearing lots of information

Coexistence of RTO and off-RTO markets eliminate bias between off-RTO and RTO

markets RTO markets should not be severely

constrained to promote off-RTO markets Allow clearing of mutually beneficial trades? Why are there mutually beneficial trades? Should off-RTO markets be subsidized? No Should the RTO be in the insurance biz? No bilateral physical markets: who pays for the

cleanup after the party?

Self supply Self supply optionsoptions

Self designed zonal configurationsbalanced schedulingself supply of ancillary servicesself designed transmission rights optional bidding in RTO/ISO marketsno bill from RTO/ISOpay or get paid for imbalances

Pre-day-ahead markets for transmission rights: CRT/TCC/TRCs/FGRs for generation capacity/resevres (ICAP) market power mitigation via options contracts

day-ahead market for reliability(valium substitute) simultaneous nodal market-clearing auctions for energy,

ancillary services and congestion allow multi-part bidding higher of market or bid cost recovery allow self scheduling allow price limit bids on ancillary and congestion

Real-time balancing myopic market markets are nodal-based LMP with fish protection

Pre-day-ahead marketsPre-day-ahead markets Annual, seasonal, monthly, weeklySimultaneous clearing of all productsDemand side bidding or capacity options (physical?)Capacity markets (two year ahead for entry) Transmission contracts

energy forward contracts: FCCs, FTCsenergy options contracts: CRTs flow gate options: FGRsFTR options for reservesUnbalanced contracts!

market power mitigation contractsbid marginal costs including start up and no loadpay higher of market clearing price or costs

Day-ahead reliability Day-ahead reliability managementmanagement

Optimize system topology over large areas reserves, real and reactive power balanced clear most congestion reserves in place including tx capacity set imports clear mutually beneficial trades interperiod interdependencies: start-up, ramp

rate and minimum load

Design principles for day-ahead RTO market

Objective: max efficiency within reliability

allow marginal cost bids(start-up/min load) self-scheduling with optional bidding simultaneous clearing of all services at LMP pay higher of bid costs or market clearing price financially binding/physically if needed low risk and transactions costs

Is a real-time market enough? In theory yescan too much real-time scheduling threaten

system stability?Neighborhood reliability of the AC load flow

What if it was a DC load flow? Simpleshould there be an incentive not to be more

than x% out of balance?first, eliminate all bias to be in the RTM

Can the real-time market Can the real-time market handle reliability by itself?handle reliability by itself?

Design principles for real-time RTO market

No other markets need be in real-time balance max efficiency within system balancedeviations from day-ahead priced at marketthose operating as scheduled in day-ahead pay

nothingphysically bindingpay market clearing pricelow risk and transactions costsfast market info: prices and quantities

non-simultaneous auctions without non-simultaneous auctions without LMP and marginal cost biddingLMP and marginal cost bidding

Socialize not privatize costhigher cost passthroughs: uplift create incentives for market power to lower the

risks of the market designhigher transactions cost of bid preparationmarket power mitigation is more difficultconstant redesign to correct flawsproblems in England, Columbia, CaliforniaAustralia moving from zonal to nodal

Zonal markets (Cal, PJM, NE, UK) Sequential markets for energy and anc services One settlement systems Infeasible markets (Cal PX and UK) Ignore nonconvexities (start-up and no-load) Ignore market power As-bid pricing all ended in administrative all ended in administrative

interventionintervention No property rights to market power

or poor market design

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30quantity

$/u

nit

demand

variable costs

competitive price

monopoly rents

monopoly price

scarcity rents

< withheld capacity >

lostsurplus

monopoly and scarcity rents

good market design allows good market design allows proper mitigationproper mitigation

Pre-day-ahead markets Tx rightsPre commitment of generatorsFeasibility of capacity markets

Day-ahead marketDemand biddingMarginal costs bid includes startup, noload and running

costsReal-time balancing market

Bid running costs if you did not bid in DAMReliability by adjusting generators via bids

No fault market power mitigation

BA

C

100 MW2/3 1/3

100 MW

100 MW

1/3

150 MW

150 MW

Max bt + B(y) βt + K(y) <= f (μ)

Sequence of auctions; forward and real-time 40,000+ nodes400, 000+ contingency constraintsCould be highly redundant constraintsK(y) can be non-convex (electromagnetic eqns)Bids are non-convex mixed integer

Solvable? Bixby says not to worry: 6 orders of magnitude in 10 years

SESW

NW

Nomogram Flowgate

NE

A Four Node Network with Nomogram Flowgate

Bidder B1 B2 B3 B4 B5 B6 S1

Bid Type FB option, NE to NW. Buy up to 100 MW

FB option, NW to SW. Buy up to 100 MW

PtP forward, NW to SW. Buy up to 100 MW

PtP option, NW to SW. Buy up to 100 MW

PtP forward, NE to NW. Buy up to 100 MW.

PAR capacity. Buy up to 100 MW.

Forward generation at SW. Sell up to 200 MW.

Bid ($/MW)

10 30 20 25 25 25 -10

Quantity awarded

100 100 0 100 40 20 200

Reduced cost

10 5 -5 0 0 25 20

PTDFs: Shadow Price:

NW to SW 0.0 1.0 0.8 0.8 0.6 0.0 0.0 25

NE to NW 1.0 0.0 -0.2 0.0 0.6 0.0 0.0 0

NE to SE 0.0 0.0 0.2 0.2 0.4 0.0 0.0 0

SE to SW 0.0 0.0 0.2 0.2 0.4 0.0 0.0 25

PAR 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5

SF nomogram

0.0 0.0 0.0 0.0 0.0 0.0 -1.0 30

( )C g S z S z S zi it ifixed

it isu

itsu

isd

itsd

ti

g dit ti

g z Git it i m in 0

z z zit i t itsu , 1 0

z z zit i t itsd

, 1 0

Maximize:

subject to: t,

i, t,

i, t,

i, t,

i, t,

zitsu, zit

sd {0,1}; all other variables 0; zit < 1

g z Git it i m ax 0

Application: Electric Power Generator Application: Electric Power Generator Unit CommitmentUnit Commitment

Plant 1 2 3

MIN Q 50 100 20MAX Q 150 200 50MC/unit 4 2 6Fixed$/hr 125 50 0StartUp$ 25 150 0ShutDown$ 0 0 0

0 Load 400

Problem 1: Single Period, 3 PlantsProblem 1: Single Period, 3 Plants

0

1

2

3

4

5

6

7

0 100 200 300 400

Quantity

Pri

ce $

/Un

it

Commodity PriceCommodity Price

0

1

2

3

4

5

6

7

8

0 100 200 300 400

Quantity

$/U

nit

Average Cost Price

Average Cost vs. PriceAverage Cost vs. Price

0

50

100

150

200

0 100 200 300 400

Quantity

Do

llars

Start P1Start P2Start P3

Startup PaymentsStartup Payments

1. Multi-Period Considerations: e.g., ramp rate limits

- Problem 2 (2 hours): Assume RR limit = 105 MW/hr, demands = 180 MW and 395 MW

- Both plants start up in period 1 because of ramp rate

- Plant 1 gets paid $150 to start up in period 1 (commodity price alone supports operation only in period 2); profit = $175

- Degeneracy/multiple duals a problem

2. Ancillary Services

3. Transmission Congestion Payments

4. Demand Bidding

Unit Commitment ExtensionsUnit Commitment Extensions

Smokestack versus High Tech (from Scarf, 1994)

Production Characteristics

Smokestack (Type 1 Unit)

High Tech (Type 2 Unit)

Capacity 16 7

Construction Cost 53 30

Marginal Cost 3 2

Average Cost at Capacity

6.3125 6.2857

Total Cost at Capacity

101 44

Formulate and Solve MIPFormulate and Solve MIP(Simulates Bid Evaluation by (Simulates Bid Evaluation by

Auctioneer)Auctioneer)Let: z1, z2 = construction decisions of types 1 & 2, respectively

q1, q2 = output for types 1 & 2

MIP:Max -i (53z1i + 3q1i) - Σi (30z2i + 2q2i)

i (q1i + q2i) = Q

-16z1i + q1i 0; z1i {0, 1}, q1i 0, i =1,2,…

-7z2i + q2i 0; z2i {0, 1}, q2i 0, i =1,2,…

Efficient OutcomeEfficient Outcome

• To optimally satisfy a demand of, say, 61 units:

#Type 1 # Type 2 Type 1 Type 2 Total

Demand Units Units Output Output Cost

61 3 2 47 14 388

Note that one Type 1 does not operate at capacity

• The output price according to the LP solution is $3– But both types make negative profits

not an equilibrium outcome

Average cost as a function of demand

for Scarf's problem

6.28

6.29

6.3

6.31

6.32

6.33

6.34

6.35

6.36

6.37

6.38

6.39

6.4

6.41

6.42

6.43

55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70demand

ave

rage c

ost

optimal value as a function of demandfor Scarf's problem

340

350

360

370

380

390

400

410

420

430

440

450

55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70demand

op

tim

al valu

e

optimal value as a function of demand

for Scarf's problem

340

350

360

370

380

390

400

410

420

430

440

450

55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70demand

optim

al

valu

e

LP That Solves the MIPLP That Solves the MIP

Max -i (53z1i + 3q1i) -Σi (30z2i + 2q2i) Duals

i (q1i + q2i) = Q (y0**)

-16z1i + q1i 0 , i =1,2,.. (y1i)

z1i = z1i* (w1i**)

q1i 0

-7z2i + q2i 0 , i =1,2,.. (y2i)

z2i = z2i* (w2i **)

q2i 0,

** Used in payment scheme

Prices at an Output of 61Prices at an Output of 61

Unit type 1 Unit type 2

(Smokestack) (High Tech)

(y) (w1i) (y1i) (w2i) (y2i)

Demand Commodity Price Start-up Capacity Start-up Capacity

61 3** -53** 0 -23** 1

**Prices paid by unit to auctioneer

Thus, each unit is paid a start-up cost, ensuring nonnegative profit (here, 0)

Dual Prices for Scarf's Problem

Unit 1(Smokestack)

Unit 2 (High Tech)

Dual Price Set

Commodity Price

Start-up Price

Capacity Price

Start-up Price

Capacity Price

Set I 3 53 0 23 -1

Set II 6.3125 0 -3.3125 -.1875 -4.3125

Set III 6.2857 .429 -3.2857 0 -4.2857

Non-Convexities in MarketsNon-Convexities in Markets• While market models often assume away non-convexities (e.g.,

integral decisions and economies of scale),

… they exist!• Electric utility industry:

– Still economies of scale in generation and especially transmission– Unit commitment: start-up, shut-down costs; minimum run levels

• Why disregard non-convexities in market models? … with convex profit maximization problems (concave

objective, convex feasible region), we can usually:– define linear (“one-part”) market clearing prices– establish existence, uniqueness properties for market equilibria– create tests for entering activities

The Problem with Non-ConvexitiesThe Problem with Non-Convexities

• Linear prices can no longer clear the market … an equilibrium cannot be guaranteed to exist

• E.g., electric power operations: – At P < P*, inadequate supply– At P > P*, a lump of additional supply enters that breaks even

(covers fixed costs), but supply exceeds demand. If force any generator to back off, its profit < 0

– “Administrative” solution to reach the optimum:• Adjust outputs to restore feasibility• Side payments to ensure no one loses money

• As Scarf (1990, 1994) then points out, there is no price test for Pareto improving entry of new production processes

Why Address Non-Convexities in Why Address Non-Convexities in Auctions Now?Auctions Now?

1) New markets for electric power have non-convexities2) A debate surrounding these power markets: the use of

prices to induce efficient, decentralized decisions a la Walrasian auctions

3) Auction mechanisms in the NYISO and PJM attempt to account for integer decisions

4) California said no to such an auction because of complaints: - these prices are not “equilibrium supporting” and

- administrative adjustments appear arbitrary (e.g., Johnson, Oren, Svoboda 1998)

5) Our result: If integral decisions can be priced, a market equilibrium can be supported

Some Related LiteratureSome Related Literature

• Scarf (1990, 1994)– Emphasized the divergence of math programming and economics– Searched (unsuccessfully) for a way to find prices in the presence of

integral choices, and for pricing tests for improvements• Gomory and Baumol (1960)

– The use of cutting plans that are combinations of existing constraints to arrive at an integer solution

– Interpreted duals of those planes; stopped short of pricing individual integer activities

• Wolsey (1981)– Pure IP with integer constraint coefficients & RHSs– Approaches to constructing price functions yielding dual problems

that satisfy weak and strong duality. Functions generally nonlinear• Williams (1996)

– Examines possible duality for integer programs, but concludes no “satisfactory” duals (Lagrange multipliers) exist

Pricing Integral ActivitiesPricing Integral Activities

• One can think of the traditional pricing approach as a misspecification of the commodity space– The commodity space could include integer decisions as an

“intermediate good”

• The pricing system derived here is similar to multi-part pricing for utilities– For example, an demand charge (fixed costs) and an energy

charge

• Buyers’ clubs with multipart contract

Our Approach Our Approach to Addressing Non-Convexitiesto Addressing Non-Convexities

1. Formulate the non-convex problem as a maximization MIP. Bids include all costs and internal constraints.

2. Solve MIP– Take advantage of modern MIP technology

3. Take integer solutions z* and define equality constraints z=z* in a LP -- “convexifying” the problem.

4. Solve LP5. Duals on z=z* are prices on the integer variables

– If market/auction participant pays those prices, together with the duals on commodity and other coupling constraints…

…. then those prices support an equilibrium– For (0,1) variables, can use only negative prices for z*=1 and positive

prices for z* = 0

General Formulation: MIPGeneral Formulation: MIPLet:

k = index for auction participants

xk, zk = activities

ck, dk = marginal benefits of activities (cost, if <0). (ckxk + dkzk is the total benefit to participant k)

Ak1, Ak2, Bk1, Bk2 = constraint coefficients

b0 = commodities to be auctioned. In double auction, b0 = 0

bk = RHS of internal constraints of participant k

MIP: max k (ckxk + dkzk)

subject to: k (Ak1xk + Ak2zk) b0

Bk1xk + Bk2zk bk

xk 0, zk {0,1}

} all k

An LP That Solves The MIPAn LP That Solves The MIP

LP(z*): Max k (ckxk + dkzk)

s.t. k (Ak1xk + Ak2zk) b0

Bk1xk + Bk2zk bk

zk = zk *

xk 0

where z* indicates an optimal value of z in MIP

} all k

Definition of EquilibriumDefinition of EquilibriumDefinition 1. A “market clearing” set of contracts has the

following characteristics:

1. Each bidder is in equilibrium in the following sense. Given• prices {y0*, wk*} and payment function Pk(xk,zk) defined by the

contract

• no restrictions on xk and zk other than k’s internal constraints (Bk1xk + Bk2zk bk)

then no bidder k can find feasible xk’, zk’ for which:

(ckxk’ + dkzk’ - Pk(xk’,zk’)) > (ckxk* + dkzk* - Pk(xk*,zk*))

Thus, the prices support the equilibrium {xk*,zk*}.

2. Supply meets demand for the commodities. I.e.,

k (Ak1xk* + Ak2zk*) < b0

A Candidate Set of Market A Candidate Set of Market Clearing Prices and QuantitiesClearing Prices and Quantities

Definition 2. Consider the contract Tk with the following terms:

1. Bidder k sells zk=zk*, xk0=xk0* (where xk0 is the subset of xk

with nonzero Ak1)

2. Bidder k pays auctioneer:

Pk(xk,zk) = y0* (Ak1xk+Ak2zk) + wk* zk.

where * indicates an optimal solution to MIP / LP(z*)

Variant. Define:

wk*’ = Max(0, wk*) if zk* = 0

= Min(0, wk*) if zk* = 1

Pk(xk,zk) = y0* (Ak1xk+Ak2zk) + wk*’ zk

Existence of Market Clearing Existence of Market Clearing Contracts for MIP AuctionContracts for MIP Auction

Theorem. T { Tk} is a market clearing set of contracts.

• Proof exploits complementary slackness conditions from auction LP to show that at the candidate equilibrium prices, {zk*,xk0*} is optimal for each bidder’s own MIP:

Max [ckxk + dkzk] - [y0* (Ak1xk+Ak2zk) + wk* zk]

s.t. Bk1xk + Bk2zk bk

xk 0, zk {0,1}

• There may be alternative optima for the bidder• Profits nonnegative

• If wk* < 0 and zk* =1, interpretable as NYISO/PJM mechanism for preventing winning bidders from losing money

• In this framework, there are payments directly for individual capacity

A Welfare ResultA Welfare Result

Corollary. If each participant k bids truthfully (submits a bid reflecting its true valuations (ckxk + dkzk) and true constraints

(Bk1xk + Bk2zk bk; xk 0; zk {0,1}),

... Then an auction defined as follows will:

(a) maximize net social benefits (k [ckxk + dkzk]) and

(b) clear the market

The auction includes the following steps:

1. The auctioneer solves MIP, yielding primal {xk0*,zk*};

2. The auctioneer solves LP(z*), obtaining prices {y0*, wk*}

3. The auctioneer imposes contract T upon the bidders

ExtensionsExtensions

• Tests for Pareto improving entry of new activities

• How useful is this really likely to be? – For small systems, where lumpiness looms large, market power is important

– For large systems, the duality gap shrinks .. into irrelevancy?

• Strategic bidding over these integral activities.– Comparisons to iterative single part bid auctions.

– Do more bidding degrees of freedom facilitate strategic behavior? What are the impacts?

• Consequences for distribution of rents in the market between consumers and producers, and among groups of producers

• Tests on larger systems

You don’t always get it right the first

time.Now you

have experience

Try LMP

Wholestic Market Design AGORAPHOBIA

Are you a Copernican or a Ptolemain?

We had to destroy the market to save it