Click to edit Master subtitle styleComplex Systems Science and CSIRO: Into the Future

Rydges Hotel, Melbourne

10-12 August 2005

NEMSIM: Practical Challenges for Agent-based Simulation of

Energy Markets

George Grozev and David BattenCSIRO Manufacturing and Infrastructure Technology

www.csiro.au

Presentation Overview

History and concepts (NEM as a CAS)

NEMSIM overview and key features

Practical challenges for agent-based simulation

www.csiro.au

Brief Historical Review of the Project

July 2002: A postdoc position awarded. Dr. Xinmin Hu started in Nov. 2002.

January 2003: Commenced as a CSS project: “Top-up” funding from CSIRO’s Centre for Complex Systems Science

October 2003: Commenced as a Theme 1 project in CSIRO’s Energy Transformed Flagship Program

April 2004: Flagship Science e-Seminar Series 2: Energy Transformed (John Wright, David Batten)

April 2005: NEMSIM Industry Focus Group Meeting (Mercure Hotel, Melbourne)

www.csiro.au

NEMSIM: National Electricity Market Simulator

Agents in NEMSIM:• 27 Scheduled Generator Companies

• 12 Non-scheduled Generator Companies

• 20 Network Service Providers

• 29 Market Customers

• 9 Traders

• An Independent System Operator (NEMMCO)

Potential Clients:• Regulators (ACCC, AER, AEMC)

• Government (DITR, SA, Tasmania)

• TNSPs (Powerlink, Transgrid)

• Customers (EUAA, ERAA, Origin, AGL)

www.csiro.au

The NEM is a Complex Adaptive System

Evolving markets on an interconnected grid about 100 interacting, autonomous agents (firms) + others

about 300 grid-connected, generating units.

Agents are intelligent, adaptive & behave differently pursue goals unique to their firms’ interests

make decisions on the basis of their own knowledge/beliefs

change strategies in the light of their and others’ experiences.

No agent knows what all the other agents are doing each agent has access to only a limited amount of information.

Some act more conservatively than others e.g. they are more constrained (e.g. by debt) than others.

www.csiro.au

Our Science – Agent-based Simulation

Equations-based models too static, aggregate or stylized to handle this complexity

Agent-based simulation computational experiments

software agents, environments and rules

agents learn and adapt strategies over simulated time

evolutionary computation and equations-based methods

can explore impacts of rule changes before their introduction

can evolve adaptive responses of competitors

collective outcomes can be unexpected, even undesirable

www.csiro.au

Presentation Overview

History and concepts (NEM as a CAS)

NEMSIM overview and key features

Practical challenges for agent-based simulation

www.csiro.au

FuelResources

GHGEmissions

Companies

Contract Market

Spot Market

Market Operator

InterconnectorsGeneratingPlants

GeneratingUnits

TransmissionLines

Demand Prices

DispatchBidding

Power Losses

Sim

ula

ted

Sce

nar

io

Simulation Engine

Reports

0

5

10

15

20

25

30

1 2 3

Graphs

HistoricalData

Environment

TechnicalInfrastructure

Agents&

Markets

SimulationLogTables

ElectricitySupplied

DailyBidding

SpotPrices

DemandEvolution

ContractPrices

SupplyEvolution

InvestmentDecisions

GHGEmissions

Tim

eH

orizo

ns

30 min Dispatch

DailyDecisions

WeeklyDecisions

MonthlyDecisions

YearlyDecisions

Longer TermDecisions

User

Scenario Evaluation

Data Input

Input

FuelResources

GHGEmissions

Companies

Contract Market

Spot Market

Market Operator

InterconnectorsGeneratingPlants

GeneratingUnits

TransmissionLines

Demand Prices

DispatchBidding

Power Losses

Sim

ula

ted

Sce

nar

io

Simulation Engine

Reports

0

5

10

15

20

25

30

1 2 3

Graphs

0

5

10

15

20

25

30

1 2 3

Graphs

HistoricalData

Environment

TechnicalInfrastructure

Agents&

Markets

SimulationLogTablesTables

ElectricitySupplied

DailyBidding

SpotPrices

DemandEvolution

ContractPrices

SupplyEvolution

InvestmentDecisions

GHGEmissions

GHGEmissions

Tim

eH

orizo

ns

30 min Dispatch

DailyDecisions

WeeklyDecisions

MonthlyDecisions

YearlyDecisions

Longer TermDecisions

30 min Dispatch

DailyDecisions

WeeklyDecisions

MonthlyDecisions

YearlyDecisions

Longer TermDecisions

UserUser

Scenario Evaluation

Data Input

Input

NEMSIM Overview

www.csiro.au

NEMSIM Overview - continued

www.csiro.au

NEMSIM – Generating Units Displays

Bid Stacks

Revenue GHG Emissions

Dispatch

www.csiro.au

Key Features

Includes all key players in the NEM

Models individual agent’s behaviour

Weather model and data from 100 years

Wholesale market model and extending to other markets, e.g. contract market

Potential effect of distributed generation

Transmission modelling

Bid strategies – e.g. lookaheads

Scenario investigations – new plants, maintenance, emergency shutdown, blackouts, new rules

Scenario comparisons

Reports – dispatch, revenue, CO2, by regions, by companies, by plants, weekly, monthly, yearly

Environmental markets, e.g. carbon trading

www.csiro.au

Other Important Features

XML editor

Simulation time control

Lookaheads

Scenario comparison

Distributed generation

New plants

Maintenance & shutdown

Reports

www.csiro.au

Area of Applications

Short-term trading analyse market bidding data

analyse “what-if” bidding scenarios

Medium-term hedging and contract markets (retailers, generators)

Long-term investment (new generators, transmission lines, distributed generation, renewables)

Greenhouse gas emissions estimates

Carbon trading (when rules are proposed)

Explore the impact of new technologies, new market rules, new grid structures, new participants

www.csiro.au

Presentation Overview

History and concepts (NEM as a CAS)

NEMSIM overview and key features

Practical challenges for agent-based simulation

www.csiro.au

Selection of Agent-based Simulation Platform

Develop our own platform

EMCAS - Argonne National Lab

DIAS/FACET – Argonne National Lab

RePast

Swinburne’s simulation framework – agent implementation of the Victorian Gas Market

www.csiro.au

Other Practical Challenges for NEMSIM

Adequately reflecting all the subtleties inherent in market-to-network interdependencies (DITR)

Developing efficient heuristic algorithms for interactive decision-making e.g. adaptive learning procedures e.g. multi-criteria decision-making

Distinguishing between counterintuitive results and programming errors

Keeping running times reasonable while adding more dynamic features

Developing confidence and trust among potential users and the market operator (NEMMCO)

www.csiro.au

Challenges of Learning in the NEM

Depending on their own competitive position, each generator behaves differently

Bidding strategies differ between states, but even more so between generators within states

Although strategies differ, we may be able to develop a generic bid function for all of them (just varying parameters/markups)

Most generators change bid capacities, occasionally changing bid prices (or price increments)

Thus each firm that owns generating units will need to be examined, if we wish to approximate reality

www.csiro.au

Potential Learning Algorithms

Genetic algorithms (see e.g. Goldberg, 1989, Mitchell, 1998, Chattoe, 1998, Dawid, 1999)

Genetic programming (see e.g. Koza, 1992)

Reinforcement learning algorithms (see e.g. Erev and Roth, 1998; Sutton and Barto, 1998)

Q-learning (see e.g. Watkins, 1989; Tesauro and Kephart, 2002)

Classifier systems (see. e.g. Holland, 1992)

Learning algorithms for automated markets (see e.g. Gjerstad and Dickhaut, 1998; Tesauro and Kephart, 1998)

www.csiro.au

• Bid acceptances/rejections• Unit utilization• Unit profitability• Market price vs. bid price• Weather and Load

NEMSIM agents can look ahead, sideways and back

• Own unit availability• Price trends/peak loads• Hedging strategy• Weather• Load forecasts

• Competing unit availability• Competing bids• Market rules

LOOK BACK (Short and Long Term Memory)

TIM

E

LOOK SIDEWAYS (Bidding rules)

LOOK AHEAD (Strategy evaluation)

Agent

www.csiro.au

Look-aheads in NEMSIM

Agents have look-ahead capabilities

• Run the simulation forward for various periods

• Test & compare a range of available strategies and plans

• Agent adopts strategy showing best possible outcome

• Strategies retested at start of each new period

• Plans/strategies changed to counter changes of others

Does a look-ahead capability add value to the existing (comparative static) approaches?

www.csiro.au

Value of a Look-ahead Capability

www.csiro.au

FG Meeting: Challenges for NEMSIM

Focus more, refine agents adaptive behaviour How agents think and interact, not just bid

Explore demand-side management options Locational issues Customers as agents

Differentiate between short and long-term Treat GHG/carbon tax/emissions trading Explore DG/wind/green power Talk to appropriate potential users

Regulators Government policy makers Network companies

www.csiro.au

Practical Advantages of NEMSIM

Practical application of a CSS methodology To a real world complex adaptive system (the NEM) Socio-economic/physical/environmental interactions

Each and every agent’s adaptive behaviour can be represented and modified

Different collective outcomes can be generated and performances compared in advance (“look-aheads”)

Conditions when unattractive outcomes occur (like price volatility & market power) can be identified

This kind of simulation goes beyond the classical simulation models in energy economics

User-friendly human-machine interface

www.csiro.au

Acknowledgments

Research and Development Group

Energy Transformed Flagship

Swinburne University of Technology

CMIT:

• George Grozev

• David Batten

• John Mo

• Miles Anderson

• Geoff Lewis

• Mario Sammut

CMAR:

• Jack Katzfey

• Marcus Thatcher

• Paul Graham – Theme Leader “Energy Futures”

• Terry Jones - Theme Leader “Low Emission Distributed Energy”

• Prof. Myles Harding

• Neale Taylor

UNSW:

• Xinmin Hu

Click to edit Master subtitle style

Thank you