PJM ©2008 PJM Advanced Control Center Project (AC²): Adventures in SOA, CIM, and Virtualization...

PJM ©2008

PJM Advanced Control Center Project (AC²):Adventures in SOA, CIM, and Virtualization

John Baranowski

2008 EMS Users Conference

September 8, 2008

PJM ©2008www.pjm.com 2

PJM AC² Objectives

• Construct second data and control center

• Implement parallel systems– High availability– Geographically diverse– Secure

• Cost effectiveness– Consolidation of company sites


Business Process Redesign

• Consolidation of operations resources– Master Coordinator– Master Dispatcher

• Enhanced use of technology to support consolidation and low cost of operations

• More economic operations with continued reliability

Supervisors

Dispatchers

Coordinators

PJM West PJM East

SS

PDPD

PDPD

GD GD

RE

TCSC

AC2 AC1

SS

MDMD

MDMD

MC MC

Technology Transition


AC2 High Level 2008 Objectives


Extended BSCs

ATC/AFC

EmergencyProcedures

eMTR

LoadResponse

eSchedules

SmartLogseSuite

MessagesGCA

AGMACM

AGC SCED 1-3

IEP

IAP

CorrelationEngine

BSCs Interface Legacy Interfaces GCA InterfaceIEP

Interface

Information Storage and Retrieval

IS&R InterfaceSCADA Interface

UI Interface

PJM Corporate Systems

Shared Architecture Components\&

AndServices

SA Interfaces

Security Interface

Identity Management,

Single Sign-On

Data Warehouse Interface

ODS

SAS

Mart

DW

User Interface Framework

IMMTNASCADA &RT APPS

RT LMP OTS

SOA ENABLED

eDART CM2/M2M

STLF/VSTLF

eORG/ORG EES OASIS

VSA/TSA

eDATA

RPM

Real time Mkt& Anciallary

SvcsDA MktFTR Auction

RT LMP Interface OTS Interface

TNA Interface IMM Interface SOA Governance

Monitoring ManagementRun-Time Governance

Design-Time Governance

SA Interfaces

AC² SOA Conceptual Model and Scope

• Development of Industry standard messaging architecture• Shared Components for re-usable services• Implementation of Enterprise Service Bus• Security built directly into the architecture


PRDPRD GMSGMS

Member Company

#1

Member Company

#1

PRD – Redundant Primary EMS/MMS

GMS – Redundant ‘AGC only’ EMS

BUCC – Non-redundant Backup EMS

MemberCompany

#2

MemberCompany

#2

Member Company

#3

Member Company

#3

MemberCompany

#4

MemberCompany

#4

MemberCompany

#5

MemberCompany

#5

Member Company

#6

Member Company

#6

MemberCompany

#7

MemberCompany

#7

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

BUCCBUCC

Current PJM System Configuration


AC1AC1 AC2AC2

Member Company

#1

Member Company

#1

MemberCompany

#2

MemberCompany

#2

Member Company

#3

Member Company

#3

MemberCompany

#4

MemberCompany

#4

MemberCompany

#5

MemberCompany

#5

Member Company

#6

Member Company

#6

MemberCompany

#7

MemberCompany

#7

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

RTUs

Future PJM System ConfigurationAC1 – Redundant “Peer” EMS/MMS

AC2 – Redundant “Peer” EMS/MMS

Each member has 2 connections

to send andreceive all data


Private Network - 10 Gigabit Ethernet

Private Network - 10 Gigabit Ethernet

AC2 Information Flow

• Field data presented simultaneously to AC1 and AC2

• Concurrent application and database operations at each data center site• High availability networks and systems / fault tolerant• Optimized recovery times to minimize interruption of customer services

AC1 AC2

TelecomVendor

TelecomVendor


Today’s Complexity

SCADA, Network

Applications, AGC

Market Mgmt

System

Dispatch

Management Tool

eMKT & eFTR

eMTR

eSuite

Energy Mgmt

System


UE&V - A Common User Interface

SCADA, Network

Applications, AGC

Market Mgmt

System

Dispatch

Management Tool

eMKT & eFTR

eMTR

eSuite

Single Sign On

Task Based

Personalization

Customization

Navigation

Security

Comm

on U

ser I

nter

face

Engy Mgmt

System

AC

2 Technology


Intelligent Event Process (IEP) Design Concept


IEP Design Approach

ESB

TNA MARKETBSCsSCADA Historian OTHER

BSCs

CEPEngine

CEPEngine

CEPEngine

COMMON OBJECT MODEL(via Integrated Model Manager)

IEPDATABASE

Intelligent Event Processor

Events H isto ry SearchFi lter

Event No te sEvent 556 – Po ssib le Line Fau lt with Overtrip

13:25 | Joe OperatorLorem ipsum dolor sit amet , consectetuer adipiscing elit . Nam interdum. Donec accumsan , purus ut v iverra pharetra , augue tellus vehicula orci, eget consectetuer neque tor tor id ante . Proin vehicula imperdiet ante . Maur is vehicula velit sed arcu . Ut aliquam pede ac arcu. Phasellus dictum condimentum nisl . Quisque elementum dictum nibh. Curabitur auctor faucibus libero . Suspendisse eu dui ut sem nonummy egestas . Praesent luctus lorem a magna .

13:37 | Bill OperatorQuisque sed arcu ut diam congue hendrer it . Fusce eleifend . In hac habitasse platea dictumst . Maecenas por ttitor ornare neque . Integer at ligula pulvinar dui bibendum laoreet . Mauris accumsan hendrerit neque. In s it amet urna ac mi blandit ornare . Phasellus tristique neque id dui . Nulla ultric ies mattis sapien . Fusce ac justo a tor tor tincidunt dignissim . Nunc sed r isus. Donec sem nisl , mollis ac, volutpat a , blandit a, risus. Donec ultr ices venenatis ante . Ut quis eros. Maecenas placerat accumsan velit . Fusce ante lectus , iaculis non , lobortis vitae , hendrerit vestibulum , sapien. Phasellus venenatis, dui in semper viverra , neque enim feugiat est , eget lacinia elit odio vestibulum massa . Aenean imperdiet ipsum quis lacus. Quisque rutrum suscipit sem .

Ackn owledge

Ackn owledge

?

?

Clear

Clear

Clear

Clear?

N ew Events

Ac kno wle dged E vents – H igh Prio rity

Ac kno wledged Ev ents

Ackn owledge

Ackn owledge

??

Ackn owledge

Ackn owledge

??

Ackn owledge

Ackn owledge

??

11 /2 6/07 13 :3 0

11 /2 6/07 13 :2 9

11 /2 6/07 13 :2 9

11 /2 6/07 13 :2 7

11 /2 6/07 13 :2 6

11 /2 6/07 13 :2 1

11 /2 6/07 13 :2 1

11 /2 6/07 13 :1 5

Line tr ip

?

Line tr ip

Line tr ip

Line tr ip

Line tr ip

Line tr ip

Line tr ip

Po ssible Line Fault w ith Overtr ip

Po ssible Bus Fau lt







A P

A P

A P

A P

A P

A P

A P

A P

AP

AP

AP

AP

AP

AP

AP

AP

3 45kV

3 45kV

3 45kV

3 45kV

3 45kV

3 45kV

3 45kV

Clear?

11/26/07 13 :17

11/26/07 13 :12

Op erating Volation

Op erating Violation

Ca lver t C liffs Voltage Limitations

Ca rlis C orner #2 CT Special Sch eme in effe ct

BG&E

AE

BG&E

AE

500kV

11/26/07 13 :24

11/26/07 13 :23

11/26/07 13 :20

Line trip

?

Line trip

Possible Line Fault with Ove rtrip

Possible Bu s Fault

Possible Line Fault with Ove rtrip

AP

AP

AP

AP

AP

AP

3 45kV

3 45kV

??

?

Ad d

E ven t ID Timestamp Event Type Event Text Comp any Zone Voltage Level

Event ID T imesta mp E ven t Type Event Text C ompany Zone Voltage L eve l

Event ID T imesta mp Even t Type Event Text C ompany Zone Voltage L eve l

…...

… ...

Alarm ID14321143221432714355

Timestamp11 /26/07 13 :2 911 /26/07 13 :2 911 /26/07 13 :2 911 /26/07 13 :2 9

B1/B2/B3 -Alarm TextStation 1/Breaker/Bre ake r A – Bre ake r A is OpenStation 1/Breaker/Bre ake r B – Bre ake r B is OpenStation 1/Breaker/Bre ake r C – Bre ake r C is OpenStation 1/Breaker/Bre ake r D – Bre ake r D is Open

561

560

559

558

557

554

553

550

556

555

552

551

549

IEP UI


Generation Control Applications Design


Key Issues and Risks

• Implementation of significant new technology – System synchronization – Information model management– SOA implementation– Advanced solutions– User interface

• Availability of key vendor and customer resources– Key power system engineers– Application developers with domain expertise– SOA design and integration architects with domain expertise

PJM ©2008

Common Information Model (CIM)

CIM-based messages for SOA-based application integration

– Extending CIM for PJM – Business Service Processes (BSPs)


CIM-based Messages for SOA Integration

• A common vocabulary is needed for integration in a SOA architecture

• CIM 11 chosen as the foundation information model for messages

• Shared data is being mapped to the CIM, and extended by– PJM AC2 SOA Implementation Project– Collaboration with vendors



• Scope– Subset of PJM applications– Shared data– Prioritized data interfaces to exclude data shared

across database links– Focused on data interfaces from EMS and MMS

software to other applications

• Producing message definitions with extensions• Extensions are intended for broader industry

application



• Message definition process– Identify shared data– Create master business definitions– Map local data elements to master definitions– Map master definitions to CIM– Extend CIM where necessary– Create message definitions based on CIM mappings– Validate message definitions with application

business owners– Refine message definitions in conjunction with

Business Service Process (BSP) designs



• Business Service Process (BSP)– A BSP is an Enterprise Service Bus (ESB) process

representing the flow of data amongst applications– Today data is shared via point-to-point interfaces,

usually on a time scheduled basis– BSPs are designed to optimize the publishing and

consuming of data across multiple applications– BSPs are being designed to be event-driven– Impact on message definitions

• Decomposition of large messages• Aggregation of messages coming from a single source

PJM ©2008

eDART

AGC

FTP

eDARTAGC AF

eDART Adapter

Business Service Process

This is a Sonic ESB Topic - a specific type of queue that supports multiple subscribers.

The role of the BSC Adapter is to listen to a specified ESB Topic and process the message upon delivery. In this example, the eDART BSC Adapter will subscribe to the Topic, consume the message and write a specified flat-file on the eDART filesystem.

The responsibility of the AGC BSC Adapter ends with successfully placing a message on the specified Topic. The routing, transformation and subsequent delivery of that message is the responsibility of the BSP.

Integration Use Case

• Transfer of flat file from EMS BSC to PJM BSC



• Challenges– CIM is still maturing in subject areas beyond the

original Power System Network Model exchange– Governance of PJM CIM messages

• New versions of CIM• Changes to PJM metadata

– Convergence of industry understanding related to CIM across vendors

PJM ©2008

Virtualization TechnologyInfrastructure

Virtualization Plans:• PJM AC2 server virtual environment• Conceptual virtual server deployment• PJM AC2 Network and Storage Area Network (SAN)

Virtualization


PJM Environment at Siemens (MN)

Physical servers:2 Virtual servers:~38

PJM Environment in PA

Physical servers:4 X Blades Virtual servers:~24

Excludes Dedicated servers

AC2 Virtual Server Development Environment

x86

Windows/Linux Platform

IBM RS/6000

AIX Platform

PJM Environment at Siemens (MN)

Physical servers:6 Virtual servers:~60

PJM Environment in PA

Physical servers:2 Virtual servers: ~20

Excludes Dedicated servers


PJM AC2 Virtual Performance Monitoring

Shared CPU, Memory, Adapters under 4 X VIO servers


AIX LPAR Layout Strategy

LPAR#1Primary 2 CPU

uncapped

LPAR#2Primary 2 CPU

uncapped

LPAR#3Primary 2 CPU

uncapped

LPAR#4Secondary 1CPU uncap.



LPAR#4Primary 2 CPU

uncapped

LPAR#5Primary 2 CPU

uncapped

LPAR#6Primary 2 CPU

uncapped




PowerVM - Adds CPU cycles on demand

Use of Dedicated Fiber Channels

Use of Integrated Virtual Ethernet (IVE)


(Redundancy is not shown to make the drawing “cleaner”)

RED Environment GREEN Environment

• Virtualizing the router allows for multiple independent route forwarders (VRF).

• Each VRF has its own routing table, and do not know about the other VRF’s.

• Solution allows for multiple environments to share network hardware, and still keep access controls at the distribution layer.

Traditional network design with new implementation


Network LAN/SAN Virtualization

Network

Virtualization

SAN & Storage

Virtualization

•Standardized on CISCO 6509 switches

•Virtual Routers

•Virtual LAN’s

•Virtual Firewalls

•Standardized on Brocade DCX (Data center Fabric)

•L-SAN’s (Logical Storage Area Net)

•Each environment in its own SAN Zone

•World Wide Port SAN Zoning

•Multiple fabrics using DCX

•Storage Virtualization through the use of Storage aggregates, volumes & shared pools.


Questions?

PJM ©2008 PJM Advanced Control Center Project (AC²): Adventures in SOA, CIM, and Virtualization...

Documents

Transcript of PJM ©2008 PJM Advanced Control Center Project (AC²): Adventures in SOA, CIM, and Virtualization...