Dissertation-v2-Decentralized Monitoring and Control of ...

ABSTRACT

HUQ, KAZI MOHAMMAD MOYEENULL. Decentralized Monitoring and Control of Electric Power Distribution System with a use case of Community Energy Storage System. (Under the direction of Dr. Mesut E. Baran and Dr. David Lubkeman) This work focuses on the platform and framework required for developing decentralized

monitoring and control applications for a distribution system. Decentralized control of

Community Energy Storage (CES) devices has been considered as a use case.

The first chapter gives an overview of energy management system and applications. Then the

development of centralized and decentralized CES control applications are discussed. The

next chapters discusses the software architecture of a typical centralized SCADA\EMS

platform and outlines the architecture required for a decentralized platform. Then, suitability

of IEC 61850 standard for decentralized monitoring and control is investigated and an

approach to developing decentralized applications in IEC 61850 enabled environment is

proposed and demonstrated. Finally, CIM standard is reviewed for exchanging power system

topology information for distributed control.

Decentralized Monitoring and Control of Electric Power Distribution System with a use case of Community Energy Storage System.

by Kazi Mohammad Moyeenull Huq

A dissertation submitted to the Graduate Faculty of North Carolina State University

in partial fulfillment of the requirements for the degree of

Doctor of Philosophy

Electrical Engineering

Raleigh, North Carolina

2015

APPROVED BY:

_______________________________ _______________________________ Dr. Mesut E. Baran Dr. David Lubkeman Committee co-chair Committee co-chair _______________________________ _______________________________ Dr. Aranya Chakrabortty Dr. Munindar Singh

ii

BIOGRAPHY

Kazi Mohammad Moyeenull Huq was born in Chittagong, Bangladesh. He received the

Bachelor of Science in Electrical Engineering from Bangladesh University of Engineering

and Technology ( BUET ). He pursued MS and Ph.D in Electrical Engineering at the North

Carolina State University.

iii

ACKNOWLEDGMENTS

I express my deepest gratitude to all the people who have helped me and guided me during

the course of study.

First, I would like to thank my advisor Dr. Mesut Baran for guiding and supporting me

throughout the research work. I also thank my co-advisor Dr. David Lubkeman for his

directions and the committee members Dr. Aranya Chakrabortty and Dr. M. Singh for their

encouragement.

I would also thank my friends and lab mates – you made the environment at FREEDM center

enjoyable.

Last but not the least, my deepest thanks goes to my family – my parents, my wife, my sister

and brother for their support and encouragement.

iv

TABLE OF CONTENTS

LIST OF TABLES…...…………………………………………………………………..vi LIST OF FIGURES……………………………………………………………………...vii

Chapter 1: Monitoring and Control of Community Energy Storage systems ........................1

Introduction .................................................................................................................1

Overview of a SCADA\EMS application ...................................................................3

1.3 A centralized EMS application development (for management of CES) ..................6

1.4 A decentralized EMS application development (for management of CES) .............10

Chapter 2: Monitoring and Control using Centralized SCADA \ EMS platform ................15

Centralized SCADA\EMS Architecture: ..................................................................15

State of the art in SCADA\EMS platform: ................................................................16

GreenBus SCADA\EMS core data services and API ...............................................23

Implementation of a centralized EMS application (CES management application) .27

Chapter 3: Monitoring and Control using a Decentralized SCADA\EMS ..........................31

3.1 Distributed approach to monitoring and control : .....................................................31

3.2 Basic Requirements for Decentralized Control .........................................................32

3.3 Decentralized Distribution Management System (DDMS) for decentralized control: .................................................................................................................................33

3.4 Application development process in DGI platform ..................................................36

3.5 A decentralized EMS Program (CES management application) implementation ....37

Chapter 4. Integration of IEC 61850 standard to the proposed DDMS ...............................41

4.1 IEC 61850 architecture: ............................................................................................41

4.2 Overview of the IEC 61850 Engineering process .....................................................47

4.3 Proposed DDMS architecture with IEC 61850 .........................................................49

4.4 Mapping DDMS functions to IEC 61850 logical nodes ...........................................53

4.5 Mapping Group management ....................................................................................53

4.6 Modeling the CES application: .................................................................................55

4.7 Simulation of the proposed IEC 61850 enabled application .....................................59

Simulation setup in DTM ..........................................................................................60

A. Setup of the MMS scheme ........................................................................................60

B. Setup of the GOOSE based scheme ..........................................................................62

v

C. Data setup ..................................................................................................................63

D. Application setup .......................................................................................................64

Simulation output ......................................................................................................64

Network traffic In the MMS based scheme : ............................................................64

Network traffic in the GOOSE based scheme ...........................................................67

Chapter 5: Utilizing CIM for exchanging topology information .........................................70

Introduction to CIM ..................................................................................................71

The requirements for exchanging topology information in CIM standard ...............72

Application of CIM standard for the DDMS ............................................................74

How to find the line impedance: ...............................................................................78

Demonstration of the utilization of CIM-XML based topology information ...........80

Chapter 6: Conclusion and Future work ..............................................................................84

Proposed modification to the DDMS architecture : .............................................................84

REFERENCES ....................................................................................................................... 88

Appendix -A ............................................................................................................................ 92

Appendix -B ............................................................................................................................ 95

vi

LIST OF TABLES

Table 1: Parts of the IEC 61850 standard documents ..............................................................42 Table 2: Portion of CIM-XML file for Load at node - 671 .....................................................81

vii

LIST OF FIGURES

Figure 1: A community microgrid scenario .............................................................................. 5 Figure 2 : Aggregation of load and generation for monitoring ................................................. 6 Figure 3: Flowchart for a centralized EMS application ........................................................... 9 Figure 4 : Generic software architecture of a SCADA\EMS system ...................................... 15 Figure 5: GEC SCADA\EMS software architecture .............................................................. 16 Figure 6: A traditional code development approach ............................................................... 20 Figure 7: An SOA based code development approach ........................................................... 21 Figure 8: Application level data exchange using AMQP ....................................................... 22 Figure 9: Networked view of GreenBus Software Architecture ............................................. 23 Figure 10: Data Acquisition and measurement processing steps ............................................ 24 Figure 11 : Data access using API and Decoupling of data .................................................... 25 Figure 12: Flowchart for a centralized EMS program ........................................................... 28 Figure 13: Location of Decentralized nodes in distribution system ....................................... 31 Figure 14: Communication hierarchy for decentralized nodes ............................................... 32 Figure 15: basic DGI Architecture .......................................................................................... 34 Figure 16: Decentralized CES control application flowchart ................................................. 40 Figure 17: Virtualization of a real device. .............................................................................. 42 Figure 18: Details of an XCBR (circuit breaker) node ........................................................... 43 Figure 19: The hierarchical structure of the IEC 61850 data model. [22] ............................. 44 Figure 20: Device Model for IEC 61850 ................................................................................ 45 Figure 21 : Data exchange at device level .............................................................................. 46 Figure 22: IEC 61850 based substation communication architecture [23] ............................. 46 Figure 23: The engineering dataflow for IEC 61850 [Ref: IEC 61850 Engineering Process by Schneider electric] .................................................................................................................. 48 Figure 24: Proposed DDMS Architecture............................................................................... 49 Figure 25: Proposed DDMS architecture incorporating IEC61850 standard ......................... 50 Figure 26: IEC 61850 communication mapping to OSI 7 layer model .................................. 51 Figure 27: Communication between DDMS and an IEC 61850 ‘not’- enabled device (approach-1) ............................................................................................................................ 52 Figure 28: Communication to DDMS between an IEC-61850 capable and not capable device (approach-2, preferred) ........................................................................................................... 52 Figure 29: A generic GGIO LN used for group management ................................................ 54 Figure 30 GGIO logical node is used to facilitate group management................................... 54 Figure 31 An example system with three community microgrids .......................................... 55 Figure 32: Dataflow among application nodes in a three community microgrid based scenario. .................................................................................................................................. 57 Figure 33: Modeling of a node in IEC61850 .......................................................................... 57 Figure 34: The SCL model of a community microgrid .......................................................... 58

viii

Figure 35: Details of a meter logical node (MMXU) ............................................................. 58 Figure 36: Simulation development in DTM environment ..................................................... 60 Figure 37: DTM Workspace setup .......................................................................................... 62 Figure 38 : Simulation host setup ........................................................................................... 62 Figure 39: GOOSE model setup ............................................................................................. 63 Figure 40: Example CSV data file for node 1 ......................................................................... 63 Figure 41: Sample output from DTM ..................................................................................... 64 Figure 42: MMS packets captured in the network ( at 152.14.125.29/FRDMPWR2 computer ) ............................................................................................................................................... 65 Figure 43: MMS packets captured in the network ( at 152.14.125.74/POWERBARAN computer ). .............................................................................................................................. 65 Figure 44: Capture of a MMS packet in network. .................................................................. 66 Figure 45: Network traffic calculation. ................................................................................... 66 Figure 46: Packet capture in the network while using the GOOSE scheme. .......................... 67 Figure 47: Network statistics generated by wireshark from a 117 second run of the GOOSE scheme..................................................................................................................................... 68 Figure 48: Exchanging Topology information using CIM standard ....................................... 70 Figure 49: IEC 61970 base packages in UML ........................................................................ 72 Figure 50: Exporting data from UML to CIM-XML format .................................................. 73 Figure 51: A 5-bus feeder ....................................................................................................... 74 Figure 52a: ConnectivityNode element shows the nodes of the system ................................. 75 Figure 53: DistributionLineSegment class structure contains the information on the lines ... 76 Figure 54: The spot loads are contained under EnergyConsumer Class ................................ 76 Figure 55: Detailed information of a member of the EnergyConsumer class ........................ 77 Figure 56: Details of the Source node in contained in the EnergySource class ..................... 78 Figure 57: Members of a DistributionLineSegment class ...................................................... 79 Figure 58: PerLengthPhaseImdedance class members ........................................................... 79 Figure 59: The impedance value of the line (per unit length) ................................................. 80 Figure 60: Extracting line information from a CIM-XML file .............................................. 81 Figure 61: Extracting Source and Load information from CIM-XML file ............................ 82 Figure 62: An Example load-flow application utilizing the information from CIM-XML. ... 82 Figure 63: Initial DDMS architecture ..................................................................................... 84 Figure 64: Proposed DDMS architecture ................................................................................ 85

1

Chapter 1: Monitoring and Control of Community Energy Storage systems

Introduction Community energy storage (CES) is the concept of using distributed battery energy storage units at residential level [1] [2]. The CES units store energy from the grid and the feed the energy back to the grid when needed. The CES units are connected to the secondary of the distribution transformer.

It is anticipated that adoption of residential PV systems will accelerate in near future. New technologies will improve the overall efficiency of the solar systems and reduce cost of ownership. Different state & federal incentive programs combined with non-measurable issues like increased environmental awareness and interest in reducing dependency on fossil fuel will encourage residential users to adopt renewable resources like PV.

When the PV sources generate a significant portion of the power demand in a distribution feeder or system, it causes certain issues to the distribution systems such as voltage fluctuation due to intermittent power output of PV systems. This may result in flicker at customer premise. In a high penetration of PV scenario, reverse power flow towards grid occurs when the load is low and the PV system generates power close to full capacity. [3] [4] [5]

The traditional distribution system is designed for unidirectional power flow from source (substation) to load. The associated control and protection schemes are designed accordingly. Reverse power flow introduces bi-directional power flow which can raise voltages above normal limits thus modifying the expected voltage profile. This can cause malfunction of voltage control devices.

Peak energy consumption affects the grid both technically and economically. In order to meet the peak energy demand a utility has to run expensive peaking power plant only during the peak hours. The transmission and distribution systems may get congested during system peak requiring system capacity upgrades. Peak energy consumption reduction can be effective measures for utilities in improving grid efficiency and deferring system upgrades. For consumers, it will result in reduced charges from the utility.

In this work, community energy storage systems are proposed to mitigate some of the major issues which are caused by high penetration of PV at power distribution system level. In order to achieve the target objectives, an ‘Energy Management System(EMS)’ is required that will calculate the optimal dispatch for the community energy storage devices. It should be noted that in electric power industry, the term EMS often refers to transmission level energy management. For distribution system, the term ‘Distribution Management System(DMS)’ is used. Here, the terms EMS and DMS will be used interchangeably to mean

2

energy management systems and in this work only distribution system related applications are considered.

From terminology usage point of view, the application that is developed for energy management of CES is termed ‘energy management application’, ‘EMS application’ and mainly ‘CES application’. The platform for running the application is called ‘Decentralized DMS’ or DDMS. DDMS refers to a generic conceptual platform while the ‘Distributed Grid Intelligence (DGI)’ is a specific software implementation by MS&T which is still an ongoing project.

There are two main architectural approaches in the development of energy management systems:–

Centralized EMS

Decentralized EMS

In a centralized EMS architecture, the field data and parameters for the whole system are collected at a central location by using a Supervisory Control and Data Acquisition (SCADA) system. The information is then saved in a database and made available to a central controller (e.g.- computer) which runs the energy management program to make the control decision. Then, the decision is sent to the control devices in the system. The advantage of such a system is that the EMS has detailed information about the whole system, which enables it to make optimal decision.

But, As the system size grows larger -

The system needs to be able to transfer a large volume of data reliably to the central SCADA\EMS system from the field sensor and send control commands back to devices.

It processing power requirements increases with volume of data. It may become difficult to operate the EMS application on a large number of data set.

Failure of communication system to meet soft or hard deadlines may degrade the performance of an EMS application or may even cause the EMS application to fail.

From security point of view, it means if the central system fails for some reason, the full system may be compromised.

On the other hand, in a decentralized EMS, there are many decentralized controllers. Each controller monitors a certain portion of the system assigned to it. Therefore, each controller has information about the area it monitors. It may share the collected information with another controller via communication network. Thus, the controllers can collaborate with each other to achieve certain objective. The controllers may report its status, certain monitored data or control setpoints to a control center. The main advantages of decentralized

3

EMS is that there’s no single point of failure like a centralized system and each controller can continue to monitor & control its designated area even if it loses communication with its neighboring controllers. The controllers solve a problem in a distributed manner. Therefore, each controller requires less computing power than a centralized system.

It should be considered that certain applications are better suited for decentralized control even if it is possible to come up with a centralized control scheme. For example, for the community energy management system which will be introduced in the next section, battery energy storage control occurs at a fast rate. There’s no requirement to optimize a large system on a second to second basis. Therefore, this scenario is better suited for decentralized control scenario. A certain area is delegated to a decentralized controller and it monitors and controls the battery energy storage devices in that area. One controller can communicate with another controller to co-operate such that together they achieve better control of the combined area. This will be explained further during application development process.

Last but not the least, the traditional power distribution system monitoring and control is limited to substation level. Therefore, the number of points for monitoring and control were limited, too. With the advent of distribution system automation and schemes like volt-var control that requires control of capacitor banks in a distribution feeder, the number of monitoring and control points are on the rise. each SCADA\EMS system can satisfactorily monitor a certain number of devices or data points. Once the limit is reached, the system needs to be expanded. However, the expansion processes have their own sets of challenges. Therefore, in order to handle the large amount of data that will be generated by the future power distribution systems, the information that can be locally processed for decision making should be done in such manner. Only relevant information should be reported to central control center.

A SCADA\EMS needs to be capable of exchanging and utilizing data with other power system softwares. This required standardization of the data exchange format. It is also termed as software interoperability. During the early phase of SCADA\EMS, most of the SCADA\EMS platforms were proprietary. It means that the user of the system was granted very limited or no options to develop new EMS software. Proprietary systems can have legal bindings that may prevent a user from getting a software developed from a third-party software development company.

These requirements for a SCADA\EMS will considered when the centralized and decentralized SCADA\EMS platforms will be analyzed.

Overview of a SCADA\EMS application A general overview of an EMS application will be given in the following section. Let’s consider an energy storage dispatch application according to the scenario shown in the Figure-1. The Feeder from a substation serves a residential area. The homes are connected to

4

the distribution feeder laterals via a distribution transformer. Here, the homes in this area have rooftop PV panels. Several nearby homes served by a distribution transformer are grouped together as ‘community microgrid’. A community energy storage device can serve the communities in each lateral.

For monitoring purpose, the load and PV generation of nearby community microgrids located in the same lateral can be aggregated. This is shown is figure-2.

The CES management application will use the real time data of the monitored aggregated PV power generation and the load consumption. Based on the collected data, it will calculate the required dispatch of the energy storage and it’ll modify the device setpoint to dispatch the actual CES hardware.

Therefore, the application needs a SCADA API to be able to get the updated data from the database. Similarly, once the dispatch setpoint for the energy storage device is calculated, the application needs an API to update the SCADA data-point in order to dispatch the associated energy storage. The static data (device configuration like Energy storage capacity) can be loaded from system model file.

This is an overview of how an EMS application works. We’ll consider the implementation details in the next chapters.

5

Energy Storage Device


Homes with PV panels




Substation Feeder

Laterals

Figure 1: A community microgrid scenario

‐

6



Substation Feeder

Laterals

Aggregated Load

Aggregated PV generation

Aggregated Load

Aggregated PV generation

Figure 2 : Aggregation of load and generation for monitoring

1.3 A centralized EMS application development (for management of CES)

A centralized community energy storage(CES) management application has been developed. The GreenBus SCADA platform [6] has been used for this purpose. The SCADA\EMS requirements and application development process will be evaluated during the process.

The application collects information regarding PV power generation and Load power consumption of a community microgrid. Then, based on the collected information and associated PV generation and Load forecast, the application dispatches an energy storage device such that the given objectives are satisfied.

The energy management objective is to minimizing the total cost of energy consumption for a day and other objectives have been selected in order to mitigate the issues that arise from high penetration of PV in a community microgrid. The issues have been discussed in the introduction chapter.

The objectives of this CES management application are –

‐ Reduction of total demand or energy consumption during the heavy load conditions. ‐ Manage the impact of intermittent PV power output on the system, by

(i) smoothing out the PV power output, (ii) prevent excessive reverse power flow back to the utility.

7

The CES management objective described earlier can be expressed mathematically as follows- First of all, the control needs to observe the power balance all the time. Note that the power balance for the system in figure-2 for any ‘i’-th interval will be

, , – , , … (i)

It should be noted that positive PL means load consumption. Similarly, positive Pg means PV supplies power to node, positive PES means the storage is in charging mode i.e. drawing power from node and positive Pg means grid supplies power to the node.

The main objective of the control is to minimize the energy or demand from the grid during especially high load conditions in the system. Price of electricity during peak load period is higher than non-peak period. Hence, the following objective function is used to differentiate these periods.

Minimize, ∗ ∑ , ∗ ∆ ∗ ∑ , ∗ ∆ … (ii)

where ‘p’ to ‘p+q’ are the non-peak hours, ‘p+q+1’ to ‘p+k’ are the peak hour, and.∆ is the control interval. Note that the function uses different weights for energy consumption during peak and non peak periods, and hence, choosing the weight W2 during peak period much higher than the weight W1 for non peak hours will result in shifting the net energy demand from the grid from peak period to non-peak period. Finally, note that PES = X is the control variable, and the controller needs an estimate of the load and PV profiles in order to determine the optimal control scheme for ES device.

(1) The energy storage device has charging and discharging power limits. If the maximum charging or discharging power of the device is denoted by Pmax, it can represented as -

…( iii)

(2) The device has maximum energy storage capacity limit. If the maximum energy storage capacity of the device is denoted by Es,max. Then at any given time the energy stored has to be within zero and Es,max. If the initial state of energy of storage device in E0, it can represented by -

0 ∑ , …(iv)

(3) ‘Limiting reverse power flow’ condition can be represented by -

– …(v)

(where, 0

(4) A smoothing condition is imposed to counter sudden change of PV output power. Smoothing is performed by injecting power from the energy storage device when the PV output power change is beyond a certain set limit ∆P whin a time interval. Thus, the smoothing condition can be represented by -

8

∆ , , ∆ …(vi) ;

where,P , P , P , ) is the power injection to the node by ES & PV at ‘i’-th interval.

Thus, the P will be limited to variation within set parameter range ∆ .

(5) The State of Charge (SOC) of the battery storage can vary within a range. The condition is imposed to avoid deep discharge or sudden rise in voltage in some batteries at the region close to 100% SOC.

... (vii)

where, ∗ ,

The higher margin can be utilized as a safety margin for special situation such as when the battery is at the higher SOC limit but needs to absorb excess PV energy. An override algorithm can be designed for the described scenario. Constraint (vi) is a refinement of constraint (iv) and these two can be combined during implementation. The problem, the objective (ii) with the constraints (iii) to (vi), is a discrete optimal control problem. The problem has single objective and has linear constraints, and hence, a linear programming technique is adopted to solve the problem, and A MATLAB program was developed for this purpose. Given an estimate of the next 24 hours load and PV power output profile, the initial control sequence can be estimated. However, since the actual PV power output may vary significantly from the forecast and considering the fact that short term forecast is more accurate than a day ahead forecast, the program needs to update the initial control sequence as the updates of PV forecast becomes available to improve the performance. Therefore, the concept of model predictive control (MPC) which is suitable for online tracking of fast changing variables is adopted here –

An optimization window is selected and forecasted PV power generation and load information is used to solve for the chosen period. For a window size has N data points, there will be N control signals X=[ X1, X2, X3 … XN ].

The first control signal X1 is sent to ES device for the first control interval. 1. For the next iteration, PV output forecasts are updated. The previous solution(s) X1 with

respective PV and load data are saved and set as a historical value for the new iterations. 2. The problem is solved with the new data to get the updated solution Xn=[ Xn1, Xn2, Xn3

… XnN ]. Where Xn1 = X1. 3. New solution Xn2 is the dispatch command for the ES during the second control interval. 4. The process is repeated. i.e- fixing initial values of [X1, X2] , updating PV forecast and

re-calculating X then applying X3 for third interval and so on.

The flowchart for the EMS application can be expressed as follows.

9

Figure 3: Flowchart for a centralized EMS application

Let’s assume that the CES_control() method incorporates the above mentioned dispatch algorithm. If this method is supplied with the correct inputs, it’ll calculate and return the required dispatch values for the CES units in the area. Now, let us can consider how to collect the input data and how to update the endpoints with the calculated dispatch by using the provided GreenBus API.

The inputs for the CES_control() method are – aggregated PV and Load measurement data for all the community microgrids in the area (referring to Figure 1 and 2). The application can collect the data in the following way -

The CES devices existing in the system can be listed by using the following API method. Here it is assumed that the system configuration file uses a consistent endpoint naming scheme. For example in the system configuration file, endpoints beginning with CES1, PV1 and Load1 should be assigned to the same community microgrid. Here the idea is, if the numeric subscripts can be used to match the devices from same community microgrid. It can be done during system modeling.

10

In this part, an overview of the centralized EMS application was discussed. The implementation specific details will be discussed after introducing the centralized SCADA API.

1.4 A decentralized EMS application development (for management of CES)

In a decentralized scenario, The system consists of multiple communities. Each community is considered a decentralized which is controlled and monitored by a decentralized controller. The communities or nodes have both local and global objectives. Therefore, the objectives for the centralized EMS can be reframed as follows:-

For the decentralized scenario, the reduction of total peak energy consumption and limiting reverse power flow objectives are considered as global objectives. Both of the objectives benefit the utility from systems operation standpoint. Smoothing is chosen an local objective since it can cause customer annoyance (due to flicker , reboot or misoperation of sensitive appliance etc.) and therefore it should be mitigated as close as possible to the source of variation.

Now, the problem can be represented mathematically as follows:

The controller needs to observe the power balance all the time. Note that the power balance for the ‘j’-th microgrid for any ‘i’-th interval will be -

P , P , – P , X … (1)

Where, positive PL values represent power consumption by load. Similarly, positive PS represents total power consumption of the microgrid from feeder. Positive Pg means PV supplies power to node, positive X means power is supplied by Battery ES. X is the control variable. The subscript ‘i j’ denotes i-th timestep for the j-th microgrid.

Therefore, the objectives and other constraints of the problem are:

(1) The main objective is to minimize the energy or demand from the grid during especially high load conditions in the system. Price of electricity during peak load period is higher than non-peak period. Hence, the following objective function is used to differentiate between these periods.

11

Minimize, f W , ∗ ∑ ∑ , ∗ ∆T W , ∗ ∑ ∑ , ∗

∆T …(2)

where ‘p’ to ‘p+q’ are the non-peak hours, ‘p+q+1’ to ‘p+k’ are the peak hour, and.∆T is the control interval. Note that the function uses different weights for energy consumption during peak and non peak periods, and hence, the weight W2 during peak period is chosen to be much higher than the weight W1 for non peak hours. It is assumed that all the microgrids in the feeder uses same W1 and W2 values.

(2) The energy storage device has charging and discharging power limits. If the maximum charging or discharging power of the device is denoted by Pmax, it can be represented as –

, …( 3)

(3) The device has maximum energy storage capacity limit. If the minimum & maximum energy storage capacity of the device in microgrid ‘j’ is denoted by (Esmin, j , Esmax, j). Then at any given time the energy stored has to be within Esmin, j and Esmax, j. If the initial state of energy of storage device in E0, j. It can represented by -

, ∑ , …(4)

(4) ‘Limiting reverse power flow’ at substation level can be represented by –

∑ , –∑ , ∑ , P ...(5)

Here,P 0 is used. It will be a negative value beyond which the RPF will be limited.

12

(5) Smoothing is performed by injecting power from the energy storage device when the PV output power change is beyond a certain set limit ∆P within a time interval. Thus, the smoothing condition can be represented by -

∆ , , ∆ … (6)

where,P , P , X is the power injection to the j-th microgrid node by ES & PV at

‘i’-th interval. Thus, the P , will be limited to variation within set parameter range ∆ .

A MATLAB program was developed for solving the problem and determining control command X . Given the estimate of the next 24 hour load and PV power output profile, it solves for the whole day’s control sequence. It is called best case scenario and the solution is used to determine the general dispatch trend of the energy storage devices.

In the decentralized scenario, there’s no central control center for ES control. But, this scenario requires a master controller to calculate how each ES will contribute to achieve the global objective. Any of the microgrid controllers can assume the role of master. The master controller will have access to the averaged PV and load forecast. It’ll receive real time PV and load measurements from the other microgrids in the group.

For example, sample PV forecasts and a load profile for a three microgrid scenario is shown in fig. – 3b

Figure 3b: Individual microgrid load forecasts and total load forecast for the three microgrids

13

It can be noted from the load plot that the time period considered can be sub-divided into several time periods -

The first time period corresponds to night till morning load pickup (which is about 7 am). During this time period CESs are charged to 50% capacity. During the time period of 7:00-16:00, PV output is rising and the load is low and flat, thus CESM charges the CESs to their full capacity, and at the same time avoids reverse power flow. This time period is divided into two periods – the second period when the PV is generating power but there’s no reverse power flow (i.e.-around 7:00-10:00 hr), the third period when the PV generation causes reverse power flow (i.e.- around 10:00-16:00 hr). The fourth period starts when the evening load begins to pick up at 16:00 hr., reaches its peak at 19:00 hr and the peak load period ends at 24:00 hr at which the load drops to its night level. During this period CESs are discharged till they reach to their lowest SOC level. A linear equation is used to find the total energy required to be discharged. Then the total energy is divided among individual ESDs in proportion to their storage capacity (Esmax). Then it is converted to the power dispatch command (X ).

Based on the time periods, the rule-based control can be described as follows. Master-slave based control scheme has been selected to implement the rule based control.

The rule-based control scheme for the master controller is as follows:

1) Master first estimates the time when reverse power flow will start by using the forecasted generation and load data. For example in Fig. 3d, the net power goes negative for five minutes at 10:45 am. Therefore, the algorithm will mark this time as the beginning time of reverse power flow. Master updates the estimation as the PV forecast is updated.

The master also calculates the total energy that flows towards the grid during the reverse power flow (RPF) condition. This is the net energy that needs to be absorbed by the CESs in order to limit the reverse power flow.

2) During the second time period, if the calculated net energy due to RPF is greater than the available chargeable energy capacity of all the CESs, then a calculated amount of energy will be discharged before the reverse power flow (RPF) begins in order to ensure that the CESs can limit the reverse power flow and will not exceed maximum storage capacity in the process.

3) During the third time period (when reverse power flow begins), at each time step, the master calculates how much energy needs to be absorbed by the CESs to limit RPF. Then it converts the energy value to expected charge/discharge command for each CES according to the ratio of their energy capacity to total ES capacity

14

4) When the fourth time period (peak hour) begins, the SOC of each of the CES will be close to the maximum allowed SOC. Then the master controller begins to dispatch the CESs linearly such that they discharge to minimum allowed SOC at the end of the peak period (fourth period).

5) Once the peak period ends, the master controller will command the CESs to charge back to initial SOC of 50% before the next day begins (during first period).

Thus, at each control update step:

1. Master uses the rules described above to determine the total energy dispatch required during this control step. Then the total energy is divided among individual CESs in proportion to their storage capacity (Esmax). Then it is converted to the power dispatch command ( /∆T). Master sends this expected charge/discharge dispatch command, X, to slave CESs using communication link.

2. Each CES controller (slave) calculates a dispatch range , , , by using

equations (7a,b). If the expected dispatch (X) is within the calculated dispatch range, smoothing is ensured and therefore the corresponding CES is dispatched. If the expected dispatch is outside the dispatch range, then the slave controller sets the dispatch value equal to the violated limit , , .

This decentralized EMS application for CES monitoring and control will use this rule based algorithm.

15

Chapter 2: Monitoring and Control using Centralized SCADA \ EMS platform

Centralized SCADA\EMS Architecture: In this chapter, the software architecture of a centralized SCADA\EMS platform will be analyzed. In order to discuss the state of the art of SCADA\EMS platforms, an advanced SCADA\EMS platform called GreenBus [7] will be studied. Then, the implementation of a community energy storage management application on a centralized SCADA\EMS platform will be discussed.

Figure 4 : Generic software architecture of a SCADA\EMS system

A generic software architecture for a traditional centralized SCADA\EMS is shown in Figure-4. The basic building blocks can be identified as – communications, real time data processing and management functions and the EMS applications. This is a stacked representation method in which each blocks utilizes the functionality provided by the layer below it & provides service to the layer above it. According to Figure-1, in a centralized SCADA system, the data from the field devices are collected to a central location via communication channels. There are specific communication protocols like DNP3 for SCADA communications in power system. Then, the data are processed (i.e.- limit check, transformation, alarm generation etc.) and saved to a historian for later retrieval. This falls under the ‘Real time data processing and management’. EMS applications (i.e.- economic dispatch, load forecast etc. ) utilize the real-time and historical data to calculate set-points for controllable devices. The set-points are then dispatched via the communication channel. If the system involves a large number of telemetry points which requires significant processing power, then specialized communication processors are used to reduce the processing load of

16

the main processor. These types of communication hardwares are called ‘Front end processor’ and they are part of the communication block.

State of the art in SCADA\EMS platform: The concepts of a basic SCADA system has been introduced in previous section. Now, it can be extended to study a state of the art SCADA\EMS platform called GreenBus system ( also known as ‘Reef’ ). The figure-5 below shows the building blocks of the GreenBus SCADA in a stacked software architecture diagram. This figure will be referred to when the different aspects of the GreenBus software architecture will be discussed. The details of the blocks are as follows:

Figure 5: GEC SCADA\EMS software architecture

The architecture of the GreenBus SCADA\EMS system comprises of the following components:

Operating system (OS) – The Operating system is a collection of softwares that manages the computer hardware resources and provide common services to the computer programs (i.e.- input/output control, file management etc.). The GreenBus platform runs on Linux operating system.

Advanced Message Queuing Protocol (AMQP) – AMQP [8] is a high performance messaging protocol. Apache Qpid is an open source implementation of AMQP used

17

in the GreenBus platform. It’s a middleware that runs on top of the operating system to provide messaging service between softwares [9]. However, It should be noted that the GreenBus SCADA\EMS system is based on service oriented architecture. Here, the service components massage each other using the message broker. Therefore, the stacked software architecture does not provide a correct operating view of the software system. Later, a networked architecture diagram will be used to show the interaction between different software components using the message broker middleware. The function of the AMQP broker is to manage all incoming and outgoing messages of the system. It receives messages from the clients and create routes, queues etc. Here, ‘clients’ are the services and/or applications that use the AMQP broker service. Therefore, use of AMQP allows for decoupling between EMS applications and the services (i.e. – data services) it consumes. AMQP also supports scalability of the system. These topics will be discussed in detail in next subsection. The Qpid AMQP client implementations are available in C, Java, Perl, PHP, Python and Ruby language. Therefore, an application can exchange message with another application if both of them have the AMQP API to communicate to the AMQP broker. It provides application level interoperability independent of programming language or OS platform. It also facilitates implementation of service-oriented architecture which is explained later.

Front end processor (FEP) – In SCADA systems, ‘Front end processor’ takes care of the communications and related data processing of the data from the field devices ( i.e.-RTU, IED etc.) and vice versa. Due to the large number of field devices, FEP can be a secondary processor that takes care of communication related pre-processing of data in order to reduce the computational load of the main processor. Conceptually, it’s a layer between communication link and database. Communication can take place via Ethernet/IP network, modem, wireless, microwave systems etc. [10] In GreenBus SCADA system, FEP is a software layer that takes care of the communication protocol (i.e.- DNP3) related conversions and encoding/decoding of data collected from the field devices and vice versa . [6].

Core Data services – The SCADA system provides data services (i.e.- measurements, controls, alarms and events ) in order to facilitate data exchange between the applications and SCADA system. Application programming interfaces (API) are provided in order to access the data services by different applications. There are other services like services for authentication, system modeling etc. These will be discussed in detail later..

Historian – A ‘Historian’ is a database for saving the data collected from field devices, control setpoints etc. Due to the high volume of data traffic, a SCADA

18

database needs to be fast, responsive and reliable. GreenBus utilizes ‘Cassandra’ database system [6] [11] [12], which is a highly available and scalable distributed database system.

Applications – EMS ‘applications’ are programs that perform specific tasks like monitoring, processing or displaying the SCADA data. The applications can process the available field device information and make certain control decisions. For example, an application may calculate the dispatch setpoint for an energy storage device. Another application may control a capacitor bank for voltage control of a given system. In the GreenBus system, the HMI is also considered an application.

Application Programming Interface (API): The purpose of an API is to define the interaction between different software components. In general, API is an approach to provide a defined method to access the services offered by a software. API provides a layer of abstraction. As a result, the API user do not need to know all the implementation details of the software services he plans to utilize.

The explanation so far is based on Figure-4, which is a stacked software architecture. The GreenBus SCADA system utilizes Service Oriented Architecture (SOA). The concepts of SOA and how it is applied in the GreenBus will be explained in the next subsection. It is relevant to mention here that due to the nature of service oriented architecture, it is more appropriate to represent the building blocks of the GreenBus as a collection of services connected via a network. It’ll be presented in Figure-6 after the service oriented architecture of GreenBus is introduced.

In this part of the report, some of the key design challenges in the architecture of the SCADA\EMS platform will be introduced. The key design challenges in the development of a SCADA\EMS platform are identified as - interoperability, application modularity and scalability [6]. The details of these issues and how they are addressed in the GreenBus platform will be discussed in the following subsection.

a) Interoperability: For example, in case of a proprietary system, an application developed by a third-party vendor may not operate in the proprietary SCADA system. This refers to software level interoperability so that the platform is open for any vendor to develop applications. In software engineering terminology, interoperability is defined as the ability of two or more systems or components to exchange information and to use the information that has been exchanged.1 [5]

1 IEEE glossary defines interoperability as the “ability of a system or a product to work with other systems or products without special effort on the part of the customer. Interoperability is made possible by the implementation of standards.” .

19

In software systems development, adoption of ‘open standards2 ’ and ‘open platform’ concept facilitates interoperability. Open Standards facilitate interoperability and data exchange among different products or services and are intended for widespread adoption.”[6] .

b) Application modularity: If a software module imports or directly accesses class members of another module, it creates strong coupling between the two modules which is not desirable. It may later result in software failure if the classes are modified later. Application modularity aids system scalability.

c) Supporting increasing volumes of diverse field data (i.e.- meter data, phasor measurements, traditional telemetry for operations etc. ).

The following approaches are used in Greenbus SCADA architecture to address the aforementioned issues: [6]

1) Open Platform : An open platform software has fully published and documented external API that allow third party developers to access and utilize the software capabilities. Therefore, an open platform facilitates interoperability and allows other entities to utilize the software or platform.

The API documents and code of the core GEC GreenBus system is freely available. Therefore, the open platform feature allows any developer to develop an application for the Greenbus SCADA system.

2) Service Oriented Architecture: Service Oriented Architecture (SOA) is a specific software architecture. SOA offers fast and low cost software development & management and flexibility in software deployment environment. It is a useful architecture for development of large and dynamic projects.

In SOA, a ‘Service’ is a unit of logic that runs in a network to handle a specific task. (i.e. - perform a calculation, access database etc.). Services interact by passing message to each other on a network.

A service is independent of other service or software since the interaction occurs through a specified interface. This property is called loose coupling, which allows modification of the implementation logic later. The message passing structure allows use of different programming languages for application development - provided that the message passing broker supports the languages. SOA supports scalability of a software product. Since the

2 International Telecommunications Union (ITU-T , an international standards development organization ) defines open standards as: “standards made available to the general public and are developed (or approved) and maintained via a collaborative and consensus driven process.

20

services communicate via message passing in a network, they don’t need to be on the same computer or server. Therefore, when any component in a system becomes a bottleneck or needs to be extended, it can be redesigned, optimized or moved to another server for better performance.

Let us consider an example with the following figure to illustrate the concept of SOA. In this example, a developer has developed a program called ‘Program1’ (Figure-6) which reads solar data (PV1 and PV2) from a database, calculates total generated power and saves it in the database. Now, he needs to develop another program (‘Program2’) which works with load data. In traditional approach-

the developer has to develop new code for reading the ‘Load’ data

he’ll have to integrate the new code to the existing codes.

he’ll compile the whole code again to create an executable program.

On the other hand , in SOA environment, the services that are required to develop the program (i.e.- getData, calc_sum, saveData ) already exists and are running in the server. The application shown as Services1 was developed earlier to perform the task described for ‘Program1’. So, the developer only needs to create a new service (‘Service2’) which will call the existing services with correct data. He’ll reuse the existing services and therefore, he’ll code for only a small portion of functionality which doesn’t exist already ( i.e.- Otherservices2 module). Since, all the services in SOA run as executable modules, the developer don’t need to stop any portion of the SCADA system to deploy his new application. He’ll compile and run the newly developed services alongside the existing pool of services.

main() {//Program1

…getData(PV1) ;getData(PV2);

R1=calc_sum(PV1,PV2);

…saveData(R1);

…}

main() {//Program2

…getData(LD1) ;getData(LD2);

R2=calc_sum(LD1,LD2);

…saveData(R2);

… }

Figure 6: A traditional code development approach

21

This is how SOA facilitates code re-usability and fast deployment of new services.

3) AMQP: It has been discussed in the introduction to GreenBus section that AMQP is messaging protocol that provides queuing, routing, message orientation, reliability and security. GreenBus uses the Apache Qpid implementation of AMQP . It’s a ‘wire level’ protocol that describes the format of the data that is sent across the network. It is an OSI application layer protocol. Therefore, the applications using the AMQP can exchange messages without any specific formatting or conversion requirements.

The Qpid AMQP implementation is available in C, Java, Perl, PHP, Python, Ruby language. Therefore, an application can exchange message with another application if both of them have the AMQP API to communicate to the AMQP broker. It is shown in the following figure. The figure-8 shows that although the applications are written in different languages, they can pass message between themselves because of the AMQP broker middleware and messenger. The figure also shows that the applications are able to exchange message among themselves inspite of the difference in OS platform and programming language. In should be noted that each service that will connect to the AMQP message bus requires a AMQP message client which provides the API to send and receive message to / from other services.

getData saveDataRet=Calc_sum

Service1()Data: PV1,PV2

service2()Data: LD1,LD2

otherService1

otherService2

Figure 7: An SOA based code development approach

22

Figure 8: Application level data exchange using AMQP

Therefore, AMQP implementation provides application level interoperability independent of programming language or OS platform. It also facilitates implementation of service-oriented architecture.

Therefore, The GreenBus platform utilizes the SOA concept. In order to apply the SOA paradigm, the project utilizes the following technologies: AMQP, Protocol buffers (ProtoBuf) and RESTful architecture. The technical details of the technologies are discussed in appendix-2.

Now that the key design challenges and the technologies used to meet those challenges in GreenBus have been covered, the architechture of the GreenBus platform can be better represented as a network of services. Therefore, the networked view of the architecture will be as follows-

23

Figure 9: Networked view of GreenBus Software Architecture

Figure-9 shows how the previously described components (in figure-5 ) interact with each other in the service oriented architecture using messages. In general, any entity in the system is considered a service. Therefore, one service can request a legitimate service from any other service component. For example, in figure-8, the application App.-1 can request a measurement reading to core data services. The same App-1 can also request the App-2 to calculate real power from some data.( assuming that it is the service provided by App-2 ).On the other hand, the HMI can request both ‘core data service’ and App-2 for real power information in order to display the data to a user. The services do not even need to reside in the same server. The services only need to be accessing via network. This is how SOA provides horizontal scalability. In GreenBus platform, this scalability applies to hardware, database and EMS software since all of them are considered services according to SOA principle.

GreenBus SCADA\EMS core data services and API As shown in Figure-4, the applications reach the SCADA data through the core data services. This data exchange process between the SCADA layer and the application is facilitated by the ‘application programming interface(API)’. API takes care of the details of the process. Therefore, API decouples the applications from directly accessing data at the SCADA layer.

24

The data exchanged by applications are of two types, static data and dynamic data. Static data are fixed parameters like line impedance or energy storage capacity. These are loaded during runtime from a configuration file. On the other hand, dynamic data are the monitored values from field, such as voltage or current value monitored at a certain location is an example of dynamic data.

In GreenBus, the processed dynamic data are termed as measurements. On the other hand, control setpoints for the field devices are called commands. Commands can be either analog or digital.

As shown in figure-10, the raw data is acquired from field using the front end processor which takes care of any protocol conversion. Then the raw data goes through some data processing steps. First , It is checked if the datapoint is over-ridden. In GreenBus, a user or application can override the value of a raw data for reasons like data inconsistency. The data point can also be marked as ‘Not in service’- in which case it will not update from the incoming raw data. Then, the configured transformations are applied (i.e.- scaling, value mapping) and event/alarms are triggered. Finally, measurements are stored in the database and published to the bus using measurement channel. The data bus contains the most recent or live data for faster access. Now, according to the software architechture in figure-4, the applications can access the data in the databus or database using the core data services. Therefore, decoupling is achieved between raw data and the data consumed by the applications via the core data serivces. The GreenBus provides API in order to access or modify the data in the databus and the database.

Figure 10: Data Acquisition and measurement processing steps

25

The decoupling concept is illustrated in figure-11. It can be seen that if an application needs to access some specific data point value, it can use the API to make a simple method call like readdata(“Dataname”). The API takes care of the intricate tasks from creating messages to request the data using the core data services. The core data services in turns gets the data from proper sources and forwards it to the application that requested the data. Finally, the application received the returned value and it doesn’t need to know the details of how the data was collected.

Figure 11 : Data access using API and Decoupling of data

The core data service provides the API to access and modify the operational data of the GreenBus system. The operational data consists of measurements, commands, events and alarms.

The API provides the following functions to access measurement services :

Access Measurement data & retrieve Measurement History

Measurement Publish – Subscribe

The Measurements service allows to query a measurement point. Measurement values can be published to the GreenBus by an application client.

The application needs to know the pointname in order to query or publish values. The point naming is dependent on how the data points are modeled in system configuration file. For example, a pointname can be “CES_demo.CES1.kW”. Here the pointname means CES1 is one of the ES devices under the location modeled by CES_demo . ‘kW’ represents the point that corresponds to the real power dispatch of the energy storage. This pointnames are defined in the system model file and are loaded to the SCADA system from the model file. A complete example code for updating a measurement point is provided in Appendix-1.

26

Example: // Get service interface for measurements (required before accessing any service type ) MeasurementService measurementService = client.getService(MeasurementService.class);

// 1. Get latest measurement for a point by name // ‘pointname’ is a string Measurement measurement = measurementService.getMeasurementByName(pointName); //2. Retrieve a list of the last five measurements for the point List<Measurement> measurementList = measurementService.getMeasurementHistory(point, limit); // int limit = 5 //3. Retrieve a list of measurements in the time interval // (Limited to ten datapoints by setting limit=10) List<Measurement> measurementList = measurementService.getMeasurementHistory(point, twentyMinutesAgo, fiveMinutesAgo, returnNewest, limit);

In the above example, The first line is required to create the service interface. Then any of the shown API functions can be called to get the data for the specific pointname. The full API documentation for GreenBus is available online.

The above approach collects data from databus only when the API methods are called. The GreenBus platform provides publisher-subscriber based approach for collecting data. In this approach, an application can subscribe to a data endpoint. If the measument value of the endpoint is updated, the update is published on the databus and the application receives the updated data. This approach is preferable if the application requires to calculate something (i.e- perform powerflow ) as soon as a certain data is updated.

So, an EMS can subscribe to a specific endpoint so that it’s notified of any event or change in data value.

27

// Build a MeasurementSubscriber callback to accept subscription events MeasurementSubscriber result = new MeasurementSubscriber(); // Get the Subscription object Subscription<Measurement> subscription = result.getSubscription(); // Start the subscription, providing the MeasurementSubscriber as a callback subscription.start(subscriber); // subscriber is the endpoint // Other code utilizing the ‘subscriber’ data // Cancel subscription to clean up resources in broker subscription.cancel();

Similarly, the other operational data ( i.e.- commands, events, alarms ) can be updates and modified. Commands are used to modify the state of the system. Events are objects that record defined occurrence or change of state of the system (i.e.- overvoltage, undervoltage, device on/off change etc). Events provide an audit log of system history. Alarms are system occurrences that require operator intervention. Alarms are closely tied to events. All alarms are associated with events, but not all events cause alarms. The example of API for each of the data types are available in the appendix-3.

Implementation of a centralized EMS application (CES management application)

The algorithm used for the centralized EMS application has been discussed in chapter 1.3. Here, we’ll consider how to collect the input data for the application from the SCADA and update the setpoints of the SCADA server to dispatch devices by using the provided API.

According to the AMQP broker implementation of GreenBus platform, an EMS program can connect to the GreenBus SCADA system in order to collect data and update set points in real time. A template is available which includes the AMQP client in Java. Once the code for the energy management program has been developed, the inputs of program can be collected from the GreenBus using the core data services API. Similarly, outputs can be dispatched. This process is shown in the flowchart below(Figure-12). The data collection and dispatch update process will be discussed here.

28

Figure 12: Flowchart for a centralized EMS program

In chapter 1.3, the algorithm used to program the CES_control() method has been discussed. If this method is supplied the correct inputs, it’ll calculate and return the required dispatch values for the CES units in the area. Now, let us can consider how to collect the input data and how to update the endpoints with the calculated dispatch by using the provided GreenBus API.

The inputs for the CES_control() method are – aggregated PV and Load measurement data for all the community microgrids in the area (referring to Figure 1 and 2). The application can collect the data in the following way -

The CES devices existing in the system can be listed by using the following API method. Here it is assumed that the system configuration file uses a consistent endpoint naming scheme. For example in the system configuration file, endpoints beginning with CES1, PV1 and Load1 should be assigned to the same community microgrid. Here the idea is, if the numeric subscripts can be used to match the devices from same community microgrid. It can be done during system modeling. Otherwise, the developed needs to manually map the related endpoint which maybe time consuming.

29

Similarly, the GreenBus SCADA can be queried to list existing PV and Load data points:

List<Entity> List_PV = entityService.getEntitiesWithType("PV"); List<Entity> List_LOAD = entityService.getEntitiesWithType("LOAD");

Then EMS application can update datapoints for each device in the system by making a method call like the following example. For multiple devices in an area , a loop is used to collect the data for the specific type of devices in the system area.

For example, in the following code, the aggregated PV measurement value of the ‘i’-th community microgrid has been collected.

// Get entity name ; //‘i’ represents the sequence of the device in the list. i= 0, … , ListSize‐1 String pointName_PV = List_PV.get(i).getName(); // Get latest measurement for the point by name Measurement measurement_PV = measurementService.getMeasurementByName(pointName_PV);

Similarly, we can collect measurement data from the PV and Load measurement points.

Based on the data, the application can call the ES_control() method and calculate the dispatch value for the CES devices. The following set of code dispatches a community energy storage device pointed by it’s endpoint mane ‘pointname_CES’ at the calculated ‘value1’.

CommandLock commandLock = commandService.createCommandExecutionLock(setpoint); Commands.CommandResult commandResult = commandService.executeCommandAsSetpoint(pointName_CES, value1); commandService.deleteCommandLock(commandLock);

List<Entity> List_CES = entityService.getEntitiesWithType("CES");

30

Then the value needs to be published in the Greenbus if we want to display the dispatch value in a HMI interface or as a trend. The following code published the value ‘value1’ in a measurement point ‘pointName_CES’ :

// Build measurement object to represent update Measurement.Builder builder1 = Measurement.newBuilder(); builder1.setName(pointName_CES ); builder1.setType(Measurement.Type.DOUBLE); builder1.setDoubleVal(value1); //Calculated value builder1.setTime(time); builder1.setUnit("kW"); builder1.setQuality(Measurements.Quality.newBuilder().build()); List<Measurement> list1 = new ArrayList<Measurement>(); list1.add(builder.build()); // Publish measurement, returning success/failure try{ boolean wasPublished = measurementService.publishMeasurements(list1); } catch (ReefServiceException ex) { System.out.println("Could not publish measurement ! " + ex); }

This is how the community energy storage devices will be dispatched and the HMI can be updated to display the command values.

Therefore, this chapter discussed the typical architecture of a centralized SCADA\EMS platform, then discussed the state of the art of in SCADA\EMS system with respect to an advanced SCADA\EMS. The design challenges and specific technologies chosen to overcome the challenges have been pointed out. Finally, an EMS application development using the Greenbus API has been discussed.

31

Chapter 3: Monitoring and Control using a Decentralized SCADA\EMS

3.1 Distributed approach to monitoring and control : With the advent of the smart grid, we new smart devices being incorporated to the distribution systems. For example, more PVs, smart meters, electric vehicles and battery energy storage devices are being added to the distribution grid. Novel devices like smart transformers are expected to be deployed in the field.

In case of decentralized control, there may be a central control center for overall monitoring of the system. It may be sufficient for the decentralized controllers to send a summary of their status or an event that is important to the control center.







Substation Feeder

Laterals


Decentralized nodes

Decentralized nodes

Central Control Center

Figure 13: Location of Decentralized nodes in distribution system

32

Figure 14: Communication hierarchy for decentralized nodes

3.2 Basic Requirements for Decentralized Control According to the previous discussion, the distributed nodes monitor devices under its control. The nodes may form a group in order to collaborate. Therefore, the software running at the nodes need to have some form of group management policy to facilitate group formation. Once the group is formed, then the group members may need to exchange different information regarding the monitored devices or other parameters. So, the nodes need some way to easily share information with another peer node. The communication requirement can be split into two parts,

- Local communication with a device.

- Communication with a peer node. (peer to peer communication)

These requirements for a decentralized platform can be summarized as follows -

1) The platform needs to provide peer to peer communication between nodes. It can be

achieved by using specific middleware or broker in the architecture.

2) The platform should be able to communicate with the devices it’ll monitor and

control in order to collect data from the devices as well as sending control signals.

3) The platform should be able to accommodate the requirements for an CES application

like group formation, coordination and leader election

33

4) Scalability of the system – effect on system performance when a significant number

of CES management controllers are added to a group or a system.

5) Detecting failure of a controller and take corrective measure.

6) Effect of Latency on the application – a. Data packets may arrive out of order b. Packet drop and delay in wireless communication technologies.

3.3 Decentralized Distribution Management System (DDMS) for decentralized control:

In this section, the architecture of a proposed DDMS platform will be introduced. In later part of the chapters the requirements will be analyzed. Some of the ongoing efforts on decentralized control [10] [13] [14] together with the theories of distributed systems [15] has been reviewed to come up with the requirements of a decentralized controller. From this point onwards, DDMS will be used as the abbreviated form of the ‘Decentralized Distribution Management System’.

In this section, a distributed controller development approach called Distributed Grid Intelligence (DGI) [13] is reviewed. The core DGI modules are:

Broker

Group management

There are other modules that facilitate the operation of the core modules. Therefore, from the discussion, a basic DGI architecture can be envisioned as follows:

34

Figure 15: basic DGI Architecture

Broker module:

The Broker module performs message delivery and routing between DGI nodes. It utilizes the modules Connection Manager and Connection in order to perform it’s task. An application can create a message using the provided API and send it to one of the peer nodes. The CMessage.hpp library provides the API to create a message and put required contents in it (i.e.- application identifier, source / destination hostname, custom message content etc.). The API also provides functions to read information from a received message.

For example, the following code constructs a message that contains the CES device configuration (e.g. - Esmax, Pmax, %SOC upper and lower limits). Then the last line sends the message to its peers in the group. The DGI message can be considered a collection of strings that are send through the network to another DGI. It should be noted that the messages intended for energy storage begins with the ‘es’ initial. ( e.g..- ‘es.Esmax’ ). This is an arbitrary convention used to identify the intended module of a message.

CMessage m1_; m1_.SetHandler("es.ReplyConfig"); m1_.m_submessages.put("es.source", GetUUID()); m1_.m_submessages.put("es.Esmax", Esmax1); m1_.m_submessages.put("es.Pmax", Pmax1);

35

m1_.m_submessages.put("es.SOC_UpperLimit", SOC_UpperLimit1); m1_.m_submessages.put("es.SOC_LowerLimit", SOC_LowerLimit1); //send the message peer->Send(m1_) ;

Group Management:

The group management module performs the tasks of group formation and leader election in a group. when a DGI entity comes online, it searches for other DGI entities and attempts to join an existing group. If it can’t find or join an existing group, then it creates a new group.

The group management module implements Garcia-Molina’s Leader election algorithm. The module do not need any input via API to perform its task. The current list of peers in a group can be queried by sending a specific message to a node in the group. Another group’s peer list can be queried in a similar manner. The response message contains the list of peers and can be parsed using provided function. The group management module sends a message to all modules whenever there is a change in group members. Upon receiving the message, the modules can update their peer list.

For example, the group management module can be queried in the following manner to collect the list of peers in a group. In this example, the temp variable will contain the identifier of the members of the group.

PeerSet temp; temp = gm::GMAgent::ProcessPeerList(msg,GetConnectionManager());

State collection:

The DGI incorporates a ‘state collection’ module. It is a distributed approach to synchronizing data between DGI nodes. Later, ‘state collection’ performance will be compared to other approached such as polling technique.

The state collection module collects the consistent snapshot of the states of the group’s node devices. Distributed systems do not have a global clock or shared memory. Therefore, collecting the global snapshot of the system is challenging considering that the system states can change anytime during the process of taking snapshot among the peer nodes. This will render the attempt useless. The DGI State Collection module is implemented based on the Chandy-Lamport algorithm [16] . Specific device states can be queried using the API. Here state means any a designated system variable such as SST voltage, CES State of charge(SOC) etc. Any module can initiate a state collection. It can specify in the initiation

36

message which state it wants to be updated. The devices are defined in device manager using a XML file as input.

For example, the following message will ask for the current snapshot of the state of the SST devices in the group. The replies will be received in two messages containing the “CollectedState.state” and "CollectedState.intransit" message header.

CMessage m_cs; m_cs.SetHandler("sc.request"); m_cs.m_submessages.put("sc.deviceType", "Sst"); m_cs.m_submessages.put("sc.valueType", "gateway"); m_cs.m_submessages.put("sc.source", GetUUID()); m_cs.m_submessages.put("sc.module", "lb"); GetPeer(GetUUID())->AsyncSend(m_sc);

3.4 Application development process in DGI platform

In order to develop an application in DGI platform, the developer will need to create and integrate a new module to the existing DGI code.

The process can be describes as follows:

Develop the algorithm that’ll be implemented.

The DGI platform uses ‘Handler’ concept for message handling. Handlers are the methods in a module which are called when a specific message is received by the DGI. The handler contains codes to enable it to respond to the received message. For example – A handler can be defined in ‘EScontrol’ module to call a method ‘ReplyWithES_Config’ when a certain message containing ‘es.configRequest’ header is received from another DGI node. Therefore, the method ‘ReplyWithES_Config’ will respond by sending back another message containing the Energy storage device configuration.

Therefore, to develop a new application module, the developer needs to define – o the messages the application can send, o the messages the application can expect to receive and o the application’s action in response to the messages.

Once the module code is developed, the new module needs to be registered and schedule into the DGI scheduler’s execution cycle.

Compile the whole code and run the DGI.

37

3.5 A decentralized EMS Program (CES management application) implementation

In this section, Distributed Grid Intelligence (DGI) – software will be considered and analyzed as a platform to realize decentralized applications for power system. A decentralized application for community energy storage management will be used for case study. The algorithm for the application has been developed in chapter-1.4.

In order to develop the decentralized application, a master-slave based approach is used. The application incorporates both master and slave mode. Once the decentralized nodes form a group, the group management functionality elects one of the nodes as ‘leader’ or ‘co-ordinator’. The application running in that node assumes the master mode while the other members automatically assumes the slave mode.

The slave nodes forward the device configuration and real time monitored data to the master node. Based on the information and the global objective, the master decided what should be the expected dispatch of all the nodes in the group. Then, it forwards the information to the individual nodes. Each node revises the received expected dispatch request based on its local objective and dispatch the energy storage under its control.

A list of methods developed for the decentralized energy storage control application are provided below. A summary of their functionality is described here -

According to the application development process described in section-3.4, the method ‘EScontrol’ contains the logic to calculate the expected dispatch in master mode. At the end of calculation it sends the calculated expected dispatch to each slave node. When the slave node received the message, it triggers a method call to ‘HandleExpectedDispatch’. It contains the logic for revising the expected dispatch values according the current status and local objective of the slave node. Once the value is revised , then the method sends dispatch command to the energy storage device using the revised dispatch value.

In order to facilitate the application specific data exchange (e.g.- CES configuration, monitored data etc), the following methods and handlers are used.

ReqConfig , HandleReqConfig, HandleReplyConfig – to exchange CES configuration

ReqToSendPglData, HandleReqToSendPglData – to exchange real time monitored data

SendPglData, HandleReceivedPglData – to exchange real time monitored data

CheckAllDataReceived, CalculateSumESCapacity etc. – methods that check different conditions or perform required data processing before calling the ‘EScontrol’ method

38

HandlePeerList, HandleAny – These handler allow the group management module to update the application in case of any change of peers or leader in the group.

The message handlers are registered at the constructor. For example- the following line in the contruction registers ‘HandleReqConfig’ in the ES control module. If the DGI node receives a message that contains “es.ReqConfig” handle in its header, the DGI keeps the message in a message queue. During the next scheduled run of the module, the DGI will invoke the ‘HandleReqConfig’ and deliver the message the it.

RegisterSubhandle("es.ReqConfig",boost::bind(&ESAgent::HandleReqConfig, this, _1, _2));

// List of developed methods and handlers for the ES Control Application. ESAgent ~ESAgent // The following methods are related to runtime scheduling of the application. Run StartStateTimer ESManage // The following methods and handlers exchange CES configuration ReqConfig HandleReqConfig HandleReplyConfig ReqToSendPglData HandleReqToSendPglData SendPglData HandleReceivedPglData // These methods check different conditions or perform required data pre=processing before //calling the ‘EScontrol’ method CheckAllDataReceived CalculateSumESCapacity // These methods contain the main logic for master and slave mode of operation EScontrol HandleExpectedDispatch // These methods handle peer-list received from the group management module // when group order ( size or leader ) changes HandlePeerList HandleAny

39

Application flowchart:

The program flowchart is shown in the flowchart in Figure-15. Its detailed explanation is as follows -

The Run() function is the starting point for the module. Then it calls the ESManage function. The ESManage manages the execution of the CES control algorithm by scheduling the function to run at given interval. If the DGI is the group leader, it asks for the other DGI nodes to start sending PV and load data to the leader by calling the ‘ReqToSendPglData’ function. This function broadcasts a message to the group members to start sending data. When the member nodes receive the message via message handler ( HandleReqToSendPglData ) , the node responds by sending back data to the Leader node. The Leader (also called master) node verifies that it received the data from the members and calls the EScontrol() function to calculate the expected dispatch values for each node. Then these expected dispatch values are sent to the member nodes via message passing.

When the member nodes receive the message containing expected dispatch value, the handler function HandleExpectedDispatch() is called. The received values are revised in order to meet the smoothing objective criterion and the physical limits of the energy storage device. Then the energy storage device is dispatched using the revised value.

The leader in a group can dynamically change due to node failure, communication failure etc. Whenever, a group formation is modified, the group management module messages each module the list of peers in the group. HandlePeerList handler is invoked when a message from the group management module is received. This handler calls the ReqConfig() function which requests the member nodes to send its’ CES device configuration. The HandleReqConfig() handler is used to collect the information and store it for calculations.

In this chapter, the requirements for a distributed platform has been reviewed and the algorithm for CES management has been implemented.

40

Figure 16: Decentralized CES control application flowchart

41

Chapter 4. Integration of IEC 61850 standard to the proposed DDMS

In this work, we propose an approach for developing an IEC 61850 based DDMS platform. First, the IEC 61580 architecture and the associated modeling process will be described. Then, we’ll discuss how the DDMS components should be modeled in an IEC-61850 compatible manner. A distributed community energy storage control application will be used as a use case to illustrate the data-model and communication related requirements. Finally, DTM software will be used as a tool to simulate and demonstrate the performance of the proposed DDMS with CES dispatch application.

4.1 IEC 61850 architecture: IEC 61850 is a standard for electrical substation automation. Although, over time the standard has evolved to include elements outside the scope of the substation. The basic vision of the IEC 61850 is to provide an interoperable digital communication for information exchange between substation devices (IEDs) for monitoring, protection and control purpose. The scope IEC 61850 is not limited to substation automation. It is adopted for other applications that are known as ‘outside the fence’ applications such as for DER, PV systems and hydro-electric systems.

In this section, a short introduction to the basic concept and architecture of IEC 61850 is provided. In IEC61850 standard, interoperability means that two devices made by different manufacturers are able to exchange data between them, make sense of the data and utilize the data without losing context such as units. Therefore, the standard addresses the communication and the data aspect in separate subparts (Table-1). IEC 61850 also standardizes the exchange of substation configuration information by using a standardized format called Substation Configuration Language (SCL). Table-1 shows the parts of the IEC 61850 standard. The part-7 of the standard is of specific interest for this work which discusses the basic communication structure and the data modeling concepts for IEC 61850 standard.

In order to achieve the interoperability, an actual device is modeled as a virtual device. As shown in Fig. 17, an actual circuit breaker (shown in the right side) is modeled as an element named XCBR ‘logical node’ in the virtual model. Fig 18 shows the details of the data members of the ‘XCBR’ LN such as their type, explanation and if it’s a mandatory or optional element. This is called ‘data modeling’ and it offers visibility into the actual device’s function.

42

Table 1: Parts of the IEC 61850 standard documents

Figure 17: Virtualization of a real device.

43

Figure 18: Details of an XCBR (circuit breaker) node

Therefore, a complete function provided by a device is broken down into smallest components that hold the information that can be exchanged between devices. These smallest components are defined as logical nodes or LNs. The LN’s are virtual representation of the real devices. The common logical nodes are standardized in IEC61850 in order to provide interoperability. Several LNs from similar or different real devices are grouped together in order to develop a function. This group is then called a Logical Device(LD). LDs are an approach to organizing the data. LNs that are required for a certain protection or control function are put under one LD. One or multiple logical devices can reside in a ‘Physical’ device. The physical device provides network connectivity for information exchange. The figure 19 shows the hierarchical structure of a data model. The standardized LN together with the hierarchical structure allow an user or application to access specific data of a device using a naming convention. For example, the current operating status (‘Loc\stVal’) of a circuit breaker(‘XCBR’) under logical device (‘RelayA’) can be accessed using the following notation – ‘RelayA\XCBR\ Loc\stVal’.

The logical node is a data structure that contains a list of relevant information. For example the circuit breaker LN contains a data (also called data object) named ‘Pos’ which is used to set the current position of a circuit breaker (Fig. 18). Each data belongs to a data attribute that specifies the type of the data. ‘Pos’ is of type ‘controllable double point’ (DPC). The IEC 61850 standard defines the LNs for the common power system devices. But, IEC61850 does

44

not dictate how a certain function should be implemented. Therefore, the function algorithms are manufacturer specific.

Figure 19: The hierarchical structure of the IEC 61850 data model. [22]

Fig. 20 shows a more detailed organization of the hierarchy in a IED (also called a server) from communication point of view and fig.-21 shows how the data are exchanged and consumed by functions(also called applications) between devices.As explained earlier,an IED contains one or multiple ‘logical devices’(LD) to organize the local device data. A function or application utilizes the data stored in the LNs. If a LN is assigned to store information from the local device it monitors and/or controls, then the LN is called primary logical node(e.g.- Metering LN, MMXU). On the other hand if the LN stores data which is calculated or derived from the primary LN data, then the LN is called secondary LN (automatic tap changer control LN, ATCC).

IEC 61850 standard provides communication schemes like ‘report control block’(RCB) in order to facilitate the data exchange between IEDs. An RCB is a list of ‘data’ points or LNs whose data is synchronized at specific time interval or event occurrence (such as change of data ). While creating an RCB profile, the data exchange mechanism (i.e.- MMS or GOOSE ) is specified.

It should be mentioned that IEC 61850 do not specify where a function or application should reside. According to the IEC 61850 philosophy, an application or a function can be executed in any hardware that can access the LNs required for the input and output data. This concept is called free-configuration. Fig-21 shows that the application or function can reside in one

45

IED, distributed among multiple IED or it may reside in the station computer. Therefore, IEC 61850 standard supports distributed application or function development.

From communication architecture point of view, organization of an IEC 61850 based substation is shown is the fig. – 22. The standard considers two separate local area netwroks (LAN) – the ‘station LAN’ connects as IEDs to each other and to a router or other mechanism for communicating to any device outside the substation network. On the other hand, the ‘process LAN’ is used to carry the raw data from the intelligent devices to the station level devices. If the measurement devices are conventional devices ( e.g. – not ethernet capable) then an adapter is required to convert the signal to digital signal and transmit to IED. For devices that use other protocols like Modbus, a protocol adapter is required [17]. Two separate networks are suggested for the process bus and the station bus. Some substations use the same network for both the station bus and the process bus.

Figure 20: Device Model for IEC 61850

46

Figure 21 : Data exchange at device level

Figure 22: IEC 61850 based substation communication architecture [23]

To summarize, the main features of IEC 61850 standard are:

‐ Data modeling: The protection and control functionalities in the substation are modelled into different standardized logical nodes that can be grouped under different logical devices. Logical nodes represent the smallest part of a power system function and associated services to access and manipulate the data. Fig. 17 shows the data model hierarchy. Fig. 21 shows a more detailed organization of the hierarchy in a server and how the data is exchanged with the devices.

47

The status information and controls contained in the logical node can be accessed and updated by functions or applications that process the data and make control decisions. For example, the LN for an automatic tap changer controller (or Automatic voltage regulator) is named ATCC. The ATCC contains measured data points such as ‘control voltage’, ‘load current’ etc. A function can be developed that utilizes the measured values stored in the ATCC LN and calculates the control command for the tap position in the same LN. An application can be the implementation of a function or a combination of different functions in order to achieve a certain feature. The inputs for a function or application can come from more than one LNs depending on the requirements of the implemented logic.

‐ Communications: The IEC 61850 provides client-server based MMS communication as well as multicast based GOOSE protocol. GOOSE is intended for fast transfer of events in protection related applications. MMS is intended for file transfer or data read/write which are not time critical. IEC 61850 provides ‘Report Control Blocks’ (RCB) that are used to manage the reporting of data in MMS based schemes.

‐ Substation Configuration Language: IEC 61850 uses Substation Configuration Language (SCL), which is an XML based language to represent and save the substation configuration. SCL is a standardized format therefore, it facilitates configuration exchange between IEDs and configuration softwares. The process of configuring the IEDs and the system as a whole is known as IEC 61850 engineering process. The IEC 61850 engineering process will be introduced in the next section under the ‘IEC 61850 engineering process’.

To facilitate IEC 61850 based device (IED) development, IEC 61850 stacks has been developed which are commercially available. The IEC 61850 communication stack is incorporated into the communication layer. The IEC 61850 stack initiates its database with the configurations loaded from the SCL schema of the system (ICD file). Therefore, the IEC 61850 data / object models know what information to exchange between which LN’s and what communication configuration needs to be used for that purpose. Any microprocessor-based device that successfully integrates the IEC61850 standard for communication is considered as an IED. IEC 61850 facilitates interoperable data exchange between the IEDs. Therefore, this architecture can be adopted for distributed monitoring and control.

4.2 Overview of the IEC 61850 Engineering process This section explains different types of configuration files defined in the IEC 61850 standard and their role.

The purpose of the IEC-61850 engineering process is to develop interoperability while exchanging IEC 61850 configuration information ( i.e.- station configuration, communication

48

configuration, HMI/single line diagram configuration etc. ). The IEC 61850 engineering process describes how an IEC 61850 compatible system should be configured. The process covers from IED configuration to system level configuration. XML based substation configuration language (SCL) format is used to provide interoperability.

According to the IEC 61850 engineering process, the ICD (‘IED Capability Description’) file is extracted from device and they represent the device capability. IID files are partially configured ICD files. The SCD (‘Substation Configuration Description’) file is created by combining capabilities from the ICD file and SSD file. SCD file contains the whole communication scheme and how individual devices or components in the system interact with each other. Finally, CID files which are the breakdown of SCD file and contain complete configurations for individual devices are loaded to the devices. It should be noted that all these file use the same XML file format defined in IEC-61850. The format is called as SCL. The workflow is shown in Fig.-3.

In order to create and edit the configuration files for this project, we used ‘SCLForge’ software by Triangle Microworks.

Figure 23: The engineering dataflow for IEC 61850 [Ref: IEC 61850 Engineering Process by Schneider electric]

49

4.3 Proposed DDMS architecture with IEC 61850 Fig.-23 represents the basic DDMS architecture. As explained in previous chapter, the DDMS architecture has three layers – the Communication layer, the RTOS layer ( Broker/ scheduler and database) and the application layer. We need to revise the DDMS architecture shown in Fig. 23 to move towards the IEC61850 based architecture as shown in fig.-19 and fig.-20. In the IEC 61850 enabled scenario, the applications will use LNs to collect necessary information. The application will also update the ‘control’ data points of a logical node with the calculated output values. Typically, a database or hardware memory is used to save and synchronize with the most recent values for the LNs. Therefore, it creates two data synchronization layers. The top layer synchronizes between the application and the database. The bottom layer synchronizes between the device data and the database. But, the API at the top layer needs to be transparent so that the application developer uses API which are consistent to the services offered by the IEC 61850.

Figure 24: Proposed DDMS Architecture

50

Figure 25: Proposed DDMS architecture incorporating IEC61850 standard

In this architecture, the communication with an IED or a DDMS node is accomplished via the IEC 61850 stack. As figure-26 shows, IEC 61850 standard uses ethernet for data transfer from sensor to IED. A suitable Hardware or software adapter can be used for legacy devices and the devices that use other protocols such as Modbus. IEC 61850 uses OSI 7 layer model for mapping the communication model. The MMS is used at station level and uses the full OSI 7 layers. The GOOSE message is primarily used to convey protection related information and a GOOSE packet has a specific time limit to reach its destination. Therefore, the IEC 61850 standard directly maps the GOOSE packet to layer-2: logical Link Control(LLC) layer in the OSI model. Sampled values generated from sensors are also mapped to Layer -2 (LLC).

51

Figure 26: IEC 61850 communication mapping to OSI 7 layer model

Therefore, if the DDMS wants to receive information from a sensor, it needs to receive sampled values at the process bus level. On the other hand, most smart devices are IEC 61850 enabled devices. For example, most smart meters have the IEC 61850 stack, therefore they act as an IED and a DDMS can poll specific data from the meter via MMS. Report Control Blocks (RCBs) can be configured in order to synchronize the most recent data at the device with the DDMS. The RCB can be defined to synchronize with the DDMS at specific time interval. It can be event triggered in which case change of any device data point with trigger an update to the DDMS.

The communication layer of the fig. 25 is elaborated in the fig 27 and fig 28. If a device is IEC 61850 capable, then the DDMS can collect information from it using the built in schemes such as RCB (fig. 27 &28, left side). In case of a device that uses a different protocol than the IEC-61850 standard, there are two approaches. A typical approach is using a protocol adapter or converter at the DDMS node(fig. 27, left side). In this case, the DDMS needs to support a collection of such protocol converters to be able to speak with different devices. From architecture point of view, if the device is custom built and uses some other protocol such as Modbus to collect information from submodules, it is preferable for the same device to host an adapter such that the data are conveyed to the DDMS in a IEC-61850 standard compatible manner. ( fig. 28, right side ).

52

Figure 27: Communication between DDMS and an IEC 61850 ‘not’- enabled device (approach-1)

Figure 28: Communication to DDMS between an IEC-61850 capable and not capable device (approach-2, preferred)

53

4.4 Mapping DDMS functions to IEC 61850 logical nodes

According to the IEC 61850 standard, logical nodes are used to map a real device to an IEC 61850 virtual device. The functions are applications use the LNs to collect the input data and then update the LNs with the output data. LNs are the only approach if a data from one IED needs to be exchanged with another IED or device.

A DDMS hosts applications that need to use the status information from another DDMS. These information need to be modeled as data in LNs In order to enable the IEC 61850 integrated DDMS to exchange the information among themselves. This chapter examines the core functionalities and applications in the proposed DDMS to enable them to exchange information in IEC 61850 enabled environment.

4.5 Mapping Group management According to the requirements analysis of a distributed platform, the platform requires a group management (GM) function in order to perform co-ordination among peers in a distributed environment. The purpose of the group management functionality is to form a group among the live nodes in a network and to elect one of the nodes as a leader. In case the leader node becomes unavailable, the group management functionality elects another live node as the leader.

The group management functionality uses a ‘leader election’ algorithm in order to designate one of the online nodes as ‘leader’. The leader can be assigned specific tasks like maintaining the list of peer nodes in the network. ‘Leader election’ algorithms are established algorithms for distributed systems in computer science [18] [15].

A group management algorithm running in a distributed node needs to communicate two pieces of information to the other DDMS nodes available in the network – 1) If it’s the current leader of the group, 2) if it’s the leader of the group, then it needs to indicate if it’s live. The other members check this signal (sometimes called ‘heartbeat’ ) at regular interval to make sure that the group leader isn’t down. If the group leader becomes inactive ( e.g.- fails to update the heartbeat signal ) then the other DDMS nodes will initiate a leader election algorithm.

The client-server based MMS communication scheme in IEC 61850 standard allows one node to read the content of another LN. Therefore, an IEC 61850 LN containing the ‘leader’ status data is required. Since no logical node exists in IEC 61850 to accommodate this functionality, according to IEC 61850 guideline- a GGIO node is used to model the leader election functionality. As shown in Fig 6, a generic GGIO LN is a custom combination of

54

data attributes supported by IEC61850. ‘Leader’ LN is created to indicate the existence of a leader in a group.

Figure 29: A generic GGIO LN used for group management

Another data attribute ‘IntIn1’ is used as counter. The leader node increases the counter value at a certain interval to signal to the other nodes that the leader is live. The other nodes always check for change in the leaders counter. If the update stalls for a specific time, the other nodes assume that the leader node is down and starts the leader election algorithm to re-elect another leader.

Figure 30 GGIO logical node is used to facilitate group management

55

Therefore, in this portion, it is proposed that when an application is developed in an IEC 61850 environment and the application needs information residing in another node – the information that needs to be exchanged should be modeled as a LN so that the application can fetch the data using a IEC 61850 supported communication method.

4.6 Modeling the CES application: A CES management application has been developed in chapter 3 as a use case for the distributed intelligent platform. The fig. 31 shows a basic system with three communities represented by the blocks. Each community has residential load, PV generation and a CES device to serve the community. Each community ( also referred to as a ‘node’) is monitored and controlled by an instance of the CES management application. The nearby applications can collaborate and collectively make decision on how to dispatch the CES devices.

The distributed community energy storage (CES) management algorithm works in a scenario where there are multiple residential communities (also termed ‘community microgrids’) with high penetration of renewable energy resources. Each community is proposed to have a community energy storage (CES) device in order to counter the impacts of high penetration of renewables. Therefore, an energy management system is required for the CES devices in

Figure 31 An example system with three community microgrids

56

order to dispatch the devices such that the given objectives are fulfilled. In this scenario, the objectives are to –

A. Minimize the cost of energy consumption during peak hours B. Smoothing the generated intermittent PV power C. Limiting reverse power flow towards grid to a certain extent.

We need to consider the data that needs to be exchanged between the DDMS nodes for the CES application to work. Besides real time monitored load and PV generation data of the community, each CES application needs information on the capacity of the CES devices serving the node. These information includes –

‐ CES device parameters: Energy Capacity, Maximum power charge/discharge capacity.

‐ State of charge (SOC) limits of CES device.

The algorithm works in master-slave configuration during collaborative operation. If there are several CES applications in the network, then one of them is selected as Master. The others assume the slave role. The slave applications forward their monitored real-time PV generation and load data to the master application. The Master app. collects and stores the CES device parameters at the beginning of its operation. Based on the information, the master application calculates the expected dispatch for all the active CES devices. The expected dispatch values are then sent to the slave applications. The slave apps. cross-check the received values. If required, the slave apps can revise the expected values before dispatching the CES devices.

Figure-32 represents the real time information exchanged in a three node scenario. Here node 2 assumes the master role. Each application node monitors the aggregated load ( LDx.Tot_Kw) and aggregated PV generation (PVx. Tot_Kw). The CES device updates the application with its current SOC. The slave nodes ( node 1 and 3 ) forward the information to the master node (node-2). The mater application calculates the CESx_dispatch and forwards it to all the slave nodes. The slave nodes revise the value (if required) and implement it as CESx.P_dispatch.

57

Figure 32: Dataflow among application nodes in a three community microgrid based scenario.

Each community microgrid consists of residential load, PV sources and a community energy storage(CES) device.In this scenario, the application nodes monitor each community and collectively make decision on how to dispatch each CES device.

Figure-33 represents the modeling of the system in fig-31 and 32 with respect to the IEC 61850 architecture described earlier in the chapter (fig-17- 19). It is assumed that there is a meter to measure the aggregated PV generation of a community. According IEC61850 conventions, the meter is modeled as MMXU1 logical node (LN). Similarly, a meter MMXU2 is used to store and represent the measured aggregated Load of the community. The DESD is modeled as a controllable inverter, ZINV. A battery model is used to store the CES parameters.

Figure 33: Modeling of a node in IEC61850

Therefore, a basic IEC61850 model of a controller device for community microgrid is created. Due to the unavailability of a general purpose IEC-61850 compatible modeling software, the ‘SCLFroge’ software by TMW is used for modeling the system.

58

The overview of the model looks as follows (Fig 34, 35) -

Figure 34: The SCL model of a community microgrid

Figure 35: Details of a meter logical node (MMXU)

As seen is Figure-35, the data field – ‘ MMXU1.TotW.instMag.f ’ represents the real-time power (watts) measured by the corresponding meter. Similarly, the value at ‘ ZINV1.OutWset.setMag.f ‘ data field represents the setpoint command for the CES device.

In this part of the report, a modified DDMS architecture is proposed to accommodate the IEC 61850 stack. Related modules and applications were modeled in IEC 61850

59

4.7 Simulation of the proposed IEC 61850 enabled application

In the previous section, a DDMS architecture incorporating the IEC 61850 standard was proposed. Then the CES application was modeled to run in a decentralized environment. In this section, we use a simulation tool to show that the model can be used to run the decentralized CES application.

For simulation development, we used the DTM simulation software [19]. This software is used to simulate and test the communications between the hardware and software for substation automation. Since, it incorporates the IEC 61850 stack and provides scripting capability, DTM is used an IEC 61850 enabled platform which is used for simulating and testing the CES application from communication perspective. DTM is used to simulate a network of application nodes exchanging information.

The decentralized CES application is originally developed in C++. DTM uses Javascript as it’s native scripting language. Therefore, the C++ application was converted to a DLL library. A DLL is conceptually an executable file. The difference is that a function or code contained in a DLL library executes only when the function is called from another executable file. A DLL file can’t execute by itself.

In the setup described by the fig.-36, the DTM acts as the main program ( or executable). It supplies the DLL of CES management application with required input parameters and calls it from DTM environment. Then, the DLL executes and returns the required outputs (e.g.- dispatch results) to the DTM.

The DTM simulates the IEC-61850 in the sense that the information transferred from one node to another utilizes IEC 61850 standard protocols. The user can setup the environment to use either MMS, GOOSE or a mix according to the application requirement. In the specific simulation that will be described in next section, three computers are used to simulate three decentralized DDMS nodes.

It should be noted that in the following simulation, only one decentralized CES application is simulated in each node. Therefore, the traffic is calculated to show the communication network load the CES application generates. But, the process can be used to find the traffic generated by any application.

60

Figure 36: Simulation development in DTM environment

Simulation setup in DTM

A. Setup of the MMS scheme The ICD profile has been created during modeling the system. In DTM, A workspace is created where we instantiate the devices by loading their ICD profile. Then we have to add the device configurations ( i.e – network configurations) to the ICD file. Once configured, the ICD’s are called CID files.

The screenshot in figure 36 and 37 shows the workspace for the CES application simulation project.

In this setup, The master needs to collect the real time data ( PV and Load ) from all the other controller nodes. Therefore, IEC61850 clients are setup at all nodes so that it can collect the data from the other nodes acting as IEC 61850 Servers. Report Control Blocks (RCB) are created to update the necessary data points ( e.g. - ‘ MMXU1.TotW.instMag.f ’ and ‘MMXU2.TotW.instMag.f ’ ) in order to reduce the data transfer and subsequent network load. The setup is shown in Figure -38 .

It should be noted that when the simulation is scaled up - if all the servers are allowed to send data to the clients, it’ll create data traffic ( for ‘n’ nodes, factorial(n) RCBs will be generated) and majority of the traffic is unnecessary because only the master node required information from other nodes. Therefore, it is suggested that the application will enable only the master node RCB to listen to the clients. The other nodes’ clients/ RCB’s will be disabled in order to avoid traffic on the network.

The base simulation contains three nodes. The simulation can run on a single computer. But, due to the hardware arrangement, the MMS packets exchanged can’t be captured. Therefore,

61

the simulation was rearranged. The master node was put on a computer ( FRDMPWR2 ), the slave nodes were put on another computer ( POWERBARAN). Now, if we monitor the data traffic to and from the FRDMPWR2 computer, we can see the data exchanged from this node to the other nodes.

62

Figure 37: DTM Workspace setup

Figure 38 : Simulation host setup

B. Setup of the GOOSE based scheme In the GOOSE based scheme, goose control datasets are created which contain the name of the data points that will be multicasted using goose. On the other hand, each node needs to know which data point to subscribe to. These points are put under the ‘Inputs’ attribute. These points are highlighted in Fig 38. The workspace remains the same as shown in Fig 36 and 37, except that the ‘Report control blocks’ are not required. Now, once the CID file is loaded, the nodes will automatically start multicasting whenever one of the configured data points changes value. The nodes will also subscribe to the data points under the ‘Inputs’.

63

Figure 39: GOOSE model setup

C. Data setup In order to simulate the real time data in each node, the data sequence can be saved in a CSV file format. In the CSV file each row begins with the name of the datapoint followed by the data array. The time interval after which the next data sample is loaded can be configured in DTM.

Figure 40: Example CSV data file for node 1

A data file is created that consists of aggregated PV and load data for each node. The DTM loads the data to the corresponding data points at designated time interval.

64

D. Application setup The DTM supports scripting in native javascript. Since the core CES dispatch algorithm is developed in C++ , the code was converted to a DLL file. Then the DLL was wrapped using .NET framework so that the ‘CESControl1’ function can be called from javascript to calculate the current dispatch value for the CES devices.

DTM allows to use ‘insighttag’ which acts as an alias to the complete path to a location in the SCL file. At the beginning. the alias for all the PV and load data points are defines. The client paths are identified in order to update the CES command values vie MMS write calls. Then, in an infinite while loop, the PV and Load data are refreshed, then the ‘CESControl1’ function is called with required arguments in order to calculate the dispatch values for each microgrid. For the slave applications, an MMS write function is called to update the value of the data point which corresponds to CES dispatch command. Then the loop repeats itself.

Simulation output Fig.-40 shows a sample output in DTM for a 3 node system simulation –

11:08:32.056 : PV Watts = 0.8274 / 0.757629990577698 / 0.792829990386963 11:08:32.056 : Load Watts = 6.70872 / 7.63691997528076 / 7.76520013809204 11:08:32.056 : O/P for step(20) =1.37023/ 1.48302/ 0.403506 11:08:33.098 : Update value from SCL 11:08:33.628 : PV Watts = 0.8274 / 1.55519998073578 / 1.10399997234344 11:08:33.628 : Load Watts = 6.864 / 7.57460021972656 / 7.68860006332397 11:08:33.629 : O/P for step (21) =1.57773463320463 / -1.25568463 / 1.68761 11:08:34.677 : Update value from SCL 11:08:35.217 : PV Watts = 1.1084 / 1.55519998073578 / 1.10399997234344 11:08:35.217 : Load Watts = 6.864 / 7.57460021972656 / 7.61199998855591 11:08:35.218 : O/P for step (22) =1.84733463320463 / 1.55152 / -0.365794

Figure 41: Sample output from DTM

Network traffic In the MMS based scheme : 1. In this three node simulation, the meter data are updated every 1 second. Each update

generates 2 report control blocks (RCB) to the other nodes to synchronize the data. 2. The CES management algorithm calculated dispatch commands every 1 seconds. The

dispatch data is written as to the inverter LN as a MMS write command. Therefore two MMS writes are generated per cycle

65

In figure 41 and 42 , a network packet capture between 16:18:20.6 to 16:18:21.6 hour is highlighted. Since MMS is not a broadcast method, MMS packets can be captured only from the involved nodes. Therefore two ‘Wireshark’ packet sniffer programs were run in two computers to see the packets.

Wireshark is not fully IEC61850 compatible. It identifies identify RCB packets as ‘unconfirmed’ MMS packets. it identifies MMS write operation as either ‘write’ or ‘confirmed request’

Within the one second timeframe, 6 MMS RCB data packets were exchanged between the three nodes. Therefore, total network traffic calculation is shown in Figure 44. The MMS write occurs only when a dispatch command is calculated. These 430 bytes are generated at each cycle. The rest of the 1577 bytes are generated due to the real time data update.

Figure 42: MMS packets captured in the network ( at 152.14.125.29/FRDMPWR2 computer )

Figure 43: MMS packets captured in the network ( at 152.14.125.74/POWERBARAN computer ). (It shows the MMS packets exchanged by 152.14.125.74 with 152.14.125.24 and 152.14.125.29 computers.)

66

Figure 44: Capture of a MMS packet in network.

Figure 45: Network traffic calculation.

From the chart we can extrapolate an equation for traffic in bytes per second –

Traffic ( in kbs, for a ‘n’ node system) = ∗ ∗ Bytes * ( number of updates per second) * ( 8 /1024 )

Therefore, for a feeder or a system with 50 DDMS nodes, it requires about 5 Mb data exchange per iteration or update of the CES application. ( number of updates per second=1 ). However, it should be mentioned that the calculations above are based on a good network condition. Therefore, it excludes any additional packet re-transmissions due to poor network

source ip destination ip bytes

74 29 262

74 24 263

29 74 263

29 24 263

24 29 263

24 74 263

29 74 132 Write

83

29 24 132 Write

83

2007 bytes/ update

15.68 Kb/ update

total Network

traffic

67

quality or any failure. For a bad network conditions, an additional probabilistic margin should be added.

Network traffic in the GOOSE based scheme

The same scenario used in the MMS scheme evaluation was updated to use GOOSE scheme. In GOOSE scheme, a GOOSE Dataset was defined. Whenever there’s a change in any of the values in the defined GOOSE dataset, The updated value will be multicasted in the network so that all the other nodes have updated information. GOOSE mechanism retransmits the data a few times based on the user defined parameter.

In this scheme we still require two MMS writes in order to write back the CES dispatch command back to the corresponding nodes.

In the simulation, the iteration time is set to 1 sec. Therefore, the data updates every second and the CES dispatch algorithm runs and generates CES control commands once every second. It can be observed in the fig. 45 below, which is a screenshot from live packets captured while testing the GOOSE scheme.

Figure 46: Packet capture in the network while using the GOOSE scheme.

68

As seen in the Fig 45, an iteration begins at 20:09:18.633 hrs.( after the highlighted MMS packet numbered-170). Lets consider only the second resolution. So, the iteration begins at 18.633 sec. Then there are 22 GOOSE packets transmissions and two MMS packets before the next iteration begins.

Figure 47: Network statistics generated by wireshark from a 117 second run of the GOOSE scheme.

A network capture of DTM run shows an estimated network traffic of 3126 Bytes/sec or 24.4 kbps(Fig.-46). The information can be used to derive functions to extrapolate the data traffic for higher number of nodes as follows. GOOSE scheme sends the same packet multiple times since the publisher doesn’t know if the subscriber is going to receive the packet. The network traffic depends on this parameter as well. An approximate traffic equation can be derived for each iteration –

Traffic ( in kb, for a ‘n’ node system ) = ( RT ) * ( average GOOSE packet size ) * … ( number of publishing nodes) + ( n-1) * ( MMS write packet size) Here, RT = number of retransmissions ( a settable parameter of GOOSE protocol)

for the simulation case, RT =7 , average GOOSE packet size = 130 Bytes , MMS write packet size = 215 Bytes.

Traffic ( in kb, for a ‘50’ node system ) = ( (7) * (130) * ( 50 ) + ( 49) * ( 215) ) * (8/1024) =437.7 kb (per update or iteration of the CES application)

Therefore, in this chapter, the approach to integrating IEC 61850 to a distributed platform has been outlined. The approach to modeling distributed applications for IEC 61850 enabled environment has been discussed and demonstrated via DTM based simulation.

69

When the MMS scheme is compared to the GOOSE scheme, it shows that traffic wise, GOOSE traffic is lower when a significant number of nodes interact with each other. There’s another consideration that a large number of client-server read/write interactions may overwhelm the master node in a MMS based scenario. On the other hand, GOOSE was initially developed to be a fast and local communication method. Therefore, if the GOOSE data packets need to travel outside the boundary of a router, a virtual LAN with VLAN tagging needs to be enabled in the router. Otherwise, the router will not allow the GOOSE packets to pass through it.

Therefore, in this chapter, an IEC- 61850 standard based communication approach is proposed for the DDMS platform. The proposed approach is modeled and implemented in a decentralized environment and the expected communication load on the network in estimated.

70

Chapter 5: Utilizing CIM for exchanging topology information

The DDMS platform is meant to be deployed it the field where each DDMS node is in control of a community microgrid node. The DDMS platform will perform mostly in grid-connected mode in co-operation with other DDMS nodes. In this dissertation, a community energy storage management application is presented as an example. Each DDMS platform is capable of hosting multiple applications. Some applications such as distributed volt-var control application or state estimator require topology specific information such as impedance and connection between nearby nodes. One approach to collecting the data is to collect the information from a central source such as the substation computer or the control center. Let’s assume that a substation computer has the model and the associated parameters of the feeders (fig-47) . A DDMS node in charge of a certain portion of the feeder can query the substation computer for information related to it’s area (e.g.- line impedance) required by a certain application. The exchange of information between the DDMS node and the substation should be in a standardized format in order to ensure interoperability and CIM standard is proposed to be used for this purpose.

An introduction to CIM standard is provided in the next section in order to provide a context to the utility of the standard. The objective is that if the DDMS receives a file in CIM format, it can parse the information and get the data required for the application.

Figure 48: Exchanging Topology information using CIM standard

71

The CIM standard uses XML based format for exchanging information. Therefore, once the DDMS node receives an XML file containing the topology and parameters of the area under the substation, the DDMS needs to parse it to collect the information for the specified area.

Introduction to CIM The CIM standard for power systems is a group of standards aimed towards interoperability for information exchange between Energy Management System(EMS) applications. It stems from an EPRI led project Control Center Application Program Interface(CCAPI) in the 1990. The realization of the project led to defining a structure for power system modeling which is the Common information model (CIM). CIM provides an XML based standardized format for sharing information among power system applications. CIM can be described as an object-oriented approach to modeling the relevant objects and their relations in the scope of electric distribution, transmission and generation. The main use case for the CIM standard is to exchange power system topology data.

The CIM standard broadly consists of IEC 61970, IEC 61968 and IEC 62325 standards. The IEC 61970 contains the core modeling approach and the abstract models to represent the the objects used by an EMS. Therefore, the base model is defined in IEC 61970-301 and it is extended for distribution management(i.e.- asset management, meter reading etc. ) by IEC 61968-11 and for market communication by IEC 62325-301.

The CIM is utilizes Unified Modeling Language (UML) to describe the models. CIM uses packages to group related model elements (fig – 48). The main IEC 61970 CIM packages are shown in the figure- 47. For example - the Core package contains basic entities that are shared by all applications. The Topology package is an extension of the Core package. It is used to model connectivity in association with the Terminal class. Similarly, the Wires package is an extension of Core and Topology packages. it models information on electrical characteristics of transmission and distribution networks. The Domain package is a dictionary of quantities and units that may be used by any class in any other package.

72

Figure 49: IEC 61970 base packages in UML

The requirements for exchanging topology information in CIM standard

The following flowchart (fig-49) shows the process of deriving a CIM/XML file from power system data. This process takes place at the DMS ( distribution management system). The resultant CIM/XML file is then transferred to the DDMS node.

73

Figure 50: Exporting data from UML to CIM-XML format

The steps can be described as –

1) A CIM profile has to be created that represents the system topology and any other additional information ( e.g.- GIS ). A CIM profile contains only the essential packages, classes and associations. In fig-49, the CIM RDF schema represents the profile which is a combination of the CIM UML model and RDF.

2) An algorithm or method is required to fill the CIM topology profile with corresponding power system data.

3) The output CIM/XML file needs to be validated for semantic, syntactic and overall completeness.

4) These steps take place within the dashed box boundary of the fig.-49 which represents the power system DMS.

In order to transfer the resulting CIM/XML file, two approaches can be used –

1) Exchange the complete topology at certain interval (e.g.- using FTP like service). 2) Exchange parts of the topology. In this approach only the change or difference is

transmitted.

This chapter outlines a standard approach to transfer power system topological data from substation to DDMS nodes. The DDMS then extracts the necessary information from the CIM/XML file and updates it’s database. Then an application can use the information as needed.

74

Application of CIM standard for the DDMS Due to lack of an open source CIM XML parser in C++, we assumed that a CIM-XML parser is available that can parse a received CIM-XML file and store the parsed information in a database. A DDMS application can fetch the information from the database to utilize it. In order to demonstrate the concept in DDMS, JSON is used as a simple database structure. Here, the idea is if we have a XML parser, it’ll save the required information in the JSON based database which the DDMS can recognize and use.

For example, let us consider a 5 bus test feeder (fig-50 ). For simplicity, it is assumed that the application under consideration requires the inputs to solve a power flow. The inputs therefore require the line impedances of the system and the power supplied by the source. These required information can be read from the CIM-XML file of the system and a DDMS node can find the impedance of a particular line.

Figure 51: A 5-bus feeder

If the CIM-XML file for the example 5 bus system is parsed, the following information is found in the XML file. An CIM-XML editor, CIMphony, is used to illustrate the information contained in a CIM-XML file. The right side of fig-51a shows the nodes of the system. Fig -51b shows the details of one of the nodes. Fig-52 and 53 shows the line information on the segments and spot loads in the system. Fig-54 shows the detailed information of one of the spot loads. Fig – 55 shows the details of the Source model and the information available in the XML file on the source node. The next section outlines an example on how to find the line impedance information from the CIM-XML file.

75

Figure 52a: ConnectivityNode element shows the nodes of the system

Figure 51b: Detail information of the Node 650

76

Figure 53: DistributionLineSegment class structure contains the information on the lines

Figure 54: The spot loads are contained under EnergyConsumer Class

77

Figure 55: Detailed information of a member of the EnergyConsumer class

78

Figure 56: Details of the Source node in contained in the EnergySource class

How to find the line impedance: Let’s assume that an application requires to know the line impedance of each lines in the 5-bus example system. In the CIM-XML file, each DistributionLineSegment class contains the length of the wire(fig-52 and fig 56). The class can directly contain the impedance information of the line or it may have information on the type of wire. If it contains the type of wire, then it’ll refer to PerLengthPhaseImdedance class which refers to PhaseImpedanceData class that contains the details of the impedance with the relevant units. Figure 56 to 58 illustrates the process.

Therefore a DDMS application can be developed which can traverse through a CIM-XML file structure and find the corresponding information. Then the application can update the DDMS database with the information. An application that requires the information can access the database to collect the data. Other information can be collected from the CIM-XML file in a similar manner.

79

Figure 57: Members of a DistributionLineSegment class

Figure 58: PerLengthPhaseImdedance class members

80

Figure 59: The impedance value of the line (per unit length)

Demonstration of the utilization of CIM-XML based topology information

It is proposed that the DDMS platform will receive the topology information from a control center DMS in CIM-XML format. Since the CIM-XML contains a detailed modeling of the feeder and associated equipments, there an XML parser is required that is capable of parsing the CIM-XML file in accordance with the CIM modeling approach. The CIM-XML format extensively uses RDF(Resource Description Framework) to make cross reference to the related equipments within an XML file.

An open source C++ based CIM-XML parser couldn’t be found. Therefore, a python based CIM-XML parser ‘PyCIM’ is used to parse a CIM-XML file and transfer the data to the cooresponding DDMS application. A simple distribution load-flow program is developed to demonstrate how the information obtained from the CIM-XML can be used by a DDMS application.

The following figure-59 - 61 show the topology information read by the load-flow application from a CIM-XML file corresponding to the 5 bus system in fig - 50. For example, in fig- 60, the load information at node 671 is printed as [“Name”, “P”, “Q”, “Phase”]= [ “671”, “1156.0” , “660”, “ABC”]. Which means the load is connected to node -671. The connected load is (1155kW, 660kVAR) and it’s a tree phase load. The information is extracted from the CIM-XML formatted information in the table -2 below.

81

Table 2: Portion of CIM-XML file for Load at node - 671

<cim:EnergyConsumer rdf:ID="_BAE8B009-5BA9-418A-AEE1-F6A4B4970653"> <cim:IdentifiedObject.name>671</cim:IdentifiedObject.name> <cim:ConductingEquipment.phases rdf:resource="http://iec.ch/TC57/2009/CIM-schema-cim14#PhaseCode.ABC"/> <cim:EnergyConsumer.LoadResponse rdf:resource="#_DB5176AE-B259-4663-BEB0-FC71DB66AED2"/> <cim:EnergyConsumer.customerCount>1</cim:EnergyConsumer.customerCount> <cim:EnergyConsumer.pfixed>1155</cim:EnergyConsumer.pfixed> <cim:EnergyConsumer.qfixed>660</cim:EnergyConsumer.qfixed> </cim:EnergyConsumer>

Figure 60: Extracting line information from a CIM-XML file

82

Figure 61: Extracting Source and Load information from CIM-XML file

Figure 62: An Example load-flow application utilizing the information from CIM-XML.

Therefore, The distribution load-flow application running on the DDMS extracted the topology ( connection and impedance), source information ( source voltage and impedance) and load information in order to perform a load-flow.

It should be noted here that, relating to the discussion in the section 5.2, the CIM-XML file is better suited for slow-changing data such as system topology. Works are going on to accommodate fast changing data in the CIM approach. Currently, it may not be the best means for conveying real time data. But, the information contained in the CIM file is sufficient to setup the database for the real time data.

Therefore, in this chapter, CIM is examined as an standard approach to transfer topological information from the DMS to the DDMS node. It is demonstrated that an application can

83

parse and use the data to perform algorithmic calculation. On the other hand, due to lack of open source resources in C/C++, a significant effort is required to develop the tools and libraries involving CIM file parsing.

84

Chapter 6: Conclusion and Future work

Proposed modification to the DDMS architecture :

In this dissertation, the requirements for a decentralized platform has been analyzed. An approach to developing co-operative decentralized algorithms have been discussed. A decentralized CES management algorithm has been developed using the approach. The utilization of IEC 61850 and CIM standards has been investigated for standardizing communication between decentralized platforms.

To review, the initial DDMS architecture is shown in fig – 62. One of the implementation disconnects is that – ideally, a database should decouple the communication layer from the application layer. But, sometimes, user developed applications tend to use an alternate database or bypass the database to directly poll the device for data. A well defined API for using the database and collecting device data can get rid of this issue.

Figure 63: Initial DDMS architecture

85

The fig.-63 shows the proposed final DDMS architecture. It incorporates the proposed IEC-61850 and CIM standards with the initial architecture.

Figure 64: Proposed DDMS architecture

There should be a standard API for the applications to access the device data. The applications should ideally access the data via a database which provides decoupling between Application layer and communication layer. The IEC smart grid standardization roadmap (June 2010) [20] outlines the relations between different communication standards, protocols and portrays how the communication requirements at different levels of the power systems industry can be standardized (fig-65). The literature from the TC 57 workgroups[20] and related to smart grid development requirements [21] [22] were reviewed in order to find any recommendations on the database structure or implementation. It appears that TC 57 workgroups do not have any recommendation on databases for IEC 61850 or CIM systems and the topic of database requirements is outside the scope of their discussion.

86

Figure 64: TC 57 reference architecture

Some of the future works regarding DDMS platform development would be –

1) A roadmap to integrating IEC 61850 to a DDMS has been outlined in the dissertation. IEC 61850 has been originally developed for static environment such as substations. Contrary to that, decentralized environment is a dynamic environment where a node may go offline any moment and another new node may join the network. IEC 61850 doesn’t have any built in mechanism to handle the dynamic environment.

2) A C/C++ based CIM-XML parser is required in order to use the CIM standard for topological data transfer. A python based parser is used on an ad-hoc basis. The parser requires the knowledge of CIM model structure which is pretty extensive. Different parser options has been considered. A XQuery based parser can be used an approach to extract information from an XML file. From initial survey, it appears that exisiting XQuery libraries such as XQilla can be used in embedded environment.

87

The database requirement analysis for a DDMS system needs to be addressed. One of the immediate needs for a database is for storing the local device data and data extracted from CIM-XML files. For storing extracted data from CIM-XML file, a property tree based structure can be used. This is an approach to store and extract nested information.

3) One of the objective of this work is to evaluate the process of development of decentralized application that co-operate with each other. During literature review, it has been noted that industrial process control systems extensively make use of decentralized and distributed control. Specific standards such as IEC 61499 (which is based on it’s predecessor - IEC 61131 standard) are used for development and deployment of distributed control. [23]. One of the efforts in the IEC 61499 standard is to create an open platform and editable Function Blocks(FB). An user can program a function block or customize an existing function block. Once the configurations are set in the configuration software, the FBs can be automatically deployed to the corresponding hardware if the hardware supports IEC 61499 stack. This can be further investigated as an approach to dynamically deploy decentralized applications to DDMS. Generic FBs can be developed such that the FBs can be connected in a graphical interface to accomplish a task (similar to working with MATLAB Simulink or PSCAD graphical user interface ) . The IEC 61499 standard adoption is still in early phase. Therefore, an imminent issue will be the unavailability of the communication interface of the FBs to the target hardware. The developer needs to accomplish this task before starting the decentralized function block development. As mentioned, this task is hardware specific so moving applications from one hardware platform to another (e.g.- from ARM based boards to beaglebone boards) may not be simple. Yet the idea of developing open and automatically deployable FBs is an interesting topic that can be further investigated.

4) A method for transferring a CIM-XML file from the DMS to the DDMS node needs to be implemented. A typical approach involves transferring the whole file using ftp like services. Another approach is to transmit the CIM-XML file once and then transmit the incremental changes. This approach requires less bandwidth, and therefore, is faster.

88

REFERENCES

[1] "CES Unit Specifications," [Online]. Available: http://www.dolantechcenter.com/Focus/DistributedEnergy/docs/CESUnitSpecifications_rev2_2.pdf . [Accessed 18 7 2014].

[2] "Distributed Energy Storage," [Online]. Available: http://www.dolantechcenter.com/Focus/DistributedEnergy/EnergyStorage.aspx.

[3] E. J.H.R., "Network impacts of high penetration of photovoltaic solar power systems," Power and Energy Society General Meeting, 2010 IEEE , vol., no., pp.1-5, 25-29 July 2010., no. doi: 10.1109/PES.2010.5589675 .

[4] F. Kateri. J. R. Agüero, "Solar PV Integration Challenges," Vols. Power and Energy Magazine, IEEE , vol.9, no.3, pp.62-71, May-June 2011 .

[5] Distribution System Voltage Performance Analysis for High-Penetration Photovoltaics Y.Liu.

[6] D. Evans, S. Hendley, A. Crain and J. S. Camilleri, " Next Generation Automation – Effective Platform Design and Practical Implementation," in Grid Interop, 2011.

[7] "GreenBus® Microgrid Solution," [Online]. Available: http://www.greenenergycorp.com/solutions/green-bus-software-platform/.

[8] "AMQP Java Broker," [Online]. Available: http://qpid.apache.org/releases/qpid-0.20/java-broker/book/. [Accessed 14 7 2014].

[9] "AMQP," [Online]. Available: http://www.amqp.org/about/what. [Accessed 9 7 2014].

[10] J. R. Davidson, "Architecture Power Systems Control Architecture," 2005. [Online]. Available: http://www.inl.gov/technicalpublications/Documents/3382953.pdf.

[11] [Online]. Available: http://cassandra.apache.org/. [Accessed 9 7 2014].

[12] "http://wiki.apache.org/cassandra/," [Online]. Available: http://wiki.apache.org/cassandra/. [Accessed 9 7 2014].

[13] R. Akella, F. Meng, D. Ditch, B. McMillin and M. Crow, ""Distributed Power Balancing for the FREEDM System,"," Smart Grid Communications (SmartGridComm), vol., no., pp.7-12, 4-6 Oct. 2010.

89

[14] A. Chuang and M. McGranaghan., "Functions of a local controller to coordinate distributed resources in a smart grid," in Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century, 2008.

[15] G. e. a. Coulouris, Distributed Systems: Concepts and Design (5th Edition), Addison-Wesley, 2011.

[16] K. Mani Chandy and Leslie Lamport, "Distributed snapshots: determining global states of distributed systems," ACM Trans. Comput. Syst., 1985.

[17] R. Mackiewicz, "Overview of IEC 61850 and Benefits ,," in Power Systems Conference and Exposition, 2006, 2006.

[18] H. Garcia-Molina, "Elections in a Distributed Computing System," in IEEE Transactions on Computers, Jan 1982.

[19] Triangle Microworks, "Distributed Test Manager (DTM)," [Online]. Available: http://www.trianglemicroworks.com/products/testing-and-configuration-tools/dtm-pages/overview. [Accessed 11 2015].

[20] SMB Smart Grid Strategic Group (SG3), "Roadmap, IEC Smart Grid Standardization.," June 2010.

[21] S. e. a. Rohjans, "Survey of smart grid standardization studies and recommendations," Smart grid communications (SmartGridComm), 2010.

[22] V. Gungor, D. Sahin, T. Kocak, S. Ergut, C. Buccella, C. Cecati and G. Hancke, "A survey on smart grid potential applications and communication requirements," Industrial Informatics, IEEE Transactions on, 2013.

[23] Vyatkin, Valeriy; and Instrument Society of America, IEC 61499 function blocks for embedded and distributed control systems design., 2007.

[24] "Protocol Buffers project site by Google.," [Online]. Available: https://code.google.com/p/protobuf/.

[25] R. T. Fielding, " Architectural styles and the design of network-based software architectures.," 2000..

90

[26] F. Mattern, "Efficient Algorithms for Distributed Snapshots and Global Virtual Time Approximation," Journal of Parallel and Distributed Computing, pp. 423 - 434 , Vol. 18, No. 4, 1993.

[27] T. MicroWorks, "IEC 61850 Source Code Library," [Online]. Available: http://www.trianglemicroworks.com/products/source-code-libraries/iec-61850-scl-pages. [Accessed 2 9 2014].

[28] S. Mohagheghi, J.-C. Tournier, J. Stoupis, L. Guise, T. Coste, C. A. Andersen and J. Dall, "Applications of IEC 61850 in distribution automation," Power Systems Conference and Exposition (PSCE), Vols. vol., no., pp.1,9,, 2011.

[29] "IEC 61850- Communication Networks and Systems in Substations, Part 1: Introduction and Overview," 2003. [Online].

[30] M. M. J. S. Z. W. S. Mohagheghi, "Modeling Distribution Automation System Components Using IEC 61850," Proc. IEEE PES General Meeting, 2009.

[31] R. Mackiewicz, " Overview of IEC 61850 and benefits," Power Engineering Society General Meeting, no. doi: 10.1109/PES.2006.1709546.

[32] C. A. G. Zhang Jianqing, "IEC 61850-Communication Networks and Systems in Substations: An Overview of Computer Science," University of Illinois at Urbana-Champaign (2007)..

[33] Y. &. C. R. H. Liang, "Understanding and Simulating the IEC 61850 Standard.," 2008.

[34] ABB, "Substation automation migration strategies to IEC 61850 using Relion protection and control," 2011.

[35] C. R. Ozansoy, "Design & Implementation of a Universal Communications Processor for Substation Integration, Automation and Protection.," Victoria University, 2006.

91

APPENDICES

92

Appendix -A

Example of a complete program to poll a measurememnt by name.[ ] The files "org.totalgrid.reef.amqp.cfg", "org.totalgrid.reef.user.cfg",

"org.totalgrid.reef.node.cfg" provide AMQP configuration, user identification and node authentication parameters respectively.

package org.totalgrid.reef.examples.measurements; import org.totalgrid.reef.client.factory.ReefConnectionFactory; import org.totalgrid.reef.client.service.list.ReefServices; import org.totalgrid.reef.client.service.MeasurementService; import org.totalgrid.reef.client.service.PointService; import org.totalgrid.reef.client.Client; import org.totalgrid.reef.client.Connection; import org.totalgrid.reef.client.ConnectionFactory; import org.totalgrid.reef.client.exception.ReefServiceException; import org.totalgrid.reef.client.settings.AmqpSettings; import org.totalgrid.reef.client.settings.UserSettings; import org.totalgrid.reef.client.service.proto.Measurements.Measurement; import org.totalgrid.reef.client.service.proto.Model.Point; import java.util.Date; import java.util.List; public class MeasurementsExample { public static void main(String[] args) throws Exception { if(args.length == 0){ args = new String[]{"org.totalgrid.reef.amqp.cfg", "org.totalgrid.reef.user.cfg", "org.totalgrid.reef.node.cfg"}; } Properties properties = PropertyReader.readFromFiles(Arrays.asList(args)); // Load broker settings from config file AmqpSettings amqp = new AmqpSettings(properties); // Load user settings (login credentials) from config file UserSettings user = new UserSettings(properties); // Create a ConnectionFactory by passing the broker settings. The ConnectionFactory is // used to create a Connection to the Reef server ConnectionFactory connectionFactory = ReefConnectionFactory.buildFactory(amqp, new ReefServices()); // Prepare a Connection reference so it can be cleaned up in case of an error Connection connection = null; int result = 0;

93

try { // Connect to the Reef server, may fail if can't connect connection = connectionFactory.connect(); // Login with the user credentials Client client = connection.login(user); // Application code or method call getMeasurementByName(client); // the method below gets a measurement } catch(ReefServiceException rse) { // Handle ReefServiceException, potentially caused by connection, login, or service request errors System.out.println("Reef service error: " + rse.getMessage() + ". check that Reef server is running."); rse.printStackTrace(); result = -1; } finally { if(connection != null) { // Disconnect the Connection object, removes clients and subscriptions connection.disconnect(); } // Terminate the ConnectionFactory to close threading objects connectionFactory.terminate(); } System.exit(result); } } //END main /***************************************** * Method: Get Measurement by Name */ public static void getMeasurementByName(Client client) throws ReefServiceException { // Get service interface for points PointService pointService = client.getService(PointService.class); // Select a specific point, get its point name

94

// Or a name can be provided if known ; like pointName = “CES_demo.CES1.kW" String pointName = pointService.getPoints().get(0).getName(); // Get service interface for measurements MeasurementService measurementService = client.getService(MeasurementService.class); // Get latest measurement for the point by name Measurement measurement = measurementService.getMeasurementByName(pointName); // Display measurement properties System.out.println("Found Measurement by name: \n" + measurement); } }

95

Appendix -B

Technologies applied to implement the service oriented architecture (SOA) architecture [6]:

Service Oriented Architecture (SOA) is a specific type of software design architecture. The key benefits of SOA is modularity, loose coupling and re-usability of services, reduced development and maintenance effort, flexibility of software development and deployment environment. SOA is useful for developing large software projects since the property of loose coupling between services allows different teams to work in parallel while adhering to the agreed upon service interface.

In SOA, the services can be placed anywhere in the network provided that the services are able to pass message to each other using the network.

The following three technologies are used to factor in and implement SOA concepts in the GreenBus platform-

Protocol buffers ( ‘Protobuf’ ) AMQP Representational State Transfer (REST) / RESTful architecture

Protocol buffer ( ‘Protobuf’ [3] ) is developed by google to serialize structured data. [24] Protocol buffers are required to serialize the service messages. Protocol buffers together with AMQP defines the GreenBus interface for the service layer according to SOA.

Representational State Transfer (REST) is an architectural style for development of distributed systems as well as web services. It is a way to create, read, update or delete information on a server using simple HTTP calls [25]. A request from any client contains all the necessary information ( i.e.- self-descriptive) to service the request. Factoring a distributed system in a RESTful style provides the following advantages:

-Uniform interface : Service consumers/providers communicate through a uniform interface (i.e. GET, PUT, POST, DELETE) which decouples client server development and supports service oriented architecture.

-Opaque transport - Clients are unaware of whether they are directly connected to the end service or a proxy. Proxies can be used for things like enforcing security.

-Stateless - Every request contains all of the information necessary to satisfy the request. Any server-side state must be addressable in some way.

96

The AMQP broker structure, Protobuf and RESTful architecture are technologies used to implement SOA in the GreenBus platform that support the scalability of the system.

--------------------------------------

Dissertation-v2-Decentralized Monitoring and Control of ...

Documents

Transcript of Dissertation-v2-Decentralized Monitoring and Control of ...