Smart Grid Communications

Presented by Solveig Ward

Smart Grid Vision

Slide I 2

Communications Requirements

Slide I 3 DOE Smart Grid Communications Requirements 2010

4

1920 1940 1960 1980 2000 2020

Extent of

Deployment

Manual network era Analog automatic network era Digital network era Data networking era

Manual

switching

2-motion

selector

switching

Semi-

electronic

switching

Digital

switching

Packet

based

switching

TDM Ethernet

Evolution of Telecommunications

Time Division Multiplexing

Slide I 5

Synchronous communications

Time division multiplexing

1 2 3 …………… 24

64 Kbps

64 Kbps

.

.

.

.

.

.

.

.

TDM

MUX

64 Kbps

64 Kbps

.

.

.

.

.

.

.

.

TDM

MUX

1

24

1

24

......................1 24

1.544 Mbps

AGGREGATE

64 Kbps

64 Kbps

.

.

.

.

.

.

.

.

TDM

MUX

64 Kbps

64 Kbps

.

.

.

.

.

.

.

.

TDM

MUX

1

24

1

24

......................1 24

1.544 Mbps

AGGREGATE

TDM Networking

Slide I 6

Conventional substation

communications

Slide I 7

Requirements

Slide I 8

Data Voice Protection

communications

Delay (latency)

tolerance*

High Moderate/Low (100 ms) Very low

(<20 ms)

Jitter (variation in

delay) tolerance*

High Moderate Very low

Stream/burst

transmission

Bursts Stream Stream

Error tolerance Low High Very low

Packet/data loss

tolerance

Moderate, by the

application requesting

retransmission

Some data loss is

acceptable until voice

quality becomes too low

No

Interruption

tolerance

Yes, by the application

requesting

retransmission

Moderate (0.5 sec)

None/very low

Protocol standard Proprietary/

standardized Standardized Proprietary

TDM versus Ethernet

Slide I 9

Protective Relaying Communications

Requirements Ethernet Communications Characteristics

Low bandwidth (< 100 kb/s)

High bandwidth (>100 Mb/s)

Fixed, predictable latency with little

wander

Variable unpredictable latency with

significant wander

Continuous, non-burst oriented

Burst (packet) oriented

Ethernet

Slide I 10

Ethernet packets

Mesh network

Why Ethernet?

• Cost!

– 1/10/100 Gbps over the same fiber

– Plug-and-play – no configuration required

– More efficient bandwidth utilization

• One platform that carries all type of traffic; voice, data

• Concerns:

– Real-time data; latency

– Protection channels – unpredictable and variable end-to-end

delays

Slide I 11

The solution - MPLS

• Ethernet is a connectionless communications system by

design

• MPLS transforms Ethernet into a connection-oriented

communications system

• Promising QoS similar to TDM systems

Slide I 12

MPLS Benefits

• Is expected to meet latency requirements for real-time

data

– Teleprotection trial by Alcatel Lucent: 15 ms end-to-end delays

• MPLS allows provisioning and management of VPNs

• Cyber Security: In the absence of mis‐configuration or

deliberate interconnection of different VPNs, it is not

possible for systems in one VPN to gain access to

systems in another VPN

– MPLS routers / applications providing encryption are becoming

available

Slide I 13

MPLS “Multi-Protocol Label Switching”

• Layer 2: Ethernet which can carry IP packets, but only over simple LANs or point-to-point WANs

• Layer 3: Internet-wide addressing and routing using IP protocols

• MPLS sits between these traditional layers, providing additional features for the transport of data across the network (Layer 2.5)

Slide I 14

Label Switching

• In a traditional IP network: – Each router performs an IP lookup (“routing”), determines a next-hop based

on its routing table, and forwards the packet to that next-hop

– Rinse and repeat for every router, each making its own independent routing decisions, until the final destination is reached

• MPLS does “label switching” instead: – The first device does a routing lookup, just like before, but instead of finding

a next-hop, it finds the final destination router and it finds a pre-determined path from “here” to that final router

– The router applies a “label” based on this information

– Next router use the label to route the traffic without needing to perform any additional IP lookups

– At the final destination router the label is removed and the packet is delivered via normal IP routing

Slide I 15

Architecture

Slide I 16

MPLS Basics

• Customer Edge

(CE) router –

managed by

customer

• Provider Edge

(PE) router –

managed by

service provider

• Provider (P) router

– managed by

service provider

Slide I 17

Combined CE/PE

Slide I 18

QoS per VRF

• VRF = virtual route forwarding; tables created and used to create traffic separation

• Implementing QoS guarantees complete control of resources (bandwidth, priority, and so

on)

• Implementing QoS allows “peaceful coexistence” of several traffic types (network

management, physical security management) with mission‐critical traffic (SCADA, PMU,

GOOSE)

Slide I 19

MPLS VPN

• By properly provisioning one physical MPLS VPN capable

infrastructure, several network “overlays” are possible

Slide I 20

SCADA

PMU

GOOSE

Communications considerations

• Bandwidth (cost)

• Latency (end-to-end delays)

• Latency Control and QoS

• Reliability (cost – redundancy)

• Monitoring

• Management

• Security

• What is the availability requirements for the application

– If data are lost, what are the consequences?

Slide I 21

Bandwidth

Slide I 22

Latency versus utilization

Slide I 23

Latency

Slide I 24

Traffic segmentation

Slide I 25

Special Protection Scheme Example

Slide I 26

PMU

Conclusions

• Properly engineered MPLS promises to provide QoS suitable

for real-time data

– Testing is needed

• Base communications requirements on application needs

– Do not send more data or more often than the application will use

– High availability (=redundancy) may not be needed in all parts of the

network and at all locations

– Low latency may not be needed for all types of data

• Engineer the telecom network to provide the best

performance / cost ratio

Slide I 27

28

29