Evolution of Network Synchronization Technologies

64
Evolution of Network Synchronization Technologies Hydro-Quebec Symposium May 16, 2016 Chuck Perry [email protected]

Transcript of Evolution of Network Synchronization Technologies

Page 1: Evolution of Network Synchronization Technologies

Evolution of NetworkSynchronization Technologies

Hydro-Quebec SymposiumMay 16, 2016

Chuck [email protected]

Page 2: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.2 © 2015 ADVA Optical Networking. All rights reserved. Confidential.2

Oscilloquartz at a Glance

• Member of the ADVA Optical Networking Group

• Focused offerings for communications, government and enterprise sync applications

• Longstanding relationship with customers worldwide • Around 100 sync focused partners in about 80 countries around the

globe

• State-of-the-art time and frequency systems

• End-to-end solutions for all markets

• Timing delivery and assurance Excellency

Innovation leader for timing distribution and assurance

Page 3: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.3 © 2015 ADVA Optical Networking. All rights reserved. Confidential.3

City

Town

Public switch

Television

Answeringmachine

PBX

IBM Compatible

Laptop computer

Telephone

Fax

IBM Compatible

Public switch

Answeringmachine

PBX Telephone

Laptop computer

Fax

City

Town

… to enable service providers to transport

bits of information within and across network

boundaries without losing any bits of information.

This is accomplished by synchronizing all

transmitted signals to a common stable frequency

source.

The Objective of Synchronization Is …

Page 4: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.4 © 2015 ADVA Optical Networking. All rights reserved. Confidential.4

Common Sync Problems

Voice Audible Click

Fax Illegible text

Email Retransmitted data

Video Corrupted video

Cell/PCS Dropped calls

SONET/TDM Data loss

ATM Corrupted data

VoIP-voice Latency (echo)

VoIP-FAX Dropped calls

QUICK PROBLEMS CHECKLIST

Poor synchronization is the most common, non-obvious, cause of service degradation

Page 5: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.5 © 2015 ADVA Optical Networking. All rights reserved. Confidential.5

Typical Hierarchical Synchronization Plan

Stratum

2

Stratum

2

Stratum

2

Stratum

2

Stratum

1 Stratum

3

Stratum

3Stratum

3Stratum

3E

Stratum

1

Stratum

3E

Stratum

3E

Stratum

1

Stratum

3E

Stratum

3EStratum

3E

Distributing a highly accurate frequency reference to all Network Elements in an effort to elevate the internal Oscillators of the network elements to that of the frequency

source (Stratum 1)

Page 6: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.6 © 2015 ADVA Optical Networking. All rights reserved. Confidential.6

ANSI Clock Standards

Stratum-1

Stratum-2

Stratum-3E

Stratum-3

1 x 10-11

1.6 x 10-8

4.6 x 10-6

Not Defined

< 255 DS1 slips,

1st 24 hrs.

1 x 10-10 per

day, 1st 24 hrs.

4.6 x 10-6 1 x 10-8 per

day, 1st 24 hrs.

1.6 x 10-8

Not Defined

4.6 x 10-6

4.6 x 10-6

SMC 2.0 x 10-5

Stratum-4 Not Defined3.2 x 10-5 3.2 x 10-5

4.6 x 10-6 2.0 x 10-5

The maximum MTIE during a reference rearrangement for

SONET interfaces is 1 ms or 20 ns in any 14 ms.

Page 7: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.7 © 2015 ADVA Optical Networking. All rights reserved. Confidential.7

The Objective of Synchronization Is …

Df = The average rate of phase accumulation (Dt/t)

1 picosec/sec 1x10-12 (1 ppt) (part per trillion)

10 picosec/sec 1x10-11

100 picosec/sec 1x10-10

1 nanosec/sec 1x10-9 (1 ppb) (part per billion)

10 nanosec /sec 1x10-8

100 nanosec /sec 1x10-7

1 msec /sec 1x10-6 (1 ppm) (part per million)

Df = fractional frequency offset

Page 8: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.8 © 2015 ADVA Optical Networking. All rights reserved. Confidential.88

Frame Slips Due to Clock Offsets

Clock Extraction

Buffer

(Typically two frames or 250 usec)

System Clock

Write Clock

Read

Clock

To

Switching

Matrix

Incoming

Signal

Frequency offsets between traffic terminating elements are

reflected in the phase alignment of the read and write clocks

of Slip Buffers. Poorly synchronized networks cause these

buffers to slip, resulting in frame slips.

Frequency offsets between traffic terminating elements are

reflected in the phase alignment of the read and write clocks

of Slip Buffers. Poorly synchronized networks cause these

buffers to slip, resulting in frame slips.

Page 9: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.9 © 2015 ADVA Optical Networking. All rights reserved. Confidential.9

TechnologyOptical Cesium

Hydrogen Maser

Cesium Standards

GPS Receivers

Rubidium OscillatorsHQQ +

Quartz Crystal Oscillators

Tuned Circuits

Stratum Levels

“Stratum 0”

Stratum-1

Stratum-2E

Stratum-2

Stratum-3E

Stratum-3

Stratum-4

1 X 10-15

1 X 10-5

Frequency Standards

9

Page 10: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.10 © 2015 ADVA Optical Networking. All rights reserved. Confidential.10

PRS - Cesium

In 1967 the esium atom was recognized as the basis

for the international standard (SI) unit of time

Atomic resonant frequency is exactly 9,192,631,770 Hz

All PRS equipment will be traced back to a Cesium standard.

Telecom Cesium devices have, typically, rT/T of 1 picosecond/second

(or a 1x10-12 frequency offset)

One DayrT = 86,400 picoseconds

One YearrT = 31,536 nanoseconds

12 YearsrT = 378 microseconds

Less than 1/2 ms time error (with respect to UTC) for the life of the tube.

(“Turn it on, and forget about it”)

10

Page 11: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.11 © 2015 ADVA Optical Networking. All rights reserved. Confidential.11

All GPS satellites contain multiple Stratum 1 Clocks (Cesium and/or Rubidium standards). The Clocks in the satellites keep accurate time to within three nanoseconds. A GPS Primary Reference Receiver will derive the accuracy of the standards and provide a very stable Stratum 1 clock source. The GPS satellites are in half synchronous orbit so they circle the earth twice a day.

PRS - Global Positioning System

11

Page 12: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.12 © 2015 ADVA Optical Networking. All rights reserved. Confidential.12

GPS Space Segment

•The space segment consists of low earth orbit (LEO) satellites, 24 in all: 21 navigational SVs and 3 active spares orbit at 11,000 nautical miles above the Earth. There are six orbital planes (with 4 SVs in each), equally spaced (60 degrees apart), and inclined about 55 degrees with respect to the equatorial plane.

12

Page 13: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.13 © 2015 ADVA Optical Networking. All rights reserved. Confidential.13

GPS Control Segment

• The GPS control segment consists of a system of monitor stations located around the world (Hawaii and Kwajalein in the Pacific Ocean; Diego Garcia in the India Ocean; Ascension Island in the Atlantic Ocean; and Colorado Springs, Colorado) a master ground station at Falcon Air Force Base in Colorado Springs, Colorado; and four large ground antenna stations that broadcast signals to the satellites.

13

Page 14: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.14 © 2015 ADVA Optical Networking. All rights reserved. Confidential.1414

Highly reliable PRS

Roof access required for antenna

Long-term capitalization factor is high

Time-of-day availableGNSS

Town

Self-contained, highly-reliable PRS

Antenna not required

Long-term capitalization factor is low

Time-of-day not availableCESIUM

City

Primary Reference Source Options

Page 15: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.15 © 2015 ADVA Optical Networking. All rights reserved. Confidential.1515

TSG Architecture

Page 16: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.16 © 2015 ADVA Optical Networking. All rights reserved. Confidential.16

Building Integrated Timing Supply - BITS

DS3

TSG

MUX

ChannelBanks

DCS STPToll

Switch

CC/DS1

ATMSwitch

ISDN

SONETADM

SONETADM

CC EC151 Mbit

MSSPRS

Stratum 1DSLDSLAM

DS1

DS1

DS1

DS1DS1

DS1

CC

CC

CC

CC/DS1

DS3

DS0

DS0

DS0

DS0

DS1OC192

OC192

OC48

OC48

OC3

OC3IP

ATMDS3

16

Page 17: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.17 © 2015 ADVA Optical Networking. All rights reserved. Confidential.1717

Inter-Office Timing Distribution - SONET

Page 18: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.18 © 2015 ADVA Optical Networking. All rights reserved. Confidential.1818

Timing Loops

Page 19: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.19 © 2015 ADVA Optical Networking. All rights reserved. Confidential.1919

Bi-Directional Switched Ring

Page 20: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.20 © 2015 ADVA Optical Networking. All rights reserved. Confidential.20

Bi-Directional Switched Ring

Page 21: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.21 © 2015 ADVA Optical Networking. All rights reserved. Confidential.2121

SSM – Sync Status Messaging

Page 22: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.22 © 2015 ADVA Optical Networking. All rights reserved. Confidential.22

SONET SSM Formats

Page 23: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.23 © 2015 ADVA Optical Networking. All rights reserved. Confidential.23

SSM Rules for Implementation

Page 24: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.24 © 2015 ADVA Optical Networking. All rights reserved. Confidential.2424

SSM Bi-Directional Ring Configuration

Page 25: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.25 © 2015 ADVA Optical Networking. All rights reserved. Confidential.2525

SSM – Bi-Directional Ring Configuration

Page 26: Evolution of Network Synchronization Technologies

Packet TimingTechnologies

Chuck [email protected]

512-431-3103

Page 27: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.27 © 2015 ADVA Optical Networking. All rights reserved. Confidential.2727

NTP Overview

Network Time Protocol (NTP) synchronizes clocks of hosts and routers in the Internet

The NTP architecture, protocol and algorithms have been evolved over the last two decades. Currently NTP Version 4 is being developed

Well-tested and widely-deployed protocol

NIST estimates 10-20 million NTP servers and clients deployed in the Internet and its tributaries all over the world. Every Windows/XP has an NTP client

NTP provides nominal accuracies of low tens of milliseconds on WANs, submilliseconds on LANs, and submicroseconds using a precision time source such as a GNSS receiver

Traditional implementations are primarily software-based. Non-deterministic delays in networking stacks contribute to significant timing inaccuracy

Page 28: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.28 © 2015 ADVA Optical Networking. All rights reserved. Confidential.28

NTP Protocol Overview

• Clock offset:• [(T2 – T1) + (T4 – T3)] / 2

• Round-trip delay:• (T4 – T1) – (T3 – T2)

Client Server

Client sends request at T1 = 10:15:00

T1

T2

T3

T4

Server receives request at T2 = 10:15:12

Server sends response at T2 = 10:15:15

Client receives response at T2 = 10:15:30

» Key Assumptions:

– Network delay is symmetric in both directions

One-way delay is half of round-trip delay

– Client and server clocks drift at the same rate

Page 29: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.29 © 2015 ADVA Optical Networking. All rights reserved. Confidential.2929

NTP Stratum Levels

S1 S1

S2 S2 S2 S2

S3 S3 S3 S3 S3

S4 S4 S4 S4

Stratum 1

Stratum 2

Stratum 3

Stratum 4

Hierarchical layering of clocks based on number of hops from primary reference source

Stratum 1 servers are synchronized with a GPS source

Stratum 2 servers use client/server mode to synchronize with up to six Stratum 1 servers and symmetric mode to synchronize with other servers on the same stratum level

Stratum 4 clocks work in client mode to synchronize with servers in Stratum 3

NTP Stratum levels are not the same as ANSI/ITU-T Stratum levels!

Page 30: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.30 © 2015 ADVA Optical Networking. All rights reserved. Confidential.30

IEEE1588 Protocol Overview

• The Slave collects the time values t1, t2, t3, t4 during a transaction and calculates final offset (o) between Master and Slave clocks canceling out network delay (d) as follows:

t2 –t1 = o + d

t4 - t3 = -o + d

o = (t2 + t3 – t1 – t4) / 2

d = (t2 – t1 + t4 – t3) / 2

Master Clock Time Slave Clock Time

Data at

Slave Clock

t1

t2

t2m

t2

Sy nc Message

Followup Message

containing v alue of t1

t1

t2

t3t

3mDelay Request

Message

t1

t2 t

3

t4

Delay Response Message

containing v alue of t4

t4

t1

t2 t

3

Time

Page 31: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.31 © 2015 ADVA Optical Networking. All rights reserved. Confidential.3131

• IEEE 1588 (commonly known as Precision Time Protocol, PTP) was ratified as a standard in September 2002

• LAN technology providing timing for the control of distributed applications

• Slow transaction rates

• Version 1 of the protocol used for applications in• Industrial automation

• Test and measurement

• Military

• Version 2 developed for telecom applications• Early adopters include Vodafone, T-Mobile, etc.

IEEE-1588 Version 1

Page 32: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.32 © 2015 ADVA Optical Networking. All rights reserved. Confidential.32

IEEE 1588v2 Enhancements

• IEEE 1588v2 meets accuracy requirements for Telecom applications • High refresh rates up to 128 messages per second

• Correction field for asymmetric measurements

• Several modes supported• Broad-cast, Multi-cast and Uni-cast are permitted

• Smaller message length to conserve bandwidth

• 72 octets (44 for 1588v2 payload)

• Multiple Master Clock selection methods• Manual, Semi-automatic, Fully-automatic

• Transparent Clocks to reduce accumulation of timing errors across network elements in cascaded topologies

• Enhanced security• Configurable network in combination with Best Master Clock algorithm for GrandMaster

• HASH codes

32

Page 33: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.33 © 2015 ADVA Optical Networking. All rights reserved. Confidential.33

Precision Time Protocol (PTP) Explained

Grandmaster(Server)

L2/L3 device

ExternalSlave (client)

1588 Packet Flow

1588

1588

1588

• Protocol used to synchronize clocks throughout a network.

• The Grandmaster (GM) “reference clock” sends a series of time-stamped messages to slaves

• Slaves receive the messages, and eliminate the round-trip delay by synchronizing to the Grandmaster.

• Frequency/Time-of-Day/Phase is recovered from the accurate time of day reference from the GM.

• Boundary Clocks (BC) can receive PTP as a reference, while providing GM functionality downstream to other clients.

Boundary Clock (BC)

ExternalSlave (client) Embedded

Slave (client)

1588

1588

1588

(BC)

(BC)

(BC)

Page 34: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.34 © 2015 ADVA Optical Networking. All rights reserved. Confidential.3434

Transparent Clock

MAC

PHY

MII

MAC

PHY

MII

PTP

UDP

IP

MAC

PHY

MII

PTP

UDP

IP

MAC

PHY

MII

Grandmaster Trasparent Clock

Transparent Clock

Slave

Grandmaster Transparent Clock Slave

A Transparent Clock contains no PTP ports.

Timestamp in incoming message is modified before sending the message out

Creates security issues, since original crypto checksum is not valid anymore

A Transparent Clock is neither a master nor a slave. It is merely a switch that adjusts a PTP message’s timestamp to compensate for its own queueing delays

IP Network

M S

Page 35: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.35 © 2015 ADVA Optical Networking. All rights reserved. Confidential.3535

Boundary Clock

PTP

UDP

IP

MAC

PHY

MII

PTP

UDP

IP

MAC

PHY

MII

PTP

UDP

IP

MAC

PHY

MII

PTP

UDP

IP

MAC

PHY

MII

Slave Master

IP Network

Grandmaster Boundary Clock

Boundary Clock

Slave

Grandmaster Boundary Clock Slave A boundary clock contains

more than one PTP port:

a slave port that is synchronized with a remote master, and

a master port that synchronizes other slaves downstream

Synchronization messages are terminated at each port and not forwarded

A Boundary Clock extends synchronization across an intermediate network element

M S

M

MS

S

Page 36: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.36 © 2015 ADVA Optical Networking. All rights reserved. Confidential.36

Mobile Backhaul / The Challenges

Application Radio Interface Backhaul

Frequency Phase Frequency Phase

CDMA 2000 ±50ppb ±3 to 10µs GPS GPS

GSM/WCDMA ±50ppb n/a ±16ppb n/a

LTE (FDD) ±50ppb n/a ±16ppb n/a

LTE (TDD) (large cell) ±50ppb ±5µs ±16ppb ±1.1µs

LTE (TDD) (small cell) ±50ppb ±1.5µs ±16ppb ±1.1µs

LTE-A MBSFN ±50ppb ±1 to 5µs ±16ppb ±1.1µs

LTE-A CoMP* ±50ppb±500nsec to

5µs±16ppb 500ns - ±1.1µs

LTE-A eICIC* ±50ppb ±1 to 5µs ±16ppb ±1.1µs

* The performance requirements of the LTE-A features are under study by 3GPP

New timing distribution architectures are Now Required!

Page 37: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.37 © 2015 ADVA Optical Networking. All rights reserved. Confidential.37

Network Synchronization Migration (L2/L3)1st mile 2nd mile Aggregation Core (IP/MPLS)

GPS PRS

SSU/TSG

CENTRAL OFFICE

FREQUENCY IS STILL REQUIRED, BUT MANY APPLICATIONS NOW REQUIRE PRECISE PHASE & TIME

Ethernet

Ethernet

Page 38: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.38 © 2015 ADVA Optical Networking. All rights reserved. Confidential.38

Network Synchronization Migration (L2/L3)1st mile 2nd mile Aggregation Core (IP/MPLS)

GPS PRS

SSU/TSG

CENTRAL OFFICE

FREQUENCY IS STILL REQUIRED, BUT MANY APPLICATIONS NOW REQUIRE PRECISE PHASE & TIME

Ethernet

Ethernet

Page 39: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.39 © 2015 ADVA Optical Networking. All rights reserved. Confidential.39

Distributed Architecture• Can be added to any exiting NE’s

• Wide Temperature range

• Simple configuration

• Small footprint and Cost effective

• Highly integrated

• Very low power (1.2W)

T-GM

GNSS

Mid-PTPGrandmaster

Packet-Based Backhaul Network

GNSS

T-SC

T-SC

T-SC

First Aggregation Node

GM closer to end application

T-SC

Mini-GM

Embedded orLow Cost SFP

GNSS/GM/T-SC

Page 40: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.40 © 2015 ADVA Optical Networking. All rights reserved. Confidential.40

GNSS only at first aggregation site1588v2 with Full/Partial On Path Support to Cell Sites

APTS in case of GNSS failure

T-GM

GNSS

PTPGrandmaster

Packet-Based Backhaul NetworkPTP unaware or partly aware

G.8265.1 /G.8275.2

RemoteBase Station

T-SC

T-SC

T-SC First Aggregation

Node

Boundary Clock &

PTP fully aware or partly aware networkG.8275.1/G.8275.2

BC

GNSS

5420

GM

Mid Scale GM with APTS in Aggregation Node– Core GM Protection

Page 41: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.41 © 2015 ADVA Optical Networking. All rights reserved. Confidential.41

GNSS only at first aggregation site1588v2 with Full/Partial On Path Support to Cell Sites

APTS in case of GNSS failure

T-GM

GNSS

PTPGrandmaster

Packet-Based Backhaul NetworkPTP unaware or partly aware

G.8265.1 /G.8275.2

RemoteBase Station

T-SC

T-SC

T-SC First Aggregation

Node

Boundary Clock &

PTP fully aware or partly aware networkG.8275.1/G.8275.2

BC

GNSS

Mid-Scale

GM

Mid Scale GM with APTS in Aggregation Node– Core GM Protection

Page 42: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.42 © 2015 ADVA Optical Networking. All rights reserved. Confidential.42

Performance Monitoring & Probing

• Software feature that measures and reports the status/state of the sync network

• Embedded in Many Synchronization products

• Analogous to Perf Mon and Bit Error Rate in traditional networks

• SLA Verification

Page 43: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.43 © 2015 ADVA Optical Networking. All rights reserved. Confidential.43

PTP Network Probe Statistics and Results

• Packet Counters (arrived, lost)

• PD (Path Delay) – min, max, avg., forward only

• MPD (Mean Path Delay) – min, max, avg., both directions

• RPDV (Residual Path Delay Variation) – min, avg., both directions (based on observed Delay Floor); Current Value and Histogram

• Network Usability (based on G.8261.1 FPP); Current Value and Histogram

Network Score -FR

Network Score -REV

Lost /Received

Page 44: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.44 © 2015 ADVA Optical Networking. All rights reserved. Confidential.44

Network Management

• Build, Monitor and Expand a Reliable Sync network

• Reduce on-site Operation with a Centralized Management process

• Performance monitoring, Fault and Event management

• Dedicated for sync network monitoring

• Provide access to• Core, edge and access sync equipment

Page 45: Evolution of Network Synchronization Technologies

1588 for Power Utilities

Chuck [email protected]

512-431-3103

Page 46: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.46 © 2015 ADVA Optical Networking. All rights reserved. Confidential.46

Why Is Timing Relevant?

There are many definitions for the smart grid, but in short …

• We need to do more with less

• Utilities need to do things differently

• Distributed generation & energy flows create network stability concerns

• New reliability and security regulation

Protection, Telecommunication, Metering and Control (PTM&C) technologies will have the biggest impact

Timing is inherent to PTM&C

Genera

tion

Tra

nsm

issio

n&

Dis

trib

ution

Power Flow

Power Flow

Page 47: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.47 © 2015 ADVA Optical Networking. All rights reserved. Confidential.47

And There Is the Regulatory Need

Au3 blackout… the case for forensics

• Blackout in north-east USA focused attention on the electric power grid

• After investigating, NERC suggested GPS time stamping throughout the grid

• Power System Outage task force endorsed NERC’s timing recommendation

NERC standard PRC-018-1

• Utilities must maintain a time accuracybetter than 2 ms in disturbance recorders

• Time shall be in Coordinated UniversalTime (UTC) format

NERC … North American Electric Reliability Corporation

Page 48: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.48 © 2015 ADVA Optical Networking. All rights reserved. Confidential.48

Timing Dependent Applications

Traditional Applications

• TDM communication networks

• Grid frequency management

• Event correlation (network and substation level)

• relays record reason for operation

• analogue trace recorders

• Control centre computers & terminal units (RTU)

• Scheduled load shedding

• Quality of Supply metering

• Energy metering (time of use tariffs)

Advanced Applications• Lightning strike monitoring• Travelling wave fault location• Synchrophasor measurement• Merging Units/Sample Values

Page 49: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.49 © 2015 ADVA Optical Networking. All rights reserved. Confidential.49

Smart Grid Timing Needs Today!

Application MeasurementAccurac

yTime Interface Sync Source

TW Fault Locator 300 m (line span) 1 μs PTP, IRIG-B, PPO GPS, 1588 GMC

Phasor Measurements ± 0.1 degree 1 μs PTP, IRIG-B (1344) GPS, 1588 GMC

Lightning Strike Correlation Grid-wide events 1 ms IRIG-B GPS

Protection Relaying events < 1 cycle 1 msPTP, IRIG-BIEC 61850

GPS, IRIG-B, 1588 GMC

Event/Disturbance Recorders < 1 cycle 1 ms PTP, IRIG-B, PPO GPS, 1588 GM

Network, Distribution & Substation Control

Grid-wide events 1 ms PTP, IRIG-BGPS, Control Centre,1588 GMC

Quality of Supply Metering Freq, time error 0.5 sec PTP, IRIG-B, PPO GPS, 1588 GMC

Bulk Metering Energy registers 0.5 sec Proprietary, PPO Proprietary

Customer Premises Metering Energy registers 1 sec NTP, Proprietary Proprietary, NTP

SCADA/EMS/PAS Grid-wide status 1 ms NTP, ASCII GPS

Frequency Measurement Frequency 1 ms N/A GPS

Sampled Values Volt/Current 1 μs PTP 1588 GM

Telecommunication SDH/PDH G.812/813PTP G.82652.048 Mbps/MHz

GPS, 1588 GMC

Page 50: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.50 © 2015 ADVA Optical Networking. All rights reserved. Confidential.50

IEEE 1588 Overview

IEEE 1588-2008 …

• Also referred to as Precision Time Protocol (PTP)

• Grandmaster “reference clock” sends time-stamped messages to IED’s

• Communication is over the Ethernet with traffic

• IED’s (slaves) eliminate round-trip delay & synchronize to the GMC

Accuracy is enhanced by:

• Hardware time-stamping (eliminate software processing delays)

• 1588 capable switches (transparent/boundary clocks)

• Best Master Clock schemes

IED

IEEE 1588Aware Switch

Grandmaster(GMC)

61850 LAN

1588 15881588 1588

Page 51: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.51 © 2015 ADVA Optical Networking. All rights reserved. Confidential.51

IEEE 1588 Power Profile

• IEEE PSRC co-ordinates with IEC TC57 WG10 (IEC 61850)

• Tasked with developing IEEE Standard PC37.238 –“Profile for Use of IEEE 1588 Precision Time Protocol in Power System Applications”

• Profile characteristics:

• LAN (Layer 2 Ethernet mapping)

• Multicast addressing only

• Peer-to-peer delay measurement

• Switches must be transparent clocks

• Holdover time defined

• http://www.pes-psrc.org/h/ Power Profile

Defined by IEEE PSRC (C37.238)

Substation LAN applications

Page 52: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.52 © 2015 ADVA Optical Networking. All rights reserved. Confidential.52

What Has Changed For Timing

From application perspective

• Accuracy increasing from 1ms to 1μs

From the regulatory perspective

• Timing of recorders is mandatory

Within the substation

• IRIG-B ports won’t be in future IED’s

• Timing is needed in the switch yard

• Sampled values make synchronization a necessity - no longer an option

• Must know clock quality and status (unmanaged GPS clocks not sustainable)

• Substations LANs change the way timing is distributed - IEEE 1588

Page 53: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.53 © 2015 ADVA Optical Networking. All rights reserved. Confidential.53

IEEE 1588 on the IEDs

• IEDs with IEEE 1588 power profile

LAN 1

LAN 2

LAN 3

Page 54: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.54 © 2015 ADVA Optical Networking. All rights reserved. Confidential.54

Interoperability Between Vendors

• Implementation of C37.238 (power profile) grandmasters, switches and slaves in IED’s

• Inter-operability of C37.238 aware components

• IEEE PSRC working groupstandard for conformance &performance testing

• IEEE 1588 plugfests

IEEE Plugfest, ISPCS ConferenceNew Hampshire, USA

Page 55: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.55 © 2015 ADVA Optical Networking. All rights reserved. Confidential.5555

Phasor Measurement Unit

A phasor measurement unit (PMU) is a device which measures the electrical waves on an electricity grid.

Page 56: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.56 © 2015 ADVA Optical Networking. All rights reserved. Confidential.5656

What is a Synchrophasor

Time synchronization allows synchronized real-time measurements of multiple remote measurement points on the grid. The resulting measurement is known as a synchrophasor.

• A mailbox-sized, monitoring device that can measure the instantaneous voltage, current and frequency at specific locations on the grid.

• They give power grid operators a near-real-time picture of what is happening on the system.

Gives power grid operators a near-real-time picture of what is happening on the system.

Page 57: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.57 © 2015 ADVA Optical Networking. All rights reserved. Confidential.57

Synchophasor Applications

Page 58: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.58 © 2015 ADVA Optical Networking. All rights reserved. Confidential.58

Is GPS Safe Enough?

N.J. man fined $32K for illegal GPS device that disrupted Newark airport system

Page 59: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.59 © 2015 ADVA Optical Networking. All rights reserved. Confidential.5959

Is GPS Safe Enough?

GPS car jammers

GPS jammer 5 band jammer L1, L2, L3, L4, L5 + RF lowjack

Page 60: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.60 © 2015 ADVA Optical Networking. All rights reserved. Confidential.6060

Jammer Hunting with UAV

A fully autonomous, unmanned aerial vehicle (UAV)-based system for locating GPS jammers, currently under development, seeks to localize a jammer to within 30 meters in less than 15 minutes in an area comparable to that of an airport

Page 61: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.61 © 2015 ADVA Optical Networking. All rights reserved. Confidential.61

Protecting The GNSSAging/Day Temperature

stability

Operational

temperature

Quartz ± 5e-10 ± 5e-9 -40 to 65C

Quartz

HQ++

± 5e-11 ± 1e-11 -40 to 65C

Rubidium ± 5e-12 ± 2e-10 -40 to 45CAging can be estimated with

GNSS and eliminated

Enter holdover

24 Hours

50

0

ns Quartz HQ++ Holdover

• Tested in the oven with temperature profile +/-20C

• Phase holdover over 24 hours below 500nsec !

Page 62: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.62 © 2015 ADVA Optical Networking. All rights reserved. Confidential.62

Timing Delivery for Substations… Today and Tomorrow

EventRecorder

GNSSPrimary

IRIG B

Protection Schemes

ADMRemoteTerminalUnit (RTU)

IRIG-B Bus

DS1G.703 9/13

1PPS

ProtectionRelay

PMU

AlarmAnnunciator

SubstationGateway

QOSMeter

SubstationSwitch

61850 LAN

PTPIEEE C37.118

NTP

RELAY ROOM

PTP Telecom ProfileG.8265.1G.8275.1

Mid PTP GM

PTPSecondary

Page 63: Evolution of Network Synchronization Technologies

Thank [email protected]

IMPORTANT NOTICE

The content of this presentation is strictly confidential. ADVA Optical Networking is the exclusive owner or licensee of the content, material, and information in this presentation. Any reproduction, publication or reprint, in whole or in part, is strictly prohibited.

The information in this presentation may not be accurate, complete or up to date, and is provided without warranties or representations of any kind, either express or implied. ADVA Optical Networking shall not be responsible for and disclaims any liability for any loss or damages, including without limitation, direct, indirect, incidental, consequential and special damages, alleged to have been caused by or in connection with using and/or relying on the information contained in this presentation.

Copyright © for the entire content of this presentation: ADVA Optical Networking.

Page 64: Evolution of Network Synchronization Technologies

© 2015 ADVA Optical Networking. All rights reserved. Confidential.64 © 2015 ADVA Optical Networking. All rights reserved. Confidential.64

Abbreviations

3ϕ Three Phase

61850 A standard for the design of an electrical substation (more detail)

AMI Advanced Metering Infrastructure

ADM Add-Drop Multiplexor (SDH/SONET terminal)

AMR Automatic Meter Reading

CES Circuit Emulation Service

CT Current Transformer

DFR Digital Fault Recorder

DNP Distributed Network Protocol

DR Demand Regulation, or

DR Disturbance Recorder

DSM Demand Side Management

EHV Extra High Voltage

EMC Electro-Magnetic Compatibility

EV Electric Vehicle

FERC Federal Energy Regulatory Commission

GMC Grandmaster Clock

GOOSE Global Object Oriented Substation Event

GUI Graphic User Interface

FTM Frequency & Time Deviation Monitor

HV High Voltage

I Current

IP Internet Protocol

IEC International Electrotechnical Commission

IED Intelligent Electronic Device

IEEE Institute of Electrical & Electronic Engineers

IRIG-B Inter-Range Instrumentation Group time-code B

NERC North American Electric Reliability Corporation

NTP Network Time Protocol

OC Ordinary Clock (PTP reference)

PAS Power Application Software

PDH Plesiochronous Digital Hierarchy

PMU Phasor Measurement Unit

PPO Programmable Pulse Output (e.g. 1PPS)

PQ Power Quality

PSN Packet Switched Network

PSRC Power System Relaying Committee

PT Potential Transformer (sometimes called a VT

PTM&C Protection, Telecommunication, Metering/Measurement and Control

PTP Precise Time Protocol

QOS Quality of Supply

RTU Remote Terminal Unit

SCADA Supervisory Control & Data Acquisition

SDH Synchronous Digital Hierarchy

SV’s Sampled Analog Values

TW Travelling Wave

UTC Universal Coordinated Time (world standard)

VT Voltage Transformer (sometimes called a PT)