UnderstandingResilientEthernetProtocolOperatio_brkopt2216

97
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 1 BRKOPT-2216 14442_04_2008_c2

Transcript of UnderstandingResilientEthernetProtocolOperatio_brkopt2216

Page 1: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 1BRKOPT-2216

14442_04_2008_c2

Page 2: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 2BRKOPT-2216

14442_04_2008_c2

Understanding Resilient Ethernet Protocol Operation

BRKOPT-2216

François Tallet

Page 3: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 3BRKOPT-2216

14442_04_2008_c2

Agenda

REP Protocol Basics

REP vs. STP

REP Is Configuration Driven

REP Is not Deterministic by Default

REP Is Fast

REP has a Local Scope

End Ports and Topology Information

Preemption and Load Balancing

Topology Change

Interaction with STP

Conclusion

Page 4: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 4BRKOPT-2216

14442_04_2008_c2

REP Protocol Basics

Page 5: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 5BRKOPT-2216

14442_04_2008_c2

REP Protocol BasicsContext

Providers using Ethernet bridges to implement rings

The Spanning Tree Protocol (STP) is not very efficient on rings

RPR requires special hardware

Software ring solutions (G.8032) are limited to rings

Resilient Ethernet Protocol (REP) must:

Perform as well as software ring solutions on rings

Using regular hardware

While supporting any kind of topology

Page 6: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 6BRKOPT-2216

14442_04_2008_c2

REP Protocol BasicsA Segment Protocol

REP operates on chain of bridges called segments

A port is assigned to a unique segment using:(config-if)# [no] rep segment {id}

A segment can have up to two ports on a given bridge

interface f1

rep segment 10

interface f2

rep segment 10

interface f1

rep segment 10

interface f2

rep segment 10

REP Segment

f1 f2f1 f2f1 f2f1 f2f1 f2

Page 7: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 7BRKOPT-2216

14442_04_2008_c2

f1 f2f1 f2f1 f2f1 f2f1 f2

f1 f2f1 f2f1 f2f1 f2f1 f2

REP Protocol BasicsBlocked Port

When all links are operational, a unique port blocks the traffic on the segment

If any failure occurs within the segment, the blocked port goes forwarding

Edge PortEdge Port f2 Blocks Traffic

f2 Unblocks Link Failure

Page 8: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 8BRKOPT-2216

14442_04_2008_c2

REP Protocol BasicsREP Segments Don’t Introduce Loops

No connectivity between E1 and E2 through the segment

The segment can be attached to any bridged domain without introducing a loop

No control protocol needed between segment and bridged domain

E2E1

Bridged Domain

E2E1

Page 9: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 9BRKOPT-2216

14442_04_2008_c2

E2E1 E2E1

REP Protocol BasicsREP Provides Two Redundant Gateways

The segment provides one level of redundancy

Hosts on the segment can reach the rest of the network through either edge port, as necessary

X X

Y

Bridged Domain Bridged

Domain

Y

Page 10: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 10BRKOPT-2216

14442_04_2008_c2

REP Protocol BasicsREP Creates a Redundant Link

Segments can be wrapped into a ring

Can be seen as a redundant link in that case

Note: Identification of edge ports requires additional configuration in that case

A BX

Y

A BX

Y

Page 11: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 11BRKOPT-2216

14442_04_2008_c2

REP Protocol BasicsA Simple Building Block

Any redundant network can be built with REP segments!

“REPification”

Page 12: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 12BRKOPT-2216

14442_04_2008_c2

REP Protocol BasicsSummary

REP segment concept is primitive

A segment is a chain of bridges

If all the links are available, REP blocks

If there is at least a failure, REP unblocks

However… any redundant network can be built with REP segments

Page 13: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 13BRKOPT-2216

14442_04_2008_c2

STP vs. REP:REP Is Configuration Driven

Page 14: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 14BRKOPT-2216

14442_04_2008_c2

REP Is Configuration DrivenSTP Is Plug-n-Play

A bridge forwards traffic on a port unless the port has been determined as being redundant

STP is plug-n-play and detect those ports by receiving BPDUs (Bridge Protocol Data Units) on them

If STP fails to detect a redundant port, consequences are catastrophic

Page 15: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 15BRKOPT-2216

14442_04_2008_c2

12

root

My Port 2 Is Not

Redundant

BA

bpdu Lost

REP Is Configuration DrivenUnidirectional Link Failure

A link is declared up by the physical layer, but only transmits traffic in one direction

Normal Scenario Unidirectional Link Failure on link A-B

Traffic Is Looping

Clockwise

Unidirectional Link Failure

12

root

My Port 2 Is

Redundant

BAbpdu

Page 16: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 16BRKOPT-2216

14442_04_2008_c2

REP Is Configuration DrivenSTP “Brain-Dead” Bridge

The bridge’s STP process is not working

BPDUs are ignored or not acted upon

Traffic is still switched in hardware

Bridge B’s STP Is Not Working

12

root

???

BAbpdu

Traffic looping in

both directions

“Brain-dead” Bridge

Page 17: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 17BRKOPT-2216

14442_04_2008_c2

REP Is Configuration DrivenSTP Solutions Built in REP

Both STP problems described earlier have solutions (UDLD, LoopGuard, dispute, bridge assurance)

Those solutions imply additional configuration or design constraints (plug-n-play aspect is lost)

REP does not attempt to be plug-n-play and will not suffer from those issues

Page 18: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 18BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

REP Is Configuration DrivenREP Adjacency

A segment port establishes an adjacency with a unique neighbor using a three-way handshake

The port is not considered operational until the adjacency is established

Periodic hellos are exchanged to maintain the adjacency

I’m 4 I’m 5, Hearing 4I’m 4, Hearing 5

Page 19: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 19BRKOPT-2216

14442_04_2008_c2

REP Is Configuration DrivenLink Integrity Checked Hop by Hop

The location of a failure is known exactly

REP blocks the failed link, thus providing maximum connectivity

I’m 3

1 2 5 6

I’m 4, Hearing 3

I’m 3

1 2 4 5

I’m 6

3 6

4

“Brain-Dead” Bridge

Unidirectional Link Failure

3

Page 20: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 20BRKOPT-2216

14442_04_2008_c2

REP Is Configuration DrivenSummary

STP works with no configuration (plug-n-play)

REP requires explicit configuration

REP requirements are strictly enforced

REP is safer than STP with no configuration

Page 21: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 21BRKOPT-2216

14442_04_2008_c2

STP vs. REP: REP Is Not Deterministic by Default

Page 22: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 22BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

REP Is Not Deterministic by DefaultLink Coming Up

Ports start in blocking state

Blocked ports generate a key when they come up

Blocked ports advertise their key to the whole segment with Blocked Port Advertisements (BPA)

A port will only unblock if it receives a BPA carrying the key it has generated

BPA P6, key6BPA P7, key7

Link 6-7 Is Brought Up

key7key6

Page 23: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 23BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

REP Is Not Deterministic by DefaultPriority Scheme

The keys include a priority and a mac address

They can be compared and no two keys are identical

The port with the “best” key unblock the other(s)

BPA P6, key7

Key6 > Key7, Port 6 Unblocks Port 7

Page 24: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 24BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

REP Is Not Deterministic by DefaultREP Removes All Blocked Ports but One

REP removes all blocked ports but one

Ports that were not blocking were not considered in the election

Other Ports Were Not Considered!

Ports 6 & 7 Took Part in Election

Page 25: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 25BRKOPT-2216

14442_04_2008_c2

REP Is Not Deterministic by DefaultNot Deterministic Without Configuration

The final state depends on the initial conditions

The blocked port is likely to be on the last link that came up

The blocked port determination is not deterministic!

Different Initial Conditions (Different Links Come Up)

Different Final State

Page 26: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 26BRKOPT-2216

14442_04_2008_c2

REP Is Not Deterministic by DefaultCase of Link Flapping with STP

Link up and link down trigger STP convergence

A link flapping will keep the network unavailable

Network Unavailable: Convergence in Progress

Network Available Network Available

A

C

BD

A

C

BD

A

C

BD

A

C

BD

Page 27: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 27BRKOPT-2216

14442_04_2008_c2

REP Is Not Deterministic by DefaultIt’s a Feature!

The last link coming up in REP is blocked

A (single) link flapping have thus no impact on REP

Link Down

Link Down

LinkUp

An optional mechanism allows moving the location of a blocked port in an operational segment (more later)

Network Available Network Available While Link FlapsNetwork

Unavailable

A

C

BD

A

C

BD

A

C

BD

Page 28: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 28BRKOPT-2216

14442_04_2008_c2

STP vs. REP:REP Is Fast

Page 29: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 29BRKOPT-2216

14442_04_2008_c2

REP Is FastSTP Convergence: Hop by Hop

A BPDU carries the failure indication hop by hop

In the worst case, the information has to travel twice across the network before complete recovery

Propagation of the Failure Indication

The Blocked Port Is Moved to the Location of the Failure

Page 30: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 30BRKOPT-2216

14442_04_2008_c2

Bridge

REP Is FastBPDU Propagation Through a Bridge

The queuing and processing of the BPDUs add some delay to the propagation

The BPDU is not just relayed, it is regenerated (the content of its fields is modified)

Switching ASIC

CPU

Output QueueInput Queue

BPDU In Regenerated BPDU Out

Data Frame In

Unmodified Data Frame Out

Page 31: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 31BRKOPT-2216

14442_04_2008_c2

REP Is FastSTP Makes No Assumption

STP “syncs” all potential redundant links during convergence of new information

Blocking the ports for the sync operation might be expensive in time (depending on the platform)

ProposalAgreement

Proposal

Sync

A

B

A

B

A

B

Page 32: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 32BRKOPT-2216

14442_04_2008_c2

REP Is FastSTP Convergence Summary

Queuing and scheduling delay (BPDUs waiting in input/output queues)

Processing delay (STP computation, port state changes)

Reliable (BPDUs retransmitted periodically)

Page 33: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 33BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

REP Is FastStable Conditions

The unique blocked port keeps sending its BPAs

Every port has stored a copy of the blocked port’s key

BPA P6, key6

Page 34: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 34BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

REP Is FastOpen, Failed and Alternate Ports

A port that is forwarding is called an open port

A port logically blocked by REP is an alternate port

A port with a link status down is called a failed port

Failed ports have better priority than alternate ones

Link Failure

Failed Ports Alternate Port

Open Port

Page 35: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 35BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

REP Is FastLink Failure

When a link fails, it sends a BPA with the key it had stored in order to unblock the previous alternate port

The convergence is a one-step process

BPA P3, key6

Final State: Key Updated

9 107 85 63 41 2

Link Failure

Failed Ports Alternate Port

Page 36: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 36BRKOPT-2216

14442_04_2008_c2

REP Is FastLink Status Layer Propagation

Link Status Layer (LSL): Responsible for maintaining the adjacency between neighbors

BPA are propagated inside the LSL frames

The transmission is reliable, but still hop by hop as LSL frame are terminated by adjacent bridges

LSL BPA

LSL LSL

LSL BPA

Periodic Hello

BPA Added to Periodic Hello

1 2 3 4 5 6

Page 37: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 37BRKOPT-2216

14442_04_2008_c2

REP Is FastLink Status Layer Summary

STP BPDU LSL BPA

Hop by Hop yes yes

Reliable yes yes

Queuing Delays

yes yes

Processing Delay

yes little

LSL BPA tx not much faster than BPDU tx

Page 38: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 38BRKOPT-2216

14442_04_2008_c2

REP Is FastHardware Flood Layer Propagation

BPAs are not modified during their propagation

The Hardware Flood Layer (HFL) defines a frame designed to carry BPAs

HFL frames are not terminated by the adjacent bridge and are switched like user data across the network

Page 39: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 39BRKOPT-2216

14442_04_2008_c2

Bridge

REP Is FastBPA Propagation Using HFL

HFL frames are both

Switched through the bridge in hardware

Queued to CPU for processing

Switching ASIC

CPU

Output QueueInput Queue

BPA in HFL Frame

Unmodified BPA in HFL Frame

Page 40: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 40BRKOPT-2216

14442_04_2008_c2

REP Is FastHFL Propagation Is Backed up by LSL

BPA messages are propagated at the same time by

HFL messages, flooded instantly (hardware support)

LSL messages, reliable but relayed in software

HFL BPA P3, key6

LSL BPA P3, key6

9 107 85 63 41 2

Page 41: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 41BRKOPT-2216

14442_04_2008_c2

REP Is FastBPA Propagation Summary

STP BPDU LSL BPA HFL BPA

Hop by Hop yes yes no

Reliable yes yes no

Queuing Delays

yes yes no

Processing Delay

yes little no

Page 42: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 42BRKOPT-2216

14442_04_2008_c2

REP Is FastHFL Needs a Dedicated VLAN

HFL messages are treated like user traffic

They are flooded on a dedicated admin VLAN

The admin VLAN controls the reach of HFL messages

If the administrative VLAN is not properly configured, REP will only rely on the LSL mechanism

Page 43: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 43BRKOPT-2216

14442_04_2008_c2

E2E1

REP Is FastREP Administrative VLAN

For HFL to function, the admin VLAN must span the path between the edge ports inside and outside of the segment

Bridged Domain

HFL Dropped by Blocked Port

HFL Flooded Network Wide on the Admin VLAN

Admin VLAN Is Not Desirable on Access Ports

Page 44: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 44BRKOPT-2216

14442_04_2008_c2

REP Is FastAdministrative VLAN Configuration

By default, REP uses VLAN 1 as an administrative VLAN

This VLAN can configured using the global configuration command:

(config)# [no] rep admin VLAN {id}

Page 45: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 45BRKOPT-2216

14442_04_2008_c2

REP Is FastHFL Frames Flooded Network Wide

HFL frames reach wherever the admin VLAN is configured

BPAs include a segment ID in their fields

What if two segments have the same ID in the network?

E2

E1

Bridged Domain

BPA F1, Key E2, Segment 1

Segment 1Segment 1

A

F1

F2

Will the BPA Sent by Remote Segment 1 Unblock A?

Page 46: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 46BRKOPT-2216

14442_04_2008_c2

REP Is FastLocal Effect of BPAs

The key included in the BPA is generated within the segment

A BPA can only unblock a port local to the segment that generated it

The key is not a security mechanism!

The admin VLAN is used to restrict the reach of BPAs forwarded with HFL to trusted devices only

Page 47: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 47BRKOPT-2216

14442_04_2008_c2

REP Is FastSummary

REP only needs to handle a very simple topology

Its BPAs don’t need processing while being relayed

BPAs are relayed reliably hop-by-hop by the LSL

BPAs are also propagated in hardware by the HFL

The administrative VLAN contains the HFL flooding

Page 48: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 48BRKOPT-2216

14442_04_2008_c2

STP vs. REP:REP Has a Local Scope

Page 49: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 49BRKOPT-2216

14442_04_2008_c2

REP Has a Local ScopeSTP Has a Global Scope

STP runs on the whole network. Segmentation impossible when there is redundancy

All the costs are calculated from the unique root: Cost configured on Site3 depend on the root in Site1

Before STP Cost Tuning

After Some STP Cost Tuning

Site 1

Site 2

Site 3Provider Network

XY

RootSite 1

Site 2

Site 3Provider Network

XY

Root

Page 50: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 50BRKOPT-2216

14442_04_2008_c2

A Switchover to a Secondary Root May Invalidate the Configuration

Site 1

Site 2

Site 3Provider Network

XY

Site 1

Site 2

Site 3Provider Network

XY

REP Has a Local ScopeSTP Local Changes Have Global Effects

Local changes have global effect

This has to be anticipated when configuring STP

Topology Depends on Remote Site

Root Root

Topology Depends on Remote Site

Page 51: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 51BRKOPT-2216

14442_04_2008_c2

REP Has a Local ScopeREP Local Changes Have Local Effect

Convergence in a segment have no impact on other segments

Local changes have local effect

REP Version of the Network

No Dependency Between the Sites

Site 1

Site 2

Site 3Provider Network

XY

Site 1

Site 2

Site 3Provider Network

XY

Failure!

No Impact

No Impact

Root Root

Page 52: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 52BRKOPT-2216

14442_04_2008_c2

B

B

REP Has a Local ScopeSide Effect of REP’s Local Scope

REP only protects against a single failure, even if there are several levels of redundancy in the network.

STP would have unblocked the green segment.

1- Red Segment Fails 2- Red Segment Fails Again

3- Green Segment Unaffected

A A B

A

Page 53: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 53BRKOPT-2216

14442_04_2008_c2

REP Has a Local ScopeInvalid Configuration

Wrong configurations can result in permanent loops

STP sees the whole network and would not allow that

B

Green and Blue Segments Are Completely Independent

Loop!

C

A B

Page 54: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 54BRKOPT-2216

14442_04_2008_c2

REP Has a Local ScopeSummary

REP segments only reacts to events within their boundaries

REP only introduces one level of redundancy

REP provides little protection against configuration errors

REP’s local scope is a desirable feature not a bug

Attempting to “fix” this would be the first step toward re-inventing STP

Page 55: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 55BRKOPT-2216

14442_04_2008_c2

End Ports and Topology Information

Page 56: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 56BRKOPT-2216

14442_04_2008_c2

End Ports and Topology InformationEnd Ports

Edge and failed ports are end ports

End ports define groups of contiguous operational links called sub-segments

REP Sub-Segment

REP Sub-Segment

End Port End Port

End Ports

REP Segment

9 107 85 63 41 2

Page 57: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 57BRKOPT-2216

14442_04_2008_c2

End Ports and Topology InformationEnd Port Advertisements

End ports send “End Port Advertisements” (EPAs)

The EPAs are propagated hop by hop reliably by the LSL across the segment (or sub-segments)

Each port that relay an EPA add some information

EPA Info 2 EPA Info 5,6 EPA Info 5,6,7,8

EPA Info 9EPA Info 7,8,9EPA Info 3,4

9 107 85 63 41 2

Page 58: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 58BRKOPT-2216

14442_04_2008_c2

End Ports and Topology InformationTopology Information

By combining the information received in each direction, a bridge collects a view of the whole sub-segment

To display the topology information, use:(exec)# show rep topology [segment id]

5 6 7 8 9 10

EPA Info 5,6 EPA Info 5,6,7,8

EPA Info 9EPA Info 7,8,9

Sub-Segment 5,6,7,8,9

A# show rep topology segment 10

REP Segment 10

BridgeName PortName Edge Role

---------- -------- ---- ----

A 5 Fail

A 6 Open

B 7 Open

B 8 Open

C 9 Pri Open

C

Info 7,8,9

+ Info 5,6

= Info 5,6,7,8,9

Info 9

+ Info 5,6,7,8

= Info 5,6,7,8,9

BA

Page 59: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 59BRKOPT-2216

14442_04_2008_c2

End Ports and Topology InformationConfiguring Edge Ports

Edge ports are essential in order to generate EPAs

However, in ring topologies, edge ports cannot be derived from the configuration

Edge ports can be specified explicitly using:(config-if)# [no] rep segment {id} edge

Explicit Configuration of the Edge Ports Is Required

f1f2

f1

f2

f1 f2

interface f1

rep segment 5

interface f2

rep segment 5

f1f2

f1

f2

f1 f2

interface f1

rep segment 5 edge

interface f2

rep segment 5 edge

Page 60: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 60BRKOPT-2216

14442_04_2008_c2

End Ports and Topology InformationSegment Complete

The EPAs also carries the ID of the edge ports

When both edge ports see each other, the segment is complete: there is no failed link (no failed port)

EPA: I’m Edge Port ID2

EPA: I’m Edge Port ID9

End Port: Edge 1

End Port: Edge 9

Alternate Port (Not Failed)

9 107 85 63 41 2

Page 61: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 61BRKOPT-2216

14442_04_2008_c2

End Ports and Topology InformationSummary

End ports are edge or failed ports

They send EPA messages that collect information about the extend of the operation (sub-)segment

EPAs determine the segment complete condition

REP Sub-Segment End Port (Edge)

End Port (Failed)

9 107 85 6

Page 62: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 62BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired Topology

Page 63: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 63BRKOPT-2216

14442_04_2008_c2

9 107 85 63 41 2

Enforcing the Desired TopologyPrimary Edge Election

The primary edge is the edge port with the best ID, the other is secondary edge

The secondary edge acknowledge the primary edge in the EPA messages it sends

When the primary receives the acknowledgement from the secondary, it detects the segment is complete

Primary Edge

Secondary Edge

Page 64: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 64BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired TopologyUse of the Primary Edge Port

The primary edge is unique in the segment

Segment wide parameter configuration will be centralized on the primary edge

In particular, it will control

The location of the segment port the user wants to block (by default, the choice is not deterministic)

The VLAN load-balancing

The election of the primary edge port can be influenced by setting:

(config-if)# [no] rep segment {id} edge primary

Page 65: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 65BRKOPT-2216

14442_04_2008_c2

f9 f10f7 f8f5 f6f3 f4f1 f2

Enforcing the Desired TopologySpecifying an Alternate Port

A desired blocked port can be specified from the primary edge, using the command:(config-if)# [no] rep block port {neighbor #} vlan all

#+1

Alternate Port

interface f2

rep segment 10 edge primary

rep block port 5 vlan all

#+2 #+3 #+4 #+5 #+6 #+7 #+8

Desired Configuration Offset Relative to Primary Edge

Page 66: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 66BRKOPT-2216

14442_04_2008_c2

f9 f10f7 f8f5 f6f3 f4f1 f2

Desired Configuration

Link 4-5 Fails

f9 f10f7 f8f5 f6f3 f4f1 f2

Enforcing the Desired TopologyStep 1: Link Failure

A link fails. The alternate port is replaced by two failed ports around the failure

The edge ports don’t receive any EPA from a peer edge port: the segment is not complete (failed)

Alternate Port

Failed Ports

Page 67: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 67BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired TopologyStep 2: Segment Complete

The link is repaired: a failed port becomes alternate, the other failed port disappears

The stable topology is not the desired one (determination of the alternate port not deterministic)

The edge ports see each other in the EPAs they send: the primary edge knows the segment is complete

New Topologyf9 f10f7 f8f5 f6f3 f4f1 f2

Page 68: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 68BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired TopologyStep 3: Preemption

The primary edge sends an EPA message instructing f6 to preempt

f6 goes alternate. There are two alternate ports!

f9 f10f7 f8f5 f6f3 f4f1 f2

Preemption Request

Two Alternate Ports

f9 f10f7 f8f5 f6f3 f4f1 f2

Page 69: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 69BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired TopologyStep 4: Preemption Completes

Port f6 preempts the segment:As if it had failed, f6 sends a BPA to unblock port f4

The segment has returned to the desired topology

Port f6 Unblocks Port f4

Back to Desired Topology

f9 f10f7 f8f5 f6f3 f4f1 f2

BPA Port f6, Key4

f9 f10f7 f8f5 f6f3 f4f1 f2

Page 70: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 70BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired TopologyTriggering Preemption

The preemption is optional

It can only be triggered after the primary edge detects the segment complete

The trigger can be automatic, after a configurable delay(config-if)# [no] rep preempt delay {seconds}

Or it can be started from the CLI prompt(exec)# rep preempt segment {id}

Page 71: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 71BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired TopologyVLAN Load Balancing Capability

REP can block at most in two different locations

A range of VLANs can be configured on the primary edge

This range will be blocked by a specified alternate port

The complementary range is blocked by the primary edge

The configuration is again entered at the primary edge:

(config-if)# [no] rep block port {neighbor#} vlan {VLAN_list}

Page 72: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 72BRKOPT-2216

14442_04_2008_c2

f9 f10f7 f8f5 f6f3 f4f1 f2

f9 f10f7 f8f5 f6f3 f4f1 f2

Enforcing the Desired TopologyVLAN Load Balancing (Step 1)

BPAs include a bitmap of VLANs

The primary edge preempts a range of VLANs

Preemption Request

Two Alternate Ports

BPA Port 2, VLANs 1-2000

Blocks VLANs 2001-4095

interface f2

rep segment 1 edge primary

rep block port 5 VLAN 2001-4095

#+1 #+2 #+3 #+4 #+5 #+6 #+7 #+8

Blocks VLANs 1-2000

Page 73: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 73BRKOPT-2216

14442_04_2008_c2

f9 f10f7 f8f5 f6f3 f4f1 f2

Enforcing the Desired TopologyVLAN Load Balancing (Step 2)

Using a command EPA message, the primary edge instructs port f6 to preempt the remaining VLANs

BPA Port 6, VLANs 2001-4095

Blocks VLANs 1-2000

Blocks VLANs 2001-4095

Blocks VLANs 2001-4095

Blocks VLANs 1-2000

f9 f10f7 f8f5 f6f3 f4f1 f2

Page 74: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 74BRKOPT-2216

14442_04_2008_c2

Enforcing the Desired TopologySummary

In a segment that is complete, two ports can “preempt” and block complementary ranges of VLANs

Those ports are

The elected primary edge

Another arbitrary port determined by configuration

Configuration can be centralized on the primary edge

The preemption can be triggered manually or after a configured delay

Page 75: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 75BRKOPT-2216

14442_04_2008_c2

Topology Changes

Page 76: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 76BRKOPT-2216

14442_04_2008_c2

Topology ChangesLearning Bridges

Bridges learn the location of the stations from the traffic they forward

Mac-addresses are added to a filtering table

Filtering Entries Populated from Conversation X-Y

After a Change in the Topology, “Starred” Entries Are Incorrect

After a failure, the filtering table must be updated

B

X X

X

Y

Y

X

X Y

X

X

X

YY

Y

Y

YY

Y

X X X

X

X

X

X

X

Y

Y

Y

Y

Y

Y

Y

Page 77: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 77BRKOPT-2216

14442_04_2008_c2

Topology Changes REP Topology Change Bit

A Topology Change (TC) bit is set in the BPA to advertise that the filtering tables need updating

All the entries in the tables are flushed

Traffic is flooded while tables are being repopulated

B

X

YY

X

Y

X X X

XX

Y

Y

Y

Y

X

X

X

Y

Y

Y

X XX

XX

XX

X

X

X

X

BPA, TC

BPA, TC

X

Page 78: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 78BRKOPT-2216

14442_04_2008_c2

Topology Changes Topology Changes Between Segments

BPAs have local effect. No flush performed outside of the segment that originated them

yY y

BPA, TC, Segment 1

BPA, TC, Segment 1

BPAs Flush Entries for X in Segment 1

Entry for X Left in Segment 2 Causes Black Holing

Entries for X Removed

Entry for X remaining

Segment 1

Segment 2

B

X

X

X

X

XX

X B

X

X

X X

Y

Page 79: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 79BRKOPT-2216

14442_04_2008_c2

Topology Changes Propagating TC Outside the Segment

Edge ports are responsible for advertising TCs to the rest of the network

Configured on edge ports

(config-if)# [no] rep stcn segment {id_list}

When the edge port receives a BPA with the TC bit set, it will generate a new BPA with TC bit set for the segments specified

Page 80: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 80BRKOPT-2216

14442_04_2008_c2

Segment 1

Topology Changes

TC Propagation

Both edge ports receive a TC indication

They are configured to propagate it to another segment

This step is achieved in software. It will introduce a slight delay

B

BPA, TC, Segment 1

BPA, TC, Segment 1

Segment 2

BPA TC for Segment 1 Converted into BPA TC for Segment 2 at the Edge

BPA, TC, Segment 2 BPA Generated

by E2

E2E1

interface E1

rep segment 1 edge

rep stcn 2

Page 81: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 81BRKOPT-2216

14442_04_2008_c2

Topology Changes Summary

After a convergence, the filtering databases need to be updated to prevent black-holing

REP BPAs include a TC bit that carry the appropriate notification within a segment

If the TC needs to propagate to another segment, explicit configuration is needed on the edge ports

Page 82: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 82BRKOPT-2216

14442_04_2008_c2

STP Interaction

Page 83: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 83BRKOPT-2216

14442_04_2008_c2

STP InteractionSTP Does Not Run on Segment Ports

Segment ports are not added to the STP

From the perspective of STP, REP ports don’t exist!

Physical Network Configuration Connectivity, as Seen by STP

No BPDU Tx or RXREP Segment

STP Region STP Region

Page 84: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 84BRKOPT-2216

14442_04_2008_c2

STP InteractionValid REP Configuration

REP configuration must not introduce loops

STP Perspective

Data Plane Perspective

No REP Port… No Loop

Traffic Not Looping

REP Segment

STP Region

STP Region

STP Region

Page 85: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 85BRKOPT-2216

14442_04_2008_c2

STP InteractionInvalid REP Configuration

REP is not plug-n-play: cannot recover from some configuration errors

STP Perspective

Data Plane Perspective

REP Segment

STP Region No REP Port… No Loop

Traffic Is Looping!

STP Region

STP Region

Page 86: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 86BRKOPT-2216

14442_04_2008_c2

STP InteractionTopology Changes and STP

To avoid black holing, mac address tables in the STP region must be updated after a reconvergence in a segment

The case is similar to the TC between segments presented earlier on

B

Y

Y

Y

Y

Y

X

X

XY

Filtering Entries Populated from Conversation X-Y

After a Change in the Topology, “Star”Entries Are Incorrect

X

X

X

X

X

Y

Y

Y

Y

Y

YSTP

Region

B

Y

Y

Y

Y

XY

X

Y

Y

Y

Y Y

YSTP

Region

X

X

X

X

Entries Cleared by REP in the Segment

X

XX

Y

YYXX XX

Page 87: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 87BRKOPT-2216

14442_04_2008_c2

STP InteractionEdge Port Generate STP TC

A configuration in the edge port allows converting BPA TC into STP TC:

(config-if)# [no] rep stcn stp

X

YSTP

Region

REP TCs Are Converted into STP TCs a the Edges

STP TC

STP TCinterface E1

rep segment 1 edge

rep stcn stp

E1 E2

X

Y

Page 88: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 88BRKOPT-2216

14442_04_2008_c2

rep

stpstp

stp

STP InteractionMigration from REP to STP

While migrating from REP to STP, the user could end up in a situation where one or several bridges are isolated

The switch cannot be configured from the network any more!

REP Ports with No Neighbors = Failed

Page 89: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 89BRKOPT-2216

14442_04_2008_c2

rep

stpstp

stp

STP InteractionFailed Open Port

REP is able to differentiate between a port that is failed because of a link failure or because of no neighbor

A port with no neighbor will be “Failed Open” if there is already a port with higher priority blocking in the segment

REP Unblocks a Failed Port to Maintain Connectivity

Page 90: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 90BRKOPT-2216

14442_04_2008_c2

STP InteractionSummary

STP is “disabled” on REP ports

Caution must be taken to avoid configuring loops

REP includes a mechanism to export TCs to STP

REP maintains connectivity while transitioning from STP to REP

Page 91: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 91BRKOPT-2216

14442_04_2008_c2

Conclusion

Page 92: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 92BRKOPT-2216

14442_04_2008_c2

ConclusionREP Was Designed to Not Be STP

REP is a simple concept

REP requires simple configuration and is robust

Its has local effect, which makes it easy to segment and operate the network

REP convergence can be independent of the diameter of the network, which makes it ideal for long chains

Page 93: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 93BRKOPT-2216

14442_04_2008_c2

ConclusionSummary REP vs. STP

REP STP

Configuration Driven:

Configuration Error Can Be Fatal

Plug-n-Play:

Operates with No Configuration

Local Scope:

Simple But with Single Level of Redundancy

Global Scope:

Can use all the Redundant Links Available

Fast:

Convergence Independent of the Number of Nodes

Slow in Large Networks:

Hop-by-Hop Transmission Suffers from Large Network Diameter

Can Operate in a Non-Deterministic Way

Always Deterministic

Page 94: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 94BRKOPT-2216

14442_04_2008_c2

ConclusionRelated Sessions

BRKOPT-2205: Deploying and Designing with the Resilient Ethernet Protocol

BRKOPT-2102: 802.17 Resilient Packet Ring Overview

Page 95: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 95BRKOPT-2216

14442_04_2008_c2

Q and A

Page 96: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 96BRKOPT-2216

14442_04_2008_c2

Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily.

Receive 20 Passport points for each session evaluation you complete.

Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.

Don’t forget to activate your Cisco Live virtual account for access to all session material on-demand and return for our live virtual event in October 2008.

Go to the Collaboration Zone in World of Solutions or visit www.cisco-live.com.

Page 97: UnderstandingResilientEthernetProtocolOperatio_brkopt2216

© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 97BRKOPT-2216

14442_04_2008_c2