Public Multi-Vendor Interoperability Event 2013 White Paper

Public Multi-VendorInteroperability Event 2013

White Paper

2

SDN, IPv6, MPLS & Ethernet World Congress 2013 Multi-Vendor Interoperability Test

EDITOR’S NOTE

In the eleven years ofEANTC interopera-bility events at MPLS &Ethernet WorldCongress, we haveevaluated multipleevolutionary technol-ogies. Starting with IP/MPLS' virtual privatenetwork and trafficengineering solutions in2003, continuing withthe packet transport-

oriented Carrier Ethernet paradigm since 2006, thepost-SDH-focused MPLS-TP in 2007, and addingIPv6 migration testing since 2011. These technol-ogies have been deployed worldwide, often incoexistence. Given their vast deployed base,continued extensions are required — which resultedin more complex test scenarios every year. By addition of SDN and extension of IPv6, MPLS,and mobile backhaul clock synchronizationscenarios, our interoperability event undertakinggrew substantially this year. We asked the 19 partic-ipating vendors to hot-stage the scenarios in our labfor three full weeks (a 50 % increase), one of whichwas exclusively dedicated to SDN and clocksynchronization testing in twostreams. Compared to last year, thetest plan grew by 140 % to 51complex test cases. More than 45vendor engineers supported thehot-staging on site in our lab inBerlin, Germany in February.It was definitely worth the effort:This year’s white paper containsmore factual, independent infor-mation about more technologysolutions than ever. In detail:

SDN. For the second time, wetackled Software-DefinedNetworking (SDN) interoperabilityin the context of Carrier Ethernet /MPLS networks. SDN is sometimes portrayed as adisruptive technology. Long story short: It is too earlyto really see a revolution coming. This would requirean eco-system of advanced protocol standards,interoperable vendor solutions and jointly supporteduse cases. Somehow SDN reminds me of electriccars: Yes, they exist today and yes, the Asianmanufacturers have the best public plans. Therevolution is not there yet. But it will very likelyhappen — the time depending on how much buyersare willing to accept setbacks today in favor of abetter vision of the future.Last year, the SDN protocol of vendors' choice wasPath Computation Element (PCE). Surprisingly, PCEvendors declined this time which left OpenFlow asthe selected management protocol. Five OpenFlowswitch and controller vendors joined our tests. Quitea few parties more were initially interested but found

that their implementations could not support ourservice provider (WAN) focused tests yet. 53 % ofthe SDN test cases we prepared did not find partiesready to test them.In the context of service provider WANs, our SDNinteroperability test explored use cases such as multi-table IPv6 multicast, policy based routing, andresilient Carrier Ethernet services. Some of thesescenarios required OpenFlow 1.2 — which only oneparticipating vendor (Huawei) supported.Compared to OpenFlow 1.0, we concluded thatOpenFlow 1.2 fits service provider use cases withdiverse services much better.

Clock Synchronization. There is a continueddefinite market need for packet-clock frequency andphase synchronization of mobile backhaul networks;with further LTE deployment and small cells cominginto the picture, it will become even more important.Our main focus this time was on advancing IEEE1588-2008 telecom profile support for boundaryclocks. The ITU is in the process of developing apartial on path support profile which requiresadvanced boundary clocks. Two vendors, BTI andEricsson, successfully participated in the boundaryclock tests of phase synchronization. We wentbeyond the ITU standard tests, introducing multipleimpairment locations. Four vendors tested trans-parent clocks successfully. In addition we tackled two

scenarios important for serviceproviders with legacy infra-structure: SyncE islands inter-connected by IEEE 1588 andTransparent SDH/SONET overPacket synchronization.IP/MPLS Multicast VPNs.For the first time worldwide,Alcatel-Lucent, Cisco, Ixia andJuniper Networks implementa-tions of Multicast VPNsuccessful interoperated at ourtest event this time. This marks atechnology advance towardsfull integration of multicasttraffic with MPLS VPNs.A new area was opened for

interoperability testing with Ethernet VPNs (EVPN) —the potential successor of VPLS for Ethernet multipointservices. While this technology is still in its earlystages, Cisco and Ixia staged the first ever publicinteroperability test.

IPv6. Do you remember that June 6, 2012 wasdeclared the »World IPv6 launch«? Nevertheless,there are still a few interoperability issues in someimplementations. Our tests of migration scenarios(6rd, Dual-Stack, DS-lite) in multi-vendor environ-ments evaluated the vendors' readiness to support amixed IPv4/IPv6 environment for the time to come.We set a major focus on end-to-end IPv6 protocoland migration support so we invited CPE vendors tojoin. This way we were able to test end-to-endadvanced address/port mapping solutions (MAP-E/MAP-T) as well as the Locator/ID Separation

Carsten RossenhövelManaging Director

Carsten RossenhövelManaging Director, EANTC

TABLE OF CONTENTS Participants and Devices.............. 3Software-Defined Networking....... 4Clock Synchronization................. 7MPLS Multicast VPN.................. 11Topology ................................. 14IPv6 Tests ................................. 19Carrier Ethernet 2.0 .................. 23Demonstration Network ............. 26Acronyms ................................ 27

3

Introduction

Protocol (LISP) to overcome current routing tablegrowth issues.

Carrier Ethernet 2.0. Carrier Ethernet 2.0conformance is a big topic for the MEF in 2013. Wetested the underlying technologies (class of service,Ethernet S-tag/C-tag interconnections, multi-pointservices support) in previous years; vendor imple-mentations are generally very mature. From ourpoint of view, one of the biggest challenges forCarrier Ethernet is efficient provisioning and servicelevel monitoring support. Our CE2.0 tests focusedareas that are not covered by the MEF certification:ITU Y.1564 service activation tests and performancemonitoring per class of service. Y.1564 support isstill very limited; this time only Ixia brought an imple-mentation. Four vendors participated in per-classperformance monitoring already, which is great,although there were a few hiccups. We will continueto evaluate both areas in future events.

INTRODUCTION

This year’s event stretched into new realms. TheEANTC team started preparations in October 2012,developing new test cases for all four main testareas. Fueled by great and ambitious proposalsfrom equipment manufacturers in the followingweeks, the list grew further until we finalized the testplan in January.The final test plan included 51 test cases, with 19 ofthose making fresh paths to SDN testing. All in allwe had the right balance to entice 19 equipmentmanufacturers to join us in the hot-staging event thattook place in our lab in Berlin in February.This time brought five first-time participants, two ofthem universities supporting open-source implemen-tations of advanced CPE protocols. Our lab in Berlin was full of activity with the morethan 45 engineers who worked closely togetherunder a multi-party Non-Disclosure Agreement. Forthe first time ever, the hot-staging tests extended tothree weeks where the first week was exclusivelydedicated to two streams, SDN and clock synchroni-zation testing. EANTC has set industry standards for planning,execution and documentation of public interopera-bility events. In our tradition, strict form was followedto ensure a level-playing field for all participants,requiring validation of all test results by EANTCengineers as a precondition for being documentedin this white paper. As usual, this white paper documents positive results(passed test combinations) individually with vendorand device names. Failed test combinations are notmentioned in diagrams; they are referenced anony-mously to describe the state of the industry. Ourexperience shows that participating vendors quicklyproceed to resolve interoperability issues after ourtest so there is no point to punish vendors. Confiden-tiality is a main cornerstone to encourage manufac-turers to participate with their latest (beta) solutions.

INTEROPERABILITY TEST RESULTS

The following sections of the white paper describeeach of the test areas and results.

Terminology. We use the term tested whenreporting on multi-vendor interoperability tests. Theterm demonstrated refers to scenarios where aservice or protocol was evaluated with equipmentfrom a single vendor only. In any case, demonstra-tions were permitted only when the topic had beencovered in the previously agreed test plan;sometimes vendors ended up with demonstrationsbecause there was no partner to test with, orbecause multi-vendor combinations failed so thatthey could not be reported.

Test Equipment. With the help of participatingtest equipment vendors, we generated andmeasured Ethernet and MPLS traffic, emulated andanalyzed control and management protocols andperformed clock synchronization analysis. We thankCalnex Solutions, Ixia, Spirent Communications andSymmetricom for their test equipment and supportthroughout the hot-staging.

PARTICIPANTS AND DEVICES

Vendor Devices

AimValley EX14

Alcatel-Lucent 7750 SR-12

Broadcom BCM956440

BTI SA-810

Calnex Paragon-X

Cernet Cernet CPE

Cisco ASR 1000ASR 9000CRS-1

ECI Telecom NPT-1200

Ericsson MINI-LINK SP 110MINI-LINK SP 210MINI-LINK SP 310MINI-LINK SP 415MINI-LINK SP 420SSR 8010

Extreme Networks Summit X670E4G-200

Hitachi AMN 6400

Huawei Smart Network Controller(SOX)Openflow Switch 1.2

4


SOFTWARE-DEFINED NETWORKING

The main idea behind Software-Defined Networking(SDN) is to enable optimized, fine-grain, appli-cation-aware resource management in a transportnetwork infrastructure. By decoupling the controlplane from the data plane of a packet transportnetwork, the control plane can be centrally imple-mented at a few servers (controllers). Each controllershould be designed to manage a large number ofswitches. Thus, SDN promotes a paradigm shift forpacket switched networks — from distributed tocentralized network control.The industry discusses many SDN use cases at thisearly time in the development cycle. SDN for datacenters is implemented as one of the first use cases.Current deployments are usually single-vendor basedand use proprietary technology to some extent.As the next logical step, the industry actively debateswhich standardized protocol or set of protocols willbest suit a wide range of SDN use cases, extendingbeyond the data center. Some discussions focus onprotocol design questions (it should be easilyextendable and backward compatible), on efficiencyand protocol reusability aspects.When we invited SDN equipment manufacturers tothis test event, EANTC proposed to evaluate threeprotocols:

• OpenFlow (versions 1.0, 1.1, and 1.2) specifiedby the Open Networking Foundation (ONF).

• Path Computation Element (PCE), standardizedby the PCE working group of the InternetEngineering Task Force (IETF). The main goal of

PCE is to centralize path computation for IP/MPLS label-switched paths.

• NETCONF, specified by the IETF’s NETCONFworking group. NETCONF has originated as aprotocol to manage network devices.

While preparing for the test event, we noticed a lotof vendor interest in OpenFlow testing and someinterest in PCE testing. Our test plan defined 12 test cases for OpenFlow,half of them requiring OpenFlow 1.1 or 1.2 support.It seems that we overestimated industry progress inthis case, and that our service provider-focused testcases for newer protocol versions were tooambitious for around half of the vendors initiallyinterested. In the end, five vendors participated inOpenFlow testing; four supported only OpenFlow 1.0.EANTC had already tested multi-vendor interopera-bility of early Path Computation Element (PCE) imple-mentations during 2012’s MPLS & Ethernet WorldCongress showcase. Surprisingly, we had to dropthe PCE test area in this event due to lack of vendorparticipation. We hope to continue PCE multi-vendorinteroperability testing at our next event.

OpenFlow 1.0 ResultsSince the OpenFlow 1.0 and 1.2 specifications areincompatible, we report results in separate subsections.The following controllers supported OpenFlow 1.0:Huawei Smart Network Controller (SOX), IxiaIxNetwork, NEC PF6800, Spirent TestCenter.In the OpenFlow 1.0 switch role, we used NECPF5240 and Extreme Networks Summit X670.

Topology Discovery in OpenFlow 1.0The Link Layer Discovery Protocol (LLDP) is adiscovery protocol for Ethernet links that is defined inthe IEEE 802.1AB specification. SDN controllers canuse this protocol to automatically discover thephysical topology in SDN.Our test setup consisted of the two interconnectedOpenFlow switches which were controlled by eachof the OpenFlow controllers, taking turns. Vendorsconfigured two flow entries on the switches via theOpenFlow controller for each combination: The FE-Table-Miss default flow entry and the FE-LLDP — anentry which specifically matched the LLDP frames.We expected the controller to transmit LLDP packetsperiodically at all switch ports using the packet-outOpenFlow message. When the switch would receivethe LLDP frames we expected it to match the FE-LLDPentry and forward the LLDP frames to the controller.Finally we expected the controller to successfullybuild and display the discovered topology to theadministrator.Figure 1 below shows the successful results fortopology discovery in OpenFlow.

Ixia ImpairNetIxNetworkIxLoadAnue 3500

Juniper Networks MX-480 3D Universal Edge Router

NEC PF5240PF5248PF6800

OE Solutions —AimValley

Smart SFP TSoPSmart SFP OAM

SpirentCommunications

Spirent TestCenter

Symmetricom SSU2000eTimeProvider 5000

UPC LISPmob home routerLISPmob mobile node

Vendor Devices

5

Software-Defined Networking

Figure 1: OpenFlow Topology Discovery Results

Both NEC and Huawei controllers displayed thediscovered topology graphically. The emulatedcontrollers (Ixia and Spirent) showed the topology ina table form.We observed that one of the OpenFlow switchessent incorrectly encapsulated LLDP messages to thecontroller, such that the frames were malformed.During the hot-staging the vendor corrected the codeand successfully tested the requested functionalitysubsequently.

Layer 2 and Layer 3 Forwarding in OpenFlowAn OpenFlow switch maintains flow tables whichare populated by an OpenFlow controller. Each flowentry is composed of header match fields, countersand actions. When a data packet arrives, the switchtries to match the header of the packet with theentries in its flow tables. If one of the entries matchesthe packet header, the switch performs actions withthe packet as specified by the flow entry. In this test case we verified the ability of anOpenFlow controller to install flow entries on anOpenFlow Switch consisting of multiple fields ofEthernet and IP headers. As soon as the entries weresuccessfully installed, we verified whether the switchcould forward the data traffic properly according tothe actions specified in the matched flow entry.The test setup consisted of three interconnectedOpenFlow switches which were controlled by asingle OpenFlow controller at a time. Threeemulated hosts were connected to the switches. Host2 and Host 3 were connected to the OpenFlowSwitch 2, Host 1 was connected to the OpenFlow

Switch 1. We applied two flows: One matched theVLAN ID and MAC destination address of Host 2,the second matched the destination IP address ofHost 3. We configured Host 1 and 2 in the same IPnetwork and Host 3 in a different IP network. On theEthernet service layer we configured a VLAN per IPnetwork. As soon as the emulated Host 1 sent trafficto Host 2, the OpenFlow Switch directly attachedmatched the VLAN and destination address of Host2 and forwarded the traffic. The next OpenFlowSwitch routed traffic between the two IP networks.Host 1 and Host 3 used this next OpenFlow Switchas default gateway. When the OpenFlow Switchtasked with IP routing received the traffic from Host 1destined to Host 3, it replaced the VLAN ID and thedestination MAC address with Host 3’s VLAN ID andMAC address.Figure 2 summarizes successful results for the Layer2 and Layer 3 Forwarding in OpenFlow test.Note that test scenarios were carried out with thethree switches taking turns in their roles for layer 2and layer 3 forwarding as described above.

Figure 2: OpenFlow Layer 2 and Layer 3 Forwarding Results(legend see Figure 1)

When we performed the test with each of theemulated controllers, we configured the flowsmanually. In the tests with non-emulated controllerimplementations, the controllers discovered thenetwork topology and installed the flow entriesdynamically.In one test run the Huawei Smart Network Controller(SOX) controller successfully controlled the switchesof OpenFlow version 1.0 and 1.2.One OpenFlow switch implementation was unableto forward flows with and without VLAN encapsu-lation simultaneously. In one of the test runs, whenwe configured flows for tagged traffic, the imple-mentation dropped the untagged LLDP packets. As aresult the tested controller could not discover thetopology. At a later time the vendor was able to fix abug in the code, and we reran the test successfully.

Extreme Summit X670

NEC

Spirent

Ixia IxNetwork

TestCenter

PF6800

HuaweiSOX

NEC PF5240

OpenFlow 1.0

OpenFlow OpenFlowControllerSwitch

OpenFlow 1.2

NEC

Ixia IxNetwork

PF6800 NEC PF5240

Extreme Summit X670

Spirent TestCenter

Huawei Openflow 1.2

HuaweiSOX

6


In two other test runs we observed unexpectedpacket loss for traffic flows that we transmitted at alow rate. We assume that the flow entry actions torewrite the destination MAC address and the VLANID were too complex for the switches. One vendorfound a workaround for the issue by changing theorder of the actions specified in the flow entry. One of the OpenFlow switch implementations didnot support rewriting of the VLAN ID (set action); theOpenFlow version 1.0 specification indeed definesthis action as optional.

Recovery in OpenFlowMechanisms for recovery from a failure of theprimary data path are a key requirement for anytransport network technology. The OpenFlow specification defines the ability tomonitor the status of the ports of an OpenFlowswitch and modify the actions of the installed flowentries if the port status changes. A controller imple-mentation can use these features to implement theresiliency mechanism by defining primary andbackup paths. Thus, the controller can switch over tothe backup path when the port status of the primarypath changes to down.Each test combination involved three interconnectedOpenFlow switches — let’s call them Switch 1,Switch 2 and Switch 3. They were controlled by oneof the OpenFlow controller in each test combination.Two emulated hosts were connected to the switches.Host 1 was connected to Switch 1, Host 2 toSwitch 2. Figure 3 shows all tested combinations.

Figure 3: OpenFlow Failure Recovery Results

(legend see Figure 1)

We installed flow entries on OpenFlow switches suchthat the traffic directed to Host 2 was transmitted viaSwitch 3. With the traffic running, we removed thelink between Switch 1 and Switch 3. We expectedthat the controller would detect and localize such afailure. We also expected that the controller wouldmodify the flow entries in the switches such that theswitches would redirect the traffic to the backuppath. We proceeded to measure the out-of-servicetime during failover and recovery process.

The combinations with non-emulated controllersshowed out-of-service times of 70 ms once and821 ms another time. One controller implementationsupported the revertive mode of operation. For thisimplementation we measured 0 ms of service inter-ruption time during recovery.While we ran the test for the emulated OpenFlowcontroller, we tested only the service interruptiontime caused by the OpenFlow Switches. Wechanged the priority of the primary path to low andof the secondary path to high. This setting forced theswitch to redirect traffic from the primary to thebackup path. In this test run we measured a serviceinterruption time of 72 ms.In one case we observed that a switch did not deletea flow entry although the controller set the flow idletimeout to 5 seconds. We expected that the switchwould remove the flow entry after the idle timeoutwhen we stop sending the traffic matching that entry.

Policy Based Routing in OpenFlowA network-wide overview of the network topologyprovides the controller with the capability tocalculate an optimal path with constraints set by anoperator. As a constraint, the operator can definelinks that the controller must exclude or includeduring the path calculation. The controller caninstruct the OpenFlow switches to create flow entriesthat adhere to the calculated shortest path.

Figure 4: OpenFlow Policy Based Routing Results


Our test setup for this test case consisted of a rangeof combinations with three interconnected OpenFlowswitches each (again called Switch 1, Switch 2, andSwitch 3) which were controlled by an OpenFlowcontroller. Three emulated hosts were connected tothe switches. Host 2 and Host 3 were connected tothe Switch 2, and Host 1 to the Switch 1. First weapplied two constraints — one for traffic directed to

Extreme Summit X670

Ixia IxNetwork

HuaweiSOX

NEC PF6800

NEC PF5240

Huawei Openflow 1.2

OpenFlowController

OpenFlowSwitch

Extreme Summit X670

NEC

Spirent

Ixia IxNetwork

TestCenter

PF6800

HuaweiSOX

NEC PF5240

7

Clock Synchronization

Host 2, another for traffic directed to Host 3. Theconstraint for Host 2 traffic was to include the linkbetween Switch 1 and Switch 2. The constraint forHost 3 traffic was to exclude the link betweenSwitch1 and Switch2. When we enabled the OpenFlow channels, each ofthe controllers was supposed to learn the networktopology via LLDP initially. Afterwards we expectedthe controller to install flow entries in the corre-sponding switches dynamically, taking into consider-ation the constraints we specified before. Thedevices shown in Figure 4 participated successfullyin multi-vendor combinations.Additionally, NEC successfully demonstrated this testusing two NEC PF5240 and one NEC PF5248.

OpenFlow 1.2 DemonstrationTo further enable OpenFlow’s industry adoption andimprove the usability and scalability of OpenFlowbased SDN, OpenFlow 1.1 and 1.2 introducedadditional features. Multicast and multi-table flowsupport was added in the OpenFlow specification1.1; IPv6 and multiple controllers were added inOpenFlow specification 1.2.

OpenFlow 1.2: Multicast IPv6 and MLD (Multicast Listener Discovery) Snooping based on Multiple Flow TablesMost common hardware architectures supportmultiple forwarding tables that allow complexforwarding decisions by smaller number of entriesper table. OpenFlow 1.1 introduces multiple tablesthat allow more efficient utilization of the switch’savailable hardware resources.Huawei’s Smart Network Controller (SOX) success-fully demonstrated IPv6 multicast combined withdynamically created multi-table flow entries forMLDv2 snooping functionality.Huawei’s Openflow 1.2 switch correctly acceptedthose multi-table flow entries and fulfilled therequired functionality.

OpenFlow 1.2: Load Balancing and Failover Using Multiple ControllersThe multi-controller is a core feature introduced inOpenFlow 1.2. The feature addresses concernsabout single points of failure and the controller’sperformance in a single-controller setup. As a gener-alized OpenFlow1.2 and 1.0 controller, Huawei’sSmart Network Controller (SOX) successfully demon-strated its multi-controller capability. Huaweiconfigured two physical controller servers to run asequals in a cluster environment and to run logicallyas a single OpenFlow controller. Smart NetworkController (SOX) dynamically assigned new physicalcontroller servers to cope with increased controltraffic and dynamically released servers when trafficdropped.

OpenFlow 1.2 ResiliencyA new fast-failure buckets group type was intro-duced in OpenFlow 1.2. The mechanism enablesthe OpenFlow 1.2 switches to change theforwarding path without requiring a round tripcommunication to the controller. Huawei’s SmartNetwork Controller (SOX) correctly provided suchfunctionality with two Huawei Openflow 1.2switches. We noticed out-of-service times exceedingthe specified limit of 50 ms during failover andrecovery.

CLOCK SYNCHRONIZATION

In this event we progressed EANTC’s packet-basedclock synchronization interoperability test programinitiated in 2008. We focused on verifying imple-mentations of grandmaster, boundary, transparent,and ordinary (slave) clocks for frequency and phasesynchronization as specified in IEEE 1588-2008. Weused GPS to provide the reference clock.

Precision Time Protocol: Transparent ClockThe goal of this test case was to verify the synchroni-zation functions of IEEE 1588-2008 implementationslocated at grandmaster, transparent, and slaveclocks. The transparent clock was inserted into theprecision time protocol (PTP) flow path between thegrandmaster and the slave clock.To verify the ability of the PTP transparent clock toupdate each PTP packet with correct delay times, wegenerated bursty traffic on the link between thetransparent and the slave clocks. In addition, wemonitored the PTP messages that left the grand-master and transparent clock, respectively, andanalyzed the timestamps and frame formats.We measured the time interval error (TIE) at a sync-out interface of the slave clock (where available) forprolonged periods of time. Based on TIE, we calcu-lated MTIE (Maximum Time Interval Error) and timeof day deviation. We expected a frequencydeviation of no more than 16 ppb (parts per Billion)and phase deviation (time of day deviation) within±1.5 μs (microseconds). In addition we measured the floor packet population(FPP) metrics described in ITU-T G.8260. FPP is theratio of the received packet count within the clusterheight to the total received packet count. The clusterheight is a packet delay range between theminimum observed packet delay and the minimumobserved packet delay plus 150 μs. The FPP is calcu-lated over a window of 200 seconds.FPP measurements evaluate the packet delayvariation (PDV) at the network boundary, immedi-ately before the packet slave clock. An FPPmeasurement lower than the FPP limit indicatesexcessive PDV. We used the 1 % FPP goal asdescribed in ITU-T G.8261.1.

8


For the transparent clocks we also measured theaccuracy of the correction field by comparing thepacket delay with the residence time — the value thetransparent clock adds to the correction field of thePTP packets. We observed peak-to-peak correctionfield accuracy ranging from 1.3 μs to 26 μs. Figure 5 below shows all successful results.

Figure 5: Transparent Clock Test

A successful run required FPP values to correctlyexceed 1 % and all MTIE measurements to pass theG.823 SEC mask.Broadcom BCM956440 and BTI SA-810 passed the±1.5 μs limit as slave clocks. BroadcomBCM956440 and Extreme E4G-200 passed the±1.5 μs limit as transparent clocks.We encountered an interoperability issue between a1-step transparent clock and a 2-step slave clock.The transparent clock updated the correction fieldonly in the sync messages while the slave clock usedthe correction field value only of the follow upmessages. Therefore the slave clock could notproperly synchronize. According to IEEE 1588-2008the slave clock should use correction fields of bothsync and follow up messages.

PTP Boundary ClockBoundary clocks provide a means to distribute timein larger networks. The boundary clock relieves thegrandmaster from handling a large number ofordinary clocks.We advanced boundary clock tests this time: Inaddition to emulating an asynchronous networkbetween the grandmaster and the boundary clock,we emulated an asynchronous network between theboundary and the ordinary clocks at the same timeas well. This scenario is representative for boundaryclocks acting within asynchronous networks.In this test we set up a PTP clocking chain consistingof grandmaster, boundary and slave clocks.Between the grandmaster and the boundary clockand between the boundary and the ordinary clockswe emulated network paths of 10 nodes each. Theimpairment generator performed this emulation withan impairment profile according to ITU-T G.8261test case 12.We extended the setup this time by adding ameasurement point on the boundary clock output,complementing the one on the slave clock. Westarted the boundary and slave clocks from free-running state and allowed one hour for the nodes tolock. When the slave clocks were locked, we startedthe time interval error (TIE) and floor packetpopulation (FPP) measurements. We performed eachof the measurements for at least five hours.For MTIE calculation we excluded the clock stabili-zation period. The stabilization period following thelock acquisition on the slave clocks ranged from 45minutes to one hour.

Figure 6: Boundary Clock Test(legend see Figure 5)

We observed successful results for this challengingsetup: All measurements passed the ITU-T G.823SEC MTIE mask. In both successful test runs weobserved FPP correctly exceeding 1 % as expected.Ericsson MINI-LINK SP 210 passed the ±1.5 μsphase deviation limit.

BroadcomBCM956440

BroadcomBCM956440

ECINPT-1200

BTISA-810

Ericsson MINI-LINK SP 210

ECINPT-1200

12:50:00Symmetricom

SSU2000e

TransparentClock

SlaveClock

ExtremeE4G-200

BroadcomBCM956440

12:50:00Symmetricom

SSU2000e

Ixia Anue 3500

12:50:00Symmetricom TimeProvider

5000


5000

Paragon-XCalnex

Clock link

Packet Switched Network (PSN)

ReferenceClock

1PPS link

PTP EVC

Data EVC

12:50:00PTP

Analyzer/

Synchronous Node

Network Impairment

1 Gbit SyncE link

Analyzer

Impairment tool

Gradmaster

(slave or boundary clock)

BoundaryClock

ExtremeE4G-200

ECINPT-1200

SlaveClock




5000

IxiaAnue 3500

9

Clock Synchronization

Precision Time Protocol and Synchronous Ethernet — Hybrid ModeIn previous test events we concluded thatSynchronous Ethernet provides reliable and precisefrequency synchronization in multi-vendor scenarios.Meanwhile, Synchronous Ethernet (SyncE) hasturned out to be the first choice of network designersfor applications requiring high-precision frequencysynchronization.However, SyncE cannot provide phase or time-of-day synchronization. If SyncE-enabled networks arerequired to deliver phase synchronization as well,PTP and SyncE can be combined (“hybrid mode”).PTP will provide phase synchronization while SyncEprovides frequency synchronization.For hybrid mode tests, we reused the previous testsetup (see PTP Boundary Clock). We enabled SyncEon the data interfaces between the Grandmasterand the boundary clocks and between the boundaryand the slave clocks. We configured the ordinaryclocks to do frequency synchronization from theSyncE interfaces, and phase synchronization fromthe PTP signal.We started the test when the clocks were in the free-running mode. We allowed the clocks first to acquirea lock. We then simultaneously introduced packetdelay variation (PDV) based on ITU-T G.8261 testcase 14 and a sinusoidal wander for theSynchronous Ethernet signal with a frequency of 0.1Hz and 40 ns peak-to-peak amplitude — the latter tochallenge PTP implementations for which slowwander is more difficult to compensate.We expected the maximum time interval error (MTIE)to pass the G.8262 EEC Option 1 mask (frequency)and expected the phase deviation to remain within±1.5 μs.One combination successfully participated in this testas boundary clock (see Figure 7), three combina-tions succeeded as transparent clocks (see Figure 8).

Figure 7: Hybrid Mode PTP and SyncE: Boundary Clock Test


In one combination, we observed an issueconcerning the sequence numbering in signalingmessages.All maximum time interval error (MTIE) measure-ments passed the G.8262 EEC Option 1 mask andthe ±1.5μs phase limit.

Figure 8: PTP and SyncE Hybrid Mode: Transparent Clock Test


Precision Time Protocol: Best Master Clock SelectionThe Precision Time Protocol (PTP) provides amechanism for slave clocks to select the best masterclock in their respective synchronization domain.The selection is performed according to the PTP attri-butes of the grandmaster (e.g. PTP priority, clockclass).In our test scenario as shown in Figure 9, two grand-masters provided timing to PTP domain A. Theboundary clock was a slave clock in domain A anda master in domain B. The boundary clock wassynchronized with the best master in PTP domain Aand supplied clock signal downstream to the slaveclock in PTP domain B.We started the test when the boundary clock(Ericsson MINI-LINK SP210) and the slave clock (ECINPT-1200) were in a free-running state. As bothSymmetricom SSU2000e grandmasters weretraceable to the same reference clock and wereconfigured with identical priorities, the boundaryclock used the grandmaster identity as a tie-breakerwhen performing the best master clock selectionalgorithm. First, we induced a switching event by modifying theclock priority of grandmaster A. The Ericsson MINI-LINK SP210 boundary clock correctly detected thechange and switched over to grandmaster clock Bbecause its priority changed. In the next step, we used an impairment tool to dropall PTP messages originating from grandmaster B.

12:50:00SymmetricomTimeProvider

BTISA-810

BroadcomBCM956440

CalnexParagon-X

5000

BoundaryClock

SlaveClock

TransparentClock

SlaveClock


ECINPT-1200

ExtremeE4G-200

BTISA-810



CalnexParagon-X

Ixia Anue3500

Ixia Anue3500

12:50:00SymmetricomSSU2000e


5000


5000

10


The boundary clock correctly detected that grand-master B was unavailable and reverted back tograndmaster A.During the whole test duration we performed timeinterval error measurements of the signal coming outof the slave clock. We expected that the transitionsbetween the grandmasters had no major impact onthe clock quality and the maximum time intervalerror (MTIE) passed the ITU-T G.823 SEC mask.

Figure 9: Precision Time Protocol:Best Master Clock Selection


SyncE Islands Synchronization via PTPIn this test scenario, we verified clock transferbetween disjoined synchronous Ethernet domains bymeans of Precision Time Protocol (PTP), a commonuse case for operators using any kind of leased linesthat do not provide synchronization.

Figure 10: SyncE Islands Synchronization via PTP


As Figure 10 shows, we connected two SyncEnetworks (Islands) over an asynchronous network.The nodes on the border between the SyncE-enablednetwork section and the asynchronous networksection ran PTP and synchronized the SyncE islandswith each other. To emulate an asynchronousnetwork we used an impairment tool that introducedPDV to the network segment between the PTP masterand the PTP slave using ITU-T G.8261 test case 13.After all clocks acquired a lock, we started the timeinterval error measurements.In both successful test runs, the maximum timeinterval error (MTIE) passed the G.823 SEC maskbut not the G.8262 EEC Option 1 mask.

Transparent SDH/SONET over Packet (TSoP) Synchronization Across Carrier Ethernet ServiceThis time we extended the program with tests forTransparent SDH/SONET over Packet (TSoP). TSoPis described in draft-manhoudt-pwe3-tsop. The draftsubmitted to the IETF defines encapsulation of SDHframes over Ethernet packet networks in a structure-agnostic way. Applications of TSoP includemigration of legacy SDH/SONET access networkand upgrading STM-1 microwave links to Ethernetpacket network for mobile backhaul.As SDH/SONET is sensitive to frequency synchroni-zation, TSoP is in turn dependent on a supplementalclocking mechanism. Synchronous Ethernet (SyncE)is one of the options which we chose to employ inthis scenario.SFP modules implementing TSoP were inserted intoCarrier Ethernet devices, providing TSoP oversynchronous Carrier Ethernet service. The testedTSoP implementation used differential clocking. Thereference clock was provided via SyncE. The following figure depicts the successful combina-tions we verified.

Figure 11: TSoP Synchronization across Carrier Ethernet Service



12:50:00Symmetricom

SSU2000e

12:50:00Symmetricom

SSU2000e

ECINPT-1200

Ixia Anue3500

Ericsson MINI- ECI

SymmetricomSSU2000e

ECI BTI

SymmetricomTimeProvider 5000

IxiaAnue 3500

IxiaAnue 3500

LINK SP 210 NPT-1200

NPT-1200 SA-810

BTISA-810

BroadcomBCM956440

ExtremeE4G-200


Ericsson MINI-LINKSP 310/

Extreme E4G-200/

ECI NPT-1200/

AimValley

BTI SA-810/

SymmetricomSSU2000e

Smart SFP TSoPAimValley

Smart SFP TSoP

IxiaAnue 3500

CalnexParagon-X

Smart SFP TSoP

AimValleySmart SFP TSoP

OE Solutions — OE Solutions —

OE Solutions —OE Solutions —

AimValley

11

MPLS Multicast VPN

During the test execution we generated SDH trafficover TSoP and monitored the packet loss and SDHerror counters. We introduced impairment to the testtraffic flow. We used a packet delay variation (PDV)profile based on the ITU-T G.8261 test case 12 anda wander profile based on the ITU-T G.8262 TDEVEEC Option 1 mask for the SyncE signal. During thetest execution we observed no packet loss and noSDH errors. In one of the test steps we emulated a unidirectionallink failure by dropping all Ethernet TSoP frames.Both TSoP endpoints detected the failure byindicating remote and local packet loss, and theSDH traffic generator detected loss of frame (LOF)condition as expected.The OE Solutions — AimValley Smart SFP TSoPsuccessfully participated in the test as TSoP.BTI SA 810, ECI NPT-1200, Ericsson MINI-LINK SP310 and Extreme E4G-200 successfully participatedas SyncE nodes and TSoP SFP module hosts.

MPLS MULTICAST VPNAs the worldwide demand grows for multicastservices, we decided to test the multi-vendor interop-erability of new innovative MPLS multicast technol-ogies in this year’s MPLS interoperability test. Multicast VPN (MVPN) is a multicast servicetechnology specified by the IETF in a series of speci-fications RFCs 6513, 6514, 4659 and 4875. TheIETF combined a range of industry ideas in theseRFCs, making their choice optional. There are twomajor ways to implement multicast for MPLS Layer 3VPNs across service provider networks: Either usingPIM or multi-protocol BGP (MP-BGP) between MPLSProvider Edge (PE) routers. The first method is ofteninformally called as draft-rosen, the later method asthe next generation (NG) MVPN. NG MVPN maintains the multicast state informationbetween the PE routers using the same architecturethat is used for unicast VPNs. Another advantage ofdeploying MVPN with MP-BGP is the ability to reusetraffic engineering features provided by MPLS andBGP. MVPN requires interoperability on multiplelevels for different protocols and options, such asRSVP-TE, mLDP, BGP, inclusive and selective trees,etc. Interoperability is a key requirement for anysuccessful multi-vendor deployment. For two weeksduring hot-staging, we tested some of the options.We publicly tested full-mesh interoperability of NGMVPN between Alcatel-Lucent, Cisco, JuniperNetworks, and Ixia for the first time in the industry.We verified functionality and two resiliency options:BGP-based and RSVP-TE-based.

IPv6 MVPN using P2MP TunnelsThe move towards larger scale multicast serviceshappens in parallel to IPv6 migration. We testedNG MVPNs together with IPv6.

Table 1: IPv6 MVPN P2MP Tunnel Test Parameters

As the first step during the test execution, vendorsconfigured PEs with one of the profiles representedin Table 1. We enabled MP-BGP on all PEs. OnceBGP sessions were established, we verified theunicast and multicast Virtual Router Forwarding(VRF) functions on each PE. We expected each PE toinstall unicast routes to remote PEs and to the CEmulticast receivers. We verified that each PE adver-tised and installed MP-BGP route type 1, also calledIntra-AS I-PMSI A-D route in the MVPN VRF. In thesecond step, we enabled a multicast source andexpected the PE connected to the multicast source toadvertise MP-BGP route type 5 Source Active A-Droute. We verified on each PE that route type 5 wasinstalled in MVPN VRF. Next, we enabled multicast receivers and generatedmulticast test traffic at the source. We verified thatthe PEs converted the customer multicast joinmessages to BGP c-multicast routes (type 6 and 7,also called Shared Tree Join and Source Tree Joinrespectively) and that the traffic was forwarded tothe receivers that joined the multicast tree. We alsoverified that the PE installed BGP c-multicast routes(type 3 and 4, also called S-PMSI A-D route and LeafA-D route) if the traffic was forwarded using S-PMSItunnels. As the last step, we disabled multicast receivers andverified that PEs pruned all BGP c-multicast routesexcluding route type 1 and type 5.

ParametersProfile 1(RSVP-TE)

Profile 2(mLDP)

Core Network IP Version IPv4PE-CE Unicast Routing StaticPE-PE Unicast Routing OSPF

PE-CE Multicast Routing IPv4: PIM-SMIPv6: MLDv2

MLDv2

PE-PE Multicast Routing MP-BGPAuto Discovery Protocol enabledTunnel Type P2MPP-Tunnel Signaling RSVP-TE mLDPP-Tunnel Encapsulation MPLS

PMSI Tunnel Type Inclusive and Selective Inclusive

Customer Multicast Route

(*,G) Static Rendezvous point (C-RP) configured on all PEs

Customer Multicast Traffic Type

Dual stack IPv4 & IPv6 IPv6

12


Figure 12: IPv6 MVPN Using P2MP Tunnels

Alcatel-Lucent 7750 SR-12, Cisco ASR 9000 andJuniper MX-480 jointly successfully participated inthe test when profile 1 was configured and RSVP-TEP-Tunnel signaling was enabled. Note that, as partof the initial staging for this profile vendors validatedIPv4 multicast interoperability using IPv4 versions ofProfile 1. Ixia IxNetwork emulated IGMP and PIM-SM at this stage. The test continued by verification ofIPv6 for Profile 1, where Ixia IxNetwork emulatedMLDv2. Alcatel-Lucent 7750 SR-12, Cisco ASR 9000, CiscoCRS-1, Ixia IxNetwork and Juniper MX-480 success-fully tested profile 2 when mLDP P-Tunnel signalingwas enabled. In this scenario Ixia IxNetworkemulated a PE router as well as MLDv2 multicastsource and receivers.

During the test setup and test configuration, weobserved several issues: 1. There was a TypeValue mismatch among

vendors. The pre-RFC implementations by twovendors support by default VRF Route-ImportExtended Community type as it was specified inthe draft version (0x010a); IETF later changedthe type value when MVPN was released as anRFC (0x010b). As a result of the mismatch, MP-BGP route types 6 and 7 were not generatedand the multicast traffic was not forwardedbetween the routers. One of the two vendors wasable to switch to RFC-compliant encoding usingCLI knob, the other modified the implementationduring the event (with a future configurationexpected). With these modifications, interopera-bility was achieved between all vendors.

2. During the test preparation the participatingvendors recognized that there are two differentincompatible PMSI Tunnel Attribute (PTA)encodings defined by RFC 6514 and RFC 4875,respectively. The Tunnel Identifier starts withExtended Tunnel ID and ends with P2MP ID asper RFC 6514. The order of the fields is reversedin RFC 4875. To ensure interoperability betweenvendors, RFC 4875 encoding was chosen.

3. During MVPNv6 test verification, we observedthat one of the implementations was unable toact as an MLDv2 router. To resolve the issue, therouter was configured with static MLD joins, andIxia IxNetwork emulated the multicast receivers.

4. During the profile 1 verification, we observedthat when one of the CEs sent PIM Joins, anotherCE also incorrectly received traffic for a fewseconds. According to the vendors, a potentialexplanation was that one of the PEs wasconfigured to join MLD statically; as a result, itcould forward the traffic received on I-PMSI tocorresponding CEs. The PEs would leave thegroup when the BGP timer expired.

5. During the test verification, we noticed thatRFC 6513 section 10 proposes two methods ofsupporting PIM-SM in ASM (Any SourceMulticast) mode. One mode (Source-Specific C-tree) calls for sending BGP type 6 and type 7routes from Egress-PEs. While the other (SharedC-trees) calls for only sending type 7 messages.We observed during the testing that the partici-pating implementations had a different defaultbehavior to handle the ASM service model andthat caused situations where joined receiverswould not get multicast traffic. The issue wascorrected with proper configuration alignment.

6. We observed that one of the implementationsalways built a Shortest Path Tree (SPT) to theMulticast Rendezvous Point. As a result no routetype 6 was advertised to the PE sender when I-PMSI tunnels were used.

MVPN RSVP-TE Profile

Alcatel-Lucent7750 SR-12

CiscoASR 9000

JuniperMX-480

CiscoCRS-1

IxiaIxNetwork


JuniperMX-480

CiscoASR 9000

MVPN mLDP Profile

Core Network

Emulated PIM-SM/MLDv2 customer network

Emulated multicast source network

Provider device /Provider Edge device

Impairmenttool

Emulator

Aggregationnode

MPLS-TP network

Access network

Link failure

13

MPLS Multicast VPN

IPv6 Multicast VPN Resiliency Service providers require resiliency solutions fornetwork services they offer. There are two options forresiliency in MVPN: Dual-homing for multicastsource and protection of PE-PE data path.In our first resiliency scenario, we tested a dual-homed multicast source. We derived the test setupfrom the MVPN functionality test case (see »IPv6MVPN using P2MP Tunnels« above) when Profile 1was configured. Each PE generated one route for itscorresponding CE-PE connection to the multicastsource. The receiving PE executed the BGP best pathalgorithm to select an optimal path to the multicastsource. During the test we shut down the interface betweenthe multicast source and the primary PE sender. Weexpected the multicast traffic be forwarded to themulticast receiver via redundant PE with low trafficloss during the convergence. Finally we tested therecovery case as we brought back the shut downinterface. In this step we expected the multicasttraffic to be sent out via primary PE sender with lowtraffic loss during the convergence.Figure 13 represents participants that successfullyparticipated in the dual-homed MVPN resiliencyscenario. We measured an out-of-service time of 4.8seconds in a failure case and no loss duringrecovery.

Figure 13: IPv6 Multicast VPN Resiliency via BGP

One of the P-tunnel methods described in RFC 6514is RSVP-TE. In case that RSVP-TE is used for P-Tunnelsignaling, the Traffic Engineering capabilities suchas Fast Reroute can be used to provision resiliencyfor MVPN. We verified this capability in our secondresiliency scenario. We used the same test setup asfor IPv6 MVPN using P2MP Tunnels with Profile 1and MPLS FRR link protection configured. Once weverified that the MVPN was operational and thetraffic was forwarded to the multicast receiverswithout traffic loss, we emulated a link failurebetween the forwarder PE and one of the receiverPEs. We expected the multicast traffic to beforwarded via an intermediate PE to the multicastreceiver with low traffic loss.Figure 14 represents successful test results for thesecond MVPN resiliency scenario.During the failover phase we observed duplicated

multicast traffic on the back-up link inside the corefor a couple of seconds — which was expectedbecause of FRR link protection. It took severalseconds for the network to converge. During theconvergence we observed duplicated traffic on a CEfor about 740 ms. We observed no traffic loss norduplication of the traffic in the recovery phase.

Figure 14: IPv6 Multicast Resiliency via RSVP-TE Fast Reroute

MPLS-TP and IP/MPLS InterworkingIn this test we verified vendor progress when itcomes to interconnecting MPLS-TP and IP/MPLSnetworks.We verified two test scenarios, multi-segmentpseudowires (MS-PW) and H-VPLS. In the firstscenario we composed a point-to-point service oftwo PW segments, one in an IP/MPLS domain andone in an MPLS-TP domain. We connected bothdomains to a stitching point, a device capable of IP/MPLS and MPLS-TP interworking. In the secondscenario, we configured a VPLS service in IP/MPLSand extended it with spoke PWs within the MPLS-TPdomain. We configured the Virtual Switch Instance(VSI) on the stitching point device.In both scenarios, we configured the PW segmentsstatically in the MPLS-TP domain. For the IP/MPLScontrol plane we used OSPF as the IGP protocol,LDP as the LSP signaling protocol and T-LDP as theservice signaling protocol.We verified that the services in the IP/MPLS domainand the MPLS-TP domain were established on eachdevice. We verified the mapping between IP/MPLSand MPLS-TP in the stitching point device. Next, wegenerated bidirectional traffic using the traffic gener-ators connected to the Access Circuits (ACs). Weobserved no traffic loss to the service traffic.The Cisco ASR 9000 successfully participated in thetest as a stitching point. Cisco ASR 9000, ECI NPT-1200 and Ericsson MINI-LINK SP 210 successfullyparticipated in the test as MPLS-TP nodes. Cisco ASR9000 successfully participated in the test as an IP/MPLS node.Ixia IxNetwork successfully emulated MPLS-TP andIP/MPLS nodes, and Spirent TestCenter successfullyemulated MPLS-TP nodes.Figure 15 depicts the successful combinations weverified.

JuniperMX-480


CiscoASR 9000



JuniperMX-480

CiscoASR 9000



14 15

Topology

Physical Network TopologyMulti–Vendor SDN, IPv6, MPLS & Carrier Ethernet Interoperability Event 2013

Network Areas Connection TypesDevice TypesProvider/Provider Edge device

IPv4/IPv6/Ethernet CE/CPE

Impairment tool & clock analyzer IP/MPLS core network

Clock link

Ethernet link

Aggregation device

Access network

Emulator

12:50:00 IEEE 1588v2 Grandmaster

MPLS–TP aggregation network

IxiaAnue 3500

Smart SFP TSoP

ContentProvider

ExtremeE4G–200

OE Solutions — AimValley

EricssonMINI–LINK SP 420

100 Gbit/s

LinksysE4200

IxiaIxNetwork

SpirentTestCenter Linksys

WRT54GL

OpenFlow–based network

OpenFlow managment network

SDNControllers

HuaweiSmart Network

NEC PF6800

Controller (SOX)

IxiaIxNetwork

SpirentTestCenter

Internet

12:50:00Symmetricom

TimeProvider 5000

NECPF5240

ExtremeX670



CiscoASR 9000

Smart SFP OAM

EX14


AimValley

IxiaIxNetwork

IxiaIxNetworkIxia

IxNetwork

EricssonSSR 8010


ECINPT–1200

CiscoASR 9000


HitachiAMN 6400

CiscoASR 9000

CiscoCRS–1

IxiaIxNetwork

JuniperMX–480

Alcatel–Lucent7750


IxiaIxNetwork

NECPF5240

NECPF5240

NECPF5248

ECINPT–1200



BTISA–810

Smart SFP TSoP

12:50:00Symmetricom SSU2000e

BroadcomBCM956440

HuaweiOF Switch 1.2

Ethernet aggregation network

16


Figure 15: MPLS-TP and IP/MPLS Interworking

MPLS-TP 1:1 OAM based ProtectionResiliency in an MPLS-TP network is one of the keyrequirements and reasons the protocol family wasinvented. In this year’s test, we focused mainly onRFC 6378, i.e. Protection State Coordination (PSC)implementations. However, we tested vendors’implementations for 1:1 LSP protection without PSCsupport as well. In the test we built two MPLS-TP LSPs using distinctlinks between a pair of devices. We configuredMPLS-TP 1:1 protection for these LSPs. A servicepseudowire (PW) used the 1:1 protected LSP. BFDContinuity Check (CC) sessions ran on both workingand protection LSPs to monitor the liveliness of theLSPs. The devices transmitted BFD-CC over GenericAssociated Channel (G-ACh) using GenericAssociated Label (GAL). For the test scenarios weused BFD with or without slow start procedure. Weconfigured the BFD intervals to either 3.33 ms (BFDmultiplier set to 3) or 15 ms (BFD multiplier set to 2).When we generated bidirectional test traffic with aconstant frame rate, we enabled bidirectionalimpairment and measured the out-of-service timeduring failover and recovery process.We expected the out-of-service time to be less than50 ms based on standard industry expectations.The following devices participated in this test case:Cisco ASR 9000, ECI NTP-1200, Ericsson MINI-LINK SP 310, Ericsson SSR 8010, HitachiAMN6400, Ixia IxNetwork.

Figure 16: MPLS-TP 1:1 OAM Based Protection

The following devices successfully participated in thefailover part of the test: Cisco ASR 9000, ECI NTP-1200, Ericsson MINI-LINK SP 310, Ericsson SSR8010, Hitachi AMN6400, Ixia IxNetwork.The following devices successfully participated in therecovery part of the test: Cisco ASR 9000, ECI NTP-1200, Ericsson MINI-LINK SP 310. For all successful test scenarios, we observed lessthan 50 ms out-of-service times for failover andrecovery.For the test combination with ECI NTP-1200 andEricsson MINI-LINK SP 310, we successfully testedfailure protection switching and MPLS-TP adminis-trative commands based on RFC 6378. During thefailover we measured 12–14 ms and during therecovery 11–14 ms out-of-service time. Interopera-bility of the following administrative commands wassuccessfully verified: Lockout of protection, ForcedSwitch and Manual Switch. For the test combination with Cisco ASR 9000 andEricsson MINI-LINK SP 210, we tested only thefailure protection switching. During the failover wemeasured 22.8–26.9 ms and during the recovery 0–8 ms out-of-service time.For the test run between Ericsson MINI-LINK SP 310and Hitachi AMN6400 we measured 9.8 ms out-of-service time during the failover. For the test run between Hitachi AMN6400 and IxiaIxNetwork, we used Ericsson SSR 8010 as interme-diate node. We measured 10.2 ms out-of-servicetime during the failover.

Static PW


LDP signaled Pseudowire (PW)

LDP signaled VPLS

IxiaIxNetwork

SpirentTestCenter

CiscoASR 9000

ECINTP-1200

IxiaIxNetwork

CiscoASR 9000

CiscoASR 9000

CiscoASR 9000

CiscoASR 9000

CiscoASR 9000

CiscoASR 9000

CiscoASR 9000

IxiaIxNetwork

CiscoASR 9000

EricssonSSR8010

Ixia Anue3500

CalnexParagon-X


ECINTP-1200

HitachiAMN6400

CiscoASR 9000


HitachiAMN6400

IxiaIxNetwork

Working path Protection path


17

Topology

In some link recovery test runs we were unable tovalidate automatic switchover: When the failed linkwas restored, one of the devices would switch to theprimary path while the other would stay on theprotection path. In such cases we observed 100 %loss in both directions until the switchover to theprimary path was performed by manual configu-ration. The vendors affected by this behavior that theissue was caused by implementation of different PSCversions. While one vendor supported version 0, theother supported version 1 of PSC. According toRFC 6378 the version field in the PSC messagesmust be set to 1.

MPLS-TP Fault ManagementMPLS-TP fault management provides tools that canautomatically indicate a disruption on a link or nodealong the data path to an MPLS-TP LSP endpoint.This indication is important to quickly identify faultsand the root cause of the failure as well as triggerprotection mechanisms. Request for comments (RFC)6427 defines the MPLS OAM channel andmessages to communicate various types of servicedisruptive conditions.An implementation raises the MPLS-TP AlarmIndication Signal (AIS) message and correspondingLink Down Indication (LDI) flags when it detectsdefects on the lower layers. An MPLS-TP devicegenerates a Lock Report (LKR) when an operatoradministratively shutdown a link. In this test, wetested LDI and LKR alarms.

Figure 17: MPLS-TP Fault Management

We tested LDI for all vendor pairs as shown in Figure17. We tested LDI by disconnecting one of the linksalong the data path. The intermediate node propa-gated the AIS (L-Flag=1, R-Flag=0) message as soonas the link was down. The node generated an AISClear (L-Flag=1, R-Flag=1) message as soon as thelink was up.

In addition, we tested LKR for two vendor pairs asshown in Figure 17. We tested LKR by administra-tively locking one of the links along the data path.The intermediate node propagated the LKR messageas soon as the link was administratively locked vianode’s CLI. The node generated an LKR Clearmessage as soon as the link was administrativelyunlocked via node’s CLI.The following devices successfully participated inthis test case: Cisco ASR 9000, ECI NTP-1200,Ericsson MINI-LINK SP 210, Hitachi AMN6400 andIxia IxNetwork.

MPLS-TP on Demand OAMOAM tools are essential to the upkeep of a network— they provide fundamental operational control andtroubleshooting mechanisms. IETF RFC 5860specifies the OAM requirements for MPLS-TP — theOAM mechanisms must operate in-band, and mustnot rely on a control plane.OAM tools like LSP ping and traceroute provide on-demand connectivity verification (CV) and pathtracing functions and enable to isolate failures, andthus simplify troubleshooting and reduce the time todetermine the root cause of a failure. LSP ping andtraceroute were standardized in IETF RFC 6426.We tested LSP ping and traceroute commands fromone end point of the MPLS-TP LSP using CLI. Weexecuted both commands in two network conditions,with and without an emulated link failure on LSPpath. OAM messages were sent over the GenericAssociated Channel (G-ACh) without IP encapsu-lation.We started the test by verifying the functionality ofthe bidirectional LSP with generation of bidirectionaltraffic between the traffic generators. Afterwards,we verified the functionality of the OAM tools andobserved the correct echo response messages withLSP ping and the list of nodes in the path with LSPtraceroute.We emulated a link failure and repeated thecommands. We verified that LSP ping timed out andLSP traceroute localized the problem.Cisco ASR 9000, Cisco ASR 9000 and EricssonMINI-LINK SP 210 successfully participated in thetest as shown in Figure 18.

Figure 18: MPLS-TP BFD on Demand OAM

Bidirectional LSPLDI

LDI/LKR

CiscoASR 9000

CiscoASR 9000

ECINTP-1200

SpirentTestCenter

IxiaIxNetwork

CiscoASR 9000


ECINTP-1200

ECINTP-1200

ECINTP-1200

HitachiAMN6400

Ixia Anue3500

HitachiAMN6400

Echo Request

CiscoASR 9000

CiscoASR 9000


Bidirectional LSP

18


MPLS-TP Static-Pseudowire RedundancyLSP linear protection often provides sufficient resil-iency to an MPLS-TP network. However, service-aware resiliency is still desired in MPLS-TP. This testcovered pseudowire (PW) redundancy in the MPLS-TP domain when the PW control plane is notavailable. RFC 6478 and RFC 6870 define a mechanism tosignal the standby status of a redundant PWsbetween their terminating nodes, using the prefer-ential forwarding status bit. The PW status is encap-sulated in a PW OAM message and transported inband with the PW data using the GenericAssociated Channel (G-ACh).We generated bidirectional traffic between theaccess circuits of the active and standby PWs. Weverified that the traffic was forwarded only via theactive PW. While the traffic was running, weemulated a link failure of the access circuitconnected to the active downstream node. Thedevices detected the failure and signaled to theupstream node and switched the traffic to theprotection PW. We measured 1.29 seconds out ofservice time during the failover protection switching.We restarted the traffic on the traffic generators andreconnected the previous removed link. We verifiedthat the devices detected the link restoration andsignaled that to the upstream node. We verified thatthe devices switched the traffic to the working PW.We measured 0.99 seconds out of service time forrecovery.Figure 19 shows the test setup for static pseudowireredundancy.

Figure 19: MPLS-TP Static-Pseudowire Redundancy

The following devices participated in the test success-fully: Cisco ASR 9000 and Ericsson MINI-LINKSP 210.During the tests, we observed that the channel typepoint code was set to 0x0022 in the G-ACh header,as set in draft versions of the specification. The IETFRFC 6478 defines 0x0027 as the code point valuefor PW OAM Messages.

Ethernet VPNEthernet VPN (E-VPN), currently under standard-ization by the IETF, replaces the data plane-basedMAC learning with control plane-based MAClearning. With E-VPN, PEs advertise MAC addressesto other PEs using Multi-Protocol BGP (MP-BGP) andVPN traffic is transported over MPLS label switchedpaths (LSPs). Provider Backbone Bridging E-VPN(PBB-EVPN) is an application of PBB (IEEE 802.1ah)to E-VPN.E-VPN and PBB-EVPN are designed with interopera-bility in mind by using the same Network LayerReachability Information (NLRI) messages whereapplicable. For example, the EVPN NLRI route type2 (MAC Advertisement Route) is used by EVPN PEsto advertise customer MAC addresses, while PBB-EVPN PEs use them to advertise backbone MACaddresses.Our test setup consisted of two PE router configuredwith PBB-EVPN in a common EVPN instance. Trafficgenerators were connected at the CE interfaces onboth routers associated to the EVPN instance.We started the test by enabling OSPF, LDP and MP-BGP sessions between the two routers and expectedto observe that all protocols sessions were estab-lished and that the BGP peers exchange correct MP-BGP EVPN NLRIs. In the next step we expected toobserve the exchange of BGP MAC AdvertisementRoute (type 2 route) carrying the relevant MACaddresses for known unicast and an InclusiveMulticast Ethernet Tag Route (type 3) for multicast orunknown unicast. We generated traffic between thetwo edge interfaces and expected to observe notraffic loss.The following figure depicts the successful test runwe verified.

Figure 20: Ethernet VPN

Cisco ASR 9000 successfully participated as a PBB-EVPN PE. Ixia IxNetwork successfully emulated aPBB-EVPN PE.During the test we also attempted to send multicasttraffic. However, we observed that although type 3Inclusive Multicast Ethernet Tag Route were correctlyexchanged, multicast packets were not forwardedon the data plane.

Working PW Protection PW

UNI

UNI

CiscoASR 9000


CiscoASR 9000

UNICisco

ASR 9000Ixia

IxNetwork

MPLS LSP

PBB EthernetVPN PE

PBB EthernetVPN PE

19

IPv6 Tests

IPV6 TESTS

The Internet community is facing multiple generalInternet Protocol version 4 (IPv4) issues — theexhaustion of IPv4 addresses is probably the mostprominent one. IPv6 technology promises to solvethe issue. The migration from IPv4 to IPv6 in serviceprovider networks is a long-lasting process and cannot be performed in one step due to the huge infra-structures involved worldwide.The IETF defined multiple migration scenarios. Themost prominent are Dual-Stack, Dual-Stack Lite and6rd. During this event we concentrated on testingDual-Stack Lite and 6rd.We also tested a recent addition to the family of IPv6migration technologies, Mapping of Address andPort (MAP), which positions itself as a statelesscounterpart to Dual-Stack Lite. MAP does not requirestateful Carrier Grade NAT (CGN) to map IPv4 toIPv6 addresses. MAP as a stateless solution promisesto solve scalability issues of stateful CGN in bigdeployments.Exponential growth of the IP routing tables is anotherserious issue. However IPv6 does not address thescalability problems of the routing tables. Migrationto IPv6 can even accelerate the tables growth.The issue was identified and documented in IETFRFC 4984, section 7. The root cause of the problemwas determined to be the overloading of the IPaddress semantic — hosts identification — byadditional semantics like location identification andtraffic engineering.The IETF working group is currently working onLocator/Identifier Separation Protocol (LISP) as asolution. The LISP working group published sevenRFC documents in January this year, including thebase LISP specification (RFC 6830) and interworkingbetween LISP and non-LISP sites (RFC 6832), aimingto solve the address overloading issue. During ourevent we tested the LISP protocol in IPv6 ServiceProvider scenario.

6rd — Rapid Deployment on IPv4 InfrastructureStandardized in IETF RC 5969, 6rd consists of twomain components, the 6rd Customer Edge (6rd CE)and 6rd Border Router (6rd BR).The 6rd CE provides dual stack connectivity towardsthe home user’s LAN and handles the tunnel encap-sulation and the mapping between IPv4 and IPv6addresses towards the provider network. On theprovider side, the 6rd BR serves as the gateway tothe IPv6 Internet and terminates the tunnels of theindividual 6rd CEs.6rd maps IPv4 addresses to IPv6 addresses. TheIPv4MaskLen parameter defines which part of theIPv4 address is encoded in the IPv6 address.We tested two scenarios. In the first scenario,IPv4MaskLen was set to zero, i.e., the entire IPv4address was encoded in the IPv6 address and theaddresses were allocated out of a global IPv4

address pool. In the second scenario, the addresseswere allocated out of a private IPv4 network andIPv4MaskLen was set to 8. The 24 least significantbits of the IPv4 addresses were encoded in the IPv6address. CGN was used to perform translationbetween the private addresses (10.0.0.0/8) andglobal Internet addresses.Our test setup consisted of two 6rd CEs, a DHCPserver and a 6rd PE. We generated traffic betweenthe 6rd CEs and the emulated IPv4 and IPv6 Internetprovided by the 6rd BR.We started the test execution by activating DHCP onthe 6rd CEs. In the next step we observed the correctIPv4 address and 6rd BR address configuration. Wethen enabled the emulated user’s home devicesconnected behind the CPE and expected to observethe appropriate IPv4 and IPv6 addresses allocatedon them. Finally we generated fully meshed IPv4 andIPv6 traffic and expected to observe no traffic loss.The following diagram depicts the successful permu-tations we verified:

Figure 21: 6rd — Rapid Deployment on IPv4 Infrastructure

As depicted in Figure 21, Cisco ASR 9000 success-fully participated in the test as 6rd BR. Cisco ASR1000 successfully participated as DHCP server.Juniper MX-480 successfully participated as both6rd BR and DHCP server.Ixia IxNetwork and Spirent TestCenter successfullyemulated the 6rd CE.

6rd Tunnel

CiscoASR 1000

JuniperMX-480

DHCP Server

CiscoASR 9000

SpirentTestCenter

LinksysE4200

JuniperMX-480

IxiaIxNetwork

LinksysE4200

SpirentTestCenter

LinksysE4200

6rd BR6rd CPE

Packet Switched

Router

CPE

Emulator

Internet

Network (PSN)

20


Dual-Stack Lite Broadband DeploymentsWhile some IPv6 migration scenarios are designedfor core networks running IPv4, Dual-Stack Lite (DS-Lite) addresses the opposite scenario — an IPv6 corenetwork with residual IPv4 deployment. Dual-StackLite is specified in IETF RFC 6333. The two basiccomponents of DS-Lite are the address family trans-lation router (AFTR) and a DS-Lite capable CPE.We connected a DS-Lite-capable CPE through anIPv6-only transport to the AFTR. We attached twoendpoints to the external interface of the AFTR, oneemulating an IPv4-only connection and the other anIPv6-only connection. Behind the DS-Lite CPE, weconnected a Dual-Stack endpoint. We generatedbidirectional traffic from the internal to the externalnetwork and expected no traffic loss.We configured the AFTR to allow up to 10 flows foreach subscriber, and generated 11 flows. Weexpected to observe no traffic loss for 10 flows andthe traffic of the remaining flow not to be forwarded.We also expected to observe logging messagesdescribing the creation and expiration of flows,along with the respective IP addresses and portnumbers.The following diagram depicts the successful permu-tations we observed in this test.

Figure 22: Dual-Stack Lite Broadband Deployments

Alcatel-Lucent 7750 SR-12 and Juniper MX-480successfully participated in the test as AFTR. IxiaIxNetwork and Spirent TestCenter successfullyemulated a DS-Lite CPE.

Port Control Protocol (PCP) in Context of Carrier-Grade NAT (CGN) for Dual-Stack LiteIn Dual-Stack Lite, IPv4 subscriber addresses aremapped to the global addresses by Carrier GradeNAT (CGN). If subscribers would like to receivetraffic from the Internet, port forwarding — similar tothe home routers performing NAT — is required. AsCGN is located in the provider network, forwardingconfiguration require a method to convey the port

mapping information to the CGN.The Port Control Protocol (PCP) offers such asolution. The base protocol is still beingstandardized in the IETF work in progress draft-ietf-pcp-base.To request the binding for incoming traffic at theCGN, the PCP client sends a MAP request. One ofthe PCP modes enables a CPE device to send thebinding information on behalf of a host connectedbehind it by using the third party PCP option.The PCP lifetime parameter specifies the amount oftime in seconds, during which the binding shall beremain valid.In our setup, an emulated CPE is connected to theAFTR, acting as a CGN and a PCP server. Wegenerated a PCP MAP request and generatedunidirectional traffic from the external network to theinternal host. We expected no traffic loss for thetraffic. After the PCP lifetime expired we expected alltraffic to be dropped.The following figure depict the test setup andsuccessful combinations.

Figure 23: Port Control Protocol (PCP) in Context of Carrier-Grade NAT (CGN) for

Dual-Stack Lite

Alcatel-Lucent 7750 SR-12 and Juniper MX-480successfully participated in the test in the role of theAFTR and PCP server. Ixia IxLoad successfullyemulated the DS-Lite CPE and PCP Client.We observed different behavior of the CGN withregards to the PCP lifetime parameter. One imple-mentation started the timer immediately following theMAP binding, while another started the timer aftertraffic stopped and restarted the timer if it receivedtraffic matching the binding. We also observed anissue with the PCP client, which allowed the PCPrequest to be sent to predefined ports only. Wealleviated the issue by using a tool of the emulator toedit the request.

IPv6 Migration: Mapping of Address and PortIPv6 migration scenarios commonly involve IPv4address sharing. Unless specifically addressed bythe migration technology, it implicitly requires statefultranslation provided by CGN. Mapping of Address and Port (MAP) is an alter-native to the CGN and aims to provide statelesstranslation of IPv4 address and port to IPv6 address.

JuniperMX-480


SpirentTestCenter

IxiaIxNetwork

AFTRDS-Lite CPE

DS-lite tunnel

Linksys+OpenWRT

JuniperMX-480


IxiaIxLoad

DS-Lite CPE/PCP Client

DS-lite tunnel

IxiaIxLoad

PCP MAP

AFTR/PCP Server

21

IPv6 Tests

Due to its stateless nature, MAP may solve scalabilityissues of stateful CGN.Configuration of MAP starts with the creation of abasic mapping rule (BMR). The BMR specifies howthe translation is performed: how many of the IPv4address bits will be used, how many ports isallocated to each subscriber, how many ports arereserved and whether the addresses are allocated incontiguous blocks.MAP has two variants. The encapsulation mode,referred to as MAP-E, uses IPv6 tunnels to enclosethe IPv4 packets. The translation mode, referred toas MAP-T, maps the IPv4 packet header to the IPv6header. MAP is a work in progress IETF draft,currently in the process of standardization in draft-ietf-softwire-map (MAP-E) and draft-ietf-softwire-map-t(MAP-T).Our test setup was designed for both MAP-E andMAP-T scenarios and consisted of a MAP borderrouter (MAP BR) and two MAP CPEs in mesh mode.We generated traffic between the MAP domain andthe external network as well as between the twoMAP CPEs. We verified the correct mapping of portnumbers for IPv4 traffic, while IPv6 traffic wasnatively forwarded. We expected the traffic betweenthe MAP CPEs to traverse directly without involvingof the MAP BR.Cisco ASR 1000 and Cisco ASR 9000 successfullyparticipated in the test in the role of a MAP-T BR.Cisco ASR 9000 successfully participated in the testin the role of a MAP-E BR. Cernet CPE successfullyparticipated in the test in the role of a MAP-T CPEand a MAP-E CPE.During the tests we observed that the CPE calculatedthe UDP checksum incorrectly while performing IPv6traceroute.The following figure depicts the successfully verifiedtest combinations in this test case.

Figure 24: IPv6 Migration: Mapping of Address and Port

Locator/ID Separation Protocol in IPv6 MigrationIn the LISP architecture, recently standardized in IETFRFC 6830, the location and identity of hosts aredecoupled. A routing locator (RLOC) identifies a site,typically assigned to a particular networkattachment point. The endpoint ID (EID) identifies ahost, which is associated to an RLOC.To communicate with the host, traffic is sent to theRLOC. A LISP map server is tasked with resolving theEID address to an RLOC address.LISP defines two major functions. The ingress tunnelrouter (ITR) is a router that is located within a LISPsite. It performs an EID-to-RLOC lookup, encapsulatesthe packet and forwards it according to the RLOC.The egress tunnel router (ETR) is a router that has atleast one RLOC associated to it. When it receivestraffic to one of its RLOCs, it strips the RLOC andforwards the packet to the destination (EID) withinthe LISP site. An xTR is a router which performs boththe ITR and ETR functions. LISP also defines the proxyingress tunnel router (PITR) and a proxy egress tunnelrouter (PETR) functions. A PxTR is a proxy device thatimplements both the PITR and the PETR functions. ThePxTR is standardized in IETF RFC 6832 and allowscommunication between hosts connected via LISPand hosts connected via non-LISP CPEs.

Figure 25: Locator/ID Separation Protocol in IPv6 Migration

MAP CPECernet CPE

MAP-T MAP-E

MAPBorder Router

CiscoASR 9000

CiscoASR 9000

CiscoASR 1000

CiscoASR 9000

LISP Map Request

UPCLISPmob

UPCLISPmob

CiscoASR 1000

CiscoASR 1000

UPCLISPmob

UPCLISPmob

CiscoASR 1000

UPCLISPmob

UPCLISPmob

CiscoASR 1000

SpirentTestCenter

SpirentTestCenter

Map Server

IPv4 Core

IPv4 Core

IPv4 Core

IPv6 Core

22


We emulated two LISP connectivity types. The firstWe used a LISP CPE, acting as xTR and performingthe lookup and encapsulation on behalf of a hostconnected to it. The second was a LISP mobile node,by itself an xTR.We tested two scenarios: communication betweentwo different LISP-sites and communication betweena LISP-site and a non-LISP site. In the latter case, anedge router performed the proxy function defined asPxTR in RFC 6832. We tested two LISP encapsula-tions: IPv4 and IPv6 EIDs over IPv6 RLOC (IPv6-onlycore network) and IPv4 and IPv6 EIDs over IPv4RLOC (IPv4-only core network).UPC LISPmob successfully participated in the test asLISP CPE (xTR) and a LISP mobile node (xTR) in IPv4-only and IPv6-only core networks.Spirent TestCenter successfully emulated a LISP CPEand a LISP mobile node (xTR) in IPv4-only corenetwork.Cisco ASR 1000 successfully participated in the testas MAP server and PxTR in IPv6-only and IPv4-onlycore networks.Cisco ASR 9000 successfully participated in the testas PxTR in IPv4-only core network.Figure 25 depicts the successful permutations weobserved in this test.

6VPE L3VPN Traffic Isolation6VPE, standardized in IETF RFC 4659, is amechanism to enable the transport of IPv6 Layer 3Virtual Private Networks (L3VPNs) over an IPv4 corenetwork.In this test, we interconnected the PEs over an IPv4core and established iBGP among them. On the PE-CE segment, we configured two services, i.e. twoDual-Stack virtual route forwarding (VRF) instances.We generated IPv4 and IPv6 routes in each VRF.

Figure 26: 6VPE L3VPN Traffic Isolation

To verify traffic isolation, each VRF had routes thatwere unique to it and routes that overlap with thoseof its partner. After we verified that the deviceslearned all the advertised routes in the appropriaterouting tables, we generated fully meshed,bidirectional traffic streams between all endpoints.We expected no traffic loss and no traffic leaking.

Figure 26 depicts the successful permutations weobserved in this test.Alcatel Lucent 7750 SR-12, Ericsson MINI-LINK SP420, Ericsson SSR 8010 and Ixia IxNetworksuccessfully participated in the test as PEs. IxiaIxNetwork and Ericsson SSR 8010 successfullyparticipated with 100GigabitEthernet interfaces.Ixia IxNetwork successfully emulated the CE devices.With one PE implementation, we observed an issuewhere the VPN traffic toward the CE is duplicated.The PE sent traffic once as expected and once withan MPLS label.

ICMPv6: IPv6 Ping and TracerouteIn this test, we verified two commonly useddiagnostic tools, ping and traceroute, which utilizeInternet Control Message Protocol (ICMPv6 — RFC4443) messages.Our test scenario consisted of three nodes. Weconnected the designated ping and tracerouteinitiator to another intermediate IPv6 node. A thirdnode was connected to the intermediate node on adifferent interface.When we executed ping and traceroute, we verifiedthat the Echo Reply messages were corresponding tothe Echo Requests that contained correct timestampsused for round trip time calculation. For thetraceroute command we expected to observe a list ofIPv6 addresses describing the intermediate andendpoint nodes.

Figure 27: ICMPv6: IPv6 Ping and Traceroute

6VPE over 1 GbE link



6VPE over 100 GbE link

EricssonSSR 8010

IxiaIxNetwork

Emulated Dual-Stack CE


JuniperMX-480


IPv6 ping IPv6 traceroute



JuniperMX-480

JuniperMX-480




EricssonSSR 8010

JuniperMX-480



EricssonSSR 8010

EricssonSSR 8010


IxiaIxNetwork

JuniperMX-480

EricssonSSR 8010

IxiaIxNetwork

23

Carrier Ethernet 2.0

Alcatel-Lucent 7750 SR-12, Ericsson MINI-LINK SP420, Ericsson MINI-LINK SP 415, Ericsson SSR8010, Juniper MX-480 successfully participated inthe test.Ixia IxNetwork successfully performed IPv6 ping andtraceroute.In one case an implementation was not able torespond to UDP traceroute. In another case an inter-mediate node was unable to initiate or respond totraceroute.Figure 27 depicts the successful combinations weverified in this test.

CARRIER ETHERNET 2.0Carrier Ethernet 2.0 (CE 2.0) is an umbrella brandfor a group of specifications and implementationagreements recently released by the Metro EthernetForum (MEF). CE 2.0 centers around three keyadditions for Carrier Ethernet (CE) services: Multi-QoS, Manageability, and Wholesale Interconnect.Most of the underlying CE 2.0 industry standardsfrom IEEE and ITU-T are well established — meaningthat the industry has already mastered the adoptionof CE 2.0. For reference, see EANTC white papers2008–2011 dealing with ENNI, Connectivity FaultManagement (CFM), traffic engineering and Classof Service testing.In this event, we evaluated the support for multipleclasses of service in our Service Activation test for E-Access services as well as CE 2.0 Manageabilityin our test of Performance Monitoring per Class ofServices for EVPL services.

Service ActivationService activation tests are an essential tool forservice providers, enabling the verification of newEthernet Virtual Connections (EVCs) according to theService Level Agreement (SLA). Service activationtests are performed after provisioning new servicesand before handing them over services to thecustomer. The test methodologies are grouped underan ITU-T standard, Y.1564 Ethernet serviceactivation test methodology.The standard describes various test methodologiesfor different services such as color blind and coloraware. It also considers different service parameterssuch as Committed Information Rate (CIR), ExcessInformation Rate (EIR), Committed Burst Size (CBS)and Excess Burst Size (EBS).During the test preparation, we agreed on twospecific configuration sets for two new services,emulating an activation by a service provider. Table2 describes the services we configured during thetest execution. We designed the two Access EthernetVirtual Private Line (E-Access EVPL) services such thatthey were aligned with MEF 33 specification. Thebandwidth profile parameters were based on MEF23.1 specifications.

Table 2: Service Activation Test Parameters

For the test execution, we configured E-Accessservices as defined in Table 2 between the accessdevices. As soon as the service was configured, theservice activation device performed the testsaccording to Y.1564. Once the service activationtest passed successfully, we performed the Y.1564performance test for 10 minutes. The performancetest evaluated the service against performanceparameters known as Service Acceptance Criteria(SAC). These parameters include Information Rate(IR), Frame Loss Ratio (FLR), Frame Transfer Delay(FTD), Frame Delay variation (FDV) and Availability(AVAIL). For each service type, the test wasperformed against the Service Acceptance Criteria(SAC) that was coordinated with participants.The following combinations successfully participatedin the service activation test for Information Rates(CIR/EIR) and for Burst Sizes (CBS/EBS) for a colorblind E-Access service: Ixia IxNetwork, EricssonMINI-LINK SP 210, Cisco ASR 1000; IxiaIxNetwork, Ericsson MINI-LINK SP 210, ECI NPT-1200; Ixia IxNetwork, ECI NPT-1200, EricssonMINI-LINK SP 210.We tested the Information Rate for the color aware E-Access service for Ixia IxNetwork, ECI NPT-1200,Ericsson MINI-LINK SP 210 pairs. We did notperform the CBS/EBS test for color aware E-Accessservice as it was not supported by the emulator.

Access EVPL 1 - Color aware service

Hig

h

PCP Ga = 5

a. G stands for Green frames

PCP G = 5 CIR = 5 Mbit/sCBS = 12,176 BEIR = 0 Mbit/sEBS = 0 B

Med

ium PCP G = 3

PCP Yb= 2

b. Y stands for Yellow frames

PCP G = 3PCP Y = 2

CIR = 10 Mbit/sCBS = 12,176 BEIR = 10 Mbit/sEBS = 12,176 B

Low

PCP G = 1PCP Y = 0, 4, 6, 7

PCP G = 1PCP Y = 0

CIR = 0 Mbit/sCBS = 0 BEIR = 15 Mbit/sEBS = 12,176 B

Access EVPL 2 - Color blind service

Med

ium PCP = 5, 3 PCP G = 5, 3 CIR = 15 Mbit/s

CBS = 12,176 BEIR = 0 Mbit/sEBS = 0 B

Low

PCP = 0, 1, 2, 4, 6, 7

PCP G = 1PCP Y= 0

CIR = 20 Mbit/sCBS= 12,176 BEIR = 30 Mbit/sEBS = 12,176 B

Color &

CoS

Forw

ardi

ng

SLA p

aram

eter

s

Clas

s lab

el

Iden

tifica

tion

Color

24


Figure 28: Service Activation

Ixia IxNetwork was used as a Service ActivationDevice. We observed that the emulator representedY.1564 test methodology steps in the GUI correctly. We performed the loopback at ENNI in two ways.Some implementations used MAC swap loopback.The loopback was performed after the bridgefunctionality of the switch and not at the ingressinterface. One implementation used the physicalloopback which is the physical connectivity of theports. For this design the MAC learning wasdisabled on the ingress port. During the test, we observed in one implementationthat the smallest CBS/EBS supported was 100kBytes. The vendor however did some dynamicsoftware changes to set CBS/EBS to 12,288 Bytes.For the other implementation, we observed thesmallest supported CBS/EBS value was 16,000Bytes. We therefore changed the service activationparameters to match CBS/EBS supported by thevendors.

Performance Monitoring per Class of ServicesAnother essential aspect for Service providers is thecapability to monitor the performance parameter ofEthernet Virtual Connection (EVC) for Service LevelAgreements (SLAs). ITU-T Y.1731 provides specifica-

tions of a tool capable of addressing this aspect. Thenewer version of the standard, G.8013/Y.1731(2011), provides the tool specifications to be able tomonitor the performance parameters per EVC andClass of Services (CoS). The standard definespriority as a specific configuration parameter forMaintenance End Point (MEP). A MEP may activatethe multiple monitoring on different CoS levels simul-taneously. In this case, each MEP needs to managethe result of monitoring per CoS level.In our event, we decided to focus on the new specifi-cations of the G.8013/Y.1731 ITU-T standard. Wetherefore distinguished between two different typesof measurements: Frame loss measurement andaverage delay measurement. We performed eachtype per point to point EVC and per CoS ID. The testwas open for any of the measurement methoddefined in ITU-T G.8013/Y.1731, meaning lossmeasurements and average delay measurementswere performed either one-way or two way.We started the test by running a baseline test. Weconfigured an EVPL service with three different Classof Services (CoS) IDs. We measured the averageframe delay and frame loss values for each devicepair with no impairment. To test the delay measurement, we added a unidirec-tional constant delay of zero, 10 and 20 milli-seconds (ms) per CoS ID 5, CoS ID 3 and CoS ID 0respectively. We expected the devices to observethat the average delay remained unchanged for CoSID 5, increased by 10 ms for CoS ID 3 andincreased by 20 ms for CoS ID 0.For loss measurement per EVC and CoS ID, we intro-duced a constant frame loss of 5 % in one directionand 15 % in the opposite direction for CoS ID 0. Weintroduced a bidirectional constant frame lossof10 % for CoS ID 3. No impairment was applied toCoS ID 5. We expected the devices to display thatthe far-end and near-end frame loss remainedunchanged for CoS ID 5, increased by 10 % forCoS ID 3, and increased by 5 % and 15 % —depending on the direction of impairment — for CoSID 0.We tested the frame loss measurement using ETH-SLM (SLM/SLR messages) for the first time during ourinteroperability event for the following pairs: OESolution AimValley Smart SFP OAM, ECI NPT-1200;Alcatel-Lucent 7750 SR-12, Cisco ASR 9000; CiscoASR 9000, ECI NPT-1200. During the test, we observed one implementationwith SLM resolution issue. The measured frame lossvalues were not observed correctly as the resolutionwas low. The vendor however fixed the measurementresolution to send 10 packets per second with themeasurement period of 100 seconds. We thereforesuccessfully participated in the test. The following combination was tested single-endedframe loss measurement with LMM/LMR messagesper EVC and CoS ID: AimValley EX14 and CiscoASR 9000. To perform the test, the Continuity CheckMessages (CCM)s was disabled to compensate for amismatch in counting logic between the devices.

E-Access1- color-aware

E-Access2- color-blind

Service activation device

CiscoASR 1000

ECINPT-1200

EricssonMINI-LINK SP 210

IxiaIxNetwork

IxiaIxNetwork

IxiaIxNetwork



ECINPT-1200

Physical link

Service activation device

Access network

Ethernet aggregation network

Ethernet physical loop

Impairment tool

Aggregation device

Emulator

Impairment

25

Carrier Ethernet 2.0

One LMM implementation counted Tx/Rx CCMs atthe same level of the MEP — according to thestandard — while the paired implementation did notcausing a false frame loss reading. Also, theimpairment was only in one direction as one of theimplementation was not copying the right receivedRx counters in the LMR message.

Figure 29: Performance Monitoring per Class of Services

All the pairs represented in Figure 29 participatedand passed the average frame delay measurementssuccessfully. During the average delay measurementtest, we observed that the emulator does a singlepacket delay measurement, hence no minimum,maximum or delay variation can be measured. Forone implementation, we performed the delaymeasurement for one Class of Service (CoS) ID atone time. For this implementation, we observed thatthe measurement was running continuously, and allresults could be logged continuously. However, the

GUI represented only the last 10 measurements, andhence, no mean value was shown in the GUI. We also observed one implementation where asingle profile could be defined per CoS ID. Wecould run all profiles at the same time. For this imple-mentation, the DM messages were sent with the rateof one per second. The delay measurement wasbased on bucket of DM messages where the bucketwas filled after a configurable time; Here, we testeda bucket of one minute. We therefore observed theexpected results after a minute. Other implementa-tions have different monitoring duration from aminute to 15 minutes. We also observed version bitmismatch for DMMs. The version bit for DMmessage is set to 0 according the old Y.1731,where it is set to 1 in the newer version. For imple-mentation supported the old standard, the inter-working was possible when the remote pairsupported DMRs with both versions or downgradethe version manually.

AimValleyEX14

CiscoASR 9000

IxiaIxNetwork

CiscoASR 1000

CiscoASR 9000

CiscoASR 1000

IxiaIxNetwork

CiscoASR 9000


ECINPT-1200

CiscoASR 9000

ECINPT-1200

OE Solutions-AimValleySmart SFP OAM



IxiaAnue 3500

IxiaImpairNet

ECINPT-1200

Y.1731 ETH-DM per EVC & CoS ID

Y.1731 ETH-LM per EVC & CoS ID

Y.1731 ETH-SLM per EVC & CoS ID

26


DEMONSTRATION NETWORK

During the intensive three weeks of testing, weverified many successful test results. Out of thesesuccessful results, we created a demonstrationnetwork as a showcase for the tested technologies.In the clock synchronization and SDN areas, wecreated two demonstrations combining both technol-ogies. The first scenario, PTP across SDN,showcases a scenario where PTP timing informationis transported over SDN switches. We connected aSymmetricom grandmaster clock to a Ericssonboundary clock, which we connected to a Extremeslave clock over a series of three SDN switches(Huawei, NEC, Extreme). Ericsson MINI-LINK SP210, Extreme E4G-200, Extreme Summit X670,Huawei OF Switch 1.2, Huawei Smart NetworkController (SOX), NEC PF5248 and SymmetricomTimeProvider 5000 are part of this setup. Ixia Anue3500 is used as a clock analyzer.The second scenario demonstrates TSoP over SyncEislands over SDN. PTP, used to synchronized thedisjoined SyncE islands, is transported in the samepath as the TSoP traffic. At the TSoP SDH/SONETinterfaces, we generated bidirectional STM-1 traffic.The point-to-point TSoP over synchronous CarrierEthernet service is configured such that one side isuntagged, while the other side is identified by aVLAN tag. The OpenFlow controller automaticallysets a customer required VLAN ID and generatesmultiple table flow entries in the OpenFlow switch.BTI SA-810, ECI NPT-1200, Ericsson MINI-LINK SP110, Extreme E4G-200, Huawei OF Switch 1.2,Huawei Smart Network Controller (SOX), OESolutions — AimValley Smart SFP TSoP and Symmet-ricom TimeProvider 5000 are part of this setup.In the IPv6 migration area, we created a setup withDual-Stack Lite and 6rd CPEs attached to AFTR (forDS-Lite) and 6rd BR over a Carrier Ethernet 2.0aggregation network. The AFTR (for DS-Lite) and 6rdBR are configured within a Dual-Stack IPv6 VRF(6VPE) service, the same VRFs which is also runningMVPN. The Carrier Ethernet 2.0 aggregationnetwork is realized via multiple EVPLs with QoS,verified via Y.1564 service activation tests andmonitored via Y.1731 for performance. Alcatel-Lucent 7750 SR-12, Cisco ASR 1000, ECI NPT-1200, Ericsson MINI-LINK SP 210, Ericsson SSR8010, Ixia IxNetwork, Juniper MX-480 and SpirentTestCenter are part of this setup.We created an advanced MVPN setup based onmLDP profile as described in the section IPv6 MVPNusing P2MP Tunnels. In this setup Alcatel-Lucent7750 SR-12, Cisco CRS-1, Cisco ASR 9000, IxiaIxNetwork and Juniper MX480 are participating asMVPN PEs. The emulated multicast CEs areconnected to the PEs over a Layer 2 servicesconfigured in the MPLS-TP aggregation network. TheLayer 2 services are provided by Cisco ASR 9000,ECI NPT-1200, Ericsson MINI-LINK SP 210, EricssonMINI-LINK SP 310, and Hitachi AMN 6400.

AcknowledgementsThis report has been edited by EANTC teammembers Spase Kovachev, Sergej Kälberer, CarstenRossenhövel, Shima Sajjad, Robert Sanchez andEldad Zack.

27

Acronyms

ACRONYMS

Term Definition6rd IPv6 Rapid Deployment

AC Access Circuit

A-D Auto Discovery

AFTR Address Family Translation Router

AIS Alarm Indication Signal

AS Autonomous System

BFD Bidirectional Forwarding Detection

BGP Border Gateway Protocol

BR Border Router

CC Continuity Check

CCM Continuity Check Message

CE Customer Edge

CE2.0 Carrier Ethernet 2.0

CFM Connectivity Fault Management

CGN Carrier Grade NAT

CLI Command Line Interface

CoS Class of Service

CPE Customer Premises Equipment

CV Connectivity Verification

DHCP Dynamic Host Configuration Protocol

DMM Delay Measurement Message

DMR Delay Measurement Reply

DS-Lite Dual-Stack Lite

ENNI External Network Network Interface

ETH-SLM Ethernet Synthetic Loss Measurement

EVC Ethernet Virtual Connections

EVPL Ethernet Virtual Private Line

EVPN Ethernet VPN

FPP Floor Packet Population

FRR Fast Re-route

G-ACh Generic Associated Channel

GPS Global Positioning System

GUI Graphical User Interface

H-VPLS Hierarchical Virtual Private LAN Service

IEEE Institute of Electrical and Electronics Engineers

IETF Internet Engineering Task Force

IGMP Internet Group Management Protocol

IGP Interior Gateway Protocol

I-PMSI Inclusive Provider Multicast Service Interface

IPv4 Internet Protocol Version 4

IPv6 Internet Protocol Version 6

ITU International Telecommunication Union

L3VPN Layer 3 Virtual Private Network

LDI Link Down Indication

LDP Label Distribution Protocol

LISP Locator/ID Separation Protocol

LLDP Link Layer Discovery Protocol

LMM Loss Measurement Message

LMR Loss Measurement Reply

LOF Loss of Frame

LSP Label Switching Router

MAC Media access control

MAP Mapping of Address and Port

MEF Metro Ethernet Forum

MLD Multicast Listener Discovery

mLDP Multipoint Label Distribution Protocol

MP-BGP Multiprotocol BGP

MPLS Multiprotocol Label Switching

MPLS-TP MPLS Transport Profile

MS-PW Multi-Segment Pseudowire

MTIE Maximum Time Interval Error

MVPN Multicast VPN

NLRI Network Layer Reachability Information

OAM Operation, Administration and Mainte-nance

ONF Open Networking Foundation

OSPF Open Shortest Path First

P2MP Point to Multipoint

PBB Provider Backbone Bridging

PCE Path Computation Element

PCP Port Control Protocol

PDV Packet Delay Variation

PE Provider Edge

PIM-SM Protocol Independent Multicast — Sparse Mode

PSC Protection State Coordination

PSN Packet Switched Network

PTP Precision Time Protocol

PW Pseudowire

PxTR Proxy Ingress/Egress Tunnel Router

QoS Quality of Service

RFC Request for Comment

RSVP-TE Resource Reservation Protocol — Traffic Engineering

SDN Software-Defined Networking

SEC SDH Equipment Clock

SFP Small Form-factor Pluggable (Trans-ceiver)

SLA Service Level Agreement

SLM Synthetic Loss Message

SLR Synthetic Loss Reply

S-PMSI Selective Provider Multicast Service Interface

STM Synchronous Transport Module

SyncE Synchronous Ethernet

TDEV Time Deviation

TSoP Transparent SDH/SONET over Packet

UNI User Network Interface

VLAN Virtual Local Area Network

VPN Virtual Private Network

xTR Ingress/Egress Tunnel Router

Term Definition


EANTC AGEuropean Advanced Networking Test Center

Upperside Conferences

Salzufer 1410587 Berlin, GermanyTel: +49 30 3180595-0Fax: +49 30 [email protected]://www.eantc.com

54 rue du Faubourg Saint Antoine75012 Paris - FranceTel: +33 1 53 46 63 80Fax: + 33 1 53 46 63 [email protected]://www.upperside.fr

This report is copyright © 2013 EANTC AG. While every reasonable effort has been made to ensure accuracy andcompleteness of this publication, the authors assume no responsibility for the use of any information contained herein.All brand names and logos mentioned here are registered trademarks of their respective companies in the UnitedStates and other countries.20140127 v5.1

Public Multi-Vendor Interoperability Event 2013 White Paper

Documents

Transcript of Public Multi-Vendor Interoperability Event 2013 White Paper