Metropolitan Ethernet Network: A move from LAN to MAN

28
Metropolitan Ethernet Network: A move from LAN to MAN Minh Huynh, Prasant Mohapatra * Computer Science Department, University of California at Davis, Davis, CA, United States Received 7 December 2006; received in revised form 21 June 2007; accepted 6 August 2007 Available online 17 August 2007 Responsible Editor: I.F. Akyildiz Abstract Ethernet has been the indisputable technology of choice for the local area networks (LANs) for more than 30 years. Its popularity is due to its versatility, plug-n-play feature, and low cost. It has transformed from a CSMA/CD technology providing low throughput to a full-duplex link increasing the throughput 1000-folds. Despite these improvements, Ether- net is still restricted to local area networks, and is not ready to become a carrier-grade technology for wider areas. How- ever, there are efforts to assist the transformation of Ethernet from the mainstream LAN technology to the possible adoption for metropolitan area networks (MENs). This paper will introduce the movement from basic Ethernet to the car- rier grade Ethernet for MENs. The paper describes the underlying technology, offered services, the state-of-the-art, and the comparison between various technologies. In the context of the move from LAN to MAN, various problems and their corresponding solutions are discussed, along with the future of Metro Ethernet Network. Ó 2007 Elsevier B.V. All rights reserved. Keywords: Metro Ethernet Network; Local area network; Metropolitan area network 1. Introduction Traditionally, Ethernet is a plug-n-play technol- ogy at the link layer intended for Local Area Net- work (LAN). Its success is in parts due to its standardization that enables the interoperation among different equipment vendors. Thus, mass production drives down the cost of Ethernet and also advances the popularity of basic Ethernet fur- ther. From the earlier Ethernet that uses CSMA/ CD and runs at 10 Mbps on the coaxial cable, Ethernet now can run full-duplex 10 Gbps links with backward compatibility. It does not need any special device to convert between equipments run- ning at different speeds displaying a true plug-n-play system. Initially, Ethernet was designed to operate on a bus topology using the coaxial cable at 10 Mbps. It was a broadcast environment where there is the possibility of frame collision. Using the CSMA/ CD, Ethernet successfully sent frame to other hosts while entering the exponential backup phase if there is a collision. Several versions of Ethernet technolo- gies existed ranging from 10 Mbps to 10 Gbps running on coaxial cable, twisted pair copper cable, and fiber optic line. However, all of the different 1389-1286/$ - see front matter Ó 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.comnet.2007.08.001 * Corresponding author. Tel.: +1 530 754 8380. E-mail addresses: [email protected] (M. Huynh), pmo- [email protected], [email protected] (P. Mohapatra). Computer Networks 51 (2007) 4867–4894 www.elsevier.com/locate/comnet

Transcript of Metropolitan Ethernet Network: A move from LAN to MAN

Page 1: Metropolitan Ethernet Network: A move from LAN to MAN

Computer Networks 51 (2007) 4867–4894

www.elsevier.com/locate/comnet

Metropolitan Ethernet Network: A move from LAN to MAN

Minh Huynh, Prasant Mohapatra *

Computer Science Department, University of California at Davis, Davis, CA, United States

Received 7 December 2006; received in revised form 21 June 2007; accepted 6 August 2007Available online 17 August 2007

Responsible Editor: I.F. Akyildiz

Abstract

Ethernet has been the indisputable technology of choice for the local area networks (LANs) for more than 30 years. Itspopularity is due to its versatility, plug-n-play feature, and low cost. It has transformed from a CSMA/CD technologyproviding low throughput to a full-duplex link increasing the throughput 1000-folds. Despite these improvements, Ether-net is still restricted to local area networks, and is not ready to become a carrier-grade technology for wider areas. How-ever, there are efforts to assist the transformation of Ethernet from the mainstream LAN technology to the possibleadoption for metropolitan area networks (MENs). This paper will introduce the movement from basic Ethernet to the car-rier grade Ethernet for MENs. The paper describes the underlying technology, offered services, the state-of-the-art, and thecomparison between various technologies. In the context of the move from LAN to MAN, various problems and theircorresponding solutions are discussed, along with the future of Metro Ethernet Network.� 2007 Elsevier B.V. All rights reserved.

Keywords: Metro Ethernet Network; Local area network; Metropolitan area network

1. Introduction

Traditionally, Ethernet is a plug-n-play technol-ogy at the link layer intended for Local Area Net-work (LAN). Its success is in parts due to itsstandardization that enables the interoperationamong different equipment vendors. Thus, massproduction drives down the cost of Ethernet andalso advances the popularity of basic Ethernet fur-ther. From the earlier Ethernet that uses CSMA/CD and runs at 10 Mbps on the coaxial cable,

1389-1286/$ - see front matter � 2007 Elsevier B.V. All rights reserved

doi:10.1016/j.comnet.2007.08.001

* Corresponding author. Tel.: +1 530 754 8380.E-mail addresses: [email protected] (M. Huynh), pmo-

[email protected], [email protected] (P. Mohapatra).

Ethernet now can run full-duplex 10 Gbps linkswith backward compatibility. It does not need anyspecial device to convert between equipments run-ning at different speeds displaying a true plug-n-playsystem.

Initially, Ethernet was designed to operate on abus topology using the coaxial cable at 10 Mbps.It was a broadcast environment where there is thepossibility of frame collision. Using the CSMA/CD, Ethernet successfully sent frame to other hostswhile entering the exponential backup phase if thereis a collision. Several versions of Ethernet technolo-gies existed ranging from 10 Mbps to 10 Gbpsrunning on coaxial cable, twisted pair copper cable,and fiber optic line. However, all of the different

.

Page 2: Metropolitan Ethernet Network: A move from LAN to MAN

4868 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

Ethernet versions kept the same frame structure forbackward compatibility.

Ethernet evolves from a LAN service intercon-necting an enterprise workgroup to running theenterprise backbone. Now extending to the MetroArea Network (MAN), Ethernet provides Ethernetservices across MAN. MAN makes up of a metrocore network and several access networks. Theaccess networks border with the subscribers net-works. Subscribers include business enterprise net-works and residential network such as DSL andcable services. The metro core is the backbone ofMAN where it interconnects the access networkshauling large trunk of traffic. In addition the metrocore provides the subscribers with access to theInternet.

Ethernet in MAN is an alternative to the tradi-tional Time Division Multiplexing (TDM) technol-ogy. Although TDM is used to delivered voiceand leased-line services, it is inefficient for deliveringthe emerging data oriented applications. Ethernetservices can offer the point-to-point line service ormultipoint-to-multipoint LAN services. LAN ser-vices connect multiple sites belonging to the sameenterprise across different physical locations into avirtual LAN as if all sites exist in a local building.

An example of emerging data oriented applica-tions running over Metro Ethernet Network(MEN) is LAN to network resources [51]. LAN tonetwork resources can offer services such as thebacking up data of enterprises at a remote andsecured site for disaster recovery. Customers canbackup and recover their data constantly acrossthe metro. For residential areas, LAN to networkresources can distribute multimedia services. Forexample, video servers can be deployed at a Pointsof Presence (POP) where the residents can accessfor broadband video on demand over an Ethernetconnection. Other services that MEN can offerinclude [51] Internet connection, Extranet, StorageArea Networks (SANs), Metro Transport, andVoIP. Around the world, different applications arethe main driving force for MEN. For example, inKorea, the growing game parlor business is thebandwidth hog. Japan focuses on the inter-officeconnection between large multi-sites enterprises thatspan across remote physical location. China andIndia are building a common platform for the resi-dential triple play: voice, video, and data.

In addition, the advantages of Ethernet such ascost effectiveness, flexibility, rapid provision ondemand, and ease of interoperability drive the adap-

tation of Ethernet into MAN. The mass productionof Ethernet equipments and the simplicity of Ether-net technicality keep the cost of having Ethernet rel-atively low compared to others competitiveprotocols. Another factor in cost saving is the easeof interoperability without third party converter orsometime without purchasing new equipments.The same Ethernet interface can support a varietyof bandwidth unlike legacy technologies. One fea-ture that makes Ethernet stands out than the restis the flexibility in bandwidth upgrade. With band-width increment as fine granularity as 1 Mbps,Ethernet offers better bandwidth efficiency thanTDM. Therefore, it is able to have rapid provisionon demand.

Ethernet in combination with VPLS is the con-vergence technology that brings mass traffictogether from diverse platform. Besides the highspeed wired networks, MEN is the cost effectivebackhaul for the mobile carriers. The Carrier Ether-net will become the common ‘‘transport layer’’ todeliver multiple services over a single connection.

In the remaining parts of this report, we willintroduce the basic Ethernet to the carrier gradeEthernet for MEN. We will explore the underlyingtechnologies, offered services, and architecturesfrom both the industry and the academia literatures.Challenges and corresponding solutions are alsodiscussed.

2. Demand for Metro Ethernet Network

We are on the verge of witnessing the transforma-tion of Ethernets from the traditional local area net-works within buildings to wider metropolitan areas.This gradual expansion of the scope is guided by thegrowing needs as well as the versatility of the proto-col. In this section, we overview the motivations andthe characteristics of Ethernet that make it a suitablecandidate for this broadening scope of usage.

2.1. Motivation for the MEN transformation

Incumbent technologies such as Private Line(PL), Frame Relay (FR), and Asynchronous Trans-fer Mode (ATM) cannot respond as fast as Ethernetto the high volume demand for new connectionsbecause of the long waiting period to establish a ded-icated physical connection. For example, an incum-bent carrier takes three to six months to deploy aT1 circuit [51]. In addition, upgrading the currentconnection exposes the inefficiencies in Time Divi-

Page 3: Metropolitan Ethernet Network: A move from LAN to MAN

Internet Acces Private Data Total Monthly Recurring Cost

$0.00

$500,000.00

$1,000,000.00

$1,500,000.00

$2,000,000.00

$2,500,000.00

$3,000,000.00

Ethernet Private Line Frame Relay

Fig. 2. Recurring Cost of Operation in a 3 year period study [44].

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4869

sion Multiplexing (TDM), such as coarse granularityof bandwidth increments resulting in oversubscrib-ing, requirement of new equipments, and changingto new service platforms and protocols. For instance,a T1 connection running at 1.5 Mbps will beupgraded to a DS3 connection at 45 Mbps. An alter-native solution is to provide multiple T1 connections[51]. Both result in purchasing of new equipments. Incontrast, Ethernet can provide bandwidth incrementwith 1 Mbps granularity. The same Ethernet proto-col can be used from 10 Mbps to 10 Gbps. It is 10times lower cost than high speed SONET interfaces[29]. Therefore, the agility to respond to customers’need and the cost efficiency drive the Ethernet expan-sion to the carrier grade for Metro Ethernet Net-work. Fig. 1 shows that the worldwide revenueforecast study from the Metro Ethernet Forum(MEF) indicating that currently FR, ATM, and PLcombined together take a larger part of the marketthan Metro Ethernet. However, with the currentgrowth rate of Ethernet, Metro Ethernet will eventu-ally take over the lead as projected. In this growthrate, 14% of Ethernet services result from new ser-vices deployment while the remaining 86% resultfrom the replacement of legacy services from a studyby the Vertical System Group [53]. In addition, Fig. 2shows that Ethernet can save more than 50% over a3 year period in a business case study from the MEF[44]. This operational cost includes Internet access,Private Data, and Monthly Recurring Cost.

2.2. Advantages of MEN

Over the last decade, bandwidth has increasedsignificantly in the backbone network making the

$66,500.00

$67,000.00

$67,500.00

$68,000.00

$68,500.00

$69,000.00

$69,500.00

$70,000.00

2002 2003 2004 2005 2006 2007

Rev

enue

USD

Mill

ions

(ATM, FR, PL) EthernetΣ

Fig. 1. Worldwide revenue forecast Metro Ethernet vs. FR,ATM, and Private Line [43].

metro as the bottleneck. Other legacy services suchas T1, T3, or ATM do not provide the flexibilityin bandwidth increment that MEN need. Further-more, Ethernet is a popular protocol in all enter-prise LANs. The choice of using Ethernet tointerconnect remote sites of an enterprise is appeal-ing for the following reasons: cost effectiveness, flex-ibility, rapid provision on demand, and ease ofinteroperability [27].

Since Ethernet equipments are very common onthe market, its material and development costs havebeen kept competitively low. Ethernet’s level oftechnical complexity is relatively lower than the oth-ers. Therefore, the capital expenditure and opera-tional expenditure are also low. This costeffectiveness drives the expenses on Ethernet servicesin MEN down compared to other technologies. Abusiness case study by the MEF in July 2003showed that compared to legacy SONET/SDH,Ethernet based services save 49% on operationalexpense and 39% on capital expenses [50]. UsingCarrier Ethernet as the common platform to back-haul mobile traffic, the saving on operationalexpense ranges from 15% by the Yankee Groupstudy to 40% by the MEF study [53]. Table 1 showsthe list prices for a monthly leasing bandwidth by

Table 1Leasing price quotes for mobile carriers

Bandwidth Verizon USA BT UK

E1/T1 (�2.048 Mbps/1.544 Mbps) 780 500–1090Ethernet 4 Mbps N/A 1000Ethernet 10 Mbps 1430 1120Ethernet 50 Mbps 2130 1450

Page 4: Metropolitan Ethernet Network: A move from LAN to MAN

4870 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

two leading carriers. From 10 Mbps and up, Ether-net is 3–5 times cheaper than E1/T1 line.

Some of the problems with the legacy services arethe long wait for the service to be installed and acti-vated and the coarse bandwidth granularity. Whatthe enterprises need is rapid provision on demandof services. For example, an enterprise might needhigh bandwidth provisioning during the day andlow bandwidth provisioning in the evening and onthe weekend. However, it is not possible to do sowith the legacy technologies. The enterprise endsup paying for the peak bandwidth for all timebecause the service installation is not on demand.In addition, enterprises have to buy the bandwidthin large chunks. It is not possible for them to havefine granularity of bandwidth increment. In con-trast, Ethernet service offers bandwidth incrementsin term of 1 Mbps. The same Ethernet interface,e.g. 1 Gpbs, can support a variety of bandwidths.Therefore, bandwidth-on-demand can be easilyprovisioned.

The plug-n-play feature of Ethernet enables asimple migration from low speed to high speed with-out any third party converter. Also, Ethernet ser-vices reduce the complexity of protocol translationbetween different platforms and systems.

2.3. The move from LAN to MAN

After recognizing the expectation for MEN, thenext step is the transformation of Ethernet fromthe LAN environment to the MAN environment.Traditional Ethernet is used to be deployed on smallsegments of workstations. These segments are thenconnected to create an intranet that traditionallyexists within the same geographical site. This setup

Fig. 3. Metro area ne

is referred to as Local Area Network (LAN) wherethe geographical area is relatively small, and all ofthe traffic belongs to the same enterprise. In contrast,Metro Ethernet Network (MEN) spans across ametropolitan area. MEN is comprised of a core net-work and several access networks as shown in Fig. 3.All the access networks connect to the core at one ortwo aggregation Ethernet switches. The customers’networks are connected to the access network; andthe core helps in interconnecting the access networks.Packets hop through multiple switches in both accessand core networks. Redundant links are used both inthe core as well as the access networks. Since trafficscoming from different enterprises traverse the samenetwork, a traffic isolation mechanism is neededinside MEN. On the other hand, there is a need tomerge traffic belonging to the same enterprise butcoming from multiple geographic locations.

To deploy MAN, there are three options. Thefirst option is to extend the core technologies suchas IP/MPLS into the access network. This createscomplexity in operation of one large network thatis very difficult to configure. There are also incom-patibility issues between different software environ-ments. Also, core technologies have highequipment cost. The second option is to deployEthernet in the access and the core. One majordrawback is that Ethernet does not support suchlarge and flat network, that is, no hierarchical struc-ture is defined. In addition, Ethernet lacks trafficmanagement functions, Service Level Agreement(SLA) mechanism, and security protection. The lastoption is the hybrid of deploying Ethernet in theaccess network and using MPLS, Resilient PacketRing (RPR), or other core technologies for the core.This has the benefit of providing Ethernet at career

twork topology.

Page 5: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4871

class with the simplicity and low cost of the tradi-tional Ethernet.

3. MEN technology

In this section, we will discuss the standardizedtechnologies that are used to support Ethernet inMEN. Since Ethernet by itself does not have all ofthe features to support services required by the cus-tomers, it relies on those that can complement it.The difficulty in these inter-networking is the map-ping from one platform to another. As always,interoperability requires the use of tunneling. InMEN, there are two main tunneling approaches:Virtual Private Wire Service (VPWS) for point-to-point and Virtual Private LAN Service (VPLS) forany-to-any or multipoint. In general, point-to-pointbehaves like a single connection while multipointbehaves like a LAN.

3.1. Point-to-point

Point-to-Point service is used to connect only twoUser Network Interfaces (UNI) together. VirtualPrivate Wire Service (VPWS) is an emulation ofL2 Virtual Private Network (VPN) for Ethernet

L2 VPN

TunnelPW

PW

Tunnel

Fig. 4. L2 VPN architecture. The pseudowire (PW) is composedof virtual circuit (VC) label and the remote PE. Tunnel serves asheader for routing within the provider network.

L2VPN

VPWSVirtual Private Wire Services

Point-to-Point

Encapsulation Signaling

Draft MartiniIETF PWE3

Draft Kompella (K)BGP Control Plane

Draft Kompella(VDraft Lasserre

LDP Control Plane

L2VPN

VPWSVirtual Private Wire Services

Point-to-Point

Encapsulation Signaling

Draft MartiniIETF PWE3

Draft Kompella (K)BGP Control Plane

Draft Kompella(VDraft Lasserre

LDP Control Plane

Fig. 5. L2 VPN proto

tunneling a point-to-point connection between twoends. Fig. 4 shows the L2 VPN architecture. VPWSis categorized into Ethernet Relay Service (ERS) orEthernet Wire Service (EWS) [42]. ERS uses theVLAN number and offers services similar to FrameRelay. The services are shared and multiplexed atthe UNI. In contrast, EWS is a port-based servicewhere traffic transporting over a port is treated asa private line. VLAN numbering is neglected inEWS. The encapsulation of VPWS uses the draft-Martini approach [26]. Fig. 5 shows the protocolsneeded to support VPWS. A list of VPN-relatedIETF drafts is listed in [42].

Draft-Martini [26] is a tunneling protocol for apoint-to-point connection. It was intended as a car-rier backhaul or high-speed connection betweenmajor sites. In MEN, it is a connection betweentwo User Network Interfaces (UNI). One of IETF’sworking groups defines pseudowire emulation edgeto edge (PW3) based on the draft-Martini. Thepseudowire is being used to offer layer 2 transportacross the MPLS core. It specifies a virtual circuitlabel and the remote Provider Edge (PE).

When an Ethernet frame enters the provider net-work, as shown in Fig. 6, the ingress router stackstwo labels on it: virtual circuit (VC) label and tunnellabel. The VC label stays the same as the frame tra-verses across the network. It is used for multiplexingpurposes when frames arrive at the destination PE.Each pair of PE has a unique VC label. The tunnellabel is locally significant at each hop for routingpurposes within the MPLS domain. It can also pro-vide multipoint-to-multipoint service but it will suf-fer from the n-squared problem whereas n2connections are required to connect n locations.Its main purpose is to support the E-Line service.

VPLSVirtual Private LAN Services

Multipoint Services

)

Encapsulation

Draft MartiniIETF PWE3

Draft KomplellaBGP

Signaling

VPLSVirtual Private LAN Services

Multipoint Services

)

Encapsulation

Draft MartiniIETF PWE3

Draft KomplellaBGP

Signaling

col classification.

Page 6: Metropolitan Ethernet Network: A move from LAN to MAN

Tunnel Label

CE

2EP1EP

CE

102 102

Ethernet Pkt Ethernet Pkt

VC Label5

1027Tunnel Label

CE

2EP1EP

CE

102 102

Ethernet Pkt Ethernet Pkt

VC Label5

1027

Fig. 6. Virtual Private Wire Service (VPWS) for point-to-point [51].

4872 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

Auto-provisioning of pseudowires is based on thecolored pool concept [42]. Different VPNs areassigned with different colors. For example, PE1and PE2 each have a green pool of attachment cir-cuits (AC). When PE1 discovers PE2, it removesan AC from its green pool and binds the AC tothe pseudowire connecting PE2. Similarly, PE2binds one its ACs from the green pool to thepseudowire connecting PE1. After the auto-discov-ery, a full mesh of pseudowires is created betweenall the ACs belonging to the same VPN.

3.2. Multipoint

In multipoint service, more than two UNIs areconnected together to form a LAN. A frame send-ing from one UNI will arrive at all the remainingUNIs within the same virtual LAN. Similar topoint-to-point service, multipoint service are catego-rized into two services: Ethernet Multipoint Service(EMS) and Ethernet Relay Multipoint Service(ERMS) [49]. EMS behaves similar to EthernetLAN like address learning and unknown addressbroadcasting. However, each UNI can only receiveone service. On the other hand, ERMS can multi-plex different services at each UNI.

Carriers have offered legacy services such asATM, Frame Relay and private line for quite some-time. Therefore, they have a large share in the mar-ket. Reports on Virtual Private LAN Service(VPLS) [37,39,40] answer the need to integrate newEthernet services while carrying on legacy serviceswith large market share. VPLS, also known as Layer2 MPLS, is built on pseudowire as a multipoint tun-neling scheme. It offers multipoint connectivity byvirtualizing enterprises with remote LAN sites ontothe same LAN supporting the E-LAN services, asshown in Fig. 7. Similar to MPLS at layer 3, VPLSoffers the same services at layer 2. The difference isin the interface between the Customer Edge equip-ment (CE) and the Provider Edge equipment (PE).

In MPLS layer 3, the CEs are IP routers as opposedto Ethernet bridge/switch/hub or router in VPLS,allowing both non-IP and IP traffic to be routed.In addition, VPLS can emulate the behavior ofEthernet LAN such as broadcasting of unknownMAC addresses and MAC address learning.

VPLS is defined in two IETF drafts: VPLS-LDP[24] and VPLS-BGP [25]. Fig. 5 shows a mapping ofthe different protocols for VPLS. VPLS-LDP usesLDP protocol as the signaling protocol to establisha full mesh of LSP between PE nodes. It is backedby majority of the vendors such as Atrica, Cisco,Extreme, Force10, Foundry, Nortel, Riverstone,Cosine Communication Inc., Laurel, Overture,Timetra, Vivace. One advantage that VPLS-BGPhas over VPLS-LDP is the discovery of the neighborPEs since BGP has that capability built in. VPLS-LDP could incorporate BGP into it or using a direc-tory-based approach such as Radius [45] that isbeing discussed within the IETF. Juniper is the onlyvendor that supports VPLS-BGP because it is oneof the first vendors to develop VPLS. Since Juniperhas invested a lot into the BGP approach, it is diffi-cult to switch to a new approach [6].

For each VPLS, VPLS-LDP creates a full meshof tunnels by first using UDP to determine neigh-bors, then establishing a TCP session to requestfor label mapping. Next, it needs to define a VPLSID and establish virtual circuit (VC) labels forLSP. The outer tunnel label and the VC label aretagged onto the front of each Ethernet packetheader for switching. Each virtual circuit LabelSwitching Path (LSP) is a bi-directional pseudowireinside the outer tunnel. Now the PEs act like bridgesand perform the following functions: Learning andaging MAC addresses on a per LSP basis, floodingof unknown frames, and replication for unknown,multicast, and broadcast frames.

To create a loop free routing environment, VPLSuses the split-horizontal technique instead of theSpanning Tree. In split-horizontal, a PE would

Page 7: Metropolitan Ethernet Network: A move from LAN to MAN

PE1 PE2

PE3

VPLS A

VPLS B

VPLS A

VPLS B

VPLS A

VPLS B

VPLS A

Physical line

CECE

PE1 PE2

PE3

VPLS A

VPLS B

VPLS A

VPLS B

VPLS A

VPLS B

VPLS A

Physical line

CECE

Fig. 7. A sample VPLS configuration [51].

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4873

not forward packets that it had received from onePE to another PE. The packet is still guaranteedto reach the destination because the networkedhas a full mesh topology. Essentially, the sourcePE broadcasts the packets to all of its adjacentneighboring PEs.

To better scale VPLS, a hierarchical VPLS(HVPLS) topology is laid out to a hub-and-spoketopology where the PEs act as hubs and the simpleswitches terminate each spoke, as shown in Fig. 8.This approach minimizes the topology of the fullmesh, reducing the number of LDP peers. Onlythe core network is needed to have a fully meshtopology. Another problem is the explosion ofMAC addresses since the MAC addresses have a flatstructure. One approach is to use routers for thecustomer/provider edge (CPE) devices. Therefore,each site is reduced to one address that the switcheshave to learn. The other approach is to limit thenumber of addresses that can be learned per accesscircuit.

PE

VPLS A

Provider MTU-s

CE

CE

CEVPLS B

Spoke connection

PE

VPLS A

Provider MTU-s

CE

CE

CEVPLS B

Spoke connection

Fig. 8. A hierarchy VPLS topology. MTU-s is Multi-Tenant Unit runnrouter or switch. CO stands for Central Office. CE is the Customer Ed

4. Metro Ethernet services

There are two services in MEN: E-Line for apoint-to-point service and E-LAN for a multipointservice. These services arise from the tunnelingapproach of MEN, specifically VPWS and VPLS.Both of these services are similar in the parametersfor the quality of service. Their difference lies inthe connectivity between end-points.

4.1. E-Line service

Ethernet Line Service (E-Line) is a point-to-pointEthernet Virtual Circuit between two user networkinterfaces (UNI) [28]. E-Line can provide a simplebest-effort service on the bi-directional line or withsome performance assurances. E-Line performanceassurance includes Committed Information Rate(CIR) and the associated Committed Burst Size(CBS), Peak Information Rate (PIR) and the asso-ciated Peak Burst Size (PBS), delay, jitter, and loss

MPLS

-rs

Provider MTU-s

CO

MPLS

-rs

Provider MTU-s

CO

ing on switches. PE-rs is Provider Edge equipment running eitherge [51].

Page 8: Metropolitan Ethernet Network: A move from LAN to MAN

CE1

CE2

CE3

UNI MEN

Point to Point EVC

UNI

UNI

CE1

CE2

CE3

UNI MEN

Point to Point EVC

UNI

UNI

Fig. 9. E-Line Service [28].

CE1

CE2

CE3

UNI MEN

UNI

UNI

CE1

CE2

CE3

UNI MEN

UNI

UNI

Fig. 10. E-LAN Services [28].

4874 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

performance assurances. Each UNI can multiplexmore than once Ethernet Virtual Circuit (EVC) ifthere are more than one EVC connected to it. InFig. 9, CE1 is multiplexing EVC from CE2 andCE3.

4.2. E-LAN service

Ethernet LAN service (E-LAN) is a multipoint-to-multipoint service connecting at least two UNIs[28]. Each UNI is connected to the same multipointEVC so that data sent from one UNI can bereceived at multiple ends, as shown in Fig. 10. Inaddition, if a new UNI is added, only the newUNI is needed to add to the multipoint EVC. Incontrast, if a new UNI is added in E-Line, a newEVC must be added to every existing UNI in thesame service. Similar to E-Line, E-LAN can providebest-effort service or quality-assured service withparameters such as CIR, CBS, PIR, PBS, jitter,delay, and loss performance.

5. Challenges

The MEF defined the five key attributes of a Car-rier Ethernet service: Standardized Services, Scala-bility, Resilience, Quality of Service, and ServiceManagement. Meeting these requirements wouldtruly enable the full potential and benefits of MEN

to support a large number of next generation appli-cations and services including: Business Services,Residential Triple-Play, and Mobile Backhaul.

5.1. Standardization

Currently, there are solutions that aim at resolv-ing issues in MEN remaining proprietary such asVLAN stacking [22], MAC-in-MAC [36], ExtremeStandby Router Protocol (ESRP) [17], or VirtualSwitch Redundancy Protocol [19]. However, inorder for E-Line and E-LAN services to be providedtransparently, it requires a ubiquitous service wherelittle or no changes to customer equipments onexisting networks. For this global compatibility toexist across different platforms, equipment vendorsmust be able to supply standardized equipments.Ideally, equipments from different vendors shouldbe able to work together to converge services suchas voice, video, and data and to converge legacytechnologies and emerging technologies. When cus-tomers need to change their subscriptions or switchto a different carrier, they would not need to com-pletely replace their existing equipments. Therefore,there should be a set of standardized requirementsthat require any Metro Ethernet certified equipmentto pass before it can be rolling off the assembly line.As of this writing, there are 17 MEF specifications[52] (MEF1, MEF5, and MEF5 are superseded byMEF10.1) that define the requirements for MENranging from service definition to test suites. How-ever, these specifications are recommendations andare not enforced as standardizations.

5.2. Scalability

With the fast growing Ethernet Services in MAN,a single MAN is expected to support thousands ofequipments that in turn providing services for mil-lion of users. The second key attribute of a CarrierEthernet require a robust and dynamic ability to sup-port growing number of users converging on a net-work where voice, video and data applicationsaggregate from variety of business enterprises andresidential areas. Furthermore, Metro Ethernet ser-vices expand from one Metro access network to otherMetro access network globally to support a widevariety of VLAN that exist in multiple remote geo-graphical regions. In addition, emerging applicationsincrease the bandwidth usage so that any MAN tech-nologies must be able to scale from 1 Mbps to10 Gbps. Unfortunately, very few Ethernet solutions

Page 9: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4875

are scalable. The traditional Ethernet managementprotocol, the Spanning Tree Protocol (STP), wouldnot be able to scale in MAN. At most, STP can spanto 7 hops. Beyond that, STP’s behavior is unpredict-able. Most technologies that were designed forEthernet were intended to run in a LAN environ-ment. Therefore, the move from LAN to MANrequires new considerations in MAN technology.

5.3. Resilience

The next key attribute of Carrier Ethernet is resil-ience that is defined by the detection and recoveryability of an Ethernet technology. The ideal networkwould perform smoothly and transparently to theuser in the face of failures meeting the demandedquality and availability as agreement in the sub-scription. After the fault detection, the networkshould autonomously recover from it. Recoverymeans that an alternate path is provided througha network reconfiguration or a backup path. Opticalnetworks set the bar at sub-50 ms for recovery timeand it has become the industry standard. There aredebates going on whether this sub-50 ms is needed.For some applications, a recovery of longer than50 ms is acceptable such as file transfer or email.

Despite its popularity and simplicity, the tradi-tional Ethernet does not meet this requirement. Therecovery time of the Spanning Tree Protocol (STP)that manages the Ethernet is in the range from 1 sto 60 s depending on the version of the STP, whichis considered drastically slow. Such performance hitscan interrupt or slow down applications, in turncause great financial loss to enterprises. Other Ether-net solutions have mixed success in this area. Someare very resilient but they require specific networkconfigurations or have high complexity that is costly.Others have lower cost in exchange for performance.

5.4. Quality of Service

One of the most important attributes that a Car-rier Ethernet must have is the support of end-to-endQuality of Service (QoS). It includes, but not limitedto, bandwidth, delay, jitter, and packet loss guaran-tees. These guarantees must be made from an end-to-end point of view. Service Level Agreements(SLAs) are the agreements on these metrics thatare negotiable between the client and the carrier toassure the end-to-end performance of voice, video,and data applications. MEF defines QoS specifica-

tion via the following metrics: CIR, frame loss,delay, and jitter.

In supporting QoS, there exist several traffic engi-neering mechanisms such as traffic policing and traf-fic shaping. Traffic policing is the act of droppingcustomers’ packets when they exceed the servicelevel agreement. It can be softening through themarking of packets that reach a certain thresholdsuch as the packet coloring scheme. The markedpackets are more likely to be dropped than theunmarked ones. Traffic shaping involves the isola-tion of traffic in each queue in order to protect fromthe burstiness of another queue by placing an upperbound on the maximum bandwidth available to atraffic class. Often included with QoS is networkload balancing that is the redirection of traffic flowsto prevent network load imbalance that leads totraffic congestion. Load balancing can be done atthe micro-level where the individual links are con-trolled in a distributed manner. Alternatively, itcan also be done at the macro-level where the loadis controlled per traffic class or domain.

In Ethernet, the widely adopted STP lacks the sup-port for assured QoS capability and load balancingcapability. At the most, it can provide marking forclass of service via the priority bits as defined in802.1p. Therefore, without further enhancement,Ethernet is not fit to be a carrier class technology.Many equipment vendors have implemented theirproprietary schemes in light of STP’s drawbacks butnone can provide a complete off-the-shelf solution.

5.5. Service Management

The last requirement for Carrier Ethernet asdefined by MEF is Operation, Administration, andMaintenance (OAM). OAM is the ability to monitor,diagnose, and manage the network autonomously orthrough a central standard interface implementationthat is vendor independence. Since the original inten-tion of Ethernet aims at LAN environment, it doesnot include the OAM capability in the standard.However, the large scale MAN environment requiresthe crucial OAM feature that exists in optical net-work. Little works have focused on this area exceptfor the on going standardizing process by the stan-dard bodies ITU, IETF, and MEF.

6. Architecture and performance

In this section, we will look at solutions devel-oped for Ethernet in response to the existing

Page 10: Metropolitan Ethernet Network: A move from LAN to MAN

SinkSource

Sink

Fig. 11. Virtual Concatenation (VC).

4876 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

challenges to push forward the transformation ofCarrier Ethernet. These challenges include resil-ience, load balancing, Quality of Service (QoS),and scalability. Efforts are ongoing in both acade-mia and industry to enhance each of these areasand to supplement the standardized protocols. Inthe MEN context, the presented solutions are cate-gorized based on where they would better bedeployed: the access network or the metro core.There is one class of solutions that would work inboth the access network or the core network butthey are not stand-alone architecture. These arecategorized as supplement solutions as they can beused concurrently with other protocols.

6.1. Metro access solutions

As earlier described and depicted in Fig. 3, a typ-ical topology of a metro access network is the meshtopology. In the dense mesh topology, there aremany redundant links to route traffic. Therefore, asuitable protocol to manage the access networkmust have a high utilization to take advantages ofthe redundant links. It must also be highly scalableto support millions of subscribers and to aggregatelarge volume of traffic from diverse platforms ontoa common platform. A summary of the solutionsin the metro access with respect to the 5 key attri-butes as defined by the MEF is shown in Table 2.

6.1.1. PESO

A proposed scheme aims to protect Ethernet traf-fic over SONET with a low overhead is calledPESO, proposed by Acharya et al. [1]. In traditionalSONET, voice traffic is supported by a primary andbackup path providing 100% protection. Thisapproach provides fast recovery upon failure butit imposes high operation cost. However, in datatraffic, whenever there is a failure, it is not necessaryto have 100% protection because it can tolerate the

Table 2Summary of the protocols for metro access

Solutions Standard Resilience Scalabi

PESO Academia Publication High ScalablAREA Industry, Proprietary High Low: li

VLANEthereal Academia Publication Low Low

SmartBridge Academia Publication Low LowSTAR Academia Publication Low Low

failure by running at a reduced rate. Depending onthe protection requirements, PESO will compute anoptimum routing path and using virtual concatena-tion (VC), as shown in Fig. 11, and Link CapacityAdjustment Scheme (LCAS) to make the necessaryrecovery. For the scenario where a single failureshould not affect more than x% of the bandwidth,PESO transforms the link capacity in the topologyto the equivalent STS-y line. Each chosen line can-not carry more than x% protected bandwidth.PESO determines the number of members in theVC. Using path augmentation maximum flow algo-rithm such as Ford and Fulkerson [47] or Edmondsand Karp [48], PESO determines the routes that thevirtual concatenation group (VCG) will take. Uponfailure, LCAS removes the failed member resultingin a continuous connection with the destinationbut the throughput has been reduced not less thanx% protected bandwidth. A variance of this protec-tion is that the provider wishes to minimize the per-formance degradation. Then first PESO must findthe value for the protected bandwidth capacitywhich is the bandwidth capacity remained after afailure between two extreme cases: if all VCG ondisjoint paths and if they are on the same path.Then PESO proceeds as earlier. The last case iswhen PESO must calculate the routing so that overprovision is supported to reconstruct the data ifthere is a fault so that the connection is still at fullthrottle in the face of failure.

Even though PESO is not ready to be used as-is,it has satisfied many of the 5 key attributes of Car-

lity QoS OAM Ready asstand-alone

e Assured bandwidth None Nomited bytag

MPLS supported None No

Assured end-to-endQoS metrics

None No

None None NoNone None No

Page 11: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4877

rier Ethernet. VC and LCAS provide high resiliencefor the network and guaranteed bandwidth for eachflow. Since assured bandwidth can be calculated,PESO can be extended to guarantee other metricsas well. In addition, PESO is better suited for themetro access network due to its ability to utilizethe redundant links when determining the routefor the VCG. What PESO missing is the OAM fea-ture before it can be used as a stand-alonearchitecture.

6.1.2. Atrica Resilient Ethernet Access (AREA)Atrica Resilient Ethernet Access (AREA) [4] is

the innovation from Atrica Network. To attain highresilience, AREA uses two recovery mechanisms:tunneling and hardware-based. The hardware-basedapproach uses a Hello messages to monitor forfailure. The tunneling protects aggregated link andnodes by redirecting traffic at wirespeed upon failuredetection to a pre-configured tunnel. Therefore, itcan achieve the recovery time of sub-50 ms. Itsupports both MPLS and VLAN tagging. For end-to-end, it uses MPLS; and for next hop, it usesEthernet VLAN. Because of the VLAN tag, AREAhas a low scalability limited by the VLAN space. Thetunnel is identified by an MPLS or VLAN protectionlabel generated by the ingress devices. Using MPLS,AREA can support the sophisticated QoS schemethat MPLS can provide. However, it is costly touse AREA in the metro access network in term ofoperational expenses and capital expenses due tothe supporting of MPLS. It is claimed to be compat-ible with Ethernet Spanning Tree Protocol family.Furthermore, AREA does not support OAM inthe original design. Therefore, it can be used as astand-alone architecture for Carrier Ethernet.

6.1.3. Ethereal

Ethereal [15] retains the distributed feature ofSpanning Tree Protocol with some new improve-ments. Ethereal, a real time connection-orientedarchitecture supporting best effort and assured ser-vice traffic at the link layer, proposes to use thepropagation order spanning tree for fast re-con-verge of the ST once a failure has been detected.Ethereal switches periodically send out hello mes-sage to the immediate neighbor switches. Absenceof any previous received hello message indicates anode/link failure. A hello message from a newswitch indicates an addition or substitution of aswitch. In either case, the switches that detect thefault immediately discard all best effort traffic, tears

down the established QoS-assured connections thattraverse through the offending link, re-converge thespanning tree, and reestablish any torn down con-nections. All of the best effort traffics are discardedbecause all destinations are unknown after a fault,which will require a flooding to deliver the traffic;and flooding without a spanning tree structurecauses loop in the network. The established QoS-assured connections that are not on the offendinglink can continue to be forwarded on the establishedpath without causing any problem. The spanningtree re-converges using the propagation order span-ning tree. The initiated switch sends out an invita-tion to its neighbors to join its spanning tree. If anode accepts, it propagates the invitation to itsneighbors. If a node receives all rejection from itsinvitation, it assumes that it is the leaf. Then it willsend a start up phase complete (SPC) to its parent.After a parent receives SPC from all of its children,it sends an SPC to its parent. This SPC propagatesuntil it reaches the root. The root will then sendtopology discovery phase complete (TDC) to all toindicate that it is now safe to forward on the estab-lished ST. If multiple nodes compete for the rootrole and send out multiple invitations, the othernodes pick the one with the lowest bridge id andpropagate the invitation. This is an improvementover the standard STP because the root role compe-tition is among groups of disjoint ST and notbetween every single switch.

Ethereal [15] demonstrate an example of runninga flow reservation mechanism. The Ethereal switcharchitecture is designed to meet the QoS require-ments for real time multimedia applications viahop-by-hop reservation. When an applicationmakes a request for connection, it sends QoSparameters, the destination IP address, and the des-tination IP port number. A Real Time Communica-tion Daemon (RTCD), developed by Ethereal,contacts the neighbor Ethereal switch and give it agenerated connection id. The connection id isunique on a per hop basic, similar to MPLS label.If the Ethereal switch can make the QoS commit-ment, then it contacts the next switch on the pathwith a unique connection id for this hop. The reser-vation propagates until it reaches the destinationand the periphery switch at the destination returnsa reply. If the reservation is successful, all theswitches on the path bind the routing entries withthe connection id and the QoS parameters. TheRTCD at the source binds the proxy Ethernetaddress with a proxy IP address into the Address

Page 12: Metropolitan Ethernet Network: A move from LAN to MAN

4878 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

Resolution Protocol (ARP) cache. Then it returnsthe proxy IP address to the request application.The proxy IP address has the format 1.1.XX.YYwhere XX.YY is the connection id. Similarly, theproxy Ethernet address has the format FF-FF-FF-FF-XX-YY where XX-YY is the connection id.The application then opens a UDP connection tothe destination with the proxy IP address. TheARP cache will translate the proxy IP address intothe proxy Ethernet address so that each switch onthe path can extract the connection id and remapsit to their locally unique connection id until theframes reach the destination. The last switch willneed to make a translation from the proxy addressto the real address before it can deliver the frameto the end host. Ethereal requires some cooperationwith the IP layer and the application. Furthermore,there is no mechanism for traffic priority to guardthe QoS commitments.

The nice feature about Ethereal is its support forflow reservation to guarantee the QoS metrics whichis one of the required key attribute for CarrierEthernet. However, Ethereal has poor scalabilitybecause of the number of connections it can make.The total number of connections it can commit toin the worse case is 2̂16 = 65,536 connections. Thisnumber is far too small for a metro access networkwhere the number of subscribers can reach millions.In addition, Ethereal dependency on Spanning Treegives it a low resilience status. Together with thelack of OAM support, Ethereal is not a completepackage for Carrier Ethereal. Ethereal can bedeployed in the metro core as well as the metroaccess. Although the scalability issue will mitigatedin the metro core, it still not ready for MAN dueto its low resilience.

6.1.4. SmartBridge

Because the Spanning Tree Protocol tends to for-ward frames toward the root, as the network sizegrows, the amount of inter-LAN traffic increasescausing a bottleneck at some bridges. Realizingthe congestion problem of inter-LAN forwarding,SmartBridge [10] was developed as a new architec-ture to scale traffic in LANs. Retaining the goodproperties of STP and combining with some goodfeatures of IP routing, SmartBridge proposes to for-ward frames along the shortest paths. It requires afull knowledge of the topology so that forwardingcan be done between hosts of known location alongthe calculated shortest path. The inventory con-struction and topology acquisition processes main-

tain the complete description of the topology.SmartBridge keeps track and update any changein network topology such as addition or removalof bridges. A host’s location information is kept ina table inside the SmartBridge. A host location revi-sion mechanism keeps track of all the hosts andupdates the table if necessary. For the purpose ofconsistency, frames with unknown source addressare dropped automatically and a topology acquisi-tion process will be initiated. Frames with unknowndestination address are flooded like the standardSTP but with slight modification to update the hostlocation table. Frames with known source and des-tination addresses are guaranteed to be forwardedon the shortest path that is calculated based on anassignment of weights so that any least-weight pathfrom source to destination is a shortest path in thetopology and the least-weight path from source todestination is unique.

SmartBridge introduces more complexity into thelink layer in order to enhance the performance of alarge network. This complexity will have a directaffect on the processing power of a bridge/switchand increase the controller traffic. The usage of IProuting increases SmartBridge’s topology utilizationby employing the redundant links. Therefore, it canbe deployed in the metro access network. However,the need to have the global topology with the stor-age for all destination addresses yields a low scala-bility making SmartBridge less desirable for themetro access. SmartBridge also lack the supportfor QoS and OAM feature. Therefore, it is notready to be deployed as the main architecture forMEN.

6.1.5. STARExploiting the fact that frames traveling on the

standard ST is not necessary the shortest path,Spanning Tree Alternate Routing [7] proposes anew forwarding scheme to enhance the forwardingperformance. The idea behind STAR is that the per-formance of a flow is affected by the length of theforwarding path. Therefore, STAR finds an alter-nate route that is shorter than the correspondingpath on the spanning tree. The metric that canbe used to determine the path might be delay,bandwidth, or any other required metric. EachSTAR-aware bridge has two routing tables: bridgeforwarding table (BF table), and host location table(HL table). The BF table indicates the forwardingport to other STAR-awared bridge along the short-est path, and the HL table maps an end host to a

Page 13: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4879

STAR-awared bridge. The BF table is found byusing a modified version of the distance vector algo-rithm. STAR uses BF table to find the shortest pathto another STAR bridge that is closest to the desti-nation; and then that bridge will deliver frame to thedestination. Since STAR use distance vector algo-rithm, for a given topology, it produces static pathsfor any computation. STAR is designed to coexistwith legacy STP bridges so that STAR deploymentcan be incremental. However, STAR has no distinc-tion between different classes of traffic. It forwardsframes along the shortest route as if they belongto the same class. It also does not provide any guar-anteed quality of service or establishes any servicelevel agreements (see Fig. 12).

Similar to SmartBridge, STAR finds alternatepaths to destination that is faster than going alongthe Spanning Tree path. However, this overlayapproach fails to distinct itself as the next genera-tion of Ethernet protocol for MEN. In the metroaccess, STAR runs the traditional Spanning TreeProtocol that has the re-convergence time of 30–60 s. This delay is not acceptable for the voice andvideo applications. Lacking the support for QoSand OAM, STAR cannot be the stand-alone solu-tion to run MEN.

6.2. Metro core solutions

The metro core topology in most cases is a ring[5] which is simpler than the mesh access network.Unlike a mesh topology, a ring topology is generallysimpler; therefore, the management is lighter andcertain behaviors are more predictable such as thedirection of traffic flows yielding a faster re-conver-gence time. The drawback with the ring topology isscalability. The latency of traversing the ring is pro-portional to the number of equipment on the ring.One solution is to break the large ring into smallerones. However, the difficulty lies on the manage-

1

3 2

4 5

6 7 8

1

2

4

6 7 8

STAR-aware bridge

Non-STAR-aware bridge

Fig. 12. (a) Physical LAN topology; (b) STAR overlay topology.

ment plane of interconnecting multi-rings. Theadvantages of using the Spanning Tree Protocol tomanage a network are low cost and simple to man-age. Therefore, it is more suitable for the core whereit only hauls large trunks of traffic from one end tothe other. The metro core topology is simple enoughfor used with a Spanning Tree Protocol while linkstate or MPLS are more complex than necessaryto set up. A summary of the solutions in the metrocore with respect to the 5 key attributes as definedby the MEF is shown in Table 3.

6.2.1. Spanning tree protocol family

Traditionally, Ethernet-based networks use thestandard spanning tree protocol (IEEE 802.1d) forrouting packets in the network. Spanning Tree Pro-tocol [20] is standardized in IEEE 802.1d. It is alayer 2 protocol that can be implemented in switchesand bridges. The spanning tree protocol (STP)essentially uses a shortest path to the central rootapproach in forming a tree that is overlaid on topof the mesh-oriented Ethernet networks, as shownin Fig. 13. Spanning tree is used primarily to avoidformation of cycles or loops in the network. UnlikeIP packets, Ethernet packets do not have a time-to-live (TTL) field. STP prevents loop in the networkby blocking redundant links. Therefore, the load isconcentrated on a single link which leaves it at riskof failures and with no load balancing mechanism.The root of the tree is chosen based on the bridgepriority, and the path cost to the root is propagatedthroughout so that each switch can determine thestate of its ports. Only the ports that are in the for-warding state can forward incoming frames. Thisensures a single path between a source and a desti-nation. Whenever there is a change in the topology,switches rerun the protocol that can take up 30–60 s. At any one time only one spanning tree dic-tates the network.

An improvement of STP is the Rapid SpanningTree Protocol RSTP [21] specified in IEEE802.1w. RSTP reduces the number of port statesto three: discarding, learning, and forwarding.Through faster aging time and rapid transition toforwarding state, RSTP is able to reduce the conver-gence time to between 1 and 3 s. In addition, thetopology change notification is propagated through-out the network simultaneously, unlike STP, inwhich a switch first notifies the root, and then theroot broadcast the changes, as shown in Fig. 14.The left of Fig. 14 shows that STP topology changeprocess takes 2 phases so that the delay lies in the

Page 14: Metropolitan Ethernet Network: A move from LAN to MAN

Table 3Summary of the protocols for metro CORE

Solutions Standard Resilience Scalability QoS OAM Ready asstand-alone

STP IEEE 802.1d Poor Low None None NoRSTP IEEE 802.1w Low Low None None NoMSTP IEEE 802.1s Low Low Manual load balance None NoViking Academia Publication High Low Some guaranteed metrics Central server NoRRSTP Industry, Proprietary Med Low None None NoEAPS IETF RFC 3619 Med to high Low (VLAN space

dependent)None None No

MRP Industry, Proprietary Med N/A None None NoH-VPLS IETF draft High Scalable N/A None No

1

2 3

4 1

2 3

4BPDU

BPDU

BPDU BPDU

BPDU

BPDU

BPDU

BPDU

RP RP

RPDPDP

DPBPDU

Root1

2 3

41

2 3

4 1

2 3

41

2 3

4BPDU

BPDU

BPDU BPDU

BPDU

BPDU

BPDU

RP RP

RPDPDP

DPBPDU

Root

Fig. 13. The distributed STP protocol broadcasts its control packet, BPDU, to elect a root switch and to select the shortest path from eachswitch to the root. Redundant links are blocked to prevent loop.

Fig. 14. Topology change notification in STP (left) and RSTP (right).

4880 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

propagation of the TCN message to the root. How-ever, the improvement in RSTP as shown on theright of Fig. 14, uses the source of the TCN messageas the root and broadcasts the TCN message fromit. Similar to STP, there is only one spanning treeover the whole network. RSTP still blocks redun-dant links to ensure loop free paths leaving the net-work underutilized, vulnerable to failures, andsupports no load balancing.

The latest upgrade to the Spanning Tree Protocolis Multiple Spanning Tree Protocol (MSTP) [23],defined in IEEE 802.1s. MSTP uses a commonspanning tree that connects all of the regions inthe topology called the Internal Spanning Tree(IST). The regions in MSTP are multiple instancesof the spanning tree. Each instance (MSTI) is an

instance of the RSTP. An instance of RSTP governsa region, where each region has its own regionalroot. The regional roots are in turn connected tothe common root that belongs to the common span-ning tree, as shown in Fig. 15. One or more VLANcan be assigned into a MSTI. By assigning differenttraffic flows into different VLAN, the network loadis balanced under the assumption that differentVLAN will take different path.

Although STP family has been used for mostEthernet networks and it could be very effective tobe used in the metro core, it has several shortcom-ings in the context of its use for MEN. These short-comings that are the reason why the Spanning TreeProtocol family is not ready to be used as-is forMEN are enumerated as follows:

Page 15: Metropolitan Ethernet Network: A move from LAN to MAN

Common Root

Regional Root for IST 1

Regional Root forIST 2

IST 1

IST 2

Fig. 15. A sample MSTP configuration.

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4881

1. Spanning trees restrict the number of ports beingused. In high-capacity Ethernets, this restrictiontranslates to a very low utilization of thenetwork.

2. Poor resiliency: a very high convergence time(STP: 30–60 s, RSTP: 1–3 s).

3. No mechanisms to balance load across thenetwork.

4. Lack of support for QoS and OAM.

6.2.2. Viking

Recognizing the poor resilience in the standardSpanning Tree, STP and RSTP, Viking, proposedby Sharma et al. [12], enhances the resiliency witha Multiple Spanning Tree architecture. Viking pre-computes multiple spanning trees so that it canchange to a backup spanning tree in the event of afailure. To precompute the spanning tree, Vikingfinds k-shortest primary path. For each of the pri-mary path, Viking finds k backup path. For eachpair of primary and backup paths, it rejects the pathif the bandwidth requirement is not satisfied. Thespanning trees are created when these paths aremerged together. The Viking server receives moni-tored information for the network condition. It usesthis information to compute the spanning tree peri-odically so that it can reply to any client query forspanning tree information.

Viking performs load balance by precomputingthe path to avoid the heavily used links. This canbe done by a cost formula that assigns the cost tothe network as more paths are added. The costformula relies on the expected cost of each linkwhich is the fraction of all the possible pathsbetween all source and destination pairs that passesthrough that link. The acceptance or rejection ofnew path requests depends on the output of the costformula that will determine if network will beoverloaded.

Viking delivers some quality of services by satis-fying bandwidth or delay requests. Although it doesprovide an end-to-end bandwidth or delay guaran-tee mechanism, it does not provide traffic policing,traffic shaping, traffic priority, or drop precedencein case of network congestion.

With respect to the 5 key attributes defined byMEF, Viking comes close to being the standalonetechnology for Carrier Ethernet to be deployed inMEN. Viking is versatile in that it can be deployedin the metro core as well as the metro access where itcan take advantages of the redundant links. Havinghigh resilience, Viking has no problem meeting thesub-50 ms criteria. However, more works areneeded to improve Viking’s support for QoS. It stilllacks required features in QoS like differentiation ofservice and assured end-to-end quality.

6.2.3. The specialized ring protocols

The following three protocols are specificallydesigned to manage ring topologies by takingadvantages of the unique characteristic of a ring.Although developed independently, they result insimilar concept. They are best deployed in the metrocore where it is most likely to be ring structure. Withthe appropriate configuration, they can achievegood resilience having sub-second re-convergence.However, they lack the support for QoS andOAM. Although suitable for the metro core, allthree protocol cannot be deployed as-is in MEN.

Riverstone Network leverages MSTP and RSTPto improve the turnaround time for port state forEthernet switches using the Rapid Ring SpanningTree Protocol (RRSTP), adhering strictly to ringtopologies [5]. Each spanning tree instance stayson its designated ring. A node will be the root switchfor that instance with a primary and alternate port.Initially, traffics enter the root will be sent on theprimary port. If a link on the primary path is bro-ken, the alternate port will open for use. Each nodeon the ring topology will be a root for an instance ofa MSTP. Since VLANs are used to distinguish span-ning tree, the VLAN space limits its scalability. Therecovery time after failure is approximately equal tothe BPDU hello time, which can be from 0.5 s to 1 s.

For link protection, Ethernet Automatic Protec-tion Switching (EAPS) is Extreme Network’s pro-prietary solution for resilience in the link layerthat also is currently specified in IETF RFC 3619[11,16]. It is designed only for a ring topology. Aring makes up of at least two switches. One of thenodes on the ring must be a master. The master

Page 16: Metropolitan Ethernet Network: A move from LAN to MAN

Link Domain

Control VLAN

Protected VLAN1

Protected VLAN2

Protected VLAN3

Link Domain

Control VLAN

Protected VLAN1

Protected VLAN2

Protected VLAN3

MasterSecondary port is blocked

Health msgdirection

X

Link down msgto master

Link down msgto master

MasterSecondary port is blocked

Health msgdirection

X

Link down msgto master

Link down msgto master

Fig. 16. (a) EAPS architecture; (b) fault detection in EAPS.

Master

Master

Ring 1 Ring 2

Master

Master

Ring 1 Ring 2

Fig. 17. A multi-ring topology.

4882 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

switch has a primary and backup port where ini-tially, traffic is sent on the primary port and thebackup port is blocked. An EAPS domain is config-ured to protect a group of VLANs as seen inFig. 16a. Multiple EAPS domains can exist on thesame ring protecting different set of VLANs. Eachdomain reserves one VLAN as the control VLAN.The control VLAN only sends and receives EAPSspecific control message. Layer 2 switching andaddress learning behave normally. Traffic belongingto a VLAN only flows through one direction on thering preventing loops from occurring. The mastersends out periodic poll from the primary port onthe control VLAN and to be received on the second-ary testing the ring connectivity. A non-masterswitch for a domain can send a link-down messageto the master if it detects a fault, as shown inFig. 16b. If a poll is timed out or a link-down mes-sage is received, the master declares a failed state,unblocks its secondary port to allow traffic goingthrough, flushes its forwarding database, and sendsa flush-DB message forcing the other switches toflush their forwarding database. The master contin-ues to send out polls on its primary port. If the linkis restored, the master blocks the secondary portand forces a database flush on all the other switches.If any of the other switches detect the recoverybefore receiving the notification from the masterswitch, it puts the traffic on the recovered port inblocked state, sets the state of the temporarilyblocked port to pre-forwarding. When it receivesthe flush-DB message from the master, it flushesthe entire forwarding database. If the state is setto pre-forwarding, it begins to forward traffic onthat port. By setting the timeout to be sub-second,the fault detection and recovery can be sub-second.Besides being limited by the VLAN space of 4096VLANs, a maximum of 64 EAPS domains can bedefined on a single switch/ring. Initial tests show

that the failover of EAPSv2 is less than 50 ms for10,000 layer 2 flows and 100 protected VLANs [46].

As an alternative to the standard STP, Foundrydelivered Metro Ring Protocol (MRP) [18]. MRPprevents loops and provides fast re-convergencefor ring topology only. A master node will be pickedfor each ring. On a given ring, the master has twointerfaces: primary and secondary. Initially, datatraffic travels on the primary path unless it fails thenthe master unblock the secondary port. The primaryinterface generates the Ring Health Packets (RHP).If the RHP messages reach the secondary port thenthe primary path is working properly. Multi-ringcan be merged to create a large topology but anMRP instance only runs on one ring and not theentire topology as seen in Fig. 17. The convergencetime after the fault detection is said to be sub-sec-ond, and there is no load balance mechanismbuilt-in.

6.2.4. Hierarchical VPLS

As an emerging technology for layer 2, VPLSuses the Martini encapsulation standard to trans-port Ethernet, ATM, and Frame Relay over thesame core network. A VPLS network can supportover a million unique labels, which means it is pos-sible to support over a million customers. VPLSalso suffers from the MAC address explosion prob-

Page 17: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4883

lem [38]. One solution is to use a router at the cus-tomer edge device that results in a single MACaddress per site. Another solution is to limit thenumber of MAC addresses that can be learned atthe provider edge per access circuit. A unique scala-bility issue with VPLS is that it uses MPLS tunnelsto create a fully meshed network and thus poten-tially requires a very large number of individualconnections. To cope with this problem, a hierarchi-cal VPLS (HVPLS) approach is needed as describedearlier.

Although HVPLS is not ready to be a CarrierEthernet technology, it is a good candidate. Havinghigh resilience as the result of taking after MPLS,HVPLS is also scalable. Even though QoS is notdefined yet in VPLS, it should not be too hard toextend it to support QoS similar to MPLS. Theone critical attribute that HVPLS misses is theOAM capability. With the appropriate configura-tion, HVPLS can be deployed to manage the wholemetro topology including the access and the core.

6.3. Supplement solutions

The following solutions that can be used as sup-plement to other protocols focus on enhancing oneof the 5 key attributes of the Carrier Ethernet. Theywould not be sufficient to serve as a stand-aloneMEN technology but they can be use concurrentwith any of the protocols that manage the metroaccess and metro core. A summary of the supple-ment solutions is shown in Table 4.

Table 4Summary of the specilized protocols

Solutions Standard Enhanced key attribute

LACP IEEE 802.3ad Load balance andresilience

TBTP AcademiaPublication

QoS: utilization and loadbalance

ESRP Industry,Proprietary

Resilience: standby node

VSRP Industry,Proprietary

Resilience: standby node

SuperSpan Industry,Proprietary

Scalability: spanning treescope

Q-in-Q Industry,Proprietary

Scalability: VLAN

MAC-in-MAC Industry,Proprietary

Scalability: MACaddresses

3 bit priority IEEE 802.1P QoS: class of serviceMEF bandwidth

profileMEFspecification

QoS: bandwidth profiling

6.3.1. Link Aggregation Control Protocol (LACP)

The Link Aggregation Control Protocol (LACP)or IEEE 802.3ad proposes to group multiple physi-cal ports together on a switch to create a single log-ical port. The benefit being that more ports can bemanaged as one connection; service provider canadd or remove bandwidth to the current connectionin chunk relative to the physical port bandwidth;load sharing and load balancing performs betweenlinks within a logical connection; and high resiliencyat the cost of reduced bandwidth if some of thephysical ports go down.

6.3.2. TBTP

Recognizing the inefficient utilization of the band-width that is caused by the overly restrictive Span-ning Tree protocol, Pellegrini et al. proposes anovel scheme, Tree-Based Turn-Prohibition (TBTP)[9], to loosen the restriction but still keep the networkoperational. STP prevents loops in the topology bypruning it down to a tree-structure path imposingsevere penalty on performance. Given a topologyand a formed spanning tree, TBTP constructs a lessrestrictive spanning tree by blocking a small numberof pairs of links around nodes, called turn, so that allcycles in a network can be broken. TBTP does notprohibit turns that are on the original spanning treein order to be backward compatible with the STP.In addition, the upper bound on the number of pro-hibited turns is at most half of the turns in the topol-ogy. Therefore, the algorithm guarantees that thetotal weight of the permitted turns is always greaterthan the total weight of the prohibited turns in thenetwork. By opening up more turns, TBTP providesmore paths to route Ethernet frames and lessen thecongestion on the main spanning tree. The benefitof TBTP is proportional to the degree of the nodes.However, TBTP did not improve on the recoverytime of the standard spanning tree protocol. SinceTBTP relies on the standard STP to re-convergebefore it can re-compute its routing, the recover timeis in the order of seconds.

6.3.3. ESRPA straight forward scheme for failure protection

is to provide backup node that normally in standbymode. In Ethernet, a switch is placed on standbyready to take over the primary equipment in casethe primary fails. Fig. 18 shows a typical exampleof an equipment redundancy protocol.

In providing node protection, Extreme has devel-oped the master and slave switch model called

Page 18: Metropolitan Ethernet Network: A move from LAN to MAN

Master Switch Slave/Backup Switch

Primary LinkBackup Link

Fig. 18. An example of providing switch redundancy.

4884 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

Extreme Standby Router Protocol (ESRP) [17] forthe mesh topology. In ESRP, there is a slave switchthat is in standby mode and all data frames to it areblocked unless the master switch fails. When themaster switch fails, the slave switch immediatelyassumes all functions of the master switch includingforwarding frames and MAC address learning with-out waiting for re-converge of the new topology.The recovery time after failure depends on the com-munication speed between the master and the slaveswitch. ESRP can scale up to 64 VLANs per ESRPdomain. ESRP is not backward compatible withSTP on the master and slave switch because bothprotocols cannot be configured to run on the sameinterface. Upon failure, the new master needs tosend out the Extreme Discover Protocol (EDP) mes-sages announcing the new master. The downstreamswitches then discard all address entries associatedwith former master. If there is no bi-directional con-nection between the new master and the down-stream switches, the recovery time could take upto five minutes. If the downstream switches arenot Extreme switches, another mechanism isrequired to inform the switches of the new master.

6.3.4. VSRPSimilar to Extreme, Foundry also has a proprie-

tary resilience protocol based on the principle of astandby switch. Foundry’s Virtual Switch Redun-

Boundary Interf

Core STP Domain 1 Core STP Domain

Fig. 19. Foundry’s SuperSpan di

dancy Protocol (VSRP) [19] has one master and atleast one backup switch. Initially, the master switchforwards all traffic. When it goes down, the down-stream switch will accept the backup switch as thenew master and the backup switch unblocks itsports for forwarding. The failover time is sub-sec-ond if all of the switches are VSRP-aware. Failovercan also occur even if the master switch is not com-pletely offline. The master switch’s priority isreduced each time a port fails. Therefore, over timeif the master’s priority is reduced to lower than thebackup switch, failover will occur.

6.3.5. SuperSpanTM

To enhance the scalability of the standard STP,Foundry Network has developed SuperSpanTM [41]that is based on the concept of divide-and-conquer.Intuitively, the network topology is divided intosmaller, easy-to-manage, and fast convergingdomains, as shown in Fig. 19. Each domain willrun a separate RSTP instance. A link going downin one domain will not cause the entire network tore-converge but its own domain. Since SuperSpanTM

uses STP, it prevents loops in the topology and for-wards frames on a single path. Between domains areboundary interfaces that connect adjacent domainstogether. They filter out local BPDU so that onedomain cannot affect another. For providers thatuse Per VLAN Spanning Tree (PVST) to isolate cus-tomers’ traffic for secured connection, Foundryclaims that SuperSpan can scale it to more than100 VLANs where normal system resources limit is.

6.3.6. VLAN tags

In Ethernet switching protocol, the IEEE 802.1Q[22] defines VLAN tag on each Ethernet frame. TheVLAN tag includes a VLAN ID to distinguish theframes belonging to different VLANs. Within a

ace

2 Core STP Domain 3

vides the provider network.

Page 19: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4885

VLAN tag, a three-bit Class of Service (CoS) fieldprovides up to eight classes of service for a VLAN.Light traffic policing can be enforced with VLANID. Enterprises use the VLAN ID to restrict broad-cast storms and to separate different services. Carri-ers use the VLAN ID to separate differentcustomers’ traffic. The mapping is one to one corre-spondence of one VLAN per customer per servicetype so that one customer cannot snoop on anothercustomer’s traffic. However, the VLAN ID field islimited to 12 bits or 4096 VLANs. This limits a car-rier to serve much less than 4096 customers becausethe carrier uses more than one VLAN on a singlecustomer to identify different services that the cus-tomer is subscribed to. Furthermore, the VLANspace is also shared with all the customers since eachcustomer will want to use the VLAN ID to identifytheir own services within their own network. If thecustomer edge equipments are layer 2 switches, thenall the customers MAC addresses will have to belearned and registered by the provider equipments[38]. This results in MAC table explosion problem,and whenever the Source Address Table periodi-cally refreshes, broadcast storm is possible. Anotherdrawback for VLAN is that the BPDUs are not

Provider NeCustomer Network

Payload

Type/Length

VLAN TCI 2Bytes

VLAN tag ID 0x8100

Src MAC6Bytes

Dest MAC6Bytes

Payload

Type/Length

VLAN TCI 2Bytes

VLAN tag ID 0x8100

Src MAC6Bytes

Dest MAC6Bytes

VLAN T2Bytes

VLAN taID 0x81

Payload

Type/Length

VLAN T2Bytes

VLAN taID 0x81

Src MA6Bytes

Dest MA6Bytes

VLAN T2Bytes

VLAN taID 0x81

Payload

Type/Length

VLAN T2Bytes

VLAN taID 0x81

Src MA6Bytes

Dest MA6Bytes

Customer VLAN tag

Add ProviderVLAN tag

Fig. 20. VLAN stacking

transparent to the provider network that couldcause undesirable results as explained later.

In the attempt to fix the VLAN ID shortageproblem, one approach, called Stacked VLAN [36]or Q-in-Q [38], is to stack another VLAN tag in frontof the original VLAN in the Ethernet header. SinceQ-in-Q is not standardized, many vendors offer thissolution as proprietary. As an Ethernet frame entersthe provider network from the customer network,the provider VLAN tag is added to the frameby the provider ingress switches, as shown inFig. 20. The second VLAN tag is used by the carriersto isolate traffic among different customers. Theegress switches of the provider network strip theprovider VLAN tag from a frame before it leavesthe provider network. Leaving the first VLAN taguntouched, the customers are free to assign its ownVLAN ID on its local network. Different enterprisescan overlap the VLAN IDs. However, Q-in-Q doesnot separate providers’ and customers’ MACaddresses. Therefore, the provider switches mustlearn all MAC addresses in the network includingthe customers’, creating the MAC table explosionproblem. The providers’ switches see both its ownnetwork and the customers’ network as one big

twork Customer Network

CI

g 00

CI

g 00

C

C

CI

g 00

CI

g 00

C

C

Payload

Type/Length

VLAN TCI 2Bytes

VLAN tag ID 0x8100

Src MAC6Bytes

Dest MAC6Bytes

Payload

Type/Length

VLAN TCI 2Bytes

VLAN tag ID 0x8100

Src MAC6Bytes

Dest MAC6Bytes

Customer VLAN tag

ProviderVLAN tag

StripProviderVLAN tag

scheme or Q-in-Q.

Page 20: Metropolitan Ethernet Network: A move from LAN to MAN

4886 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

network. Since there is no separation between pro-vider’s MAC addresses and customers’ MACaddresses, complications arise for Ethernet controlprotocols (e.g. BPDU). For example, a customer’sBPDU must not interact with the provider’s network.However, the Spanning Tree protocol is identified bythe fixed MAC address 01-80-C2-00-00-00. Thismeans that a spanning tree recalculation request fora customer’s network might trigger a ST recalculationin the provider’s network as well. From the scalabilityperspective, the provider is able to support up to 4096customers and still able to support separation of traf-fic among different customers. Each incoming C-VLAN ID and C-VLAN CoS is mapped to P-VLANand P-VLAN CoS while keeping C-VLAN ID andC-VLAN CoS unchanged. Therefore, the customerVLAN ID and VLAN CoS is preserved. Anotherapproach is that the provider uses both VLAN fields.For every incoming customer VLAN, it is translatedto the provider 24 bit VLAN. Therefore, the pro-vider can support up to 16 millions customers butthe difficulty lies in the translation between VLANIDs. If separation among different customers’ trafficis not required, the provider can set the P-VLAN tobe of service type and aggregate customer’s trafficinto P-VLAN ID based on the service needed e.g.E-Line, E-LAN, WAN, VoIP, and so on.

Provider NeCustomer Network

Payload

Type/Length

VLAN tag

Src MAC6Bytes

Dest MAC6Bytes

Payload

Type/Length

VLAN tag

Src MAC6Bytes

Dest MAC6Bytes

Src MA6Bytes

Dest MA6Bytes

Payload

Type/Length

VLAN t

Src MA6Bytes

Dest MA6Bytes

Src MA6Bytes

Dest MA6Bytes

Payload

Type/Length

VLAN t

Src MA6Bytes

Dest MA6Bytes

Customer MAC

Add Provider MAC

Fig. 21. MAC-in-M

6.3.7. MAC-in-MAC

Another tunneling scheme is the MAC-in-MAC(M-in-M) [36] that can isolate the provider’s MACaddresses from the customers’ MAC addresses.Hence, it resolves some drawbacks of 802.1q andQ-in-Q, such as customer control protocol transpar-ency and enhancing the scalability. Similar to theQ-in-Q concept, M-in-M prepends the provider’sMAC source and destination addresses to the Ether-net header at the provider ingress switch, as shownin Fig. 21. Switches within the provider network usethe provider MAC addresses to deliver frames to theegress switches. At the egress switches, the providerMAC addresses are stripped before any framesleave the provider network. M-in-M also has notbeen standardized. Nortel’s M-in-M proprietarysolution prepends provider’s MAC source address,MAC destination address, P-EtherType, P-VLANtag, and P-Service Label into the Etherframe frame.Since MAC addresses are allowed to overlapbetween the provider and customer, the spanningtree protocol’s fixed MAC address no longer posesa problem for the provider network. In addition,the provider network needs to learn only MACaddresses from its own switches and not all the cus-tomer’s MAC addresses as before. Therefore, theMAC table explosion is mitigated. However, the

twork Customer Network

C

C

ag

C

C

C

C

ag

C

C

Customer MAC

Provider MAC

Strip Provider MAC

Payload

Type/Length

VLAN tag

Src MAC6Bytes

Dest MAC6Bytes

Payload

Type/Length

VLAN tag

Src MAC6Bytes

Dest MAC6Bytes

AC approach.

Page 21: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4887

complication is at the ingress and egress switcheswhere the translation/mapping between customers’MAC addresses and provider’s MAC addresses.These ingress and egress switches still need to learnall of the customers’ MAC addresses in order tomake the translation. It is possible for Q-in-Q andM-in-M to coexist as a hybrid solution. As sug-gested by Nortel Network [36], the Q-in-Q is usedin the Metro Access and M-in-M is used in theMetro Aggregation Network, as shown in Fig. 22.When the customer frames pass to the metro accessnetwork, the provider VLAN tag is added on top ofthe original frame. The provider MAC labels arestacked on the frames coming from the metroaccess. Both additional labels are stripped off asthey move out of their respective area.

6.3.8. Traffic class of service

As a simple protocol, the standard IEEE 802.1Ddoes not describe any sophisticated scheme for QoS,admission control, or traffic policing as in DiffServe,IntServe, and MPLS. Light traffic policing is possi-ble by defining VLAN configurations that map traf-fic type to priority queues. However, there are noguaranteed services such as bandwidth reservation.The standard defines eight traffic types, in order ofpriority: background, spare, best effort, excellenteffort, controlled load, video, voice, and networkcontrol. Network control has the ‘‘no loss’’ require-

Metro Access

Network

Customer Network

MAg

C

Customer FrameQ-in-Q

P-VLAN is addedM

P-

Provider Service Label

Provider VLAN tag

Provider Ethertype

Provider Src MAC

Provider DestMAC

Provider Service Label

Provider VLAN tag

Provider Ethertype

Provider Src MAC

Provider DestMAC

Fig. 22. A hybrid approach of M-in-M and Q-in-Q. The provider servmapped to.

ment to maintain and support the network infra-structure. Voice must be less than 10 ms delay andvideo must be 100 ms delay. Control load is impor-tant business application traffic that is subjected tosome form of ‘‘admission control’’. Depending onthe number of queues on a switch, 802.1D dividesthese traffic types among the priority queues (Table5). For example, if there are three queues, then thetraffic types are divided as follow: {best effort, excel-lent effort, background}, {controlled load, video},and {voice, network control}. The 802.1P definesa 3 bit user priority field within the Ethernet headerfor differentiation of services. Then there are a max-imum of eight priorities can be supported on aswitch.

To cope with the QoS insufficiency in Ethernet,MEF defines traffic management mechanisms thatinclude bandwidth profiling at the edge and insidethe MEN [30]. At the MEN ingress, the CustomerEquipment VLAN ID (CE-VLAN ID) is mappedto an EVC. Then, the EVC or a combination ofEVC and the Customer Equipment VLAN Classof Service (CE-VLAN-CoS) determine the Class ofService (CoS) instance to be used inside the MEN.Bandwidth profiling is a set of parameters that theservice providers use to control the incoming trafficinto the MEN so that they meet the Service LevelSpecification (SLS) that is an agreement with thecustomers. These parameters are input into a

etrogregation/

ore Network

AC-in-MACMAC is added

Customer Payload

Customer Ethertype

Customer VLAN tag

Customer Src MAC

Customer DestMAC

Customer Payload

Customer Ethertype

Customer VLAN tag

Customer Src MAC

Customer DestMAC

ice label contains the service id for which the customer frame is

Page 22: Metropolitan Ethernet Network: A move from LAN to MAN

Table 5Recommened user priority to traffic class mappings [22].

User priority Number of available traffic classes (queues)

1 2 3 4 5 6 7 8

0 (default) 0 0 0 1 1 1 1 21 0 0 0 0 0 0 0 02 0 0 0 0 0 0 0 13 0 0 0 1 1 2 2 34 0 1 1 2 2 3 3 45 0 1 1 2 3 4 4 56 0 1 2 3 4 5 5 67 0 1 2 3 4 5 6 7

Note: User priority value mapping to traffic types are as follow-ing: 0 – best effort, 1 – background, 2 – spare, 3 – excellent effort,4 – controlled load, 5 – video, 6 – voice, 7 – network control.

4888 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

bandwidth profile algorithm that verifies the confor-mance of the traffic and performs the necessaryoperations including dropping or recoloring thetraffic frames to indicate the drop precedence. Theframe color and their meaning are: green indicatesthat the frame is in-profile, yellow indicates thatthe frame is out-of-profile and drop if the networkis congested, and red means that the frame is tobe dropped immediately. The six parameters to con-trol the traffic rate are Committed Information Rate(CIR), Committed Burst Size (CBS), Excess Infor-mation Rate (EIR), Excess Burst Size (EBS), Cou-pling Flag (CF), and Color Mode (CM). CMindicates whether or not the frame color is takeninto account when determining the conformity ofthe traffic. The value of CF has the effect of control-ling the volume of the yellow frames traffic. Thealgorithm uses token buckets to determine if thetraffic conforms to the SLS. Initially, there are twobuckets that are full of tokens. As frames enterthe provider’s network, tokens from the first bucket(green bucket) is decremented by the size of theframes. If the green bucket is not empty, the frameis CIR-conformant and is allowed into the network.If it is empty, the second bucket (yellow bucket) isdecremented. If yellow tokens are available, thenthe frame is colored yellow and allowed into the net-work. If there are no more yellow token, then theframe is declared red and discarded. Bandwidthprofiling can be applied per ingress UNI, perEVC, or per CoS. The profiling granularity goesin the order of increasing granularity from peringress UNI to per CoS since each UNI is composedof multiple EVCs, and each EVC is composed ofmultipled CoS. In addition, the document alsodefines the frame delay performance, frame jitterperformance, and frame loss ration. Frame delay

performance is the P-percentile of the delay for allgreen frames successfully delivered for a UNI pairwithin a time interval. Frame jitter performance isthe P-percentile of the difference of the one-waydelay of green frame pairs that arrive at the ingressUNI within a time interval. The frame loss ratio isthe percentage of the number of green frames thatare loss over the total number of green frames thatarrive at the UNI.

7. Operation, administration, and maintenance

(OAM)

One of the features that imposes a high degree ofobstruction to Ethernet from being a standalonecarrier-grade technology is the lack of Operation,Administration, and Maintenance (OAM). Othertechnologies such as SONET and ATM haveOAM capabilities within the data link layer [14].OAM deals with the network performance monitor-ing and maintenance. It includes predefined vari-ables about the status of the network such asdelay, loss, jitter, and bandwidth availability thatcan be sent in-band or out-of-band. In-band signal-ing, like SONET/SDH, sends OAM informationattached to the data. This is a closer measurementof the network performance. However, there is arisk that the OAM information might affect andbias the measurement. Out-of-band signaling sendsthe OAM information on a different path than thedata path like MPLS echo request. Therefore, care-ful implementation must be enforced to ensure thatthe OAM frames are treated as closely as possible tothe data frames in order to get an accurate reading.Ref. [2] speculates that out-of-band will likely be thecase for Ethernet OAM. To measure frame delayand jitter, [2] suggests one way to measure delay/jit-ter but it requires clock synchronization at endhosts. One approach is to use primary referenceclock (PRC) like in SONET that is derived fromGPS. However, this technique is still complex andnot widely available for Ethernet. Roundtrip mea-surement is not accurate because the reverse direc-tion might not take the same path. Frame loss andthroughput are also important and can be measuredby counting packet arrival at one end in a giventime. Late frames must be dropped from the count-ing for certain applications such as voice and video.Link level failure detection uses hello messages andpasses on the information about the failure in bothup and down stream. Service failure detection is alsoneeded to in case nodes and links are online but

Page 23: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4889

flows get interrupted such as in the case of poisonforwarding table. A connectionless approach is hardfor OAM because its network resources are spreadthroughout the network as opposed to the naileddown path in a connection-oriented paradigm [2].Moreover, Ethernet runs a risk of broadcast stormthat is a challenge for traffic management such asenforcing SLA on customers.

7.1. MEF standards

The MEF current work on defining OAM forEthernet does not focus on single link OAM mech-anisms. That would overlap with some of theIEEE’s drafts. However, MEF defines OAM mech-anisms on multilink such as edge-to-edge intra-car-rier OAM, edge-to-edge inter-carrier OAM, andend-to-end customer OAM. They suggest that themeasurements must be per VLAN and be with thedata plane. This means that in-band signaling willbe used for more accurate measurements and userdata is mixed with OAM. Currently, the draftincludes connectivity, latency, loss, and jitter forSLA metrics. The defined OAM frame is the sameas the data frame but is differentiated by the multi-cast address for OAM discovery and the Ethertypefield. An OAM barrier filters out OAM at the edgesof the domain to prevent leaking OAM from oneprovider to another provider or customer. Domainsare defined as intra-provider, inter-provider, andcustomer-to-customer. For discovery operations,an edge switch sends a multicast ping request. Otheredge switches response to the ping. Then the reques-ter constructs a list of all the edge switches. Auto-matic discovery is useful for plug-n-play anddiagnostic. Loss measurement is performed by uni-cast ping n times, the packet loss is m/n wherem � n requests are responded. Latency is measuredthrough roundtrip time. For delay measurement,the source attaches a ‘‘relative’’ timestamp in therequest. The receiver calculates the delay by theinter-transmit time which is recorded in the time-stamp or the inter-received time via the actual pingreceived time.

7.2. ITU standards

ITU works on the requirements for EthernetOAM or Y.1730 and Ethernet OAM mechanismY.17ethoam [3]. Y.1730 defines the motivation andrequirements for user-plane OAM including therequired OAM functions for point-to-point and

multipoint-to-multipoint in dedicated and sharedaccess. Y.17ethoam defines the mechanisms forfault management, performance measurement, anddiscovery. Ethernet OAM frames format is alsoincluded.

7.3. IETF standards

IETF [13] is working on a draft entitled ‘‘Ether-net in the First Mile (EFM) OAM MIB’’ for singleEthernet link OAM. It is expected to complementSNMP management by defining the basic functionsat layer 2 supporting directly connected Ethernetstations. The draft focuses on three areas: link faultindicator, link monitor, and control remote loop-back. Link fault indicator enables one Ethernetend host to signal the other end that the path isnon-operational. In addition, it allows a mechanismto operate in unidirectional mode so that the linkcontinues to operate in one direction even thoughthe reverse direction has failed. Link monitoringincorporates into SNMP the ability for an end hostto signal the occurrences of certain important eventsvia layer two. There are also mechanisms for anEthernet station to query its adjacent neighbor forthe status of its interface. Remote loopback is whenan Ethernet host station echoes back every receivedpacket onto the link. The draft defines object con-trolling the loopback and reading the status of theloopback state.

8. Security

In the past, LANs have been under the control ofan organization in a small and contained area thatwould not span across different networks. There-fore, layer 2 is considered to be trusted and littlework have been done in security for layer 2. How-ever, this assumption is invalidated as long as thereare inside-attackers. The movement of extendingEthernet to the metro core opens up more opportu-nities for attackers to exploit the vulnerabilities ofthe network. Current intrusion detection mecha-nisms, filtering rules, and firewall only work at layer3 and above. Therefore, it does not directly protectthe vulnerable Metro Ethernet network where layer2 is the underlying technology. Marro [8] shows thatby exploiting the lack of authentication in BPDUmessages, an inside-attacker can perform Denial ofService (DOS) attacks by creating loop in the span-ning tree or preventing the tree formation. A host

Page 24: Metropolitan Ethernet Network: A move from LAN to MAN

1

2

33

ST before the attack

ST after the attack

AttackerAttacker

Fig. 23. Loop forms when topology is under attack.

4890 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

can also snoop the network by impersonate as aswitch to gain confidential data.

8.1. Vulnerabilities

The most obvious and crucial weakness in Ether-net is the lack of authentication for Bridge ProtocolData Unit (BPDU) messages. BPDU messages areused to administrate the operations of an Ethernetnetwork. Any end host connected to a switch cangenerate well-formed BPDU message forcing theswitch to process. This enables the host to masquer-ade as a working switch to join in the active topol-ogy. As the result, the attackers can perform a DOSattack, disrupting data forwarding, and Man-in-the-Middle attack, snooping traffic going through thatoriginally was not intended for the attacker.

Ethernet runs the standard 802.1d Spanning TreeProtocol family that includes vulnerability in whichthe root role is not fully monitored. For example,when a switch discovers a change in topology, itkeeps sending a Topology Change Notification(TCN) BPDU up the tree toward the root until itreceives an acknowledgement from the immediateneighbor. In STP, after the root receives this BPDU,it is up to root to generate subsequence Configura-tion Message BPDUs with the Topology Change(TC) flag on. However, the originator does not keeptrack of the pending operation. Therefore, a com-promised root can acknowledge the TCN BPDUand not generate subsequence BPDU with the TCflag on, the rest of the switches will not detect anyfurther changes aiding the success of a DOS attack.

There are two categorized attack-approachresulted from the combination of the two mentionedvulnerabilities: flooding attack and topologyengagement attacks [8]. The following subsectionsdescribe the variants and attack scenarios.

8.2. Flooding attacks

A flooding attack is a brute force attack thatsends a steady flood of bogus BPDUs disruptingthe normal behavior of the Spanning Tree Algo-rithm. The first variant of flooding attack is a floodof Configuration Message BPDUs with the TC flagon. The second variant of flooding attack sends asteady flow of bogus Topology Change Notificationmessage propagating up the tree. The last variant offlooding sends special messages from the attackerclaiming to be new to the topology and having rootpath cost of zero. Its purpose is to poison the for-

warding table of the target switch. Theseapproaches force the spanning tree algorithm tocontinuously recalculating so that it would not gointo forwarding state. Thus, a DOS attack is suc-cessful when no data packet is forwarded.

The experiments from [8] suggest that the higherthe target switch is in the tree hierarchy, the moreeffective the flooding attack is to degrade the net-work performance. One possible reason is that thehigher level switches handle more trunk traffic thanswitches that are lower in the hierarchy. Therefore,they consume more computational resources so thatthey are more susceptible to resource drainingattacks.

While under a flooding attack, it is possible forthe topology to form a loop. When a switch underattack is computationally compromised, it absorbsall incoming BPDUs and does not generate anynew ones. This is switch 1 from Fig. 23. Conse-quently, it is unable to participate in the ST proto-col. The ports of the uncompromised switches thatface the compromised switch believe that they areports connecting to a non-switch node. Then, theoriginal tree topology changes to a new one neglect-ing the compromised switch. However, the logicalconnectivity still uses the old path. This means thatthe ports of the uncompromised switches that facethe compromised switch do not change roles andstay forwarding. When a station that sends ICMPrequests stop receiving consistent ICMP replies traf-fic through the compromised switch, it issues ARPrequests that open up the loop for the topology.

8.3. Topology engagement attacks

In this class of attack, the attacker claims to beone of the switches participating in the spanningtree protocol. It is different from flooding in thatit only sends a single BPDU per hello time period.In the BPDU, the original root bridge ID and otherparameters are preserved except for the root pathcost which is incremented by a hop. By gaining a

Page 25: Metropolitan Ethernet Network: A move from LAN to MAN

AttackerAttacker

AttackerAttacker

22

1144

33

66

55AttackerAttacker

AttackerAttacker

2211

443366

55

Fig. 24. Topology segmentation is created by topology engagement attack.

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4891

role in the topology, it is able to snoop traffic goingthrough. The closer to the root the attacker is, thehigher the volume of traffic the attacker can snoop.In addition, the attacker can stop propagating theTCN information up the tree defeating the resiliencemechanism of STP. The general case of this attack iscalled internal node role claiming. A more specificcase of the attack is when the attacker claims theroot role. A host can claim to be a root by sendinga configuration BPDU with the bridge ID lowerthan the current root’s ID. When the fake rootreceives a TCN BPDU from a regular switch, itacknowledges that switch but it fails to set the TCflag on in its subsequent configuration BPDUs. Asa result, the TC information does not propagatedown the tree to invoke the ST recalculation. Thisattack strikes at the resilience mechanism of theSpanning Tree Protocol. It is the direct result ofthe vulnerability in which the root role is not fullymonitored. However, it is not effective againstRSTP because RSTP does not rely on the root topropagate the topology change information. Whilebeing the root, the attacker can change any param-eter in the BPDU to cause further instability in thenetwork. A variant of this attack engages theattacker to more than one host through multi-home.It is similar to the single host attack except, theattacker send the BPDU message per hello timeper interface (home) targeting multiple switches.Both of these variants can do snooping of trafficas well. As the result of snooping, the attackerdefeats the purpose of VLAN separation. Confiden-tial traffic between different enterprises can bescreened by the attacker.

Finally, the attacker can segment the networktopology by having two or more hosts claim to havethe same bridge ID that is lower the root bridge ID.Each attacking host target a different switch in thenetwork sending a single BPDU message per hellotime per interface. All switches in the networkreceive more than one advertisement on the samenew root. Because the shortest path is picked, thetopology is segmented as shown in Fig. 24.

9. Current deployments

At the present, service providers tend to offerEthernet over some optical network in MEN so thatthe services can enjoy the de-facto sub-50 ms recov-ery time. The common trend is to have Ethernetdeploys at the access and aggregation part of themetro area network. PRP or MPLS is the technol-ogy for the metro core. The customer premiseequipment can be an Ethernet switch or a router.

There are several deployments of Metro EthernetNetwork around the globe supporting a wide rangeof applications. AT&T installs a multipoint VPNE-LAN providing high-speed connection among 12locations of the Clarian Health Center in Indiana,USA [31]. The services include voice and data inter-connecting their healthcare systems providing acces-ses to up-to-date medical research, clinical expertise,and patient values to reduce the variance of carebetween physicians giving the same patient and phys-ical condition. The city of Roanoke, Virginia, USA,deploys Ethernet over SONET network using RPR[35]. The city has OC-48 SONET backbone betweenten sites. They are also running two RPRs at OC-12to minimize disruption from future expansions andnew applications. Their application is a web portalthat allows citizens to pay parking tickets and taxonline, obtain registrations and permits, view parksrecreation information, and satellite images of prop-erties for real estates and development purposes. Autility company in Idaho, USA, the Idaho Falls Pow-ers (IFP), creates a communication network of theirown after realizing the flexibility and scalability ofmetro Ethernet technology avoiding the constrainedof the incumbent carriers for new leased services [32].Their network includes an installation of LuminousNetworks PacketwaveTM platforms deploying inRPR rings with MPLS abstraction. It can providethe transportation of multiple services with variousCoS. To its customer, IFP offers transport of missioncritical data, Ethernet private line, multiple servicessuch as VoIP, and video surveillance. IFP leaseswavelength instead of fiber strands with resiliency

Page 26: Metropolitan Ethernet Network: A move from LAN to MAN

4892 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

and automatic protection for the fiber. At the sametime, IFP requires little service level staff to maintainthe network.

A Korean-based wholesale provider of metroEthernet, PowerComm, offers Ethernet services toresidents in Seoul and integrating with their existingcable network at the same time [33]. PowerCommwants to provide gigabit Ethernet with the reliabilityas the traditional SONET/SDH but at a lower cost.It requires one metro backbone infrastructure tounify its existing cable network with the new last-mile Ethernet services. Gigabit Ethernet links aredeployed in parallel between all of its Points of Pres-ence (POP). The Gigabit Ethernet POPs are con-nected parallel with the central backbone POPsand to the Hybrid Fiber Coax cable network. TheMetro Access connections aggregate into districtsand regional POPs and then into the redundantGigabit Ethernet Network backbone. The result isa ring of rings and each of the Ethernet rings runsRRSTP from Riverstone (see Section 6.2.3). Power-Comm also implements layer 2 MPLS E-Line ser-vices and VPLS E-LAN services. The access portsand regional rings use hardware-based rate limitingand shaping to control bandwidth and QoS in a sin-gle management system.

In Spain, a company, called Al-Pi, saw the poten-tial of Metro Ethernet in 2001 and decided todeploy Gigabit Ethernet services in Barcelona [34].At the start, it offers LAN to LAN services usingthe enterprises class Ethernet switches. Theseswitches lack features such as sub-50 ms resilience,SLA control, high scalability, and end-to-end QoSguarantee. Then it makes a move toward opticalMetro Ethernet using dense fiber network. Now itcan offer carrier class services to its customer.

10. Acronym

1. AC: Attachment Circuit2. AREA: Atrica Resilient Ethernet Access3. ATM: Asynchronous Transfer Mode4. BGP: Border Gate Protocol5. BPDU: Bridge Protocol Data Unit6. CBS: Committed Burst Size7. CE: Customer Edge/Equipment8. CIR: Committed Information Rate9. CoS: Class of Service

10. CPE: Customer/Provider Equipment11. CSMA/CD: Carrier Sense Multiple Access

with Collision Detection12. CWDM: Coarse Wave Division Multiplexing

13. DOS: Denial of Service14. DWDM: Dense Wave Division Multiplexing15. EAPS: Ethernet Automatic Protection

Switching16. EDP: Extreme Discover Protocol17. E-LAN: Ethernet LAN service (a multipoint

service)18. E-Line: Ethernet Line service (a point-to-point

service)19. EoA: Ethernet over ATM20. EoMPLS: Ethernet over MPLS21. EoS: Ethernet over SONET22. EoWDM: Ethernet over Wave Division

Multiplexing23. ERS: Ethernet Relay Service24. ESCON: Enterprise System Connection25. ESRP: Extreme Standby Router Protocol26. EVC: Ethernet Virtual Circuit27. EWS: Ethernet Wire Service28. FDDI: Fiber Distributed Data Interface29. FR: Frame Relay30. HVPLS: Hierarchical VPLS31. IEEE 802.17: Resilient Packet Ring Protocol32. IEEE 802.1ad: Q-in-Q or VLAN Stacking33. IEEE 802.1D: Spanning Tree Protocol34. IEEE 802.1P: LAN Layer 2 CoS Protocol for

Traffic Prioritization35. IEEE 802.1Q: VLAN tag36. IEEE 802.1s: Multiple Spanning Tree

Protocol37. IEEE 802.1w: Rapid Spanning Tree Protocol38. IEEE 802.3: Ethernet Protocol39. IEEE 802.3ad: Link Aggregation40. IETF: Internet Engineer Task Force41. IP: Internet Protocol42. IST: Internal Spanning Tree43. ITU-T: International Telecommunications

Union-Telecommunications Standard Sector44. L2: Layer 245. LAN: Local Area Network46. LCAS: Link Capacity Adjustment Scheme47. LDP: Label Distribution Protocol48. LSP: Label Switching Path49. MAC: Media Access Control50. MAN: Metropolitan Area Network51. MEF: Metro Ethernet Forum52. MEN: Metropolitan Ethernet Network53. M-in-M: MAC-in-MAC54. MPLS: Multi-Protocol Label Switching55. MRP: Metro Ring Protocol56. MSTI: Multiple Spanning Tree Instance57. MSTP: Multiple Spanning Tree Protocol

Page 27: Metropolitan Ethernet Network: A move from LAN to MAN

M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894 4893

58. MTU: Multi Tenant Unit59. OAM: Operation, Administration, and Main-

tenance60. PBS: Peak Burst Size61. PE: Provider Edge/Equipment62. PIR: Peak Information Rate63. PL: Private Line64. POP: Points of Presence65. PRC: Primary Reference Clock66. PW: Pseudowire67. Q-in-Q: VLAN Stack68. QoS: Quality of Service69. RHP: Ring Health Packet70. RPR: Resilient Packet Ring71. RRSTP: Rapid Ring Spanning Tree Protocol72. RSTP: Rapid Spanning Tree Protocol73. SDH: Synchronous Digital Hierarchy74. SLA: Service Level Agreement75. SLS: Service Level Specification76. SONET: Synchronous Optical Network77. SPC: Startup Phase Complete78. STP: Spanning Tree Protocol79. TCN: Topology Change Notification80. TDM: Time Division Multiplexing81. TTL: Time-To-Live82. UNI: User Network Interface83. VC: Virtual Circuit84. VCG: Virtual Circuit Group85. VLAN: Virtual LAN86. VPLS: Virtual Private LAN Service87. VPN: Virtual Private Network88. VPWS: Virtual Private Wire Service89. VSRP: Virtual Switch Redundancy Protocol90. WDM: Wave Division Multiplexing

References

[1] S. Acharya, B. Gupta, P. Risbood, A. Srivastava, PESO: lowoverhead protection for Ethernet over SONET transport, in:Proceedings of IEEE INFOCOM 2004.

[2] D. Cavendish, Operation, administration, and maintenanceof ethernet services in wide area networks, IEEE Commu-nications Magazine (2004).

[3] S. Clavenna, Standardizing Ethernet Services, January9th, 2004. <http://www.lightreading.com/document.asp?doc_id=45328&print=true>.

[4] T. Gimpelson, Atrica makes Ethernet resilient, NetworkWorld Fusion, January 02, 2002. <http://www.nwfu-sion.com/edge/news/2002/0122atrica.html>.

[5] G. Holland, Carrier class metro networking: the highavailability features of Riverstone’s RS metro routers,Riverstone networks technology whitepaper #135. <http://www.riverstonenet.com>.

[6] M. Jander, VPLS: the future of VPNs? March 21st,2003. <http://www.lightreading.com/document.asp?doc_id=30038>.

[7] K. Lui, W.C. Lee, K. Nahrstedt, STAR: a transparentspanning tree bridge protocol with alternate routing, ACMSIGCOMM Computer Communications Review 32 (3)(2002).

[8] G.M. Marro, Attacks at the data link layer, Master Thesis atUC Davis 2003.

[9] F. De Pellegrini, D. Starobinski, M.G. Karpovsky, L.B.Levitin, Scalable cycle-breaking algorithms for gigabitEthernet backbones, in: Proceedings of IEEE INFOCOM2004.

[10] T.L. Rodeheffer, C.A. Thekkath, D.C. Anderson, Smart-Bridge: a scalable bridge architecture, in: Proceedings ofACM SIGCOMM 2000.

[11] S. Shah, M. Yip, Extreme networks, Ethernet automaticprotection switching EAPS RFC 3619, October 2003.

[12] S. Sharma, K. Gopalan, S. Nanda, T. Chiueh, Viking: amulti-spanning-tree Ethernet architecture for metropolitanarea and cluster networks, in: Proceedings of IEEE INFO-COM 2004.

[13] M. Squire, Ethernet in the First Mile (EFM) OAM MIB,IETF drafts October 2004. <http://www.ietf.org/internet-drafts/draft-ietf-hubmib-efm-mib-02.txt>.

[14] M. Squire, Metro Ethernet Forum OAM, Hatteras Net-works. <http://www.metroethernetforum.org/presentations/MEF%20OAM%202003-12-01.pdf>.

[15] S. Varadarajan, T. Chiueh, Automatic fault detection andrecovery in real time switched Ethernet networks, in:Proceedings of IEEE INFOCOM 1999.

[16] Extreme Networks, Ethernet automatic protection switchingEAPS, whitepaper. <http://www.extremenetworks.com>.

[17] Extreme Networks, Extreme Standby Router ProtocolTM andVirtual Routing Redundancy Protocol, Whitepaper. <http://www.extremenetworks.com>.

[18] Foundry Networks, Foundry switch and router installationand basic configuration guide, configuring metro fea-tures. <http://www.foundrynet.com/services/documentation/sribcg/Metro.html> (Chapter 13).

[19] Foundry Networks, Foundry switch and router installationand basic configuration guide, configuring metro fea-tures. <http://www.foundrynet.com/services/documentation/sribcg/Metro.html#61625> (Chapter 13).

[20] IEEE Information technology – telecommunications andinformation exchange between systems – local and metro-politan area networks – common specifications. Part 3:Media Access Control (MAC) bridges, ISO/IEC 15802-3,ANSI/IEEE Std 802.1D, 1998.

[21] IEEE Standard for Local and metropolitan area networks –common specifications. Part 3: Media Access Control(MAC) bridges – Amendment 2: Rapid ReconfigurationAmendment to IEEE Std 802.1D, 1998 Edition, IEEE Std802.1w-2001.

[22] IEEE Standard for Local and Metropolitan Area Networks:Virtual Bridged Local Area Networks, IEEE Std 802.1Q-1998.

[23] IEEE Standards for Local and metropolitan area networksVirtual Bridged Local Area Networks – Amendment 3:Multiple Spanning Trees Amendment to IEEE Std 802.1QTM,1998 Edition, IEEE Std 802.1s-2002.

Page 28: Metropolitan Ethernet Network: A move from LAN to MAN

4894 M. Huynh, P. Mohapatra / Computer Networks 51 (2007) 4867–4894

[24] IETF VPLS (LDP). <http://www.ietf.org/internet-drafts/draft-ietf-l2vpn-vpls-ldp-03.txt>.

[25] IETF VPLS (BGP). <http://www.ietf.org/internet-drafts/draft-ietf-l2vpn-vpls-bgp-02.txt>.

[26] IETF Ethernet encapsulation (PWE3). <http://www.ietf.org/internet-drafts/draft-ietf–pwe3-ethernet-encap-05. txt>.

[27] MEF, Metro Ethernet Networks – a technical overview.<http://www.metroethernetforum.org>.

[28] MEF, Metro Ethernet Services. <http://www.metroether-netforum.org>.

[29] MEF, CapEx/OpEx of Traditional TDM or otherL2 Services. <http://www.metroethernetforum.org/Portal/S_OpExCapEx.asp>.

[30] MEF, MEF 5: Traffic management specification: phase 1.<http://www.metroethernetforum.org/TechSpec.htm>.

[31] MEF, Clarian health deploys a metro area E-LAN solutionto support patient care. <http://www.metroethernetfo-rum.org/MEF_SuccessStories.htm>.

[32] MEF, IDAHO FALLS POWER – century old utilityprovider moves in to the 21st century. <http://www.metro-ethernetforum.org/MEF_SuccessStories.htm>.

[33] MEF, Korea-based PowerComm, a wholesale provider ofmetro Ethernet services. <http://www.metroethernetfo-rum.org/MEF_SuccessStories.htm>.

[34] MEF, Spains Al-Pi: delivering broadband metro Ethernetthru Giganet. <http://www.metroethernetforum.org/MEF_SuccessStories.htm>.

[35] MEF, Optical Ethernet and the City of Roanoke Virginia.<http://www.metroethernetforum.org/MEF_SuccessStories.htm>.

[36] Nortel Networks, Service Delivery Technologies for MetroEthernet Networks, Nortel Networks Whitepaper, Septem-ber 19, 2003. <http://www.nortel.com/solutions/optical/col-lateral/nn-105600-0919-03.pdf>.

[37] Riverstone Networks, An overview of virtual private LANservice: a new approach to LAN to LAN communication,Riverstone Networks Technology whitepaper #200. <http://www.riverstonenet.com>.

[38] Riverstone Networks, Scalability of Ethernet Services Net-works. <http://www.riverstonenet.com/solutions/ethernet_scalability.shtml>.

[39] VPLS.ORG, Virtual Private LAN Services (VPLS) Tech-nical Overview. <http://vpls.org/vpls_technical_overview.shtml>.

[40] VPLS.ORG, VPLS Standards. <http://vpls.org>.[41] Foundry Networks, SuperSpanTM a break-through for layer 2

Ethernet networks, foundry networks, Whitepaper, Novem-ber 2001. <http://www.foundrynet.com>.

[42] P. Knight, C. Lewis, Layer 2 and 3 virtual private networks:taxonomy, technology, and standardization efforts, IEEECommunications Magazine (2004).

[43] Metro Ethernet Webinar, Business benefits and key applica-tions for the enterprise, November 6th, 2003.

[44] Metro Ethernet Forum, Ethernet in the metro: status;services; and the role of metro Ethernet forum, Presentation.

[45] C. Rigney, A. Rubens, W. Simpson, S. Willens, Remoteauthentication dial in user service (RADIUS), IETFRFC2138 April 1997. <http://www.ietf.org/rfc/rfc2138.txt>.

[46] Extreme Networks Ethernet Automatic Protection SwitchingEvaluation Report. Technical Report Reference: 80056 Issue

1.1 (30/7/03). <http://www.extremenetworks.com/technol-ogy/competitive/Default.asp>.

[47] J.L.R. Ford, D.R. Fulkerson, Flows in Network, PrincetonUniversity Press, 1962.

[48] J. Edmonds, R.M. Karp, Theoretical improvements inalgorithmic efficiency for network flow problems, Journalof ACM 19 (2) (1990).

[49] F. Brockners, N. Finn, S. Phillips, Metro Ethernet –deploying the extended campus using ethernet technology,in: Proceeding of the 28th IEEE International Conference onLocal Computer Networks 2003.

[50] MEF, The Metro Ethernet Network: comparison to LegacySONET/SDH MANs for Metro Data Service Providers,Metro Ethernet Forum Whitepaper July 2003.

[51] S. Halabi, Metro Ethernet, Cisco Press, Indianapolos, IN,2003.

[52] MEF, The Technical Specifications of the Metro EthernetForum. <http://metroethernetforum.org/techspec.htm>.

[53] MEF, Carrier Ethernet the technology of choice for accessnetworks, March 2006.

Minh Huynh received his B.S. degree incomputer science from the University ofCalifornia, Davis in 2002.

His research interest is on MetroEthernet Network. The focus of hisresearch is on resilience, network loadbalancing, QoS, and security.

He interned at Siemens Technology-to-Business on Metro Ethernet Networks.He was on the student volunteer commit-tee for Globecom 2006 in San Francisco.

Prasant Mohapatra received his Ph.D. in

computer engineering from the Pennsyl-vania State University in 1993.

He is currently a Professor in theDepartment of Computer Science at theUniversity of California, Davis. He hasheld Visiting Scientist positions at IntelCorporation, Panasonic Technologies,Institute of Infocomm Research (I2R),Singapore, and National ICT Australia(NICTA). His research interests are in

the areas of wireless networks, sensor networks, Internet protocolsand QoS. His research has been funded though grants from the

National Science Foundation, Intel Corporation, Siemens, Pan-asonic Technologies, Hewlett Packard, and EMC Corporation.

He was/is on the editorial board of the IEEE Transactions oncomputers, IEEE Transaction on Parallel and Distributed Sys-tems, ACM WINET, and Ad Hoc Networks. He has been on theprogram/organizational committees of several international con-ferences. He was the Program Vice-Chair of INFOCOM 2004,and the Program Co-Chair of the First IEEE InternationalConference on Sensor and Ad Hoc Communications and Net-works (SECON 2004). He has been a Guest Editor for IEEENetwork, IEEE Transactions on Mobile Computing, and theIEEE Computer.