Network Moniting

IEEE Communications Magazine July 201344 0163-6804/13/$25.00 2013 IEEE

INTRODUCTIONCarrier networks rely on pricy, tightly integrated,and monolithic machinery which is neither easyto configure optimally nor troubleshoot. Carrierequipment follows widely accepted network stan-dards, such as those introduced by the ThirdGeneration Partnership Project (3GPP). Howev-er, typical implementations rely on vendor-spe-cific hardware platforms. Operators often endup in vendor lock-ins as entire areas must becovered with equipment from the same vendorfor maximum efficiency. These problems aremanifested in the unlicensed spectrum wirelessindustry too. For example, the control and provi-sioning of wireless access points (CAPWAP)standard (RFC 5415) enables a single accesscontroller to manage numerous WiFi accesspoints, possibly procured from a variety of ven-dors. In practice, CAPWAP multivendor inter-operable deployments are uncommon. Morecritically, the closed nature of modern networkequipment prevents the research communityfrom exploring new paradigms using real-worldequipment. For mobile operators, although new

revenue opportunities arise, such as machine-type communication (MTC), standardizationtakes years from concept definition to commer-cial equipment that can realize the servicebecoming widely available. In short, the carriernetwork paradigm we have come to trust hasreached its limits.

On these grounds, software-defined network-ing (SDN) emerged, aiming at a shift toward aflow-centric model that employs inexpensivehardware [1], a logically centralized networkcontroller, and assorted applications that utilizecontroller-exposed information to orchestrateservice delivery in the network [2]. SDN takescues from the way we run networks today: mod-ern networks, be they campus, data center,enterprise, or carrier networks, follow domain-centered deployment and operation. In thisparadigm, each domain could develop its ownnetwork services, addressing specific user needs.Presently, this can only be done on an end-to-end basis (i.e., on top of IP), as only vendors canmodify the highly sophisticated but primarilyhardware-based network elements. Instead, SDNadvocates the use of simple but software-pro-grammable network switches. This model cantake advantage of modern agile programmingmethodologies, which enable network softwareto be developed, enhanced, and upgraded atmuch shorter cycles than what we experiencetoday with state-of-the-art hardware-centric net-work machinery.

The first generation of SDN development isclosely associated with OpenFlow [2], but thelasting contribution of this early phase work isushering in radical new thinking in the way wedesign and realize networks. SDN explicitly sep-arates the control and data planes in a mannermore daring than other carrier-grade architec-tures, such as the 3GPP Evolved Packet Core(EPC). Moreover, SDN has triggered significantinterest in network function virtualization(NFV), as indicated by the latest developmentsin standardization organizations such as theInternet Engineering Task Force (IETF) andEuropean Telecommunications Standards Insti-tute (ETSI). In this context, we advance thestate of the art in SDN through three contribu-tions. First, we introduce the software-definedmobile network (SDMN) architecture, ablueprint for carrier-grade flow-based forward-ing and a rich environment for innovation at the

ABSTRACTMobile carrier networks follow an architec-

ture where network elements and their inter-faces are defined in detail throughstandardization, but provide limited ways todevelop new network features once deployed. Inrecent years we have witnessed rapid growth inover-the-top mobile applications and a 10-foldincrease in subscriber traffic while ground-break-ing network innovation took a back seat. Weargue that carrier networks can benefit fromadvances in computer science and pertinenttechnology trends by incorporating a new way ofthinking in their current toolbox. This articleintroduces a blueprint for implementing currentas well as future network architectures based ona software-defined networking approach. Ourarchitecture enables operators to capitalize on aflow-based forwarding model and fosters a richenvironment for innovation inside the mobilenetwork. In this article, we validate this conceptin our wireless network research laboratory,demonstrate the programmability and flexibilityof the architecture, and provide implementationand experimentation details.

FUTURE CARRIER NETWORKS

Kostas Pentikousis, Yan Wang, and Weihua Hu, Huawei Technologies

MobileFlow: Toward Software-DefinedMobile Networks

PENTIKOUSIS LAYOUT_Layout 1 6/26/13 11:50 AM Page 44

IEEE Communications Magazine July 2013 45

core of the mobile network. The first publicationof concept validation laboratory results is oursecond contribution. The hindsight and imple-mentation details provided should be of greatinterest to the research community at large.Finally, backed by experimental testing, weexplore the programmability, flexibility, andopenness of the proposed architecture, detailinghow mobile networks can be developed anddeployed in the future.

The remainder of this article is organized asfollows. After reviewing pertinent related work,we provide a brief background on EPC. We thenpresent SDMN, detailing the core components,and explain the different models supported.Finally, we illustrate our research laboratoryexperiments and conclude the article.

RELATED WORKSince the introduction of OpenFlow [2], promi-nent SDN work has addressed several topicssuch as network virtualization [1, 3], data centerand cloud networking [4, 5], acceleration invalue-added network service development [6],and network management and control platforms[7], taking an implementation- and experimenta-tion-oriented approach from the beginning.Wireless networking has also been considered,although salient work in this area, to date, hasfocused primarily on campus-like wirelessdeployments. The feasibility of mobility manage-ment with vertical handovers between IEEE802.11 and IEEE 802.16 networks was exploredin [8]. Although this work is indicative of futuredevelopments in wireless networking, it does notaddress the challenges mobile operators facewhen extending a model originally designed forwired networks to the wireless domain. Specifi-cally, carrier-grade software-defined mobile net-works have not received the attention theydeserve given their ubiquity and importance inconnecting the majority of Internet users aroundthe world.

Mobile operators are interested in maturelimited-risk technologies that are highly opti-mized to make the most out of scarce (licensed)wireless resources, scalable to handle billions ofusers as todays networks can, and ready to fos-ter innovation inside the network. A recent posi-tion paper on software-defined cellular networks[9] makes the case that SDN can simplify cellu-lar networks and lower management costs. Weconcur with Li et al. [9] that as one introducesSDN in a mobile network, the challenges lyingahead are significant. Unfortunately, the authorsdo not provide an architectural blueprint detail-ing their proposal; nor do they report experi-mental results. On the other hand, Kempf et al.[10] present a detailed study on the evolution ofEPC toward a model where the control planecan be lifted up from the core network ele-ments and henceforth reside in a data center.The authors review the current EPC architectureand document their implementation, which isbased on OpenFlow 1.2 protocol extensions, at asufficient level of detail so that it can be replicat-ed by others in the SDN community. The article,which we view as complementary to our work,emphasizes the role that data centers can play in

future carrier networks and lists possible archi-tectural modifications to EPC, but is very terseon prototype details. Indeed, the authors experi-ence with using the prototype appears as a workin progress, and the article does not provideexperimentation results. The work presented inthis article can coincide with that described in[10], but takes a different approach that is not asclosely knit to the use of data centers whiledeploying the proposed architecture. NFV iscurrently expected to be a driving force in futurecarrier network standardization, and there aremany different paths one can follow in both thesystem design phase and actual deployment. Weargue in this article that SDMN is demonstrablyflexible enough to accommodate numerous waysforward.

This article advances the state of the art inthe public peer-reviewed SDN literature bydescribing a concrete architectural proposal thatapplies the core SDN principles to mobile carri-er networks. We detail the development and useof a proof-of-concept prototype to validate thekey components of the architecture, followingthe SDN tradition of implementation-orientedexperimentation.

MOBILE BROADBAND TODAYEPC is a special-purpose, flat, all-IP architecturestandardized by 3GPP [1113]. EPC, whichalong with the evolved Universal MobileTelecommunications System (UMTS)terrestrial radio access network (E-UTRAN)forms the foundation of the Evolved Packet Sys-tem (EPS) and fourth generation (4G) networks,is currently deployed by dozens of operatorsaround the world. EPC made significant stepsforward in terms of embracing packet-switchednetworking, aiming at reduced complexity andincreased scalability through data and controlplane separation. EPC acts as a unifying routingfabric in the core network used by different3GPP-standardized radio technologies as well asnon-3GPP access technologies such as IEEE802.11, among others.

EPC is a highly optimized mobile serviceoverlay that capitalizes on IP- or Ethernet-basedtransport to deliver a diverse range of servicesthrough secure communications with quality ofservice (QoS) guarantees and seamless mobilitysupport [1113]. Figure 1 illustrates the key enti-ties, functional elements, and interfaces involvedin typical EPS operation during a voice call,video streaming, or online access to web content.We do not delve into the details of EPS opera-tion due to space constraints (interested readersshould consider [1113, references therein]), butnote that traffic from/to the user equipment(UE) is effectively directed through a set of gen-eral packet radio service (GPRS) Tunneling Pro-tocol (GTP)/Proxy Mobile IP (PMIP)tunnels from the attached eNB to the servinggateway (SGW) to the packet data network(PDN) gateway (PGW) and onward to the cor-responding PDN; see [11] for more details. Asillustrated in Fig. 1, user packets are encapsulat-ed/decapsulated and processed successively bydifferent functional elements associated withoperational and management functions such as

EPC is a

special-purpose, flat,

all-IP architecture

standardized by

3GPP. EPC, which

along with E-UTRAN,

forms the foundation

of the EPS and 4G

networks, is currently

deployed by dozens

of operators around

the world.


IEEE Communications Magazine July 201346

charging, policing, authentication, authorization,and accounting (AAA) operations, mobilitymanagement, and so on. Each box in thisarchitecture typically stands for a physical net-work device, often powered by specialized soft-ware and hardware, adhering to carrier-gradeoperation requirements. This orientation towardspecialized hardware stems, in part, from theevolution of telecommunication networks [14,references therein].

This mobile network architecture has been ingeneral successful as manifested by the numberof mobile subscribers and the multibillion dollarindustry around it. However, as new technolo-gies such as content distribution and cloud com-puting enter the mobile operator domain, thecomplex nature of mobile broadband technolo-gies starts to become an impediment to sustain-able future growth. EPC is a fit-for-purposeclosed system based on standardized interfaces,where every component performs specific func-tions, and each of the dozens of interfaces has aunique definition. Further evolution of themobile broadband architecture along this line ofthinking has certain drawbacks. First, operatorswill have to deal with higher capital and opera-tional expenditures (CAPEX/OPEX) at a timewhen average revenue per user (ARPU) isdecreasing. Progress down this evolutionarypath, based on this so-called stovepipe designmodel, is bound to incur an increasing rate ofcosts in the future. Meanwhile, operations staffhave to maintain many types of network ele-ments and follow the evolution of dozens ofinterfaces. Second, as higher CAPEX/OPEXforces some operators to refrain from investingfurther, those who do invest face long time-to-market periods as it is taxing to add new fea-tures. Interested operators must push a wholeindustry to standardize these new features, andthen wait for vendors to actually implementthem. This process does assure quality standardsdevelopment, but prevents operators who arewilling to endeavor first into new domains fromdoing so in a fast pace. Finally, a direct conse-quence of the specialization of each network ele-ment, which is defined solely for EPC, is limitedopenness and flexibility. Once purchased, EPCelements can be controlled through standardinterfaces, but cannot be extended through theuse of open interfaces or application program-

ming interfaces (APIs). If the operator expectsthat new kinds of network services would appealto its subscribers, existing equipment may be oflittle use in developing and deploying said ser-vices.

SOFTWARE-DEFINED MOBILENETWORK

In order to address the challenges introduced inthe previous sections, and motivated by the coreprinciples of the SDN paradigm, we set out todefine what we refer to as the software-definedmobile network (SDMN). A top-level require-ment for SDMN is to provide maximum flexibili-ty, openness, and programmability to futurecarriers without mandating any changes in UE.In this way, operators can innovate inside theirdomain without having to depend on either over-the-top (OTT) service providers or UE vendorsto support their innovations. More important,SDMN aims at making the forwarding substratefully software-driven, enabling the creation of anynetwork type on demand (as seen later in thisarticle), and opens the mobile network to innova-tion through new service enablers.

Figure 2 illustrates the central elements ofSDMN, contrasting it visually with todays EPC(Fig. 1). The key enablers in our architectureconsist of the MobileFlow forwarding engine(MFFE) and MobileFlow controller (MFC).MFFEs are interconnected by an underlyingIP/Ethernet transport network. Recall from Fig.1 that in EPC, the SGW and PGW are user planeelements, while the mobility management entity(MME) is a control plane element. Figure 2 illus-trates a new split that decouples mobile networkcontrol from all user plane elements. This waythe (new) user plane (i.e., the MFFE) becomessimple, stable, and high-performing, while thecontrol plane (i.e., the MFC and mobile networkapplications) can be implemented in a logicallycentralized manner. Forwarding in the MFFEcan be fully defined in software, while the controlsoftware can flexibly steer user traffic to differentservice enablers (e.g., deep packet inspection[DPI], video caching, and optimization) that canbe distributed throughout the mobile network, asshown in Fig. 2. This design, which follows in thetradition first explored in [6], permits us to facili-

Figure 1. Evolved packet system.

MME

eNB SGW PGW

IP/Ethernet-based transport network

Gx

GiS5/S8S1-U

S1-MME S6aMobilenetwork

LTE-Uu

HSS PCRF

UE

PDN/Internet

Switch/router

Switch/router

Switch/router

Switch/router

Control partForwarding part

User IP trafficRadio bearerGTP/PMIPtunnel

ASPserver

A top-level require-

ment for SDMN is to

provide maximum

flexibility, openness,

and programmability

to future carriers

without mandating

any changes in UE.

In this way,

operators can

innovate inside their

domain without

having to depend on

either over-the-top

(OTT) service pro-

viders or UE vendors

to support their

innovations.



tate swift network innovation through networkfunction virtualization. For example, in this case,the carrier may opt to either reuse existing DPIequipment or employ virtualized functionality ina data center.

Note also that the figure illustrates two UEdevices. The traffic from one of them is handledby a set of MFFEs, while the traffic from the sec-ond one is offloaded to the Internet primarilyusing the underlying OpenFlow transport network.Moreover, the depiction of virtualized networkfunctions, such as DPI, and their interconnectionwith specific MMFEs is only indicative.

Effectively, we introduce a new stratum basedon a logically centralized MobileFlow controllerthat can steer forwarding. MFFEs employ net-work virtualization, are suitable for multitenantmobile networks, and can support advanced andsecurity-related network functions. MFFEs aremore complex than an OpenFlow switch, butmuch simpler than a router or a PGW, as themajority of control plane functionality is fac-tored out to the MFC and the associated Mobile-Flow applications. In other words, significantparts of the functionality of todays EPC can becentralized into a carrier-grade control plane(upper part of Fig. 2). Figure 2 also depicts theOpenFlow-based decoupling of the IP/Ethernettransport network, contrasting it with the afore-mentioned mobile network control- and forward-ing-plane decoupling. As can be surmised fromFig. 2, OpenFlow alone is not sufficient forimplementing a carrier-grade mobile networkuser plane. We distinguish an SDMN from anOpenFlow-based network, as MFFEs are not

switch-level equipment. MFFEs must supportcarrier-grade functionality such as network layer(L3) tunneling and flexible charging.

Our MobileFlow architecture can easily inter-act with legacy EPC network elements. In thecontrol plane, the mobile network applicationsrunning on top of an MFC can function as anMME or the gateway control part of a physicalbox (indicated as GW-C), and thus can interop-erate with other legacy MMEs, SGWs, or PGWsusing the well established 3GPP-defined inter-faces, as shown in Fig. 2. In the user plane, theMFFE can be interconnected with legacy EPCelements with a common IP/Ethernet transportnetwork. The control messages can be transmit-ted via the MFFE in an in-band manner or via adedicated out-of-band network.

Next, we look into the details of each of thearchitectural components of Fig. 2 and provideimplementation insights based on our experiencewith an SDMN in our research laboratory.

MOBILEFLOW FORWARDING ENGINEMFFEs include standard mobile network tunnelprocessing capabilities (e.g., GTP-U and GREencapsulation/decapsulation facilities) [11]. Con-sequently, MFFEs, with the support of an MFC,can be integrated with legacy EPC equipment.An MFFE may also include the key functionalityof a wireless access node, such as a radio inter-face to manage radio bearers. As illustrated inFig. 2, an SDMN does not mandate any changesin UE. From a deployment perspective, onecould envision wireless access MFFEs in paralleloperation with existing eNodeBs, for example, as

Figure 2. Software defined mobile network.

MF ctrl if

Decoupling ofcontrol and

forwarding inIP/Eth network

PDN/Internet

MF: MobileFlowOF: OpenFlow

S1-MME

SmfSmf

Smf

S5/S8

S11

S10Decoupling of controlfrom user plane

Legacy MMEMobile NW app

(MME, GW-C, etc)

MobileFlowcontroller

Control plane

LegacyeNodeB

Legacy SGW

Legacy PGW

Switch/router

OpenFlow network

OF ctrl ifOF fwdengine

Internet



OF controller

Control planeNetwork app

(routing, MPLS,VPN, etc.)

App serviceproviderserver

UE

UE

MF ctrl interface MF ctrl ifRadio

if DPIVideo

caching andoptimization

MF fwdengine

MF fwdengine

MF fwdengine

MFFEs include

standard mobile

network tunnel

processing

capabilities, such as,

for instance,

GTP-U and GRE

encapsulation/

decapsulation facili-

ties. Consequently,

MFFEs, with the sup-

port of a MobileFlow

controller, can be

integrated with lega-

cy EPC equipment.



well as the incremental use of MFFEs in thecore network as SGW/PGWs are retired.

Each MFFE communicates with an MFCthrough a lightweight protocol that implementsthe MobileFlow control interface (Smf). Due tospace limitations we cannot delve into the detailsof this protocol, but suffice it to say that theMFFEs use the flow rules received from theMFC to encapsulate/decapsulate user trafficand/or forward packets to the transport network.Our current protocol implementation borrowsfrom the flow-entry model advocated by theOpen Networking Forum (ONF). Although weare still at a proof-of-concept stage, and morework is foreseen in the coming months, the per-formance we are observing in the laboratory isvery promising, since the burdensome gatewaycontrol part is factored out of the user plane.Eventually, each MFFE featuring the lightweightSmf-driven control layer will be a high-perfor-mance, highly reliable network element that canhandle a much larger number of flow entriesthan commodity OpenFlow switches do. This isbecause a future carrier mobile network shouldbe able to handle very fine-grained policies, atthe very minimum on par with what is now avail-able for commercial PGWs. The comparativeadvantage is that with MFFEs, one can innovatemuch faster and, if so desired, can aim for dif-ferent optimalities than what is possible today.

From an implementation perspective, MFFEcan be based on a stripped-down mobile gatewaythat can handle network-layer (layer 3, L3) tun-nel processing in the data plane with all othertunnel control processes moved up to the MFC.Alternatively, one can extend an OpenFlowswitch to support the MobileFlow functionsmentioned above. A hybrid model can be usedas a combined forwarding element in the sameIP/switch network level to facilitate 3GPP-IP

convergence. For example, in Fig. 2, the MFFEcan be combined with an OpenFlow switch,while the MFC can be combined with the Open-Flow controller.

MOBILEFLOW CONTROLLERFigure 3 zooms in on the MFC and illustrates itsmain functional blocks: the mobile networkfunction, the mobile network abstraction, andthe functional blocks corresponding to threeinterfaces. The MFC southbound interface con-trols MFFE operation. The horizontal interfaceis used for communication with other MFCs,and can be employed for intra- and trusted inter-domain cooperation. Operators with very largeinfrastructure typically segment their networkaccording to administrative, regulatory, techni-cal, or other reasons. In contrast to previouswork [8, 9], our architecture specifies this inter-face from the beginning as we aim at very largecarrier-grade deployments with high demands onreliability and availability that can scale out.Finally, the northbound interface facilitates fastnetwork service and application development, aswe discuss below.

The MFC relies on network-level abstraction.The corresponding functional block in Fig. 3includes topology auto-discovery, topologicalresource view, network resource monitoring, andnetwork resource virtualization. The networkfunctions block includes tunnel processing,mobility anchoring, routing, and charging, amongothers. For instance, support for creating andestablishing GTP-U tunnels is one of the net-work functions implemented in our testbed pro-totype, described later in this article. Theseabstractions correspond to high-level descrip-tions and are agnostic to the particular MFFEimplementation. Thus, an operator can freelyuse MFFEs from different vendors. In addition,the operator can use MFFEs to adopt anddeploy novel mobile network architectures thatdo not rely on tunneling.

A cornerstone of the SDN paradigm is thepossibility of extending current and creating newfunctionality through programmatic APIs. AnSDMN is compatible with advances in the Open-Flow-based ecosystem. Moreover, Fig. 3 illus-trates the open interfaces and associated APIsintroduced by the SDMN. Note that the SDMNcontroller does not have an explicit interface tolegacy infrastructure. Any legacy interfacing canbe handled efficiently by MobileFlow applica-tions. The SDMN northbound interface, forexample, can be used by MobileFlow applica-tions to implement different network functionali-ty. The MobileFlow stratum can be engaged inmanaging user traffic selectively depending onseveral criteria.

Returning to Fig. 2, recall that some trafficcan be handled directly by the underlying Open-Flow-based transport network, while other trafficmay require special treatment or support fromadvanced services. For example, mobility man-agement, DPI, video optimization, and fine-grained QoS support can be selectively activatedand handled at the MobileFlow stratum for pre-mium traffic, while offloaded traffic can be han-dled by the transport network. We detail in theImplementation section later in this article

Figure 3. MobileFlow controller.

Mobile network applications(EPC, FMC, CDN, etc.)

Northbound interface

Mobile networkfunctions

OtherMobileFlowcontrollers


Mobile networkabstraction

Radioi/f MFFE MFFE MFFE

MobileFlowprotocol

Horizontal interface

Southbound interface



how this is employed to realize different cellulararchitectures.

MOBILE NETWORK APPLICATIONSMobile network applications can be developedby taking advantage of the SDMN northboundinterface. Figure 4 illustrates the two models weexplored while developing a fully EPS-compliantnetwork based solely on an SDMN. The firstmodel, illustrated in Fig. 4a, adopts a networkfunction virtualization approach: a MobileFlowapplication implements the control part func-tionality of each of the EPS elements (e.g.,eNodeB-C, SGW-C, PGW-C, S/PGW-C, andGGSN-C). Then each of these applications,which can run in a cloud computing environmentsimilar to what is proposed in [10], interacts withthe MFC to manage a respective MFFE, effec-tively implementing a virtual gateway. Thismodel features a 1:1 mapping between the virtu-al gateway and MFFE via the MobileFlow con-troller. In other words, an EPC virtual topologyis constructed in the MobileFlow stratum, asshown in Fig. 4a, according to the relationshipbetween the corresponding reference architec-ture (Fig. 1) network elements (eNB-C, SGW-C,PGW-C, MME, PCRF, and HSS). Value-addedfunctionality (e.g. video optimization) can beintroduced by implementing the correspondingapplication and mapping it to an MFFE. Thismodel is akin to the one introduced in [3], andalthough control may be centralized, applicationinteraction is logically distributed.

The second model, shown in Fig. 4b, adopts a1:m mapping between one all-encompassingMobileFlow application and several MFFEs.The application uses the SDMN northboundinterface to access the substrate resource topolo-gy from the MobileFlow controller. In addition,it dictates the traffic paths and operations withinthe MFFE topology based on the networkabstraction obtained from the MFC (Fig. 3).This model does not have to follow currentlystandardized interfaces and can therefore pro-vide fertile ground for developing novel mobilenetwork systems. For example, we can develop avirtual EPC gateway which aggregates function-ality by factoring in that is currently distributedbetween several network elements, thus optimiz-

ing control signaling, while incorporating variousSDMN-based service enablers, as shown in Fig.4b.

Both models support m:1 mapping betweenMobileFlow applications and MFFEs, as eachapplication belongs to a different control plane,therefore enabling multitenancy in a straightfor-ward manner. Note that the first model is partic-ularly suitable for integrating legacyinfrastructure transparently into an SDMN sys-tem. The operator is free to employ the formermodel for one part of the network and the latterone for another part in which it can experimentwith and readily deploy new network services. Infact, through our support for multitenancy weexpect that it is possible to have both models inthe same part of the network, although this isleft as future work with respect to prototypeimplementation. Regardless of which model ispreferred, SDMN enables flexible on-demandmapping between MobileFlow applications andtheir MFFE counterparts, as we explain in thefollowing section.

MOBILITY MANAGEMENTSupporting mobility in an SDMN, whether basedon existing standards by 3GPP and IETF ornewly proposed by the research community, canbe realized by introducing the correspondingapplications above the northbound interface ofthe MFC. For instance, as shown in Fig. 4a,where we take EPS as an illustrative case, wecan use SDMN to implement the current gener-ation of control functionality in a 3GPP mobilenetwork. The wireless access MFFE with itseNB-C acts as an eNodeB. The MFFE with itsassociated SGW-C acts as the SGW, and theMFFE with its associated PGW-C acts as thePGW. In this case, mobility is managed by thevirtualized but standard-adhering MME. Theinterfaces between MME and eNB-C/SGW-Care as defined by 3GPP (i.e., S1-MME and S11).The interface between SGW-C and PGW-C alsofollows the 3GPP-defined GTP-C part of S5/S8.The corresponding MFFEs handling the usertraffic are controlled by their -C control coun-terparts with respect to processing GTP-U pack-ets. The main difference from currentdeployments is that the -C virtualized function

Figure 4. MobileFlow application models.

InternetIP transport

Control plane

Novel mobilenetwork application

HSS

MME

SGW-C

PCRF

Virtual EPCtopology

PGW-CeNB-CMobileFlow controller

MobileFlow controller

MFFE

RadioI/F MFFEUE

MFFE

InternetIP transport

(b)(a)

MFFEDFI

MFFEVideo opt.

UE

MFFEMFFE

RadioI/F MFFE

Supporting mobility

in SDMN, whether

based on existing

standards by 3GPP

and IETF or newly

proposed by the

research community,

can be realized by

introducing the cor-

responding applica-

tions above the

northbound interface

of the MFC.



translates the GTP signaling/context into flowrules and sends the rules via the MFC to each ofthe MFFEs involved in handling the particularflow. In other words, via the Smf interface wecan control packet forwarding behavior, such asGTP tunnel encapsulation/decapsulation, qualityof service (QoS) metering, forwarding, and so on,without having this type of control plane func-tionality in the boxes deployed in the field. Themost challenging aspect here is how to split thefunctionality of an eNB, which is more complexthan splitting the control and data planes in anS/P-GW. The interface between eNB-U andeNB-C consists of the forwarding control andwireless bearer control. The wireless bearer con-trol is relevant to the radio handoffs required inmobility management processes, and at this stageof development this design aspect is part of ourongoing work. With respect to the prototypeimplementation we describe in the following sec-tion, the reader should keep in mind that thecurrent code addresses only the split of S/PGW.

Looking forward, one can experiment on asmall scale and then widely deploy novel mobili-ty management solutions. In an SDMN this isdone by upgrading or changing altogether therespective MobileFlow applications withoutreplacing the deployed MFFEs. For example,Fig. 4b indicates that we can deploy a flat mobil-ity management scheme which does not dependon S/PGW-like anchor nodes. This opens upopportunities for introducing simpler handoverand gateway change processes, which todayrequire lengthy standardization efforts. Earlierwork on OpenPipes [6] and OpenRoads [8]explored a range of possibilities for optimizedtraffic steering through different service enablerssuch as video optimization, we discussed earlierin this article and illustrated explicitly by the dif-ferent paths for the red and blue flows in Fig.4b. Adopting a design based on locator/identifiersplit, including LISP, HIP, and others discussedin [15], can easily be accommodated. An SDMNaims to provide the same level of flexibility inhandling mobility of users, flows, and content,but with carrier-grade performance. We are cur-rently actively following the developments in theIETF Distributed Mobility Management (DMM)working group, and we plan to showcase in thefuture how SDMN can implement the novelsolutions standardized there.

IMPLEMENTATIONWe validated the SDMN architecture throughresearch prototyping using COTS x86 based gen-eral-purpose servers and OpenFlow-based com-ponents. To showcase the flexibility of ourarchitecture we developed the On-DemandMobile Network (ODMN) prototype, whichdemonstrates the core benefits of our Mobile-Flow approach. In particular, we show that theMobileFlow stratum enables the software-baseddefinition of different mobile networks (3G, 4G,FMC, and others) with different specifications(radio coverage, subscription numbers, PDP/bearer numbers, maximum bandwidth, etc.)according to administrator demand.

The ODMN is effectively a virtual networkmanagement system in which network resources

include control and forwarding plane elements.We implement control plane network functionsas mobile network applications (e.g., SGW-C,PGW-C, MME, and PCRF) and run them ongeneral-purpose servers. Forwarding planeresources consist of radio access nodes andMFFEs. MFFEs may be implemented solely insoftware or using specific hardware platforms.By supporting virtualization for both control-and forwarding-plane resources, it is easy to cre-ate multiple virtual mobile networks with differ-ent types and specifications over the samehardware environment, as illustrated in Fig. 5.The same MFFE can be reused by differenttypes of virtual networks. Each virtual mobilenetwork can evolve and be upgraded solely byreplacing the virtual machine running the corre-sponding MobileFlow application on the controlplane servers MFFEs remain unchanged.

Our testbed environment includes five serversin the control plane, as illustrated in the bottomleft part of Fig. 6. Servers marked C1C4 runmultiple KVM virtual machines with differentMFCs and applications (MME, SGW-C, PGW-C, GGSN-C, SGSN, etc.). A fifth server acts asthe ODMN management server handling thecreation, modification, and termination of virtualmobile networks. For example, Fig. 6 illustratesthat we created three different networks. In theforwarding plane there are three radio accessnodes (eNB12, NodeB) and three servers(F1F3). eNB1 is a real Huawei eNodeB, whileNodeB, radio network controller (RNC), andeNB2 are emulated using the Huawei NetworkTraffic Emulator (NTE). We developed eachMFFE by extending the Open vSwitch code(available from www.openvswitch.org) to supportGTP tunneling. All control plane and forwardingplane servers are interconnected by commercialoff-the-shelf (COTS) LAN switches.

We also developed the ODMN testbed oper-ator graphical user interface (Fig. 6, top left),which we refer to as the ODMN dashboard. Thisdashboard provides an overview of the physicalresource topology and, more important, is usedfor all virtual resource allocations describednext. The dashboard GUI also displays, overeach network element icon, the virtual resourceinstances. Finally, the right side of Fig. 6 por-trays different snapshots from the virtual net-works we instantiate for the demonstrationpresented in this article.

Without loss of generality, we use ODMN todeploy three different mobile networks concur-rently on the research testbed. The first network(marked in yellow in Fig. 6) is a standard 3Gnetwork composed of an NTE emulated RNC, avirtualized serving GPRS support node (SGSN),and a virtualized gateway GPRS support node(GGSN). As the color code indicates in thedashboard (Fig. 6, top left), the SGSN corre-sponds to an SGSN virtual machine (VM) onC1; the GGSN is composed of an MFFE VM onF3 and a corresponding GGSN-C VM on C4.The second virtual network (shown in red in Fig.6) is a 4G network including the real eNodeB(eNB1), a virtual MME (C3), and a virtualizedS/PGW. The virtual S/PGW comprises an MFFEVM on F1 and the corresponding S/PGW-C VMon C3. The third virtual network (in green) is

We validated the

SDMN architecture

through research

prototyping using

COTS x86 based

general-purpose

servers and Open-

Flow-based compo-

nents. To showcase

the flexibility of our

architecture we

developed the

ODMN prototype,

which demonstrates

the core benefits of

our MobileFlow

approach.



another 4G network including an NTE emulatedeNodeB (eNB2), a virtual MME (C1), a virtualSGW, and a virtual PGW. The virtual SGWcomprises an MFFE VM (F2) and the corre-sponding SGW-C VM on C2. The virtual PGWis composed of an MFFE VM on F3 and thecorresponding PGW-C VM on C2.

In ODMN, a COTS LTE Android smartphone can successfully attach to the virtual 4Gmobile network (red network) via a real-worldeNodeB (eNB1) through the standard 3GPPattachment procedure. Since the virtual S/PGW(corresponding to MFFE VM on F1) connectsthrough the laboratory LAN to the Internet, theuser of the LTE smartphone can surf the webthrough the ODMN-based virtual mobile net-work. This essential functional test verified thatthe basic 3GPP attachment and bearer establish-ment procedures are correctly implementedusing an SDMN. However, the prototype doesnot fully support charging and policing; enhanc-ing this aspect is part of our ongoing implemen-tation work.

For the virtual 3G network (yellow networkin Fig. 6) and virtual 4G network (green net-work), we verified their on-demand perfor-mance capability for the maximum bandwidthand number of PDP/bearer contexts. We specifythe maximum bandwidth to 100 and 200 Mb/sfor the virtual 3G and 4G networks, respectively.We specify the maximum number of PDP/bearercontexts to 200 for both virtual networks. We

employed NTE to emulate 400 PDP/bearersattached to the two networks with 400 Mb/smaximum bandwidth, respectively, and verifiedthat the ODMN can successfully match the max-imum bandwidth and number of PDP/bearers asper each test specification. As illustrated above,virtualization is explicitly supported, while multi-tenancy is taken to a new level as mobile net-works with different types and specifications canbe deployed on demand.

OPERATOR BENEFITSBy explicitly separating the transport stratumfrom the mobile network one, we introduce anew degree of freedom in our system, which isnot present in an OpenFlow-based network.First, the two strata can be developed anddeployed independently. This means that amobile operator can continue to benefit fromthe reliability of current EPC equipment whilerolling out a commodity OpenFlow-based trans-port network to reduce costs. Simultaneously,the operator can start to deploy MFFEs, whichcan interoperate with legacy equipment (e.g., viaGTP/PMIP tunnels), and experiment with newservices built on the software-defined part of themobile network. Without the MobileFlow stra-tum, an operator would effectively have to discardall current equipment, including specialized mid-dleboxes, and reintroduce the same functionalityover the transport network applications space.This latter approach has many risks due to the

Figure 5. On-demand mobile network.

VM


Software basedMFFE

Hardware basedMFFE

MFFE: MobileFlowforwarding engine



VM

Novel mobile network4G network3G network

Control planeservers

management

Forwardingplane

physicalresource

management

Control planeVM

management

Forwardingplane virtual

resourcemanagement

Virtual networkmanagementframework

??VM

VMVM

VM VMSGSN GGSN-C

VM

VMPGW-C

MMESGW-C

SGW-C?? ??

3G

4GWi-Fi

IP transport network Internet

VM VM VM

This essential func-

tional test verified

that the basic 3GPP

attachment and

bearer establishment

procedures are cor-

rectly implemented

using SDMN. How-

ever, the prototype

does not fully sup-

port charging and

policing; enhancing

this aspect is part of

our ongoing imple-

mentation work.



lack of carrier-grade mature OpenFlow products.Instead, our architecture enables operators tochoose the best of both worlds and migrate to afully software-defined network on their own ini-tiative and pace. Furthermore, the architecture isforward-looking, so the operator can also replacethe transport stratum without losing its invest-ment in the MobileFlow stratum. This is of partic-ular interest, for example, when considering fixedmobile convergence (FMC).

As our research prototype proof-of-conceptresults indicate, SDMN enables operators to useinfrastructure resources to orchestrate ondemand the creation of various mobile networkpipes based on different mobile architectures(3G, 4G, FMC, etc.) using the appropriateMobileFlow applications. While doing so, theoperator can manage traffic steering for value-added services, enabling new possibilities forimproving the end-user quality of experience aswell as optimizing the cost of handling lesser-value traffic flows. An example of the latterincludes offloading selected traffic to the closest

egress at the transport network level withouthaving to traverse the mobile core, as illustratedin Fig. 2. Since the MFC maintains a completeview of the infrastructure network topology,MobileFlow applications can be used to orchas-trate various service chains, thus delivering apersonalized user experience.

Last but certainly not least, SDMN enablesspeedy and smooth mobile network evolutionand differentiation. As the example in Fig. 5illustrates, an operator can start using SDMNwith a 3G network in place and evolve it into a4G network by allocating more resources as therollout proceeds. Changing important mecha-nisms at the core is also decoupled from evolu-tion at the edge. Today this would require atechnology swap, which is costly, time-consumingand may introduce competitive deployment racesthat can hurt operator profits. Through a phasedapproach that can follow each operators ownpace, future SDMN-based networks can bedeployed and operated. Moreover, within thesame domain, a carrier can allocate a dynamic

Figure 6. ODMN prototype testbed and dashboard user interface.

Select C1

ID CPU1 MME VN1

Type: Huawei RH2295CPU: 16 coreRAM: 40GBDisk: 2TB

55%83%40%

C1 C2 C3

Controlplane

Data plane

C4

F3F1

F2

Testbed Control serverC1 C2 C3 C4

MFFE 3

MFFE 2

MFFE 1

Switch 4

Switch 5

Switch 3Switch 1

Switch2

eNodeB 1

RNCNodeB

eNodeB 2

LTE Smartphone 1

LTE Smartphone 2

3GSmartphone

Emulated

ODMNdashboard

ODMNmanagement

server

Overall resourceview

Virtual networkview

eNB2

eNB1

NodeBRNC

GGSNRNCNodeB

SGSN

S/PGW

eNodeB1

MME

VN: EPC-3G-11

Virtual networkEPC-3G-11EPC-4G-22EPC-4G-33



VN: EPC-4G-22

VN: EPC-4G-33 MME

PGW

SGWeNodeB2

RAM Role VN

2 SGW-CVN23 PGW-CVN3

General

VM list



mix of control and user plane resources from thevirtualized network infrastructure to address dif-ferent pain points as they emerge (e.g., signalingstorms or spikes in content distribution duringpopular events) on much shorter timescales thantoday. Within this context, operators may pro-cure (or develop) different sets of MobileFlowapplications based on the subscriber groups theyaddress and thus further differentiate themselvesfrom their competition.

CONCLUSIONWe introduce SDMN, an architecture for futurecarrier networks that builds on the decoupling ofdata and control in the mobile network userplane and a new MobileFlow stratum, which cansignificantly increase the operator innovationpotential. The SDMN open interfaces and APIsfoster service innovation, increasing the capabilityof the operator to roll out new network featureswhile reducing time to market for new services.Our proof-of-concept prototype demonstratesthat by employing the SDMN architecture a carri-er can configure on-demand and in detail the net-work architecture, radio coverage, gatewaylocation, PDP/bearers number, maximumthroughput, and so on. Moreover, as we haveseen, an SDMN operator can orchestrate newservice chains and evolve its network by updatingor replacing MobileFlow applications.

Our ODMN testbed implementation vali-dates the programmability and flexibility of theSDMN approach. That said, many challenges lieahead as we focus on improving performance,ease of resource configuration and management,scalability, and code maturity. ODMN providesthe foundation for further research and experi-mentation in this area. We expect that SDMNscan catalyze operator innovations in a range ofareas, from speedy introduction of new servicesto improved monitoring and management of net-work resources and services, to controllingCAPEX/OPEX, to personalized handling of sub-scriber traffic to yield more revenue and reducechurn. With SDMN, operators can differentiatetheir offerings at an unprecedented scale, whileprotecting their current investment in traditionalcellular equipment.

ACKNOWLEDGMENTWe are grateful to the anonymous reviewers fortheir constructive comments and suggestions,which helped us to hone the core message ofthis article. We thank Dr. Jiang Li and HuaHuang for their comments and suggestions,which helped improve the overall quality of thiswork. The views expressed herein are solelythose of the authors and do not necessarily rep-resent the views of Huawei Technologies.

REFERENCES[1] A. Greenhalgh et al., Flow Processing and the Rise of

Commodity Network Hardware. SIGCOMM CCR, vol.39, no. 2, ACM, 2009, pp. 2026.

[2] N. McKeown et al., OpenFlow: Enabling Innovation inCampus Networks, SIGCOMM CCR, vol. 38, no. 2.ACM, 2008, pp. 6974.

[3] M. R. Nascimento et al., Virtual Routers as A Service:the RouteFlow Approach Leveraging Software-DefinedNetworks, Proc. CFI, Seoul, Korea. ACM, 2011.

[4] C. J. Sher Decusatis et al., Communication withinClouds: Open Standards and Proprietary Protocols forData Center Networking, IEEE Commun. Mag., vol. 50,no. 9, Sept. 2012.

[5] M. Bari et al., Data Center Network Virtualization: ASurvey, IEEE Commun. Surveys & Tutorials, vol. PP, no.99 (early access article).

[6] G. Gibb et al., OpenPipes: Prototyping high-speed net-working systems, Proc. SIGCOMM (Demo), Barcelona,Spain, 2009.

[7] T. Koponen et al., Onix: A Distributed Control Platformfor Large-Scale Production Networks, Proc. OSDI, Van-couver, BC, Canada, 2010.

[8] K.-K. Yap et al., Blueprint for Introducing Innovationinto Wireless Mobile Networks, Proc. SIGCOMM VISAWksp., New Delhi, India, 2010.

[9] E. Li et al.., Toward Software-Defined Cellular Net-works, Proc. EWSDN, Darmstadt, Germany, 2012.

[10] J. Kempf et al., Moving the Mobile Evolved PacketCore to the Cloud, Proc. WiMob, Barcelona, Spain,2012.

[11] I. Ali et al., Network-based Mobility Management inthe Evolved 3GPP Core Network, IEEE Commun. Mag.,vol. 47, no. 2, Feb. 2009.

[12] H. Ekstrm, QoS Control in the 3GPP Evolved PacketSystem, IEEE Commun. Mag., vol. 47, no. 2, Feb.2009.

[13] J.-J. Pastor Balbs et al., Policy and Charging Controlin the Evolved Packet System, IEEE Commun. Mag.,vol. 47, no. 2, Feb. 2009.

[14] K. Pentikousis et al., Design Considerations for Mobil-ity Management in Future Infrastructure Networks,Proc. ITU TELECOM WORLD Technical Symp., Geneva,Switzerland, 2011.

[15] B. Sousa et al., Multihoming Management for FutureNetworks, ACM/Springer Mobile Networks and Appli-cations, vol. 16, no. 4, Aug. 2011, pp. 50517.

BIOGRAPHIESKOSTAS PENTIKOUSIS ([email protected]) is a seniorresearch engineer at Huawei Technologies, Berlin, Ger-many. He earned his Bachelors degree in informatics fromAristotle University of Thessaloniki, and his Masters anddoctoral degrees in computer science from Stony BrookUniversity. At Huawei he worked on 3GPP EPC researchtopics beyond Rel. 12, carrier-grade SDN, and NFV.

YAN WANG ([email protected]) is a seniorresearch engineer at Huawei Technologies, Shanghai,China. He earned his doctoral degree in communicationand information systems from the State Key Laboratory onAdvanced Optical Communication Systems and Networks,Shanghai Jiao Tong University (SJTU), China. At Huawei heworked as the leader of the Software Defined Mobile Net-work (SDMN) group. His research interests include 3GPPpacket core research topics, carrier-grade SDN, and NFV.

WEIHUA HU ([email protected]) is a senior researchengineer at Huawei Technologies in Shanghai, China. Heearned his Bachelors and Masters degrees in control theo-ry and control project from SJTU. At Huawei he worked on3GPP packet core research topics, carrier-grade SDN, andNFV.

Our ODMN testbed

implementation vali-

dates the pro-

grammability and

flexibility of the

SDMN approach.

That said, many chal-

lenges lie ahead as

we focus on improv-

ing performance,

ease of resource

configuration and

management, scala-

bility, and code

maturity.


/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages false /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages false /GrayImageDownsampleType /Average /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages false /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages false /MonoImageDownsampleType /Average /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier (CGATS TR 001) /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /False

/CreateJDFFile false /Description > /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ > /FormElements false /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MarksOffset 6 /MarksWeight 0.250000 /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /UseName /PageMarksFile /RomanDefault /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /LeaveUntagged /UseDocumentBleed false >> > ]>> setdistillerparams> setpagedevice

Network Moniting

Documents

Transcript of Network Moniting