A Robust Optimization Based Energy-Aware Virtual Network...
Transcript of A Robust Optimization Based Energy-Aware Virtual Network...
Research ArticleA Robust Optimization Based Energy-Aware Virtual NetworkFunction Placement Proposal for Small Cell 5G Networks withMobile Edge Computing Capabilities
Bego Blanco Ianire Taboada Jose Oscar Fajardo and Fidel Liberal
Faculty of Engineering of Bilbao University of the Basque Country (UPVEHU) Alameda Urquijo sn 48013 Bilbao Spain
Correspondence should be addressed to Bego Blanco begonablancoehueus
Received 4 May 2017 Revised 8 August 2017 Accepted 28 August 2017 Published 12 October 2017
Academic Editor Wolfgang Kellerer
Copyright copy 2017 Bego Blanco et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
In the context of cloud-enabled 5G radio access networks with network function virtualization capabilities we focus on the virtualnetwork function placement problem for a multitenant cluster of small cells that provide mobile edge computing services Underan emerging distributed network architecture and hardware infrastructure we employ cloud-enabled small cells that integratemicroservers for virtualization execution equipped with additional hardware appliances We develop an energy-aware placementsolution using a robust optimization approach based on service demand uncertainty in order to minimize the power consumptionin the system constrained by network service latency requirements and infrastructure terms Then we discuss the results of theproposed placement mechanism in 5G scenarios that combine several service flavours and robust protection values Once theimpact of the service flavour and robust protection on the global power consumption of the system is analyzed numerical resultsindicate that our proposal succeeds in efficiently placing the virtual network functions that compose the network services in theavailable hardware infrastructure while fulfilling service constraints
1 Introduction
In this new era of an always-connected and hypercommuni-cated society 5G targets a set of ambitious requirements interms of network performance-related key performance indi-cators such as very low delays and high data rates [1ndash3] Forthis reason from an operatorrsquos perspective finding a tradeoffbetween user performance improvement and CAPEXOPEXreduction becomes essential
In this context Cloud-Enabled Radio Access Networks(CE-RANs) become a feasible solution to enhance userexperience and decrease total costs [4] CE-RAN architectureleverages network function virtualization (NFV) [5ndash7] to splitthe radio protocol stack between physical and virtualizedfunctions the latter running in edge clouds This edge cloudmay also be used to place service instances at the edge of themobile network and in the proximity of end users Thus thisinclusion of edge cloud service capabilities in the network notonly improves quality of service but also simplifies deploy-ment and management in multitenant scenarios [8]
Through the deployment of Virtualized Network Func-tions (VNFs) on Virtual Machines (VMs) operators willexecute edge radio and service VNFs inside data-centers oncommodity hardware with the advantage of reducing capitalexpenses In this novel paradigm general-purpose comput-ing in networks has been realized along with the virtualiza-tion of network functions which enables the automation ofnetwork service provisioning and management In this workwe focus on an emerging 5G scenario amultitenant cluster ofsmall cells (SCs) that provides edge radio and service func-tions by means of virtualization technologies In particularwe employ cloud-enabled SCs that integrate microservers forvirtualization execution equipped with IT resources (CPURAM and storage) and additional hardware appliances suchas GPUs DSPs or FPGAs (Figure 1) Thus in this whole pic-ture we could define a network service (NS) as a collection ofVNFs required to deploy a complete 5G mobile service (eginnovative video content delivery at the edge integrated withradio monitoring and management) for the end users of theSC network operator
HindawiMobile Information SystemsVolume 2017 Article ID 2603410 14 pageshttpsdoiorg10115520172603410
2 Mobile Information Systems
SC small cell NF network function
PNF physical NF VNF virtualized NF
Mobile core network
PNF PNF PNF
Mobile edgecomputing
SC SC SC
Dist
ribut
ed cl
oud-
enab
led
radi
o ac
cess
net
wor
k
VNF VNF VNF
Figure 1 Cloud-enabled RAN architecture based on small cells
Nevertheless the success of the presented novel approachundoubtedly depends on the VNF placement strategy usedto allocate VNFs into the available resources given a set ofservice constraints However defining efficient mechanismsto allocate the software-based components of NS onto thenetworked data-centers (set ofmicroservers that conform thecloud-enabled SC cluster in our case) is a challenging taskTherefore the relevance of designing intelligent VNF place-ment algorithms becomes fundamental while it is notdefined in standards [9]
Besides current trends towards green information andcommunication technologies target the improvement ofenergy consumption in all the nodes including the networknodes themselves In this paper we will particularly discusshow to derive an energy-aware VNF placement solution fora cluster of cloud-enabled SCs in a 5G deployment sce-nario With this purpose we will deal with the objective ofminimizing power consumption of both the microserversand SDN-enabled switches to deploy the overall virtualizedinfrastructure needed (including virtual switches within eachSC and physical ones interconnecting SCs)
In order to address the VNF placement problem underthe aforementioned novel 5G scenario this paper proposesa placement mechanism that minimizes power consumptionsubject to bit rate and latency constraints imposed by thenetwork services and is aware of the available underlyingresources Existing literature provides a wide variety of VNFplacement approaches ranging from accurate Integer LinearProgramming (ILP) formulations to heuristic algorithmsHowever these solutions aremore focused on the use of NFVover big centralized data-centers rather than over small edgeclouds devoted to provide radio access and edge networkingservices In addition the analyzed studies rarely consideruncertainty in the definition of the problem
Hence the main contribution of this work is the designof an energy-aware VNF placement expert system for
a CE-RAN environment with 5G SCs We use constraintprogramming to solve the discrete optimization problemof resource assignment combined with robust optimizationtechniques (RO) [10] to develop an energy-awareVNFplace-ment policy Thus our approach differentiates from othersin the use of RO which allows us to generate a suboptimalplacement policy that enables the introduction of uncertaintyin the problem In particular we introduce service demanduncertainty in order to capture the variable nature of trafficflows size and per-service job execution burden
The paper has been organized in six sections Section 2reviews the related work In Section 3 we describe the VNFplacement problem under a 5G environment Section 4presents our placement proposal using the RO approach pre-viously providing the problem model Next in Section 5 weanalyze the performance of the proposed placement solutionFinally Section 6 gathers the conclusions of this work
2 Related Work
Nowadays mobile operators see network virtualization asthe key towards enabling flexible 5G network architectureFor this reason the problem of VNF placement has beenattracting increasing attention during the last years [11ndash25]Nonetheless the optimal placement of virtual elements tophysical resources aimed at minimizing power consumptionwith service performance constraints for the 5G scenariounder study is a complex issue
VNFplacement is a resource allocation discrete or combi-natorial optimization problem that can be considered equiv-alent to a knapsack problem In consequence it can be solvedusing exact methods or heuristics depending on the numberof VNF instances and the complexity of the resources incompetition As typically provided in the literature (see [12ndash14]) an ILP approach optimally solves the placement problemin reasonable time when the number of service instances islow In reference to heuristics-based placement proposals thevariety of tools employed for solving the problem is hugefor example [15] uses a greedy algorithm whereas [16] usesa binary search heuristic
Even though some of the aforementioned relevant worksdeal with energy-related optimization taking into accountdelay bounds (eg the objective in [12 13] is minimizing thenumber of CPUs constrained by latency) those solutions donot usually consider optimalsuboptimal power consumptionitself However it is worth mentioning the few works wehave found deal with power-based VNF placement proposals[17ndash21] for our case study Reference [17] aims at minimizingpower consumption allowing users to meet delay require-ments using a genetic algorithm approach while [18 19] pro-pose heuristic algorithms that give power reduction improve-ment Besides [20] presents a suboptimal placement solutionthat combines traffic and energy cost optimization derivedby means of a Markov approximation with matching theoryApart from that research works in [22ndash24] go a step forwardand cope with the migration problem of active microserversneeded in low traffic conditions to turn off or suspendmicroservers so as to achieve energy savings
Mobile Information Systems 3
NS request Service orchestrator NS catalog
VNF catalog
Available virtualizedresources
Placement algorithm
VNF1 VNF2 VNF3 VNF4 VNF5VNFchain
Infrastructure manager
SLA
Cluster network
microserver microserver microserver
Clou
d-RA
N
VM
VM
Virtualization layer
HW resources HW resources HW resources
VNF1VM
VNF3VM
VNF5VM
VNF4VM
VNF2VM
VM
Figure 2 VNF placement process
Moreover a common assumption of the different opti-mization models proposed to solve different versions of theVNFs placement problem is that input data is perfectlyknown which is difficult in practice Unfortunately smalldeviations in input data values may usually lead to situationswhere a previously found optimal solution is not even feasibleanymore and consequently the presence of uncertain datamay produce useless placement solutions In order to copewith uncertain input parameters RO [10] is applied to solveoptimization problems Indeed RO has been successfullyemployed in different contexts such as in resource allocationin OFDM networks [26] with uncertainty in channel stateinformation in cloud resource provisioning without perfectknowledge of demands and price [21] and even in the virtualnetwork embedding problem on a physical network substratein recent works [27ndash30] that usually deal with uncertainservice demands
Nevertheless to the best of our knowledge except thework presented in [21] there is no VNF placement proposalbased on RO Therefore to achieve our goal of designingan energy-aware placement solution using RO approach thatconsiders service demand uncertainty [21] has been relevantAuthors develop a novel power-based placement solutionwith uncertainty inVNFsCPUdemands based onRO theoryTo this aim they take into account the physical serversand network resources along with their energy efficiencyNonetheless they focus on the core network part whereas ourresearch is oriented to combined deployment of radioservicenetwork functions at the edge Additionally the placementproblem considered in this paper takes into account a het-erogeneous edge cloudmade up of a cluster of interconnectedSCs
3 Problem Description
Thenetworking environment targeted in ourwork is a 5Gdis-tributed edge cloudThis environment uses SCs that evolve tobuild multitenant CE-RANs The enhanced SCs deploymentis offered to multiple operatorsservice providers to servetheir network services in the virtualized infrastructure andengage them in a multitenant system Figure 2 shows the NSrequest and subsequent placement decision process in thisjoint radio-cloud architecture
In such an architecture whenever a tenant needs todeploy a new service itmakes a request to the ServiceOrches-trator functional element which is implemented as theNFV Orchestrator (NFVO) in ETSI-MANO terminologyThe orchestrator analyzes its NS catalog to extract thecorrespondent service chain A service chain is the sequenceof VNFs that composes the requested 5G service
The VNF chain extracted from the catalog is later used asan input for the placement algorithm together with the ser-vice constraints imposed by the key performance indicators(KPI) of the corresponding Service Level Agreement (SLA)between the tenant and the CE-RAN operator In our workthe SLA KPIs include the following network parametersmaximum accepted latency for the NS aggregated user bitrate and a robust protection parameterThis protection para-meter is related to the uncertainty of the user traffic demandWe assume that the network service has a mean aggregatedtraffic demand that is used to size the necessary resourcesin order to serve tenantrsquos user But along the service timethe user traffic demand fluctuates around this mean valueaffecting the amount of resources needed The robust pro-tection parameter is therefore related to the expected peak
4 Mobile Information Systems
traffic demand deviation and will be used to overestimate theallocation of virtualized resources in order to be prepared foreventual user peak demands
Next the placement algorithm inspects the VNF catalogfor the parametrization of theVNF instances needed tomatchthe SLA and checks the available resources provided throughthe virtualized infrastructure VNFs are characterized by theuse of resources (ie number of cores RAM and storage)according to the aggregated bit rate (ie characterized aggre-gated traffic flows of a NS) served by the NS they belongtoMoreover the service latency of eachVNF also depends onthe traffic load supported by the VNF instance Lastly virtu-alized resources may also include other hardware appliancessuch as hardware accelerators (HWA) that can improve theperformance of a VNF in terms of latency This way theseadditional appliances help matching SLA with the tenant butat a higher energy cost
All the VNFs composing the service chains of therequested network services must be allocated in the availablevirtualized resources including a given number of CPUshardware appliances RAM and storage space and networkbandwidth We assume that each core runs a single VM thatis devoted to the execution of a single VNF instance but oneVNF instance can scale from one to many VMs upon the ser-vice requirements of theNS Besides the transmission latencyalong the network topology between two VNFs allocatedin different microservers depends on the traffic load of theinvolved links and the size of the flow to be transmittedWith this information the placement algorithm assigns eachVNF instance to one or more VMs with the objective ofminimizing the energy consumption of the whole systemwhile matching the latency constraints for the NS
Finally the Infrastructure Manager functional elementVirtualized Infrastructure Manager (VIM) in ETSI-MANOterminology receives the outcome of the placement decisionand maps each VNF to the assigned resources
It is important to point out that a VNF chain includesradio-related and service-related instances The placementalgorithm introduced in this work considers a scenario withseveral tenants each of them requesting one or more NSswhich share the available resources Considering the afore-mentioned communication particularities of 5G Figure 3shows an example of a simple complete placement result ofa multitenant environment
This example considers a distributed edge cloud shared bytwo tenants Tenant 1 offers two network services the first onechaining a virtualized firewall (vFW) an intrusion detectionfunction (vID) a watermarking process (vWM) and finally acaching service (vCache) The second NS of Tenant 1 chainsthe vFW and a video analytics (VA) functionality for onlinesurveillance Tenant 2 offers a single network service thanchains the vFW a vCache service and a virtual transcodingunit (vTU) All NSs are associated with their correspondentlatency constraint
In the example both NSs belonging to the same tenantshare the radio VNF (small cell VNF or SCVNF) In this waythe common radio operations are performed in a coherentway between the different edge services of a tenant and thedata paths of the specific service instances per tenant are split
SCVNF SCVNF
vFW
VA
vFW
vWM
vCachevFW
vCache
vTU
HW
Acc
CPU
s
vID
NS2
NS1
NS1
NS1
NS1
NS1
NS1
NS1 N
S3NS3
NS1
NS2
Network topology
vFW vCache vTU
vFW VA
vFW vID vWM vCache ltlat1
ltlat2
ltlat3
Tena
nt 1
SCVNF
SCVNF
Tena
nt 2
NS1
NS2
NS3
vIDlowast
Figure 3 Multitenant placement example lowast indicates that ldquovIDrdquouses a hardware acceleration appliance
The SCVNF of each tenant will be shared by all theNSs of thistenant which means that the use of resources of the SCVNFmust then consider the sumof the aggregated flows of theNSsof the same tenant
The distinctive features of 5G communications have twoimportant implications over the placement algorithm Firstlyevery tenant allocates a unique SCVNF instance at the edgecloud in order to support the control plane and data planeoperations of all the SCs associated with that tenant Andsecondly since the SCVNFmanages the uplink and downlinktraffic between themobile users and the core network it leadsto circular forwarding paths when including edge serviceinstances In this paper we consider that the core networkfunctionalities of themobile network operator are outside theCE-RAN which means that the CE-RAN only deploys radioand edge service instances In order to keep the data pathbetween the mobile users and the core network functions alltheNSs need to start and end at the SCVNFwhichwill handlethe user plane traffic tofrom the core network
The example also shows how some VNFs needmore thanoneVM to execute the associated functionalities and howoneVNF uses a HWA tomeet the latency constraint of the server
Section 5 will show the result of more complex and com-plete placement experiments For the sake of generality thispaper does not focus on the specific modelling of the trafficpatterns the energy consumption schemes or the VNF net-work service performance Section 5 provides some referenceoutcomes on the potential benefits of the introduced VNFplacement algorithm in the considered network scenario but
Mobile Information Systems 5
the applicability of the solution to other scenarios shouldbe fed by experimental modelling data specific to the targetscenario
4 Robust Constraint-Based VNFPlacement Proposal
Once the 5G VNF placement problem under study is pre-sented in this section we propose a placement solution tothat problem using the RO approach To this end first weformulate the energy-aware VNF placement model in thenext subsection Then we provide our RO-based placementproposal in Section 42
41 ProblemModelling As stated before the VNF placementproblem addressed in this work focuses on the assignment ofthe VNFs composing each NS to the provided 5G networkinfrastructure Therefore it constitutes a discrete or com-binatorial optimization problem where the minimizationof energy consumption constitutes the optimality criterionSummary of Model Indexes Sets Parameters and DecisionVariables reports and summarizes the mathematical nota-tions used for the model indexes sets input parameters anddecision variables of VNF placement problem that is definedin the following
The Problem Model The energy-aware VNF placement opti-mization problem
min120587isinΠ
119875120587 (1)
st 119889120587119894 lt 119863max119894 forall119894 isin I 120587 isin Π (2)
119898120587119903119909 lt 119872119903119909 forall119903 isin R 119909 isin X 120587 isin Π (3)
119905119897120587119909 lt BW119909 forall119909 isin X 120587 isin Π (4)
119875120587 = sum119894isinI
sum119895isinJ119894
119875120587119894119895 (5)
119875120587119894119895
= sum119909isinX
sum119910isinY119909
119875cpu119909119910119894119895 119860119909119910
119894119895 + sum119894isinI
sum119895isinJ119894
119875switch119894119895119871 119894119895
+ sum119909isinX
119875sc119909119860119909119910
119894119895
(6)
119889120587119894 = sum119895isinJ119894
119889120587119894119895 (7)
119889120587119894119895
= sum119909isinX
sum119910isinY119909
119889119901119909119910119894119895 119860119909119910
119894119895
+ sum1199091isinX
sum1199101isinY1199091
sum1199092isinX
sum1199102isinY1199092
11988911989711990911199101 11990921199102119894119895 11986011990911199101119894119895 11986011990921199102119894119895+1
(8)
sum119894isinI
sum119895isinJ119894
119860119909119910119894119895 = 1 forall119909 isin X 119910isin Y119909 (9)
119898120587119903119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119898119903119894119895119860119909119910
119894119895 (10)
119905119897120587119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119879119894119860119909119910
119894119895 1198601199091015840119910
119894119895+1 (11)
We consider a set X of small cells that compose the CE-RAN and a set Y119909 of cores included in the microserver ofthe SC 119909 Additionally we also consider a set of I networkservices offered by the CE-RAN operator to the differenttenants and a set J119894 of VNFs that form the functionalchain of the NS 119894 Finally R represents the set of differentresources available in the microservers for the execution ofthe VNFs (storage RAM hardware accelerators etc) Henceour constraint-based expert systemmust determine in whichVM 119910 of a SC 119909 each VNF 119895 of a NS 119894 is located
Let Π denote the set of all possible power-aware andlatency- and resource-constrained VNF placement policieswhich decide the mapping of each VNF of available NSs toavailable VMs in the provided SCs We formulate our VNFplacement optimization problem aimed atminimizing powerconsumption with delay requirements as (1)
In reference to the energy part we define the total powerconsumption 119875120587 as the sum of the energy demand of all theallocated VNFs 119875120587119894119895 (5) Then for each VNF instance thepower consumption is expressed in (6) as the sum of thepower consumption of VNF 119895 of NS 119894 due to the CPU useof VM 119910 of SC 119909 the power consumption of the physicalswitch due to the forwarded traffic of the VNF and the powerconsumption of SC 119909 because of being switched on
The power consumption of the cores that run the VMsdepends of the aggregated user traffic served by the corre-spondent VNF and this relationship may be nonlinear Itmust be also understood that when a VNF instance scales toa higher number of cores the individual load of the involvedCPUs decreases leading to a lower power consumption percore Additionally the energy consumption of the networkingdevices (eg an Ethernet switch) depends on the aggregatedflow that traverses the device to forward data packets betweenSCs and the number of active ports Finally it is realized thatthe power wasted because the SC is active is constant notaffecting the optimization results Besides we express per-NSdelay in (7) as the sum of per-VNF delays and the per-VNFdelay in (8) as the sum of the processing delays introducedby the execution of the VNF instance plus the path delayintroduced by the traffic going from one VNF to the next onein the chain defined in the correspondent NS
On the one hand the processing delay in the CPU growswith the supported user traffic load On the other hand weconsider that the pathlink delay is composed of several linkdelays fromtoVMs tofrom the switches plus the delay in theswitches Hence depending on the adopted placement policy120587 each NS will follow a different path through the networkIn such a way if the VNFs belonging to a NS are placed in thesame SC the service chain traverses virtualinternal links andthe virtual switch of the selected SC whereas if the VNFs of
6 Mobile Information Systems
a NS are located in different SCs external links and the physi-cal switch is also used therefore generally 11988911989711990911199101 11990921199102119894119895 =11988911990911199101vs + 119889vs
1199091+ 119889vs1199091 119904 + 119889119904 + 119889119904vs1199092 + 119889vs
1199092+ 119889vs11990921199102 but for
a given VNF if the next VNF in its chain holds in the sameSC its link delay is only affected because of passing throughthe virtual switch Note that in case a VNF is the last VNF inthe chain the link delay is null with the sums referring to thelink delay for that 119895 being null
In order to guarantee that each VM holds only a singleVNF constraint (9) must be satisfied Nevertheless weconsider that each VNF of a NS can be instantiated as a VMthat is executed over several cores of the same SC Apart fromthat as previously detailed in Section 3 the first VNF and thelast VNF in a service chain are the same (the SCVNF) in orderto perform the specific functions that allow the split betweencontrol and data plane required in 5G communicationsTherefore it implies that 1198601199091199101198941 = 119860119909119869119894119894119895
Finally the model must also include the constraintsreferred to as the available resources In this sense (3) statesthat the consumption of the resource 119903 due to the placementof VNFs in the microserver of the SC 119909119898120587119903119909 must not exceedthe total amount of resource 119872119903119909 where 119898120587119903119909 is defined in(10) Similarly (4) constrains the traffic through the networklinks 119905119897120587119909 to the maximum capacity of the links BW119909 as thesum of the traffic flows of the network services that must beforwarded from a VNF in one SC to the next VNF in anotherSC
42 Robust Constraint-Based Solution As described inSection 2 many existing placement works assume perfectknowledge on input parameters pointing thus to the for-mulation of a deterministic optimization problem Unfortu-nately in the real world the estimated values of the inputparameters may differ due to biased data unrealistic assump-tions or numerical errors affecting the obtained optimalsolution and its performance This potential deviation ofthe nominal or expected input may lead to the violation ofthe problem constraints and therefore make the obtainedsolution suboptimal or even unfeasible
The uncertainty of modelling parameters has been tra-ditionally handled through stochastic programming andsensitivity analysis but robust optimization techniques haverecently arisen as a powerful tool to manage the impact ofuncertain input sets RO seeks an uncertainty-immunizedsolution with an acceptable performance under the real-ization of the uncertain inputs becoming a conservativeworst-case oriented methodology The main tools of RO areuncertainty sets and a robust counterpart problem [31] Theuncertainty of the input data is described later in this sectionwhile the robust counterpart problem is the deterministicmodel previously detailed in Section 41
Here we propose a robust constraint-based placementsolution that deals with service demand uncertainty Withthis RO approach we consider a tradeoff between theproblem optimization and the protection from deviationscaused by input parameters uncertainty This uncertainty inservice demand has an impact on several components on theformulation previously presented in Section 41 On the one
hand we consider that a VM which is running a specificVNF requires an expected use of CPU and thus a certainconstant power consumption due to CPU Neverthelessthe uncertainty in traffic demand may cause uncertainty inthe CPU requirementuse and consequently in its energyconsumption and processing delays On the other hand bothenergy consumption and delays in switches and links dependon that demand uncertainty as well
In order to obtain a proposal for the placement problemwith uncertain service demand we employ the Soyster [10]method used in the framework of RO That technique isalso known as the worst-case approach where the solutionsare achieved using the most extreme expected values of theuncertain variables Therefore that kind of resolution guar-antees feasible solutions for any value of the uncertain servicedemand
In reference to the modelling of the uncertain trafficdemand per NS 119894 we consider an expected traffic demand119864[119879119894] = 119879119894 plus a term due to uncertainty in the servicedemandUTmax119894 We assume that the randomvariableUTmax119894is symmetrically distributed within [minusΔ119879119894 +Δ119879119894] sdot 119879119894 andwith mean zero Furthermore the sum of deviations of theuncertain service demand should not exceed the worst-casemaximum value 119879max119894 fulfilling (12) In this way the valueof 119879max119894 controls the tradeoff between the robustness andthe impact on the optimization higher 119879max119894 leads to worseobjective function values but protects from more parameterdeviations
119880119879119894Δ119879119894
le 119879max119894 forall119894 (12)
5 Analysis of the Results
This section presents the results obtained with the simulationof the proposed robust constraint-based expert system for theplacement of VNF chains in an emerging 5G distributed edgecloud As previously introduced in Section 2 the mathemat-ical VNF placement problemmodel presented in Section 4 isa combinatorial or discrete optimization problem equivalentto the well-known knapsack problem This means that thedecision problem is NP-complete while the optimizationproblem is NP-hard [11] The proposed optimization modelhas been solved using constraint programming as a powerfulparadigm for solving combinatorial search problems [32]
We usedMinizinc [33] a free and open-source constraintmodelling language to model the optimization and con-straint satisfaction VNF placement problem in a high-levelsolver-independent wayThen Minizinc compiles this modelinto FlatZinc a solver input language that is understoodby a wide range of solvers In our case Gecode solver [34]provided the best performance in the search of the optimalsolution
The experiments that are described and discussed belowhave been conducted in a single computer with a 27GHzIntel Core i5 processor and 16GB RAM This simple con-figuration provides solutions within minutes for simpleexperiments with loose constraint satisfaction When theexperiments demand tighter latency or resource constraints
Mobile Information Systems 7
Virt
ual i
nfra
struc
ture
Corenetwork
Figure 4 Evaluation scenario
the solving time increases to hours depending on the specificconfiguration of the experiment
Next we first describe the analysis scenario and the con-figuration of the performed experiments to later discuss theresults of those experimentsThe objective of the experimentsis to demonstrate that the placement mechanism is able toallocate multitenant service chains to the available resourcesconsidering latency constraints For this reason the numeri-cal values employed in the next section are just experimentaland do not respond to real world measurements
51 Description of the Analysis Scenario Our evaluationscenario is depicted in Figure 4 We consider a 10-SC deploy-ment for example to cover a football stadium for periodicalflash eventsThe SCs are connected in a star topology throughan Ethernet switch Each SC is composed of 8 cores 16GBRAM 100GB of storage and 1HWA (eg GPU)
We also consider 2 NSs that can be offered at 3 serviceflavours (bronze silver and gold) that determine the latencyconstraint for eachNS and the nominal aggregate user bit rateserved as shown in Figure 5
Table 1 shows the experimental energy model and Table 2shows the modelling of the VNFs that compose the proposedNSs for the user traffic demand that will be employed in theevaluation experiments The user traffic values include thenominal bit rates of services NS1 and NS2 (130Mbps and90Mbps resp) as well as the values when a 20 protectionparameter is applied to the nominal values (156Mbps and108Mbps resp) in order to study the effect of robust optimi-zation techniques on the placement decision
As a final remark the placement behavior of VNFs canbe altered by the use of HWA The assignment of a HWA toa VNF implies that the service latency of the correspondentinstance is reduced 3 times at a cost of incrementing thepower consumption by 10 It is important to note that whenthe VNF instance needs more than 1 CPU for executingwithout the assistance of a HWA then if the placementalgorithm assigns it to a HWA the VNF will need a singlecore
52 Discussion of the Placement Results As stated before theevaluation scenario includes 3 tenants that hire a combination
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
2 Mobile Information Systems
SC small cell NF network function
PNF physical NF VNF virtualized NF
Mobile core network
PNF PNF PNF
Mobile edgecomputing
SC SC SC
Dist
ribut
ed cl
oud-
enab
led
radi
o ac
cess
net
wor
k
VNF VNF VNF
Figure 1 Cloud-enabled RAN architecture based on small cells
Nevertheless the success of the presented novel approachundoubtedly depends on the VNF placement strategy usedto allocate VNFs into the available resources given a set ofservice constraints However defining efficient mechanismsto allocate the software-based components of NS onto thenetworked data-centers (set ofmicroservers that conform thecloud-enabled SC cluster in our case) is a challenging taskTherefore the relevance of designing intelligent VNF place-ment algorithms becomes fundamental while it is notdefined in standards [9]
Besides current trends towards green information andcommunication technologies target the improvement ofenergy consumption in all the nodes including the networknodes themselves In this paper we will particularly discusshow to derive an energy-aware VNF placement solution fora cluster of cloud-enabled SCs in a 5G deployment sce-nario With this purpose we will deal with the objective ofminimizing power consumption of both the microserversand SDN-enabled switches to deploy the overall virtualizedinfrastructure needed (including virtual switches within eachSC and physical ones interconnecting SCs)
In order to address the VNF placement problem underthe aforementioned novel 5G scenario this paper proposesa placement mechanism that minimizes power consumptionsubject to bit rate and latency constraints imposed by thenetwork services and is aware of the available underlyingresources Existing literature provides a wide variety of VNFplacement approaches ranging from accurate Integer LinearProgramming (ILP) formulations to heuristic algorithmsHowever these solutions aremore focused on the use of NFVover big centralized data-centers rather than over small edgeclouds devoted to provide radio access and edge networkingservices In addition the analyzed studies rarely consideruncertainty in the definition of the problem
Hence the main contribution of this work is the designof an energy-aware VNF placement expert system for
a CE-RAN environment with 5G SCs We use constraintprogramming to solve the discrete optimization problemof resource assignment combined with robust optimizationtechniques (RO) [10] to develop an energy-awareVNFplace-ment policy Thus our approach differentiates from othersin the use of RO which allows us to generate a suboptimalplacement policy that enables the introduction of uncertaintyin the problem In particular we introduce service demanduncertainty in order to capture the variable nature of trafficflows size and per-service job execution burden
The paper has been organized in six sections Section 2reviews the related work In Section 3 we describe the VNFplacement problem under a 5G environment Section 4presents our placement proposal using the RO approach pre-viously providing the problem model Next in Section 5 weanalyze the performance of the proposed placement solutionFinally Section 6 gathers the conclusions of this work
2 Related Work
Nowadays mobile operators see network virtualization asthe key towards enabling flexible 5G network architectureFor this reason the problem of VNF placement has beenattracting increasing attention during the last years [11ndash25]Nonetheless the optimal placement of virtual elements tophysical resources aimed at minimizing power consumptionwith service performance constraints for the 5G scenariounder study is a complex issue
VNFplacement is a resource allocation discrete or combi-natorial optimization problem that can be considered equiv-alent to a knapsack problem In consequence it can be solvedusing exact methods or heuristics depending on the numberof VNF instances and the complexity of the resources incompetition As typically provided in the literature (see [12ndash14]) an ILP approach optimally solves the placement problemin reasonable time when the number of service instances islow In reference to heuristics-based placement proposals thevariety of tools employed for solving the problem is hugefor example [15] uses a greedy algorithm whereas [16] usesa binary search heuristic
Even though some of the aforementioned relevant worksdeal with energy-related optimization taking into accountdelay bounds (eg the objective in [12 13] is minimizing thenumber of CPUs constrained by latency) those solutions donot usually consider optimalsuboptimal power consumptionitself However it is worth mentioning the few works wehave found deal with power-based VNF placement proposals[17ndash21] for our case study Reference [17] aims at minimizingpower consumption allowing users to meet delay require-ments using a genetic algorithm approach while [18 19] pro-pose heuristic algorithms that give power reduction improve-ment Besides [20] presents a suboptimal placement solutionthat combines traffic and energy cost optimization derivedby means of a Markov approximation with matching theoryApart from that research works in [22ndash24] go a step forwardand cope with the migration problem of active microserversneeded in low traffic conditions to turn off or suspendmicroservers so as to achieve energy savings
Mobile Information Systems 3
NS request Service orchestrator NS catalog
VNF catalog
Available virtualizedresources
Placement algorithm
VNF1 VNF2 VNF3 VNF4 VNF5VNFchain
Infrastructure manager
SLA
Cluster network
microserver microserver microserver
Clou
d-RA
N
VM
VM
Virtualization layer
HW resources HW resources HW resources
VNF1VM
VNF3VM
VNF5VM
VNF4VM
VNF2VM
VM
Figure 2 VNF placement process
Moreover a common assumption of the different opti-mization models proposed to solve different versions of theVNFs placement problem is that input data is perfectlyknown which is difficult in practice Unfortunately smalldeviations in input data values may usually lead to situationswhere a previously found optimal solution is not even feasibleanymore and consequently the presence of uncertain datamay produce useless placement solutions In order to copewith uncertain input parameters RO [10] is applied to solveoptimization problems Indeed RO has been successfullyemployed in different contexts such as in resource allocationin OFDM networks [26] with uncertainty in channel stateinformation in cloud resource provisioning without perfectknowledge of demands and price [21] and even in the virtualnetwork embedding problem on a physical network substratein recent works [27ndash30] that usually deal with uncertainservice demands
Nevertheless to the best of our knowledge except thework presented in [21] there is no VNF placement proposalbased on RO Therefore to achieve our goal of designingan energy-aware placement solution using RO approach thatconsiders service demand uncertainty [21] has been relevantAuthors develop a novel power-based placement solutionwith uncertainty inVNFsCPUdemands based onRO theoryTo this aim they take into account the physical serversand network resources along with their energy efficiencyNonetheless they focus on the core network part whereas ourresearch is oriented to combined deployment of radioservicenetwork functions at the edge Additionally the placementproblem considered in this paper takes into account a het-erogeneous edge cloudmade up of a cluster of interconnectedSCs
3 Problem Description
Thenetworking environment targeted in ourwork is a 5Gdis-tributed edge cloudThis environment uses SCs that evolve tobuild multitenant CE-RANs The enhanced SCs deploymentis offered to multiple operatorsservice providers to servetheir network services in the virtualized infrastructure andengage them in a multitenant system Figure 2 shows the NSrequest and subsequent placement decision process in thisjoint radio-cloud architecture
In such an architecture whenever a tenant needs todeploy a new service itmakes a request to the ServiceOrches-trator functional element which is implemented as theNFV Orchestrator (NFVO) in ETSI-MANO terminologyThe orchestrator analyzes its NS catalog to extract thecorrespondent service chain A service chain is the sequenceof VNFs that composes the requested 5G service
The VNF chain extracted from the catalog is later used asan input for the placement algorithm together with the ser-vice constraints imposed by the key performance indicators(KPI) of the corresponding Service Level Agreement (SLA)between the tenant and the CE-RAN operator In our workthe SLA KPIs include the following network parametersmaximum accepted latency for the NS aggregated user bitrate and a robust protection parameterThis protection para-meter is related to the uncertainty of the user traffic demandWe assume that the network service has a mean aggregatedtraffic demand that is used to size the necessary resourcesin order to serve tenantrsquos user But along the service timethe user traffic demand fluctuates around this mean valueaffecting the amount of resources needed The robust pro-tection parameter is therefore related to the expected peak
4 Mobile Information Systems
traffic demand deviation and will be used to overestimate theallocation of virtualized resources in order to be prepared foreventual user peak demands
Next the placement algorithm inspects the VNF catalogfor the parametrization of theVNF instances needed tomatchthe SLA and checks the available resources provided throughthe virtualized infrastructure VNFs are characterized by theuse of resources (ie number of cores RAM and storage)according to the aggregated bit rate (ie characterized aggre-gated traffic flows of a NS) served by the NS they belongtoMoreover the service latency of eachVNF also depends onthe traffic load supported by the VNF instance Lastly virtu-alized resources may also include other hardware appliancessuch as hardware accelerators (HWA) that can improve theperformance of a VNF in terms of latency This way theseadditional appliances help matching SLA with the tenant butat a higher energy cost
All the VNFs composing the service chains of therequested network services must be allocated in the availablevirtualized resources including a given number of CPUshardware appliances RAM and storage space and networkbandwidth We assume that each core runs a single VM thatis devoted to the execution of a single VNF instance but oneVNF instance can scale from one to many VMs upon the ser-vice requirements of theNS Besides the transmission latencyalong the network topology between two VNFs allocatedin different microservers depends on the traffic load of theinvolved links and the size of the flow to be transmittedWith this information the placement algorithm assigns eachVNF instance to one or more VMs with the objective ofminimizing the energy consumption of the whole systemwhile matching the latency constraints for the NS
Finally the Infrastructure Manager functional elementVirtualized Infrastructure Manager (VIM) in ETSI-MANOterminology receives the outcome of the placement decisionand maps each VNF to the assigned resources
It is important to point out that a VNF chain includesradio-related and service-related instances The placementalgorithm introduced in this work considers a scenario withseveral tenants each of them requesting one or more NSswhich share the available resources Considering the afore-mentioned communication particularities of 5G Figure 3shows an example of a simple complete placement result ofa multitenant environment
This example considers a distributed edge cloud shared bytwo tenants Tenant 1 offers two network services the first onechaining a virtualized firewall (vFW) an intrusion detectionfunction (vID) a watermarking process (vWM) and finally acaching service (vCache) The second NS of Tenant 1 chainsthe vFW and a video analytics (VA) functionality for onlinesurveillance Tenant 2 offers a single network service thanchains the vFW a vCache service and a virtual transcodingunit (vTU) All NSs are associated with their correspondentlatency constraint
In the example both NSs belonging to the same tenantshare the radio VNF (small cell VNF or SCVNF) In this waythe common radio operations are performed in a coherentway between the different edge services of a tenant and thedata paths of the specific service instances per tenant are split
SCVNF SCVNF
vFW
VA
vFW
vWM
vCachevFW
vCache
vTU
HW
Acc
CPU
s
vID
NS2
NS1
NS1
NS1
NS1
NS1
NS1
NS1 N
S3NS3
NS1
NS2
Network topology
vFW vCache vTU
vFW VA
vFW vID vWM vCache ltlat1
ltlat2
ltlat3
Tena
nt 1
SCVNF
SCVNF
Tena
nt 2
NS1
NS2
NS3
vIDlowast
Figure 3 Multitenant placement example lowast indicates that ldquovIDrdquouses a hardware acceleration appliance
The SCVNF of each tenant will be shared by all theNSs of thistenant which means that the use of resources of the SCVNFmust then consider the sumof the aggregated flows of theNSsof the same tenant
The distinctive features of 5G communications have twoimportant implications over the placement algorithm Firstlyevery tenant allocates a unique SCVNF instance at the edgecloud in order to support the control plane and data planeoperations of all the SCs associated with that tenant Andsecondly since the SCVNFmanages the uplink and downlinktraffic between themobile users and the core network it leadsto circular forwarding paths when including edge serviceinstances In this paper we consider that the core networkfunctionalities of themobile network operator are outside theCE-RAN which means that the CE-RAN only deploys radioand edge service instances In order to keep the data pathbetween the mobile users and the core network functions alltheNSs need to start and end at the SCVNFwhichwill handlethe user plane traffic tofrom the core network
The example also shows how some VNFs needmore thanoneVM to execute the associated functionalities and howoneVNF uses a HWA tomeet the latency constraint of the server
Section 5 will show the result of more complex and com-plete placement experiments For the sake of generality thispaper does not focus on the specific modelling of the trafficpatterns the energy consumption schemes or the VNF net-work service performance Section 5 provides some referenceoutcomes on the potential benefits of the introduced VNFplacement algorithm in the considered network scenario but
Mobile Information Systems 5
the applicability of the solution to other scenarios shouldbe fed by experimental modelling data specific to the targetscenario
4 Robust Constraint-Based VNFPlacement Proposal
Once the 5G VNF placement problem under study is pre-sented in this section we propose a placement solution tothat problem using the RO approach To this end first weformulate the energy-aware VNF placement model in thenext subsection Then we provide our RO-based placementproposal in Section 42
41 ProblemModelling As stated before the VNF placementproblem addressed in this work focuses on the assignment ofthe VNFs composing each NS to the provided 5G networkinfrastructure Therefore it constitutes a discrete or com-binatorial optimization problem where the minimizationof energy consumption constitutes the optimality criterionSummary of Model Indexes Sets Parameters and DecisionVariables reports and summarizes the mathematical nota-tions used for the model indexes sets input parameters anddecision variables of VNF placement problem that is definedin the following
The Problem Model The energy-aware VNF placement opti-mization problem
min120587isinΠ
119875120587 (1)
st 119889120587119894 lt 119863max119894 forall119894 isin I 120587 isin Π (2)
119898120587119903119909 lt 119872119903119909 forall119903 isin R 119909 isin X 120587 isin Π (3)
119905119897120587119909 lt BW119909 forall119909 isin X 120587 isin Π (4)
119875120587 = sum119894isinI
sum119895isinJ119894
119875120587119894119895 (5)
119875120587119894119895
= sum119909isinX
sum119910isinY119909
119875cpu119909119910119894119895 119860119909119910
119894119895 + sum119894isinI
sum119895isinJ119894
119875switch119894119895119871 119894119895
+ sum119909isinX
119875sc119909119860119909119910
119894119895
(6)
119889120587119894 = sum119895isinJ119894
119889120587119894119895 (7)
119889120587119894119895
= sum119909isinX
sum119910isinY119909
119889119901119909119910119894119895 119860119909119910
119894119895
+ sum1199091isinX
sum1199101isinY1199091
sum1199092isinX
sum1199102isinY1199092
11988911989711990911199101 11990921199102119894119895 11986011990911199101119894119895 11986011990921199102119894119895+1
(8)
sum119894isinI
sum119895isinJ119894
119860119909119910119894119895 = 1 forall119909 isin X 119910isin Y119909 (9)
119898120587119903119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119898119903119894119895119860119909119910
119894119895 (10)
119905119897120587119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119879119894119860119909119910
119894119895 1198601199091015840119910
119894119895+1 (11)
We consider a set X of small cells that compose the CE-RAN and a set Y119909 of cores included in the microserver ofthe SC 119909 Additionally we also consider a set of I networkservices offered by the CE-RAN operator to the differenttenants and a set J119894 of VNFs that form the functionalchain of the NS 119894 Finally R represents the set of differentresources available in the microservers for the execution ofthe VNFs (storage RAM hardware accelerators etc) Henceour constraint-based expert systemmust determine in whichVM 119910 of a SC 119909 each VNF 119895 of a NS 119894 is located
Let Π denote the set of all possible power-aware andlatency- and resource-constrained VNF placement policieswhich decide the mapping of each VNF of available NSs toavailable VMs in the provided SCs We formulate our VNFplacement optimization problem aimed atminimizing powerconsumption with delay requirements as (1)
In reference to the energy part we define the total powerconsumption 119875120587 as the sum of the energy demand of all theallocated VNFs 119875120587119894119895 (5) Then for each VNF instance thepower consumption is expressed in (6) as the sum of thepower consumption of VNF 119895 of NS 119894 due to the CPU useof VM 119910 of SC 119909 the power consumption of the physicalswitch due to the forwarded traffic of the VNF and the powerconsumption of SC 119909 because of being switched on
The power consumption of the cores that run the VMsdepends of the aggregated user traffic served by the corre-spondent VNF and this relationship may be nonlinear Itmust be also understood that when a VNF instance scales toa higher number of cores the individual load of the involvedCPUs decreases leading to a lower power consumption percore Additionally the energy consumption of the networkingdevices (eg an Ethernet switch) depends on the aggregatedflow that traverses the device to forward data packets betweenSCs and the number of active ports Finally it is realized thatthe power wasted because the SC is active is constant notaffecting the optimization results Besides we express per-NSdelay in (7) as the sum of per-VNF delays and the per-VNFdelay in (8) as the sum of the processing delays introducedby the execution of the VNF instance plus the path delayintroduced by the traffic going from one VNF to the next onein the chain defined in the correspondent NS
On the one hand the processing delay in the CPU growswith the supported user traffic load On the other hand weconsider that the pathlink delay is composed of several linkdelays fromtoVMs tofrom the switches plus the delay in theswitches Hence depending on the adopted placement policy120587 each NS will follow a different path through the networkIn such a way if the VNFs belonging to a NS are placed in thesame SC the service chain traverses virtualinternal links andthe virtual switch of the selected SC whereas if the VNFs of
6 Mobile Information Systems
a NS are located in different SCs external links and the physi-cal switch is also used therefore generally 11988911989711990911199101 11990921199102119894119895 =11988911990911199101vs + 119889vs
1199091+ 119889vs1199091 119904 + 119889119904 + 119889119904vs1199092 + 119889vs
1199092+ 119889vs11990921199102 but for
a given VNF if the next VNF in its chain holds in the sameSC its link delay is only affected because of passing throughthe virtual switch Note that in case a VNF is the last VNF inthe chain the link delay is null with the sums referring to thelink delay for that 119895 being null
In order to guarantee that each VM holds only a singleVNF constraint (9) must be satisfied Nevertheless weconsider that each VNF of a NS can be instantiated as a VMthat is executed over several cores of the same SC Apart fromthat as previously detailed in Section 3 the first VNF and thelast VNF in a service chain are the same (the SCVNF) in orderto perform the specific functions that allow the split betweencontrol and data plane required in 5G communicationsTherefore it implies that 1198601199091199101198941 = 119860119909119869119894119894119895
Finally the model must also include the constraintsreferred to as the available resources In this sense (3) statesthat the consumption of the resource 119903 due to the placementof VNFs in the microserver of the SC 119909119898120587119903119909 must not exceedthe total amount of resource 119872119903119909 where 119898120587119903119909 is defined in(10) Similarly (4) constrains the traffic through the networklinks 119905119897120587119909 to the maximum capacity of the links BW119909 as thesum of the traffic flows of the network services that must beforwarded from a VNF in one SC to the next VNF in anotherSC
42 Robust Constraint-Based Solution As described inSection 2 many existing placement works assume perfectknowledge on input parameters pointing thus to the for-mulation of a deterministic optimization problem Unfortu-nately in the real world the estimated values of the inputparameters may differ due to biased data unrealistic assump-tions or numerical errors affecting the obtained optimalsolution and its performance This potential deviation ofthe nominal or expected input may lead to the violation ofthe problem constraints and therefore make the obtainedsolution suboptimal or even unfeasible
The uncertainty of modelling parameters has been tra-ditionally handled through stochastic programming andsensitivity analysis but robust optimization techniques haverecently arisen as a powerful tool to manage the impact ofuncertain input sets RO seeks an uncertainty-immunizedsolution with an acceptable performance under the real-ization of the uncertain inputs becoming a conservativeworst-case oriented methodology The main tools of RO areuncertainty sets and a robust counterpart problem [31] Theuncertainty of the input data is described later in this sectionwhile the robust counterpart problem is the deterministicmodel previously detailed in Section 41
Here we propose a robust constraint-based placementsolution that deals with service demand uncertainty Withthis RO approach we consider a tradeoff between theproblem optimization and the protection from deviationscaused by input parameters uncertainty This uncertainty inservice demand has an impact on several components on theformulation previously presented in Section 41 On the one
hand we consider that a VM which is running a specificVNF requires an expected use of CPU and thus a certainconstant power consumption due to CPU Neverthelessthe uncertainty in traffic demand may cause uncertainty inthe CPU requirementuse and consequently in its energyconsumption and processing delays On the other hand bothenergy consumption and delays in switches and links dependon that demand uncertainty as well
In order to obtain a proposal for the placement problemwith uncertain service demand we employ the Soyster [10]method used in the framework of RO That technique isalso known as the worst-case approach where the solutionsare achieved using the most extreme expected values of theuncertain variables Therefore that kind of resolution guar-antees feasible solutions for any value of the uncertain servicedemand
In reference to the modelling of the uncertain trafficdemand per NS 119894 we consider an expected traffic demand119864[119879119894] = 119879119894 plus a term due to uncertainty in the servicedemandUTmax119894 We assume that the randomvariableUTmax119894is symmetrically distributed within [minusΔ119879119894 +Δ119879119894] sdot 119879119894 andwith mean zero Furthermore the sum of deviations of theuncertain service demand should not exceed the worst-casemaximum value 119879max119894 fulfilling (12) In this way the valueof 119879max119894 controls the tradeoff between the robustness andthe impact on the optimization higher 119879max119894 leads to worseobjective function values but protects from more parameterdeviations
119880119879119894Δ119879119894
le 119879max119894 forall119894 (12)
5 Analysis of the Results
This section presents the results obtained with the simulationof the proposed robust constraint-based expert system for theplacement of VNF chains in an emerging 5G distributed edgecloud As previously introduced in Section 2 the mathemat-ical VNF placement problemmodel presented in Section 4 isa combinatorial or discrete optimization problem equivalentto the well-known knapsack problem This means that thedecision problem is NP-complete while the optimizationproblem is NP-hard [11] The proposed optimization modelhas been solved using constraint programming as a powerfulparadigm for solving combinatorial search problems [32]
We usedMinizinc [33] a free and open-source constraintmodelling language to model the optimization and con-straint satisfaction VNF placement problem in a high-levelsolver-independent wayThen Minizinc compiles this modelinto FlatZinc a solver input language that is understoodby a wide range of solvers In our case Gecode solver [34]provided the best performance in the search of the optimalsolution
The experiments that are described and discussed belowhave been conducted in a single computer with a 27GHzIntel Core i5 processor and 16GB RAM This simple con-figuration provides solutions within minutes for simpleexperiments with loose constraint satisfaction When theexperiments demand tighter latency or resource constraints
Mobile Information Systems 7
Virt
ual i
nfra
struc
ture
Corenetwork
Figure 4 Evaluation scenario
the solving time increases to hours depending on the specificconfiguration of the experiment
Next we first describe the analysis scenario and the con-figuration of the performed experiments to later discuss theresults of those experimentsThe objective of the experimentsis to demonstrate that the placement mechanism is able toallocate multitenant service chains to the available resourcesconsidering latency constraints For this reason the numeri-cal values employed in the next section are just experimentaland do not respond to real world measurements
51 Description of the Analysis Scenario Our evaluationscenario is depicted in Figure 4 We consider a 10-SC deploy-ment for example to cover a football stadium for periodicalflash eventsThe SCs are connected in a star topology throughan Ethernet switch Each SC is composed of 8 cores 16GBRAM 100GB of storage and 1HWA (eg GPU)
We also consider 2 NSs that can be offered at 3 serviceflavours (bronze silver and gold) that determine the latencyconstraint for eachNS and the nominal aggregate user bit rateserved as shown in Figure 5
Table 1 shows the experimental energy model and Table 2shows the modelling of the VNFs that compose the proposedNSs for the user traffic demand that will be employed in theevaluation experiments The user traffic values include thenominal bit rates of services NS1 and NS2 (130Mbps and90Mbps resp) as well as the values when a 20 protectionparameter is applied to the nominal values (156Mbps and108Mbps resp) in order to study the effect of robust optimi-zation techniques on the placement decision
As a final remark the placement behavior of VNFs canbe altered by the use of HWA The assignment of a HWA toa VNF implies that the service latency of the correspondentinstance is reduced 3 times at a cost of incrementing thepower consumption by 10 It is important to note that whenthe VNF instance needs more than 1 CPU for executingwithout the assistance of a HWA then if the placementalgorithm assigns it to a HWA the VNF will need a singlecore
52 Discussion of the Placement Results As stated before theevaluation scenario includes 3 tenants that hire a combination
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mobile Information Systems 3
NS request Service orchestrator NS catalog
VNF catalog
Available virtualizedresources
Placement algorithm
VNF1 VNF2 VNF3 VNF4 VNF5VNFchain
Infrastructure manager
SLA
Cluster network
microserver microserver microserver
Clou
d-RA
N
VM
VM
Virtualization layer
HW resources HW resources HW resources
VNF1VM
VNF3VM
VNF5VM
VNF4VM
VNF2VM
VM
Figure 2 VNF placement process
Moreover a common assumption of the different opti-mization models proposed to solve different versions of theVNFs placement problem is that input data is perfectlyknown which is difficult in practice Unfortunately smalldeviations in input data values may usually lead to situationswhere a previously found optimal solution is not even feasibleanymore and consequently the presence of uncertain datamay produce useless placement solutions In order to copewith uncertain input parameters RO [10] is applied to solveoptimization problems Indeed RO has been successfullyemployed in different contexts such as in resource allocationin OFDM networks [26] with uncertainty in channel stateinformation in cloud resource provisioning without perfectknowledge of demands and price [21] and even in the virtualnetwork embedding problem on a physical network substratein recent works [27ndash30] that usually deal with uncertainservice demands
Nevertheless to the best of our knowledge except thework presented in [21] there is no VNF placement proposalbased on RO Therefore to achieve our goal of designingan energy-aware placement solution using RO approach thatconsiders service demand uncertainty [21] has been relevantAuthors develop a novel power-based placement solutionwith uncertainty inVNFsCPUdemands based onRO theoryTo this aim they take into account the physical serversand network resources along with their energy efficiencyNonetheless they focus on the core network part whereas ourresearch is oriented to combined deployment of radioservicenetwork functions at the edge Additionally the placementproblem considered in this paper takes into account a het-erogeneous edge cloudmade up of a cluster of interconnectedSCs
3 Problem Description
Thenetworking environment targeted in ourwork is a 5Gdis-tributed edge cloudThis environment uses SCs that evolve tobuild multitenant CE-RANs The enhanced SCs deploymentis offered to multiple operatorsservice providers to servetheir network services in the virtualized infrastructure andengage them in a multitenant system Figure 2 shows the NSrequest and subsequent placement decision process in thisjoint radio-cloud architecture
In such an architecture whenever a tenant needs todeploy a new service itmakes a request to the ServiceOrches-trator functional element which is implemented as theNFV Orchestrator (NFVO) in ETSI-MANO terminologyThe orchestrator analyzes its NS catalog to extract thecorrespondent service chain A service chain is the sequenceof VNFs that composes the requested 5G service
The VNF chain extracted from the catalog is later used asan input for the placement algorithm together with the ser-vice constraints imposed by the key performance indicators(KPI) of the corresponding Service Level Agreement (SLA)between the tenant and the CE-RAN operator In our workthe SLA KPIs include the following network parametersmaximum accepted latency for the NS aggregated user bitrate and a robust protection parameterThis protection para-meter is related to the uncertainty of the user traffic demandWe assume that the network service has a mean aggregatedtraffic demand that is used to size the necessary resourcesin order to serve tenantrsquos user But along the service timethe user traffic demand fluctuates around this mean valueaffecting the amount of resources needed The robust pro-tection parameter is therefore related to the expected peak
4 Mobile Information Systems
traffic demand deviation and will be used to overestimate theallocation of virtualized resources in order to be prepared foreventual user peak demands
Next the placement algorithm inspects the VNF catalogfor the parametrization of theVNF instances needed tomatchthe SLA and checks the available resources provided throughthe virtualized infrastructure VNFs are characterized by theuse of resources (ie number of cores RAM and storage)according to the aggregated bit rate (ie characterized aggre-gated traffic flows of a NS) served by the NS they belongtoMoreover the service latency of eachVNF also depends onthe traffic load supported by the VNF instance Lastly virtu-alized resources may also include other hardware appliancessuch as hardware accelerators (HWA) that can improve theperformance of a VNF in terms of latency This way theseadditional appliances help matching SLA with the tenant butat a higher energy cost
All the VNFs composing the service chains of therequested network services must be allocated in the availablevirtualized resources including a given number of CPUshardware appliances RAM and storage space and networkbandwidth We assume that each core runs a single VM thatis devoted to the execution of a single VNF instance but oneVNF instance can scale from one to many VMs upon the ser-vice requirements of theNS Besides the transmission latencyalong the network topology between two VNFs allocatedin different microservers depends on the traffic load of theinvolved links and the size of the flow to be transmittedWith this information the placement algorithm assigns eachVNF instance to one or more VMs with the objective ofminimizing the energy consumption of the whole systemwhile matching the latency constraints for the NS
Finally the Infrastructure Manager functional elementVirtualized Infrastructure Manager (VIM) in ETSI-MANOterminology receives the outcome of the placement decisionand maps each VNF to the assigned resources
It is important to point out that a VNF chain includesradio-related and service-related instances The placementalgorithm introduced in this work considers a scenario withseveral tenants each of them requesting one or more NSswhich share the available resources Considering the afore-mentioned communication particularities of 5G Figure 3shows an example of a simple complete placement result ofa multitenant environment
This example considers a distributed edge cloud shared bytwo tenants Tenant 1 offers two network services the first onechaining a virtualized firewall (vFW) an intrusion detectionfunction (vID) a watermarking process (vWM) and finally acaching service (vCache) The second NS of Tenant 1 chainsthe vFW and a video analytics (VA) functionality for onlinesurveillance Tenant 2 offers a single network service thanchains the vFW a vCache service and a virtual transcodingunit (vTU) All NSs are associated with their correspondentlatency constraint
In the example both NSs belonging to the same tenantshare the radio VNF (small cell VNF or SCVNF) In this waythe common radio operations are performed in a coherentway between the different edge services of a tenant and thedata paths of the specific service instances per tenant are split
SCVNF SCVNF
vFW
VA
vFW
vWM
vCachevFW
vCache
vTU
HW
Acc
CPU
s
vID
NS2
NS1
NS1
NS1
NS1
NS1
NS1
NS1 N
S3NS3
NS1
NS2
Network topology
vFW vCache vTU
vFW VA
vFW vID vWM vCache ltlat1
ltlat2
ltlat3
Tena
nt 1
SCVNF
SCVNF
Tena
nt 2
NS1
NS2
NS3
vIDlowast
Figure 3 Multitenant placement example lowast indicates that ldquovIDrdquouses a hardware acceleration appliance
The SCVNF of each tenant will be shared by all theNSs of thistenant which means that the use of resources of the SCVNFmust then consider the sumof the aggregated flows of theNSsof the same tenant
The distinctive features of 5G communications have twoimportant implications over the placement algorithm Firstlyevery tenant allocates a unique SCVNF instance at the edgecloud in order to support the control plane and data planeoperations of all the SCs associated with that tenant Andsecondly since the SCVNFmanages the uplink and downlinktraffic between themobile users and the core network it leadsto circular forwarding paths when including edge serviceinstances In this paper we consider that the core networkfunctionalities of themobile network operator are outside theCE-RAN which means that the CE-RAN only deploys radioand edge service instances In order to keep the data pathbetween the mobile users and the core network functions alltheNSs need to start and end at the SCVNFwhichwill handlethe user plane traffic tofrom the core network
The example also shows how some VNFs needmore thanoneVM to execute the associated functionalities and howoneVNF uses a HWA tomeet the latency constraint of the server
Section 5 will show the result of more complex and com-plete placement experiments For the sake of generality thispaper does not focus on the specific modelling of the trafficpatterns the energy consumption schemes or the VNF net-work service performance Section 5 provides some referenceoutcomes on the potential benefits of the introduced VNFplacement algorithm in the considered network scenario but
Mobile Information Systems 5
the applicability of the solution to other scenarios shouldbe fed by experimental modelling data specific to the targetscenario
4 Robust Constraint-Based VNFPlacement Proposal
Once the 5G VNF placement problem under study is pre-sented in this section we propose a placement solution tothat problem using the RO approach To this end first weformulate the energy-aware VNF placement model in thenext subsection Then we provide our RO-based placementproposal in Section 42
41 ProblemModelling As stated before the VNF placementproblem addressed in this work focuses on the assignment ofthe VNFs composing each NS to the provided 5G networkinfrastructure Therefore it constitutes a discrete or com-binatorial optimization problem where the minimizationof energy consumption constitutes the optimality criterionSummary of Model Indexes Sets Parameters and DecisionVariables reports and summarizes the mathematical nota-tions used for the model indexes sets input parameters anddecision variables of VNF placement problem that is definedin the following
The Problem Model The energy-aware VNF placement opti-mization problem
min120587isinΠ
119875120587 (1)
st 119889120587119894 lt 119863max119894 forall119894 isin I 120587 isin Π (2)
119898120587119903119909 lt 119872119903119909 forall119903 isin R 119909 isin X 120587 isin Π (3)
119905119897120587119909 lt BW119909 forall119909 isin X 120587 isin Π (4)
119875120587 = sum119894isinI
sum119895isinJ119894
119875120587119894119895 (5)
119875120587119894119895
= sum119909isinX
sum119910isinY119909
119875cpu119909119910119894119895 119860119909119910
119894119895 + sum119894isinI
sum119895isinJ119894
119875switch119894119895119871 119894119895
+ sum119909isinX
119875sc119909119860119909119910
119894119895
(6)
119889120587119894 = sum119895isinJ119894
119889120587119894119895 (7)
119889120587119894119895
= sum119909isinX
sum119910isinY119909
119889119901119909119910119894119895 119860119909119910
119894119895
+ sum1199091isinX
sum1199101isinY1199091
sum1199092isinX
sum1199102isinY1199092
11988911989711990911199101 11990921199102119894119895 11986011990911199101119894119895 11986011990921199102119894119895+1
(8)
sum119894isinI
sum119895isinJ119894
119860119909119910119894119895 = 1 forall119909 isin X 119910isin Y119909 (9)
119898120587119903119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119898119903119894119895119860119909119910
119894119895 (10)
119905119897120587119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119879119894119860119909119910
119894119895 1198601199091015840119910
119894119895+1 (11)
We consider a set X of small cells that compose the CE-RAN and a set Y119909 of cores included in the microserver ofthe SC 119909 Additionally we also consider a set of I networkservices offered by the CE-RAN operator to the differenttenants and a set J119894 of VNFs that form the functionalchain of the NS 119894 Finally R represents the set of differentresources available in the microservers for the execution ofthe VNFs (storage RAM hardware accelerators etc) Henceour constraint-based expert systemmust determine in whichVM 119910 of a SC 119909 each VNF 119895 of a NS 119894 is located
Let Π denote the set of all possible power-aware andlatency- and resource-constrained VNF placement policieswhich decide the mapping of each VNF of available NSs toavailable VMs in the provided SCs We formulate our VNFplacement optimization problem aimed atminimizing powerconsumption with delay requirements as (1)
In reference to the energy part we define the total powerconsumption 119875120587 as the sum of the energy demand of all theallocated VNFs 119875120587119894119895 (5) Then for each VNF instance thepower consumption is expressed in (6) as the sum of thepower consumption of VNF 119895 of NS 119894 due to the CPU useof VM 119910 of SC 119909 the power consumption of the physicalswitch due to the forwarded traffic of the VNF and the powerconsumption of SC 119909 because of being switched on
The power consumption of the cores that run the VMsdepends of the aggregated user traffic served by the corre-spondent VNF and this relationship may be nonlinear Itmust be also understood that when a VNF instance scales toa higher number of cores the individual load of the involvedCPUs decreases leading to a lower power consumption percore Additionally the energy consumption of the networkingdevices (eg an Ethernet switch) depends on the aggregatedflow that traverses the device to forward data packets betweenSCs and the number of active ports Finally it is realized thatthe power wasted because the SC is active is constant notaffecting the optimization results Besides we express per-NSdelay in (7) as the sum of per-VNF delays and the per-VNFdelay in (8) as the sum of the processing delays introducedby the execution of the VNF instance plus the path delayintroduced by the traffic going from one VNF to the next onein the chain defined in the correspondent NS
On the one hand the processing delay in the CPU growswith the supported user traffic load On the other hand weconsider that the pathlink delay is composed of several linkdelays fromtoVMs tofrom the switches plus the delay in theswitches Hence depending on the adopted placement policy120587 each NS will follow a different path through the networkIn such a way if the VNFs belonging to a NS are placed in thesame SC the service chain traverses virtualinternal links andthe virtual switch of the selected SC whereas if the VNFs of
6 Mobile Information Systems
a NS are located in different SCs external links and the physi-cal switch is also used therefore generally 11988911989711990911199101 11990921199102119894119895 =11988911990911199101vs + 119889vs
1199091+ 119889vs1199091 119904 + 119889119904 + 119889119904vs1199092 + 119889vs
1199092+ 119889vs11990921199102 but for
a given VNF if the next VNF in its chain holds in the sameSC its link delay is only affected because of passing throughthe virtual switch Note that in case a VNF is the last VNF inthe chain the link delay is null with the sums referring to thelink delay for that 119895 being null
In order to guarantee that each VM holds only a singleVNF constraint (9) must be satisfied Nevertheless weconsider that each VNF of a NS can be instantiated as a VMthat is executed over several cores of the same SC Apart fromthat as previously detailed in Section 3 the first VNF and thelast VNF in a service chain are the same (the SCVNF) in orderto perform the specific functions that allow the split betweencontrol and data plane required in 5G communicationsTherefore it implies that 1198601199091199101198941 = 119860119909119869119894119894119895
Finally the model must also include the constraintsreferred to as the available resources In this sense (3) statesthat the consumption of the resource 119903 due to the placementof VNFs in the microserver of the SC 119909119898120587119903119909 must not exceedthe total amount of resource 119872119903119909 where 119898120587119903119909 is defined in(10) Similarly (4) constrains the traffic through the networklinks 119905119897120587119909 to the maximum capacity of the links BW119909 as thesum of the traffic flows of the network services that must beforwarded from a VNF in one SC to the next VNF in anotherSC
42 Robust Constraint-Based Solution As described inSection 2 many existing placement works assume perfectknowledge on input parameters pointing thus to the for-mulation of a deterministic optimization problem Unfortu-nately in the real world the estimated values of the inputparameters may differ due to biased data unrealistic assump-tions or numerical errors affecting the obtained optimalsolution and its performance This potential deviation ofthe nominal or expected input may lead to the violation ofthe problem constraints and therefore make the obtainedsolution suboptimal or even unfeasible
The uncertainty of modelling parameters has been tra-ditionally handled through stochastic programming andsensitivity analysis but robust optimization techniques haverecently arisen as a powerful tool to manage the impact ofuncertain input sets RO seeks an uncertainty-immunizedsolution with an acceptable performance under the real-ization of the uncertain inputs becoming a conservativeworst-case oriented methodology The main tools of RO areuncertainty sets and a robust counterpart problem [31] Theuncertainty of the input data is described later in this sectionwhile the robust counterpart problem is the deterministicmodel previously detailed in Section 41
Here we propose a robust constraint-based placementsolution that deals with service demand uncertainty Withthis RO approach we consider a tradeoff between theproblem optimization and the protection from deviationscaused by input parameters uncertainty This uncertainty inservice demand has an impact on several components on theformulation previously presented in Section 41 On the one
hand we consider that a VM which is running a specificVNF requires an expected use of CPU and thus a certainconstant power consumption due to CPU Neverthelessthe uncertainty in traffic demand may cause uncertainty inthe CPU requirementuse and consequently in its energyconsumption and processing delays On the other hand bothenergy consumption and delays in switches and links dependon that demand uncertainty as well
In order to obtain a proposal for the placement problemwith uncertain service demand we employ the Soyster [10]method used in the framework of RO That technique isalso known as the worst-case approach where the solutionsare achieved using the most extreme expected values of theuncertain variables Therefore that kind of resolution guar-antees feasible solutions for any value of the uncertain servicedemand
In reference to the modelling of the uncertain trafficdemand per NS 119894 we consider an expected traffic demand119864[119879119894] = 119879119894 plus a term due to uncertainty in the servicedemandUTmax119894 We assume that the randomvariableUTmax119894is symmetrically distributed within [minusΔ119879119894 +Δ119879119894] sdot 119879119894 andwith mean zero Furthermore the sum of deviations of theuncertain service demand should not exceed the worst-casemaximum value 119879max119894 fulfilling (12) In this way the valueof 119879max119894 controls the tradeoff between the robustness andthe impact on the optimization higher 119879max119894 leads to worseobjective function values but protects from more parameterdeviations
119880119879119894Δ119879119894
le 119879max119894 forall119894 (12)
5 Analysis of the Results
This section presents the results obtained with the simulationof the proposed robust constraint-based expert system for theplacement of VNF chains in an emerging 5G distributed edgecloud As previously introduced in Section 2 the mathemat-ical VNF placement problemmodel presented in Section 4 isa combinatorial or discrete optimization problem equivalentto the well-known knapsack problem This means that thedecision problem is NP-complete while the optimizationproblem is NP-hard [11] The proposed optimization modelhas been solved using constraint programming as a powerfulparadigm for solving combinatorial search problems [32]
We usedMinizinc [33] a free and open-source constraintmodelling language to model the optimization and con-straint satisfaction VNF placement problem in a high-levelsolver-independent wayThen Minizinc compiles this modelinto FlatZinc a solver input language that is understoodby a wide range of solvers In our case Gecode solver [34]provided the best performance in the search of the optimalsolution
The experiments that are described and discussed belowhave been conducted in a single computer with a 27GHzIntel Core i5 processor and 16GB RAM This simple con-figuration provides solutions within minutes for simpleexperiments with loose constraint satisfaction When theexperiments demand tighter latency or resource constraints
Mobile Information Systems 7
Virt
ual i
nfra
struc
ture
Corenetwork
Figure 4 Evaluation scenario
the solving time increases to hours depending on the specificconfiguration of the experiment
Next we first describe the analysis scenario and the con-figuration of the performed experiments to later discuss theresults of those experimentsThe objective of the experimentsis to demonstrate that the placement mechanism is able toallocate multitenant service chains to the available resourcesconsidering latency constraints For this reason the numeri-cal values employed in the next section are just experimentaland do not respond to real world measurements
51 Description of the Analysis Scenario Our evaluationscenario is depicted in Figure 4 We consider a 10-SC deploy-ment for example to cover a football stadium for periodicalflash eventsThe SCs are connected in a star topology throughan Ethernet switch Each SC is composed of 8 cores 16GBRAM 100GB of storage and 1HWA (eg GPU)
We also consider 2 NSs that can be offered at 3 serviceflavours (bronze silver and gold) that determine the latencyconstraint for eachNS and the nominal aggregate user bit rateserved as shown in Figure 5
Table 1 shows the experimental energy model and Table 2shows the modelling of the VNFs that compose the proposedNSs for the user traffic demand that will be employed in theevaluation experiments The user traffic values include thenominal bit rates of services NS1 and NS2 (130Mbps and90Mbps resp) as well as the values when a 20 protectionparameter is applied to the nominal values (156Mbps and108Mbps resp) in order to study the effect of robust optimi-zation techniques on the placement decision
As a final remark the placement behavior of VNFs canbe altered by the use of HWA The assignment of a HWA toa VNF implies that the service latency of the correspondentinstance is reduced 3 times at a cost of incrementing thepower consumption by 10 It is important to note that whenthe VNF instance needs more than 1 CPU for executingwithout the assistance of a HWA then if the placementalgorithm assigns it to a HWA the VNF will need a singlecore
52 Discussion of the Placement Results As stated before theevaluation scenario includes 3 tenants that hire a combination
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
4 Mobile Information Systems
traffic demand deviation and will be used to overestimate theallocation of virtualized resources in order to be prepared foreventual user peak demands
Next the placement algorithm inspects the VNF catalogfor the parametrization of theVNF instances needed tomatchthe SLA and checks the available resources provided throughthe virtualized infrastructure VNFs are characterized by theuse of resources (ie number of cores RAM and storage)according to the aggregated bit rate (ie characterized aggre-gated traffic flows of a NS) served by the NS they belongtoMoreover the service latency of eachVNF also depends onthe traffic load supported by the VNF instance Lastly virtu-alized resources may also include other hardware appliancessuch as hardware accelerators (HWA) that can improve theperformance of a VNF in terms of latency This way theseadditional appliances help matching SLA with the tenant butat a higher energy cost
All the VNFs composing the service chains of therequested network services must be allocated in the availablevirtualized resources including a given number of CPUshardware appliances RAM and storage space and networkbandwidth We assume that each core runs a single VM thatis devoted to the execution of a single VNF instance but oneVNF instance can scale from one to many VMs upon the ser-vice requirements of theNS Besides the transmission latencyalong the network topology between two VNFs allocatedin different microservers depends on the traffic load of theinvolved links and the size of the flow to be transmittedWith this information the placement algorithm assigns eachVNF instance to one or more VMs with the objective ofminimizing the energy consumption of the whole systemwhile matching the latency constraints for the NS
Finally the Infrastructure Manager functional elementVirtualized Infrastructure Manager (VIM) in ETSI-MANOterminology receives the outcome of the placement decisionand maps each VNF to the assigned resources
It is important to point out that a VNF chain includesradio-related and service-related instances The placementalgorithm introduced in this work considers a scenario withseveral tenants each of them requesting one or more NSswhich share the available resources Considering the afore-mentioned communication particularities of 5G Figure 3shows an example of a simple complete placement result ofa multitenant environment
This example considers a distributed edge cloud shared bytwo tenants Tenant 1 offers two network services the first onechaining a virtualized firewall (vFW) an intrusion detectionfunction (vID) a watermarking process (vWM) and finally acaching service (vCache) The second NS of Tenant 1 chainsthe vFW and a video analytics (VA) functionality for onlinesurveillance Tenant 2 offers a single network service thanchains the vFW a vCache service and a virtual transcodingunit (vTU) All NSs are associated with their correspondentlatency constraint
In the example both NSs belonging to the same tenantshare the radio VNF (small cell VNF or SCVNF) In this waythe common radio operations are performed in a coherentway between the different edge services of a tenant and thedata paths of the specific service instances per tenant are split
SCVNF SCVNF
vFW
VA
vFW
vWM
vCachevFW
vCache
vTU
HW
Acc
CPU
s
vID
NS2
NS1
NS1
NS1
NS1
NS1
NS1
NS1 N
S3NS3
NS1
NS2
Network topology
vFW vCache vTU
vFW VA
vFW vID vWM vCache ltlat1
ltlat2
ltlat3
Tena
nt 1
SCVNF
SCVNF
Tena
nt 2
NS1
NS2
NS3
vIDlowast
Figure 3 Multitenant placement example lowast indicates that ldquovIDrdquouses a hardware acceleration appliance
The SCVNF of each tenant will be shared by all theNSs of thistenant which means that the use of resources of the SCVNFmust then consider the sumof the aggregated flows of theNSsof the same tenant
The distinctive features of 5G communications have twoimportant implications over the placement algorithm Firstlyevery tenant allocates a unique SCVNF instance at the edgecloud in order to support the control plane and data planeoperations of all the SCs associated with that tenant Andsecondly since the SCVNFmanages the uplink and downlinktraffic between themobile users and the core network it leadsto circular forwarding paths when including edge serviceinstances In this paper we consider that the core networkfunctionalities of themobile network operator are outside theCE-RAN which means that the CE-RAN only deploys radioand edge service instances In order to keep the data pathbetween the mobile users and the core network functions alltheNSs need to start and end at the SCVNFwhichwill handlethe user plane traffic tofrom the core network
The example also shows how some VNFs needmore thanoneVM to execute the associated functionalities and howoneVNF uses a HWA tomeet the latency constraint of the server
Section 5 will show the result of more complex and com-plete placement experiments For the sake of generality thispaper does not focus on the specific modelling of the trafficpatterns the energy consumption schemes or the VNF net-work service performance Section 5 provides some referenceoutcomes on the potential benefits of the introduced VNFplacement algorithm in the considered network scenario but
Mobile Information Systems 5
the applicability of the solution to other scenarios shouldbe fed by experimental modelling data specific to the targetscenario
4 Robust Constraint-Based VNFPlacement Proposal
Once the 5G VNF placement problem under study is pre-sented in this section we propose a placement solution tothat problem using the RO approach To this end first weformulate the energy-aware VNF placement model in thenext subsection Then we provide our RO-based placementproposal in Section 42
41 ProblemModelling As stated before the VNF placementproblem addressed in this work focuses on the assignment ofthe VNFs composing each NS to the provided 5G networkinfrastructure Therefore it constitutes a discrete or com-binatorial optimization problem where the minimizationof energy consumption constitutes the optimality criterionSummary of Model Indexes Sets Parameters and DecisionVariables reports and summarizes the mathematical nota-tions used for the model indexes sets input parameters anddecision variables of VNF placement problem that is definedin the following
The Problem Model The energy-aware VNF placement opti-mization problem
min120587isinΠ
119875120587 (1)
st 119889120587119894 lt 119863max119894 forall119894 isin I 120587 isin Π (2)
119898120587119903119909 lt 119872119903119909 forall119903 isin R 119909 isin X 120587 isin Π (3)
119905119897120587119909 lt BW119909 forall119909 isin X 120587 isin Π (4)
119875120587 = sum119894isinI
sum119895isinJ119894
119875120587119894119895 (5)
119875120587119894119895
= sum119909isinX
sum119910isinY119909
119875cpu119909119910119894119895 119860119909119910
119894119895 + sum119894isinI
sum119895isinJ119894
119875switch119894119895119871 119894119895
+ sum119909isinX
119875sc119909119860119909119910
119894119895
(6)
119889120587119894 = sum119895isinJ119894
119889120587119894119895 (7)
119889120587119894119895
= sum119909isinX
sum119910isinY119909
119889119901119909119910119894119895 119860119909119910
119894119895
+ sum1199091isinX
sum1199101isinY1199091
sum1199092isinX
sum1199102isinY1199092
11988911989711990911199101 11990921199102119894119895 11986011990911199101119894119895 11986011990921199102119894119895+1
(8)
sum119894isinI
sum119895isinJ119894
119860119909119910119894119895 = 1 forall119909 isin X 119910isin Y119909 (9)
119898120587119903119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119898119903119894119895119860119909119910
119894119895 (10)
119905119897120587119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119879119894119860119909119910
119894119895 1198601199091015840119910
119894119895+1 (11)
We consider a set X of small cells that compose the CE-RAN and a set Y119909 of cores included in the microserver ofthe SC 119909 Additionally we also consider a set of I networkservices offered by the CE-RAN operator to the differenttenants and a set J119894 of VNFs that form the functionalchain of the NS 119894 Finally R represents the set of differentresources available in the microservers for the execution ofthe VNFs (storage RAM hardware accelerators etc) Henceour constraint-based expert systemmust determine in whichVM 119910 of a SC 119909 each VNF 119895 of a NS 119894 is located
Let Π denote the set of all possible power-aware andlatency- and resource-constrained VNF placement policieswhich decide the mapping of each VNF of available NSs toavailable VMs in the provided SCs We formulate our VNFplacement optimization problem aimed atminimizing powerconsumption with delay requirements as (1)
In reference to the energy part we define the total powerconsumption 119875120587 as the sum of the energy demand of all theallocated VNFs 119875120587119894119895 (5) Then for each VNF instance thepower consumption is expressed in (6) as the sum of thepower consumption of VNF 119895 of NS 119894 due to the CPU useof VM 119910 of SC 119909 the power consumption of the physicalswitch due to the forwarded traffic of the VNF and the powerconsumption of SC 119909 because of being switched on
The power consumption of the cores that run the VMsdepends of the aggregated user traffic served by the corre-spondent VNF and this relationship may be nonlinear Itmust be also understood that when a VNF instance scales toa higher number of cores the individual load of the involvedCPUs decreases leading to a lower power consumption percore Additionally the energy consumption of the networkingdevices (eg an Ethernet switch) depends on the aggregatedflow that traverses the device to forward data packets betweenSCs and the number of active ports Finally it is realized thatthe power wasted because the SC is active is constant notaffecting the optimization results Besides we express per-NSdelay in (7) as the sum of per-VNF delays and the per-VNFdelay in (8) as the sum of the processing delays introducedby the execution of the VNF instance plus the path delayintroduced by the traffic going from one VNF to the next onein the chain defined in the correspondent NS
On the one hand the processing delay in the CPU growswith the supported user traffic load On the other hand weconsider that the pathlink delay is composed of several linkdelays fromtoVMs tofrom the switches plus the delay in theswitches Hence depending on the adopted placement policy120587 each NS will follow a different path through the networkIn such a way if the VNFs belonging to a NS are placed in thesame SC the service chain traverses virtualinternal links andthe virtual switch of the selected SC whereas if the VNFs of
6 Mobile Information Systems
a NS are located in different SCs external links and the physi-cal switch is also used therefore generally 11988911989711990911199101 11990921199102119894119895 =11988911990911199101vs + 119889vs
1199091+ 119889vs1199091 119904 + 119889119904 + 119889119904vs1199092 + 119889vs
1199092+ 119889vs11990921199102 but for
a given VNF if the next VNF in its chain holds in the sameSC its link delay is only affected because of passing throughthe virtual switch Note that in case a VNF is the last VNF inthe chain the link delay is null with the sums referring to thelink delay for that 119895 being null
In order to guarantee that each VM holds only a singleVNF constraint (9) must be satisfied Nevertheless weconsider that each VNF of a NS can be instantiated as a VMthat is executed over several cores of the same SC Apart fromthat as previously detailed in Section 3 the first VNF and thelast VNF in a service chain are the same (the SCVNF) in orderto perform the specific functions that allow the split betweencontrol and data plane required in 5G communicationsTherefore it implies that 1198601199091199101198941 = 119860119909119869119894119894119895
Finally the model must also include the constraintsreferred to as the available resources In this sense (3) statesthat the consumption of the resource 119903 due to the placementof VNFs in the microserver of the SC 119909119898120587119903119909 must not exceedthe total amount of resource 119872119903119909 where 119898120587119903119909 is defined in(10) Similarly (4) constrains the traffic through the networklinks 119905119897120587119909 to the maximum capacity of the links BW119909 as thesum of the traffic flows of the network services that must beforwarded from a VNF in one SC to the next VNF in anotherSC
42 Robust Constraint-Based Solution As described inSection 2 many existing placement works assume perfectknowledge on input parameters pointing thus to the for-mulation of a deterministic optimization problem Unfortu-nately in the real world the estimated values of the inputparameters may differ due to biased data unrealistic assump-tions or numerical errors affecting the obtained optimalsolution and its performance This potential deviation ofthe nominal or expected input may lead to the violation ofthe problem constraints and therefore make the obtainedsolution suboptimal or even unfeasible
The uncertainty of modelling parameters has been tra-ditionally handled through stochastic programming andsensitivity analysis but robust optimization techniques haverecently arisen as a powerful tool to manage the impact ofuncertain input sets RO seeks an uncertainty-immunizedsolution with an acceptable performance under the real-ization of the uncertain inputs becoming a conservativeworst-case oriented methodology The main tools of RO areuncertainty sets and a robust counterpart problem [31] Theuncertainty of the input data is described later in this sectionwhile the robust counterpart problem is the deterministicmodel previously detailed in Section 41
Here we propose a robust constraint-based placementsolution that deals with service demand uncertainty Withthis RO approach we consider a tradeoff between theproblem optimization and the protection from deviationscaused by input parameters uncertainty This uncertainty inservice demand has an impact on several components on theformulation previously presented in Section 41 On the one
hand we consider that a VM which is running a specificVNF requires an expected use of CPU and thus a certainconstant power consumption due to CPU Neverthelessthe uncertainty in traffic demand may cause uncertainty inthe CPU requirementuse and consequently in its energyconsumption and processing delays On the other hand bothenergy consumption and delays in switches and links dependon that demand uncertainty as well
In order to obtain a proposal for the placement problemwith uncertain service demand we employ the Soyster [10]method used in the framework of RO That technique isalso known as the worst-case approach where the solutionsare achieved using the most extreme expected values of theuncertain variables Therefore that kind of resolution guar-antees feasible solutions for any value of the uncertain servicedemand
In reference to the modelling of the uncertain trafficdemand per NS 119894 we consider an expected traffic demand119864[119879119894] = 119879119894 plus a term due to uncertainty in the servicedemandUTmax119894 We assume that the randomvariableUTmax119894is symmetrically distributed within [minusΔ119879119894 +Δ119879119894] sdot 119879119894 andwith mean zero Furthermore the sum of deviations of theuncertain service demand should not exceed the worst-casemaximum value 119879max119894 fulfilling (12) In this way the valueof 119879max119894 controls the tradeoff between the robustness andthe impact on the optimization higher 119879max119894 leads to worseobjective function values but protects from more parameterdeviations
119880119879119894Δ119879119894
le 119879max119894 forall119894 (12)
5 Analysis of the Results
This section presents the results obtained with the simulationof the proposed robust constraint-based expert system for theplacement of VNF chains in an emerging 5G distributed edgecloud As previously introduced in Section 2 the mathemat-ical VNF placement problemmodel presented in Section 4 isa combinatorial or discrete optimization problem equivalentto the well-known knapsack problem This means that thedecision problem is NP-complete while the optimizationproblem is NP-hard [11] The proposed optimization modelhas been solved using constraint programming as a powerfulparadigm for solving combinatorial search problems [32]
We usedMinizinc [33] a free and open-source constraintmodelling language to model the optimization and con-straint satisfaction VNF placement problem in a high-levelsolver-independent wayThen Minizinc compiles this modelinto FlatZinc a solver input language that is understoodby a wide range of solvers In our case Gecode solver [34]provided the best performance in the search of the optimalsolution
The experiments that are described and discussed belowhave been conducted in a single computer with a 27GHzIntel Core i5 processor and 16GB RAM This simple con-figuration provides solutions within minutes for simpleexperiments with loose constraint satisfaction When theexperiments demand tighter latency or resource constraints
Mobile Information Systems 7
Virt
ual i
nfra
struc
ture
Corenetwork
Figure 4 Evaluation scenario
the solving time increases to hours depending on the specificconfiguration of the experiment
Next we first describe the analysis scenario and the con-figuration of the performed experiments to later discuss theresults of those experimentsThe objective of the experimentsis to demonstrate that the placement mechanism is able toallocate multitenant service chains to the available resourcesconsidering latency constraints For this reason the numeri-cal values employed in the next section are just experimentaland do not respond to real world measurements
51 Description of the Analysis Scenario Our evaluationscenario is depicted in Figure 4 We consider a 10-SC deploy-ment for example to cover a football stadium for periodicalflash eventsThe SCs are connected in a star topology throughan Ethernet switch Each SC is composed of 8 cores 16GBRAM 100GB of storage and 1HWA (eg GPU)
We also consider 2 NSs that can be offered at 3 serviceflavours (bronze silver and gold) that determine the latencyconstraint for eachNS and the nominal aggregate user bit rateserved as shown in Figure 5
Table 1 shows the experimental energy model and Table 2shows the modelling of the VNFs that compose the proposedNSs for the user traffic demand that will be employed in theevaluation experiments The user traffic values include thenominal bit rates of services NS1 and NS2 (130Mbps and90Mbps resp) as well as the values when a 20 protectionparameter is applied to the nominal values (156Mbps and108Mbps resp) in order to study the effect of robust optimi-zation techniques on the placement decision
As a final remark the placement behavior of VNFs canbe altered by the use of HWA The assignment of a HWA toa VNF implies that the service latency of the correspondentinstance is reduced 3 times at a cost of incrementing thepower consumption by 10 It is important to note that whenthe VNF instance needs more than 1 CPU for executingwithout the assistance of a HWA then if the placementalgorithm assigns it to a HWA the VNF will need a singlecore
52 Discussion of the Placement Results As stated before theevaluation scenario includes 3 tenants that hire a combination
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mobile Information Systems 5
the applicability of the solution to other scenarios shouldbe fed by experimental modelling data specific to the targetscenario
4 Robust Constraint-Based VNFPlacement Proposal
Once the 5G VNF placement problem under study is pre-sented in this section we propose a placement solution tothat problem using the RO approach To this end first weformulate the energy-aware VNF placement model in thenext subsection Then we provide our RO-based placementproposal in Section 42
41 ProblemModelling As stated before the VNF placementproblem addressed in this work focuses on the assignment ofthe VNFs composing each NS to the provided 5G networkinfrastructure Therefore it constitutes a discrete or com-binatorial optimization problem where the minimizationof energy consumption constitutes the optimality criterionSummary of Model Indexes Sets Parameters and DecisionVariables reports and summarizes the mathematical nota-tions used for the model indexes sets input parameters anddecision variables of VNF placement problem that is definedin the following
The Problem Model The energy-aware VNF placement opti-mization problem
min120587isinΠ
119875120587 (1)
st 119889120587119894 lt 119863max119894 forall119894 isin I 120587 isin Π (2)
119898120587119903119909 lt 119872119903119909 forall119903 isin R 119909 isin X 120587 isin Π (3)
119905119897120587119909 lt BW119909 forall119909 isin X 120587 isin Π (4)
119875120587 = sum119894isinI
sum119895isinJ119894
119875120587119894119895 (5)
119875120587119894119895
= sum119909isinX
sum119910isinY119909
119875cpu119909119910119894119895 119860119909119910
119894119895 + sum119894isinI
sum119895isinJ119894
119875switch119894119895119871 119894119895
+ sum119909isinX
119875sc119909119860119909119910
119894119895
(6)
119889120587119894 = sum119895isinJ119894
119889120587119894119895 (7)
119889120587119894119895
= sum119909isinX
sum119910isinY119909
119889119901119909119910119894119895 119860119909119910
119894119895
+ sum1199091isinX
sum1199101isinY1199091
sum1199092isinX
sum1199102isinY1199092
11988911989711990911199101 11990921199102119894119895 11986011990911199101119894119895 11986011990921199102119894119895+1
(8)
sum119894isinI
sum119895isinJ119894
119860119909119910119894119895 = 1 forall119909 isin X 119910isin Y119909 (9)
119898120587119903119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119898119903119894119895119860119909119910
119894119895 (10)
119905119897120587119909 = sum119894isinI
sum119895isinJ119894
sum119910isinY
119879119894119860119909119910
119894119895 1198601199091015840119910
119894119895+1 (11)
We consider a set X of small cells that compose the CE-RAN and a set Y119909 of cores included in the microserver ofthe SC 119909 Additionally we also consider a set of I networkservices offered by the CE-RAN operator to the differenttenants and a set J119894 of VNFs that form the functionalchain of the NS 119894 Finally R represents the set of differentresources available in the microservers for the execution ofthe VNFs (storage RAM hardware accelerators etc) Henceour constraint-based expert systemmust determine in whichVM 119910 of a SC 119909 each VNF 119895 of a NS 119894 is located
Let Π denote the set of all possible power-aware andlatency- and resource-constrained VNF placement policieswhich decide the mapping of each VNF of available NSs toavailable VMs in the provided SCs We formulate our VNFplacement optimization problem aimed atminimizing powerconsumption with delay requirements as (1)
In reference to the energy part we define the total powerconsumption 119875120587 as the sum of the energy demand of all theallocated VNFs 119875120587119894119895 (5) Then for each VNF instance thepower consumption is expressed in (6) as the sum of thepower consumption of VNF 119895 of NS 119894 due to the CPU useof VM 119910 of SC 119909 the power consumption of the physicalswitch due to the forwarded traffic of the VNF and the powerconsumption of SC 119909 because of being switched on
The power consumption of the cores that run the VMsdepends of the aggregated user traffic served by the corre-spondent VNF and this relationship may be nonlinear Itmust be also understood that when a VNF instance scales toa higher number of cores the individual load of the involvedCPUs decreases leading to a lower power consumption percore Additionally the energy consumption of the networkingdevices (eg an Ethernet switch) depends on the aggregatedflow that traverses the device to forward data packets betweenSCs and the number of active ports Finally it is realized thatthe power wasted because the SC is active is constant notaffecting the optimization results Besides we express per-NSdelay in (7) as the sum of per-VNF delays and the per-VNFdelay in (8) as the sum of the processing delays introducedby the execution of the VNF instance plus the path delayintroduced by the traffic going from one VNF to the next onein the chain defined in the correspondent NS
On the one hand the processing delay in the CPU growswith the supported user traffic load On the other hand weconsider that the pathlink delay is composed of several linkdelays fromtoVMs tofrom the switches plus the delay in theswitches Hence depending on the adopted placement policy120587 each NS will follow a different path through the networkIn such a way if the VNFs belonging to a NS are placed in thesame SC the service chain traverses virtualinternal links andthe virtual switch of the selected SC whereas if the VNFs of
6 Mobile Information Systems
a NS are located in different SCs external links and the physi-cal switch is also used therefore generally 11988911989711990911199101 11990921199102119894119895 =11988911990911199101vs + 119889vs
1199091+ 119889vs1199091 119904 + 119889119904 + 119889119904vs1199092 + 119889vs
1199092+ 119889vs11990921199102 but for
a given VNF if the next VNF in its chain holds in the sameSC its link delay is only affected because of passing throughthe virtual switch Note that in case a VNF is the last VNF inthe chain the link delay is null with the sums referring to thelink delay for that 119895 being null
In order to guarantee that each VM holds only a singleVNF constraint (9) must be satisfied Nevertheless weconsider that each VNF of a NS can be instantiated as a VMthat is executed over several cores of the same SC Apart fromthat as previously detailed in Section 3 the first VNF and thelast VNF in a service chain are the same (the SCVNF) in orderto perform the specific functions that allow the split betweencontrol and data plane required in 5G communicationsTherefore it implies that 1198601199091199101198941 = 119860119909119869119894119894119895
Finally the model must also include the constraintsreferred to as the available resources In this sense (3) statesthat the consumption of the resource 119903 due to the placementof VNFs in the microserver of the SC 119909119898120587119903119909 must not exceedthe total amount of resource 119872119903119909 where 119898120587119903119909 is defined in(10) Similarly (4) constrains the traffic through the networklinks 119905119897120587119909 to the maximum capacity of the links BW119909 as thesum of the traffic flows of the network services that must beforwarded from a VNF in one SC to the next VNF in anotherSC
42 Robust Constraint-Based Solution As described inSection 2 many existing placement works assume perfectknowledge on input parameters pointing thus to the for-mulation of a deterministic optimization problem Unfortu-nately in the real world the estimated values of the inputparameters may differ due to biased data unrealistic assump-tions or numerical errors affecting the obtained optimalsolution and its performance This potential deviation ofthe nominal or expected input may lead to the violation ofthe problem constraints and therefore make the obtainedsolution suboptimal or even unfeasible
The uncertainty of modelling parameters has been tra-ditionally handled through stochastic programming andsensitivity analysis but robust optimization techniques haverecently arisen as a powerful tool to manage the impact ofuncertain input sets RO seeks an uncertainty-immunizedsolution with an acceptable performance under the real-ization of the uncertain inputs becoming a conservativeworst-case oriented methodology The main tools of RO areuncertainty sets and a robust counterpart problem [31] Theuncertainty of the input data is described later in this sectionwhile the robust counterpart problem is the deterministicmodel previously detailed in Section 41
Here we propose a robust constraint-based placementsolution that deals with service demand uncertainty Withthis RO approach we consider a tradeoff between theproblem optimization and the protection from deviationscaused by input parameters uncertainty This uncertainty inservice demand has an impact on several components on theformulation previously presented in Section 41 On the one
hand we consider that a VM which is running a specificVNF requires an expected use of CPU and thus a certainconstant power consumption due to CPU Neverthelessthe uncertainty in traffic demand may cause uncertainty inthe CPU requirementuse and consequently in its energyconsumption and processing delays On the other hand bothenergy consumption and delays in switches and links dependon that demand uncertainty as well
In order to obtain a proposal for the placement problemwith uncertain service demand we employ the Soyster [10]method used in the framework of RO That technique isalso known as the worst-case approach where the solutionsare achieved using the most extreme expected values of theuncertain variables Therefore that kind of resolution guar-antees feasible solutions for any value of the uncertain servicedemand
In reference to the modelling of the uncertain trafficdemand per NS 119894 we consider an expected traffic demand119864[119879119894] = 119879119894 plus a term due to uncertainty in the servicedemandUTmax119894 We assume that the randomvariableUTmax119894is symmetrically distributed within [minusΔ119879119894 +Δ119879119894] sdot 119879119894 andwith mean zero Furthermore the sum of deviations of theuncertain service demand should not exceed the worst-casemaximum value 119879max119894 fulfilling (12) In this way the valueof 119879max119894 controls the tradeoff between the robustness andthe impact on the optimization higher 119879max119894 leads to worseobjective function values but protects from more parameterdeviations
119880119879119894Δ119879119894
le 119879max119894 forall119894 (12)
5 Analysis of the Results
This section presents the results obtained with the simulationof the proposed robust constraint-based expert system for theplacement of VNF chains in an emerging 5G distributed edgecloud As previously introduced in Section 2 the mathemat-ical VNF placement problemmodel presented in Section 4 isa combinatorial or discrete optimization problem equivalentto the well-known knapsack problem This means that thedecision problem is NP-complete while the optimizationproblem is NP-hard [11] The proposed optimization modelhas been solved using constraint programming as a powerfulparadigm for solving combinatorial search problems [32]
We usedMinizinc [33] a free and open-source constraintmodelling language to model the optimization and con-straint satisfaction VNF placement problem in a high-levelsolver-independent wayThen Minizinc compiles this modelinto FlatZinc a solver input language that is understoodby a wide range of solvers In our case Gecode solver [34]provided the best performance in the search of the optimalsolution
The experiments that are described and discussed belowhave been conducted in a single computer with a 27GHzIntel Core i5 processor and 16GB RAM This simple con-figuration provides solutions within minutes for simpleexperiments with loose constraint satisfaction When theexperiments demand tighter latency or resource constraints
Mobile Information Systems 7
Virt
ual i
nfra
struc
ture
Corenetwork
Figure 4 Evaluation scenario
the solving time increases to hours depending on the specificconfiguration of the experiment
Next we first describe the analysis scenario and the con-figuration of the performed experiments to later discuss theresults of those experimentsThe objective of the experimentsis to demonstrate that the placement mechanism is able toallocate multitenant service chains to the available resourcesconsidering latency constraints For this reason the numeri-cal values employed in the next section are just experimentaland do not respond to real world measurements
51 Description of the Analysis Scenario Our evaluationscenario is depicted in Figure 4 We consider a 10-SC deploy-ment for example to cover a football stadium for periodicalflash eventsThe SCs are connected in a star topology throughan Ethernet switch Each SC is composed of 8 cores 16GBRAM 100GB of storage and 1HWA (eg GPU)
We also consider 2 NSs that can be offered at 3 serviceflavours (bronze silver and gold) that determine the latencyconstraint for eachNS and the nominal aggregate user bit rateserved as shown in Figure 5
Table 1 shows the experimental energy model and Table 2shows the modelling of the VNFs that compose the proposedNSs for the user traffic demand that will be employed in theevaluation experiments The user traffic values include thenominal bit rates of services NS1 and NS2 (130Mbps and90Mbps resp) as well as the values when a 20 protectionparameter is applied to the nominal values (156Mbps and108Mbps resp) in order to study the effect of robust optimi-zation techniques on the placement decision
As a final remark the placement behavior of VNFs canbe altered by the use of HWA The assignment of a HWA toa VNF implies that the service latency of the correspondentinstance is reduced 3 times at a cost of incrementing thepower consumption by 10 It is important to note that whenthe VNF instance needs more than 1 CPU for executingwithout the assistance of a HWA then if the placementalgorithm assigns it to a HWA the VNF will need a singlecore
52 Discussion of the Placement Results As stated before theevaluation scenario includes 3 tenants that hire a combination
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
6 Mobile Information Systems
a NS are located in different SCs external links and the physi-cal switch is also used therefore generally 11988911989711990911199101 11990921199102119894119895 =11988911990911199101vs + 119889vs
1199091+ 119889vs1199091 119904 + 119889119904 + 119889119904vs1199092 + 119889vs
1199092+ 119889vs11990921199102 but for
a given VNF if the next VNF in its chain holds in the sameSC its link delay is only affected because of passing throughthe virtual switch Note that in case a VNF is the last VNF inthe chain the link delay is null with the sums referring to thelink delay for that 119895 being null
In order to guarantee that each VM holds only a singleVNF constraint (9) must be satisfied Nevertheless weconsider that each VNF of a NS can be instantiated as a VMthat is executed over several cores of the same SC Apart fromthat as previously detailed in Section 3 the first VNF and thelast VNF in a service chain are the same (the SCVNF) in orderto perform the specific functions that allow the split betweencontrol and data plane required in 5G communicationsTherefore it implies that 1198601199091199101198941 = 119860119909119869119894119894119895
Finally the model must also include the constraintsreferred to as the available resources In this sense (3) statesthat the consumption of the resource 119903 due to the placementof VNFs in the microserver of the SC 119909119898120587119903119909 must not exceedthe total amount of resource 119872119903119909 where 119898120587119903119909 is defined in(10) Similarly (4) constrains the traffic through the networklinks 119905119897120587119909 to the maximum capacity of the links BW119909 as thesum of the traffic flows of the network services that must beforwarded from a VNF in one SC to the next VNF in anotherSC
42 Robust Constraint-Based Solution As described inSection 2 many existing placement works assume perfectknowledge on input parameters pointing thus to the for-mulation of a deterministic optimization problem Unfortu-nately in the real world the estimated values of the inputparameters may differ due to biased data unrealistic assump-tions or numerical errors affecting the obtained optimalsolution and its performance This potential deviation ofthe nominal or expected input may lead to the violation ofthe problem constraints and therefore make the obtainedsolution suboptimal or even unfeasible
The uncertainty of modelling parameters has been tra-ditionally handled through stochastic programming andsensitivity analysis but robust optimization techniques haverecently arisen as a powerful tool to manage the impact ofuncertain input sets RO seeks an uncertainty-immunizedsolution with an acceptable performance under the real-ization of the uncertain inputs becoming a conservativeworst-case oriented methodology The main tools of RO areuncertainty sets and a robust counterpart problem [31] Theuncertainty of the input data is described later in this sectionwhile the robust counterpart problem is the deterministicmodel previously detailed in Section 41
Here we propose a robust constraint-based placementsolution that deals with service demand uncertainty Withthis RO approach we consider a tradeoff between theproblem optimization and the protection from deviationscaused by input parameters uncertainty This uncertainty inservice demand has an impact on several components on theformulation previously presented in Section 41 On the one
hand we consider that a VM which is running a specificVNF requires an expected use of CPU and thus a certainconstant power consumption due to CPU Neverthelessthe uncertainty in traffic demand may cause uncertainty inthe CPU requirementuse and consequently in its energyconsumption and processing delays On the other hand bothenergy consumption and delays in switches and links dependon that demand uncertainty as well
In order to obtain a proposal for the placement problemwith uncertain service demand we employ the Soyster [10]method used in the framework of RO That technique isalso known as the worst-case approach where the solutionsare achieved using the most extreme expected values of theuncertain variables Therefore that kind of resolution guar-antees feasible solutions for any value of the uncertain servicedemand
In reference to the modelling of the uncertain trafficdemand per NS 119894 we consider an expected traffic demand119864[119879119894] = 119879119894 plus a term due to uncertainty in the servicedemandUTmax119894 We assume that the randomvariableUTmax119894is symmetrically distributed within [minusΔ119879119894 +Δ119879119894] sdot 119879119894 andwith mean zero Furthermore the sum of deviations of theuncertain service demand should not exceed the worst-casemaximum value 119879max119894 fulfilling (12) In this way the valueof 119879max119894 controls the tradeoff between the robustness andthe impact on the optimization higher 119879max119894 leads to worseobjective function values but protects from more parameterdeviations
119880119879119894Δ119879119894
le 119879max119894 forall119894 (12)
5 Analysis of the Results
This section presents the results obtained with the simulationof the proposed robust constraint-based expert system for theplacement of VNF chains in an emerging 5G distributed edgecloud As previously introduced in Section 2 the mathemat-ical VNF placement problemmodel presented in Section 4 isa combinatorial or discrete optimization problem equivalentto the well-known knapsack problem This means that thedecision problem is NP-complete while the optimizationproblem is NP-hard [11] The proposed optimization modelhas been solved using constraint programming as a powerfulparadigm for solving combinatorial search problems [32]
We usedMinizinc [33] a free and open-source constraintmodelling language to model the optimization and con-straint satisfaction VNF placement problem in a high-levelsolver-independent wayThen Minizinc compiles this modelinto FlatZinc a solver input language that is understoodby a wide range of solvers In our case Gecode solver [34]provided the best performance in the search of the optimalsolution
The experiments that are described and discussed belowhave been conducted in a single computer with a 27GHzIntel Core i5 processor and 16GB RAM This simple con-figuration provides solutions within minutes for simpleexperiments with loose constraint satisfaction When theexperiments demand tighter latency or resource constraints
Mobile Information Systems 7
Virt
ual i
nfra
struc
ture
Corenetwork
Figure 4 Evaluation scenario
the solving time increases to hours depending on the specificconfiguration of the experiment
Next we first describe the analysis scenario and the con-figuration of the performed experiments to later discuss theresults of those experimentsThe objective of the experimentsis to demonstrate that the placement mechanism is able toallocate multitenant service chains to the available resourcesconsidering latency constraints For this reason the numeri-cal values employed in the next section are just experimentaland do not respond to real world measurements
51 Description of the Analysis Scenario Our evaluationscenario is depicted in Figure 4 We consider a 10-SC deploy-ment for example to cover a football stadium for periodicalflash eventsThe SCs are connected in a star topology throughan Ethernet switch Each SC is composed of 8 cores 16GBRAM 100GB of storage and 1HWA (eg GPU)
We also consider 2 NSs that can be offered at 3 serviceflavours (bronze silver and gold) that determine the latencyconstraint for eachNS and the nominal aggregate user bit rateserved as shown in Figure 5
Table 1 shows the experimental energy model and Table 2shows the modelling of the VNFs that compose the proposedNSs for the user traffic demand that will be employed in theevaluation experiments The user traffic values include thenominal bit rates of services NS1 and NS2 (130Mbps and90Mbps resp) as well as the values when a 20 protectionparameter is applied to the nominal values (156Mbps and108Mbps resp) in order to study the effect of robust optimi-zation techniques on the placement decision
As a final remark the placement behavior of VNFs canbe altered by the use of HWA The assignment of a HWA toa VNF implies that the service latency of the correspondentinstance is reduced 3 times at a cost of incrementing thepower consumption by 10 It is important to note that whenthe VNF instance needs more than 1 CPU for executingwithout the assistance of a HWA then if the placementalgorithm assigns it to a HWA the VNF will need a singlecore
52 Discussion of the Placement Results As stated before theevaluation scenario includes 3 tenants that hire a combination
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mobile Information Systems 7
Virt
ual i
nfra
struc
ture
Corenetwork
Figure 4 Evaluation scenario
the solving time increases to hours depending on the specificconfiguration of the experiment
Next we first describe the analysis scenario and the con-figuration of the performed experiments to later discuss theresults of those experimentsThe objective of the experimentsis to demonstrate that the placement mechanism is able toallocate multitenant service chains to the available resourcesconsidering latency constraints For this reason the numeri-cal values employed in the next section are just experimentaland do not respond to real world measurements
51 Description of the Analysis Scenario Our evaluationscenario is depicted in Figure 4 We consider a 10-SC deploy-ment for example to cover a football stadium for periodicalflash eventsThe SCs are connected in a star topology throughan Ethernet switch Each SC is composed of 8 cores 16GBRAM 100GB of storage and 1HWA (eg GPU)
We also consider 2 NSs that can be offered at 3 serviceflavours (bronze silver and gold) that determine the latencyconstraint for eachNS and the nominal aggregate user bit rateserved as shown in Figure 5
Table 1 shows the experimental energy model and Table 2shows the modelling of the VNFs that compose the proposedNSs for the user traffic demand that will be employed in theevaluation experiments The user traffic values include thenominal bit rates of services NS1 and NS2 (130Mbps and90Mbps resp) as well as the values when a 20 protectionparameter is applied to the nominal values (156Mbps and108Mbps resp) in order to study the effect of robust optimi-zation techniques on the placement decision
As a final remark the placement behavior of VNFs canbe altered by the use of HWA The assignment of a HWA toa VNF implies that the service latency of the correspondentinstance is reduced 3 times at a cost of incrementing thepower consumption by 10 It is important to note that whenthe VNF instance needs more than 1 CPU for executingwithout the assistance of a HWA then if the placementalgorithm assigns it to a HWA the VNF will need a singlecore
52 Discussion of the Placement Results As stated before theevaluation scenario includes 3 tenants that hire a combination
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
8 Mobile Information Systems
VNF1 VNF2 VNF3 VNF5 VNF6
VNF1 VNF2 VNF3 VNF4
NS1
NS2
Service flavourBronze Silver Gold
NS1
NS2 Max Flow = 90MbpsMax Lat = 90ms Max Lat = 80ms
Max Flow = 130-<JM
Max Lat = 150GM
Max Lat = 110GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 120GM
Max Flow = 90Mbps
Max Flow = 130-<JM
Max Lat = 100GM
Figure 5 Description of NSs and service levels
Table 1 Energy models of the cores of the microservers and of the network switch related to the usage percentage
Usage 0 10 20 30 40 50 60 70 80 90 100119875router 0 1 2 5 7 9 11 13 14 15 15119875cpu 5 50 70 100 130 150 165 170 180 185 190
Table 2 VNF characterization for the service flavours of the evaluation scenario
VNF1 VNF2 VNF3 VNF4 VNF5 VNF6Number of cores per instance
90Mbps 1 2 1 1 1 1108Mbps 1 2 2 1 1 1130Mbps 1 2 2 1 1 1156Mbps 1 2 2 1 1 1
CPU usage90Mbps 30 20 90 50 50 20108Mbps 40 30 60 60 60 30130Mbps 40 30 60 60 60 30156Mbps 40 30 90 40 20 30
Service latency90Mbps 10ms 15ms 30ms 35ms 20ms 25ms108Mbps 10ms 15ms 35ms 40ms 25ms 30ms130Mbps 10ms 15ms 35ms 40ms 25ms 30ms156Mbps 12ms 16ms 40ms 50ms 30ms 35ms
of network servicesflavours to the SC operator In this wayin the following first we are going to evaluate the behavior ofthe placement algorithm for the three service levels and nextwe will show the effect of applying RO techniques to thoseresults
521 Bronze Flavour Scenario Results with 0 Robust Pro-tection Figure 6 shows the placement results of an scenariowhere all the network services operate at bronze flavourDifferent colours represent each tenant in order to dif-ferentiate the VNFs that belong to its NS and differenttones distinguish NSs of the same tenant As a result ofthe placement algorithm all the NSs are allocated in theavailable virtualized resources with energy consumption costof 3712W meeting all the latency constraints The figure
displays the assignment of VNFs to the SCs highlightingsome placement particularities As explained in Section 3the VNF chains of the same tenant share their SCVNF Inour example Tenant 1 and Tenant 2 both have two NSsthat are served with a single SCVNF Furthermore eachCPU executes a single VM but some VNF instances needtwo cores to manage the user demand In connection withthis behavior the VNF scalability feature is also visible inFigure 6 observing the placement of for example VNF3for the different NSs NS1 with a higher aggregated userdemand needs two CPUs to instantiate VNF3 but since NS2operates at a lower user demand the instantiation of VNF3needs just a single core Note as well that bronze serviceflavour does not require HW acceleration to meet the latencyconstraints of the NSs Finally it can also be observed that
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mobile Information Systems 9
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Bronze Bronze Bronze Bronze Bronze
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
VNF1VNF3
VNF1
VNF6
VNF1
SCVNF
VNF3VNF2
VNF1VNF3
VNF1VNF2
VNF2VNF3
VNF5
VNF5VNF2
VNF5
SCVNF
VNF1
VNF5
VNF3
VNF4
VNF5
VNF6
SCVNF
VNF1
VNF1VNF2
VNF3
VNF4
VNF1
960Mbps
960Mbps
780Mbps
780Mbps
780Mbps
260Mbps
VNF2
VNF6
CPU
s
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
Switc
hH
A A
cc
132GM
102GM
132GM
102GM
130GM
37127
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
150GM 110GM 150GM 110GM 150GM
Figure 6 Placement results of bronze flavour setting with 0 robust protection
the algorithm tries to group asmanyVNFs as possible in eachserver in order to reduce the data traffic across the switch andminimize its load and therefore its energy waste
522 Silver Flavour Scenario Results with 0 Robust Pro-tection The evaluation experiment analyzed in this sectionstudies the outcome of the placement algorithm when thelatency constraints become more strict with silver serviceflavour Due to the further limitation of maximum latencyvalues (20 stricter) the placement mechanism is compelledto use HW acceleration units Figure 7 shows that at thisservice level all the NSs must use HW acceleration in oneof the VNFs composing the service chain As a consequencethe resulting total latency of the NSs decreases to meet theservice level requirements but in return the global powerconsumption grows by 22 up to 4547W
Apart from that the placement results illustrate the samebehavior as the bronze flavoured example of Section 521regarding the allocation of VNFs in the available resourcesscale-inout features and multitenancy
The evaluation experiments of Sections 521 and 522operate with a nominal aggregated user traffic to perform theoptimization decision-making process However a perfectlyknown user traffic demand is seldom available Usuallythere is a certain level of uncertainty in the behavior of theuser demand We include robust optimization techniques to
allow the placement algorithm to insert a defined level ofuncertainty and protect the system against the undesirableeffects of demand peaksThe evaluation examples in Sections523 and 524 include a robust protection parameter andanalyze its effect on the global power consumption
523 Silver Flavour Scenario Results with 20 Robust Protec-tion This section studies the placement outcome of the eval-uation scenario described in the previous section introducinga 20 robust protection parameter This modification on thescenario configuration implies an increase of the 20 of theaggregated user traffic as an input of the decision-makingprocess in order to be prepared for potential demand peaksduring the service operation
Figure 8 exhibits that VNF3 of NS2 must scale as a con-sequence of the traffic growth and now needs two CPUs foreach instance The traffic increase also has an impact on theprocessing latency of the VNFs causing an adjustment in theuse of HW accelerators As a consequence some VNFs thatrequired 2CPUs for their operation in the previous evalu-ation scenario now only need a single core in combinationwith the HWA Therefore in some cases the higher powerconsumption that is implicit to the use of HWAs may bebalanced with a decrease on the number of CPUs required forthe VNF Besides the higher aggregated user traffic involvesa more intense use of the network switch These factors have
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
10 Mobile Information Systems
SCVNF
VNF6
VNF1VNF1
VNF2
VNF3
VNF3
VNF3
VNF5
VNF2
VNF1
VNF2
SCVNF
VNF1
VNF2
VNF4
VNF5
VNF6
SCVNF
VNF2
VNF3
VNF4
VNF5
VNF6
VNF3
VNF1
Finallatency
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF1VNF1VNF2VNF2
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 0 0 0 0 0
Peak ow
960Mbps
960Mbps
780Mbps
780Mbps
260Mbps
780MbpsSw
itch
100GM
118GM
116GM
45477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
120GM 120GM 120GM
CPU
sH
A A
cc
88GM
88GM
90GM 90GM
Figure 7 Placement results of silver flavour setting with 0 robust protection
an impact on the power consumption of the componentsof the network resulting in a global energy consumption of5047W which implies an increment of 11 compared to theunprotected placement
524 Gold Flavour Scenario Results with 20 Robust Protec-tion Finally the last evaluation example sets even stricterlatency constraints to theNSs to be placed and includes a 20robust protection parameter Figure 9 depicts the results ofthe placement algorithm with this configuration Again thereduction of latency values forces the increment of the use ofHW acceleration In this case the main consequence is thespreading of the VNFs across the SCs to use the single HWAavailable in each cellThe use of HWAs the increase of powerconsumption due to higher user aggregated traffic and thehigher network traffic traversing the switch cause the globalenergy consumption to grow up to 5569W
As a final remark Figure 10 shows the comparison ofthe energy consumption and use of resources (cores activeCESCs and HWA) related to the robust protection level ina wider collection of scenarios In particular the charts ofFigure 10 compare the results of the placement algorithm forthe three service levels described in the previous sections(bronze silver and gold) combined with three robust protec-tion levels (0 or no protection 10 and 20) Obviouslythe main trend is that both power and resource consumptionincrease with higher values of protection but the associatedcost can be assumed in favor of reducing the impact of
unexpected user demand peaks However there are somecases in which the protection parameter does not have soadverse implications For example it can be observed howthe number of used cores in the silver service level with a10 protection level is lower than in the case of no robustprotection The reason of this behavior is how the algorithmuses HWA appliances in this case The increase of the usertraffic demand in the protected services implies the scaling upof someVNFs that nowmust usemore cores for the operationof theNSsThen the placement algorithm decides to useHWaccelerators for those specific VNFs compensating this waythe higher energy cost of HWAs with a lower use of CPUcores With all these considerations that can be extractedfrom a deeper analysis of robust placement results robustoptimization can be applied to decision-making problemsnot ending just with the optimization problem
The proposed steps in this paper can be used in anyplacement decision-making context and its goal is to providea tool that helps system administrator to evaluate the possibleimpact of any decision on the performance of the deploymentunderneath
6 Conclusions
This paper introduces a constraint-based expert system tosupport the decision-making process of VNF chain place-ment in distributed edge cloud networks The placement
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mobile Information Systems 11
SCVNF
VNF5
VNF6VNF5VNF2
VNF3
VNF5
VNF1
VNF2
VNF3
VNF1
VNF2
VNF3
VNF6
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF1
VNF2
VNF3
VNF4
840Mbps
840Mbps
936Mbps
624Mbps
312Mbps
312Mbps
936Mbps
VNF6
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF5VNF6VNF5VNF1VNF4VNF4
VNF2
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Silver Silver Silver Silver Silver
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
116GM
110GM
118GM
50477
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
150GM 110GM 150GM 110GM 150GM
79GM
79GM
CPU
sH
A A
ccSw
itch
Figure 8 Placement results of silver flavour setting with 20 robust protection
algorithm applies robust optimization techniques to theminimization of the global power consumption of the systemsubject to the bit rate and latency constraints of the networkservices The power consumption of the components of thesystem depends on the usage percentage which is a con-sequence of the aggregated user traffic demand The modelassumes that the relationships between the user trafficdemand the power consumption and the use of the resourcesmay be nonlinear Besides the placement algorithmconsidersthe 5G communication particularities of both control anddata plane and allows multitenancy Finally the algorithmalso includes VNF scaling features and the use of hardwareacceleration
The results show that the placement algorithm succeeds inallocating theVNFs that compose theNSs of different tenantsin the available resourcesmeeting the latency constraints andminimizing the energy consumption of thewhole systemThepresented evaluation examples demonstrate the features ofthe constraint-based expert system and illustrate the impactof the selected service flavour and robust protection on theglobal power consumption and therefore the exploitationcost of the 5G distributed edge cloud
Further researchwill extend the characterization of VNFsto include variable output bit rate This feature will modelthe behavior of for example transcoding functions A future
version of the algorithm will also incorporate the existenceof a virtual switch to connect the VMs running in the samemicroserver Another enhancement of the model will bethe combination of heterogeneous SCs with different con-figurations and resource consumption models Finally theobtained placement results will be used for a technoeconomicstudy that will analyze the relationship between the serveraggregated user traffic cost of the required infrastructure andthe pricing of the services This study will also consider theintegration of the placement decision problem with the radioside
Summary of Model Indexes SetsParameters and Decision Variables
Indexes
120587 Index for VNF placement policy119909 Index for SC microserver119910 Index for core number in each microserver119894 Index for NS119895 Index for VNF in a network service chain119903 Index for physical resources in
SC microserver
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
12 Mobile Information Systems
VNF6VNF5VNF1VNF6VNF2SCVNF
VNF2
VNF3
VNF5
VNF3
VNF1
VNF2
VNF3
VNF5
SCVNF
VNF1
VNF1
VNF3
VNF4
SCVNF
VNF2
VNF3
VNF4
VNF1
Tenant 1NS1
NS2
Tenant 2NS1
NS2
Tenant 3 NS1
Total energy consumption
VNF6VNF5VNF1VNF6VNF2VNF5VNF6VNF4VNF4
VNF2
840Mbps
840Mbps
936Mbps
936Mbps
312Mbps
312Mbps
312Mbps
312Mbps
312Mbps
Finallatency
Tenant 1 Tenant 2 Tenant 3
NS1 NS2 NS1 NS2 NS1
Service type Gold Gold Gold Gold Gold
Max ow
Max latency
Deviation 20 20 20 20 20
Peak ow
55697
130-<JM 90-<JM 130-<JM 130-<JM90-<JM
156-<JM 108-<JM 156-<JM 108-<JM 156-<JM
100GM 100GM 100GM
79GM
95GM
89GM
79GM
86GM
CPU
sH
A A
ccSw
itch
80GM 80GM
Figure 9 Placement results of gold flavour setting with 20 robust protection
34
32
30
28
260
10
20Robust protection ()
Num
ber o
f use
dco
res
10864200
10
20Robust protection ()
HW
AN
umbe
r of u
sed
0
10
20Robust protection ()
987654N
umbe
r of a
ctiv
eCE
SCs
6000
5500
5000
4500
4000
35003000
010
20
Ener
gy co
nsum
ptio
n(W
)
Robust protection ()
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Gold
Silver
Bronze
Figure 10 Comparison of energy and resource consumption for different levels of robust protection
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mobile Information Systems 13
Sets
Π Set of possible VNF placement policies119883 Set of SC microservers119884 Set of cores composing a SC microserver119868 Set of offered NSs119869119894 Set of VNFs in NS 119894119877 Set of resources available in
each SC microserver
Input Parameters
119879119894 Contracted maximum aggregated usertraffic for network service 119894
119863max119894 Maximum latency allowed for NS 119894119872119903119909 Amount of resource 119903 available in
SC microserver 119909119875sc119909 Power consumption of SC 119909 because of
being switched on119875cpu119909119910119894119895 Power consumption of VM 119910 of SC 119909 due
to CPU use of VNF 119895 of NS 119894119875switch119894119895 Power consumption in the physical switch
due to CPU use of VNF 119895 of NS 119894
Decision Variables
119875120587 Total power consumption ofplacement policy 120587
119875120587119894119895 Power consumption of VNF 119895 of NS 119894 inplacement policy 120587
119898120587119903119909 Consumption of resource 119903 in SCmicroserver 119909 in placement policy 120587
119905119897120587119909 Traffic of the link that connects SCmicroserver 119909 with the switchinplacement policy 120587
119860119909119910119894119895 Binary variable that expressesthe assignment of VNF 119895 of NS 119894 toVM 119910 in SC 119909
119871 119894119895 Binary variable to express the use of thenetwork switch to forward the flow ofVNF 119895 of NS 119894 to the next VNF
119889119894 Delay of NS 119894119889119894119895 Delay of VNF 119895 of NS 119894119889119901119909119910119894119895 Processing delay in the CPU of VNF 119895 of
NS 119894 in VM 119910 of SC 1199091198891198971199091119910111990921199102119894119895 Path delay for NS 119894 between VNF 119895
(located in VM 1199101 of SC 1199091) and VNF119895 + 1 (located in VM 1199102 of SC 1199092)
119889vs119909119910 Link delay between the virtual switch inSC 119909 and the VM 119910 in SC 119909
119889vs119909 119904 Link delay between the virtual switch inSC 119909 and the physical switch
119889vs119909 Delay in the virtual switch of SC 119909
119889119904 Delay in the physical switch
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The research leading to these results has been supportedby the EU funded H2020 5G-PPP Project SESAME (GrantAgreement 671596) and the Spanish MINECO Project5GRANVIR (TEC2016-80090-C2-2-R)
References
[1] ITU-R M2083 IMT Vision-Framework and overall objectivesof the future development of IMT for 2020 and beyond 2015
[2] P Demestichas A Georgakopoulos D Karvounas et al ldquo5G onthe Horizon key challenges for the radio-access networkrdquo IEEEVehicular Technology Magazine vol 8 no 3 pp 47ndash53 2013
[3] I F Akyildiz S Nie S-C Lin and M Chandrasekaran ldquo5Groadmap 10 key enabling technologiesrdquo Computer Networksvol 106 pp 17ndash48 2016
[4] ETSI Mobile-edge computing Introductory technical whitepaper 2014
[5] B Han V Gopalakrishnan L Ji and S Lee ldquoNetwork functionvirtualization challenges and opportunities for innovationsrdquoIEEE Communications Magazine vol 53 no 2 pp 90ndash97 2015
[6] R Mijumbi J Serrat J Gorricho N Bouten F De Turckand R Boutaba ldquoNetwork function virtualization state-of-the-art and research challengesrdquo IEEE Communications Surveys ampTutorials vol 18 no 1 pp 236ndash262 2016
[7] Ericsson White Paper Cloud-RANndash the benefits of virtualiza-tion centralization and coordination 2015
[8] B Blanco J O Fajardo I Giannoulakis et al ldquoTechnologypillars in the architecture of future 5G mobile networks NFVMEC and SDNrdquo Computer Standards and Interfaces vol 54 pp216ndash228 2017
[9] ETSI Network Functions Virtualization (NFV) Managementand Orchestration 2014
[10] A Ben-Tal L E Ghaoui and A Nemirovski Robust Optimiza-tion Aharon Ben-Tal Laurent El Ghaoui Arkadi Nemirovski-Google Books Princeton University Press Princeton NewJersey NY USA 2009
[11] J Gil Herrera and J F Botero ldquoResource Allocation in NFVA Comprehensive Surveyrdquo IEEE Transactions on Network andService Management vol 13 no 3 pp 518ndash532 2016
[12] B Addis D Belabed M Bouet and S Secci ldquoVirtual networkfunctions placement and routing optimizationrdquo in Proceedingsof the 4th IEEE International Conference on Cloud NetworkingCloudNet rsquo15 pp 171ndash177 October 2015
[13] H Moens and F De Turck ldquoVNF-P a model for efficientplacement of virtualized network functionsrdquo in Proceedingsof the 10th International Conference on Network and ServiceManagement (CNSM rsquo14) pp 418ndash423 Rio de Janeiro BrazilNovember 2014
[14] A Gupta M F Habib P Chowdhury M Tornatore and BMukherjee ldquoOn service chaining using Virtual Network Func-tions in Network-enabled Cloud systemsrdquo in Proceedings of the9th IEEE International Conference on Advanced Networks andTelecommuncations Systems ANTS rsquo15 pp 1ndash3 December 2015
[15] M Bouet J Leguay T Combe and V Conan ldquoCost-basedplacement of vDPI functions in NFV infrastructuresrdquo Interna-tional Journal of Network Management vol 25 no 6 pp 490ndash506 2015
[16] M C Luizelli L R Bays L S Buriol M P Barcellos andL P Gaspary ldquoPiecing together the NFV provisioning puzzle
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
14 Mobile Information Systems
efficient placement and chaining of virtual network functionsrdquoin Proceedings of the 14th International Symposium on IntegratedNetwork Management IM rsquo15 pp 98ndash106 IEEE TorontoCanada May 2015
[17] S Kim Y Han and S Park ldquoAn energy-Aware service functionchaining and reconfiguration algorithm in NFVrdquo in Proceedingsof the 1st International Workshops on Foundations and Applica-tions of Self-Systems FAS-W rsquo16 pp 54ndash59 September 2016
[18] V Eramo A Tosti and E Miucci ldquoServer resource dimen-sioning and routing of service function chain in NFV networkarchitecturesrdquo Journal of Electrical and Computer Engineeringvol 2016 Article ID 7139852 12 pages 2016
[19] K Hida and S-I Kuribayashi ldquoVirtual routing function allo-cation method for minimizing total network power consump-tionrdquo World Academy of Science Engineering and TechnologyInternational Journal of Electrical Computer Energetic Elec-tronic and Communication Engineering vol 10 pp 997ndash10022016
[20] C Pham N H Tran S Ren W Saad and C S Hong ldquoTraffic-aware and energy-efficient vnf placement for service chainingjoint sampling and matching approachrdquo IEEE Transactions onServices Computing 2017
[21] A Marotta and A Kassler ldquoA power efficient and robust virtualnetwork functions placement problemrdquo in Proceedings of the28th International Teletraffic Congress ITC rsquo16 pp 331ndash339September 2016
[22] C Pham H D Tran S I Moon K Thar and C S Hong ldquoAgeneral and practical consolidation framework in CloudNFVrdquoin Proceedings of the 2015 International Conference on Infor-mation Networking ICOIN rsquo15 pp 295ndash300 IEEE CambodiaCambodia January 2015
[23] V Eramo M Ammar and F G Lavacca ldquoMigration energyaware reconfigurations of virtual network function instances inNFV architecturesrdquo IEEE Access vol 5 pp 4927ndash4938 2017
[24] N E Khoury S Ayoubi and C Assi ldquoEnergy-aware placementand scheduling of network trafficflowswith deadlines on virtualnetwork functionsrdquo in Proceedings of the 5th IEEE InternationalConference on CloudNetworking CloudNet rsquo16 pp 89ndash94 IEEEPisa Italy October 2016
[25] V Eramo E Miucci M Ammar and F G Lavacca ldquoAnapproach for service function chain routing and virtual func-tion network instance migration in network function virtual-ization architecturesrdquo IEEEACM Transactions on Networking2017
[26] V Jumba S Parsaeefard M Derakhshani and T Le-NgocldquoEnergy-efficient robust resource provisioning in virtualizedwireless networksrdquo in Proceedings of the IEEE InternationalConference on Ubiquitous Wireless Broadband ICUWB rsquo15 pp1ndash5 IEEE Montreal Canada October 2015
[27] S Coniglio A Koster and M Tieves ldquoData uncertainty invirtual network embedding robust optimization and protectionlevelsrdquo Journal of Network and SystemsManagement vol 24 no3 pp 681ndash710 2016
[28] I Takouna K Sachs and C Meinel ldquoMultiperiod robustoptimization for proactive resource provisioning in virtualizeddata centersrdquo Journal of Supercomputing vol 70 no 3 pp 1514ndash1536 2014
[29] G Chochlidakis and V Friderikos ldquoRobust virtual networkembedding for mobile networksrdquo in Proceedings of the 26thIEEE Annual International Symposium on Personal Indoor andMobile Radio Communications PIMRC rsquo15 pp 1867ndash1871 IEEEKowloon Hong Kong September 2015
[30] S Coniglio A M Koster and M Tieves ldquoVirtual networkembedding under uncertainty exact and heuristic approachesrdquoin Proceedings of the 11th International Conference on the Designof Reliable Communication Networks DRCN rsquo15 pp 1ndash8 IEEEKansas Kan USA March 2015
[31] D Bertsimas and M Sim ldquoThe price of robustnessrdquoOperationsResearch vol 52 no 1 pp 35ndash53 2004
[32] F Rossi P Van Beek and T Walsh Handbook of ConstraintProgramming Elsevier Amsterdam Netherland 2006
[33] httpwwwminizincorg[34] C Schulte G Tack and M Z Lagerkvist Modeling and pro-
gramming with gecode 2010 Schulte Christian and Tack Guidoand Lagerkvist Mikael
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Submit your manuscripts athttpswwwhindawicom
Computer Games Technology
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Distributed Sensor Networks
International Journal of
Advances in
FuzzySystems
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014
International Journal of
ReconfigurableComputing
Hindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
Applied Computational Intelligence and Soft Computing
thinspAdvancesthinspinthinsp
Artificial Intelligence
HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014
Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Electrical and Computer Engineering
Journal of
Hindawi Publishing Corporation
httpwwwhindawicom Volume 2014
Advances in
Multimedia
International Journal of
Biomedical Imaging
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Advances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
RoboticsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Computational Intelligence and Neuroscience
Industrial EngineeringJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Human-ComputerInteraction
Advances in
Computer EngineeringAdvances in
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014