Applying Machine Learning to Intent-based Networking and

NFV Scaling Strategies

Diego R. Lopez, Telefonica

Glen Lockhart, Openet

The CogNet Project

• CogNet is building a network management solution based on machine-learning• Relying on SDN and the NFV architecture framework

• Focused on 5G use cases

• Committed to produce and demonstrate a general framework for smart management

• Able to support different application scenarios

• Monitoring and SLA enforcement

• Traffic identification and security

• Resource management

• Reliability enhancement

• E2E service management

• <Your favourite scenario here>

• And for sure• Relying on open source

• Willing to contribute back to the community

• Seeking for opportunities of further engagement

The CogNet Architecture

•A machine learning module (the CSE) connected to two streams

• Data, inbound

• Policies, outbound

•Supporting an integrated SDN/NFV model

• Following the SDN/NFV integration models considered by ETSI and ONF

The CogNet Implementation

• CogNet is committed to make its first demonstration in September

• The team is working in a first implementation release

• Based on open source• Monasca, Kafka and InfluxDB for

data collection• PANDAS, Spark MLlib, Hive and

Hadoop for the machine learning process

• Drools as policy engine• ODL as tenant controller• OSM for orchestration• OPNFV Brahmaputra as NFVI

• And open data models• JSON for (most) interfaces• SUPA for policy definitions

A Dialectical Approach to Intent

• Thesis: Translate intent into action• Understanding intent statements• Mapping onto technologies• Considering current state• A controlled but flexible language

• Antithesis: Address environment constraints• Provided by network management• Evolved from general principles defined by humans• Equivalent but not necessarily equal language space• The archetypal SLA enforcement

• Synthesis: Conflict resolution• Among action requests• And with management constraints• Hast to be initiated before translation• Understandable by humans for auditing purposes

Intent in the CogNet Architecture

• Intent can be expressed as different kinds of user input

• At least two essential computational intelligence technologies

• Linguistic interfaces• Machine learning mechanisms

• Identify application points and support composition

• Horizontal: federation• Vertical: recursion

Applying Intent Dialectics

• The generic 5G slicing• Slices defined by intent expressions

• Matched against operator infrastructure intent declaration

• Compatible with other slice definitions

• Supporting recursion, applying slice user intent expressions

• Supporting federation, across network and slice providers

• Personalized services• Composing intent expressions

• User, service provider, slice operator, infrastructure…

• Support for personal auditing

• Privacy preservation according to personal requests

• Reduced service TTM• Applying business rules as intent

• Domain-specific languages to capture customer requirements

• Bridging the gap between support systems and network management

• The BOSS concept

• Computational intelligence is here to help• And the CogNet project is paving its way

Applying Machine Learning to Intent-based Networking and

NFV Scaling Strategies

Diego R. Lopez, Telefonica

Glen Lockhart, Openet

Introduction

•As more network functions are virtualized and deployed in the cloud we want to be able to scale VNF clusters automatically to improve:• System availability

• Operating costs

Cost Savings

0

5

10

15

20

25

30

35

JAN-15

FEB-15

MAR-15

APR-15

MAY-15

JUN-15

JUL-15

AUG-15

SEP-15

OCT-15

NOV-15

DEC-15

JAN-16

FEB-16

MAR-16

APR-16

MAY-16

JUN-16

JUL-16

AUG-16

SEP-16

OCT-16

NOV-16

DEC-16

Planned vs Actual

Hadoop Cluster

YARN

High Level Architecture

Apache Spark

VIM (Open Stack)

Metrics Source

(Ceilometer)VNF-M

MANO Layer

Scaling Events

Collection Agent

Apache Kafka

Hybrid Model

Reactive Scaling

Predictive Scaling

Voting Engine

RPS Metrics VNFOrchestrator

Algorithm Selection

• We used Random Forest algorithm provided in Apache Spark MLlib to train our model

Selecting Training Data

• Biased towards more recent data to take trending performance into account.

• Take all data points from last week

• Take from the 5th minute of the hour from the week before that

• Take from the 15th minute of the hour from the week before that

• Take from the 30th minute of the hour from the week before that

Training Data - Code

JavaRDD<List<Object>> metricsRDD= inputFileRDD.map(new InputParser(fields));

….

String query = "SELECT datetime, rps FROM dataTable WHERE (minOfDay%5) = 0";

DataFrame week1= /* do SparkSQL query against metricsRDD */

….

DataFrame biasedData = week0.unionAll(week1)

.unionAll(week2)

.unionAll(week3)

.orderBy("datetime");

Training the model - Code

JavaRDD<LabeledPoint> biasedData= getTrainingData();

RandomForestModel model = RandomForest.trainRegressor(inputLabeledPointRDD,

new HashMap<>(),

100, "auto", "variance",

5,

32,

12345);

Predicting against the Model - Code

public Tuple2<Double, Double> predictModel(RandomForestModelmodel, double[] data) {

Vector v = Vectors.dense(Arrays.copyOfRange(data, 1, data.length));

Tuple2<Double, Double> predictionAndLabel =

new Tuple2<>(data[0], model.predict(v));

return predictionAndLabel;

}

Purely Reactive versus Hybrid Model

Summary

• Predictive auto-scaling delivers clear advantages over a reactive model, providing VNF’s with the resources they need, before they need them, ensuring that your service can provide timely responses 24/7