Introduction to NBL

~Since 2002

1

Agenda

• Introduction

• Tool Development

• RealFlow

• Software Defined Network and OpenFlow

2

Introduction

• 3rd-Party Testing Lab

• Self-designed Tools

• RealFlow Test

• Research Team

• Look forward for the future

3

4

Test Coverage

Area DUT/FUT Test Coverage

Bridging and

Routing

Ethernet L2/L3 Switch Functionality, Conformance, Stability,

RFC 2544/2889

NCSec UTM, Anti-Virus, IPS, SSL VPN,

IPSec VPN, P2P/IM Management

Functionality, Interoperability, Stability,

Session Capacity and Rate, Accuracy

WLAN and

VoIP

SOHO Router, DSL Router, IAD

Gateway, SIP Phone, SIP

Gateway, SIP Proxy, Access

Point

Voice Quality, Mobility, Functionality,

Interoperability, Stability, Session Capacity

and Rate

Bridging and

Routing

WLAN and VoIP

5

Public Benchmark Event

2001網路安全閘道器網路頻寬管理器

網頁交換器網路服務品質

2002電子商務網站

無線區域網路產品網路安全閘道器

Internet內容遞送

2010Network

Anti-botnet Solution

Benchmarking

2004Wireless LAN SOHO路由器

VoIP產品

2005VoIP互通性插拔大會網路/內容安全產品

2006入侵防禦系統

10GbE Ethernet SwitchVoWLAN 產品的語音品質

2007P2P Friendly Properties of NAT

Wireless SIP Residential Gateways

2009SOHO NAT於真實網路流

量下的穩定性表現

Benchmarking, Workshop, Publish

2013International SDN

Symposium Taiwan

6

Industrial Customers

http://www.broadcom.com/

http://www.broadcom.com/

http://www.box-sol.com.tw/

http://www.box-sol.com.tw/

http://www.draytek.com.tw/

http://www.draytek.com.tw/

http://www.fortinet.com/

http://www.fortinet.com/

http://www.cht.com.tw/index.php

http://www.cht.com.tw/index.php

http://www.trendmicro.com/tw/

http://www.trendmicro.com/tw/

http://www.quantatw.com/

http://www.quantatw.com/

http://www.paralink.com.tw/

http://www.paralink.com.tw/

http://www.fon.com/tw/

http://www.fon.com/tw/

http://zh.wikipedia.org/zh-tw/File:Mtk-LOGO.jpg

http://zh.wikipedia.org/zh-tw/File:Mtk-LOGO.jpg

Self-Designed Test Tool更新日期：2010/2/10

NBL開發工具/系統目前已授權提供多家國內廠商

適合的待測物類型(DUT)

工具/系統名稱(Tool

Name)

與預估開發人力

規格/特色(SPEC & Features)

應用說明(Description)

工具類型測試類型

SOHO Router

DSL Router

IAD Gateway

Security Appliance

In-Lab Live

Test(ILLT)

(6人月)

1. HW: PC

2. SW: Java and C programs

3. Continuously increasing real-world traffic

4. Support Bridge/Router/NAT mode

5. Automatically check the status of DUT

6. Stateless and Stateful Replay

[ 穩定性測試與除錯 ]

利用錄製、播放流量的技術，完整呈現真實網路環境。在實驗室內部再造真實網路流量環境，與Field Test相較之下，在測試上更具彈性、網路流量涵蓋範圍更大、並且更容易重製Bug。

流量產生、結果解析、自動化

Stability

Security Appliance

UTM

Anti-Virus

IPSP2P/IM

Network Forensics

PCAP Library

(3人月)

1. HW: PC

2. SW: Java and C progrmas

3. A database of PCAP files

4. Continuously increasing real-world traffic

5. Classified netowrk traffic such as P2P,

Attack

[ 網路流量資料庫 ]

流量資料庫，儲存各種未分類與已分類的PCAPs。目前分類的類別包括P2P、Attack、HTTP、SMTP、POP3、FTP等。

[ 協助漏判(FN)與誤判(FP)情況的改善 ]

可應用在 "找出可能造成產品漏判與誤判的網路流量"，進一步提供這些可疑的網路流量(i.e. PCAP files)給研發人員參考，方便找出其特徵值或行為模式。


Accuracy

QoS

SOHO Router

Security Appliance

UTM

Anti-Virus

IPSP2P/IM

Network Forensics

APP-Test

(5人月)

1. HW: PC


3. 支援多種Applications:

HTTP、FTP、Mail、SIP/RTP、IM、P2P、 Skype、Streaming (ex. Youtube、MLB、NBA)

4. 自動分析DUT所產生的Log資訊來判斷測試結果

[自動產生各種類型的應用]

透過使用者端應用軟體，自動產生各類應用層網路流量，例如測試QoS時，可產生各類streaming應用，檢查各streaming應用是否有達到頻寬的保證。

[自動解析測試結果]

從各類待測物的output中(ex. Log)收集相關資訊，來判斷本次的測試結果為Pass or

Fail


Functionali

ty

Self-Designed Test Tool (cont.)



Name)





SSL VPN

SSL VPN Tunnel

Capacity

(2人月)

1. HW: PC


3. Generate more than 500 SSL VPN Tunnels

in a single PC

4. Support background traffic in the tunnel

such as HTTP, CIFS

測試SSL VPN Tunnel數量，使用一台PC即可建立超過500條以上的Tunnel，並且在每條建立起來的Tunnel中可傳送應用層流量，如網頁的瀏覽、網路芳鄰抓檔。


Performanc

e

PPTP/L2TP VPN

PPTP/L2TP VPN

Tunnel Capacity

(2人月)

1. HW: PC


3. Generate more than 500 PPTP/L2TP VPN

Tunnels in a single PC

4. Support background traffic in the tunnel

such as HTTP, CIFS

測試PPTP/L2TP VPN Tunnel數量，使用一台PC即可建立超過500條以上的Tunnel，並且在每條建立起來的Tunnel中可傳送應用層流量，如網頁的瀏覽、網路芳鄰抓檔。


Performanc

e

Switch

IGMP Snooping

(2人月)

1. HW: SmartBits

2. SW: Tcl/Tk programs

3. Support IGMP v1/v2/v3

測試switch在IGMP snooping的功能是否正確。流量產生、

結果解析、自動化

Functionali

ty

General

MIB-AutoChecker

(3人月)

1. HW: PC

2. SW: Net-SNMP and Perl programs

3. MIB Object Accessibility

4. MIB Definition Customization

5. SNMP v1, v2c and v3 Support

即自動化 MIB 物件存取技術。MIB AutoChecker 可匯入自訂 ASN.1的物件檔案，它以 Net-SNMP 動作為基礎，自動地展開指定的數個群組的所有物件並加以存取，記錄下所有訊息使得測試者很快就找出有問題的部份。以Net-SNMP為基礎，可以在MIB中指定的結點與其下的sub-tree自動化存取物件。

自動化Functionali

ty



工具/系統名稱(Tool Name)





Switch/Router

ANVL-MultiParser

(3人月)

1. HW: PC

2. SW: ANVL and Perl programs清楚解析出ANVL的測試結果，讓研發人員可以快速找出產品的問題點。結果解析

Conforman

ce

Switch/Router

Performance Test

Suites(PTS)

(3人月)

1. HW: SmartBits

2. SW: Tcl/Tk and PHP programs

3. 圖型化使用者介面：DUT Configurator、

SmartBits Configurator、Test Editor、Test/

Result Viewer

4. 自動化組態待測物：5. 自動化收集測試結果及產生測試報告

Layer 2/3 Switch的效能測試自動化。

自動化Performanc

e

Switch/Router

Auto-ANVL

(3人月)

1. HW: PC

2. SW: ANVL and Perl programs

3. 圖型化使用者介面：DUT Configurator、IxANVL Configurator、Test Editor、Test

Viewer、Result Viewer、Preference

4. 自動化組態待測物5. 自動化執行Test Suite及測試案例6. 自動化剖析測試記錄檔及分析結果7. 自動化產生測試報告

Layer 3 Switch的符合性測試自動化。

自動化Conforman

ce

SOHO Router

DSL Router

WebUIAutoChecker

(2人月)

1. HW: PC

2. SW: Java programs

3. Web GUI invalid value auto check

4. Suppoer Firefox and IE

Web GUI 自動檢測工具，將Web GUI輸入各種錯誤值的測試自動化，提高測試效率與準確性。自動化

Functionali

ty




Name)





WiFi Phone

WLAN Card

Access Point

WLAN Mobility Test

System(WMTS)

(6人月)

Mobility Test Scenarios:

1. HW: Azimuth

2. SW: Tcl/Tk

3. Distance Test for Computer Adapter

4. Distance Test for Mobile Phone

5. Motion Adaptation Test for Computer

Adapter

6. Motion Adaptation Test for Mobile Phone

7. Motion Roaming Test for Computer

Adapter

8. Motion Roaming Test for Adaptation for

Mobile Phone

9. Switch Roaming Test for Computer

Adapter

10. Switch Roaming Test for Adaptation for

Mobile Phone

Programmable Channel Model:

11. Free Space and Typical

Home/Office/Street Channels

12. More channels can be added by the

programmable parameters

Traffic Generation/Analysis:

1. HW: Abacus and PC

2. SW: IxChariot and Perl programs

3. Ixia IxChariot® (with the scripts and QoS

selective)

4. Abacus® for PESQ/PSQM analog testing

5. NBL traffic analyzers for link adaptation

and re- association

自動化WLAN移動測試系統（WMTS）可進行Roaming的延遲分析。提供具有可參數化channel模擬與WMM支援的Distance

Adaptation和Roaming測試Scenarios

For Distance, Adaptation, and Roaming

scenarios with a parameterized channel

approximation and WMM support

The WLAN Mobility Test System (MTS) for

Azimuth W-Series Platform® version 1.3 can

support 8 novel mobility test scenarios and

integrate standard traffic utilities like

IxChariot® and VQT®, including QoS

enabled configuration. A programmable

channel model is introduced for the mobility

control to offer the approximation of signal

fading for the in-lab wireless testing, which is

more stable and repeatable than the outdoor

testing.

結量產生、結果解析、自動化

from Lab Test to RealFlow Test

11

Solutions

12

真實網路流量測試

Real Network

•Users

•Beta Site

Replay Tech.

•Capture

•Replay

Test System

•PCAP Library•In-Lab Live Test (IL2T)

RealFlow

Beta Site

Zone 1 End-user software

Zone 2 Ethernet L2/L3 Switch Wireless AP

Zone 3 Core Router

Zone 4 (Inline, one-in-one-out)

UTM, IPS, Anti-Virus, QoSFirewall

Zone 5 (Sniff)

Network Forensic Anti-Malware/Botnet

Zone 6 (ILLT)

SOHO Router, Home Gateway Broadband Gateway DSL Router, IAD Gateway

13

RealFlow Certification

•Function

•PerformancePhase

1StabilityPhase

2

14

USG 2000ZyXEL Security Appliance

DIR-300D-Link SOHO Router

DIR-615C1D-Link SOHO Router

Mini guardLionic Security

Appliance

DIR-655D-Link SOHO Router

Experiment in OpenFlow

Service

Solution

• Solution Prototype•SDN Enabled Wi-Fi

Tool

• Test Tool develop and Automation•Cloud based conformance test tool based on oftest

•SDN Enabled Testbed

Service

• Test/Benchmark of SDN•Device and Apps

•OpenFlow Certification

16

Approved Labs

Why SDN - Requirement

• Too many different kinds of devices with different operation method for 1 or 2 persons

• Devices with high cost

• Impossible to fully match the campus network administration policy

What solution we need?

• An unify control message and method to integrated different devices from different vendors

• An easy way to develop the related service/software and deploy the suitable devices with suitable spec. and cost

• An flexible method to integrate the other services and devices

Advantage of OpenFlow (Manage)

• Open source with low cost

• A generic protocol to control different kinds of devices

• The network administrator could choose the suitable HW devices and policy/SLA for each site and account(person)

• Don’t need to add more human resource to manage more devices (cost again)

Advantage of OpenFlow (Technical)

• Control the forwarding table of switch/router and wireless devices from different vendors with the same message

• Control the forwarding policy by the software development of each site with different rule.

• Easy to control the traffic to integrated with other services

Campus Wi-Fi Solution

Campus Wi-Fi Solution (cont.)

Example: Auth Workflow

Example: System control

Web UI

RYU

AP

REST APP

OpenFlow

OpenFlow

CURL

OpenFlow 1.3Experimenter message

RESTAPI POST

Feature List

• Auto-Provisioning and Configuration through experimenter message

• Multi-path routing/switching with fail-over and switch-over

• Switch and AP control and management

• Multicast Routing

• Streaming Control

• Wi-Fi AP-STA Association Load Balance

28

SDN Test (concept)

APP/Controller

Topology Generator•Automatically•Physical devices•Simulator

Monitor•Flow Checker(App for Testing)

Test bedTraffic Generator•Trackable mechanism

Test Script•Test Methodology•Test Plan

Test Controller

Monitor

Test bed

Traffic Generator

Topology Generator

Test Script

Report Generator

Test Portal

SDN Test (Lab and Auto)

SDN Network Device

1. Execute Automation Test Program

MonitorSystem information collector

CPU Loading, Link utilization, counters, client simulator

Packet GeneratorScriptableError packetsAttack traffic

Trackable packets and mechanism*L4+ level packet generator

Topology generatorGUI based traffic tracker

Lab test automationSDN-enabled Layer 1 switchFlow entry add/removeControllable VM migrationTest plan/case/script/reporter*

SDN-enabledLayer 1 switch

2. Topology Generator generate a topology

3. Monitor Start to gather information DUT & SUT

4. Start to generate the required packets

5. If any Link Needs to be Removed During Test Period,

the L1 Switch can Do it.

SDN Test (BetaSite)

Traffic ReproducerReal-time time machineReproduce defect of AppsReproduce trafficReproduce topology by simulator and emulator

Lab Test AutomationFlow entry add/removeControllable VM migrationTest plan/case/script/reporter*

1. Execute Automation Test Program

Topology Generator

GUI based traffic tracker2. Automatically Gather the Topology Information

MonitorSystem information collectorCPU Loading, Link utilization, counters, client simulator

3. Monitor Start to gather information DUT & SUT

4. Traffic Start

SDN Network Device

5. 1 Capture traffic

5. 2 Replay traffic to reproduce bugs

PKTPKT

Packet Generator

oftest GUI

oftest GUI (cont.)

基於SDN和Cloud架構之無線/寬網技術與服務

SDN-enabled Cloud-based Wireless and Broadband Network Technologies &

Services

Project Goal

• Establish the first end-to-end test platform for SDN in Asia

• Contribute to the open networking (SDN related) standards and set up international reputation

• Cultivate human talents with SDN related technology and service knowledge and experience to build an SDN industry in Taiwan

• Strengthen SDN IP portfolio for Taiwan industry

• Enhance the competitive advantage of Taiwan ICT industry

Technology Layered Architecture

38

IoT/M2M ②

Northbound API ①③④⑥⑧

Southbound API

Controller

Switch

Testb

ed

and M

eth

ods④

Cam

pus F

ield

Tria

l①②③④

North

ern

TW

Fie

ld T

rial ①②

③④⑥⑧

OA&MService APP

Network APP Security④⑧

Perfo

rmance④

Accountin

g①

Config

ura

tion①③⑥

Fault

①CHT (Wireless/BB/Security) ②CHT (Cloud/Billing/IoT) ③D-Link ④EstiNet

⑤MediaTek ⑥ Arcadyan ⑦ Xinguard ⑧ Inventec

Enterprise/Campus ③⑥⑧

Broadband Technologies① ⑦

Network Virtualization

V&T

SDN Chip⑤

SDN Switch OS④

Multi-controller④

Cloud Technologies②④⑧

Wireless Technologi

es①

Smart Data Pricing ②

Common controller①

Subproject E – Integration and Field Test

39

小型SDN環境(小型OpenFlow Switch)

InternetOpenFlowWiFi

有線網路

無線網路

OpenFlow CPE

終端SDN環境

雲端資料中心

x4 x2

雲端SDN環境

行動網路

UE

TL Core Network

小型SDN環境(小型OpenFlow

Switch)

Internet中型SDN環境(中型OpenFlow

Switch )

OpenFlowWiFi

有線網路

無線網路

SDN Controllers

OpenFlow CPE

終端SDN環境

雲端資料中心

x4

x2

雲端SDN環境

行動網路

UE

NCTU Core Net. emulator

小型SDN環境(小型OpenFlow

Switch)

Internet

OpenFlowWiFi

有線網路

無線網路

SDN Controllers

OpenFlow CPE

雲端資料中心

x4

x2雲端SDN環境

CHT

NCTU NTHU

SDN Core Switch(中型OpenFlow Switch )

中型SDN環境(中型OpenFlow

Switch )

SDN Master Controllers

SDN Controllers

中型SDN環境(中型OpenFlow

Switch )

Internet

SDN APP

SDN APP

SDN APP

SDN Enabled Wi-Fi Solution

40

Outlines• System Architecture

• Topology Example

• Environment Description

• Features vs. Components

• Vendor-Defined Feature Specification

• Feature: Auto Provision Function

• Feature: Authentication Portal Function

• Demo: OAM Web System

41

System Architecture

42

Topology Example

43

Environment Description (1/3)

• SDN Controller

Ryu SDN Framework 3.8

Ubuntu 12.04

• SDN Access Point

Device: TPLink TL-WR1043ND V2, AP222, …

OS: OpenWRT trunk

Patch: ofsoftswitch13

• SDN OAM Server / SDN Portal Server

PHP 5.5

Apache 2.4.10

• SDN Database Server

MySQL 5.5


• Ryu SDN Framework

Python-based Project

Written in Python language

Environment

OS: Ubuntu 10.04 or higher

Network simulation tool: Mininet

Features provided

Component-based SDN framework

Support OpenFlow v1.0, v1.2, v1.3, v1.4

Support various protocols for managing network devices, such as OpenFlow, Netconf, OF-config, etc.

Provide REpresentational State Transfer (REST) service

45


46

Controller

Ryu App

data in JSON format

REST A

PI

PHP

Operations via HTTP request with URL.

• GET• POST• DELETE• PUT

HTTP request

OA

M W

eb

UI

curl HTTP request

Mininet

(Simulated network)

OpenFlowSwitch(e.g., Access Point)

REpresentational State Transfer (REST) Service

Features vs. ComponentsSDN Controller

ComponentSDN Access Point OAM App

System Management

Generic Wireless Control

Auto Provision Function

Authentication PortalFunction

Device InformationManagement

Channel Non-overlapping

Quality of Service

Roaming

47

Vendor-Defined Feature Specification (1/2)

• Vendor-defined features

Implemented with “Experimenter Messages”

Experimenter MessageAn optional field (padding) in OpenFlow protocol

Provide SDN vendors to develop self-defined functions and services

• Experimenter Message Format

48

Header:

EXPERIMENTER_ID

(8 bytes)

EXP_TYPE

(8 bytes)

Data:

EXP_SUBTYPE

(1 byte)

PAYLOAD

(255 bytes)

• The format should be defined in both controller and access points.• Corresponding handler function will be triggered according to the EXP_TYPE and EXP_SUBTYPE.

Vendor-Defined Feature Specification (2/2)

49

Experimenter Message Type and Subtype List Example

Feature: Auto Provision Function (1/3)

50

System Scenario


51

Trigger Proxy Mode of Configured AP Automatically

Trigger/Close Proxy Mode of Configured AP Manually by Administrator


52

Relay the Configuration Request from Unconfigured AP and Response

Feature: Authentication Portal Function (1/3)

53

Topology Example


54

Message Flow


55

Demo Portal Webpage

Demo: OAM Web System

56

[Demo URL] http://140.113.243.175/login.php

Login Webpage & Dashboard Webpage

http://140.113.243.175/login.php

Introduction to NBL

Technology

Transcript of Introduction to NBL