Measurement tools and the adoption of Abilene Observatory Jin Tanaka & Takatoshi Ikeda APAN-JP NOC...

Measurement tools and the adoption of Abilene Observatory

Jin Tanaka & Takatoshi IkedaAPAN-JP NOC

APAN Cairns meeting, July 2004

Background Measurement Tools in APAN-JP Selection and Provision of Measurement

data Deployment plan of Observatory

Outline

Background Various measurement tools are installed independently by

research projects in APAN-JP.Operators can’t grasp the whole measurement tools.Some demonstrations are being done without any notifications.

Multiple measurement tools sometimes deal with the same data simultaneously.

We can see the cases that the same data are measured with multiple machines.Many researchers have paid attention to throughput data and routing information as the research material.

Operators are investigating the network condition in spots.Operators are required to provide the researchers not only with connectivity but with more advanced services as needed.Operators are required to collaborate with the researchers, just before the events or the demonstrations.

Members of APAN-JP NOC

Network Researchers

Network Engineers

24 x 7 Operators

Experiments & Advanced Services

Basic Services including PerformanceTrouble Tickets

<Job Functions><Categories>

Measurement Tools in APAN-JP

Throughput -1-

Machine name Installed tools

Function Project Administrator

Note

Perf3 Iperf Bandwidth measurement

Internet End-to-end Performance Monitoring - Bandwidth to the World (IEPM-BW) project

KDDI Labs http://www.slac.stanford.edu/comp/net/bandwidth-tests/hercules/html/scatplots/node1.jp.apan.net.html

Perf4&7 Iperf Bandwidth measurement

Operation tool of NOC

KDDI Labs -

perf5&perf6&genkai-perf1&genkai-perf2

Web100*¹FastTCPIperfBWCTLOriginal tools

Bandwidth measurement

High-performancenetwork for eVLBI

M.Hirabaru(NICT)

http://mrtg.koganei.itrc.net/

*¹ http://www.web100.org/


Throughput -2-



Note

fs01&fs02 FastTCP Bandwidth measurement

FastTCP test across TransPAC

R.JayaramS.Low(caltech)

-

? GNET-1*² Bandwidth measurement

Bandwidthchallenge

O.Tatebe(AIST)

https://datafarm.apgrid.org/software/

bwctl IperfBWCTL

Scheduling and resource allocation wrapper for Iperf

piPEs Y.Kitamura(NICT)

-

*² http://www.gtrc.aist.go.jp/gnet/



Note

rv mrtd Routing information (BGP)

APII BGP collector for APAN Measurement WG

M.Hirabaru(NICT)


maru1 & maru2

Zebra *TcpdumpOriginal tools

Routing information (BGP)

Analysis of the relation between traffic and routing

M.Maruyama(U-Tokyo)

http://maru2.jp.apan.net

ope1 net-snmpTcpdumpOriginal tools

Routing information(OSPF,IS-IS)

Research for IGP stability

Z. Sheu(NICT)

http://ope1.jp.apan.net/cgi-bin/lsa-stat?net=apan&area=backbone

ns2 Original tools Routing information(BGP)


KDDI Labs http://ns2.jp.apan.net/nr/http://ns2.jp.apan.net/bgproute/


Routing

* http://www.zebra.org/

Machine name

Installed tools


Note

? GNET-1 One-way Delaymeasurement

Bandwidthchallenge

O.Tatebe(AIST)

https://datafarm.apgrid.org/software/

pe2(koganei.wide.ad.jp)qpe

OWAMP One-way Delaymeasurement

Research for One-Way Delay

Z. Sheu(NICT)

http://pe2.koganei.wide.ad.jp/cgi-bin/owd-stat

ns2 Smoke ping*⁴

Round Trip Timemeasurement


KDDI Labs http://www.jp.apan.net/measurement/rtt/


Latency

*⁴ http://people.ee.ethz.ch/~oetiker/webtools/smokeping/

Machine name

Installed tools


Note

ns CVSOriginal tools

Config backup and history


NOC Engineer -


Router


NetflowMachine name Installed

toolsFunction Project Administra

torNote

nms3 Flow tools Netflow Observatory KDDI Labs & NOC Engineer

-

Machine name

Installed tools


Note

ns syslogdOriginal tools

Reporting router information


NOC Engineer -


Syslog


Usage



Note

ns MRTGRRDTool

Trafficmeasurement


NOC Engineer http://www.jp.apan.net/NOC/tm/

mrtg CricketRRDToolOriginal tools

Trafficmeasurement


NOC Engineer http://mrtg.jp.apan.net/cricket/router-interfaces/

mrtg.koganei.itrc.net

MRTGOriginal tools

Traffic measurement

M.Hirabaru(NICT)


Selection and Provision of Measurement Data

Sort out data required for researchers

Throughput Iperf, BWCTL

Latency OWAMP,RRD

Netflow cflow,flowtools

GNET-1 IGP Routing information Tcpdump … etc.

to be provided by the Observatory system that NOC members will coordinate.

to be adopted into the Observatory system as “new” common tools.

　

Evaluation ＆ Selection

Advanced backbone services

Collocated systems assisted by NOC Engineers

to be left as the private tools.


Potential “New” Common Tool ~Example~

GNET-1 Developed by AIST (The National Institute of Advanced Industrial Scie

nce and Technology, Japan) GNET-1 is a gigabit network testbed, which consists of four Gigabit Et

hernet interfaces, four fast and large amount of SRAMs and a large-scale FPGA connected to the them.

Function Wide area network emulation Network instrumentation Traffic shaping Traffic generation

Powerful tool for developing network-aware grid softwarelike GridMPI and Gfarm


TransPAC/APAN Observatory Systems We are planning to provide the services of two categorie

s: Advanced backbone data

Backbone data collected by NOC using equipment located in TransPAC/APAN nodes. Data item and data format will become

the same as those of Abilene observatory.

Collocation Systems assisted by NOC engineers Data collected by respective research projects using their own equipments colloc

ated in XP. Accounts should be provided for NOC operators and engineers. Permission of exporting data should be authorized by the projects.

Just started by the Joint TransPAC and APAN-JP NOC members.


Backbone Data Collection

Necessity for backbone measurement Advanced Backbone measurement is required for improving NOC

services.

Advanced data enables to allocate the network resources effectively.

It helps operators to find “where is a bottle-neck?” It activates network researches with advanced backbone data.

E2E Measurement

Backbone Measurement

Observatory

NOC operators can’t coordinate the E2E measurement.

Congestion of measuring activities will be relieved.


Proposal of TransPAC/APAN Observatory

Measurement should be done in the best conditions Observatory systems should be installed in the best environments of TransPAC/APAN network.i.e. To be set as close to the router interfaces as possible, using the best interface cards of Jumbo

Frame.

NOC should maintain the systems and export data to authorized researchers. A good common platform for performance measurement should be installed at e

ach XP. Common measurement method should be developed

Common tools are installed, providing the common data. Data should be formatted in the global standards. i.e. GGF, piPEs, IPPM(IETF), IRPM …etc.


Formation made by Observatory

Observatory system will grow up R&E networks!

Observatory•Developed by NOC researchers & maintained by NOC engineers/operators

•Common Tools with High Priority

•Output data in standard format

Data with authentication

Software,Papers, etc.

ResearchersResearchers

can get measurement data of the global networks and work together with foreign researchers.

NOC Advanced Service

NOC Basic ServiceHelpful in operation

Deployment plan of Observatory

Observatory system could be deployed across the TransPAC circuits.

This deployment is to : get data over the long latency link cooperate with the Abilene Observatory

Deployment plan of Observatory

Location Japan

Tokyo XP U.S.

To be decided by Indiana University folks

Average RTT 190ms

TokyoXP

Chicago/Indianapolis

Los Angeles

Deployment plan of Observatory Machines

There are ４ machines in each side for observatory. (NMS1-4)The configuration is the same as that of Abilene Observatory : optimized for observatory based on their experience made easy to cooperate with the Abilene Observatory

managed by NOC Hardware Specification

Chassis : SuperMicro SuperServer 6013P CPU : Dual 3.06GHz Intel Xeon Memory : 2 x 1GB DDR266 Disk : 2 x SCSI 73GB (RAID1)(NMS2~4 ） 3 x SCSI 73GB (RAID5)(NMS1 ） NIC : Dual 10/100/1000 on-board Intel 1000base-SX （ NMS1)

Deployment plan of Observatory Roles of NMS

host OS Measurement tool Network

NMS1 Linux 2.4.26 web100 2.3.8

Throughput(regularly)

iperf GbE to router & switchMTU 9000

NMS2 Linux 2.4.26 web100 2.3.8

Throughput(on-demand)

iperf GbE to switchMTU 9000

NMS3 FreeBSD 5.2.1-REL-p8

NetflowUsage

Flow-toolsNet-snmp +RRD

GbE to switchMTU 9000

NMS4 FreeBSD 5.2.1-REL-p8

Latecy OWAMP GbE to switchMTU 9000

Routing, Router, Syslog and Usage data will be evaluated for the deployment. NICs on all servers are for Gigabit Ethernet.

Deployment plan of Observatory Network Layout (TokyoXP)

NMS1

NMS2

NMS3

NMS4

Procket PRO/ 8812

GigE Fibre

GigE Metal

GigE Metal

GigE Metal

GigE Metal

GigE Fibre

Deployment plan of Observatory Rack Front View

Deployment plan of Observatory Rack Rear View

Deployment plan of Observatory Network Layout (U.S.)

Indiana University folks will talk about the network layout.

(For reference) Current Situation of StarLight site

No L3 device Create VLAN for the observatory over the TransPAC circuit. It is L2 connectivity betwe

en TokyoXP and StarLight site In the future, L3 device might be installed, and reconfiguration of network layout will b

e studied. Power supply

AC power unprotected

at the cost of circuit reliability -- 意味を教えて！

Current Status Current Status (TokyoXP)

The machines has installed the OS and connected the network Ready to collect data

Netflow Ready to collect a Netflow data with Juniper router

Problem with the Procket routers:The Procket routers create the UDP data of MTU size for netflow, IP packet containing UDP header and IP header exceed the MTU size. Hence fragmentation is occurred and flow-tools cannot receive packets correctly.

->will be fixed in the next software version

Throughput and Latency Ready to collect throughput and latency data with the peering machine located

in Chicago, managed by John-san. Need to tune some parameter for good throughput data

(buffer , flow-control ) Our L2 switch didn’t perform wire-rate switching. Some packets was disc

arded at input port while testing a high bit rate . we will investigate in detail .

Roadmap under discussion with Indiana University, but the f

ollowing schedule is envisaged: 2004.7 Netflow & piPEs (temporal) Website was formed: http://www.jp.apan.net/NOC/Observatory/

2004.8 start to develop the real system 2004.10 Connection with Abilene Observatory 2005.1 Presentation at APAN Bangkok meeting

Deployment into APAN We expect that Observatory system will be

widely deployed in APAN.

North America

KoreaTokyo

Indonesia

USA

Vietnam

TaiwanHong Kong

Philippines

Australia

Thailand

Malaysia

Singapore

China Genkai

TransPAC Observatory

APAN Observatory

References Abilene Observatory

http://abilene.internet2.edu/observatory/ http://loadrunner.uits.iu.edu/~neteng/nms/

Tools web100 http://www.web100.org/ GNET-1 http://www.gtrc.aist.go.jp/gnet/ zebra http://www.zebra.org/ Smokeping

http://people.ee.ethz.ch/~oetiker/webtools/smokeping/ Flow-tools http://www.splintered.net/sw/flow-tools/ OWAMP http://e2epi.internet2.edu/owamp/ Iperf http://dast.nlanr.net/Projects/Iperf/

Thank you

Please feel free to contact us if you have any questions and comments

Jin Tanaka – [email protected] Takatoshi Ikeda – [email protected]

Measurement tools and the adoption of Abilene Observatory Jin Tanaka & Takatoshi Ikeda APAN-JP NOC...

Documents

Transcript of Measurement tools and the adoption of Abilene Observatory Jin Tanaka & Takatoshi Ikeda APAN-JP NOC...