1

Network Measurement SummaryESCC, Feb. 2006

Joe Metzger

ESnet Engineering GroupLawrence Berkeley National Laboratory

2

General Trends 1

• There has been a community shift in the focus of network measurement effortso From supporting the people running the networks

o To supporting the people using the network

o How we get there- Backbone providers agree on one set of tools to archive and

publish measurement data– ESnet, Internet2, GEANT, most European NRENs and others are all

collaborating on PerfSONAR– Standardization on collection tools may also happen

- The user community develops analysis and visualization tools that leverage the backbone provider data to meet their needs

– EGEE network analysis tool– PerfsonarUI– MonALISA?

3

General Trends 2

• 10 Gigabit networking will present new challengeso The high costs of 10GE routers is pushing people to

deploy different architectures using lower cost switches.o Many of these lower cost devices provide very limited

diagnostic and debugging support. o Over subscription of circuits and bottlenecks within

equipment may cause un-expected packet loss.o The differences between OC192, 10GE LANPHY, 10GE

WANPHY and almost 10GE line-rate equipment may cause buffering problems.

o These issues will lead to problems that will be challenging to identify and resolve.

4

ESnet Deployed Tools & Capabilities

ESnet Performance Centers SNMP & Flow Stats Owamp PerfSONAR

Utilization Measurement Archive

Traceroute Visualization

5

ESnet Performance Centers

A web interface that lets ESnet users perform tests to points in the ESnet core

•Ping•Traceroute•Iperf•Pathload•Pathrate•Pipechar

6

2 SNMP Collection Systems

• One system based on MRTGo Collects interface utilization, error and discard data every

minute from ~1383 physical and logical interfaces

o Used for day to day operations.

o Portions accessible via http://netinfo.es.net

• Second collection system based on netSNMPo Collects interface utilization

o Monthly reporting

o Details back to 1990- http://www1.es.net/pub/esnet-stats/

• The two systems back each other up

7

Netflow

• Collect at ESnet ingress points

• Trendingo Identify large traffic flows

o Observation: About 100 host pairs generate 30% of all ESnet traffic

• Peering Analysiso ESnet has around 100 peers

- Most of them at multiple locations

- Look at “what if” scenarios for traffic engineering

• Diagnosing problemso Identifying endpoints of a flow that saturated a site access

link

8

OWAMP

• Initially deployed to measure & compareo DOE Lab to DOE Lab performance

o DOE Lab to University performance

o Mesh between- FERMI, LBNL, BNL

- OSU, NCS, SDSC

• Very small network changes can generate measurable differences in latency.

• OWAMP monitors deployed in most ESnet hubs

9

PerfSONAR Utilization MA

• Provide ESnet users and network researchers access to ESnet link utilization datao Deployed a PerfSONAR Measurement Archive (MA)

- Uses the “EGEE” tagged release of PerfSonar Java MA

- Provides a web services interface using NMWG V2 schema

- Data source is RRD files populated by SNMP system– Copied to MA every 5 minutes

o Provides link capacity and link utilization information for over 800 ESnet interfaces.

• Software that uses the ESnet MAo EGEE network diagnostic toolo CNM (DFN Customer Network Management tool)o PerfSONARUI o Traceroute Visualizer

10

PerfSONAR Traceroute Visualizer

• Displays a web form

• User pastes a traceroute output into the form

• Parses the form data to find IP addresses

• Displays the traceroute with interface utilization graphs for all the interfaces it can find.

• Developed a quick demo to show the value of a standardized measurement infrastructure

• Show demo if time permits

11

Traceroute

ESnet PerfSONAR Traceroute Visualizer

Trace SubmittedTracing route to cache3.bnl.gov 130.199.3.21 over a maximum of 30 hops 1 1 ms 1 ms 1 ms joem-fe-stub.es.net 198.124.224.5 2 21 ms 21 ms 21 ms chi-ameslab.es.net 134.55.208.38 3 21 ms 21 ms 21 ms chicr1-ge0-chirt1.es.net 134.55.209.189 4 41 ms 41 ms 41 ms aoacr1-oc192-chicr1.es.net 134.55.209.58 5 43 ms 43 ms 43 ms bnl-oc48-aoacr1.es.net 134.55.209.130 6 43 ms 43 ms 43 ms bnl-esbnl.es.net 198.124.216.114 Analyzing Trace

12

CNM from DFN 1

13

CNM from DFN 2

14

PerfsonarUI 1

15

PerfsonarUI 2

16

PerfsonarUI 3

17

ESnet Future Plans 1

• Generalo Continue the PerfSONAR collaboration with Internet2,

GEANT, and the European National Research & Education Networks

o Deploy additional components of the PerfSONAR architecture as it is developed

- Lookup service

- Topology service

- Bandwidth measurement points

- Latency measurement points

- Authentication service

o Enhance & extend our end to end monitoring efforts- Include more LHC sites

- Interoperate with other tools used in LHC community

18

ESnet Future Plans 2

• SNMPo Use PerfSONAR to continue to publish ESnet interface

data

• Latencyo Integrate existing ESnet latency MPs with PerfSONAR

o Deploy new latency MP’s in new hubs

• Bandwidtho Integrate ESnet Performance Centers with PerfSONAR

- GE capable bandwidth MPs

o Deploy new PerfSONAR bandwidth MPs- 10 GE capable

19

Suggestions for Sites

• Deploy test servers at the edge of your network.o OWAMP for measuring latencyo IPERF/BWCTL for measuring bandwidtho Internet2 is measurement workshops

• Collect interface utilization datao Capture data with MRTG, Cricket, SNAPP or other tools.

• Join the PerfSONAR development team if you are actively developing measurement tools

• Think about data export policies

• Plan to deploy PerfSONAR in the summero After we generate a release…

20

Summary

• ESnet will deploy tools:o To run the backbone

o To provide the data necessary for end users to develop and tune applications

• Sites should deploy tools that will allow them to:o Determine if their applications are working correctly

o Determine if the local network is working correctly

o Generate useful information for diagnosing problems

• Use these tools to:o Continuously document performance so you know when it

changes

o Share your network measurements results

21

Pointers

o http://www.perfsonar.net

o http://netinfo.es.net

o https://performance.es.net

o http://e2epi.internet2.edu