Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput...
Transcript of Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput...
![Page 1: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/1.jpg)
Internet2 NetFlow Weekly Reports
Stanislav Shalunov
Internet2 Fall Member Meeting, Indianapolis, 2003-10-13
![Page 2: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/2.jpg)
What is NetFlow?
• Originally a Cisco proprietary technology
• Now supported by other vendors and being standartized in
the IETF (ipfix WG)
• Used to be a method to speed up packet forwarding
– Cache the next interface for given 〈src, dst, proto, ports,
tos〉
– Look in the cache hash table before you consult the rout-
ing table
• It was realized it’s useful for accounting purposes
• Not any longer used for optimization, but the accounting use
is growing more widespread
1
![Page 3: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/3.jpg)
NetFlow version 5
• There are several versions of NetFlow (5, 7, 8 are widespread)
• Provide different fields and different levels of aggregation
• NetFlow v5 gives you records with the following:
src ip, dst ip, packets, octets, start time, end time,
src port, dst port, proto, tos, src as, dst as, if in, if out
• Can use for accounting purposes, but there’s more
2
![Page 4: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/4.jpg)
Methodology of Data Collection
• Collect 1% sampled NetFlow data from all core Abilene routers
• Collection done at ITEC-Ohio with flow-tools
• Throw away data coming from interfaces between core nodes
• flow-tools now include SNMP hooks for that
• Concatenate the rest of the data
• Ship the resulting files (5–25 GB) to our RAID array daily
• Resulting view treats Abilene as a single data-forwarding unit
3
![Page 5: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/5.jpg)
Methodology of Data Processing
• The goal is to capture long-term trends
• Weekly averages for everything, hence weekly reports
• Daily averaging too volatile, monthly would take too long
• Two data sets: one the complete thing, one “bulk TCP”
• Bulk TCP is a TCP connection that transferred > 10 MB
• For full data set can do traffic composition
• For bulk TCP data set can do more, including throughput
• Traffic composition studies are routine (though most do not
look at file sharing), but looking at bulk TCP is unique
4
![Page 6: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/6.jpg)
Data Presentation
• Find it at http://netflow.internet2.edu/weekly/
• Weekly: new report added, time-series graphs updated
• The heart: TCP throughput analysis (includes CDF)
• Time-series graphs
• Traffic composition: finally you know what uses Abilene
• Salient points:
– Median bulk TCP throughput is around 2.5 Mb/s
– 95th percentile is around 15 Mb/s
– A decreasing fraction of traffic is file sharing
– Bulk TCP throughput appears to be increasing
5
![Page 7: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/7.jpg)
Top 10 Connections
• Top 10 bulk TCP performers
• Only a single connection from a given AS to a given AS can
be listed
• Two independent table are produced: one for measurement
flows and one for the rest
• If you are going for records, check if your flows show up
– Validation
– Independent verification
6
![Page 8: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/8.jpg)
Costs, Tools Used
• Capacity overhead of data collection is negligible
• Need a machine with disk space ($100 for 40 GB now)
• FOTS (free off the shelf) flow-tools for collection
• Custom-written stuff for analysis (around 2 man-months)
– CWEB program to make a pass over complete data set
– Perl programs to post-process and handle presentation
• CWEB part is available as documentation of classification
• Perl part can be released if there’s interest
7
![Page 9: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/9.jpg)
8
![Page 10: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/10.jpg)
Throughput Distribution of Bulk TCPs
• The shape of the curve mostly similar for different weeks.
• Sometimes one encounters unusual shapes.
– Denial of service attacks
– Sets of major demos
• The tail wiggles more than the body
• The virtually straight (on log-log scale) line from 0 to 130 Mb/s
or so virtually every week
– No explanation
– No “theoretic” shape for this curve
9
![Page 11: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/11.jpg)
10
![Page 12: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/12.jpg)
Median, 5th, and 95th Percentile of Bulk TCPThroughput
• Good news: keeps increasing (generally)
• Higher percentiles go up more than median
– The high-end users had better luck than the masses
– The wizard gap is widening
• But:
– Impact of Cisco to Juniper change (different eviction time-
outs)
– Mostly can be explained by changes in OS composition
(newer OSes have larger window size)
– Impact of file-sharing decreases (file-sharing was always
below median, so decreases of file-sharing translate to
increases of the median)
11
![Page 13: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/13.jpg)
12
![Page 14: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/14.jpg)
Median of Bulk TCP Throughput
• Same as on previous plot, but less compressed
• Clearly the trend is up
13
![Page 15: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/15.jpg)
14
![Page 16: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/16.jpg)
Percentage of Data Transfer Traffic
• NNTP, HTTP, FTP, and Rsync
• Passive FTP not included (difficult to characterize)
• Goes up in summer and during winter break (because there’s
less file sharing and interactive use)
• Fairly stable otherwise
15
![Page 17: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/17.jpg)
16
![Page 18: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/18.jpg)
Percentage of File-Sharing Traffic
• The general trend is downwards
• A lot of shifting to new applications
• Some must have spilled over into unidentified
• Correlates well with political and legislative events
17
![Page 19: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/19.jpg)
18
![Page 20: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/20.jpg)
Percentage of Measurement Traffic
• Iperf, ICMP, IPMP
• A sizeable chunk of network capacity
• Probably more than desired
19
![Page 21: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/21.jpg)
20
![Page 22: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/22.jpg)
Percentage of Encrypted Traffic
• SSH, HTTPS, IPsec
• General trends seems to be up
• Looking at port numbers, so don’t know what fraction of
unidentified is encrypted
21
![Page 23: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/23.jpg)
22
![Page 24: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/24.jpg)
Percentage of Audio/Video Traffic
• Multicast, Real, Windows Media, etc.
• More volatile than other traffic categories
• Event-related
• Windows Media a minor (but increasing) fraction of Real
23
![Page 25: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/25.jpg)
24
![Page 26: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/26.jpg)
Percentage of Advanced Applications Traffic
• UNIDATA LDM, BBFTP, IBP, GsiFTP, McIDAS
• Mostly LDM with a smidgen of BBFTP
• Even more volatile than audio/video
• The fewer users, the more volatility, generally
25
![Page 27: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/27.jpg)
26
![Page 28: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/28.jpg)
Percentage of Games Traffic
• Not a big traffic source
• Games are generally design for the masses
• The masses have DSL or cable at best
• Most games are designed so that the bandwidth of dialup is
enough
• Advanced application waiting to happen?
27
![Page 29: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/29.jpg)
28
![Page 30: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/30.jpg)
Percentage of Miscellaneous Traffic
• Mail, Port 0, AFS, DNS, X11, AIM, Telnet, MS Windows,
Squid, NFS, SOCKS, IRC, IDENT, NTP, SNMP, Portmap-
per, RTIP
• Known traffic that doesn’t fit other categories
• Quite stable
29
![Page 31: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/31.jpg)
30
![Page 32: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/32.jpg)
Percentage of Unidentified Traffic
• Quite smoothly changing
• Seems to be negatively correlated with file-sharing in winter
• Seems to be positively correlated with file-sharing in summer
• An unknown fraction might be file-sharing
31
![Page 33: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/33.jpg)
32
![Page 34: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/34.jpg)
Percentage of NNTP Traffic
• The most talkative single application on Internet2
• Some percentage swings are related to file-sharing changes
• Most bytes are in binary groups
33
![Page 35: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/35.jpg)
34
![Page 36: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/36.jpg)
Percentage of HTTP Traffic
• Less on Internet2 than on commodity Internet
• Not a lot of traffic (in relative terms), huge utility
– Email has even less traffic and even more utility
• The upward trend is mostly just the decrease in other types
of traffic
• Is likely to decrease over time (but likely retain the utility, if
similar to email)
35
![Page 37: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/37.jpg)
36
![Page 38: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/38.jpg)
Percentage of FTP Traffic
• Active FTP only
• Passive FTP is a part of unidentified
• Peculiarly, percentage decreases during breaks
– Therefore, mostly used interactively
37
![Page 39: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/39.jpg)
38
![Page 40: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/40.jpg)
Percentage of BitTorrent Traffic
• A relatively new file-sharing application
• Open-source
• Originally developed for the distribution of Linux CD images
• Used for other kinds of files nowadays
• Seems to become more and more popular—fast
39
![Page 41: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/41.jpg)
40
![Page 42: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/42.jpg)
Percentage of FastTrack Traffic
• The protocol used by KaZaa
• Was virtually gone in the end of summer
• Made a small comeback early fall
• Unlikely to come back big, at least in the original form
41
![Page 43: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/43.jpg)
42
![Page 44: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/44.jpg)
Percentage of eDonkey2000 Traffic
• A second-tier file-sharing application
• Remarkably resilient and stable
• Other file-sharing applications come and go, but eDonkey is
still here
43
![Page 45: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/45.jpg)
44
![Page 46: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/46.jpg)
Percentage of Gnutella Octets
• A file-sharing application
• Used to be a KaZaa competitor
• Steadily losing popularity
• Unlikely to come back
45
![Page 47: Internet2 NetFlow Weekly ReportsPerl part can be released if there’s interest 7. 8. Throughput Distribution of Bulk TCPs The shape of the curve mostly similar for di erent weeks.](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0c59547e708231d434f597/html5/thumbnails/47.jpg)
Summary
• Performance is going up
• Wizard gap is widening
• Quantity of file-sharing is going down
• Top 10 tables can help in validation and verification of test
results
• http://netflow.internet2.edu/weekly/
46