Licentiate Seminar: On Measurement and Analysis of Internet Backbone Traffic Wolfgang John...
-
Upload
cuthbert-gregory -
Category
Documents
-
view
212 -
download
0
Transcript of Licentiate Seminar: On Measurement and Analysis of Internet Backbone Traffic Wolfgang John...
Licentiate Seminar:
On Measurement and Analysis of Internet Backbone Traffic
Wolfgang John Department of Computer Science and Engineering
Chalmers University of TechnologyGöteborg, Sweden
2008-02-29Licentiate Seminar Wolfgang John
Internet, 1983Internet, 2005
Why measure Internet traffic? (1)
The Internet is changing in size
ARPANET, 1969
2008-02-29Licentiate Seminar Wolfgang John
The Internet is changing in application
Why measure Internet traffic? (2)
2008-02-29Licentiate Seminar Wolfgang John
• The Internet– is constantly developing– is used differently in different locations– is heterogeneous
The Internet is not understood in its entirety!
INTERconnected NETworks
Why measure Internet traffic? (3)
INTER NET
2008-02-29Licentiate Seminar Wolfgang John
• Operational purpose– Troubleshooting, provisioning, planning ….
• Scientific purpose– Protocols, infrastructure and services
– Performance properties
– Internet simulation models
– Security measures
Why measure Internet traffic? (4)
2008-02-29Licentiate Seminar Wolfgang John
Thesis Objectives
1. Guidelines for Internet measurement
2. Current traffic characteristics
3. Traffic decomposition
4. Inconsistent behavior
2008-02-29Licentiate Seminar Wolfgang John
Outline
• Measurement approaches
• Internet measurement challenges
• The MonNet project
• Scientific contribution
• Results– Four studies included
• Conclusions
Measurement
Analysis
2008-02-29Licentiate Seminar Wolfgang John
Measurement approaches
Network traffic measurement
Active Passive
Software Hardware
Online Offline
Flows Packets
Complete Headers
Different protocol levels
Statistical summaries
Transport layer
2008-02-29Licentiate Seminar Wolfgang John
Internet measurement challenges (1)
• Legal considerations
• Ethical and moral considerations
• Operational considerations
• Technical considerations
2008-02-29Licentiate Seminar Wolfgang John
Measurement challenges (3)
Technical considerations
• Data amount– Exhausting I/O and storage access speeds
• Data reduction techniques– Filtering, sampling, packet truncation
• Timing– Clock synchronization
2008-02-29Licentiate Seminar Wolfgang John
The MonNet Project (1)
Technical Solution
10 GbpsGöteborg
splitterBorås
10 Gbps
Processing Platform and Storage
Measurement Node 2
Measurement Node 1
2008-02-29Licentiate Seminar Wolfgang John
The MonNet Project (2)
Internet
Internet
Regiona
l ISPsRegiona
l ISPs
Göteborg
Stockholm
Other smaller Univ. and Institutes
Göteborgs Univ.
Student-Net
Chalmers Univ.
Measurement location
Borås
•April 2006 148 traces (20 minutes) 11 billion packets, 7.6 TB of data
•Sept. – Nov. 2006 554 traces (10 minutes) 28 billion packets, 19.5 TB of data
2008-02-29Licentiate Seminar Wolfgang John
Scientific ContributionLevel of
complexity
Quantification of inconsistent
behavior
Traffic characterization
Packet level Flow level Traffic classes
Stu
dy I
Stu
dy II
Stu
dy IV
Stu
dy III
Upcoming
2008-02-29Licentiate Seminar Wolfgang John
Study I: Packet Level Analysis
• Updated packet-level characteristics of Internet traffic
• Inconsistencies in headers will appear
– Network attacks and malicious traffic– Active OS fingerprinting– Buggy applications or protocol stacks
2008-02-29Licentiate Seminar Wolfgang John
• High level analysis does not necessarily show differences → detailed analysis does!
• 2 main reasons for directional differences: – Malicious traffic
• the Internet is “unfriendly”
– P2P• Göteborg is a P2P source• P2P is changing traffic characteristics
e.g. packet sizes, TCP termination, TCP option usage
Study II: Flow level analysis
2008-02-29Licentiate Seminar Wolfgang John
Study III: Classification Method (1)
• Classification of flow traffic without payload
• Heuristics to identify nature of endpoints
• Rules based on connection patterns and port numbers– 5 rules for P2P traffic
– 10 rules to classify other types of traffic• remove ‘false positives’ from P2P
2008-02-29Licentiate Seminar Wolfgang John
Study III: Classification Method (2)
# connections in 106 Amount of data in TB
Comparison of classification methods for P2P traffic
2008-02-29Licentiate Seminar Wolfgang John
Study III: Classification Method (3)
• Previous classification methods on packet header traces don’t work well on backbone data
• Proposal of refined and updated heuristics– Simple and fast method to decompose traffic– No payload required– Effectively used even on short traces (10 min)
• 0.2% of the data left unclassified
2008-02-29Licentiate Seminar Wolfgang John
Study IV: Classification Results (1)
Tuesday, 18.04.2006
2008-02-29Licentiate Seminar Wolfgang John
Study IV: Classification Results (2)
Application breakdown April till Nov. 2006
2008-02-29Licentiate Seminar Wolfgang John
Study IV: Classification Results (3)
Connection establishment for traffic classes
2008-02-29Licentiate Seminar Wolfgang John
Study IV: Classification Results (4)
• Behavior of P2P traffic– Unsuccessful TCP connection attempts increasing
– Serving peers terminate with FIN and RSTDecreased from 20% to 8%
– UDP overlay traffic doubled
• TCP options deployment differs– P2P behaves as expected
– Web traffic shows artifacts of client-server pattere.g. popular web-servers neglecting SACK option
2008-02-29Licentiate Seminar Wolfgang John
Summary
1. Guidelines for Internet measurement• Experiences of the MonNet project
2. Current traffic characteristics• Packet and flow level
3. Traffic decomposition• Traffic classification method
4. Inconsistent behavior• Packet header anomalies
• Malicious traffic flows
2008-02-29Licentiate Seminar Wolfgang John
General remarks
• Internet today is essential, but still not understood entirely• Large-scale traffic measurements uncommon
– A lot of analysis is done on outdated datasets
• Each study generated as much questions as answers• Reconsider measurement process (duration, payload…)• A lot of open questions …
…get more answers in two years…