Post on 22-Dec-2015
University Computing ServicesUniversity Computing Services
EDUCAUSE Mid-Atlantic Regional Conference16 January 2003
An Infrastructure and An Infrastructure and Accounting Response to Accounting Response to
Peer to Peer Traffic VolumePeer to Peer Traffic Volume
Dr. Michael R MundraneDirector of Telecommunications
Rutgers University Computing Services
University Computing ServicesUniversity Computing Services
CopyrightCopyright
Copyright Michael R Mundrane 2003. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
University Computing ServicesUniversity Computing Services
AgendaAgenda
• Statement of Problem
• Objectives
• Approach
• Results
• Conclusions
University Computing ServicesUniversity Computing Services
Statement of ProblemStatement of Problem
Is he kidding? P2P is the problem!
University Computing ServicesUniversity Computing Services
Network EvolutionNetwork Evolution
• Sporadic
• Unequally funded
• Unstructured
• Immediacy
• Complex
• Point services
• Faculty centric
University Computing ServicesUniversity Computing Services
Application ModelsApplication Models
• Limited customer interface • Minimal administration• Centralized management• Centralized storage• hub and spoke infrastructure• Minimal bandwidth
Terminal Host
University Computing ServicesUniversity Computing Services
Application ModelsApplication Models
• Rich customer interface
• Medium administration
• Centralized management
• Hybrid storage (server and client)
• Tiered network infrastructure
• Bandwidth server/s dependant
Client Server
University Computing ServicesUniversity Computing Services
Application ModelsApplication Models
• Rich user interface
• High touch administration
• Distributed management (costly)
• Distributed storage (difficult to manage)
• Complex mesh infrastructure
• High bandwidth
Peer Peer
University Computing ServicesUniversity Computing Services
Cooperative?!?Cooperative?!?
A. Badges? We don’t see no stinking badges!
Q. Excuse me, would you please forward the business activity associated with your traffic so that we can adjust our records?
University Computing ServicesUniversity Computing Services
ObjectivesObjectives
More than near term survival!
University Computing ServicesUniversity Computing Services
Essential CharacteristicsEssential Characteristics
• Preserve behavior
• Ensure access
• Moderate impact
• Protect privacy
• Avoid value judgments
• Apply to new applications
University Computing ServicesUniversity Computing Services
AssumptionsAssumptions
• Large number of hosts
• Small number of problems
• Service consumers
• Many random light providers
• Few heavy providers
• Responsive community
University Computing ServicesUniversity Computing Services
Just Use Traffic ShapingJust Use Traffic Shaping
University Computing ServicesUniversity Computing Services
Just Use Traffic ShapingJust Use Traffic Shaping
• Cisco routers
• CAR – traffic class
• MicroCAR – identified flow
day
Gigabytes
bits
byte
M
G
K
M
day
onds
ond
32.1
8024,1024,1
sec400,86
sec
Kilobits128
University Computing ServicesUniversity Computing Services
Just Use QoSJust Use QoS
University Computing ServicesUniversity Computing Services
Just Use QoSJust Use QoS
• Classification
• Differentiation
• Admission control
• Provisioning
• Bandwidth
• Latency
• Jitter
University Computing ServicesUniversity Computing Services
QoS DifferentiationQoS Differentiation
P2P Other
10Mbit 90Mbit
University Computing ServicesUniversity Computing Services
QoS DifferentiationQoS Differentiation
10Mbit
Differentiation w/o admission control only
defers the problem!
University Computing ServicesUniversity Computing Services
Rutgers NetworkRutgers Network
• 40,000+ hosts
• 1200+ networks
• 200+ routers
• 17 zones
• 7 campuses
• 3 regions
• 1 autonomous system
University Computing ServicesUniversity Computing Services
ApproachApproach
No single solution!
University Computing ServicesUniversity Computing Services
Best Network PracticesBest Network Practices
• Modular
• Layered
• Aggregated
• Scalable
• Uniform
• Deterministic
• Comprehensible
University Computing ServicesUniversity Computing Services
DeviceDevice DeviceDevice
DeviceDevice DeviceDevice
Intra-building Backbone
Building
Intra-building BackboneIntra-building Backbone
RUNet ~ 1200
University Computing ServicesUniversity Computing Services
BuildingBuilding BuildingBuilding
BuildingBuilding BuildingBuilding
Inter-building Backbone
Zone
Inter-building BackboneInter-building Backbone
RUNet 17
University Computing ServicesUniversity Computing Services
ZoneZone ZoneZone
ZoneZone ZoneZone
Intra-campus Backbone
Campus
Intra-campus BackboneIntra-campus Backbone
RUNet 7
University Computing ServicesUniversity Computing Services
CampusCampus CampusCampus
CampusCampus CampusCampus
Inter-campus backbone
Region
Inter-campus BackboneInter-campus Backbone
RUNet 3
University Computing ServicesUniversity Computing Services
MANMAN MANMAN
MANMAN MANMAN
Inter-region Backbone
Autonomous System
Inter-region BackboneInter-region Backbone
RUNet 1
University Computing ServicesUniversity Computing Services
CharacteristicsCharacteristics
• Geographic independence
• Shallow topology
• Similar (not optimal) paths
• Low latency
• Uniform characteristics
• 1 autonomous system
University Computing ServicesUniversity Computing Services
Collect DataCollect Data
• Netflow
• Source/Destination address
• Source/Destination ports
• Protocol
• Packets/Octets/Flows
• Start/End time
University Computing ServicesUniversity Computing Services
Raw DataRaw Data
• 10 minute granularity
• Each source
• Each destination
• 1,000,000 addresses
• 10,000,000 records
• 1 Gigabytes, 1 day
University Computing ServicesUniversity Computing Services
Rollup DataRollup Data
• Rutgers sources/sinks
• Data >= 1024, 10 minutes
• Data >= 6*1024, 1 hour
• Data >= 24*6*1024, 1 day
• 20,000 unique hosts
• 20,000 records
• 1 Megabyte
University Computing ServicesUniversity Computing Services
Filtered DataFiltered Data
• Rutgers sources/sinks
• Data >= 512 Megabytes, 1 Day
• 125 unique hosts
• 125 records
• 50 Kilobytes
University Computing ServicesUniversity Computing Services
ReductionReduction
10,000,000 99.799%20,000 0.200%125 0.001%
10,020,125
Addresses
1,000,000 98.027%20,000 1.961%125 0.012%
1,020,125
Records1,073,741,824 99.898%
1,048,576 0.098%51,200 0.005%
1,074,841,600
Size
1,000 90.090%100 9.009%10 0.901%
1,110
Model
University Computing ServicesUniversity Computing Services
DistributionDistribution
• Reread entire data set
• Limit to filtered only
• Rollup based on external address
• Preserve individual distributions
• Useful to reduce contact
University Computing ServicesUniversity Computing Services
Questionable DistributionQuestionable Distribution
University Computing ServicesUniversity Computing Services
Good DistributionGood Distribution
University Computing ServicesUniversity Computing Services
Storage
Process ModelProcess Model
Rollup
Internet
NetflowFilterDistribution
Analyze
University Computing ServicesUniversity Computing Services
Residence AssumptionsResidence Assumptions
• RFC1918 address space
• Large number of hosts
• Small number of problems
• Service consumers
• No service providers
• Unresponsive community
University Computing ServicesUniversity Computing Services
Set LimitsSet Limits
• 2048 MB download
• 512 MB upload
• 7 day granularity
• Sliding window
• Enforcement
University Computing ServicesUniversity Computing Services
ReferenceReference
• 4 movies
• 400 songs
• 45,000 web pages
• 2048 Megabytes
University Computing ServicesUniversity Computing Services
Oracle
Process ModelProcess Model
Table
Rollup
Table
Enforce
Table
GatherInternet
Netflow
WWW
Custom ACL
University Computing ServicesUniversity Computing Services
Traffic ShapingTraffic Shaping
• 1 Day on
• 7 Days off
• Multiplexed
• 1:8 ratio
• Automatic
• Aggregated
• Not legalistic
Load
Impact
University Computing ServicesUniversity Computing Services
Differentiated ServiceDifferentiated Service
• Residence facilities
• Other locations
• Two traffic classes
• 1:2 host distribution
• 1:1 bandwidth allocation
• CAR enforced
University Computing ServicesUniversity Computing Services
ResultsResults
Some pains, some gains!
University Computing ServicesUniversity Computing Services
Extra EffortsExtra Efforts
• Registration
• Port Address Translation
• Split horizon DNS
• Help desk/Appeals
• Address hopping
• Proxy services
• Oracle
University Computing ServicesUniversity Computing Services
90% Data Sinks90% Data Sinks
University Computing ServicesUniversity Computing Services
99.99% Data Sinks99.99% Data Sinks
University Computing ServicesUniversity Computing Services
90% Data Sources90% Data Sources
University Computing ServicesUniversity Computing Services
99.99% Data Sources99.99% Data Sources
University Computing ServicesUniversity Computing Services
Internet TrafficInternet Traffic
University Computing ServicesUniversity Computing Services
ConclusionsConclusions
• Modest applications with broad demographics have profound impact.
• Students have free time.
• Network best practices never more important.
• Cooperative generic methods can be effective (w/ encouragement).
• No magic bullet.
University Computing ServicesUniversity Computing Services
Questions?
mundrane@td.rutgers.edu