Proposed Diffserv Evaluation for Emerge Sites Volker Sander [email protected] Argonne National...
-
Upload
marvin-warren -
Category
Documents
-
view
220 -
download
0
Transcript of Proposed Diffserv Evaluation for Emerge Sites Volker Sander [email protected] Argonne National...
Proposed Diffserv Evaluation for Emerge
SitesVolker Sander
Argonne National Laboratory1/20/2000
2
Outline
Basic GARA overview New GARA-features Basic set of experiments Installation of GARA Running the experiments using GARA Timeline
3
The Basic Diffserv Concept
Aggregate based scheduling implements EF PHB
Classify and Mark
packets
Shape Traffic
Flow based
Classify and Mark
packets
Shape Traffic
4
Proposed Implementation
Client
GARA API
GARA Diffserv
Resource Manager
Server
5
The Multidomain Extension
Server
GARA Diffserv
Resource
Manager
Client
GARA API GARA Diffserv
Resource
Manager
6
GARA Globus Architecture for Reservation and
Allocation Three important contributions:
– Support for Advance Reservations (AG)
– Uniform treatment of underlying resources> Network, disk, CPU etc…
> Currently supported:
Differentiated Services, DPSS, and DSRT
– Layered architecture enables generic co-reservation agents
> User is able to select API best suitable for his need
7
GARA Basics—Reservations There is a generic “reservation”, which has
several properties:– Start Time (“now” or future) and Duration
– Resource type/Underlying resource identifier
– Resource-specific (bandwidth, % CPU…) All reservations are treated uniformly:
– Create/Modify (Given properties)=> Returns Reservation Handle
– Destroy
– Monitor (Callbacks or Polling)
8
A picture of what happens(creating a reservation)
Gatekeeper(Authenticate and Authorize)
CreateUser Program or Agent
Reservation Handle
Resource Manager
Gatekeeper Service
Local Resource Manager API (LRAM)
Resource (Router...)
9
A picture of what happens(claiming a reservation)
User Programbind(handle)
LRAM API
Resource Manager
Setup reservation
ResourceEnforce Reservation
Note: GARA supports third party binding
10
11
Example: Network reservation In advance, make a reservation:
RSL:& (reservation-type=network)
(start-time=now) (duration=3600)
(endpoint-a=128.135.11.4) (endpoint-b=128.135.11.1)
(bandwidth=1000) (protocol=udp)
(reservation-subtype=low-latency)
Handle: Opaque string H1
At run-time:
– Bind(H1,RSL)
RSL:& (which-endpoint=a) (endpoint-a-port=5000)
(endpoint-b-port=5000)
12
The Resource Manager To track advanced reservations, we use a simple “slot
table” We assume exclusive access to resource through the
resource manager We aren’t tied to this particular resource manager
% of available resource allocated
0%
100%
Time
13
End-to-End Network Reservations Algorithm for a single network reservation
– Discover which gatekeepers/resource managers need to be contacted (MDS)
– Make reservation at each one.
User process Workstation
Resource Manager Resource Manager
MDSEnd-to-End Co-Reservation Agent
14
The GARA API
globus_gara_reservation_create() Gatekeeper contact RSL reservation specification Reservation Handle or Error
globus_gara_reservation_modify() Old Reservation Handle RSL reservation specification New Reservation Handle or Error
15
The GARA API (continued) globus_gara_reservation_cancel()
Reservation Handle Error
globus_gara_reservation_status() Reservation Handle Status/Error
globus_gara_reservation_callback_register() Reservation Handle Callback Function/User Parameter Error
16
New GARA features New Reservation-Subtypes
– low-latency> Used to support jitter sensitive applications using the same EF
aggregate behavior as high-throughput TCP applications
> Based on traffic shaping and Priority-Queuing
> Currently being evaluated
– background> Used to support bulk transfers, including deadline support. Takes at
least the amount of premium traffic required to fulfill the deadline; more if not used actively
Monitoring of the network edges– Provides feedback to applications when they send too
fast
17
Network + CPU Reservations
0
20000
40000
60000
80000
100000
0 50 100 150 200 250
Time
Ban
dw
idth
(K
bp
s)
TCP Flow Competitive Traffic
18
Example of bulk transfer (LAN)
0
20000
40000
60000
80000
100000
0 50 100 150 200 250
Time
Ban
dw
idth
(K
bp
s)
backgroundforegroundcompetitive
19
Example of bulk transfer (WAN)
0
10000
20000
30000
40000
50000
0 20 40 60 80 100 120 140 160 180
Time
Ban
dw
idth
(K
bp
s)
backgroundforegroundcompetitive
20
Feedback from Monitoring the Edges
Receiver Bound BW
0
1000
2000
3000
4000
5000
6000
7000
8000
90001
11
21
31
41
51
61
71
81
91
10
1
11
1
12
1
13
1
14
1
15
1
16
1
17
1
18
1
Seconds
KB
/s
Series2
21
Evaluation Tools UDP Traffic Generator
– Modified Version of Andy Adamson’s gen_send and gen_recv
> Evaluate admission control
> Creating competing traffic
– MGEN/Drec> Evaluate delay and Jitter for Premium UDP Flow
Modified Version of ttcp – GARA-enabled (wait for reservation)
– Support for a desired application rate> long term controllong term control
> short term controlshort term control
– Consecutive bandwidth reporting
– Bulk transfer ttcp
22
Basic Experiments Evaluate Admission Control Evaluate premium TCP Flows under Congestion
– Analyze network traces with tcpdump Evaluate UDP
– Delay
– Jitter Evaluate Inter-Domain Issues
Fundamental idea: Get the necessary knowledge to be able to experiment with REAL applications
23
Basic Experiment I Goal: Proof of Admission Control
– Gather packet statistics from gen_recv for different configurations and different traffic profiles:
UDP Rate comparison for different CAR buffer sizes(Desired rate: >40Mbps; Average rate limit 40 Mbps)
Packet size 530 Bytes
36500
37000
37500
38000
38500
39000
39500
40000
40500
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
Time (s)
Ac
tua
l R
ate
(K
bp
s)
100KB
1MB
2MB
Buffer size
24
Basic Experiment I
gen_send
GARA Diffserv
Resource Manager
gen_recv
25
Basic Experiment II Goal: Proof of Premium Quality for TCP flows
– Run modified ttcp with different traffic and reservation profiles; gather receiver bound statistics
– Create congestion using gen_send/recvStable Long Term TCP Traffic
(Attempted Rate: 6000KB/s, Buffer size: 2000000 bytes)
2000
3000
4000
5000
6000
7000
8000
9000
1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97Time
Ac
tua
l R
ate
(K
bs
)
Rate
Limit
26
Basic Experiment II Enhancement
– Use tcpdump and tcptrace get a deep overview on the behavior
– Use netlogger_enabled ttcp_gara and gen_recv to publish data into MDS. “Real Time Visualization”
27
Basic Experiment II
GARA Diffserv
Resource Managerttcp -t
ttcp -r
gen_send gen_recv
28
Basic Experiment III Goal: Demonstrate Premium Quality for UDP flows
– Run mgen/drec with different traffic and reservation profiles; gather receiver bound statistics
– Create congestion using gen_send/recv
29
Basic Experiment III
GARA Diffserv
Resource Manager
mgen
drec
gen_send gen_recv
30
Basic Experiment IV Goal: Proof Premium Quality for UDP flows
sharing one EF aggregate– Run mgen/drec with different traffic and
reservation profiles; gather receiver bound statistics
– Run high-throughput ttcp using a correct reservation
– Create congestion using gen_send/recv
31
Basic Experiment IV
GARA Diffserv
Resource Manager
ttcp -t
ttcp -r
mgen drec
gen_recv
gen_send
32
Additional Experiments
Bulk-Transfer Reevaluate inter-domain behavior
– Add Traffic-Shaping between domains Test Inter-Domain Admission control Prepare the Instrumentation Go with real applications
33
Installing GARA Get GARA
– (currently through E-Mail from sander/roy @mcs.anl.gov) Get SSLeay 0.9.0 and install it Get Globus 1.1 configure and compile it
deploy is not necessary for doing tests but recommended
Run grid-cert-request to apply for a Grid X509 Certificate
add “gara-service stderr_log,local_cred - $GARA_DIR/gara/programs/globus_gatekeeper_gara_service -d”
to $GLOBUS_DEPLOY_DIR/etc/globus-services Read gara/docs/* Edit gara/build Run gara/build Customize GARA
– gara/resource_manager/programs/diffserv_manager.conf
– gara/resource_manager/programs/setup_flow.cfg
– (Samples are in the documentation directory)
34
Building GARA $GARA_DIR/build #!/bin/ksh CC=cc CXX=CC CFLAGS="-g -v -mt" GLOBUS_DIR=/soft/pub/packages/globus-1.1.0/development/sparc-sun-solaris2.7_pt hreads_standard_debug SSL_LIBRARY_DIR=/soft/pub/packages/SSLeay-0.9.0/lib
export CC CXX CFLAGS GLOBUS_DIR SSL_LIBRARY_DIR
configure make clean make depend make all # # For portability reasons you might not want to do a make withdsrt.. # in case: just comment out this line. #make withdsrt
Run build
35
Configuring GARA resource_manager/programs/diffserv_manager.confPublicationMethod web mds
LDAPDistinguishedName "dslnet1.mcs.anl.gov:34880:/C=US/O=Globus/O=Argonne Nati
onal Laboratory/OU=MCS/CN=roy”
MDSPublishPassword “password”
WEBPublishFile /home/sander/public_html/diffserv_manager_slots.html
NoOfRouters 2
IPAddressesServed[1] 140.221.48.162,140.221.48.146,140.221.48.164,140.221.48.1
65,140.221.48.166,140.221.48.167,140.221.48.34
IPAddressesServed[2] 140.221.48.98,140.221.48.114,140.221.48.102,140.221.48.10
3,140.221.48.104,140.221.48.105
Parameters: "LDAPDistinguishedName",
"MDSPublishPassword",
"WEBPublishFile",
"FILEPublishFile",
"Quantity",
"ClearSlotTable",
"SlotTableFilename",
"Verbose",
"PublicationMethod",
"DontReallyBindReservations"
36
Configuring GARA resource_manager/programs/setup_flow.conf1_ROUTER 140.221.48.145
1_USERNAME my_userid
1_PASSWORD password
1_INTERFACE1 FastEthernet1/0/1 140.221.48.162
1_INTERFACE2 FastEthernet1/0/1 140.221.48.164
1_INTERFACE3 FastEthernet1/0/1 140.221.48.165
1_INTERFACE4 FastEthernet1/0/1 140.221.48.166
1_INTERFACE5 FastEthernet1/0/1 140.221.48.167
1_INTERFACE6 FastEthernet1/0/0 140.221.48.146
#
#
2_ROUTER baku
2_USERNAME my_userid
2_PASSWORD password
2_INTERFACE1 FastEthernet1/0/1 140.221.48.98
2_INTERFACE2 FastEthernet1/0/1 140.221.48.102
2_INTERFACE3 FastEthernet1/0/1 140.221.48.103
2_INTERFACE4 FastEthernet1/0/1 140.221.48.104
2_INTERFACE5 FastEthernet1/0/1 140.221.48.105
2_INTERFACE6 FastEthernet1/0/0 140.221.48.114
37
Setting up the routers Get new IOS XE beta release
IOS (tm) RSP Software (RSP-PV-M), Experimental Version 12.0(19991129:193607) [rolsen-conn_isp_llq 144]
Available at ftp://ftpeng.cisco.com/rolsen/rsp-pv-mz.120699
Edge-Routers (Update of setup_flow required)class-map match-all EF
match ip dscp 46
class-map match-all priority-qos
match qos-group 99
policy-map emerge
class EF
bandwidth 30000
queue-limit 500
class priority-qos
priority
policy-map set-precedence
– Attach service-policy set-precendence to every input
– Attach service-policy emerge to every output interface
38
Setting up the routers
Interior-Routers (Update of setup_flow required)class-map match-all EF
match ip dscp 46
policy-map emerge
class EF
bandwidth 30000
queue-limit 500
– Attach service-policy emerge to every output interface
39
Updating setup_flow
setup_flow is the script which is configuring the router Possible update of setup_flow (is)
197 expect_after default { close; continue }
198
199 expect "name: " { send "$username\r" }
200 expect "word: " { send "$password\r" }
Might be (without tacacs+)197 expect_after default { close; continue }
198
199 expect ”word: " { send "$password\r” }
200 expect “#” { send “enable\r” }
201 expect "word: " { send "$password\r" }
40
Updating the MDS
End-to-end API searches for end-systems in MDS Publish information for all end-systems into MDS
dslnet1.cldif:
---------------------
dn: hn=dslnet1.mcs.anl.gov, ou=Mathematics and Computer Science Division,
o=Argonne National Laboratory, o=Globus, c=US
…. If the machine is already in MDS this is enough
networkreservationcontact:
dslnet1.mcs.anl.gov:30000:/C=US/O=Globus/O=Argonne National
Laboratory/OU=MCS/CN=roy197 expect_after default { close; continue }
– run grid-info-update Tools will be available soon
41
The GARA Source Tree Subdirectories represent different layers
– end2end
– gara (gara/java GUI)
– lram
– resource_manager
– slot_manager
– common
– logging
– simple_multi
– (dsrt)
We are providing examples for every layer docs directory contains lots of useful
information
42
Doing Further Tests
Select your favorite application Determine port numbers edit (set endpoint systems) and run script:
gara/gara/utilities/demo_make_reservation Create congestion Try demo_modify_reservation for getting
different premium bandwidth Try demo_cancel_reservation Or use the GUI
43
Timeline Establish connectivity (January)
– mostly done!
– Setup accounts for Emerge-User (January!)
– Check connectivity Install and customize GARA (Jan - Feb)
– partly done
– Configure and check with GUI without any application (Just look at differv_manager and router)
Do basic experiments (February-March) Start with multi-domain issues (March-April) Port real applications to the Testbed (Feb-?)
44
Questions...
Now or later… (sander/roy @ mcs.anl.gov)