July 2000 PPNCG Meeting R. Hughes-Jones Performance Measurements of LANs MANs and SuperJANET III...
19
July 2000 PPNCG Meeting R. Hughes-Jones Performance Measurements of LANs MANs and SuperJANET III This is PRELIMINARY Baseline data for Grid development Latency Throughput / Bandwidth Packet Jitter Performance of 2 Mbit Managed Bandwidth Link TCP/IP Stream performance
-
date post
22-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of July 2000 PPNCG Meeting R. Hughes-Jones Performance Measurements of LANs MANs and SuperJANET III...
July 2000 PPNCG Meeting R. Hughes-Jones
Performance Measurements ofLANs MANs and SuperJANET III
This is PRELIMINARY Baseline data for Grid development Latency Throughput / Bandwidth Packet Jitter Performance of 2 Mbit Managed Bandwidth Link TCP/IP Stream performance
July 2000 PPNCG Meeting R. Hughes-Jones
Latency Round trip times measured using Request-Response UDP frames Latency as a function of frame size
Slope gives sum of individual data transfer rates end-to-end
Histograms of individual measurements On LAN Latency histograms ~ 2 s wide
PC-eth sw-PC
y = 0.1077x + 109.3
y = 0.1872x + 127.7
y = 0.0802x + 17.987
y = 0.108x + 244.94
y = 0.1088x + 107.15
0
100
200
300
400
500
600
700
0 500 1000 1500 2000 2500 3000
Message length bytes
La
ten
cy u
s
0
5
10
15
20
25
30
35
40
45
50
Cu
mla
tive
nu
mb
er
of
tim
eo
uts
PC-HEP switch-PCPC=PCdt PC-HEP switch-PCCumlative timeoutsLinear (Series5)Linear (Series4)Linear (dt PC-HEP switch-PC)Linear (Series7)Linear (Series8)
PC software 0.01 s/byte
PC-eth sw-Sun
y = 0.2642x + 127.74
y = 0.107x + 109.71
y = 0.3734x + 236.27
0
100
200
300
400
500
600
700
800
900
0 200 400 600 800 1000 1200 1400
Message length bytes
La
ten
cy u
s
PC-sw-SunPC=PCdt PC-sw-Sun
Sun software 0.036 s/byte
July 2000 PPNCG Meeting R. Hughes-Jones
Latency: Man-DL
ATM MAN 7 routers ~0.8ms / router Router+link 0.4 s/byte DL-Liv ~10 ms rtt - agreement
y = 2.2589x + 5643.6
y = 2.1519x + 5533.9
y = 0.107x + 109.71
0
2000
4000
6000
8000
10000
12000
0 500 1000 1500 2000 2500 3000
Message length bytes
La
ten
cy u
s
0
50
100
150
200
250
300
350
400
450
500
La
ten
cy f
or
PC
=P
C u
s
PC-SJIII-DL PCdt PC-SJIII-DL PCPC=PC
64 bytes pc-SJ-DL req-resp
0
100
200
300
400
500
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
256 bytes pc-SJ-DL req-resp
0
100
200
300
400
500
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
1024 bytes pc-SJ-DL req-resp
0
100
200
300
400
500
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)1500 bytes pc-SJ-DL req-resp
0
100
200
300
400
500
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
512 bytes pc-SJ-DL req-resp
0
100
200
300
400
500
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
Sigma ~12 ms
July 2000 PPNCG Meeting R. Hughes-Jones
Latency: Man-IC
Sigma decreases with size
3.4 -> 1.7 ms Packet loss falls 5 -> 1 %
64 bytes pc-SJ-ICpc req-resp
020406080
100120140
15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
1500 bytes pc-SJ-ICpc req-resp
050
100150200250300350
15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
250 bytes pc-SJ-ICpc req-resp
0
20
40
60
80
100
120
15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
512 bytes pc-SJ-ICpc req-resp
0
50
100
150
200
15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
512 bytes pc-SJ-ICpc req-resp
0
50
100
150
200
250
15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
July 2000 PPNCG Meeting R. Hughes-Jones
Latency: Man-UCL
Slope 1.84 s/byte Sigma decreases with size
3.7 -> 1.7 ms Packet loss falls 3 -> 1 %
y = 1.8391x + 17321
y = 0.107x + 109.71
0
5000
10000
15000
20000
25000
0 200 400 600 800 1000 1200 1400
Message length bytes
La
ten
cy u
s
0
50
100
150
200
250
300
350
400
450
500
La
ten
cy f
or
PC
=P
C u
s
man-ucldt man-uclPC=PC
64 bytes pc-SJ-DL req-resp
0
100
200
300
400
10000 15000 20000 25000 30000 35000 40000
Latency us
N(t
)
256 bytes pc-SJ-DL req-resp
0
100
200
300
400
500
600
10000 15000 20000 25000 30000 35000 40000
Latency us
N(t
)
1024 bytes pc-SJ-DL req-resp
0
200
400
600
800
10000 15000 20000 25000 30000 35000 40000
Latency usN
(t)
1500 bytes pc-SJ-DL req-resp
0
200
400
600
800
1000
10000 15000 20000 25000 30000 35000 40000
Latency us
N(t
)
512 bytes pc-SJ-DL req-resp
050
100150200250300350
10000 15000 20000 25000 30000 35000 40000
Latency us
N(t
)
July 2000 PPNCG Meeting R. Hughes-Jones
Latency: Man-GLA
Sigma decreases with frame size 2.6 -> 2.3 ms
NO Packet loss
64 bytes Man_Gla req-resp
0
50
100
150
200
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
1400 bytes Man_Gla req-resp
0
50
100
150
200
250
0 5000 10000 15000 20000 25000 30000 35000
Latency us
N(t
)
256 bytes Man_Gla req-resp
0
50
100
150
200
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
512 bytes Man_Gla req-resp
0
50
100
150
200
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
1024 bytes Man_Gla req-resp
0
50
100
150
200
250
0 5000 10000 15000 20000 25000 30000 35000
Latency us
N(t
)
y = 0.562x + 17019
y = 1.5487x + 11383
0
5000
10000
15000
20000
25000
0 200 400 600 800 1000 1200 1400
Message length bytes
La
ten
cy u
s
0
100
200
300
400
500
600
700
800
900
1000
PC
to
PC
La
ten
cy u
s
PC-SJIII-Gla PC min timePC=PCLinear ( min time)
DL slope 2.5 s/byte cf min time slope 1.5 s/byte
July 2000 PPNCG Meeting R. Hughes-Jones
64 bytes pc-SJIII-sun req-resp
0
50
100
150
200
250
300
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
256 bytes pc-SJIII-sun req-resp
050
100150200250300350
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
1024 bytes pc-SJIII-sun req-resp
0
50
100
150
200
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
1500 bytes pc-SJIII-sun req-resp
0
50
100
150
200
250
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
512 bytes pc-SJIII-sun req-resp
0
50
100
150
200
250
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
y = 2.9022x + 24016
0
10000
20000
30000
40000
50000
60000
0 200 400 600 800 1000 1200 1400
Message length bytes
La
ten
cy u
s
PC-SJIII-RALSunPC-MB-RALSun
Latency: Man-RAL
Sigma 4.5 -> 6.7 ms NO Packet loss
July 2000 PPNCG Meeting R. Hughes-Jones
y = 6.488x + 16071
y = 6.1089x + 15838
y = 0.3734x + 236.27
0
5000
10000
15000
20000
25000
30000
35000
0 200 400 600 800 1000 1200 1400
Message length bytes
La
ten
cy u
s
0
200
400
600
800
1000
1200
1400
1600
1800
2000
La
ten
cy P
C=
Su
n u
s
PC-MB-RALSundt PC-MB-RALSunPC-sw-Sun
Latency: Man-RAL Managed BW64 bytes pc-SJIII-sun req-resp
0
100
200
300
400
500
600
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
256 bytes pc-SJIII-sun req-resp
0
100
200
300
400
500
600
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
1024 bytes pc-SJIII-sun req-resp
0
100
200
300
400
500
600
10000 15000 20000 25000 30000 35000 40000 45000
Latency usN
(t)
1500 bytes pc-SJIII-sun req-resp
0100200300400500600700
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
512 bytes pc-SJIII-sun req-resp
0
100
200
300
400
500
600
10000 15000 20000 25000 30000 35000 40000 45000
Latency us
N(t
)
5 routers ~0.8ms / router Slope 6.5 s/byte A 2Mbit link uses 4 s/byte NO Packet loss Sigma ~ 2.0 ms Sigma 4.5 ->6.7 ms
July 2000 PPNCG Meeting R. Hughes-Jones
Latency: Man-MC Pink main campus router Blue NNW – SuperJANET
Nice exponential - queuing
64 bytes pc-MC req-resp
0
100
200
300
400
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
gw
-me
tro
0
500
1000
1500
2000
2500
N(t
) g
w-m
c
1400 bytes pc-MC req-resp
0
100
200
300
400
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
gw
-me
tro
0
500
1000
1500
2000
N(t
) g
w-m
c
256 bytes pc-MC req-resp
0
100
200
300
400
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
gw
-me
tro
0
500
1000
1500
2000
2500
N(t
) g
w-m
c
512 bytes pc-MC req-resp
0
100
200
300
400
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
gw
-me
tro
0
500
1000
1500
2000
2500
3000
N(t
) g
w-m
c
1024 bytes pc-MC req-resp
0
100
200
300
400
0 5000 10000 15000 20000 25000 30000
Latency us
N(t
)
gw
-me
tro
0
500
1000
1500
2000
2500
N(t
) g
w-m
c
July 2000 PPNCG Meeting R. Hughes-Jones
Bandwidth
Send a burst of UDP frames spaced at regular intervals Vary the frame size and the frame transmit spacing Record
The time to send and the time to receive the frames The number received, the number lost, number out of order
Zero stats
OK done
●●●
Get remote statistics
Send statistics
Send data frames atregular intervals ●●●
Time to send
Time to receive
July 2000 PPNCG Meeting R. Hughes-Jones
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Re
cve
ive
Tim
e/p
ack
et
us
100 bytes 200 bytes
400 bytes 600 bytes
800 bytes 1000 bytes
1200 bytes 1400 bytes
Bandwidth: Man-DL
ATM MAN 10 Mbit link into the ICFA PC No frames lost when not queuing
0
2
4
6
8
10
12
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Re
cv W
ire
Ra
te M
bit
/s
100 bytes 200 bytes 400 bytes 600 bytes800 bytes1000 bytes1200 bytes 1400 bytes
July 2000 PPNCG Meeting R. Hughes-Jones
0
5
10
15
20
25
30
35
40
45
50
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Recv
Wire
Rat
e M
bits/
s
100 bytes 200 bytes400 bytes 600 bytes800 bytes 1000 bytes1200 bytes 1400 bytes
Bandwidth: Man-IC
NNW – ATM WAN – London Man 1-2% frame loss when not queuing for all frame sizes
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Recv
Tim
e/pa
cket
us
100 bytes 200 bytes400 bytes 600 bytes800 bytes 1000 bytes1200 bytes 1400 bytes
July 2000 PPNCG Meeting R. Hughes-Jones
0
2
4
6
8
10
12
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Recv
Wire
rate
Mbi
ts/s
100 bytes 200 bytes400 bytes 800 bytes600 bytes 1000 bytes1200 bytes 1400 bytes
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Recv
eive
Tim
e/pa
cket
us
100 bytes 200 bytes400 bytes 800 bytes600 bytes 1000 bytes1200 bytes 1400 bytes
Bandwidth: Man-UCL
NNW – ATM WAN – London Man Suspect a 10 Mbit link at UCL 1-2% frame loss when not queuing for all frame sizes IC-UCL frame loss ~0.1%
July 2000 PPNCG Meeting R. Hughes-Jones
0
5
10
15
20
25
30
35
40
45
50
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Recv
Wire
Rat
e M
bit/s
100 bytes 200 bytes400 bytes 600 bytes800 bytes 1000 bytes1200 bytes 1400 bytes
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 200 400 600 800 1000 1200 1400
Transmit Time/packet us
Recv
Tim
e/pa
cket
us
100 bytes 200 bytes400 bytes 600 bytes800 bytes 1000 bytes1200 bytes 1400 bytes
Bandwidth: Man-GLA
NNW – ATM WAN – Glasgow NO frame loss when not queuing Similar behavior to Man-IC
July 2000 PPNCG Meeting R. Hughes-Jones
0
500
1000
1500
2000
2500
0 500 1000 1500 2000 2500
Transmit Time/packet us
Recv
Tim
e/pa
cket
us
100 bytes 200 bytes
400 bytes 600 bytes
800 bytes 1000 bytes
1200 bytes 1400 bytes
0
5
10
15
20
25
30
35
0 500 1000 1500 2000 2500
Transmit Time/packet us
Recv
Wire
Rat
e in
clud
ing
ATM
Mbi
t/s
100 bytes 200 bytes
400 bytes 600 bytes
800 bytes 1000 bytes
1200 bytes 1400 bytes
Bandwidth: Man-RAL
PC Manc. – NNW – ATM WAN – LAN RAL Sun Odd bursts of frame loss when not queuing 24 Mbit link into RAL
July 2000 PPNCG Meeting R. Hughes-Jones
0
1000
2000
3000
4000
5000
6000
7000
8000
0 500 1000 1500 2000 2500 3000
Transmit Time/packet us
Recv
Tim
e/pa
cket
us
100 bytes 200 bytes 400 bytes 600 bytes 800 bytes 1000 bytes 1200 bytes 1400 bytes
0
0.5
1
1.5
2
2.5
0 500 1000 1500 2000 2500 3000Transmit Time/packet us
Recv
Wire
Rat
e in
clud
ing
ATM
Mbi
t/s
100 bytes 200 bytes
400 bytes 600 bytes
800 bytes 1000 bytes
1200 bytes 1400 bytes
Bandwidth: Man-RAL Managed BW
PC Manc. LAN – ATM PVC – RAL Sun NO frame loss when not queuing Suggests PVC iS CBR, given 1 cell every 75
July 2000 PPNCG Meeting R. Hughes-Jones
Packet Jitter
Same protocol as for the Throughput tests Send frames at regular fixed time intervals Measure the difference between consecutive
frame arrivals using the Pentium cycle counter.
July 2000 PPNCG Meeting R. Hughes-Jones
64 bytes pc-gw-metro
0100200300400500600700
0 500 1000 1500 2000 2500 3000 3500
Inter-Frame time us
N(t
) 10
00u
s
0
100
200
300
400
500
N(t
) 20
00u
s
Packet Jitter: Man-MC
PRELIMINARY !!
HEP – gw-metro
Major peaks 160 s Satellites 40 s ATM cell 2.74 s ?
1000 byte frames appear to arrive too quickly.
1400 bytes pc-gw-metro
0
200
400
600
800
1000
0 500 1000 1500 2000 2500 3000 3500
Inter-Frame time us
N(t
) 10
00u
s
0
100
200
300
400
500
N(t
) 20
00u
s
July 2000 PPNCG Meeting R. Hughes-Jones
IC-UCL
0
500
1000
1500
2000
2500
0 500 1000 1500 2000
Inter-frame time us
N(t
)
0
1000
2000
3000
4000
5000
6000
7000
8000100 bytes
1400 bytes
Packet Jitter: IC-UCL
PRELIMINARY !!
1000 s frame spacing
Structure for 100 byte frames
Satellites at 40 s ATM cell 2.74 s ?
