Interactions Between Delayed Acks and Nagle’s Algorithm in HTTP and HTTPS: Problems and Solutions...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
1
Transcript of Interactions Between Delayed Acks and Nagle’s Algorithm in HTTP and HTTPS: Problems and Solutions...
Interactions Between Delayed Acks and Nagle’s Algorithm in HTTP and HTTPS: Problems
and Solutions
Arthur Goldberg
Robert Buff
New York University
March 1999
2
Client Browser Web ServerTCP/IPApplication
W1: write() less than MSS
W2: write() less than MSS
175
ms
(Win
32)
Ack
del
ay
Network
20 m
s
TCP segment with W1
TCP segment with W2
ACK for W1
Nagle - Delayed Ack Interaction
TCP/IP Application
read() W1
read() W2
write() W1/W2 response
read() W1/W2 response
segment with response
3
Observations
• Applies to request/response applications other than the Web
• Diagram to scale, and generous with message travel times
• Client blocks until receipt of response• ACK delay depends on TCP
implementation• Occurs with client/server roles reversed, too
4
Solution Opportunities: In Order of Increasing Implementation Difficulty and Impact
• Application: Change buffering, disable Nagle
• Sockets API: Broaden semantics
• TCP implementation: Decrease ACK delay
• TCP standard: Change Nagle
5
Network Communications Goals
• Application layer: minimize median response time - deliver data to a receiver as soon as it’s ready
• Transport and Network layers: avoid gross inefficiencies - collect data into the largest possible segments
6
Assumptions
• Network– modest delay– high bandwidth– minimal packet lost
• Application– client/server– many modules
7
The Silly Window Syndrome (SWS)
Sender Receiver
TCP window fullread 1 byte
read 1 byte
data
ack
Advertise 1 byte of available window
1 byte of data
ack
Advertise 1 byte of available window
1 byte of data
Overhead: 3 packets and 3*40 = 120 bytes of TCP/IP headers transport 1 byte of data!
write() some
8
TCP’s SWS Avoidance
• Receiver– avoid advertising small TCP window
advances– delay acknowledgements
• Sender– Nagle algorithm: delays transmission of
partially filled segments until all previously transmitted data has been acknowledged
9
Receiver SWS Avoidance in the TCP Spec, RFC 1122
• The receiver's SWS avoidance algorithm determines when the right window edge may be advanced;– For realistic receive buffers (greater than twice the
MSS) window advances are announced in increments of MSS.
• A TCP SHOULD implement a delayed ACK … the delay MUST be less than 0.5 seconds, and in a stream of full-sized segments there SHOULD be an ACK for at least every second segment.
10
Sender SWS Avoidance in the TCP Spec
• A TCP SHOULD implement … [Nagle 84] to coalesce short segments. However, there MUST be a way for an application to disable the Nagle algorithm on an individual connection.
• The Nagle algorithm is …: If there is unacknowledged data … then the sending TCP buffers all user data … until the outstanding data has been acknowledged or until the TCP can send a full-sized segment …
11
Test Application• Execute 100 iterations
System calls Typical
Client Server Applicationsemantics
TCPsegments
connect() listen(); accept() TCPhandshake
write() request (16 B) Dataread()write() First part of
response (10 B)
Data
read() ack(optional)
write() Second part ofresponse (10 B)
Data
read() ack(optional)
12
LabMachines connected by unloaded 100 Mbps Ethernet
* Win95 separated by 1 router
Name Processor Operating SystemNT4Wa Pentium, 100 MHz NT Workstation 4.00.1381NT4Wb Pentium, 200 MHz NT Workstation 4.00.1381NT4S Pentium, 233 MHz NT Server 4.00.1381, SP 3NT5S Pentium, 233 MHz NT Server 5.00.1671, beta 1Win95 Pentium, 100 MHz Windows 95Win98 Pentium, 90 MHz Windows 98Linux i486, 66 MHz Linux 2.0.31Solaris Sun SPARCstation 5 SunOS 5.6
13
Linux/NT, Nagle OffSegment,payload [bytes]
Delta[ms]
1 TCP handshake
2 0.5 0.5 ms cumulative
3 0.9 1.4
4 Request, 16 9.7 11.1
5 First response, 10 1.5 12.6
6 Second response, 10 1.2 13.8
7 (ack) 13.6 27.4
8 Request, 16 32.5 59.9
9 First response, 10 1.3 61.2
10 Second response, 10 1.2 62.4
and so on
14
NT/NT, Nagle OnSegment,payload [bytes]
Delta[ms]
1 TCP handshake
2 0.3 0.3 ms cumulative
3 0.3 0.6
4 Request, 16 9.1 9.7
5 First response, 10 3.6 13.3
6 (ack) 114.4 127.7
7 Second response, 10 0.2 127.9
8 Request, 16 8.3 136.2
9 First response, 10 3.5 139.7
10 (ack) 188.3 328.0
11 Second response, 10 0.2 328.2
and so on
15
Solaris/NT, Nagle OnSegment,payload [bytes]
Delta[ms]
1 TCP handshake
2 0.4 0.4 ms cumulative
3 0.7 1.1
4 Request, 16 3.1 4.2
5 First response, 10 0.8 5.0
6 (ack) 0.6 5.6
7 Second response, 10 0.3 5.9
8 Request, 16 2.1 8.0
9 First response, 10 0.8 8.8
10 (ack) 41.8 50.6
11 Second response, 10 0.3 50.9
and so on
16
Summary
Delay of ack [ms] Transaction [ms]
Client Server Nagle on Nagle off Nagle on Nagle off
NT4Wa NT4S 187.3 0.4 191.4 7.9
NT4S NT4Wa 187.0 0.2 191.6 3.0
NT4Wb Win95 193.1 194.1 193.1 199.0
Win95 NT4Wb 208.8 0.7 209.5 2.1
NT4Wb Win98 192.5 193.8 198.1 199.8
Win98 NT4Wb 192.5 1.0 192.9 1.5
NT4Wa NT5S 195.9 0.3 196.7 0.7
NT5S NT4Wa 197.4 0.2 198.6 1.6
NT4Wa Solaris 194.2 0.3 196.7 3.0
Solaris NT4Wa 46.2 0.6 47.5 4.3
NT4Wa Linux 188.7 0.3 191.8 3.9
Linux NT4Wa 15.8 –––– 21.5 4.4
17
ACK delays
Linux 2 Adjusted to interarrivalspacing, observed at 15 to20 ms
Solaris 50 ms mean (100 ms intervaltimer, according to Paxson)
Win32 Maximum 200 ms, increaserapidly during connection
18
HTTPS, new session keySegment,payload [bytes]
Delta[ms]
1 TCP handshake
2 0.2
3 0.4
4 SSL, 93 203.3
Initial 0.6 ms for TCP handshake and 203.3 ms for SSLcalculations at the server are omitted.
5 SSL, 726 1.4 205.3 ms cumulative
6 SSL, 204 88.5 293.8
7 SSL, 6 116.2 410.0
8 (ack) 156.1 566.1
9 SSL, 61 0.3 566.4
10 HTTPS Request, 46 10.7 577.1
11 HTTPS Resp 1, 306 3.3 580.4
12 HTTPS Resp 2, 1021 1.3 581.7
13 (ack) 1.5 583.2
14 (ack) 6.4 589.6
15 (ack) 0.2 589.8
19
HTTPS, session key reusedSegment,payload [bytes]
Delta[ms]
1 TCP handshake
2 0.2 0.2 ms cumulative
3 0.3 0.5
4 SSL, 108 11.1 11.6
5 SSL, 79 1.4 13.0
6 (ack) 133.1 146.1
7 SSL, 67 0.3 146.4
8 SSL, 67 4.5 150.9
9 (ack) 182.8 333.7
10 Request, 46 0.4 334.1
11 HTTPS Resp 1, 306 3.1 337.2
12 HTTPS Resp 2, 1021 1.3 338.5
13 (ack) 1.5 340.0
14 (ack) 6.0 346.0
15 (ack) 0.1 346.1
20
HTTPS Potential Savings
SSL handshake Entire HTTPS transactionSession key
Delay No delay Slowdown Delay No delay Slowdown
New 565 ms 409 ms 38 % 589 ms 433 ms 36 %
Reused 333 ms 17 ms 1826 % 346 ms 30 ms 1046 %
21
HTTP, GIF from Netscape
Segment,payload [bytes]
Delta[ms]
1 TCP handshake
2 3.0 3.0 ms cumulative
3 0.4 3.4
4 HTTP GET, 390 1.7 5.1
5 HTTP RESP, 1944 10.5 15.6
6 HTTP GET, 414 164.3 179.9
7 HTTP RESP, 354 8.8 188.7
8 (ack) 199.4 388.1
9 HTTP RESP, 3255 12.9 401.0
and so on
22
Application AlternativesStrategy Approx
Appdelay
IPpkts
Issues
Writev RTT/2 1 Ideal performance, butimpossible to implement if manymodules write
Coalescedwrites
RTT/2 1 Like writev, but more possiblycopying
Two writes,Nagle off
RTT/2 +
2 Easy to build; low delay; risky, aspacket count can surge withmany writes
Two writes ACKdelay +3RTT/2
3 slow, 3 IP packets
23
Alternative: Disable Nagle on the last write of an application message
• Advantages– easy to do– removes delayed ACK from critical path– eliminate risk of small packets flood
• Disadvantages– two extra system calls per application message– possibly one extra IP packet– complicated for a programmer
24
Related Work• Nagle - delayed ACK interactions widely
recognized, e.g. Stevens 98, Microsoft 98 and Sun 98
• P-HTTP recorded by Heidemann 97
• Nielsen 97, 98: test a typical HTTP/1.1 server page load
– (Client ran on a DEC Alphastation 400 4/233, UNIX 4.0a)
Situation Time (sec) Time (sec)Nagle 0.48 0.27NoNagle 0.45 0.21
25
Possible Solutions
• Sockets API– Include a flush
• TCP implementation– Decrease ACK delay, especially in Win32
• TCP standard– Desensitize SWS avoidance
• SWS avoidance clearly being triggered prematurely and introducing unnecessary delays
26
Nagle modification proposed
• Minshall, “A Suggested Modification to Nagle's Algorithm”, http://search.ietf.org/internet-drafts/draft-minshall-nagle-00.txt
• If a TCP has less than a full-sized packet to transmit, and if any previous less than full-sized packet has not yet been acknowledged, do not transmit a packet.
• Will not solve SSL problem above
27
Todos
• Measure production delayed ACK costs
• Examine other application protocols
• Develop high-performance sockets
Interactions Between Delayed Acks and Nagle’s Algorithm in HTTP and HTTPS: Problems
and Solutions
Arthur Goldberg
Robert Buff
New York University
March 1999