UDP-BASED DATA TRANSFER PROTOCOL (UDT)By:Cecil S. Briones

UDP-BASED DATA TRANSFER PROTOCOL (UDT)is a high performance data transfer protocol designed for transferring large volumetric datasets over high speed wide area networks.

UDTInitial versions were developed and tested on very high speed networks (1Gb/s, 10Gb/s, etc.); however, recent versions of the protocol have been updated to support the commodity Internet as well. For example, the protocol now supports rendezvous connection setup, which is a desirable feature for traversing NAT firewalls using UDP.has an open source implementation which can be found on SourceForge. It is one of the most popular solutions for supporting high speed data transfer and is part of many research projects and commercial products.

USER DATAGRAM PROTOCOL (UDP) is one of the core members of the Internet Protocol Suite, the set of network protocols used for the Internet. With UDP, computer applications can send messages, in this case referred to as datagrams, to other hosts on an Internet Protocol (IP) network without requiring prior communications to set up special transmission channels or data paths. The protocol was designed by David P. Reed in 1980 and formally defined in RFC 768.

BACKGROUND

UDT was developed by Yunhong Gu during his PhD studies at the National Center for Data Mining (NCDM) of University of Illinois at Chicago. Dr. Gu continues to maintain and improve the protocol after graduation.The UDT project started in 2001, when inexpensive optical networks became popular and triggered a wider awareness of TCP efficiency problems over wide area high speed networks. The first version of UDT, also known as SABUL (Simple Available Bandwidth Utility Library), was designed to support bulk data transfer for scientific data movement over private networks. SABUL used UDP for data transfer and a separate TCP connection for control messages.SABUL was later renamed to UDT starting with version 2.0, which was released in 2004. UDT2 removed the TCP control connection in SABUL and used UDP for both data and control information. UDT2 also introduced a new congestion control algorithm that allowed the protocol to run "fairly and friendly" with concurrent UDT and TCP flows.

UDT3 (2006) extended the usage of the protocol to the commodity Internet. Congestion control was tuned to support relatively low bandwidth as well. UDT3 also significantly reduced the use of system resources (CPU and memory). Additionally, UDT3 allows users to easily define and install their own congestion control algorithms.

UDT4 (2007) introduced several new features to better support high concurrency and firewall traversing. UDT4 allowed multiple UDT connections to bind to the same UDP port and it also supported rendezvous connection setup for easier UDP hole punching.A fifth version of the protocol is currently in the planning stage. Possible features include the ability to support multiple independent sessions over a single connection.Moreover, since the absence of a security feature for UDT has been an issue with its initial implementation in a commercial environment, Bernardo (2011) has developed a security architecture for UDT as part of his PhD studies. This architecture however is undergoing enhancement to support UDT in various network environments (i.e., optical networks).

PROTOCOL ARCHITECTURE

UDT is built on top of User Datagram Protocol (UDP) by adding congestion control and reliability control mechanisms. UDT is an application level, connection oriented, duplex protocol that supports both reliable data streaming and partial reliable messaging.AcknowledgingAIMD with Decreasing IncreaseConfigurable Congestion ControlRendezvous Connection Setup

ACKNOWLEDGINGUDT uses periodic acknowledgments (ACK) to confirm packet delivery, while negative ACKs (loss reports) are used to report packet loss. Periodic ACKs help to reduce control traffic on the reverse path when the data transfer speed is high, because in these situations, the number of ACKs is proportional to time, rather than the number of data packets.

AIMD WITH DECREASING INCREASEUDT uses an AIMD (additive increase multiplicative decrease) style congestion control algorithm. The increase parameter is inversely proportional to the available bandwidth (estimated using the packet pair technique), thus UDT can probe high bandwidth rapidly and can slow down for better stability when it approaches maximum bandwidth. The decrease factor is a random number between 1/8 and 1/2. This helps reduce the negative impact of loss synchronization.In UDT, packet transmission is limited by both rate control and window control. The sending rate is updated by the AIMD algorithm described above. The congestion window, as a secondary control mechanism, is set according to the data arrival rate on the receiver side.

CONFIGURABLE CONGESTION CONTROLThe UDT implementation exposes a set of variables related to congestion control in a C++ class and allows users to define a set of callback functions to manipulate these variables. Thus, users can redefine the control algorithm by overriding some or all of these callback functions. Most TCP control algorithms can be implemented using this feature with fewer than 100 lines of code.

RENDEZVOUS CONNECTION SETUPUDT supports both traditional client/server connection setup mode and a new rendezvous connection setup mode. The latter is widely used for firewall traversing when both peers are behind firewalls.

USE SCENARIOSUDT is widely used in high performance computing area to support high speed data transfer over optical networks. For example, GridFTP, a popular data transfer tool in grid computing, uses UDT as its data transfer protocol.Over the commodity Internet, UDT has been used in many commercial products for fast file transfer over wide area networks.Because UDT is purely based on UDP, it has also been used in many situations where TCP is at a disadvantage to UDP. These scenarios include Peer-to-Peer applications, video/audio communication and many others.

THATS ALL FOLKS[ nisi Dominus frusta]