1

Non-blocking I/O

Computing Network Programming

2

Outline

– Socket operations that cause blocking and how there operations behave for non-blocking sockets

– non-blocking read and write– non-blocking connect

– daytime client

– web client

– non-blocking accept

3

Motivation

• If we use non-blocking I/O, the process can some useful task after initiating the operation, or after detecting that I/O operation can not be completed (we don’t waste time by blocking and sleeping)

• If we use non-blocking I/O, we can write network programs who perform better in terms of time

– Establish simultaneous connections and data transfers between a server and client: example: netscape browsers

4

Socket operations that cause blocking

– By default sockets are blocking– if socket function call can not complete immediately,

process is put into sleep mode until kernel completes the operation

– Socket calls that can block• Input operations: read, readv, recv, recvmsg, recvfrom

– block until some data arrives (TCP) or until a complete UDP datagram arrives (UDP)

– for non-blocking socket, if input can not received, function returns immediately with error code EWOULDBLOCK

5

Socket operations that cause blocking

• Output operations: write, writev, send, sendto, sendmsg– block until socket sendbuffer has room (TCP).

» For non-blocking socket, output operation returns immedialy if there no room with error EWOULDBLOCK.

– never block for UDP.

• Accepting incoming connections: accept– if there is no new connection available accept will block until a

new connection is established.

– For a non-blocking socket accept return immediately if there is no new connection available with an error EWOULDBLOCK

6

Socket operations that cause blocking• Initiating outgoing connections: connect

– connect for TCP blocks the process until a TCP connection is established (until client receives the ACK of its SYN: at least one round-trip time)

– for non-blocking socket, if connect is called for TCP, the connection is initiated and connect returns with error EINPROGRESS. (sometimes connection can be established immediately if two process are at the same machine, in which case connect will return 0 = OK).

7

How to set a socket non-blocking mode

int flags;int socketfd;….sockfd = socket(AF_INET, SOCK_STREAM, 0);

if ((flags = fcntl(fd, F_GETFL, 0)) < 0)err_sys(“F_GETFL error”),

flags = flags | O_NONBLOCK;if (fcntl(fd, F_SETFL, flags) < 0)

err_sys(“F_SETFL error”),

We use fcntl function to set a socket to non-blcoking mode.We saw this function earlier when we studied the socket options

8

Non-blocking Read and WriteWe will use our echo client again to show not blocking readand writes. Focus on str_cli() function

We had developed earlier version that was using select on stdin and socket. But it is still using blocking I/O, because after obtaining a descriptor to be readable/writebale, the I/O operation can still block (see next slide)

We will now develop echo client that is completely blocking free, hence it is more efficient.

However, buffer management is more complex with non-blocking I/O, and programs can be longer.

Tradeoff between performance and development effort

9

void str_cli(FILE *fp, int sockfd){ int maxfdp1; fd_set rset; char sendline[MAXLINE], recvline[MAXLINE];

FD_ZERO(&rset); for ( ; ; ) { FD_SET(fileno(fp), &rset); FD_SET(sockfd, &rset); maxfdp1 = max(fileno(fp), sockfd) + 1; Select(maxfdp1, &rset, NULL, NULL, NULL); if (FD_ISSET(sockfd, &rset)) { /* socket is readable */ if (Readline(sockfd, recvline, MAXLINE) == 0) /* can BLOCK here */ err_quit("str_cli: server terminated prematurely"); Fputs(recvline, stdout); } if (FD_ISSET(fileno(fp), &rset)) { /* input is readable */ /* can BLOCK here */ if (Fgets(sendline, MAXLINE, fp) == NULL) /* can BLOCK here */ return; /* all done */ Writen(sockfd, sendline, strlen(sendline)); } }}

Using select (still can block)

10

Non-blocking I/O: Buffer Management

We maintain two buffers called “to” and “fr”:to: contains data going from standard input (keyboard) to the server (socket)

fr: contains data arriving from server (socket) to standard output (screen)

to buffer

fr buffer

stdin

stdout

keyboard

screen

to/from Server

echo client process

sockfd

str_cli() function reads from keyboard into “to” buffer and writesinto socket, and reads from socket into “fr” buffer and writes to screen

11

Buffers and Pointers

already sent data to send to the serveravailable space to read into

fom stdin

stdin

sockettooptr

toiptr &to[MAXLINE]

already sent data to send to standard outputavailable space to read into

from socket

socket

stdoutfroptr

friptr &fr[MAXLINE]

to

fr

12

str_cli() function of echo client 5 void str_cli(FILE *fp, int sockfd) 6 { 7 int maxfdp1, val, stdineof; 8 ssize_t n, nwritten; 9 fd_set rset, wset; 10 char to[MAXLINE], fr[MAXLINE]; 11 char *toiptr, *tooptr, *friptr, *froptr; 12 13 val = Fcntl(sockfd, F_GETFL, 0); 14 Fcntl(sockfd, F_SETFL, val | O_NONBLOCK); 15 16 val = Fcntl(STDIN_FILENO, F_GETFL, 0); 17 Fcntl(STDIN_FILENO, F_SETFL, val | O_NONBLOCK); 18 19 val = Fcntl(STDOUT_FILENO, F_GETFL, 0); 20 Fcntl(STDOUT_FILENO, F_SETFL, val | O_NONBLOCK); 21 22 toiptr = tooptr = to; /* initialize buffer pointers */ 23 friptr = froptr = fr; 24 stdineof = 0; 25

13

26 maxfdp1 = max(max(STDIN_FILENO, STDOUT_FILENO), sockfd) + 1; 27 for ( ; ; ) { 28 FD_ZERO(&rset); 29 FD_ZERO(&wset); 30 if (stdineof == 0 && toiptr < &to[MAXLINE]) /* if buffer has space */ 31 FD_SET(STDIN_FILENO, &rset); /* read from stdin */ 32 if (friptr < &fr[MAXLINE]) 33 FD_SET(sockfd, &rset); /* read from socket */ 34 if (tooptr != toiptr) /* if there is data to write */ 35 FD_SET(sockfd, &wset); /* data to write to socket */ 36 if (froptr != friptr) 37 FD_SET(STDOUT_FILENO, &wset); /* data to write to stdout */ 38 39 Select(maxfdp1, &rset, &wset, NULL, NULL); 40 41

str_cli() continued

14

42 if (FD_ISSET(STDIN_FILENO, &rset)) { 43 if ( (n = read(STDIN_FILENO, toiptr, &to[MAXLINE] - toiptr)) < 0) { 44 if (errno != EWOULDBLOCK) 45 err_sys("read error on stdin"); 46 47 } else if (n == 0) { 48 49 stdineof = 1; /* all done with stdin */ 50 if (tooptr == to) 51 Shutdown(sockfd, SHUT_WR);/* send FIN */ 52 53 } else { 54 toiptr += n; /* # just read */ 55 FD_SET(sockfd, &wset); /* try and write to socket below */ 56 } 57 }

str_cli() continued

15

64 if (FD_ISSET(sockfd, &rset)) { 65 if ( (n = read(sockfd, friptr, &fr[MAXLINE] - friptr)) < 0) { 66 if (errno != EWOULDBLOCK) 67 err_sys("read error on socket"); 68 } else if (n == 0) { 69 if (stdineof) 70 return; /* normal termination */ 71 else 72 err_quit("str_cli: server terminated prematurely"); 73 } else { 74 friptr += n; /* # just read */ 75 FD_SET(STDOUT_FILENO, &wset); /* try and write below */ 76 } 77 }

str_cli() continuedI deleted some code that is not very important hencethere is shift in line numbers! Don’t get confused.

16

89 if (FD_ISSET(STDOUT_FILENO, &wset) && ( (n = friptr - froptr) > 0)) { 90 if ( (nwritten = write(STDOUT_FILENO, froptr, n)) < 0) { 91 if (errno != EWOULDBLOCK) 92 err_sys("write error to stdout"); 93 } else { 94 froptr += nwritten; /* # just written */ 95 if (froptr == friptr) 96 froptr = friptr = fr; /* back to beginning of buffer */ 97 } 98 }

str_cli() continued

17

105 if (FD_ISSET(sockfd, &wset) && ( (n = toiptr - tooptr) > 0)) { 106 if ( (nwritten = write(sockfd, tooptr, n)) < 0) { 107 if (errno != EWOULDBLOCK) 108 err_sys("write error to socket"); 109 110 } else { 111 tooptr += nwritten; /* # just written */ 112 if (tooptr == toiptr) { 113 toiptr = tooptr = to; /* back to beginning of buffer */ 114 if (stdineof) 115 Shutdown(sockfd, SHUT_WR); /* send FIN */ 116 } 117 } 118 } 119 } 120 }

str_cli() continued

End of Function

18

Other ways of implementing echo client

• Non-blocking I/O increses performance but it is complex

• We can split the client into 2 processes using fork() and obtain a simpler program– one child will handle keyboard to socket

transfer– other child will handle socket to screen transfer

19

Echo Client with 2 Processes

parent

child

serverfork()

client

One TCP connection (full duplex)stdin

stdout

Server and Child share the same socket- one socket, one recv buffer, one send buffer in the kernel

20

client code with fork()#include "unp.h"voidstr_cli(FILE *fp, int sockfd){ pid_t pid; char sendline[MAXLINE], recvline[MAXLINE];

if ( (pid = Fork()) == 0) { /* child: server -> stdout */ while (Readline(sockfd, recvline, MAXLINE) > 0) Fputs(recvline, stdout); kill(getppid(), SIGTERM); /* in case parent still running */ exit(0); }

/* parent: stdin -> server */ while (Fgets(sendline, MAXLINE, fp) != NULL) Writen(sockfd, sendline, strlen(sendline)); Shutdown(sockfd, SHUT_WR); /* EOF on stdin, send FIN */ pause(); /* is used only to measure the time correctly */ return;}

21

Non-blocking Connect()

• We have seen I/O functions on non-blocking sockets

• Now we will see how to connect behaves on non-blocking sockets

• We will see an application that uses this approach: netscape browser

• opens multiple TCP connections simultaneously

22

Example ScenerioWe access and download a web page (/) and that webpage contains links to other objects (GIF files) and we would liketo download these objects simultaneously over different connections

(object may reside on different servers)Two steps- establish a separate TCP connection for each object- transfer the object over the established connection

Do these two steps for each objects. 3 approaches

1- totally serialized (one connection at a time) 2- establish connections first one by one serially, then transfer data simulaneously3- establish connections and transfer objects all simultaneously (we will see how to do this)

23

Establishing simultaneous Connections

Serverhttpd

Clientbrowser

Other web servers and objects

Download main web page

Links

Main page dowloadover this connection

Client will download the other objects specified in the main page.

24

Performance Improvement with simultaneous connections

10

4

15

10

4

15 1510

4

29 unit time

15 unit time 15 unit time

One connection at a time

Two connections simultaneously

Three connections simulatenously

25

How to use non-blocking socket with connect– First call connect()

• initiates TCP connection and returns.

– Then call select()• when select returns (either there is timeout, or socket

available for reading or for both reading and writing)– Check the error value with getsockopt function and

SO_ERROR socket option– IF the error is zero (no error)

» then connection established successfully, we can read from and write to the socket.

– IF the error is non-zero» connection could not estanlished successfully, either because of

select timeout or some other error like connection refused, TCP timeout, hostunreachable, etc.

26

Example programWeb client

> web 3 www.foo.com / image1.gif image2.gif image3.gif

- retrieves first root home page: /- then establishes simultaneous connections to retrive 3 objects

image1.gif, image2.gif, image3.gif

thereby simulates a web browser operation.

27

#include "unp.h"#define MAXFILES 20#define SERV "80" /* port number or service name */struct file { char *f_name; /* filename */ char *f_host; /* hostname or IPv4/IPv6 address */ int f_fd; /* descriptor */ int f_flags; /* F_xxx below */} file[MAXFILES];

#define F_CONNECTING 1 /* connect() in progress */#define F_READING 2 /* connect() complete; now reading */#define F_DONE 4 /* all done */

#define GET_CMD "GET %s HTTP/1.0\r\n\r\n" /* globals */int nconn, nfiles, nlefttoconn, nlefttoread, maxfd;fd_set rset, wset; /* function prototypes */void home_page(const char *, const char *);void start_connect(struct file *);void write_get_cmd(struct file *);

First look at the program header file web.h

28

int main(int argc, char **argv){ int i, fd, n, maxnconn, flags, error; char buf[MAXLINE]; fd_set rs, ws;

if (argc < 5) err_quit("usage: web <#conns> <hostname> <homepage> <file1> ..."); maxnconn = atoi(argv[1]); nfiles = min(argc - 4, MAXFILES); for (i = 0; i < nfiles; i++) { file[i].f_name = argv[i + 4]; file[i].f_host = argv[2]; file[i].f_flags = 0; } printf("nfiles = %d\n", nfiles); home_page(argv[2], argv[3]); /* retrieves the main (root) page */ FD_ZERO(&rset); FD_ZERO(&wset); maxfd = -1; nlefttoread = nlefttoconn = nfiles; nconn = 0;

/* …. Will be continued */

web.c intialization

29

voidhome_page(const char *host, const char *fname){ int fd, n; char line[MAXLINE];

fd = Tcp_connect(host, SERV); /* blocking connect() */

n = snprintf(line, sizeof(line), GET_CMD, fname); Writen(fd, line, n); /* send the GET command */

/* receive the main page */ for ( ; ; ) { if ( (n = Read(fd, line, MAXLINE)) == 0) break; /* server closed connection */

printf("read %d bytes of home page\n", n); /* do whatever with data */ } printf("end-of-file on home page\n"); Close(fd);}

Retrieving the main page (/): home_page() function

We will use blockingconnect() and read()while retrieving the main page.

But for rest of the objects, we will use non-blockingconnect and I/O operations.

30

void start_connect(struct file *fptr){ int fd, flags, n; struct addrinfo *ai;

ai = Host_serv(fptr->f_host, SERV, 0, SOCK_STREAM); /* obtain addrinfo for server */ fd = Socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol); /* create the socket */ fptr->f_fd = fd; flags = Fcntl(fd, F_GETFL, 0); /* Set socket nonblocking */ Fcntl(fd, F_SETFL, flags | O_NONBLOCK); /* Initiate non-blocking connect to the server. */ if ( (n = connect(fd, ai->ai_addr, ai->ai_addrlen)) < 0) { if (errno != EINPROGRESS) err_sys("nonblocking connect error"); fptr->f_flags = F_CONNECTING; FD_SET(fd, &rset); /* select for reading and writing */ FD_SET(fd, &wset); if (fd > maxfd) maxfd = fd; } else if (n >= 0) /* connect is already done */ write_get_cmd(fptr); /* write() the GET command */}

Initiating a connection: start_connect() function

31

voidwrite_get_cmd(struct file *fptr){ int n; char line[MAXLINE];

n = snprintf(line, sizeof(line), GET_CMD, fptr->f_name); Writen(fptr->f_fd, line, n); printf("wrote %d bytes for %s\n", n, fptr->f_name);

fptr->f_flags = F_READING; /* clears F_CONNECTING */

FD_SET(fptr->f_fd, &rset); /* will read server's reply */ if (fptr->f_fd > maxfd) maxfd = fptr->f_fd;}

Sending HTTP GET command to the serverwrite_get_cmd() function

32

main() function continued

while (nlefttoread > 0) { /* more files need to be downloaded */ while (nconn < maxnconn && nlefttoconn > 0) { /* find a file to read and start connection to the server for file*/ for (i = 0 ; i < nfiles; i++) if (file[i].f_flags == 0) break; if (i == nfiles) err_quit("nlefttoconn = %d but nothing found", nlefttoconn); start_connect(&file[i]); nconn++; nlefttoconn--; } rs = rset; ws = wset; n = Select(maxfd+1, &rs, &ws, NULL, NULL);

for (i = 0; i < nfiles; i++) { flags = file[i].f_flags; if (flags == 0 || flags & F_DONE) continue; fd = file[i].f_fd; /* continues on the next slide */

33

if (flags & F_CONNECTING && (FD_ISSET(fd, &rs) || FD_ISSET(fd, &ws))) { n = sizeof(error); if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &error, &n) < 0 || error != 0) { err_ret("nonblocking connect failed for %s", file[i].f_name); } /* connection established */

FD_CLR(fd, &wset); /* no more writeability test */ write_get_cmd(&file[i]); /*send the GET command to server */ } else if (flags & F_READING && FD_ISSET(fd, &rs)) { if ( (n = Read(fd, buf, sizeof(buf))) == 0) {

Close(fd); /* end of file reached */ file[i].f_flags = F_DONE; /* clears F_READING */ FD_CLR(fd, &rset); nconn--; nlefttoread--; } else { /* we are not doing any special processing on the file after we read */ printf("read %d bytes from %s\n", n, file[i].f_name);

} } } } exit(0);} /* END OF PROGRAM */

main() function continued