Chapter 5

TCP Client/Server Example

TCP Client-Server Example TCP echo server: main and str_echo TCP echo client: main and str_cli Normal startup and termination POSIX signal handling Handling SIGCHILD, interrupted system call

s, and preventing zombies Connection abort before accept returns Crash of server process

SIGPIPE signal Crash, reboot, shutdown of server

host Summary of TCP example Data format: passing string or

binary

TCP Echo Server and Client

To expand this example to other applications,just change what the server does with the client input.

Many boundary conditions to handle: signal, interruptedsystem call, server crash, etc. The first version does nothandle them.

TCP Echo Server main() Algorithm A typical fork()-based concurrent server Algorithm outline :

Create socket Bind it to a designated port (supposedly to be a well-known port) Allow incoming traffic for any local network interface (wildcard a

ddress: INADDR_ANY) Convert it to a listening socket

Set up a listening queue Loop around forever :

Block in call to accept(), wait for a client to connect Spawn a child to handle each client upon successful connection

Close listening socket Execute str_echo()

Close connected socket for child. ‹—— parent no wait

5

TCP Echo Server: main function

#include "unp.h"intmain(int argc, char **argv){

int listenfd, connfd;pid_t childpid;socklen_t clilen;struct sockaddr_in cliaddr, servaddr;listenfd = Socket(AF_INET, SOCK_STREAM, 0);bzero(&servaddr, sizeof(servaddr));servaddr.sin_family = AF_INET;servaddr.sin_addr.s_addr = htonl(INADDR_ANY);servaddr.sin_port = htons(SERV_PORT);Bind(listenfd, (SA *) &servaddr, sizeof(servaddr));

tcpcliserv/tcpserv01.c

9877

Listen(listenfd, LISTENQ);for ( ; ; ) {

clilen = sizeof(cliaddr);connfd = Accept(listenfd, (SA *) &cliaddr, &clilen);if ( (childpid = Fork()) == 0) { /* child process */Close(listenfd); /* close listening socket */str_echo(connfd); /* process the request */exit(0);}Close(connfd); /* parent closes connected socket*/

}}

str_echo() Algorithm It provides very simple service for each client

It reads data from a client and echoes it back to the client

Algorithm outline : Read a buffer from the connected socket

If n (number of characters read) > 0, Echo back to the client (writen(): p. 89), read again

Else if n < 0 & EINTR (got interrupt), read again Else just n < 0 (error occurred), display error message (and te

rminate child process in err_sys()) Else if n = 0 (receipt of FIN from client, the normal scenario),

return

8

TCP Echo Server: str_echo function

#include "unp.h"voidstr_echo(int sockfd){

ssize_t n;char line[MAXLINE];for ( ; ; ) {if ( (n = Readline(sockfd, line, MAXLINE)) == 0)return; /* connection closed by other end */Writen(sockfd, line, n);}

}

lib/str_echo.c

TCP Echo Client main() Algorithm Algorithm outline :

Check number of commandline arguments It must be 2 (program name and server address) Quit if not 2 (call to sys_quit())

Open socket Fill in internet socket address structure Connect to server Call str_cli() to handle the rest of the client processing Exit when no more user input

Note: All errors end up in termination of the client in this function. Real applications may need to recover differently

10

TCP Echo Client: main function#include "unp.h"int main(int argc, char **argv){

int sockfd;struct sockaddr_in servaddr;if (argc != 2)

err_quit("usage: tcpcli <IPaddress>");sockfd = Socket(AF_INET, SOCK_STREAM, 0);bzero(&servaddr, sizeof(servaddr));servaddr.sin_family = AF_INET;servaddr.sin_port = htons(SERV_PORT);Inet_pton(AF_INET, argv[1], &servaddr.sin_addr);Connect(sockfd, (SA *) &servaddr, sizeof(servaddr));str_cli(stdin, sockfd); /* do it all */exit(0);

}

tcpcliserv/tcpcli01.c

TCP Echo Client: str_cli function

#include "unp.h"voidstr_cli(FILE *fp, int sockfd){

char sendline[MAXLINE], recvline[MAXLINE];

while (Fgets(sendline, MAXLINE, fp) != NULL) {

Writen(sockfd, sendline, strlen(sendline));if (Readline(sockfd, recvline, MAXLINE) == 0)

err_quit("str_cli: server terminated prematurely");Fputs(recvline, stdout);}

}

lib/str_cli.c

Normal Startup (1/2) To watch the sequence of client/server To start the server in background :

linux% tcpserv01 &[1] 17870

To check the status of all sockets on a system (-a) before the client starts :

linux% netstat –aActive Internet connections (servers and established)Proto Recv-Q Send-Q Local Address Foreign Address Statetcp 0 0 *:9877 *:* LISTEN Note : The output above shows only partial results, and the outp

ut format may be different from system to system

13

Normal Startup (2/2) To start the client on the same machine (using the loop

back address) :linux% tcpcli01 127.0.0.1

Then, check the status of all sockets again :linux% netstat –aActive Internet connections (servers and established)Proto Recv-Q Send-Q Local Address Foreign Address Statetcp 0 0 localhost:9877 localhost:42758 ESTABLISHEDtcp 0 0 localhost:42758 localhost:9877 ESTABLISHEDtcp 0 0 *:9877 *:* LISTEN Note : The first tcp connection is for the server child, and the sec

ond is for the client, and the third is the server parent

14

Normal Termination (1/2)

Client 離開 str_cli 執行 exit 離開 main Client 結束時，所有已開啟的 socket 會被 kernel

自動關閉。 Client 的 TCP 會送 FIN 至 server ， server 的 TCP 會回 ACK.

Client 到 server child 這一半的 connection 已關閉

Fgets

Normal Termination (2/2)

當 server TCP 收到 FIN 時， server child 正等在 readline ，故 readline 會傳回0 ，導致 server child 離開 str_echo 回到 main 。

Server child 執行 exit 會使 server TCP關閉另一半 (server child 到 client) 的連結。

Server child 程序結束成為 zombie

Zombie Process

( 另一個系統的輸出 )

Cleaning Up Zombie Processes

Child process 變成 zombie 後不會自行消失

Zombie 保留 child 結束時的狀態以備 parent 取得，因此會佔用 kernel 空間

Parent 應該要主動清除 zombie 必須要處理 kernel 發出的 signal(SIGCH

LD)

POSIX Signal Handling Signal (software interrupt): sent by one

process to another process (or to itself) or by the kernel to a process

SIGCHLD: by the kernel to the parent when a child process is terminated

Disposition of a signal: catch the signal by a specified signal

handler SIG_IGN: ignore it SIG_DFL: default: terminate or ignore

signal Function That Enables System Call Restart#include "unp.h"Sigfunc *signal(int signo, Sigfunc *func){

struct sigaction act, oact;act.sa_handler = func;sigemptyset(&act.sa_mask);act.sa_flags = 0;if (signo == SIGALRM) {

#ifdef SA_INTERRUPTact.sa_flags |= SA_INTERRUPT; /* SunOS 4.x */

#endif} else {

#ifdef SA_RESTARTact.sa_flags |= SA_RESTART; /* SVR4, 44BSD */

#endif}if (sigaction(signo, &act, &oact) < 0)return(SIG_ERR);return(oact.sa_handler);

}讓被某些 singal 中斷的system call 能夠 restart

Posix 標準 signal disposition函數 sigaction 使用上較為複雜，故作者用此 signal函數來包裝 sigaction

lib/signal.c

Signal Function That Enables System Call Restart (cont.)

Sigfunc *Signal(int signo, Sigfunc *func) {Sigfunc *sigfunc;if ( (sigfunc = signal(signo, func)) == SIG_ERR)err_sys("signal error");return(sigfunc);

}

POSIX signal semantics:1. Once a signal handler is installed, it remains installed.2. The signal being delivered is blocked while a signal handler is executing.3. By default, signals are not queued.

Signal 是將 signal包裝起來的函數

Handling Zombies

Whenever we fork children, we must waitfor them to prevent them from becomingZombies

•To do this, we establish a signal handler tocache SIGCHLD

and within the handler we call wait.

Signal Handler for SIGCHLD

#include "unp.h"voidsig_chld(int signo){

pid_t pid;int stat;pid = wait(&stat);printf("child %d terminated\n", pid);return;

}

tcpcliserv/sigchldwait.c

child 的process id

呼叫 wait 以免讓 child 變成zombie

Interrupted System Calls 當 kernel 發出 SIGCHLD signal 而被 parent 捕

捉時， parent 是 block 在 accept 這個 system call 中。此 system call 被 interrupt 去執行 handler

當 signal handler return 後，如果此 interrupted system call 沒有被 restart ，此 system call會傳回錯誤 (EINTR) 我們定義的 signal 函數有設定要 restart system call

EINTR 的錯誤可導致我們設計的 Accept 函數中止程式執行 (abort)

Handling Interrupted SystemCalls Definition: Slow system call “system call that can

block forever” Some kernels automatically restart some

interrupted system calls, while some don’t We must prepare for slow system calls to return

EINTR

for ( ; ; ){clilen = sizeof (cliaddr);if ( (connfd = accept (listenfd, (SA) &cliaddr, &clilen)) < 0) {

if (errno == EINTR) continue; /* back to for ( ) */else err_sys (“accept error”);

}我們自行 restart accept這個 system call

Weakness of Wait Unix signals are normally not queued multiple occurrences of the same sign

al only cause the handler to be called once

It’s a problem when multiple children terminate at the same time

Solution: use waitpid instead of wait in the handler to kill all zombies

Client terminates all five connectionscatching all SIGCHLD signals in server parent

SIGCHLD Handler UsingWaitpid

#include "unp.h"voidsig_chld(int signo){

pid_t pid;int stat;while ( (pid = waitpid(-1, &stat, WNOHANG)) > 0)

printf("child %d terminated\n", pid);return;

}

#include <sys/wait.h>pid_t waitpid (pid_t pid, int *statloc, int options);

returns: process ID if OK, o, or -1 on error

Wait for the first terminated childNot block if there areno terminated children

Final (correct) Version of TCP Echo Server

handling SIGCHLD, EINTR from accept, zombies

#include "unp.h"intmain(int argc, char **argv){

int listenfd, connfd;pid_t childpid;socklen_ t clilen;struct sockaddr_in cliaddr, servaddr;void sig_chld(int);listenfd = Socket(AF_INET, SOCK_STREAM, 0);bzero(&servaddr, sizeof(servaddr));servaddr.sin_family = AF_INET;servaddr.sin_addr.s_addr = htonl(INADDR_ANY);servaddr.sin_port = htons(SERV_PORT);Bind(listenfd, (SA *) &servaddr, sizeof(servaddr));Listen(listenfd, LISTENQ);

tcpcliserv/tcpserv04.c

Final (correct) Version of TCP Echo Server (cont.)

Signal(SIGCHLD, sig_chld); /* must call waitpid() */for ( ; ; ) {

clilen = sizeof(cliaddr);if ( (connfd = accept(listenfd, (SA *) &cliaddr, &clilen)) < 0) {if (errno == EINTR)

continue; /* back to for() */else

err_sys("accept error");}if ( (childpid = Fork()) == 0) { /* child process */

Close(listenfd); /* close listening socket */str_echo(connfd); /* process the request */exit(0);

}Close(connfd); /* parent closes connected socket */}

}

Connection Abort Before accept Returns

implementation dependent !

In BSD, kernel handles this. accept does not return.In SVR4, accept is returned with EPROTO.In POSIX.1g, accept is returned with ECONNABORTEDThe server can ignore the error and just call accept again

Crashing of Server ChildProcess

Procedure:1. Kill the child. Server TCP sends FIN to client TCP, which responds with an ACK. (TCP half-close) (The client process is blocked in fgets when client TCP receives FIN.)2. SIGCHLD signal is sent to the server parent.3. User inputs. The client process calls writen to send data to server.4. The server TCP responds with an RST.5. The client process returns from readline, 0, when client TCP receives RST.6. The client process terminates.

The client is blocked on the call to fgets when FIN is received.(it should block on input from either source)Solution:Use select or poll to block on either socket or stdio

Client 沒辦法同時等待兩個 input

SIGPIPE Signalwhen writing to a socket that has received an RST

Procedure:1. The client writes to a crashed server process. An RST is received at the client TCP and readline returns 0 (EOF).2. If the client ignores the error returned from readline and write more, SIGPIPE is sent to the client process.3. If SIGPIPE is not caught, the client terminates with no output

Problem:Nothing is output even by the shell to indicate what has happened.(Have to use “echo $?”to examine the shell’s return value of last command.)Solution:1. Setting the signal disposition to SIG_IGN2. Catch the SIGPIPE signal for further processing. (handle EPIPE

error returned from write).

An Example to Show SIGPIPE To invoke tcpcli11 which has two write operations to show a

n example of writing to a closed socket The first write to the closed socket is to solicit RST from the server TCP The second write is to generate SIGPIPE from the local process. An sample run :

linux% tcpcli11 127.0.0.1Hi there # user input in boldHi there # echoed back from server

# terminate server child process thenBye # then type this line purposelyBorken pipe # output by the shell because of SIGPIPE

Note: To write to a socket which has received a FIN is OK. However, it is an error to write to a socket hat has received an RST

34

str_cli() – Calling writen() Twice tcpcliserv/str_cli11.c

35

Crash of Server Host Scenario

1. client and server run on different hosts 2. make a connection between client and server 3. client types something to transmit data to the server 4. disconnect the server host from the network (destination unr

eachable) 5. client types something again.

client TCP continuously retx data and timeout around 9 min The client process will then return with the error ETIMEDOUT. If some intermediate router determined that the server host was do

wn and responded with an ICMP “destination unreachable” message, the error returned will then be either EHOSTUNREACH or ENETUNREACH

To quickly detect: timeout on readline, SO_KEEPALIVE socket option, heartbeat functions

Reboot of Server Host The client does not see the server hos

t shut down Client sends data to server after the se

rver reboots server TCP responds to client data wit

h an RST because it loses all connection information

readline returns ECONNRESET

Shutdown of Server Host (byOperator)

init process sends SIGTERM to all processes We can catch this signal and close all

open descriptors by ourselves init waits 5-20 sec and sends

SIGKILL to all processes all open descriptors are closed by

kernel

Summary of TCP Example From client’s perspective:

socket and connect specifies server’s port and IP

client port and IP chosen by TCP and IP respectively

From server’s perspective: socket and bind specifies server’s local

port and IP listen and accept return client’s port and

IP

TCP Client/Server – Client’s Perspective

40

TCP Client/Server – Client’s Perspective

41

Data Format: Text Strings

server process gets two numbers (in a line of text) from client and returns their sum

In str_echo: sscanf converts string to long integer, snprintf converts long back to string

str_echo() – Adding 2 Numbers tcpcliserv/str_echo08.c

43

Data Format: Binary Structure Passing binary structure between client and

server does not work when the client and server are run on hosts with

different byte orders or sizes of long integer Since different implementations can store the s

ame C datatype differently. Suggestions:

pass in string only explicitly define the format of data types (e.g. R

PC’s XDR -- external data representation)

str_cli() – Sending 2 Binary Int’s tcpcliserv/str_cli09.c

45

str_echo() – Adding 2 Binary Int’s tcpcliserv/str_echo09.c

46

Beware of Different Byte Orders Due to the big-endian and little-endian implementations, s

ending binary numbers between different machine architectures may end up with different results

An example of two big-endian SPARC machines : solaris% tcpcli09 12.106.32.254 11 12 # user input in bold 33 # result back from server -11 -14 -55

An example of big-endian SPARC and little-endian Intel machines : linus% tcpcli09 206.168.112.96 1 2 # user input in bold 3 # It seems to work -22 -77 -16777314 # oops! It does not work!

47