What happens when many clients want to contact the server at the same time ?

The iterative approach consist in “enqueuing” the requests (this is done automatically by the system) and serving them one by one : Accept the next connection and build the socket Read request Process request (send the response)

If there is a client with a small request will have to wait until the large requests are over

If more than the allowed number of clients at socket queue request a service during this time they are rejected !!!

There may be also some delays in serving clients by waiting some information (client are asked to type in something)

There are many Disk I/O operations in a file serving scenario which are normally “slow”and do not require network traffic or CPU

So there is a lot of resources standing idle with this schema

A sequential (iterative) server attending more than a client

A SERVER A CLIENT

A CLIENT

A CLIENT

4444

During the transfer of the file the server cannot hear at the port 4444 for requests

A SERVER A

CLIENT

A CLIENT

A CLIENT

4444

Only after the transmition is over the server can attend another request

A SERVER A CLIENT

A CLIENT

A CLIENT

4444

This may involve some delay (typing, transmitting large files)

A SERVERA CLIENT

A CLIENT

A CLIENT

4444

Sometimes a large waiting time for the client may mean a time out for the connection

A SERVER A

CLIENT

A CLIENT

A CLIENT

4444

Timeout

The Concurrent Server A concurrent server can serve many clients at the

same time While transferring a file, it can still keep listening

for requests Every time a request comes, a new parallel line of

statements execution is started for attending this request. After this the server can “hear” again at the server socket for further requests

Different approaches have been developed to implement parallel lines of executions within a program

The operative system plays an important role

In concurrent servers there is a main program which listens for client requests

A SERVER

A CLIENT

A CLIENT

A CLIENT

4444

After a client is contacted, a new line of execution is created which will attend the

request in parallel.

A SERVER

A CLIENT

A CLIENT

A CLIENT

4444

The main program will return to the listening loop while the first client is being attended, so

it can accept another request

A SERVER

A CLIENT

A CLIENT

A CLIENT

4444

… and create another parallel line of execution for attending the client

A SERVER

A CLIENT

A CLIENT

A CLIENT

4444

… so a request of a third client can be immediately be accepted

A SERVER

A CLIENT

A CLIENT

A CLIENT

4444

… thus having the three clients being attended in parallel

A SERVER

A CLIENT

A CLIENT

A CLIENT

4444

The Concurrent Server Algorithm

Master program of the server1. Create a server socket 2. Wait for and accept a client’s request

3. Create a parallel line of execution to attend the requirement of the new client4. Goto 2.

Slave (parallel) process :1. Receive the parameters of the communication established by the master (socket or input/output data streams)2. Attend client’s request (read filename, transfer data)3. Return (disappear !)

Parallel Lines of execution If there is only 1 CPU, why create parallel processes?

It often makes server programming easier. Because there are more CPU involved !!!!!

The concept of parallel processes implemented at the S.O level appeared with UNIX and C.

In C a new process can be created by executing the fork() statement

int i = fork() creates an exact copy of the running process. Both continue executing the same program with the same variables.

For the “father” the value of i will be the id of the created process. For the child process this value will be 0

When programming concurrent servers the father will be the main process accepting requests and the child will process the client’s request

A simplified example of the use of for for implementing concurrent servers

main() { int pid, msock, ssock; msock = passivesock(port, “tcp”, qlen); /* see chapter 10.4 from Internetworking with tcp/ip from Douglas

Commer for the implementation */

while(1) { ssock = accept(msock, &fsin, &alen);

pid = fork(); if (pid == 0) { process client’s request; return; } }

Problems with fork() in UNIX The creation of a new process is a time costly procedure

In some texts it is suggested to create a set of processes at the beginning which will be activated later by the main program. When a client arrives the parameters of the communication are passed vi pipes to child.

The new created process duplicates all the variables !!! It is not easy to manage the processes which ended in an

abnormal way. They keep the resources !!!. Sometimes is preferred to use the “select” statement: it

basically selects from an array of sockets the first one which has available data to read.

In JAVA is better to use Threads

A thread is a sequence or a flow of instructions which are executed in parallel to the main sequence of the program. It has a start and an end.

A thread can only be created and lives inside an already running process. A process can start as many threads as necessary.

Because of this, the main program has a better control of the threads it started. They can be created, started, suspended, or reactivated by the program.

Threads are implemented as methods of a certain class. When this method (normally called run) is activated, it starts to run in parallel to the method that called it.

As any other method, it can define its own set of variables

Using threads to implement concurrent servers

Main (master) program:

Create ServerSocket ss;while(true) {

Socket s = ss.accept();Create new thread with the socket s as parameter;start executing thread;

}

Define a thread class with a method called run which implements the processing of the clients request

Exampleimport java.io.*; import java.net.*; public class ArchServidor { public static void main(String[] args) throws Exception { Socket cs = null; while(true) { System.out.println("Waiting for a client ... "); cs = ss.accept(); System.out.println(cs.getInetAddress()+" contacted "); System.out.println(“Creating thread to serve request"); ServeFileThread t = new ServeFileThread(cs); t.start(); } }}

Implementation of Threads

Define a new class which is an extension of the Thread class and overwrite its run() method.Thread is an existing class This thread class has a run method (originally with no

statements) which will be executed in parallel when activated. In order to execute this method a program must create an

object of the extended class and apply the start() method to it. The definition of the new Thread class should look like

this:public class MyThread extends Thread {

And somewhere should appear: public void run() { //instructions to be executed in parallel }

Examplepublic class ServeFileThread extends Thread { Socket cs; public ServeFileThread(Socket x) { cs = x; } public void run() {

BufferedReader inSocket = new BufferedReader( new InputStreamReader(cs.getInputStream()));

String fileName = inSocket.readLine(); FileInputStream inFile = new

FileInputStream(fileName); OutputStream outSocket = cs.getOutputStream(); System.out.println("sending file "+fileName); int b; byte ab[] = new byte[1024];

while ((b= inFile.read(ab,0,1024) ) != -1) { outSocket.write(ab,0,b); }

outSocket.close(); inFile.close(); inSocket.close(); } }

About threads and servers See ActiveClock2 for an example of threads

whose execution can be stopped and resumed This is NOT the way the to stop threads (look at

the API why suspend and resume are deprecated) See ActiveClock3 for a better solution. Servers normally implement an infinite loop for

accepting clients. In order to allow a clean stop of the server it

should be normally be implemented as a thread

Example2-1import java.io.*; import java.net.*; public class ArchServidor extends Thread { static boolean seguir = true; public static void main(String[] args) throws Exception { //get resources ArchServidor as = new ArchServidor(); as.start(); //release resources Scanner s = new Scanner(System.in); System.out.println(“press enter to stop server propperly”); String s.nextLine(); seguir = false; }

Example2-2public void run() { Socket cs = null; while(seguir) { System.out.println("Waiting for a client ... "); cs = ss.accept(); System.out.println(cs.getInetAddress()+" contacted "); System.out.println(“Creating thread to serve request"); ServeFileThread t = new ServeFileThread(cs); t.start(); } }}

Sometimes you cannot program the server as an extension of a thread

For example, if the server has to implement a graphical interface it should be declared as an extension of a Frame, if it is programmed as an Applet it must extend the Applet class

The we can use the interface Runnable, which means the Server object will also be seen as a Runnable class and has to implement the run() method.

To initiate a parallel execution of the run method a Thread object must be created. In the creation a pointer to the server object (this) should be passed as parameter. With this, the Thread object will have access to the run method.

The run method implemented in the server class will be executed by performing start() on the new created Thread object.

First Examplepublic class MyServer implements Runnable {

MyServer() {

//this is the constructor

}

public void run() {

//code to be executed in a thread

}

public static void main(String args[]) {

MyServer mc = new MyServer();

Thread t = new Thread(mc);

t.start();

}

}

Second Examplepublic class MyServer implements Runnable {

MyServer() {

//this is the constructor

Thread t = new Thread(this);

t.start();

}

public void run() {

//code to be executed in a thread

}

public static void main(String args[]) {

MyServer mc = new MyServer();

}

}

Example: A simple concurrent web server First version: (Httpd1.java) only to serve files (html and images) but it is

easily extensible for serving different types of requirements For each client a new thread is created The filename of the request is not filtered in order to keep the program

small

bc1

GET fileX HTTP1.1

bc2

GET fileY HTTP1.1

tb2

tb1

read fileYread fileX

send fileY

send fileX

tbi = thread for browser ibci = browser for computer i

A web server generating dynamic html content Instead of asking for a filename the browser asks for a classname. In order

to tell the server that the requested file is a class the browser has to send a specific

When this is the case, an object of this class is created and a certain (predefined process ) method is invoked

The method of this class writes directly to the browser through the output stream of the socket

bc2

GET /servlet/servletX HTTP1.1

tb2

Creates object

send fileX

tbi = thread for browser ibci = browser for computer ioti = object created for thread i

ot2

writes output generated dynamically(e.g. the current time)

GET /servlet/servletY?querystring HTTP1.1

A web server generating dynamic html content With this schema we can define different classes of objects which extend from

BaseServlet and overwrite the process method in order to have them doing different things

For Example, a class calls MyServlet and extends BasicServlet and overwrites the process method

Browser contacts server with request /servlet/MyServlet Servlet creates a MyServlet object (casted as BasicServlet) and invokes

process method The process method of MyServlet will be performed

Extends

MyServlet

BasicServlet

Creates object

ot2 Is of class

See: Echo

Implementing state in a web server

The server must keep tracking of the different users that contact it

The client must submit this information The Echo2 class keeps track of a

“shopping cart” for different users The client must submit a request like:

u=username&p=productcode&q=number

A number provider

N++

NoSyncronClient

Ask number

Give Number

Ask numberAsk Number

Give Number Give Number

NoSyncronClientNoSyncronClient

NoSincronServer

Critical regions in Java A critical region is a piece of code that should be

executed by only one thread at the same time to prevent concurrent access to some variables or resources.

Java provides basically two methods for controlling concurrent access to resourcesYou can declare a whole method as a critical region

Just put synchronized at the beginning of the method declaration

You can use the semaphore of any object Create any object Object o = new Object() Start the critical region with synchronized(o) {…….}

Another way to use semaphores in java is with wait() and notify()

Broadcasting a text to many clients

Hello

Hello

Hello

Hello

Hello

Hello

The server: receiving a new client

4444

Client contacts server at 4444

The server: receiving a new client

4444

A new socket is placed and the output channel is opened They are kept in an array

The server: Broadcasting a message

When a message is entered the server will distribute it to the connected clients

BroadcastCliente

BraodcastServerNF

Type a message

It is now easy to extend this to a chat system

Hello

Hello

Hello

Hello

Hello

Conditions for implementing a chat system

Server must be listening to requests of new clients AND to messages which are sent by already connected

Client must be listening to messages from the server AND to the keyboard for messages the user wants to transmit.

There are many approaches for implementing this in TCP/IP

No one is the “absolute correct” solution, all them have their advantages and drawbacks

Normally (like everywhere in computers) “faster” solutions will use more memory

Solution 1a The server:

1. A thread listens to new clients trying to join the chat party

2. When a new clients connects, a PrintWriter and a BufferedReader for that client are created. The print Writer is kept in a vector

3. A thread is created and receives a pointer to the PrintWriters’ vector and the BufferedReader. It reads the input from the client and writes it to all PrintWriters (clients)

The client 1. The main method (procedure) will contact the server, create a

thread and start it. Then it will read lines from the keabord and send it to the server

2. A thread will read input from the server and display it on a text area

See: Chat1Server.java, Chat1ClientThread.java & Chat1ClientNW

Solution 1b The server:

1. A thread listens to new clients trying to join the chat party

2. When a new clients connects, a PrintWriter and a BufferedReader for that client are created. The print Writer is kept in a vector

3. A thread is created and receives a pointer to the PrintWriters’ vector and the BufferedReader. It reads the input from the client and writes it to all PrintWriters (clients)

The client 1. Graphical interface for reading lines from the keyboard (there is

a thread which triggers the execution of the actionPerformed method) and sends it to the server

2. A thread will read input from the server and display it on a text area

See: Chat1Server.java, Chat1ClientThread.java & Chat1Client

Solution 1 Schema

Client1

Client2

Client3

New Client

Thread

PrintWriter

BufferedReader

Solution 2 The server:1. The server has only one thread listening at one port for attending

all the requests of the clients.

2. The clients contact the server for registering (reg), sending a message (msg) or logging out (des)

3. The server keeps a PrintWiter vector to keep track about which clients participate in the chat party. Additionally, it keeps a vector with the nicknames of the clients

The client 1. Graphical interface for reading lines from the keyboard (there is

a thread which triggers the execution of the actionPerformed method) and sends it to the server. It also shows the nicknames of the participants

2. A thread will read input from the server. This can be a message (msg) or a refreshment of the login list

See: CICChat.java & CICServer.java

Solution 2 Schema

Client1

Client2

New Client

Solution 2

Client1

Client2

Client3

Peer-to-peer solution for TCP/IP

Every program should be client and server at the same time

When a new member wants to join the party, he/she shoud contact anyone of the group and ask for the list of contacts

After retrieving the list of contacts (hostnames or addresses) he/she should open an input and output channel with everyone (including the one who provided the list)

Peer-to-peer solution: a user starts a chat party by listening on a port for others wanting to join

A user wanting to join contacts the initiator. An InputStream and an OutputStream is opened on each

A third user may contact anyone and recover the list of participants, in this case each one has the address of the other

in the list

Then contacts all of them opening Input/Output streams. All them now have two entries on their participant list

Particularities of this implementation

The program is written in the ChatPtP.java file In order to allow testing in one computer, it is necessary to

give the port number at which the program will listen for newcomers

java ChatPtP localport starts a new Chat party java ChatPtP remotehost remoteport localport joins an

existing party (on remotehost at remoteport a program is waiting for new members)

The is one thread for listening to newcomers Another listens for input on keyboard and sends it to all

connected participants There is one thread listening for input for each of the other

participants Not very easy, isn’t it ?

Problems of this implementation

What happens if one user leaves the chat party ? What happens if during the second connection

phase (retrieving the list and contacting the participants) another newcomer joins the party by asking a third one for the list ?

This is a very critical problem which has been more or less addressed in some (sometimes hard to understand) papers.