ECE 297

Concurrent Servers Process, fork & threads

ECE 297

ECE 297

Cache

CacheCache Cache Cache

How do you handle cache updates?

How do you handlecache invalidation?

Keep it simple

Process-basedserver

ECE 297

file

How do you handle concurrent access to files?

Careful with writingto the same filein different processes!

Process-basedserver

ECE 297

Process versus thread IProcess• Unit of resource ownership with respect to the execution

of a single program• Can encompass more than one thread of execution

– E.g., Web browser: More than one thread (process) per window/tab, GUI, rendering engine etc.

– E.g., Web server: More than one thread for handling requests

Thread• Unit of execution• Belongs to a process• Can be traced (i.e., list the sequence of instructions)

ECE 297

Process versus thread II

• A.k.a. lightweight process (LWP), threads, multi-threaded processes

ECE 297

Process versus thread III

Per process items• Address space• Global variables• Open files• Child processes• Pending alarms• Signal and signal

handlers• Accounting information

Per thread items• Program counter• Registers• Stack

ECE 297

Use• Processes are largely

independent and often compete for resources

Use• Threads are part of the

same “job” and are actively and closely cooperating

OS OS

Threads Threads

Process 1 Process 2 Process 3 Process

ECE 297

Threads

OS

Threads

Thread 1’sstack

Process

Thread-based server

• Server design alternatives– Thread-per-request– Thread-per-client– Thread-per connection

• The new thread can access all resources held by the process that created it

• For example, the cache, open data files, global variables are all available to the threads– Unlike for process-based servers

ECE 297

pthreads API overview

• pthread_create(…): creates a thread• pthread_wait(…): waits for a specific

thread to exit• pthread_exit(…): terminates the

calling thread• pthread_yield(…): calling thread

passes control voluntarily to another thread

p is for POSIX

Thread priority, initial stack size, …; NULL for defaults

Pointer to argument for

function

ECE 297

pthreads API I

#include <pthread.h>

pthread_create(pthread_t *tid,

const pthread_attr_t *attr,

void *(*func) (void *),

void *arg);

Returns 0, if OK, positive Exx on error

p is for POSIX

Thread ID

Function to execute; the

actual “thread”

ECE 297

pthreads API IV

• pthread_self(void)– Returns thread ID to caller

• pthread_detach(pthread_t thread)– Indicates to system that storage for thread can be

reclaimed

• There are many other pthread API calls, the above should suffice for our purposes

p is for POSIX

ECE 297

Thread-based servervoid *thread(void *vargp); int *connfdp;int main(int argc, char **argv) { … pthread_t tid; … listenfd = socket(…); … listen(listenfd, …)

// main server loop for( ; ; ) { connfdp = malloc(sizeof(int)); … *connfdp = accept(listenfd,

(struct sockaddr *) &clientaddr, &clientlen);

pthread_create(&tid, NULL, thread, (void *) connfdp);

} // for} // main

We create the thread to handle

the connected client.

ECE 297

The actual thread to handle the client

void *thread(void *vargp) { int connfd;

// detached to avoid a memory leak pthread_detach(pthread_self());

connfd = *((int *)vargp); free(vargp);

// do the work, service the client

close(connfd); return NULL;}

This is where the client gets serviced

ECE 297

listenfd = socket(AF_INET, SOCK_STREAM, 0)…bind(listenfd, …)

listen(listenfd, …)

for( ; ; ){ …connfd = accept(listenfd, …);…if ( (childPID = fork()) == 0 ){// The Child!

close(listenfd); //Close listening socketdo the work //Process the requestexit(0);

} …close(connfd); //Parent closes connfd

}

Concurrent server template

ECE 297

Issues with thread-based servers

• Must be careful to avoid unintended sharing of variables

• For example, what happens if we pass the address of connfd to the thread routine?pthread_create(&tid, NULL, thread,

(void *)&connfd);

• Must protect access to

intentionally shared data– Here, we got around this by creating a new

variable, but in general …

Would be a shared variable

Complications

• Imaging a global variable counter in the process

– For example the storage server in-memory cache (more complex structure)

– Or the connfd variableLet’s dissect the issue in

detail

!

Shared data & synchronization

ECE 297

TableTableTable

What happens if multiple threads concurrently accessshared process state (i.e., memory)?

Concurrently manipulating shared data

• Two threads execute concurrently as part of the same process

• Shared variable (e.g., global variable)– counter = 5

• Thread 1 executes– counter++

• Thread 2 executes– counter—

• What are the possible values of counter after Thread 1 and Thread 2 executed?

ECE 297

counter

ECE 297

Machine-level implementation

• Implementation of “counter++”

register1 = counter

register1 = register1 + 1 counter = register1

• Implementation of “counter--” register2 = counter register2 = register2 – 1 counter = register2

ECE 297

Possible execution sequences

counter++

counter--

Context Switch

counter++

counter--

Context Switch

Context Switch

Context Switch

Context Switch

ECE 297

Interleaved execution• Assume counter is 5 and interleaved execution of

counter++ (P) and counter– (C) T1: r1 = counter (register1 = 5)T1: r1 = r1 + 1 (register1 = 6)T2 : r2 = counter (register2 = 5)T2 : r2 = r2– 1 (register2 = 4)T1 : counter = r1 (counter = 6)T2 : counter = r2 (counter = 4)

• The value of counter may be either 4 or 6, where the correct result should be 5.

contextswitch

ECE 297

Race condition

• Race condition: – Several threads manipulate shared data

concurrently. The final value of the data depends upon which thread finishes last.

• In our example (interleaved execution) for c++ last, result would be 6, and for c-- last, result would be 4 (correct result should be 5)

• To prevent race conditions, concurrent processes must be synchronized.

ECE297

The moral of this story

• The statementscounter++;counter--;must each be executed atomically.

• Atomic operation means an operation that completes in its entirety without interruption.

• This is achieved through synchronization primitives (semaphores, locks, condition variables, monitors, disabling of IRPs …).

Synchronization primitives

• Semaphore (cf. ECE344)• Monitor (cf. ECE344)• Condition variable (cf. ECE344)• Lock

– Prevent data inconsistencies due to race conditions

– A.k.a. mutex (mutual exclusion)– Use to protect shared data within a process– Can not be used across processes

• Need to use semaphore instead

ECE 297

Mutex: Mutual exclusion

pthread_mutex_lock(pthread_mutex_t *mtpr)

pthread_mutex_unlock(pthread_mutex_t *mtpr)

Returns 0, if OK, positive Exx on error

• There are other abstractions, but the mutex should suffice for us

• NB: In ECE344 we learn how to implement locks.

ECE 297

The pthreads mutex (lock)

pthread_mutex_t my_cnt_lock = PTHREAD_MUTEX_INITIALIZER;

int counter=0;

pthread_mutex_lock( & my_cnt_lock );counter++;pthread_mutex_unlock( & my_cnt_lock );…

ECE 297

Mutex is for mutual exclusion

ECE 297

For statically allocated mutexes.

pthread_mutex_lock(& my_cnt_lock);

counter++;

pthread_mutex_unlock(& my_cnt_lock);

pthread_mutex_t my_cnt_lock = PTHREAD_MUTEX_INITIALIZER

Guaranteed to execute atomically

pthread_mutex_lock(& my_cnt_lock);counter--;

pthread_mutex_unlock(& my_cnt_lock);

Guaranteed to execute atomically

ECE 297

Possible execution sequences

counter++

counter--

Context Switch Context Switch

lock

unlock

lock

unlock

counter++

lock

unlock

counter--

lock

unlock

lock

lock

lock

lock

Watch out for I

• For all shared data access you must use a synchronization mechanism

• For Milestone 4 based on threads, you can get by with the mutexes

• Other useful mechanisms in pthreads are– pthread_join(…)– pthread_cond_wait(…) & pthread_cond_signal()

• Bugs due to race conditions are extremely difficult to track down– Non-deterministic behaviour of code

ECE 297

Watch out for II

• You can not make any assumption about thread execution order or relative speed

• Threaded code must use thread-safe functions– Functions that use no static variables, no global

variables, don’t return pointers to static variables• Otherwise need to protect call to non-thread-safe code with

mutexes• Non-thread-safe code also called non-reentrant code

– Function local data is allocated on the stack• Deadlocks

– Code halts, as threads may wait indefinitely on locks– Cause is programmer error or poorly written code

ECE 297

ECE 297

Pros & cons of threads-based servers

• Probably the simplest option –No zombies, no signal handling, no onerous data

structures

• “Easy” to share data structures between threads–Logging information, data files, cache, …

• Thread creation is more efficient than process creation

• Enables concurrent processing of requests from multiple clients

ECE 297

Pros & cons cont.’d

• Unintentional sharing can introduce subtle and hard to reproduce race conditions

• malloc an argument (struct) for each thread and pass pointer to variable to thread and free after use

• Keep global variables to a minimum• If a thread references a global variable

•protect it with a mutex or • think carefully about whether unprotected variable is safe–e.g., one writer thread vs. multiple readers is OK.