Processes and threads

30
Perl Programming Perl Programming Course Course Processes and threads Processes and threads Krassimir Berov I-can.eu

description

This is the twelfth set of slightly updated slides from a Perl programming course that I held some years ago. I want to share it with everyone looking for intransitive Perl-knowledge. A table of content for all presentations can be found at i-can.eu. The source code for the examples and the presentations in ODP format are on https://github.com/kberov/PerlProgrammingCourse

Transcript of Processes and threads

Page 1: Processes and threads

Perl Programming Perl Programming CourseCourse

Processes and threadsProcesses and threads

Krassimir Berov

I-can.eu

Page 2: Processes and threads

ContentsContents

1.1. What is a processWhat is a process

2.2. What is a threadWhat is a thread

3.3. ComparisonComparison

4.4. ThreadsThreads

• Threaded Program ModelsThreaded Program Models

• Creating and identifying ThreadsCreating and identifying Threads

• Thread ManagementThread Management

• Sharing DataSharing Data

• Synchronization and controlSynchronization and control

Page 3: Processes and threads

ContentsContents

5.5. ProcessesProcesses

• forkfork

• pipepipe

• openopen

• ......

Page 4: Processes and threads

What is a processWhat is a process

• processprocess

• An instance of a running programAn instance of a running program

• Two or more separate processes could be Two or more separate processes could be running the same program independently running the same program independently at the same timeat the same time

• A computer program itself is just a A computer program itself is just a passive collection of instructions, while a passive collection of instructions, while a process is the actual execution of those process is the actual execution of those instructionsinstructions

Page 5: Processes and threads

What is a threadWhat is a thread

• threadthread

• a thread is a flow of control through a program a thread is a flow of control through a program with a single execution pointwith a single execution point

• there can be several there can be several threads of executionthreads of execution within within a processa process

• multiple threads share the same program code, multiple threads share the same program code, operating system resources and operating operating system resources and operating system permissions as the process they belong system permissions as the process they belong toto

Page 6: Processes and threads

ComparisonComparison

• Thread versus processThread versus process

Page 7: Processes and threads

ComparisonComparison

• ProcessesProcesses

• run independently and do not share run independently and do not share resourcesresources

• the fork() system call in UNIX causes creation the fork() system call in UNIX causes creation of a new processof a new process

• on Windows it is emulated by using threadson Windows it is emulated by using threads

• The return value from fork() is used to The return value from fork() is used to distinguish the parent from the childdistinguish the parent from the child

• the parent receives the child's process id, but the parent receives the child's process id, but the child receives zerothe child receives zero

Page 8: Processes and threads

ComparisonComparison

• ProcessesProcesses

• The new process (child process) is an exact The new process (child process) is an exact copy of the calling process (parent process) copy of the calling process (parent process) except for the following:except for the following:

• The child process has a unique process IDThe child process has a unique process ID

• The child process has a different parent process The child process has a different parent process ID (i.e., the process ID of the parent process)ID (i.e., the process ID of the parent process)

• The child process has its own copy of the parent's The child process has its own copy of the parent's descriptorsdescriptors

(2)

Page 9: Processes and threads

ComparisonComparison

• ThreadsThreads

• A thread is an entity within a process that A thread is an entity within a process that consists of the schedulable part of the consists of the schedulable part of the processprocess

• Creating new threads is faster Creating new threads is faster

• Thread creation induces a peer Thread creation induces a peer relationship between all the threads of a relationship between all the threads of a processprocess

Page 10: Processes and threads

ComparisonComparison

• ThreadsThreads

• All threads can shareAll threads can share

• the parent process IDthe parent process ID

• the process memorythe process memory

• the process datathe process data

• the process permissionsthe process permissions

• the Table with opened filesthe Table with opened files

(2)

Page 11: Processes and threads

ComparisonComparison

• ThreadsThreads

• Every thread has its ownEvery thread has its own

• thread IDthread ID

• separate point of executionseparate point of execution

• thread-local storagethread-local storage

(2)

Page 12: Processes and threads

Threaded Program ModelsThreaded Program Models

• Three basic ways that you can structure a Three basic ways that you can structure a threaded programthreaded program

• Boss/Worker – one Boss/Worker – one bossboss thread and one or more thread and one or more workerworker threads threads

• Work Crew – several threads are created that do Work Crew – several threads are created that do essentially the same thing to different pieces of essentially the same thing to different pieces of datadata

• Pipeline – a task is divided into a series of stepsPipeline – a task is divided into a series of steps

• the results of one step are passed to the thread the results of one step are passed to the thread processing the next stepprocessing the next step

• Each thread does one thing to each piece of dataEach thread does one thing to each piece of data

Page 13: Processes and threads

Creating Creating and identifying Threadsand identifying Threads

• ExampleExample#threads_create.pl#threads_create.plBEGIN {BEGIN { use Config;use Config; $Config{useithreads} $Config{useithreads} or die('Threads support needed.');or die('Threads support needed.');}}use strict;use warnings;use strict;use warnings;use threads;use threads;$|++;$|++;while (1){while (1){ sleep 1;sleep 1; my $thread_id = threads->self()->tid;my $thread_id = threads->self()->tid; threads->create(\&a_thread,$thread_id,[localtime]);threads->create(\&a_thread,$thread_id,[localtime]); #OR#OR threads->new(\&a_thread,$thread_id,[localtime]);threads->new(\&a_thread,$thread_id,[localtime]);}}#...#...

Page 14: Processes and threads

Thread ManagementThread Management

• Waiting For A Thread To ExitWaiting For A Thread To Exit• joinjoin

• waits for a thread to exit, waits for a thread to exit, • cleans up after it, cleans up after it, • returns any data the thread may have producedreturns any data the thread may have produced

#threads_management.pl#threads_management.plwhile ($i<30){while ($i<30){ sleep 1;sleep 1; my $odd = threads->create(\&a_thread,[localtime]);my $odd = threads->create(\&a_thread,[localtime]); print $odd->print $odd->join()join(),$/;,$/; my $even = threads->new(\&a_thread,[localtime]);my $even = threads->new(\&a_thread,[localtime]); print $even->print $even->join()join(),$/;,$/; $i++;$i++;}}#...#...

Page 15: Processes and threads

Thread ManagementThread Management

• Ignoring A ThreadIgnoring A Thread

• detachdetach• the thread runs until it's finishedthe thread runs until it's finished• Perl will clean up after it automaticallyPerl will clean up after it automatically• may not be joinedmay not be joined• any return data is lostany return data is lost

#threads_management2.pl#threads_management2.plwhile ($i<30){while ($i<30){ sleep 1;sleep 1; my $odd = threads->create(\&a_thread,[localtime]);my $odd = threads->create(\&a_thread,[localtime]); $odd->$odd->detach()detach();; my $even = threads->new(\&a_thread,[localtime]);my $even = threads->new(\&a_thread,[localtime]); $even->$even->detach()detach();; $i++;$i++;}}#...#...

Page 16: Processes and threads

Thread ManagementThread Management

• Process and Thread TerminationProcess and Thread Termination

• an action that terminates a process will an action that terminates a process will terminate all running threads.terminate all running threads.

• perl does an exit() when the main thread exitsperl does an exit() when the main thread exits

#threads_management3.pl#threads_management3.plmy @threads = ();my @threads = ();while ($i<30){while ($i<30){ push @threads, threads->new(\&a_thread,[localtime]);push @threads, threads->new(\&a_thread,[localtime]); push @threads, threads->new(\&a_thread,[localtime]);push @threads, threads->new(\&a_thread,[localtime]); $i++;$i++;}}#uncomment and run again#uncomment and run again#print $_->join foreach @threads;#print $_->join foreach @threads;#...#...

Page 17: Processes and threads

Sharing DataSharing Data

• by default, no data is sharedby default, no data is shared

• all the data associated with the current thread is all the data associated with the current thread is copied to the new thread, and is subsequently copied to the new thread, and is subsequently private to that new threadprivate to that new thread

• all happens within the current processall happens within the current process#sharing_data.pl#sharing_data.plmy @threads = ();my @threads = ();while ($i<30){while ($i<30){ push @threads, threads->new(\&a_thread,[localtime]);push @threads, threads->new(\&a_thread,[localtime]); push @threads, threads->new(\&a_thread,[localtime]);push @threads, threads->new(\&a_thread,[localtime]); $i++;$i++;}}#uncomment and run again#uncomment and run again#print $_->join foreach @threads;#print $_->join foreach @threads;#...#...

Page 18: Processes and threads

Sharing DataSharing Data

• by default, no data is sharedby default, no data is shared

• all the data associated with the current thread is all the data associated with the current thread is copied to the new thread, and is subsequently copied to the new thread, and is subsequently private to that new threadprivate to that new thread

• all happens within the current processall happens within the current process

• use threads::shared and the :shared attributeuse threads::shared and the :shared attributeto share datato share data

• only simple values or references to shared only simple values or references to shared variables are allowedvariables are allowed

Page 19: Processes and threads

Sharing DataSharing Data

•Race conditionsRace conditions● caused by unsynchronized access to shared caused by unsynchronized access to shared

datadata

● there's no way to be sure that nothing has there's no way to be sure that nothing has happened to the shared data between the time happened to the shared data between the time you access it and the time you update ityou access it and the time you update it

● $a += 5 or $a++ are not guaranteed to be $a += 5 or $a++ are not guaranteed to be atomicatomic

Page 20: Processes and threads

Synchronization and controlSynchronization and control

• locklock

• takes a shared variable and puts a lock on ittakes a shared variable and puts a lock on it

• no other thread may lock the variable until the no other thread may lock the variable until the variable is unlocked by the thread holding the lockvariable is unlocked by the thread holding the lock

• Unlocking happens automatically when the locking Unlocking happens automatically when the locking thread exits the block that contains the call to the thread exits the block that contains the call to the lock() functionlock() function

• blocks the thread until the variable being locked is blocks the thread until the variable being locked is availableavailable

• your thread can be sure that no other thread can lock your thread can be sure that no other thread can lock that variable until the block containing the lock exitsthat variable until the block containing the lock exits

• does not prevent access to the variable, only lock does not prevent access to the variable, only lock attemptsattempts

Page 21: Processes and threads

Synchronization and controlSynchronization and control

• lock – Examplelock – Example#deadlock.pl#deadlock.pluse threads; use threads::shared;use threads; use threads::shared;my $a :shared = 4;my $a :shared = 4;my $b :shared = 'foo';my $b :shared = 'foo';my $thr1 = threads->create(sub {my $thr1 = threads->create(sub { lock($a)lock($a);; sleep(2);sleep(2); lock($b)lock($b);; $a++; $b .= $a;$a++; $b .= $a;})->join ;})->join ;my $thr2 = threads->create(sub {my $thr2 = threads->create(sub { lock($b)lock($b);; sleep(2);sleep(2); lock($a)lock($a);; $a++; $b .= $a;$a++; $b .= $a;})->join ;})->join ;print $thr1,$/,$thr2,$/;print $thr1,$/,$thr2,$/;

Page 22: Processes and threads

Synchronization and controlSynchronization and control

• QueuesQueues

• A queue is a special thread-safe object A queue is a special thread-safe object that lets you put data in one end and take that lets you put data in one end and take it out the other without having to worry it out the other without having to worry about synchronization issuesabout synchronization issues

• add lists of scalars onto the end with add lists of scalars onto the end with enqueue()enqueue()

• pop scalars off the front of it with pop scalars off the front of it with dequeue()dequeue()

Page 23: Processes and threads

Synchronization and controlSynchronization and control

• SemaphoresSemaphores

• generic locking mechanismgeneric locking mechanism

• behave very much like lockable scalars, behave very much like lockable scalars, except that they can't hold dataexcept that they can't hold data

• must be explicitly unlockedmust be explicitly unlocked

• by default, semaphores behave like locksby default, semaphores behave like locks

• see also: perlthrtut/Advanced Semaphoressee also: perlthrtut/Advanced Semaphores

Page 24: Processes and threads

ProcessesProcesses

• forkforkDoes a fork(2) system call to create a new Does a fork(2) system call to create a new process running the same program at the same process running the same program at the same pointpoint

• returns the child pid to the parent process, 0 to the returns the child pid to the parent process, 0 to the child process, or undef if the fork is unsuccessfulchild process, or undef if the fork is unsuccessful

• file descriptors are shared, while everything else is file descriptors are shared, while everything else is copiedcopied

• beginning with v5.6.0, Perl will attempt to flush all beginning with v5.6.0, Perl will attempt to flush all files opened for output before forking the child files opened for output before forking the child processprocess

Page 25: Processes and threads

ProcessesProcesses

• fork – Examplefork – Example#io-socket-tcp-server-fork.pl#io-socket-tcp-server-fork.pl#...#...sub REAPER {sub REAPER { 1 until (-1 == waitpid(-1, WNOHANG));1 until (-1 == waitpid(-1, WNOHANG)); $SIG{CHLD} = \&REAPER;$SIG{CHLD} = \&REAPER;}}$SIG{CHLD} = \&REAPER;$SIG{CHLD} = \&REAPER;

print "Server ($0) running on port $port...\n";print "Server ($0) running on port $port...\n";while (my $connection = $server->accept) {while (my $connection = $server->accept) { if (my $pid = fork){if (my $pid = fork){ handle_connection($connection,$$);handle_connection($connection,$$); }}}}$server->close();$server->close();

Page 26: Processes and threads

IPCIPC

• pipe READHANDLE,WRITEHANDLEpipe READHANDLE,WRITEHANDLE• Opens a pair of connected pipes like the Opens a pair of connected pipes like the

corresponding system call. corresponding system call. • Perl's pipes use IO buffering, so you may need to set Perl's pipes use IO buffering, so you may need to set

$| to flush your WRITEHANDLE after each command, $| to flush your WRITEHANDLE after each command, depending on the application.depending on the application.

pipe (READ, WRITE);pipe (READ, WRITE);select WRITE;select WRITE;$| = 1;$| = 1;#...#...

Page 27: Processes and threads

IPCIPC

• system PROGRAM LISTsystem PROGRAM LIST

• exactly the same thing as exec LIST , except exactly the same thing as exec LIST , except that a fork is done first, and the parent that a fork is done first, and the parent process waits for the child process to process waits for the child process to completecomplete

• The return value is the exit status of the The return value is the exit status of the program as returned by the wait callprogram as returned by the wait call

• see perlfunc/systemsee perlfunc/system

Page 28: Processes and threads

IPCIPC

• openopen

open(MAIL, "|/usr/lib/sendmail -oi -t") open(MAIL, "|/usr/lib/sendmail -oi -t") or die "can't fork sendmail: $!";or die "can't fork sendmail: $!";

print MAIL <<EOF;print MAIL <<EOF;From: $0 From: $0 To: you\@example.comTo: you\@example.comSubject: blahSubject: blah

EOFEOF

close(MAIL) close(MAIL)

Page 29: Processes and threads

Processes and threadsProcesses and threads

• RessourcesRessources• Professional Perl Programming Professional Perl Programming

(Chapter 22 – Creating and Managing Processes)(Chapter 22 – Creating and Managing Processes)

• perldoc perlthrtutperldoc perlthrtut

• http://en.wikipedia.org/wiki/Process_%28computing%29http://en.wikipedia.org/wiki/Process_%28computing%29

• http://en.wikipedia.org/wiki/Thread_%28computer_science%29http://en.wikipedia.org/wiki/Thread_%28computer_science%29

• http://gauss.ececs.uc.edu/Users/Franco/ForksThreads/forks.htmlhttp://gauss.ececs.uc.edu/Users/Franco/ForksThreads/forks.html

• perldoc perlipcperldoc perlipc

• perldoc perlforkperldoc perlfork

Page 30: Processes and threads

Processes and threadsProcesses and threads

Questions?Questions?