CSCI 243 The Mechanics of Programming The OS: Process...

CSCI 243The Mechanics of Programming

Timothy Fossum ([email protected])

TVF / RIT 20191

The OS: Process Creation

TVF / RIT 20191 CS243: The OS: Process Creation

Process Creation

• Typically, via a system call• Process executes a system call

• OS creates a new process in response

• Chicken/egg problem:• To create a process, execute a syscall

• To execute a syscall, must already be in a process


Resolving the Problem

• OS typically creates first user-level process manually• UNIX®/Linux®: init

• All other processes are descended from it

UNIX ®is a registered trademark of The Open Group Ltd.Linux® is the registered trademark of Linus Torvalds in the United States and other countries.


Process Creation

• Classic method:

• Duplicates the executing process• Executing same code, at same place

• Same content in data areas

• A few differences:• Different PID, PPID

• Some file descriptors and other resources not inherited by child

• Some OS-level settings and data structures are not inherited

• Return value from system call

• Child may have its own runtime stack

pid_t fork( void );


Process Creation

• Child and parent are at exactly the same point• To each, it appears that they just returned from the fork()

• Return values:• -1: failure - no child was created, errno contains error code

• 0: success - this is the child process

• Else: success - this is the parent process

• Typical code structure:id = fork();switch( id ) { case 1: /* error */ break; case 0: /* executing in child */ break; default: /* executing in parent */}


Process Creation

• Originally, fork() duplicated all allocated memory• Wasteful - will all be thrown away when exec() occurs

• BSD introduced vfork() variation• Doesn’t duplicate any memory - much faster

• Parent and child share all memory

• Potentially dangerous (why?)

• Linux introduced clone()• More options, more flexible


Process Creation

• AT&T introduced copy-on-write version of fork()• Relies on VM page-sharing

• No memory is duplicated initially

• A page is only duplicated when one process writes to it• Duplicate is created for that process

• Other process(es) continue using the original version

• Reduces memory usage• The only unique-to-process pages are ones that are different

• No need to discard lots of allocated memory on an exec()


Child Status

• Sometimes, parent waits for child to terminate

• Blocks process until any child has a state change• Terminates, stopped by a signal, resumed by a signal

• If no children, returns immediately

• Two return values:• Intrinsic: PID of child, or -1 if no child

• Termination status of child in status (if not NULL)

• Examine with macros (defined in <sys/wait.h>)• WIFEXITED(status)

• WEXITSTATUS(status)

• WIFSIGNALED(status)

pid_t wait( int *status );


Child Status

• Variant:

• Waits for a specific child (pid)• > 0: specific PID

• -1: any child

• Same return values• Intrinsic: PID of child, or -1 if no child

• Termination status of child in status (if not NULL)

• Can modify behavior with options parameter• Default: waits for termination

• WNOHANG: return immediately if child hasn’t exited

pid_t waitpid( pid_t pid, int *status, int options);


Child Status

• Why bother using wait()?

• All terminating processes have a status• return from main(), call to exit()

• If parent is still active:• Child “hangs around” until parent waits for it

• If parent never waits for it, child stays in “undead” (zombie) state

• wait() by parent reaps the child & cleans it up

• What if parent is gone?• Child is automatically “reparented” to the init process

• init loops forever calling wait()


Example – fork_demo1.c (1/4)

#include <stdio.h>#include <stdlib.h>#include <unistd.h>#include <sys/types.h>#include <sys/wait.h>

int main( void ) { pid_t id, my_id; char *msg; int status;

// start by having the original process report // its identity

my_id = getpid(); // ask OS for our PID printf( "Initial process has PID %d\n", my_id ); fflush( stdout );



// create the child process

id = fork(); switch( id ) {

case 1: // the fork() failed perror( "fork" ); exit( EXIT_FAILURE );

case 0: // we are the child process msg = "child"; my_id = getpid(); break;

default: // we are the parent process msg = "parent"; break; }



// at this point, both the parent and the child // are executing this code; the only differences // are that 'my_id' contains different PIDs in // the two processes, and 'msg' points to a // different string.

printf( "In %s, PID is %d\n", msg, my_id ); fflush( stdout );

// child should exit using _exit()

if( id == 0 ) { puts( "Child is now exiting." ); _exit( EXIT_SUCCESS ); }



// parent will wait for child to exit id = wait( &status ); if( id < 0 ) { perror( "wait" ); } else { printf( "Parent: child %d terminated, status %d\n", id, WEXITSTATUS(status) ); }

puts( "Parent is now exiting." );

exit( EXIT_SUCCESS );}


Example – fork_demo1.c Results

$ ./fork_demo1Initial process has PID 8097In parent, PID is 8097In child, PID is 8098Child is now exiting.Parent: child 8098 terminated, status 0Parent is now exiting.

$


Issues With exit()

• Why two versions?

• Which to use?

• Parent: exit()• Flushes all i/o buffers

• Runs atexit() code

• Cleans up user-space data structures

• Calls _exit()

• Child: _exit()• Just exits

• Won’t mess up parent’s i/o

void exit( int status );void _exit( int status );


Hic Sunt Dracones!

#include <unistd.h>

int main( void ) {

while( 1 ) { fork(); }

return( 0 );}


Inheritance

• Child inherits most open i/o connections from parent• Both processes can read/write them

• Why isn’t the output completely garbled?

• File i/o requires:• Map of where the file lives on the disk

• Offset indicating where, within the file, the next i/o operation begins

• Remember what open() does:

x = 3offset map

x = open(...);

Open file table I-node table

Process System


Inheritance

• Child inherits parent’s process-level data structures• This includes the PCB FD table

• As each process writes, the (shared) offset is updated

x = 3offset map

fork();

Process 1 System

x = 3

Process 2

Open file table I-node table


Variations

• Behavior varies • From one OS to another

• From one OS version to another of the same OS

• Same OS on different hardware

• Also with i/o system being used


Shared I/O

• Other order: fork(), then open()

fork();

Process System

x = 3offset map

x = open(...);

struct file

struct vnode

x = 3offset map


The exec() Family

• Use fork() to get a new process into execution

• More useful to get a new program into execution!

• Do this with the exec() system call family


The exec() Family

• Six variations• Two basic ways to pass parameters

• Three variations on other behavior

• Parameters:• Specification of which program to run (all)

• Command-line arguments to pass to the program (all)

• Environment the program will inherit (some)

• We’ll look at two versions


The exec() Family

• Here are two of the variations:

• First argument is the path to the file to execute

• First argument is the name of the file to execute• Not a full path to it

• The OS locates the file by using the PATH environment variable

• Common behavior:• Second argument is an array of char*

• New program inherits the environment from the current program

• Return value:• On success, nothing – it doesn’t return!

• On error, -1, and errno contains an error code

int execv( const char *path, char *const argv[] );

int execvp( const char *file, char *const argv[] );


Example – exec_demo1.c (1/3)#include <stdio.h>#include <stdlib.h>#include <unistd.h>#include <sys/types.h>#include <sys/wait.h>

int main( void ) {pid_t id, my_id;char *argv[4];int status;

// start by having the original process report// its identity

my_id = getpid(); // ask OS for our PIDprintf( "Initial process has PID %d\n", my_id );fflush( stdout );

// create the child processid = fork();switch( id ) {case 1: // the fork() failed

perror( "fork" );exit( EXIT_FAILURE );


Example – exec_demo1.c (2/3)case 0: // we are the child process

// report our identitymy_id = getpid();printf( "Child is PID %d, running 'echo'\n", my_id );

// child will execvp() an 'echo' commandargv[0] = "echo"; // command nameargv[1] = "hello,"; // first argumentargv[2] = "world!"; // second argumentargv[3] = NULL; // terminating NULL pointer

execvp( "echo", argv );

// we only get here if the execvp() failedperror( "execvp" );

// use _exit() to avoid problems with the shared// stdout still being used by our parent_exit( EXIT_FAILURE );

// will never reach this statement!break;


Example – exec_demo1.c (3/3)default: // we are the parent

break;

}

// parent will wait for child to exitid = wait( &status );if( id < 0 ) {

perror( "wait" );} else {

printf( "Parent: child %d terminated, status %d\n",id, WEXITSTATUS(status) );

}




Example – exec_demo1.c Results

$ ./exec_demo1Initial process has PID 8099Child is PID 8100, running 'echo'hello, world!Parent: child 8100 terminated, status 0Parent is now exiting.

$


Example – exec_demo2.c (1/4)#include <stdio.h>#include <stdlib.h>#include <unistd.h>#include <sys/types.h>#include <sys/wait.h>

int main( void ) {pid_t id, my_id;char *argv[4];int status;

// start by having the original process report// its identity

my_id = getpid(); // ask OS for our PIDprintf( "Initial process has PID %d\n", my_id );fflush( stdout );

// create the child processid = fork();switch( id ) {case 1: // the fork() failed

perror( "fork" );exit( EXIT_FAILURE );


Example – exec_demo2.c (2/4)case 0: // we are the child process

// report our identitymy_id = getpid();printf( "Child is PID %d\n", my_id );

// child will execv() an 'echo' commandargv[0] = "echo"; // command nameargv[1] = "hello,"; // first argumentargv[2] = "world!"; // second argumentargv[3] = NULL; // terminating NULL pointer

// the 'echo' command may be in one of several// different places on different systems

// first, try /usr/binputs( "Child trying /usr/bin/echo" );execv( "/usr/bin/echo", argv );

// if that failed, let’s try /binputs( "Child trying /bin/echo" );execv( "/bin/echo", argv );


Example – exec_demo2.c (3/4)// both failed!perror( "execv" );

// use _exit() to avoid problems with the shared// stdout still being used by our parent_exit( EXIT_FAILURE );

// will never reach this statement!break;

default: // we are the parentbreak;

}


Example – exec_demo2.c (4/4)default: // we are the parent

break;

}

// parent will wait for child to exitid = wait( &status );if( id < 0 ) {

perror( "wait" );} else {

printf( "Parent: child %d terminated, status %d\n",id, WEXITSTATUS(status) );

}




Example – exec_demo2.c Results

$ ./exec_demo2Initial process has PID 27006Child is PID 27007Child trying /usr/bin/echoChild trying /bin/echohello, world!Parent: child 8102 terminated, status 0Parent is now exiting.

$


Interprocess Communicaton

• UNIX and Linux provide many ways to do this

• The original method was the pipe

• Essentially, an i/o channel• What is written into the pipe can be read from the other end


Interprocess Communicaton

• Original implementation: unallocated disk sectors• Taken from root filesystem

• Limit of 512 bytes in pipe at one time

• Number of pipes limited by available space in root filesystem

• Modern implementation: OS memory buffers• Capacity varies

• Typically, PIPE_BUF defines maximum atomic write()

• Solaris 10: <sys/param.h> defines as 5120

• Linux: <linux/limits.h> defines as 4096


Pipes

• Process-level view: pair of file descriptors

• To allocate:

• Creates a pipe with two open file descriptors

• Readable descriptor in fd[0]

• Writable descriptor in fd[1]

• Returns 0 on success, else -1

• To use, processes must have a common ancestor• Ancestor creates pipe & executes fork()

• Open descriptors are inherited, so parent and child can use pipe

int pipe( int fd[2] );


Pipes

• Alternate method in Linux:

• Options: O_NONBLOCK, O_CLOEXEC

• Must define _GNU_SOURCE before including <unistd.h>

int pipe2( int fd[2], flags );


Pipe Creation

close(fd[0]); close(fd[1]);fd[0] \

fd[1]

fd[0]

\ fd[1]

Process 1 Process 2

int fd[2];pipe(fd);

fd[0]

fd[1]

fork();fd[0]

fd[1]

fd[0]

fd[1]


Pipes and Standard I/O

• Works fine when you write the programs yourself• Create pipe, create child

• Parent and child know which descriptors connect to the pipe

• What if parent and/or child exec() other programs?


Pipes and Standard I/O

• Convention: read input from stdin, write output to stdout

• Need to “move” open FDs to 0 and 1

• Solution: duplicate them, then close the originals


Moving FDs

• Methods:

• Creates a duplicate of fd

• Closes newfd if open

• Duplicates oldfd onto newfd

• Always returns lowest-numbered available FD

int dup( int fd );

int dup2( int oldfd, int newfd );


Creating Parent stdout → Child stdin Pipeline

• Call pipe()

• Call fork()

• In parent:• Close readable side of pipe

• Close 1

• Dup writable side of pipe

• Close original writable side of pipe

• exec() desired program

• In child:• Close writable side of pipe

• Close 0

• Dup readable side of pipe

• Close original readable side of pipe

• exec() desired progam


Process 1 Process 2

close(fd[1]);close(fd[0]);

01

fd[0] \fd[1]

01

fd[0]\ fd[1]

Moving FDs – dup2()


01

fd[0] \fd[1] \

01

\ fd[0]\ fd[1]

01

fd[0] \fd[1]

01

fd[0]\ fd[1]

dup2(fd[1],1); dup2(fd[0],0);


Moving FDs – Original dup()

Process 1 Process 2

close(fd[1]);close(0);

close(fd[0]);close(1);

01 \

fd[0] \fd[1]

\ 01

fd[0]\ fd[1]

dup(fd[0]);dup(fd[1]);

01

fd[0] \fd[1]

01

fd[0]\ fd[1]


01

fd[0] \fd[1] \

01

\ fd[0]\ fd[1]


pipes1.c (1/4)

#include <stdlib.h>#include <stdio.h>#define MAX_LINE_LEN 100

void do_child( int to_parent[], int from_parent[] ){

/* Take care of our input pipe */ close( from_parent[ 1 ] ); /* close unneeded side */ dup2( from_parent[ 0 ], 0 ); /* dup readable side to stdin */ close( from_parent[ 0 ] ); /* close original pipe output */

/* Take care of our output pipe */ close( to_parent[ 0 ] ); /* close unneeded side */ dup2( to_parent[ 1 ], 1 ); /* dup writable side to stdout */ close( to_parent[ 1 ] ); /* close original pipe input */

/* Execute the desired program */ execlp( "cat", "cat", NULL ); perror( "exec" ); _exit( EXIT_FAILURE );}


pipes1.c (2/4)

char buffer[ MAX_LINE_LEN ];char buffer2[ MAX_LINE_LEN ];

int main( int ac, char **av ){ int to_child[ 2 ]; int from_child[ 2 ]; int len;

/* Create the pipes, then fork. */

if( pipe( to_child ) < 0 || pipe( from_child ) < 0 ){ perror( "pipe" ); exit( EXIT_FAILURE ); }


pipes1.c (3/4)

switch( fork() ){ case 1: perror( "fork" ); exit( EXIT_FAILURE );

case 0: /* child process */ do_child( from_child, to_child ); /* NOTREACHED */

default: /* parent process */ /* ** Close the output side of the output pipe and the ** input side of our input pipe. */ close( to_child[ 0 ] ); close( from_child[ 1 ] ); break; }


/* ** Get input one line at a time, pass it through the ** other program and print the results. */ while( fputs( "? ", stdout ), fgets( buffer, sizeof( buffer ), stdin ) != NULL ) {

len = strlen( buffer ); if( write( to_child[ 1 ], buffer, len ) < 0 ) { perror( "write" ); exit( EXIT_FAILURE ); } printf( "We wrote: %s", buffer ); len = read( from_child[ 0 ], buffer2, sizeof( buffer2 ) ); if( len < 0 ) { perror( "read" ); exit( EXIT_FAILURE ); } printf( "We got back: %s", buffer2 ); }


pipes1.c (4/4)


← What happened here?

$ ./pipes1? Hello!We wrote: Hello!We got back: Hello!? How are you today?We wrote: How are you today?We got back: How are you today?? I'm fineWe wrote: I'm fineWe got back: I'm fineou today??

$

pipes1.out


pipes2.c (1/4)

#include <stdlib.h>#include <stdio.h>#define MAX_LINE_LEN 100

void do_child( int to_parent[], int from_parent[] ){ /* Take care of our input pipe */ close( from_parent[ 1 ] ); /* close unneeded side */ dup2( from_parent[ 0 ], 0 ); /* dup readable side to stdin */ close( from_parent[ 0 ] ); /* close original pipe output */

/* Take care of our output pipe */ close( to_parent[ 0 ] ); /* close unneeded side */ dup2( to_parent[ 1 ], 1 ); /* dup writable side to stdout */ close( to_parent[ 1 ] ); /* close original pipe input */

/* Execute the desired program */ execlp( "tr", "tr", "[aeiou]", "[AEIOU]", NULL ); perror( "exec" ); _exit( EXIT_FAILURE );}


pipes2.c (2/4)

int main( int ac, char **av ){ int to_child[ 2 ]; int from_child[ 2 ]; char buffer[ MAX_LINE_LEN ]; char buffer2[ MAX_LINE_LEN ]; int len;

/* Create the pipes, then fork. */ if( pipe( to_child ) < 0 || pipe( from_child ) < 0 ){ perror( "pipe" ); exit( EXIT_FAILURE ); }


pipes2.c (3/4)

switch( fork() ){ case 1: perror( "fork" ); exit( EXIT_FAILURE );

case 0: /* child process */ do_child( from_child, to_child ); /* NOTREACHED */

default: /* parent process */ /* ** Close the output side of the output pipe and the ** input side of our input pipe. */ close( to_child[ 0 ] ); close( from_child[ 1 ] ); break; }


/* ** Get input one line at a time, pass it through the ** other program and print the results. */ while( fputs( "? ", stdout ), fgets( buffer, sizeof( buffer ), stdin ) != NULL ){ len = strlen( buffer ); if( write( to_child[ 1 ], buffer, len ) < 0 ) { perror( "write" ); exit( EXIT_FAILURE ); } printf( "We wrote: %s", buffer ); len = read( from_child[ 0 ], buffer2, sizeof( buffer2 ) ); if( len < 0 ) { perror( "read" ); exit( EXIT_FAILURE ); } buffer2[len] = ‘\0’; printf( "We got back: %s", buffer2 ); }


pipes2.c (4/4)


← No output yet, entered next input← Entered ^D, still no output← Interrupted program — is tr broken?

pipes2.out

$ ./pipes2? Hi thereWe wrote: Hi therehow are you?^D^C

$ tr ‘[aeiou]’ ‘[AEIOU]’Hi thereHI thErEhow are you?hOw ArE yOU?^D

$


while( fputs( "? ", stdout ), fgets( buffer, sizeof( buffer ), stdin ) != NULL ){ len = strlen( buffer ); if( write( to_child[ 1 ], buffer, len ) < 0 ){ perror( "write" ); exit( EXIT_FAILURE ); } printf( "We wrote: %s", buffer ); } len = read( from_child[ 0 ], buffer2, sizeof( buffer2 ) ); while( len > 0 ){ buffer2[ len ] = '\0'; printf( "We got back: %s", buffer2 ); len = read( from_child[ 0 ], buffer2, sizeof( buffer2 ) ); } if( len < 0 ){ perror( "read" ); exit( EXIT_FAILURE ); }


pipes3.c (partial)


← Entered EOF, but no output!← Interrupted the program

$ ./pipes3? Hi thereWe wrote: Hi there? How are you today?We wrote: How are you today?? I'm fine.We wrote: I'm fine.? ^D^C

$

pipes3.out


pipes4.c (partial) while( fputs( "? ", stdout ), fgets( buffer, sizeof( buffer ), stdin ) != NULL ){ len = strlen( buffer ); if( write( to_child[ 1 ], buffer, len ) < 0 ){ perror( "write" ); exit( EXIT_FAILURE ); } printf( "We wrote: %s", buffer ); } close( to_child[1] ); len = read( from_child[ 0 ], buffer2, sizeof( buffer2 ) ); while( len > 0 ){ buffer2[ len ] = '\0'; printf( "We got back: %s", buffer2 ); len = read( from_child[ 0 ], buffer2, sizeof( buffer2 ) ); } if( len < 0 ){ perror( "read" ); exit( EXIT_FAILURE ); } exit( EXIT_SUCCESS );}


← Program never ended, interrupt

\ → All from one read! /

pipes4.out$ ./pipes4

? Hi there!

We wrote: Hi there!

? How are y'all today?

We wrote: How are y'all today?

? I'm fine.

We wrote: I'm fine.

? ^DWe got back: HI thErE!

HOw ArE y'All tOdAy?

I'm fInE.

$ ls l largedata

rwr—— 1 csci243 course 423024 Nov 27 12:50 largedata

$ ./pipes4 < largedata > tmpfl

^C

$ ls l tmpfl

rwr—— 1 csci243 course 204800 Nov 27 12:50 tmpfl


More Pipe Notes

• Can have multiple processes using a single pipe• Must all have a common ancestor who created the pipe

• No way to determine which process wrote which data into pipe

• No way to ensure that a specific process reads specific data from pipe

• To use with stdio:

• fd must be open already

• mode must agree with how fd was opened

FILE *fdopen( int fd, char *mode );


Using Pipes With Stdio

• Can create a pipe to the stdin or stdout of a program:

• Creates a process for cmd

• On success, returns a stream connected to:

• stdin of cmd if mode is “w”, stdout of cmd if mode is “r”

• Returns NULL on error

• When done, use pclose() to close:

FILE *popen( char *cmd, char *mode );

int pclose( FILE *stream );


FIFOs

• Like pipes, but FIFOs have names in the filesystem• No file actually attached to the name

• Essentially, named pipes

• When a process opens a FIFO, it blocks until another process opens the FIFO

• When second process opens FIFO, a pipe-like structure is created


Notes on FIFOs

• Like pipes:• Can have multiple processes reading/writing

• Bi-directional - either process can read and write them

• Unlike pipes:• Processes don’t have to be related

• Any process that has permission to open the FIFO can use it

• Only real differences:• How they are created by processes

• Presence in the system (i.e., filesystem entry)


Dealing With Multiple Input Sources

• Process can have lots of open FDs• Files, pipes, terminal

• Maximum number is OS-dependent

• Consider a process with this behavior:• Has multiple open input FDs

• Each FD is the readable side of a pipe from another process

• Can get input from any process at any time



• Problem: read() from a FD blocks until data arrives• Can’t just read() from each FD in sequence

• Must correctly predict which FD will have data to avoid blocking

• If you read from an FD which never has data, you block forever

• How to deal with this situation?



• Multi-thread the program• One thread per FD

• Thread can block, but other threads can continue

• Set reads to be non-blocking using fcntl()• However, want blocking when there is no data to read at all

• Better solution: polling!


Polling

• Usage:

• Based on this structure

• fd is a file descriptor

• events is a bit mask indicated event(s) to watch for

• revents is a bit mask indicating which event(s) occurred

• fds array contains nfds elements

• Third argument is upper limit on blocking (ms)• Negative → infinite timeout

int poll( struct pollfd fds[], nfds_t nfds, int timeout );

struct pollfd { int fd; /* file desc to poll */ short events; /* events of interest on fd */ short revents; /* events that occurred on fd */};


Polling

• Set up one entry for each FD you want to watch

• Lots of different events• POLLIN or POLLRDNORM

• Normal priority data is available for reading - won’t block

• POLLOUT or POLLWRNORM

• Normal priority data can be written without blocking

• POLLERR

• Error condition (output)

• POLLHUP

• Hangup (output)

• POLLNVAL

• Invalid request - fd not open (output)

• Call to poll() returns # of FDs with pending events

CSCI 243 The Mechanics of Programming The OS: Process...

Documents

Transcript of CSCI 243 The Mechanics of Programming The OS: Process...