dow

nloa

d ta

sk p

roce

eds,

the

prog

ram

as

a w

hole

sho

uldn

’t s

tall—

it s

till

mus

t res

pond

to s

cree

n re

draw

s, m

ouse

clic

ks, a

nd s

o on

.

Para

llel

proc

essi

ng c

omes

to

the

resc

ue h

ere,

too

. B

y pe

rfor

min

g su

ch l

ong-

runn

ing

task

s in

par

alle

l wit

h th

e re

st o

f the

pro

gram

, the

sys

tem

at l

arge

can

rem

ain

resp

onsi

veno

mat

ter h

ow b

usy

som

e of

its

part

s m

ay b

e. M

oreo

ver,

the

para

llel p

roce

ssin

g m

odel

is a

nat

ural

fit

for

str

uctu

ring

suc

h pr

ogra

ms

and

othe

rs;

som

e ta

sks

are

mor

e ea

sily

conc

eptu

aliz

ed a

nd c

oded

as

com

pone

nts

runn

ing

as in

depe

nden

t, p

aral

lel e

ntit

ies.

The

re a

re t

wo

fund

amen

tal

way

s to

get

tas

ks r

unni

ng a

t th

e sa

me

tim

e in

Pyt

hon—

proc

ess

fork

s an

d sp

awne

d th

read

s. F

unct

iona

lly,

both

rel

y on

und

erly

ing

oper

atin

gsy

stem

ser

vice

s to

run

bit

s of

Pyt

hon

code

in p

aral

lel.

Proc

edur

ally

, the

y ar

e ve

ry d

if-fe

rent

in te

rms

of in

terf

ace,

por

tabi

lity,

and

com

mun

icat

ion.

For

inst

ance

, at t

his

wri

t-in

g di

rect

pro

cess

fork

s are

not

supp

orte

d on

Win

dow

s und

er st

anda

rd P

ytho

n (t

houg

hth

ey a

re u

nder

Cyg

win

Pyt

hon

on W

indo

ws)

.

By

cont

rast

, Py

thon

’s t

hrea

d su

ppor

t w

orks

on

all

maj

or p

latf

orm

s. M

oreo

ver,

the

os.s

pawn

fam

ily o

f ca

lls p

rovi

des

addi

tion

al w

ays

to l

aunc

h pr

ogra

ms

in a

pla

tfor

m-

neut

ral

way

tha

t is

sim

ilar

to f

orks

, an

d th

e os

.pop

en a

nd o

s.sy

stem

cal

ls a

ndsu

bpro

cess

mod

ule

we

stud

ied

in C

hapt

ers

2 an

d 3

can

be u

sed

to p

orta

bly

spaw

npr

ogra

ms

wit

h sh

ell c

omm

ands

. The

new

er mul

tipr

oces

sing

mod

ule

offe

rs a

ddit

iona

lw

ays

to r

un p

roce

sses

por

tabl

y in

man

y co

ntex

ts.

In t

his

chap

ter,

whi

ch i

s a

cont

inua

tion

of

our

look

at

syst

em i

nter

face

s av

aila

ble

toPy

thon

pro

gram

mer

s, w

e ex

plor

e Py

thon

’s b

uilt

-in

tool

s fo

r st

arti

ng t

asks

in p

aral

lel,

as w

ell a

s co

mm

unic

atin

g w

ith

thos

e ta

sks.

In s

ome

sens

e, w

e’ve

alr

eady

beg

un d

oing

so—

os.s

yste

m, os.

pope

n, a

nd sub

proc

ess,

whi

ch w

e le

arne

d an

d ap

plie

d ov

er t

he la

stth

ree

chap

ters

, are

a fa

irly

por

tabl

e w

ay t

o sp

awn

and

spea

k w

ith

com

man

d-lin

e pr

o-gr

ams,

too.

We

won

’t r

epea

t ful

l cov

erag

e of

thos

e to

ols

here

.

Inst

ead,

our

em

phas

is in

this

cha

pter

is o

n in

trod

ucin

g m

ore

dire

ct te

chni

ques

—fo

rks,

thre

ads,

pip

es,

sign

als,

soc

kets

, an

d ot

her

laun

chin

g te

chni

ques

—an

d on

usi

ng P

y-th

on’s

bui

lt-i

n to

ols

that

sup

port

the

m,

such

as

the os

.for

k ca

ll an

d th

e th

read

ing,

queu

e, a

nd mul

tipr

oces

sing

mod

ules

. In

the

next

cha

pter

(and

in th

e re

mai

nder

of t

his

book

), w

e us

e th

ese

tech

niqu

es in

mor

e re

alis

tic

prog

ram

s, s

o be

sur

e yo

u un

ders

tand

the

basi

cs h

ere

befo

re fl

ippi

ng a

head

.

One

not

e up

fron

t: a

ltho

ugh

the

proc

ess,

thre

ad, a

nd IP

C m

echa

nism

s w

e w

ill e

xplo

rein

this

cha

pter

are

the

prim

ary

para

llel p

roce

ssin

g to

ols i

n Py

thon

scri

pts,

the

thir

d pa

rty

dom

ain

offe

rs a

ddit

iona

l opt

ions

whi

ch m

ay s

erve

mor

e ad

vanc

ed o

r sp

ecia

lized

rol

es.

As

just

one

exa

mpl

e, th

e M

PI fo

r Py

thon

sys

tem

allo

ws

Pyth

on s

crip

ts to

als

o em

ploy

the

Mes

sage

Pas

sing

Int

erfa

ce (

MPI

) st

anda

rd,

allo

win

g Py

thon

pro

gram

s to

exp

loit

mul

tipl

e pr

oces

sors

in v

ario

us w

ays

(see

the

Web

for

deta

ils).

Whi

le s

uch

spec

ific

ex-

tens

ions

are

bey

ond

our

scop

e in

this

boo

k, th

e fu

ndam

enta

ls o

f mul

tipr

oces

sing

that

we

will

exp

lore

her

e sh

ould

app

ly to

mor

e ad

vanc

ed te

chni

ques

you

may

enc

ount

er in

your

par

alle

l fut

ures

.

178

|Ch

apte

r 5: 

Para

llel S

yste

m To

ols

Fork

ing P

roce

sses

Fork

ed p

roce

sses

are

a t

radi

tion

al w

ay t

o st

ruct

ure

para

llel t

asks

, and

the

y ar

e a

fun-

dam

enta

l par

t of

the

Uni

x to

ol s

et. F

orki

ng is

a s

trai

ghtf

orw

ard

way

to

star

t an

inde

-pe

nden

t pr

ogra

m, w

heth

er it

is d

iffer

ent

from

the

cal

ling

prog

ram

or

not.

For

king

isba

sed

on t

he n

otio

n of

cop

ying

pro

gram

s: w

hen

a pr

ogra

m c

alls

the

for

k ro

utin

e, t

heop

erat

ing

syst

em m

akes

a n

ew c

opy

of t

hat

prog

ram

and

its

proc

ess

in m

emor

y an

dst

arts

run

ning

tha

t co

py in

par

alle

l wit

h th

e or

igin

al. S

ome

syst

ems

don’

t re

ally

cop

yth

e or

igin

al p

rogr

am (i

t’s

an e

xpen

sive

ope

rati

on),

but

the

new

cop

y w

orks

as

if it

wer

ea

liter

al c

opy.

Aft

er a

for

k op

erat

ion,

the

ori

gina

l cop

y of

the

pro

gram

is c

alle

d th

e pa

rent

pro

cess

,an

d th

e co

py c

reat

ed b

y os

.for

k is

cal

led

the

child

pro

cess

. In

gene

ral,

pare

nts c

an m

ake

any

num

ber o

f chi

ldre

n, a

nd c

hild

ren

can

crea

te c

hild

pro

cess

es o

f the

ir o

wn;

all

fork

edpr

oces

ses

run

inde

pend

entl

y an

d in

par

alle

l und

er th

e op

erat

ing

syst

em’s

con

trol

, and

child

ren

may

con

tinu

e to

run

aft

er th

eir

pare

nt e

xits

.

Thi

s is

pro

babl

y si

mpl

er in

pra

ctic

e th

an in

the

ory,

tho

ugh.

The

Pyt

hon

scri

pt in

Ex-

ampl

e 5-

1 fo

rks

new

chi

ld p

roce

sses

unt

il yo

u ty

pe th

e le

tter

q a

t the

con

sole

.

Exa

mpl

e 5-

1. P

P4E

\Sys

tem

\Pro

cess

es\f

ork1

.py

"for

ks c

hild

pro

cess

es u

ntil

you

typ

e 'q

'"

impo

rt o

s

def

chil

d():

prin

t('H

ello

fro

m ch

ild'

, o

s.ge

tpid

())

os._

exit

(0)

# e

lse

goes

bac

k to

par

ent

loop

def

pare

nt()

:

wh

ile

True

:

ne

wpid

= o

s.fo

rk()

if n

ewpi

d ==

0:

chil

d()

else

:

pr

int(

'Hel

lo f

rom

pare

nt',

os.

getp

id()

, ne

wpid

)

if

inp

ut()

==

'q':

bre

ak

pare

nt()

Pyth

on’s

pro

cess

for

king

too

ls,

avai

labl

e in

the

os

mod

ule,

are

sim

ply

thin

wra

pper

sov

er s

tand

ard

fork

ing

calls

in th

e sy

stem

libr

ary

also

use

d by

C la

ngua

ge p

rogr

ams.

To

star

t a

new

, par

alle

l pro

cess

, cal

l the

os.

fork

bui

lt-i

n fu

ncti

on. B

ecau

se t

his

func

tion

gene

rate

s a

copy

of t

he c

allin

g pr

ogra

m, i

t re

turn

s a

diff

eren

t va

lue

in e

ach

copy

: zer

oin

the

child

pro

cess

and

the

proc

ess

ID o

f the

new

chi

ld in

the

pare

nt.

Fork

ing P

roce

sses

|17

9

Prog

ram

s ge

nera

lly t

est

this

res

ult

to b

egin

diff

eren

t pr

oces

sing

in t

he c

hild

onl

y; t

his

scri

pt, f

or in

stan

ce, r

uns

the ch

ild

func

tion

in c

hild

pro

cess

es o

nly.

†

Bec

ause

fork

ing

is in

grai

ned

in th

e U

nix

prog

ram

min

g m

odel

, thi

s sc

ript

wor

ks w

ell o

nU

nix,

Lin

ux, a

nd m

oder

n M

acs.

Unf

ortu

nate

ly, t

his

scri

pt w

on’t

wor

k on

the

stan

dard

vers

ion

of P

ytho

n fo

r Win

dow

s to

day,

bec

ause

for

k is

too

muc

h at

odd

s w

ith

the

Win

-do

ws

mod

el.

Pyth

on s

crip

ts c

an a

lway

s sp

awn

thre

ads

on W

indo

ws,

and

the

mul

tipr

oces

sing

mod

ule

desc

ribe

d la

ter

in t

his

chap

ter

prov

ides

an

alte

rnat

ive

for

runn

ing

proc

esse

s po

rtab

ly, w

hich

can

obv

iate

the

need

for

proc

ess

fork

s on

Win

dow

s in

con

-te

xts

that

con

form

to it

s co

nstr

aint

s (a

lbei

t at s

ome

pote

ntia

l cos

t in

low

-lev

el c

ontr

ol).

The

scr

ipt

in E

xam

ple

5-1

does

wor

k on

Win

dow

s, h

owev

er,

if yo

u us

e th

e Py

thon

ship

ped

wit

h th

e C

ygw

in s

yste

m (

or b

uild

one

of

your

ow

n fr

om s

ourc

e-co

de w

ith

Cyg

win

’s li

brar

ies)

. Cyg

win

is a

fre

e, o

pen

sour

ce s

yste

m t

hat

prov

ides

ful

l Uni

x-lik

efu

ncti

onal

ity

for

Win

dow

s (a

nd is

des

crib

ed f

urth

er in

“M

ore

on C

ygw

in P

ytho

n fo

rW

indo

ws”

on

page

185

). Y

ou c

an fo

rk w

ith

Pyth

on o

n W

indo

ws

unde

r C

ygw

in, e

ven

thou

gh it

s beh

avio

r is n

ot e

xact

ly th

e sa

me

as tr

ue U

nix

fork

s. B

ecau

se it

’s c

lose

eno

ugh

for

this

boo

k’s

exam

ples

, tho

ugh,

let’

s us

e it

to r

un o

ur s

crip

t liv

e:

[C:\

...\

PP4E

\Sys

tem\

Proc

esse

s]$

pyth

on f

ork1

.py

Hell

o fr

om p

aren

t 72

96 7

920

Hell

o fr

om c

hild

792

0

Hell

o fr

om p

aren

t 72

96 3

988

Hell

o fr

om c

hild

398

8

Hell

o fr

om p

aren

t 72

96 6

796

Hell

o fr

om c

hild

679

6q

The

se m

essa

ges

repr

esen

t thr

ee fo

rked

chi

ld p

roce

sses

; the

uni

que

iden

tifie

rs o

f all

the

proc

esse

s in

volv

ed a

re f

etch

ed a

nd d

ispl

ayed

wit

h th

e os

.get

pid

call.

A s

ubtl

e po

int:

the ch

ild

proc

ess

func

tion

is a

lso

care

ful t

o ex

it e

xplic

itly

wit

h an

os.

_exi

t ca

ll. W

e’ll

disc

uss t

his c

all i

n m

ore

deta

il la

ter i

n th

is c

hapt

er, b

ut if

it’s

not

mad

e, th

e ch

ild p

roce

ssw

ould

live

on

afte

r the

chi

ld fu

ncti

on re

turn

s (r

emem

ber,

it’s

just

a c

opy

of th

e or

igin

alpr

oces

s). T

he n

et e

ffec

t is

that

the

child

wou

ld g

o ba

ck to

the

loop

in par

ent

and

star

tfo

rkin

g ch

ildre

n of

its

own

(i.e

., th

e pa

rent

wou

ld h

ave

gran

dchi

ldre

n). I

f yo

u de

lete

the

exit

cal

l an

d re

run,

you

’ll l

ikel

y ha

ve t

o ty

pe m

ore

than

one

q t

o st

op,

beca

use

mul

tipl

e pr

oces

ses

are

runn

ing

in th

e pa

rent

func

tion

.

In E

xam

ple

5-1,

eac

h pr

oces

s ex

its

very

soo

n af

ter

it s

tart

s, s

o th

ere’

s lit

tle

over

lap

inti

me.

Let

’s d

o so

met

hing

slig

htly

mor

e so

phis

tica

ted

to b

ette

r illu

stra

te m

ulti

ple

fork

edpr

oces

ses

runn

ing

in p

aral

lel.

Exa

mpl

e 5-

2 st

arts

up

5 co

pies

of i

tsel

f, e

ach

copy

cou

nt-

ing

up to

5 w

ith

a on

e-se

cond

del

ay b

etw

een

iter

atio

ns. T

he tim

e.sl

eep

stan

dard

libr

ary

†A

t lea

st in

the

curr

ent P

ytho

n im

plem

enta

tion

, cal

ling os

.for

k in

a P

ytho

n sc

ript

act

ually

cop

ies

the

Pyth

onin

terp

rete

r pr

oces

s (i

f you

look

at y

our

proc

ess

list,

you

’ll s

ee tw

o Py

thon

ent

ries

aft

er a

fork

). B

ut s

ince

the

Pyth

on in

terp

rete

r re

cord

s ev

eryt

hing

abo

ut y

our

runn

ing

scri

pt,

it’s

OK

to

thin

k of

for

k as

cop

ying

you

rpr

ogra

m d

irec

tly.

It r

eally

will

if P

ytho

n sc

ript

s ar

e ev

er c

ompi

led

to b

inar

y m

achi

ne c

ode.

180

|Ch

apte

r 5: 

Para

llel S

yste

m To

ols

call

sim

ply

paus

es th

e ca

lling

pro

cess

for a

num

ber o

f sec

onds

(you

can

pas

s a

float

ing-

poin

t val

ue to

pau

se fo

r fr

acti

ons

of s

econ

ds).

Exa

mpl

e 5-

2. P

P4E

\Sys

tem

\Pro

cess

es\f

ork-

coun

t.py

"""

fork

bas

ics:

sta

rt 5

cop

ies

of t

his

prog

ram

runn

ing

in p

aral

lel

with

the

orig

inal

; ea

ch c

opy

coun

ts u

p to

5 o

n th

e sa

me s

tdou

t st

ream

--fo

rks

copy

pro

cess

mem

ory,

inc

ludi

ng f

ile

desc

ript

ors;

for

k do

esn'

t cu

rren

tly

work

on

Wind

ows

with

out

Cygw

in:

use

os.s

pawn

v or

mul

tipr

oces

sing

on

Wind

ows

inst

ead;

spa

wnv

is r

ough

ly l

ike

a fo

rk+e

xec

comb

inat

ion;

"""

impo

rt o

s, t

ime

def

coun

ter(

coun

t):

#

run

in

new

proc

ess

for

i in

ran

ge(c

ount

):

ti

me.s

leep

(1)

# s

imul

ate

real

wor

k

pr

int(

'[%s

] =>

%s'

% (

os.g

etpi

d(),

i))

for

i in

ran

ge(5

):

pi

d =

os.f

ork(

)

if

pid

!=

0:

pr

int(

'Pro

cess

%d

spaw

ned'

% p

id)

# i

n pa

rent

: co

ntin

ue

el

se:

coun

ter(

5)

#

els

e in

chi

ld/n

ew p

roce

ss

os

._ex

it(0

)

# r

un f

unct

ion

and

exit

prin

t('M

ain

proc

ess

exit

ing.

')

#

par

ent

need

not

wai

t

Whe

n ru

n, t

his

scri

pt s

tart

s 5

proc

esse

s im

med

iate

ly a

nd e

xits

. All

5 fo

rked

pro

cess

esch

eck

in w

ith

thei

r fir

st c

ount

dis

play

one

sec

ond

late

r an

d ev

ery

seco

nd t

here

afte

r.N

otic

e th

at c

hild

pro

cess

es c

onti

nue

to ru

n, e

ven

if th

e pa

rent

pro

cess

that

cre

ated

them

term

inat

es:

[C:\

...\

PP4E

\Sys

tem\

Proc

esse

s]$

pyth

on f

ork-

coun

t.py

Proc

ess

4556

spa

wned

Proc

ess

3724

spa

wned

Proc

ess

6360

spa

wned

Proc

ess

6476

spa

wned

Proc

ess

6684

spa

wned

Main

pro

cess

exi

ting

.[4

556]

=>

0[3

724]

=>

0[6

360]

=>

0[6

476]

=>

0[6

684]

=>

0[4

556]

=>

1[3

724]

=>

1[6

360]

=>

1[6

476]

=>

1[6

684]

=>

1[4

556]

=>

2[3

724]

=>

2[6

360]

=>

2

Fork

ing P

roce

sses

|18

1

[647

6] =

> 2

[668

4] =

> 2

...m

ore

outp

ut o

mitt

ed...

The

out

put o

f all

of th

ese

proc

esse

s sh

ows

up o

n th

e sa

me

scre

en, b

ecau

se a

ll of

them

shar

e th

e st

anda

rd o

utpu

t st

ream

(an

d a

syst

em p

rom

pt m

ay s

how

up

alon

g th

e w

ay,

too)

. Tec

hnic

ally

, a fo

rked

pro

cess

get

s a

copy

of t

he o

rigi

nal p

roce

ss’s

glo

bal m

emor

y,in

clud

ing

open

file

des

crip

tors

. Bec

ause

of t

hat,

glo

bal o

bjec

ts li

ke fi

les

star

t ou

t w

ith

the

sam

e va

lues

in a

chi

ld p

roce

ss, s

o al

l the

pro

cess

es h

ere

are

tied

to th

e sa

me

sing

lest

ream

. But

it’s

impo

rtan

t to

rem

embe

r th

at g

loba

l mem

ory

is c

opie

d, n

ot s

hare

d; if

ach

ild p

roce

ss c

hang

es a

glo

bal o

bjec

t, it

cha

nges

onl

y it

s ow

n co

py. (

As

we’

ll se

e, t

his

wor

ks d

iffer

entl

y in

thre

ads,

the

topi

c of

the

next

sec

tion

.)

The f

ork/

exec

Com

bina

tion

In E

xam

ples

5-1

and

5-2

, chi

ld p

roce

sses

sim

ply

ran

a fu

ncti

on w

ithi

n th

e Py

thon

pro

-gr

am a

nd t

hen

exit

ed.

On

Uni

x-lik

e pl

atfo

rms,

for

ks a

re o

ften

the

bas

is o

f st

arti

ngin

depe

nden

tly

runn

ing

prog

ram

s th

at a

re c

ompl

etel

y di

ffer

ent

from

the

pro

gram

tha

tpe

rfor

med

the fo

rk c

all.

For

inst

ance

, Exa

mpl

e 5-

3 fo

rks

new

pro

cess

es u

ntil

we

type

q ag

ain,

but

chi

ld p

roce

sses

run

a b

rand

-new

pro

gram

inst

ead

of c

allin

g a

func

tion

inth

e sa

me

file.

Exa

mpl

e 5-

3. P

P4E

\Sys

tem

\Pro

cess

es\f

ork-

exec

.py

"sta

rts

prog

rams

unt

il y

ou t

ype

'q'"

impo

rt o

s

parm

= 0

whil

e Tr

ue:

parm

+=

1

pi

d =

os.f

ork(

)

if

pid

==

0:

#

cop

y pr

oces

s

os

.exe

clp(

'pyt

hon'

, 'p

ytho

n',

'chi

ld.p

y',

str(

parm

)) #

ove

rlay

pro

gram

asse

rt F

alse

, 'e

rror

sta

rtin

g pr

ogra

m'

# s

houl

dn't

ret

urn

else

:

pr

int(

'Chi

ld i

s',

pid)

if i

nput

() =

= 'q

': b

reak

If y

ou’v

e do

ne m

uch

Uni

x de

velo

pmen

t, th

e fo

rk/exe

c co

mbi

nati

on w

ill p

roba

bly

look

fam

iliar

. The

mai

n th

ing

to n

otic

e is

the

os.

exec

lp c

all i

n th

is c

ode.

In

a nu

tshe

ll, t

his

call

repl

aces

(ov

erla

ys)

the

prog

ram

run

ning

in t

he c

urre

nt p

roce

ss w

ith

a br

and

new

prog

ram

. Bec

ause

of t

hat,

the

com

bina

tion

of os.

fork

and

os.

exec

lp m

eans

sta

rt a

new

proc

ess a

nd ru

n a

new

pro

gram

in th

at p

roce

ss—

in o

ther

wor

ds, l

aunc

h a

new

pro

gram

in p

aral

lel w

ith

the

orig

inal

pro

gram

.

182

|Ch

apte

r 5: 

Para

llel S

yste

m To

ols

os.e

xec c

all f

orm

ats

The

arg

umen

ts t

o os

.exe

clp

spec

ify t

he p

rogr

am t

o be

run

by

givi

ng c

omm

and-

line

argu

men

ts u

sed

to s

tart

the

pro

gram

(i.e

., w

hat

Pyth

on s

crip

ts k

now

as sy

s.ar

gv).

If

succ

essf

ul, t

he n

ew p

rogr

am b

egin

s run

ning

and

the

call

to os.

exec

lp it

self

neve

r ret

urns

(sin

ce t

he o

rigi

nal p

rogr

am h

as b

een

repl

aced

, th

ere’

s re

ally

not

hing

to

retu

rn t

o).

Ifth

e ca

ll do

es re

turn

, an

erro

r has

occ

urre

d, s

o w

e co

de a

n as

sert

aft

er it

that

will

alw

ays

rais

e an

exc

epti

on if

rea

ched

.

The

re a

re a

han

dful

of o

s.ex

ec v

aria

nts

in t

he P

ytho

n st

anda

rd li

brar

y; s

ome

allo

w u

sto

con

figur

e en

viro

nmen

t va

riab

les

for

the

new

pro

gram

, pa

ss c

omm

and-

line

argu

-m

ents

in d

iffer

ent f

orm

s, a

nd s

o on

. All

are

avai

labl

e on

bot

h U

nix

and

Win

dow

s, a

ndth

ey r

epla

ce th

e ca

lling

pro

gram

(i.e

., th

e Py

thon

inte

rpre

ter)

. exe

c co

mes

in e

ight

fla-

vors

, whi

ch c

an b

e a

bit c

onfu

sing

unl

ess

you

gene

raliz

e:

os.e

xecv

(pro

gram

, co

mman

dlin

eseq

uenc

e)T

he b

asic

“v”

exe

c fo

rm is

pas

sed

an e

xecu

tabl

e pr

ogra

m’s

nam

e, a

long

wit

h a

list

or tu

ple

of c

omm

and-

line

argu

men

t str

ings

use

d to

run

the

exec

utab

le (t

hat i

s, th

ew

ords

you

wou

ld n

orm

ally

type

in a

she

ll to

sta

rt a

pro

gram

).

os.e

xecl

(pro

gram

, cm

darg

1, c

mdar

g2,.

.. c

mdar

gN)

The

bas

ic “

l” exe

c fo

rm is

pas

sed

an e

xecu

tabl

e’s

nam

e, fo

llow

ed b

y on

e or

mor

eco

mm

and-

line

argu

men

ts p

asse

d as

ind

ivid

ual

func

tion

arg

umen

ts.

Thi

s is

the

sam

e as

os.

exec

v(pr

ogra

m, (cm

darg

1, c

mdar

g2,.

..))

.

os.e

xecl

pos

.exe

cvp

Add

ing

the

lett

er p

to th

e ex

ecv

and ex

ecl

nam

es m

eans

that

Pyt

hon

will

loca

te th

eex

ecut

able

’s d

irec

tory

usi

ng y

our

syst

em s

earc

h-pa

th s

etti

ng (

i.e., PA

TH).

os.e

xecl

eos

.exe

cve

Add

ing

a le

tter

e to

the ex

ecv

and ex

ecl

nam

es m

eans

an

extr

a, la

st a

rgum

ent i

s a

dict

iona

ry c

onta

inin

g sh

ell e

nvir

onm

ent v

aria

bles

to s

end

to th

e pr

ogra

m.

os.e

xecv

peos

.exe

clpe

Add

ing

the

lett

ers

p an

d e

to t

he b

asic

exe

c na

mes

mea

ns t

o us

e th

e se

arch

pat

han

d to

acc

ept a

she

ll en

viro

nmen

t set

ting

s di

ctio

nary

.

So w

hen

the

scri

pt i

n E

xam

ple

5-3

calls

os.

exec

lp,

indi

vidu

ally

pas

sed

para

met

ers

spec

ify a

com

man

d lin

e fo

r the

pro

gram

to b

e ru

n on

, and

the

wor

d py

thon

map

s to

an

exec

utab

le f

ile a

ccor

ding

to

the

unde

rlyi

ng s

yste

m s

earc

h-pa

th s

etti

ng e

nvir

onm

ent

vari

able

(PA

TH).

It’

s as

if w

e w

ere

runn

ing

a co

mm

and

of t

he f

orm

pyt

hon

chil

d.py

1in

a s

hell,

but

wit

h a

diff

eren

t com

man

d-lin

e ar

gum

ent o

n th

e en

d ea

ch ti

me.

Fork

ing P

roce

sses

|18

3

Spaw

ned c

hild

prog

ram

Just

as

whe

n ty

ped

at a

she

ll, th

e st

ring

of a

rgum

ents

pas

sed

to os.

exec

lp b

y th

e fo

rk-

exec

scri

pt in

Exa

mpl

e 5-

3 st

arts

ano

ther

Pyt

hon

prog

ram

file

, as s

how

n in

Exa

mpl

e 5-

4.

Exa

mpl

e 5-

4. P

P4E

\Sys

tem

\Pro

cess

es\c

hild

.py

impo

rt o

s, s

yspr

int(

'Hel

lo f

rom

chil

d',

os.g

etpi

d(),

sys

.arg

v[1]

)

Her

e is

thi

s co

de in

act

ion

on L

inux

. It

does

n’t

look

muc

h di

ffer

ent

from

the

ori

gina

lfo

rk1.

py, b

ut it

’s re

ally

runn

ing

a ne

w p

rogr

am in

eac

h fo

rked

pro

cess

. Mor

e ob

serv

ant

read

ers m

ay n

otic

e th

at th

e ch

ild p

roce

ss ID

dis

play

ed is

the

sam

e in

the

pare

nt p

rogr

aman

d th

e la

unch

ed c

hild

.py

prog

ram

; os.

exec

lp s

impl

y ov

erla

ys a

pro

gram

in t

he s

ame

proc

ess:

[C:\

...\

PP4E

\Sys

tem\

Proc

esse

s]$

pyth

on f

ork-

exec

.py

Chil

d is

455

6He

llo

from

chi

ld 4

556

1

Chil

d is

592

0He

llo

from

chi

ld 5

920

2

Chil

d is

316

Hell

o fr

om c

hild

316

3q

The

re a

re o

ther

way

s to

star

t up

prog

ram

s in

Pyth

on b

esid

es th

e fo

rk/e

xec

com

bina

tion

.Fo

r ex

ampl

e, t

he os.

syst

em a

nd os.

pope

n ca

lls a

nd sub

proc

ess

mod

ule

whi

ch w

e ex

-pl

ored

in C

hapt

ers

2 an

d 3

allo

w u

s to

spa

wn

shel

l com

man

ds. A

nd th

e os

.spa

wnv

call

and mu

ltip

roce

ssin

g m

odul

e, w

hich

we’

ll m

eet

late

r in

thi

s ch

apte

r, a

llow

us

to s

tart

inde

pend

ent

prog

ram

s an

d pr

oces

ses

mor

e po

rtab

ly. I

n fa

ct, w

e’ll

see

late

r th

at mul

tipr

oces

sing

’s p

roce

ss s

paw

ning

mod

el c

an b

e us

ed a

s a

sort

of p

orta

ble

repl

acem

ent f

oros

.for

k in

som

e co

ntex

ts (

albe

it a

less

eff

icie

nt o

ne)

and

used

in c

onju

ncti

on w

ith

the

os.e

xec*

cal

ls s

how

n he

re to

ach

ieve

a s

imila

r ef

fect

in s

tand

ard

Win

dow

s Py

thon

.

We’

ll se

e m

ore

proc

ess

fork

exa

mpl

es la

ter

in t

his

chap

ter,

esp

ecia

lly in

the

pro

gram

exit

s an

d pr

oces

s co

mm

unic

atio

n se

ctio

ns, s

o w

e’ll

fore

go a

ddit

iona

l exa

mpl

es h

ere.

We’

ll al

so d

iscu

ss a

ddit

iona

l pro

cess

topi

cs in

late

r cha

pter

s of

this

boo

k. F

or in

stan

ce,

fork

s are

revi

site

d in

Cha

pter

12

to d

eal w

ith

serv

ers a

nd th

eir z

ombi

es—

dead

pro

cess

eslu

rkin

g in

sys

tem

tabl

es a

fter

thei

r dem

ise.

For

now

, let

’s m

ove

on to

thre

ads,

a s

ubje

ctw

hich

at l

east

som

e pr

ogra

mm

ers

find

to b

e su

bsta

ntia

lly le

ss fr

ight

enin

g…

184

|Ch

apte

r 5: 

Para

llel S

yste

m To

ols

Mor

e on C

ygwi

n Pyt

hon f

or W

indo

wsA

s men

tion

ed, t

he os.fork

cal

l is p

rese

nt in

the

Cyg

win

ver

sion

of P

ytho

n on

Win

dow

s.E

ven

thou

gh t

his

call

is m

issi

ng in

the

sta

ndar

d ve

rsio

n of

Pyt

hon

for

Win

dow

s, y

ouca

n fo

rk p

roce

sses

on

Win

dow

s w

ith

Pyth

on if

you

inst

all a

nd u

se C

ygw

in. H

owev

er,

the

Cyg

win

fork

cal

l is

not a

s ef

ficie

nt a

nd d

oes

not w

ork

exac

tly

the

sam

e as

a fo

rk o

ntr

ue U

nix

syst

ems.

Cyg

win

is a

free

, ope

n so

urce

pac

kage

whi

ch in

clud

es a

libr

ary

that

att

empt

s to

pro

vide

a U

nix-

like

API

for u

se o

n W

indo

ws

mac

hine

s, a

long

wit

h a

set o

f com

man

d-lin

e to

ols

that

im

plem

ent

a U

nix-

like

envi

ronm

ent.

It

mak

es i

t ea

sier

to

appl

y U

nix

skill

s an

dco

de o

n W

indo

ws

com

pute

rs.

Acc

ordi

ng to

its

FAQ

doc

umen

tati

on, t

houg

h, “

Cyg

win

fork

() e

ssen

tial

ly w

orks

like

ano

ncop

y on

wri

te v

ersi

on o

f for

k() (

like

old

Uni

x ve

rsio

ns u

sed

to d

o). B

ecau

se o

f thi

sit

can

be

a lit

tle

slow

. In

mos

t ca

ses,

you

are

bet

ter

off u

sing

the

spa

wn

fam

ily o

f cal

lsif

poss

ible

.” S

ince

thi

s bo

ok’s

for

k ex

ampl

es d

on’t

nee

d to

car

e ab

out

perf

orm

ance

,C

ygw

in’s

fork

suf

fices

.

In a

ddit

ion

to th

e fo

rk c

all,

Cyg

win

pro

vide

s ot

her U

nix

tool

s th

at w

ould

oth

erw

ise

not

be a

vaila

ble

on a

ll fla

vors

of

Win

dow

s, i

nclu

ding

os.mkfifo

(di

scus

sed

late

r in

thi

sch

apte

r). I

t als

o co

mes

wit

h a gcc

com

pile

r en

viro

nmen

t for

bui

ldin

g C

ext

ensi

ons

for

Pyth

on o

n W

indo

ws

that

will

be

fam

iliar

to U

nix

deve

lope

rs. A

s lo

ng a

s yo

u’re

will

ing

to u

se C

ygw

in li

brar

ies

to b

uild

you

r app

licat

ion

and

pow

er y

our P

ytho

n, it

’s v

ery

clos

eto

Uni

x on

Win

dow

s.

Like

all

thir

d-pa

rty

libra

ries

, th

ough

, C

ygw

in a

dds

an e

xtra

dep

ende

ncy

to y

our

sys-

tem

s. P

erha

ps m

ore

crit

ical

ly, C

ygw

in c

urre

ntly

use

s the

GN

U G

PL li

cens

e, w

hich

add

sdi

stri

buti

on re

quir

emen

ts b

eyon

d th

ose

of st

anda

rd P

ytho

n. U

nlik

e us

ing

Pyth

on it

self,

ship

ping

a p

rogr

am th

at u

ses

Cyg

win

libr

arie

s m

ay re

quir

e th

at y

our p

rogr

am’s

sou

rce

code

be

mad

e fr

eely

ava

ilabl

e (t

houg

h R

edH

at o

ffer

s a

“buy

-out

” op

tion

whi

ch c

anre

lieve

you

of t

his

requ

irem

ent)

. Not

e th

at th

is is

a c

ompl

ex le

gal i

ssue

, and

you

sho

uld

stud

y C

ygw

in’s

lice

nse

on y

our o

wn

if th

is m

ay im

pact

you

r pro

gram

s. It

s lic

ense

doe

s,ho

wev

er, i

mpo

se m

ore

cons

trai

nts

than

Pyt

hon’

s (P

ytho

n us

es a

“B

SD”-

styl

e lic

ense

,no

t the

GPL

).

Des

pite

lice

nsin

g is

sue,

Cyg

win

sti

ll ca

n be

a g

reat

way

to

get

Uni

x-lik

e fu

ncti

onal

ity

on W

indo

ws

wit

hout

ins

talli

ng a

com

plet

ely

diff

eren

t op

erat

ing

syst

em s

uch

asLi

nux—

a m

ore

com

plet

e bu

t ge

nera

lly m

ore

com

plex

opt

ion.

For

mor

e de

tails

, se

eht

tp://

cygw

in.c

om o

r ru

n a

sear

ch fo

r C

ygw

in o

n th

e W

eb.

See

also

the

sta

ndar

d lib

rary

’s multiprocessing

mod

ule

and os.spawn

fam

ily o

f ca

lls,

cove

red

late

r in

this

cha

pter

, for

alt

erna

tive

way

to s

tart

par

alle

l tas

ks a

nd p

rogr

ams

onU

nix

and

Win

dow

s th

at d

o no

t re

quir

e fo

rk a

nd e

xec

calls

. To

run

a si

mpl

e fu

ncti

onca

ll in

par

alle

l on

Win

dow

s (r

athe

r th

an o

n an

ext

erna

l pro

gram

), a

lso

see

the

sect

ion

on s

tand

ard

libra

ry t

hrea

ds l

ater

in

this

cha

pter

. T

hrea

ds, multiprocessing,

and

os.s

pawn

cal

ls w

ork

on W

indo

ws

in s

tand

ard

Pyth

on.

Fork

ing P

roce

sses

|18

5

Four

th E

ditio

n U

pdat

e: A

s I

was

upd

atin

g th

is c

hapt

er i

n Fe

brua

ry 2

010,

Cyg

win

’sof

ficia

l Pyt

hon

was

sti

ll Py

thon

2.5

.2.

To

get

Pyth

on 3

.1 u

nder

Cyg

win

, it

had

to

bebu

ilt f

rom

its

sour

ce c

ode.

If

this

is s

till

requ

ired

whe

n yo

u re

ad t

his,

mak

e su

re y

ouha

ve gcc

and

mak

e in

stal

led

on y

our

Cyg

win

, the

n fe

tch

Pyth

on’s

sou

rce

code

pac

kage

from

pyt

hon.

org,

unp

ack

it, a

nd b

uild

Pyt

hon

wit

h th

e us

ual c

omm

ands

:

./co

nfig

ure

make

make

tes

t

su

do m

ake

inst

all

Thi

s w

ill in

stal

l Pyt

hon

as pyt

hon3

. The

sam

e pr

oced

ure

wor

ks o

n al

l Uni

x-lik

e pl

at-

form

s; o

n O

S X

and

Cyg

win

, the

exe

cuta

ble

is c

alle

d py

thon

.exe

; els

ewhe

re it

’s n

amed

pyth

on. Y

ou c

an g

ener

ally

ski

p th

e la

st t

wo

of t

he a

bove

ste

ps if

you

’re

will

ing

to r

unPy

thon

3.1

out

of

your

ow

n bu

ild d

irec

tory

. B

e su

re t

o al

so c

heck

if P

ytho

n 3.

X is

ast

anda

rd C

ygw

in p

acka

ge b

y th

e ti

me

you

read

thi

s; w

hen

build

ing

from

sou

rce

you

may

hav

e to

twea

k a

few

file

s (I

had

to c

omm

ent-

out a

#de

fine

in Mod

ules

/mai

n.c)

, but

thes

e ar

e to

o sp

ecifi

c an

d te

mpo

ral t

o ge

t int

o he

re.

Thre

ads

Thr

eads

are

ano

ther

way

to s

tart

act

ivit

ies

runn

ing

at th

e sa

me

tim

e. In

sho

rt, t

hey

run

a ca

ll to

a fu

ncti

on (

or a

ny o

ther

type

of c

alla

ble

obje

ct)

in p

aral

lel w

ith

the

rest

of t

hepr

ogra

m. T

hrea

ds a

re s

omet

imes

cal

led

“lig

htw

eigh

t pr

oces

ses,

” be

caus

e th

ey r

un in

para

llel l

ike

fork

ed p

roce

sses

, but

all

of th

em ru

n w

ithi

n th

e sa

me

sing

le p

roce

ss. W

hile

proc

esse

s ar

e co

mm

only

use

d to

sta

rt in

depe

nden

t pr

ogra

ms,

thr

eads

are

com

mon

lyus

ed fo

r ta

sks

such

as

nonb

lock

ing

inpu

t cal

ls a

nd lo

ng-r

unni

ng t

asks

in a

GU

I. T

hey

also

pro

vide

a n

atur

al m

odel

for

alg

orit

hms

that

can

be

expr

esse

d as

ind

epen

dent

lyru

nnin

g ta

sks.

For

app

licat

ions

tha

t ca

n be

nefit

from

par

alle

l pro

cess

ing,

som

e de

vel-

oper

s co

nsid

er th

read

s to

off

er a

num

ber

of a

dvan

tage

s:

Perf

orm

ance

Bec

ause

all

thre

ads

run

wit

hin

the

sam

e pr

oces

s, t

hey

don’

t ge

nera

lly in

cur

a bi

gst

artu

p co

st to

cop

y th

e pr

oces

s it

self.

The

cos

ts o

f bot

h co

pyin

g fo

rked

pro

cess

esan

d ru

nnin

g th

read

s ca

n va

ry p

er p

latf

orm

, but

thre

ads

are

usua

lly c

onsi

dere

d le

ssex

pens

ive

in te

rms

of p

erfo

rman

ce o

verh

ead.

Sim

plic

ityT

o m

any

obse

rver

s, th

read

s ca

n be

not

icea

bly

sim

pler

to p

rogr

am, t

oo, e

spec

ially

whe

n so

me

of th

e m

ore

com

plex

asp

ects

of p

roce

sses

ent

er th

e pi

ctur

e (e

.g.,

proc

ess

exit

s, c

omm

unic

atio

n sc

hem

es, a

nd z

ombi

e pr

oces

ses,

cov

ered

in C

hapt

er 1

2).

Shar

ed g

loba

l mem

ory

On

a re

late

d no

te, b

ecau

se th

read

s ru

n in

a s

ingl

e pr

oces

s, e

very

thre

ad s

hare

s th

esa

me

glob

al m

emor

y sp

ace

of t

he p

roce

ss. T

his

prov

ides

a n

atur

al a

nd e

asy

way

for

thre

ads

to c

omm

unic

ate—

by fe

tchi

ng a

nd s

etti

ng n

ames

or

obje

cts

acce

ssib

leto

all

the

thre

ads.

To

the

Pyth

on p

rogr

amm

er, t

his

mea

ns t

hat

thin

gs li

ke g

loba

l

186

|Ch

apte

r 5: 

Para

llel S

yste

m To

ols