Multiprocessor Synchronization Algorithms (20225241) Lecturer: Danny Hendler The Mutual Exclusion...

Multiprocessor Synchronization

Algorithms (20225241)

Lecturer: Danny Hendler

The Mutual Exclusion problem

2

The mutual exclusion problem(Dijkstra, 1965)

We need to devise a protocolthat guarantees mutuallyexclusive access by processesto a shared resource (such asa file, printer, or code that must be performed in isolation)

3

The problem model (reads/writes)

• Shared-memory multiprocessor: multiple processes

• Processes can apply atomic reads and writes to shared registers

• Completely asynchronous

4

Mutex: formal definition

loop foreverRemainder codeEntry codeCritical section (CS)Exit code

end loop

Remainder code

Entry code

CS

Exit code

5

Mutex Requirements

• Mutual exclusion: No two processes are at their CS at the same time.

• Deadlock-freedom: If a process is trying to enter its critical section, then some process eventually enters its critical section.

• Starvation-freedom (optional): If a process is trying to enter its critical section, then this process must eventually enter its critical section.

Assumption: processes do not fail-stop while performing the entry, CS, or exit code.

6

Candidate algorithm 1.

Program for process 0

1. await turn=02. CS of process 03. turn:=1


1. await turn=12. CS of process 13. turn:=0

initially: turn=0

Does algorithm1 satisfy mutex?

Does it satisfy deadlock-freedom?

Yes

No

7


Program for both processes

1. await lock=02. lock:=13. CS4. lock:=0

initially: lock=0



No

Yes

8


initially: flag[0]=false, flag[1]=false




1. flag[0]:=true2. await flag[1]=false3. CS of process 04. flag[0]:=false


1. flag[1]:=true2. await flag[0]=false3. CS of process 14. flag[1]:=false

No

Yes

9

Peterson’s 2-process algorithm(Peterson, 1981)

initially: b[0]=false, b[1]=false, turn=0 or 1


1. b[0]:=true2. turn:=03. await (b[1]=false or

turn=1)4. CS5. b[0]:=false


1. b[1]:=true2. turn:=13. await (b[0]=false or

turn=0)4. CS5. b[1]:=false

10

Kessels’ single-writer algorithm(Kessels, 1982)

initially: b[0]=false, b[1]=false, turn[0], turn[1]=0 or 1


1. b[0]:=true2. local[0]:=turn[1]3. turn[0]:=local[0]4. Await (b[1]=false or

local[0]<>turn[1]5. CS6. b[0]:=false


1. b[1]:=true2. local[1]:=1-turn[0]3. turn[1]:=local[1]4. Await (b[0]=false or

local[1]=turn[0]5. CS6. b[1]:=false

A single-writer register is a register that can be written by a single process only.

Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

11

Mutual exclusion for n processes:

Tournament trees0

0 1

0 1 2 3

0 1 2 3 4 5 6 7

Level 0

Level 1

Level 2

Processes

A tree-node is identified by: [level, node#]Synchronization Algorithms and Concurrent Programming

Gadi Taubenfeld © 2006

12

Tournament tree based on Peterson’s 2-process alg.

Program for process i1. node:=i2. For level = 0 to log n-1 do {3. id:=node mod 24. node:= node/2 5. b[level,2node+id]:=true6. turn[level,node]:=id7. await (b[level,2node+1-id]=false or turn[level,node]=1-id)8. } 9. CS10. for level=log n –1 downto 0 do {11. node:= i/2level 12. b[level,2node+id]:=false // id is value set at this node in entry13. }

VariablesPer node: b[level, 2node], b[level, 2node+1], turn[level,node]Per process (local): level, node, id.

13

The tournament tree using Peterson’s 2-process algorithm satisfies both mutual-exclusion

and starvation-freedom.

14

Contention-free step complexity

The worst-case number of steps for a process to enter the CS when it runs by itself.

What’s the contention-free step complexity of Peterson’s tournament

tree? log n

Can we do better?

15

Lamport’s fast mutual exclusion algorithmVariables

Fast-lock, slow-lock initially 0want[i] initially false

Program for process i1. want[i]:=true2. fast-lock:=i3. if slow-lock<>0 then4. want[i]:=false5. await slow-lock=06. goto 17. slow-lock:=i8. if fast-lock <> i then9. want[i]:=false10. for j:=1 to n do await want[j] = false11. if slow-lock <> i then12. await slow-lock = 013. goto 114. CS15. slow-lock:=016. want[i]:=false

16

Schematic for Lamport’s fast mutual exclusion

Indicate contentionwant[i]:=true, fast-lock:=i

Is there contention?slow-lock< > 0?

yes

Barrierslow-lock:=i

CS

Wait until CS is releasedwant[i]:=false, await slow-

lock:=0

Is there contention?fast-lock < > i?

no

no

EXIT

yes Wait until no other process can cross the Barrier

Not last to cross Barrier?slow-lock < > i?

yesWait until CS is released

no

17

Lamport’s fast mutual exclusion algorithm satisfies both mutual-exclusion and deadlock-freedom.

18

First in First Out (FIFO)

entry code

exit code

criticalsection

remainder• Mutual Exclusion

• Deadlock-freedom

• Starvation-freedom

doorway

waiting

• FIFO: if process p is waiting and process q has not yet started the doorway, then q will not enter the CS before p.


20

time

Lamport’s Bakery Algorithm

0 0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

1

1

2 2

2 2

1

1

0

2

2

0

3

3

2

2

0

4

4waiting

en

try

remainder


21

Implementation 1code of process i , i {1 ,..., n}

number[i] := 1 + max {number[j] | (1 j n)}for j := 1 to n (<> i) { await (number[j] = 0) (number[j] > number[i])}critical sectionnumber[i] := 0

1 2 3 4 n

number integer0 0 0 0 0 0

Answer: No, it can deadlock!

Does this implementation work?


22

time

Implementation 1: deadlock

0 0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

1

1

2 2

2 2

1

1

0

waiting

en

try

remainder

deadlock


23

number[i] := 1 + max {number[j] | (1 j n)}for j := 1 to n (<> i) { await (number[j] = 0) (number[j],j) > number[i],i)

// lexicographical order

}critical sectionnumber[i] := 0

1 2 3 4 n


Answer: It does not satisfy mutual exclusion!

Implementation 2code of process i , i {1 ,..., n}

Does this implementation work?


24

time

Implementation 2: no mutual exclusion

0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

0

1

0 0

2 2

1

1

0

2 2

waiting

en

try

remainder

1 2 2

0


25

The Bakery Algorithmcode of process i , i {1 ,..., n}

1: choosing[i] := true2: number[i] := 1 + max {number[j] | (1 j n)}3: choosing[i] := false4: for j := 1 to n do {5: await choosing[j] = false 6: await (number[j] = 0) (number[j],j) (number[i],i)7: }8: critical section9: number[i] := 0

1 2 3 4 n

choosing bitsfalse


false false false false false

Doorway

Waiting Bakery

26

Computing the maximumcode of process i , i {0 ,..., n-1}

local1 := 0for local2 := 1 to n { local3 := number[local2] if local1 < local3 then {local1 := local3}}number[i] := 1+local1

0123

n-1

numberchoosingfalsefalsefalsefalsefalse

false

00000

0

The correctness of the Bakery algorithm depends on an implicit assumption on the implementation of computing the maximum (statement 2). Below we give a correct implementation. For each process, three additional local registers are used. They are named local1, local2, local3 and their initial values are immaterial.


27

Question: Computing the maximum

code of process i , i {0 ,..., n-1}

local1 := ifor local2 := 1 to n { if number[local1] < number[local2] then {local1 := local2}}number[i] := 1+ number[local1]

0123

n-1

numberchoosingfalsefalsefalsefalsefalse

false

00000

0

Is the following implementation also correct? That is, does the Bakery algorithm solve the mutual exclusion problem when the following implementation is used? Justify your answer.For each process, two additional local registers are used. They are named local1, local2, and their initial values are immaterial.


time

0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

1 1

1 1

1

1

0

1

waiting

en

try

remainder 00

?

1

1

?local1 2

1Passed process 1

Waiting for process 2

The 2nd maximum alg. doesn’t work


29

Properties of the Bakery algorithm

• Satisfies Mutual exclusion and first-come-first-served.

• The size of number[i] is unbounded.– In practice this is not a problem if 64-bit tickets are

used

• There is no need to assume that operations on the same memory location occur in some definite order; it works correctly even when it is allowed for reads which are concurrent with writes to return an arbitrary value.

The Black-White Bakery AlgorithmThe Black-White Bakery AlgorithmBounding the space of the Bakery Algorithm

Bakery (FIFO, unbounded)

The Black-White Bakery Algorithm

FIFOBounded space+ one bit


31

time

The Black-White Bakery AlgorithmThe Black-White Bakery Algorithm

0 0 0 0 0

doorway

1 2 3 4 5 n

CS

exit

0

1

0 0

2 2

1

1

0

2

2

0

1

2

2

0

2waiting

en

try

remainder

1 20 201 2

1

1

00

color bit


32

The Black-White Bakery Algorithm

1 2 3 4 n

choosing

Data StructuresData Structures

mycolor

number

color bit

bits

bits

{0,1,...,n}

{black,white}


33

The Black-White Bakery Algorithmcode of process i , i {1 ,..., n}

choosing[i] := truemycolor[i] := colornumber[i] := 1 + max{number[j] | (1 j n) (mycolor[j] = mycolor[i])}choosing[i] := falsefor j := 0 to n do { await choosing[j] = false if mycolor[j] = mycolor[i] then await (number[j] = 0) (number[j],j) (number[i],i) (mycolor[j] mycolor[i]) else await (number[j] = 0) (mycolor[i] color) (mycolor[j] = mycolor[i]) fi }critical sectionif mycolor[i] = black then color := white else color := black finumber[i] := 0


34

A space lower bound fordeadlock-free mutex

(Burns & Lynch, 1993)

How many registers must an n-process deadlock-free mutual exclusion algorithm use if it can only usesingle-writer registers?

We now prove that the same result holds for multi-reader-multi-writer registers, regardless of their

size.

35

Some definitions required for the proof

• Configuration

• A quiescent configuration

• Indistinguishable configurations

• A P-quiescent configuration

• A covered register

• An execution

36

Example of indistinguishabilityExecution x is indistinuishable from execution y to process p

execution x• p reads 5 from r1• q writes 6 to r1• p writes 7 to r1• q writes 8 to r1• p reads 8 from r1

execution y• p reads 5 from r1• p writes 7 to r1• q writes 6 to r1• q reads 6 from r1• q writes 8 to r1• p reads 8 from r1

8r1 8r1

• q writes 6 to r1

6

The values of the shared registers must also be the same


37

Illustration for Lemma 1

C

pi-quiescent,W covered by P

Quiescent

D

C ~ Dpi

(by pi)

C1pi in CS

pi in CS

(by pi) D1

Q

Quiescent

(By pj)

R

Pj in CS

Q1

Q ~ Q1

pj

(By pj)

Z

Both pi, pj in CS!

Based on the proof in “Distributed Computing”, by Hagit Attiya & Jennifer Welch

38

Illustration for the simple part of Lemma 2

C1

{pk,…,pn-1}-quiescent p0…pk-1 cover W

pk runs until it covers x

' (pk only)

D1Quiescent

{pk+1,…,pn-1}-quiescent W U {x} covered

C'2

D‘1 ~ D1

P-{pk}

(by p0… pk-1) C2

{pk,…,pn-1}-quiescentp0…pk-1 cover W


D'1

p0… pk-1 write to W and exit

x is coveredP-{pk} in remainder

39

Illustration for the general part of Lemma 2


D0 D'i'i

quiescent

D1

1

quiescent

C11

{pk,…,pn-1}-quiescent p0…pk-1 cover W1

C22

{pk,…,pn-1}-quiescent p0…pk-1 cover W2

Ci

2… i

{pk,…,pn-1}-quiescent p0…pk-1 cover Wi

{pk+1,…,pn-1}-quiescent W U {x} covered

C’j

i+1i+1… j

D‘i ~ Di

{pk+1,…,pn-1}-i

Di

quiescent

Cj

i+1i+1… j

{pk+1,…,pn-1}-quiescent p0…pk-1 cover Wi

40

A matching upper bound:the one-bit algorithm

initially: b[i]:=false

Program for process i1. repeat2. b[i]:=true; j:=13. while (b[i] = true) and (j < i) do4. if (b[j]=true then 5. b[i]:=false6. await b[j]=false7. j:=j+18. until b[i]=true9. for (j:=i+1 to n) do10. await b[j]=false 11. Critical Section12. b[i]=false

41

Read-Modify-Write (RMW) operations

Read-modify-write (w, f)do atomically prev:=w w:=f(prev) return prev

Fetch-and-add(w, Δ)do atomically prev:=w w:= prev+Δ return prev

Test-and-set(w)do atomically prev:=w w:=1 return prev

42

Mutual exclusion using test-and-set

Program for process I

1. await test&set(v) = 02. Critical Section3. v:=0

initially: v:=0

Mutual exclusion?Yes

Deadlock-freedom?

NoStarvation-freedom?

Yes

43

Mutual exclusion using general RMW

Program for process I

1. position:=RMW(v, <v.first, v.last+1> )

2. repeat3. queue:=v4. until queue.first = position.last5. Critical Section6. RMW(v, <v.first+1, v.last> )

initially: v:=<0,0>

How many bits does this algorithm require?

Unbounded number, but can be improved to 2 log2 n

44

Lower bound on the number

of bits required for mutual exclusion

Multiprocessor Synchronization Algorithms (20225241) Lecturer: Danny Hendler The Mutual Exclusion...

Documents

Transcript of Multiprocessor Synchronization Algorithms (20225241) Lecturer: Danny Hendler The Mutual Exclusion...