Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.
-
date post
20-Dec-2015 -
Category
Documents
-
view
215 -
download
2
Transcript of Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.
![Page 1: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/1.jpg)
Introduction to Lock-freeData-structures and algorithms
Micah J BestMay 14/09
![Page 2: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/2.jpg)
Introduction to the Introduction
• Some problems with locks:– Deadlock: The condition where there are at
least two processes A and B such that A holds a lock on a resource required by B to complete and B holds a lock on a resource required by A to complete and they wait indefinitely.
– In general Deadlock avoidance is hard if not NP-Complete.
![Page 3: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/3.jpg)
(Some) Problems with Locks
• Priority Inversion: A low-priority process holds a lock on a resource desired by a high priority process.
• Not very granular, a small percentage of the operations performed in the critical section may actually modify shared memory, hurting performance
![Page 4: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/4.jpg)
If not locks than what?
• Without hardware support Lock-free algorithms are possible, but not really practical (exceptions do exist: see Lamport’s Bakery Algorithm)
• Hardware support can be generalized as atomic operations.
![Page 5: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/5.jpg)
What is Lock-free? (Definitions)• Atomic: From the Greek ἄτομος (atomos)
meaning indivisible or uncuttable. • Atomic Operations: A set of operations that
execute as if they were a single simple operation.
![Page 6: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/6.jpg)
What is Lock-free? (Definitions)
• Lock-free: Algorithms/Data Structures that can be invoked or accessed in a parallel context accessing shared memory without a mechanism to protect their critical section (such as a mutex)
• Critical Section: A set of instructions necessary to complete a single complete operation on a shared memory resource. (Ex: Pop operation on a stack)
![Page 7: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/7.jpg)
Compare and Swap (CAS)
• For integers n and n' and a memory location a
• CAS( n, a, n' )if the value at address a is n
write the value of n' to address a return true
otherwise return false
![Page 8: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/8.jpg)
Compare and Swap (CAS)
• Executes atomically, often the values of n and n' represent memory addresses
• Supported in some form as far back as the IBM 370 and available on almost all modern general purpose microprocessors (Pentium, Power PC, etc)
![Page 9: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/9.jpg)
Compare and Swap (CAS)
• Often used to implement 'busy waiting’
... success <- false Do success <- CAS( n, a, n') Until success ...
![Page 10: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/10.jpg)
Compare and Swap (CAS)
• Example: Pushing an item onto a lock-free stack Let Top be address of the top of the stack
Let Next(I) where I is a valid address of a stack item be the address where the address of the next item after I is stored
Let New be the address of the item for the stack
![Page 11: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/11.jpg)
Compare and Swap (CAS)
• Example: Pushing an item onto a lock-free stack
Push ( New )Success <- falseDo
T' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
![Page 12: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/12.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Head
![Page 13: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/13.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
We create a new item
7
![Page 14: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/14.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
Read the value of Top
7
T’
![Page 15: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/15.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
Set the new item’s Next to T’
7
T’
![Page 16: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/16.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
Suppose another thread adds an item before the assignment and the CAS
7
T’
3
![Page 17: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/17.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
T’ != Top so the CAS will fail – we must start again
7
T’
3
![Page 18: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/18.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
7
T’
3
![Page 19: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/19.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
7
T’
3
![Page 20: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/20.jpg)
Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false
DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )
Until Success
5 3
Top
7
T’
3
This time the CAS succeeds and we are done
![Page 21: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/21.jpg)
ABA problem
• Where the value read(and stored) by a process (A) is changed by another process (B) to the same value as read by A, but causes some other state change that causes A to produce an incorrect result
![Page 22: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/22.jpg)
ABA problem
• Consider the stack and two threads A and B and the following (incorrect) pop operation:Let Value(I), where I is a valid address of a stack item, be the value associated with I
Pop ()Do
T' <- Top’N' <- Next(T')V <- Value(T') Success <- CAS( T', Top, N' )
Until Success Return V
![Page 23: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/23.jpg)
ABA problemA B
5 3
Top
72
![Page 24: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/24.jpg)
ABA problemA
Do
T' <- TopN' <- Next(T’)V <- Value(T')
B
5 3
Top
72
V = 7
T’
N’
![Page 25: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/25.jpg)
ABA problemA
Do
T' <- Top’N' <- Next(T’)V <- Value(T')
B
Pop()
5 3 2
V = 7
T’
N’
7 is returned
Top
![Page 26: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/26.jpg)
ABA problemA
Do
T' <- Top’N' <- Next(T’)V <- Value(T')
B
Pop()
Push( 42 )
5 3 2
V = 7
T’42
The new item is created in
the old memory location
N’Top
![Page 27: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/27.jpg)
ABA problemA
Do
T' <- Top’N' <- Next(T’)V <- Value(T')
Success <- CAS( T', Top, N’ )
B
Pop()
Push( 42 )
5 3 2
V = 7
T’
N’Top
42
The CAS willSucceed
![Page 28: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/28.jpg)
ABA problemA
Do
T' <- Top’N' <- Next(T’)V <- Value(T')
Success <- CAS( T', Top, N’ )Until Success Return V
B
Pop()
Push( 42 )
5 3 2
V = 7
T’
N’Top
42
7 will be returned
![Page 29: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/29.jpg)
ABA problemA
Do
T' <- Top’N' <- Next(T’)V <- Value(T')
Success <- CAS( T', Top, N’ )Until Success Return V
B
Pop()
Push( 42 )
5 3 2
V = 7
T’
N’Top
42
7 will be returned- again.
![Page 30: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/30.jpg)
What is Wait-free?
• Wait free: Algorithms/Data Structures that can be invoked or accessed in a parallel context accessing shared memory such that execution time is guaranteed and predictable.
• Almost always lock-free.• Not all lock-free constructions are wait-free
(most use some form of 'busy waiting' which is inherently unpredictable)
![Page 31: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/31.jpg)
Multi-Producer/Consumer Circular Queues
(The Problem)
• Circular queue: An array of fixed size, to be accessed in a FIFO manner
• Multi-producer: More than one process may write to the queue at once
• Multi-consumer: More than one process may read from the queue at once
![Page 32: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/32.jpg)
Multi-Producer/Consumer Circular Queues
(The Problem)
• The advantages of lock free:– After a cell in the array has been 'reserved' for
a particular process - writing or reading will cause no contention.
– Only maintenance of queue parameters need made safe
![Page 33: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/33.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)
• The advantages of lock free:– After a cell in the array has been 'reserved' for
a particular process - writing or reading will cause no contention.
– Only maintenance of queue parameters need made safe
![Page 34: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/34.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Sequential Version:
Let A(i) be the element of the array at position i (starting at 0) Let Size be the number of cells in the arrayLet Head be the index of the first free cell for writing Let Tail be the index of the first occupied cell for writingLet Write(i) represent the work to write the data in question to array position i
![Page 35: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/35.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Sequential Version:
Enqueue()
If not head = tail
Write( Head )
Head <- ( Head + 1 ) mod Size
else
Fail
![Page 36: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/36.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Obvious not correct in a parallel contextConsider:
A B
![Page 37: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/37.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Obvious not correct in a parallel contextConsider:
A BEnqueue()
![Page 38: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/38.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Obvious not correct in a parallel contextConsider:
A BEnqueue()
Enqueue()
![Page 39: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/39.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Obvious not correct in a parallel contextConsider:
A BEnqueue()
Enqueue()
Write( Head )
![Page 40: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/40.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Obvious not correct in a parallel contextConsider:
A BEnqueue()
Enqueue()
Write( Head )
Write( Head )
![Page 41: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/41.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Obvious not correct in a parallel contextConsider:
A BEnqueue()
Enqueue()
Write( Head )
Write( Head )
Head<-( Head + 1 ) mod Size
![Page 42: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/42.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Obvious not correct in a parallel contextConsider:
A BEnqueue()
Enqueue()
Write( Head )
Write( Head )
Head<-( Head + 1 ) mod Size
Head<-( Head + 1 ) mod Size
Values will be written to the same place
![Page 43: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/43.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)
• Need additional information:Let NumWriters and NumReaders be the maximum amount of writers and readers allowed in the queue respectively
Initially NumWriters = Size and NumReaders = 0
![Page 44: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/44.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• First solution: Advance the head pointer
atomicallyLet *Head be the memory address of the value HeadLet *NR be the address of NumReadersEnqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Increase NumReaders
![Page 45: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/45.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)
• Just when you thought is was (thread) safe:Consider the following series of events:
Queue is Empty
HeadNumReaders = 0
![Page 46: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/46.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Just when you thought is was (thread) safe:
Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointer
HeadNumReaders = 0
![Page 47: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/47.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Just when you thought is was (thread) safe:
Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing
HeadNumReaders = 0
23423234
![Page 48: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/48.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Just when you thought is was (thread) safe:
Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointer
HeadNumReaders = 0
234232343
![Page 49: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/49.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Just when you thought is was (thread) safe:
Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writing
HeadNumReaders = 0
23458334
2342323431
![Page 50: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/50.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Just when you thought is was (thread) safe:
Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writingThread B finishes writing
HeadNumReaders = 0
23423234319
234583349839274
![Page 51: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/51.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Just when you thought is was (thread) safe:
Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writingThread B finishes writingThread B succeeds in increasing NumReaders
HeadNumReaders = 1
234232343195
234583349839274
![Page 52: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/52.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Just when you thought is was (thread) safe:
Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writingThread B finishes writingThread B succeeds in increasing NumReaders
HeadNumReaders = 1
234232343195
234583349839274
This can now be read – before
writing has finished!
![Page 53: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/53.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• A 'busy waiting' solution
Let WriteMarker be equal to Head when Queue is created
Enqueue() ( NumWriters is decremented )Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
![Page 54: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/54.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
2423423
We begin as another thread is already writing to the queue
H’
WriteMarker
![Page 55: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/55.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
24234234
Copy the head index
H’
WriteMarker
![Page 56: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/56.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
242342348
Advance head(assume we succeed)
H’
WriteMarker
![Page 57: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/57.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
2423423485
Write to the buffer
H’
WriteMarker
97234
![Page 58: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/58.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
24234234858
Write to the buffer
H’
WriteMarker
9723498734
![Page 59: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/59.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
242342348586
Write to the buffer
H’
WriteMarker
972349873402393
![Page 60: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/60.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
2423423485867
Keeping looping until the other thread advances WriteMarker
H’
WriteMarker
972349873402393
![Page 61: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/61.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
24234234858678
Keeping looping until the other thread advances WriteMarker
H’
WriteMarker
972349873402393
![Page 62: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/62.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
242342348586784
Keeping looping until the other thread advances WriteMarker
H’
WriteMarker
972349873402393
![Page 63: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/63.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
242342348586784
Keeping looping until the other thread advances WriteMarker
H’
WriteMarker
972349873402393
![Page 64: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/64.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
242342348586784
Now we advance WriteMarker ourselves.
(Why don’t we need to CAS this?)
H’
WriteMarker
972349873402393
![Page 65: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/65.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Enqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )
Until SuccessWrite( H' )Do
W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders
Head
242342342342343
242342348586784
Finally we increase NumReaders now that the data
is safe to be read
H’
WriteMarker
972349873402393
![Page 66: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/66.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Everything works better when we all cooperateLet Done(i) be a boolean variable associated with each cell in the Array, Initially
falseLet *WM be the address of WriteMarkersEnqueue() ( NumWriters is decremented )
Success <- falseDo
H' <- HeadSuccess <- CAS( H', *H, (H' + 1) mod Size )
Until SuccessWrite( H' )
![Page 67: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/67.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)• Everything works better when we all cooperate(con’t)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
![Page 68: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/68.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
2423423
8653869
WriteMarker
972349873402393
Done: false false false falsefalse true false
The story so far:
![Page 69: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/69.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
2423423
8653869
WriteMarker
972349873402393
Done: false false false falsefalse true false
The story so far:There are two cells currently
being written
![Page 70: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/70.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
2423423
8653869
WriteMarker
972349873402393
Done: false false false falsefalse true false
The story so far:This is ‘us’
![Page 71: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/71.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
2423423
8653869
WriteMarker
972349873402393
Done: false false false falsefalse true false
Writing continues
![Page 72: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/72.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
24234237
86538694
WriteMarker
972349873402393
Done: false false false falsefalse true false
Writing continues
![Page 73: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/73.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
2423423745
8653869445
WriteMarker
972349873402393
Done: false false false falsefalse true false
Writing continues
![Page 74: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/74.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
24234237458
86538694454
WriteMarker
972349873402393
Done: false false false falsefalse true false
Writing continues
![Page 75: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/75.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
2423423745893
8653869445489
WriteMarker
972349873402393
Done: false false false falsefalse true false
Writing continues
![Page 76: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/76.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse true false
Writing finishes
![Page 77: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/77.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsetrue true true
Both threads update Done
![Page 78: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/78.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsetrue true true
Both threads try to move the write pointer
![Page 79: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/79.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsetrue true true
We succeed. This means that the other thread has
failed. It will finish knowing that we will ‘clean things up’
![Page 80: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/80.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse true true
Update the Done flag
![Page 81: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/81.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse true true
Let another reader in – now that we know it’s safe.
![Page 82: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/82.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse true true
Repeat until all everything is in a proper state or
somebody else takes over.
![Page 83: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/83.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse false true
Repeat until all everything is in a proper state or
somebody else takes over.
![Page 84: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/84.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse false false
Repeat until all everything is in a proper state or
somebody else takes over.
![Page 85: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/85.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse false false
Repeat until all everything is in a proper state or
somebody else takes over.
![Page 86: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/86.jpg)
Multi-Producer/Consumer Circular Queues
(The Algorithm)Done(H') <- true
Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )
W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success
Done( W ) <- falseIncrease NumReaders
While Success
Head
242342374589334
865386944548973
WriteMarker
972349873402393
Done: false false false falsefalse false false
Exit with the knowledge of a job well done
![Page 87: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/87.jpg)
Multi-Producer/Consumer Circular Queues
(Proof - Sketch)• Everything works better when we all cooperate
• Criteria for correctness: – 1) At all times: ( WriteMarker - Tail ) mod Size >=
NumReaders – 2) Enqueue always terminates– 3) At any point in execution if no Enqueue operation is
in progress then WriteMarker = Head– 4) Once an item is enqueued in cell i no other
enqueue operation will write to i until it is dequeued (follows from 1 and case analysis)
![Page 88: Introduction to Lock-free Data-structures and algorithms Micah J Best May 14/09.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d435503460f94a1e7b1/html5/thumbnails/88.jpg)
Some Performance Results