Art of Multiprocessor Programming1 Concurrent Queues Companion slides for The Art of Multiprocessor...

Art of Multiprocessor Programming

1

Concurrent Queues

Companion slides forThe Art of Multiprocessor Programming

by Maurice Herlihy & Nir Shavit

Modified by Rajeev Alur For CIS 640 University of Pennsylvania


2Art of Multiprocessor Programming

2

Filterclass Filter implements Lock { int[] level; // level[i] for thread i int[] victim; // victim[L] for level L

public Filter(int n) { level = new int[n]; victim = new int[n]; for (int i = 1; i < n; i++) { level[i] = 0; }}…

}

level

victim

n-1

n-1

0

1

0 0 0 0 0 04

2

2

Thread 2 at level 4

0

4



3

Filterclass Filter implements Lock { …

public void lock(){ for (int L = 1; L < n; L++) { level[i] = L; victim[L] = i;

while ((k != i level[k] >= L) && victim[L] == i ); }} public void unlock() { level[i] = 0; }}


44

Queues & Stacks

• pool of items


55

Queues

deq()/ enq( )

Total orderFirst in

First out


66

Stacls

pop()/

push( )

Total orderLast in

First out


77

Bounded

• Fixed capacity• Good when resources an issue


88

Unbounded

• Unlimited capacity• Often more convenient

…


9

Blockingzzz

…Block on attempt to remove from empty

stack or queue


10

Blockingzzz

…Block on attempt to add to full bounded stack or

queue


11

Non-BlockingOuch

! Throw exception on attempt to remove

from empty stack or queue


1212

Queue: Concurrency

enq(x) y=deq()

enq() and deq() work at

different ends of the object

tail head


1313

Concurrency

enq(x)

Challenge: what if the queue is empty or full?

y=deq()ta

ilhead


1414

Bounded Queue

Sentinel

head

tail


1515

Bounded Queue

head

tail

First actual item


1616

Bounded Queue

head

tail

Lock out other deq() calls

deqLock


1717

Bounded Queue

head

tail

Lock out other enq() calls

deqLock

enqLock



18

Not Done Yet

head

tail

deqLock

enqLock

Need to tell whether queue is

full or empty


1919

Not Done Yet

head

tail

deqLock

enqLock

Permission to enqueue 8 items

permits

8


2020

Not Done Yet

head

tail

deqLock

enqLock

Incremented by deq()Decremented by enq()

permits

8


2121

Enqueuer

head

tail

deqLock

enqLock

permits

8

Lock enqLock


2222

Enqueuer

head

tail

deqLock

enqLock

permits

8

Read permits

OK


2323

Enqueuer

head

tail

deqLock

enqLock

permits

8

No need to lock tail


2424

Enqueuer

head

tail

deqLock

enqLock

permits

8

Enqueue Node


2525

Enqueuer

head

tail

deqLock

enqLock

permits

87

getAndDecrement()


2626

Enqueuer

head

tail

deqLock

enqLock

permits

8 Release lock7


2727

Enqueuer

head

tail

deqLock

enqLock

permits

7

If queue was empty, notify waiting

dequeuers


2828

Unsuccesful Enqueuer

head

tail

deqLock

enqLock

permits

0

Uh-oh

Read permits


2929

Dequeuer

head

tail

deqLock

enqLock

permits

8

Lock deqLock


3030

Dequeuer

head

tail

deqLock

enqLock

permits

7

Read sentinel’s next field

OK


3131

Dequeuer

head

tail

deqLock

enqLock

permits

7

Read value


3232

Dequeuer

head

tail

deqLock

enqLock

permits

7

Make first Node new sentinel


3333

Dequeuer

head

tail

deqLock

enqLock

permits

8

Increment permits


3434

Dequeuer

head

tail

deqLock

enqLock

permits

7Release deqLock


3535

Unsuccesful Dequeuer

head

tail

deqLock

enqLock

permits

8

Read sentinel’s next field

uh-oh


3636

Bounded Queue

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}


3737

Bounded Queue


Enq & deq locks


3838

Digression: Monitor Locks

• Java Synchronized objects and Java ReentrantLocks are monitors

• Allow blocking on a condition rather than spinning

• Threads: – acquire and release lock– wait on a condition


3939

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Acquire lock


4040



Release lock


4141



Try for lock, but not too hard


4242



Create condition to wait on


4343



Guess what this method does?


4444

Lock Conditions

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }


4545


Lock Conditions

Release lock and wait on condition


4646


Lock Conditions

Wake up one waiting thread


4747


Lock Conditions

Wake up all waiting threads


4848

Await

• Releases lock associated with q• Sleeps (gives up processor)• Awakens (resumes running)• Reacquires lock & returns

q.await()


4949

Signal

• Awakens one waiting thread– Which will reacquire lock

q.signal();


5050

Signal All

• Awakens all waiting threads– Which will each reacquire lock

q.signalAll();


5151

A Monitor Lock

Cri

tical S

ecti

on

waiting roomLock(

)

unLock()


5252

Unsuccessful Deq

Cri

tical S

ecti

on

waiting roomLock

()

await()

Deq()

Oh no, Empty!


5353

Another One

Cri

tical S

ecti

on

waiting roomLock

()

await()

Deq()

Oh no, Empty!


5454

Enqueur to the Rescue

Cri

tical S

ecti

on

waiting roomLock(

)

signalAll()

Enq( )

unLock()

Yawn!Yawn!


5555

Yawn!

Monitor Signalling

Cri

tical S

ecti

on

waiting room

Yawn!

Awakend thread might still lose lock to outside contender…


5656

Dequeurs Signalled

Cri

tical S

ecti

on

waiting room

Found it

Yawn!


5757

The Bounded Queue



5858

Bounded Queue Fields


Enq & deq locks


5959



Enq lock’s associated condition


6060



Num permits: 0 to capacity


6161



Head and Tail


6262

Enq Method Part Onepublic void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}


6363

public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Enq Method Part One

Lock and unlock enq

lock


6464


Enq Method Part One

If queue is full, patiently await further

instructions …


6565


Be Afraid

How do we know the permits field won’t

change?


6666


Enq Method Part One

Add new node


6767


Enq Method Part One

If queue was empty, wake frustrated dequeuers


6868

Enq Method Part Deux

public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }


6969


public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }Are there dequeuers to be signaled?


7070



Lock and unlock deq

lock


7171



Signal dequeuers that queue no longer

empty


7272

The Enq() & Deq() Methods

• Share no locks– That’s good

• But do share an atomic counter– Accessed on every method call– That’s not so good

• Can we alleviate this bottleneck?– By splitting the counter


7373

A Lock-Free Queue (Michael&Scott)

Sentinel

head

tail


7474

Compare and Set

CAS


7575

Enqueue

head

tail

Enq( )


7676

Enqueue

head

tail


7777

Logical Enqueue

head

tail

CAS


7878

Physical Enqueue

head

tail

Enqueue Node

CAS


7979

Enqueue

• These two steps are not atomic• The tail field refers to either

– Actual last Node (good)– Penultimate Node (not so good)

• Be prepared!


8080

Enqueue

• What do you do if you find– A trailing tail?

• Stop and help fix it– If tail node has non-null next field– CAS the queue’s tail field to tail.next


8181

When CASs Fail

• During logical enqueue– Abandon hope, restart– Still lock-free (why?)

• During physical enqueue– Ignore it (why?)


8282

Dequeuer

head

tail

Read value


8383

Dequeuer

head

tail

Make first Node new sentinel

CAS


8484

Enq Method public void enq(T x) { Node node = new Node(x); while (true) { Node last = tail.get(); Node next = last.next.get(); if (last == tail.get()) { if (next == null) { if (last.next.compareAndSet(next,node)) { tail.compareAndSet(last, node); return; } } else { tail.compareAndSet(last, next); } } }


8585

Memory Reuse?

• What do we do with nodes after we dequeue them?

• Java: let garbage collector deal?• Suppose there is no GC, or we

prefer not to use it?


8686

Dequeuer

head

tail

CAS

Can recycle


8787

Simple Solution

• Each thread has a free list of unused queue nodes

• Allocate node: pop from list• Free node: push onto list• Deal with underflow somehow …


8888

Why Recycling is Hard

Free pool

head tail

Want to redirect

tailfrom

grey to red

zzz…


8989

Both Nodes Reclaimed

Free pool

zzz

head tail


9090

One Node Recycled

Free pool

Yawn!

head tail


9191

Why Recycling is Hard

Free pool

CAShead tail

OK, here I go!


9292

Epic FAIL

Free pool

zOMG what went wrong?

head tail

Bad news


9393

The Dreaded ABA Problem

Head pointer has value AThread reads value A

head tail


9494

Dreaded ABA continued

zzz Head pointer has value BNode A freed

head tail


9595


Yawn! Head pointer has value A againNode A recycled & reinitialized

head tail


9696


CAS succeeds because pointer matcheseven though pointer’s meaning has changed

CAShead tail


9797

The Dreaded ABA Problem

• Is a result of CAS() semantics– blame Sun, Intel, AMD, …

• Not with Load-Locked/Store-Conditional– Good for IBM?


9898

Dreaded ABA – A Solution

• Tag each pointer with a counter• Unique over lifetime of node• Pointer size vs word size issues• Overflow?

– Don’t worry be happy?– Bounded tags?

• AtomicStampedReference class


9999

Atomic Stamped Reference

• AtomicStampedReference class– Java.util.concurrent.atomic package

address S

Stamp

Reference

Can get reference and stamp atomically

Art of Multiprocessor Programming1 Concurrent Queues Companion slides for The Art of Multiprocessor...

Documents

Transcript of Art of Multiprocessor Programming1 Concurrent Queues Companion slides for The Art of Multiprocessor...