A Mile-High View of Concurrent Algorithms Hagit Attiya Technion.

A Mile-High View of Concurrent Algorithms

Hagit AttiyaTechnion

May 1, 2008 Concurrent algorithms @ COVA 2

A Simplistic View of Concurrent SystemsA collection of processes

Each a sequential thread of execution

Communicating through shared data structures


Alternative Routes for Developing Concurrent AlgorithmsRefinement: Implementing high-level ADT from

lower-level ADTs– Safety conditions, liveness properties

– Hierarchical

Transactional: Support for running sequential applications concurrently– Safety conditions, liveness properties


Abstract Data Types (ADT)

Cover most concurrent applications– At least encapsulate their data needs

– An object-oriented programming point of view Abstract representation of data

& set of methods (operations) for accessing it– Signature

– Specification

data


data

data

Implementing High-Level ADT


Implementing High-Level ADT

data

data

-------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------------------

Using lower-level ADTs

&procedures


Lower-Level Operations

High-level operations translate into primitives on base objects– Obvious: read, write (restrictions?)– Common: compare&swap (CAS)– LL/SC, Double-CAS (2CAS,

DCAS), kCAS, …– Generic: read-modify-write (RMW), kRMW

Low-level operations are often implemented from more primitive operations– A hierarchy of implementations


Executing Operations

P1

invocation response

P2

P3


Interleaving Operations

Concurrent execution


Interleaving Operations

)External (behavior


Interleaving Operations, or Not

Sequential execution


Interleaving Operations, or Not

Sequential behavior: invocations & response alternate and match (on process & object)

Sequential specification: All the legal sequential behaviors, satisfying the semantics of the ADT– E.g., for a (LIFO) stack: pop returns the last item pushed


Correctness: Sequential consistency

[Lamport, 1979]

For every concurrent execution there is a sequential execution that– Contains the same operations

– Is legal (obeys the sequential specification)

– Preserves the order of operations by the same process


Sequential Consistency: Examples

push(4)

pop():4push(7)

Concurrent (LIFO) stack

push(4)

pop():4push(7)

Last In First Out


Sequential Consistency: Examples

push(4)

pop():7push(7)

Concurrent (LIFO) stack

Last In First Out


Example 1: Multi-Writer Registers

Add logical time (Lamport timestamps) to values

Write(v)

read TS1,...,read TSn

TSi = max TSj +1

write v,TSi

Read only own value

Read()

read v,TSi

return v

Once in a while


write max TSj to TSi

~[Attiya, Welch TOCS 1994]

Using (multi-reader) single-writer registers

Need to ensure writes are eventually visible


Multi-Writer Registers: Proof

Write(v,X)


TSi = max TSj +1

write v,TSi

Read(X)

read v,TSi

return v

Once in a while


write max TSj to TSi

Create sequential execution: – Place writes in timestamp order– Insert reads after the appropriate write



Create sequential execution: – Place writes in timestamp order– Insert reads after the appropriate write

Legality is immediate Per-process order is preserved since a read returns a

value (with timestamp) larger than the preceding write by the same process


Sequential Consistency is not Composable

enq(Q1,X) enq(Q2,X) deq(Q1,Y)enq(Q2,Y) enq(Q1,Y) deq(Q2,X)

The execution is not sequentially consistent


Sequential Consistency is not Composable

enq(Q1,X) deq(Q1,Y)enq(Q1,Y)enq(Q2,X)enq(Q2,Y) deq(Q2,X)

The execution projected on each object is sequentially consistent

Bad news forverification!


Correctness: Linearizability

[Herlihy & Wing, 1990]

For every concurrent execution there is a sequential execution that– Contains the same operations

– Is legal (obeys the specification of the ADTs)

– Preserves the real-time order of non-overlapping operations

Each operation appears to takes effect instantaneously at some point between its invocation and its response (atomicity)


Linearizable Multi-Writer Registers

Add logical time to values

Write(v,X)


TSi = max TSj +1

write v,TSi

Read(X)


return value with max TS

Using (multi-reader) single-writer registers[Vitanyi & Awerbuch, 1987]


Multi-writer registers: Linearization orderCreate linearization:

– Place writes in timestamp order– Insert each read after the appropriate write

Write(v,X)


TSi = max TSj +1

write v,TSi

Read(X)


return value with max TS



Create linearization: – Place writes in timestamp order– Insert each read after the appropriate write

Legality is immediate Real-time order is preserved since a read returns a value

(with timestamp) larger than all preceding operations


Linearizability is Composable

The whole system is linearizable each object is linearizable

Allows to implement and verify each object separately

Good news forverification!


Example 3: Atomic Snapshot

m components Update a single component Scan all the components

“at once” (atomically)

Provides an instantaneous view of the whole memory

update

ok

scan

v1,…,vm


Atomic Snapshots: Algorithm

Update(v,k)

A[k] = v,seqi,iScan()

repeat

read A[1],…,A[m]

read A[1],…,A[m]

if equal

return A[1,…,m]

Linearize:• Updates with their writes• Scans inside the double collects

double collect

[Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]


Atomic Snapshot: LinearizabilityDouble collect (read a set of values twice)

If equal, there is no write between the collects– Assuming each write has a new value (seq#)

Creates a “safe zone”, where the scan can be linearized

read A[1],…,A[m] read A[1],…,A[m]

write A[j]


Liveness Conditions (Eventual)

Wait-free: every operation completes within a finite number of (its own) steps no starvation for mutex

Nonblocking: some operation completes within a finite number of (some other process) steps deadlock-freedom for mutex

Obstruction-free: an operation (eventually) running solo completes within a finite number of (its own) steps– Also called solo termination

wait-free nonblocking obstruction-free


Liveness Conditions (Bounded)

Wait-free: every operation completes within a bounded number of (its own) steps no starvation for mutex

Nonblocking: some operation completes within a bounded number of (some other process) steps deadlock-freedom for mutex

Obstruction-free: an operation (eventually) running solo completes within a bounded number of (its own) steps– Also called solo termination

Bounded wait-free bounded nonblocking bounded obstruction-free


Wait-free Atomic Snapshot[Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]

Embed a scan within the Update.

Update(v,k)

V = scan

A[k] = v,seqi,i,V

Scan()

repeat

read A[1],…,A[m]

read A[1],…,A[m]

if equal

return A[1,…,m]

else record diff

if twice pj

return Vj

Linearize:• Updates with their writes• Direct scans as before• Borrowed scans with source

direct scan

borrowedscan


Atomic Snapshot: Borrowed ScansInterference by process pj

And another one… pj does a scan inbeteween

Linearizing with the borrowed scan is OK.

write A[j]

read A[j]… …

read A[j]… …

embedded scan write A[j]

read A[j]… …

read A[j]… …


Alternative Routes for Developing Concurrent AlgorithmsRefinement: Implementing high-level ADT from

lower-level ADTs– Safety conditions, liveness properties

– Hierarchical

Transactional: Support for running sequential applications concurrently– Safety conditions, liveness properties


The Transactional Approach

[Herlihy, Moss, ISCA 1993]

Systematic approach for implementing concurrent data structures

Program sequentially But run concurrently Should appear to execute sequentially No high-level signature / semantics

– The sequential program is the specification


Transactional Synchronization

A transaction aggregates a sequence of resource accesses to be executed atomically– A sequence of atomic actions

– Like in database systems A transaction ends either

by committing– all its updates take effect

or by aborting– no update is effective

Read XWrite XRead ZRead Y

Read XWrite XRead ZRead Y


Safety: Serializability

[Papadimitriou 79][Weikum, Vossen, 2002, Chapter 3]

An analogue of sequential consistency Any interleaving of the transactions yields a

result that could be achieved in a sequential execution of the same set of transactions (a serialization)– Just the committed transactions?

(Aborted transactions have no effect.) Final state serializability


Safety: View Serializability

[Yannakakis 1984]

What about intermediate values read?– Could be “corrected” later– But still cause harm, e.g., division by 0

Any interleaving of the transactions has an equivalent sequential execution of the same transactions– Where all reads return the same value

Makes no sense in the context of implementing a high-level ADT– Where the internals are not exposed


Safety: Strict Serializability

[Papadimitriou 79][Bernstein, Shipman & Wong, 1979]

The serialization must preserve the real-time order of (non-overlapping) transactions An analogue of linearizability Called ordered serializabililty in W&V

Strict view serializability


Opacity

[Guerraoui & Kapalka, PPoPP 08]

Essentially, strict view serializability – Applied to all transactions

(not only committed or completed ones)

– Generalized to arbitrary object types (not just reads and writes)


Liveness Properties

As in high-level implementations But restricted to successful completions

(commit)

E.g., wait-freedom every transaction eventually commits,obsruction-freedom every transaction (eventually) running solo terminates


NonblockingObstruction-free

Obstruction-free

A Shift in Terminology

< 2003 ≥ 2003

Lock-free

Nonblocking

Wait-free

Lock-free

Wait-free


Where’s the Confusion?

[Herlihy & Wing, TOPLAS 1990]

[Herlihy, TOPLAS 1991]

[Herlihy, Luchangco, Moir, ICDCS 2003]


The Road Ahead

Concurrent algorithms pose a great challenge

It is “easy” to write – Correct algorithms and let

efficiency take care of itself

– Efficient algorithms and let correctness take care of itself

But very hard to write correct & efficient algorithms

Systematically…

A Mile-High View of Concurrent Algorithms Hagit Attiya Technion.

Documents

Transcript of A Mile-High View of Concurrent Algorithms Hagit Attiya Technion.