sequential verificationfor serializability

H. Attiya

Queen Mary

POPL’10 Madrid, Spain 20-January-2010

G. Ramalingam N. Rinetzky

Technion MSR India

Monday, 12 April 2010

• goal verify concurrent data structures

• problem interleaving

• solution ignore problem

Monday, 12 April 2010

• goal verify concurrent data structures

• problem interleaving

• solution ignore problem

1. verify assuming single-threaded contexts

2. conclude results hold in multithread contexts

Monday, 12 April 2010

• goal verify concurrent data structures

• problem interleaving

• solution ignore problem

1. verify assuming single-threaded contexts

2. conclude results hold in multithread contexts

Monday, 12 April 2010

main idea

• sequential verification for serializability

• use serilizability for sequential verification

• memory safety, assertions, data invariants, ...

⇒data structure “behaves well”

when used by a single thread

any concurrent execution of the data structure operations is

serializable

Monday, 12 April 2010

this talk

• sequential verification for serializability

⇒data structure “behaves well”

when used by a single thread

any concurrent execution of the data structure operations is

serializable

Monday, 12 April 2010

preliminaries

• executions

• concurrent, sequential, non-interleaved

• completability

• conflict-serializability (CS)

• CS-locking protocols

Monday, 12 April 2010

executions

complete operation operation

operation

|| ||(concurrent) execution

complete operation operation complete operation

; ;sequential execution

add s and e

Monday, 12 April 2010

non-interleaved executions

• a thread is given only one chance to run

• several operations may not complete

• total order between operations complete operation operation operation

|; |;non-interleaved execution

Monday, 12 April 2010

non-interleaved executions

• a thread is given only one chance to run

• several operations may not complete

• total order between operations complete operation operation operation

|; |;non-interleaved execution

Monday, 12 April 2010

non-interleaved executions

• a thread is given only one chance to run

• several operations may not complete

• total order between operations complete operation operation operation

|; |;non-interleaved execution

complete operation operation complete operation

; ;sequential execution

Monday, 12 April 2010

completability

• termination => completability

Monday, 12 April 2010

conflict-serializability (CS)

complete operation operation

operation

read Ywrite Y|| ||

write Xwrite X

conflict conflict

Monday, 12 April 2010

conflict-serializability (CS)

complete operation operation

operation

read Ywrite Y|| ||

write Xwrite X

conflict conflict

complete operation operation operation

|; |;non-interleaved execution

Monday, 12 April 2010

conflict-serializability (CS)

complete operation operation

operation

read Ywrite Y|| ||

write Xwrite X

conflict conflict

complete operation operation operation

|; |;non-interleaved execution

Monday, 12 April 2010

conflict-serializability (CS)

complete operation operation

operation

read Ywrite Y|| ||

write Xwrite X

conflict conflict

complete operation operation operation

non-interleaved execution

~~~ conflict

|; |;

Monday, 12 April 2010

• CS-locking protocols

• common programming discipline to obtain conflict-serializability

• lock-protected accesses

• special locking patterns

• ...

CS in practice

Monday, 12 April 2010

CS-locking protocols• common programming discipline to obtain

conflict-serializability

• two-phase locking [Papadimitiriou ’79]

• tree (hand-over-hand) locking [Kedem et al. ’81]

• dag locking [Chaudhri et al. ’95]

⊨concurrent execution

locking protocol

concurrent execution:

⇒ ⊨ CSconcurrent execution

∀

Monday, 12 April 2010

our contributions

Monday, 12 April 2010

sequential reductions for CS-locking protocols

• two-phase locking

• tree (hand-over-hand) locking

• dag locking

• progressive local CS-locking protocols

⇒∀⊨sequential execution∀ ⊨concurrent

executionlocking

protocol

locking protocol

completable∧

Monday, 12 April 2010

sequential verificationfor conflict-serializability

⇒∀⊨sequential execution∀ ⊨concurrent

executionlocking

protocol

⊨concurrent execution

locking protocol

concurrent execution:

⇒ ⊨ CSconcurrent execution

∀

locking protocol

completable∧

Monday, 12 April 2010

sequential verificationfor conflict-serializability

∀ ⊨concurrent execution

locking protocol

⇒∀⊨concurrent execution

locking protocol∀ ⊨concurrent

execution CS

⇒⇒⊨sequential execution∀

locking protocol

completable∧

Monday, 12 April 2010

⇒sequential verificationfor conflict-serializability

∀ ⊨concurrent execution

locking protocol

∀ ⊨concurrent execution CS

⇒⊨sequential execution∀

locking protocol

completable∧

Monday, 12 April 2010

⇒ ⇒sequential verificationfor conflict-serializability

∀ ⊨concurrent execution

locking protocol

∀ ⊨concurrent execution CS

⇒⊨sequential

execution∀ locking protocol

finite sequence of operations terminates

∧

⊨sequential execution∀

locking protocol

completable∧

∀

Monday, 12 April 2010

example

• dynamic tree (hand-over-hand) locking

• heap-allocated tree-like data structures

• hand-over-hand traversal

• lock object before using its fields

• lock child before unlock parent

• never lock object twice

• ...

Monday, 12 April 2010

1. verify hand-over-hand list implementation using the most general single-thread client

1. correctly follows the protocol

2. every call terminates ⇒ completability

2. apply sequential reduction theorem

3. conclude hand-over-hand list correctly follows the protocol

verification outline

Monday, 12 April 2010

bool find(int v){ ACQUIRE H; if (H != null){ Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null){ acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

hand-over-hand locking

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

bool b = false; if (c & c.d==v) b = true;

release(p); if (c) release(c); return b; } else { ...

insert(int v){ ACQUIRE H; if (H != null){ Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null){ acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

if (c & c.d<v){ Node x = alloc Node; p.n = x; x.d = v; x.n = c; }

release(p); if (c) release(c); } else { ...

Type Node { int d; Node n;}Node H = null;

remove(int v){ ACQUIRE H; if (H != null){ Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null){ acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

: remove(18)

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n n

p c y

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n n

p c y

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

hand-over-hand locking

-4 7 18 25 33-9Hn n n

p c y

n

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

: remove(18)

Monday, 12 April 2010

insert(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

find(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

sequential verificationof hand-over-hand-locking

Monday, 12 April 2010

sequential shape reasoning

if (c & c.d==v){ Node y = c.n; c.n = null; p.n = y; }

release(p); if (c) release(c); } else { ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c);

while (c & c.d<v){ release(p); p = c; c = p.n; acquire(c); }

-4 7 18 25 33-9Hn n n n n

p c

Monday, 12 April 2010

✓ ✓✓insert(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

find(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

∀ sequential execution ⊨hand-over-hand locking

Monday, 12 April 2010

✓ ✓✓insert(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

find(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

sequential termination

-4 7 18 25 33-9Hn n n

p c y

nMonday, 12 April 2010

ACNI-reduction

⇒⇒

⊨LP∀ complete operation operation operation

sequential reductionfor hand-over-hand locking

⊨∀ complete operation operation complete operation

NI-reduction

complete operation operation

operation

∀ ⊨

|; |;

; ;

|| || LP

LPsequential

termination+

Monday, 12 April 2010

✓ ✓✓insert(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

remove(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

find(int v) { ACQUIRE H; if (H != null) { Node p = H; acquire(p); RELEASE H; Node c = p.n; if (c != null) { acquire(c); ...

∀ concurrent execution ⊨hand-over-hand-locking

Monday, 12 April 2010

“ignoring interleavings”

-4 7 18 25 33-9Hn n

p c y

n

p cp

|| || ||...

Monday, 12 April 2010

sequential reductionfor CS-locking protocols

• NI-reduction

• ACNI-reduction

⇒∀⊨sequential execution

locking protocol∀ ⊨concurrent

executionlocking

protocolcompletable

∧

Monday, 12 April 2010

NI-reduction

LP∀ complete operation operation operation

complete operation operation

operation

∀ ⊨LP

|; |;

|| ||

⊨⇒

Monday, 12 April 2010

complete operation

ACNI-reduction

complete operation operation operation

|; |;

; ;

complete operation operation

LP∀

∀ ⊨LP

⊨⇒

Monday, 12 April 2010

non-interleaved execution ⊭ LP

int Y = 0;

setY(){ ACQUIRE Y; Y = 1; RELEASE Y; while (true) skip;}

|; ACQUIRE Y; Y=1; RELEASE Y; ACQUIRE Y; y=Y; RELEASE Y; violate LP

getY(){ ACQUIRE Y; int y = Y; RELEASE Y; if (y == 1)

violate LP}

Monday, 12 April 2010

∀ sequential execution ⊨ LP

int Y = 0;

setY(){ ACQUIRE Y; Y = 1; RELEASE Y; while (true) skip;}

getY(){ ACQUIRE Y; int y = Y; RELEASE Y; if (y == 1)

violate LP}

ACQUIRE Y; y=Y; RELEASE Y; violate LP|; ACQUIRE Y; Y=1; RELEASE Y; skip; skip; skip; ...

Monday, 12 April 2010

execution not completable

int Y = 0;

setY(){ ACQUIRE Y; Y = 1; RELEASE Y; while (true) skip;}

getY(){ ACQUIRE Y; int y = Y; RELEASE Y; if (y == 1)

violate LP}

ACQUIRE Y; Y=1; RELEASE Y; skip; skip; skip; ...

Monday, 12 April 2010

completability weaker than sequential termination

int Y = 0;

setY(){ ACQUIRE Y; Y = 1; RELEASE Y; while (*) skip;}

getY(){ ACQUIRE Y; int y = Y; RELEASE Y; if (y == 1)

violate LP}

ACQUIRE Y; Y=1; RELEASE Y; skip; skip; skip; ...

Monday, 12 April 2010

int Y = 0;

setY(){ ACQUIRE Y; Y = 1; RELEASE Y; while (*) skip;}

getY(){ ACQUIRE Y; int y = Y; RELEASE Y; if (y == 1)

violate LP}

ACQUIRE Y; Y=1; RELEASE Y; skip; skip; skip; ...

completability weaker than sequential termination

skip; skip

complete operation

Monday, 12 April 2010

complete operation

ACNI-reduction

complete operation operation operation

|; |;

; ;

complete operation operation

LP∀

∀ ⊨LP

⊨⇒completable

∧

Monday, 12 April 2010

ACNI-reduction

⇒⇒

⊨LP∀ complete operation operation operation

sequential reductionfor CS-locking protocols

⊨LP∀

complete operation operation complete operation

NI-reduction

complete operation operation

operation

∀ ⊨LP

|; |;

; ;

|| ||

completable∧

Monday, 12 April 2010

shape analysis

• TVLA [Sagiv et al., TOPLAS’02][Lev-Ami, Sagiv, SAS’00]

• hand-over-hand locking

• termination

singly linked list 4 sec. 4 MB

binary tree 125 sec. 91 MB

Monday, 12 April 2010

main results• sequential reductions for CS-locking

protocols

• local CS-locking protocols

• progressive local CS-locking protocols

• sequential analyses for serializability

• verifiable properties and techniques

• shape analysis adapting TVLA ‘s existing shape analyses of sequential lists & trees

Monday, 12 April 2010

thank you!

ignore your problemsand they (sometimes) go away ...

Monday, 12 April 2010