04/18/23 1PSU’s CS 587

16. Concurrency Control and Recovery(only for DBs with updates…..!)- Review Concurrency Control

Transaction ACID Isolation

• Schedules• Guaranteeing isolation

• Serializability• Serializability ⇔ Isolation• Locking• Strict Two Phase Locking• Strict 2PL ⇒Serializable

04/18/23 2PSU’s CS 587

Learning Objectives

Define ACID, schedule, isolated, equivalent, serializable, S2PL, conflict serializable, precedence graph, recoverable.

Know the implications on slide 25 and when the converses hold

Explain lock management, multiple granularity locks, phantoms, locking in BTrees, optimistic concurrency control

04/18/23 3PSU’s CS 587

Example Transaction

Transfer $100 from A to B Read A; Verify A; Write A-100; then Read B; Verify B; Write B+100

Are all 6 steps necessary? Which steps require disc access? When can an abort occur without

damage? Write is as in a program’s write

What damage can an abort cause? How can you avoid such damage?

04/18/23 4PSU’s CS 587

Transaction (cont.)

User (application developer) must indicate: Begin transaction read/write/modify statements intermixed

with other programming language statements

plus either commit - indicates successful completion or abort - indicates program wants to roll back

(erase the transaction) All or nothing! (Atomic)

04/18/23 5PSU’s CS 587

Supporting the ACIDACID Properties of Transactions

AAtomicity: All actions in a transaction happen intheir entirety or not at all.

CConsistency: If the DB starts in a consistent state, (this notion is defined by the user; some of it

may be enforced by integrity constraints) and if a transaction executes with no other queries active, then the DB ends up in a consistent state.

IIsolation: Each transaction is isolated from othertransactions. The effect on the DB is asif each transaction executed by itself.

DDurability: If a transaction commits, its changes to the database state persist.

RecoverySystem

RecoverySystem

ConcurrencyControlSystem

Programmers

04/18/23 6PSU’s CS 587

Isolation/Concurrency ControlT1: BEGIN A+=100, B-=100 END

T2: BEGIN A=1.06*A, B=1.06*B END

What is each of these transactions doing? A schedule of T1 and T2 is an interleaving of the steps of these transactions so that each transaction’s

order is preserved.

04/18/23 7PSU’s CS 587

Which of these is a Schedule of T1 and T2?

T1: A+=100, B-=100T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100 T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100 T2: B=1.06*B, A=1.06*A

T1: A+=100, B-=100 T2: A=1.06*A,B=1.06*B

04/18/23 8PSU’s CS 587

Isolated Schedules

A schedule is isolated if its effect on the DB is as if each transaction executed by itself, serially.

Which of the schedules on the next page is isolated?

Hint: Calculate the effect of the schedule on a sample state of the DB, for example A has $1,000, B has $500. This won’t tell you the effect on all states, but it’s helpful information.

04/18/23 9PSU’s CS 587

Which Schedules are Isolated?T1: A+=100, B-=100

T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100 T2: A=1.06*A, B=1.06*B

Goal of Concurrency Control subsystem: Guarantee only isolated schedules.

T1: A+=100, B-=100 T2: A=1.06*A,B=1.06*B

T1: A+=100,B-=100 T2: A=1.06*A, B=1.06*B

04/18/23 10PSU’s CS 587

Equivalent Schedules Two schedules are equivalent if given any

starting DB state, they produce the same result.

Which of these schedules is equivalent?

T1: A+=100, B-=100T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100 T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100 T2: A=1.06*A,B=1.06*B

04/18/23 11PSU’s CS 587

Serializable Schedules A schedule is serializable if it is

equivalent to a serial schedule. Which of these schedules is serializable?

T1: A+=100, B-=100T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100 T2: A=1.06*A, B=1.06*B

T1: A+=100, B-=100 T2: A=1.06*A,B=1.06*B

04/18/23 12PSU’s CS 587

The goal of Concurrency Control Recall the goal of concurrency control: To

ensure that all schedules are isolated

Theorem: A schedule is Serializable ⇔ it is Isolated ⇒: Serializable ⇒ equivalent to some serial schedule,

and in a serial schedule, each Xact. is isolated⇐: If each xact runs alone, the schedule must be

serial

But serializability is hard to verify

How can we, in real time, check each schedule?

So the Concurrency Control Subsystem needs more work.

04/18/23 13PSU’s CS 587

Locking

Transaction must get a lock – before it can read or update data

There are two kinds of locks: shared (S) locks and exclusive (X) locks

To read a record you MUST get an S lockTo modify or delete a record you MUST get an X lock

Lock info maintained by a “lock manager”

04/18/23 14PSU’s CS 587

How Locks Work

If a Xact has an S lock on a data object , new transactions can get S locks on that object, but not X locks.

If a Xact has an X lock, no other Xact can get any lock (S or X) on that data object.

If a transaction can’t get a lock, it waits (in a queue).

-- S X

--

S

X

ok

ok

ok no no

ok ok

ok no

Lock compatibility

lock on data item

lock

you

wan

t

04/18/23 15PSU’s CS 587

Strict Two Phase Locking Protocol (S2PL)

Strict 2PL is a way of managing locks during a transaction A Xact gets (S and X) locks gradually, as

needed The Xact holds all locks until end of transaction

(commit/abort)

time

# of locksheld by a

transaction T

All locksare releasedat the end, upon commit or abort

0

21

4

3

5

04/18/23 16PSU’s CS 587

Strict 2PL guarantees serializability

Idea of the Proof: a Strict 2PL schedule is equivalent to the serial schedule in which each transaction runs instantaneously at the time that it commits

This is huge: A property of each transaction (S2PL) implies a property of any set of transactions (serializability) No need to check serializability of any schedules

Real DBMSs use S2PL to enforce serializability

In reality, users can and do choose lower levels of concurrency for all but the most sensitive transactions

04/18/23 17PSU’s CS 587

17. Concurrency Control Conflicts

Conflicting Actions Conflict Equivalent Conflict Serializable Conf. Ser. ⇒

Serializable Precedence Graph

Conf. Serializable ⇔Precedence graph is acyclic

Strict 2PL ⇒Recoverable

2PL ¬⇒ Recoverable

Locks Management Deadlocks

• Waits-for

Multiple Granularity Phantoms

• Predicate, Index locking

Locking in B+ Trees Optimistic CC

• Inefficiency of locking• Optimistic CC idea

04/18/23 18PSU’s CS 587

17.1 Conflict Serializable Schedules

Conflicting actions: Actions that access the same data and at least one of which is a write Note that changing the order of these two actions

might yield different results. Two schedules are conflict equivalent if:

They involve the same actions of the same transactions in the same order

Every pair of conflicting actions is ordered the same way

Schedule S is conflict serializable if S is conflict equivalent to some serial schedule

17. CC

04/18/23 19PSU’s CS 587

Which are conflict serializable?

T1: R(A), W(A) T2: R(A), W(A), R(B)

T1: R(B), W(A), W(B) T2: R(A), W(A), R(B)

T1: R(A), W(A)T2: W(A)T3: W(A)

17. CC

T1: R(A),W(A), R(B),W(B) T2: R(A),W(A), R(B),W(B)

04/18/23 20PSU’s CS 587

Conflict Serializable ⇒ Serializable

If two actions do not conflict, then commuting them results in an equivalent schedule.

Suppose S is conflict serializable. Then there is a sequence of commuting actions I = {I1,…,In} so thata) Each of the Ii commutes nonconflicting actions

b) I applied to S is a serial schedule Because of (a), I does not change the state of

any database. Thus S, and I applied to S, are equivalent and I applied to S is serial (b), so S is serializable.

04/18/23 21PSU’s CS 587

Serializable does NOT imply Conflict Serializable

Equivalent to what serial schedule? Therefore it is a serializable schedule

Why is it not conflict serializable? (for now just give an intuitive reason, later we will have a proof)

T1: R(A), W(A)T2: W(A)T3: W(A)

04/18/23 22PSU’s CS 587

Precedence graphs

Why is this graph not conflict serializable?

The cycle in the graph illustrates the problem. T1 must precede T2, and T2 must precede T1, in any conflict equivalent serial schedule.

T1: R(A), W(A), R(B), W(B)T2: R(A), W(A), R(B), W(B)

T1 T2A

B

Precedence graph

17. CC

04/18/23 23PSU’s CS 587

Precedence Graph Precedence graph: One node per Xact;

edge from Ti to Tj if an action of Ti precedes and conflicts with an action of Tj.

Theorem: Schedule is conflict serializable if and only if its precedence graph is acyclic ⇒If there is a cycle in the graph, it cannot be

serializable (see previous page & generalize) ⇐ If the graph is acyclic, the schedule is

equivalent to a topologically sorted order of the actions.

17. CC

04/18/23 24PSU’s CS 587

Example of acyclic graph

Is this graph acyclic? What is a topological sort of it? Is a schedule, for which this is a precedence

graph, equivalent to a serial schedule? Can we move all actions of T4 to occur before T2,

without reversing conflicting actions? How about T1 before T4?

T1 T2

T4 T3

04/18/23 25PSU’s CS 587

SummaryEach Xact in a schedule is Isolated

Isolated Xact: same results as if it ran alone

The schedule is Serializable

Serializable Schedule: Same result as a serial schedule

The schedule is Conflict Serializable

Conflict Serializable : Conflict Equivalent to a Serializable Schedule

The Schedule’s Precedence Graph is Acyclic

The schedule is consistent with Strict 2PL.

Strict 2PL: There is a locking schedule where all locks are held until EOT

Deadlock is possible

Deadlock: There is a cycle in the Waitsfor graph.

04/18/23 26PSU’s CS 587

Strict 2PL⇒Recoverable

A schedule is recoverable if, during it, all transactions commit only after all transactions whose data they have read commit.

Why is recoverability desirable? Otherwise, T1 may read the data of T2 ( a so-called dirty read), then T1 commit, then T2 abort and roll back. Then T1 has read a value that does not exist.

04/18/23 27PSU’s CS 587

Two-Phase Locking (2PL) Two-Phase Locking Protocol

Each Xact must obtain a S (shared) lock on object before reading, and an X (exclusive) lock on object before writing.

A transaction can not request additional locks once it releases any locks.

2PL implies that all schedules have acyclic precedence graphs, so are serializable.

However, they are not recoverable, so Strict 2PL is used in practice.

17. CC

04/18/23 28PSU’s CS 587

Lock Management Lock and unlock requests are handled by the lock

manager Lock table entry:

IDs of transactions currently holding a lock Type of lock held (shared or exclusive) Pointer to queue of lock requests

• If there is an S lock on an object O and T1 requests an X lock, what happens? What if then T2 requests an S lock?

Locking and unlocking have to be atomic operations How is this enforced?

Lock upgrade: transaction that holds a shared lock can be upgraded to hold an exclusive lock if no one else has a shared lock.

17. CC

04/18/23 29PSU’s CS 587

New Lock

Type of lock?

Queue empty?

S

EnQ lock?

N Y

Grant S lock

N

Type of lock?

Y

S

EnQ

X

lock?

X

Grant X lock

N

EnQY

Managing a new lock (simplified)

04/18/23 30PSU’s CS 587

Managing a lock release (simplified)Release Lock

Other locks?Y Exit

N

DeQ xact from Q, give it a lock

What type of lock was it?

ExitX

Is there an S lock on top of the Q?

Y

S

N

04/18/23 31PSU’s CS 587

Deadlocks

Deadlock: Cycle of transactions waiting for locks to be released by each other.

Two ways of dealing with deadlocks: Deadlock prevention Deadlock detection

17. CC

04/18/23 32PSU’s CS 587

Deadlock Prevention Theory

Assign priorities based on timestamps. • Older transactions get higher priority.

Assume Ti wants a lock that Tj holds. Two policies are possible:

• Wait-Die: It Ti has higher priority, Ti waits for Tj; otherwise Ti aborts

• Wound-wait: If Ti has higher priority, Tj aborts; otherwise Ti waits

If a transaction re-starts, make sure it has its original timestamp

Practice: http://dev.mysql.com/doc/refman/5.0/en/innodb-deadlocks.html

17. CC

04/18/23 33PSU’s CS 587

Deadlock Detection

Create a waits-for graph: Nodes are transactions There is an edge from Ti to Tj if Ti is waiting

for Tj to release a lock Periodically check for cycles in the

waits-for graph Note that waits-for graph is opposite

direction of precedence graph.

17. CC

04/18/23 34PSU’s CS 587

Deadlock Detection (Continued)Example:

T1: S(A), R(A), S(B)T2: X(B),W(B) X(C)T3: S(C), R(C)

X(A)T4: X(B)

T1 T2

T4 T3

T1 T2

T4 T3

17. CC

04/18/23 35PSU’s CS 587

Multiple-Granularity Locks

Hard to decide what granularity to lock (tuples vs. pages vs. tables).

Shouldn’t have to decide! Data “containers” are nested:

Tuples

Tables

Pages

Database

contains

17. CC

04/18/23 36PSU’s CS 587

Solution: New Lock Modes, Protocol

Allow Xacts to lock at each level, but with a special protocol using new “intention” locks:

Before locking an item, Xact must set “intention locks” on all its ancestors.

IX(IS): Intend to X(S) lock a subset.

SIX: S & IX at the same time. Used to scan and update selected records.

-- IS IX

--

IS

IX

S X

S

X

17. CC

04/18/23 37PSU’s CS 587

Multiple Granularity Lock Protocol Each Xact starts from the root of the

hierarchy. To get S or IS lock on a node, must hold IS

or IX on parent node. To get X or IX or SIX on a node, must hold

IX or SIX on parent node. Must release locks in bottom-up order. Sometimes hard to decide granularity of

locks. Can start small and use lock escalation.

17. CC

04/18/23 38PSU’s CS 587

Examples

T1 scans R, and updates a few tuples: T1 gets an SIX lock on R, then repeatedly

gets an S lock on tuples of R, and occasionally upgrades to X on the tuples.

T2 uses an index to read only part of R: T2 gets an IS lock on R, and repeatedly

gets an S lock on tuples of R. T3 reads all of R:

T3 gets an S lock on R. OR, T3 could behave like T2; can

use lock escalation to decide which.

-- IS IX

--

IS

IX

S X

S

X

17. CC

04/18/23 39PSU’s CS 587

Dynamic Databases: Phantoms

If we allow updates, even Strict 2PL will not assure serializability: T1 finds oldest sailor in each rank T2 inserts(Rohi,1,27) and deletes John Schedule is

This schedule is Strict 2PL, but not serializable! Result of this schedule is (1,Pehr)(2,Lorr) Result of T1;T2 is (1,Pehr)(2,John) Result of T2;T1 is (1,Rohi)(2,Lorr)

17. CC

Name

Rank

Age

Pehr 1 25

John 2 26

Lorr 2 23

T1 rank 1 T1 rank 2

T2 inserts Rohi, deletes John

04/18/23 40PSU’s CS 587

The Problem

When T1 retrieved the oldest sailor of rank 1, it locked each sailor of rank 1 with a read lock.

None of these locks applied to the new record (a phantom) inserted by T2.

We need a mechanism to prevent phantoms; to allow T1 to lock present and future sailors with rank 1.

There are two such mechanisms, index locking and predicate locking.

17. CC

04/18/23 41PSU’s CS 587

Index Locking

If there is a dense index on the rating field using Alternative (2), T1 should lock the index page(s) containing the data entries with rating = 1. If there are no records with rating = 1, T1 must

lock the index page where such a data entry would be, if it existed!

If there is no suitable index, T1 must lock all pages, and lock the file/table to prevent new pages from being added, to ensure that no new records with rating = 1 are added.

r=1

Data

Index

04/18/23 42PSU’s CS 587

Predicate Locking

Grant lock on all records that satisfy some logical predicate, e.g. age > 2*salary.

Index locking is a special case of predicate locking for which an index supports efficient implementation of the predicate lock. What is the predicate in the sailor example?

In general, predicate locking has a lot of locking overhead.

17. CC

04/18/23 43PSU’s CS 587

Locking in B+ Trees

How can we efficiently lock a B+ tree? Btw, don’t confuse this with multiple

granularity locking! One solution: Ignore the tree structure,

just lock pages while traversing the tree, following 2PL.

This has terrible performance! Root node (and many higher level nodes)

become bottlenecks because every tree access begins at the root. This single threads all updates to the tree.

17. CC

04/18/23 44PSU’s CS 587

Two Useful Observations

Higher levels of the tree only direct searches for leaf pages.

For inserts/deletes, a node on a path from root to modified leaf must be locked (in X mode, of course), only if a split can propagate up to it from the modified leaf.

We can exploit these observations to design efficient locking protocols that guarantee serializability even though they violate 2PL.

17. CC

04/18/23 45PSU’s CS 587

A Tree Locking Algorithm

Search: Start at root and go down; repeatedly, S lock child then unlock parent.

Insert/Delete: Start at root and go down, obtaining X locks as needed. Once child is locked, check if it is safe: If child is safe, release all locks on ancestors.

Safe node: Node such that the change will not propagate up beyond this node. Inserts: Node is not full. Deletes: Node is not half-empty.

17. CC

04/18/23 46PSU’s CS 587

Example

ROOT

A

B

C

D E

F

G H I

35

20*

38 44

22* 23* 24* 35* 36* 38* 41* 44*

Do:1) Search 38*2) Delete 38*3) Insert 45*4) Insert 25*

23

20

35

3523 3538 44

04/18/23 47PSU’s CS 587

Optimistic CC (Kung-Robinson) Locking is a conservative approach in which

conflicts are prevented. Disadvantages: Lock management overhead. Deadlock detection/resolution. Lock contention for heavily used objects.

If conflicts are rare, we might be able to gain concurrency by not locking, and instead checking for conflicts before Xacts commit.

A version of this optimistic approach is used by PostgreSQL and Oracle

17. CC

04/18/23 48PSU’s CS 587

Kung-Robinson Model

Xacts have three phases: READ: Xacts read from the database,

but make changes to private copies of objects.

VALIDATE: Check for conflicts. WRITE: Make local copies of changes

public.ROOT

old

new

modifiedobjects

17. CC

04/18/23 49PSU’s CS 587

Validation

Each Xact is assigned a numeric id. Just use a timestamp.

Xact ids assigned at end of READ phase, just before validation begins.

ReadSet(Ti): Set of objects read by Xact Ti.

WriteSet(Ti): Set of objects modified by Ti.

17. CC

04/18/23 50PSU’s CS 587

Test 1

For all i and j such that TSi < TSj, check that Ti completes before Tj begins.

Ti

TjR V W

R V W

17. CC

04/18/23 51PSU’s CS 587

Test 2

For all i and j such that Ti < Tj, check that: Ti completes before Tj begins its Write phase + WriteSet(Ti) ReadSet(Tj) is empty.

Ti

TjR V W

R V W

Does Tj read dirty data? Does Ti overwrite Tj’s writes?

17. CC

04/18/23 52PSU’s CS 587

Test 3 For all i and j such that Ti < Tj, check that:

Ti completes Read phase before Tj does + WriteSet(Ti) ReadSet(Tj) is empty + WriteSet(Ti) WriteSet(Tj) is empty.

Tj

TiR V W

R V W

Does Tj read dirty data? Does Ti overwrite Tj’s writes?

17. CC

04/18/23 53PSU’s CS 587

Overheads in Optimistic CC

Must record read/write activity in ReadSet and WriteSet per Xact. Must create and destroy these sets as needed.

Must check for conflicts during validation, and must make validated writes ``global’’. Critical section can reduce concurrency. Scheme for making writes global can reduce

clustering of objects. Optimistic CC restarts Xacts that fail

validation. Work done so far is wasted; requires clean-up.

17. CC