19 Concurrency & Recovery

1

CS 338: Computer Applications in Business: Databases (Fall 2014)

1992-2014 by Addison Wesley & Pearson Education, Inc., McGraw Hill, Cengage Learning Slides adapted and modified from Fundamentals of Database Systems (5/6) (Elmasri et al.), Database System Concepts (5/6) (Silberschatz et al.), Database Systems (Coronel et al.), Database Systems (4/5) (Connolly et al. ), Database Systems: Complete Book (Garcia-Molina et al.)

Concurrency Control & Recovery Sections 22.1, 23.1-23.2

CS 338: Computer Applications in Business: Databases 1992-2014 by Addison Wesley & Pearson Education, Inc., McGraw Hill, Cengage Learning Slides adapted and modified from Fundamentals of Database Systems (5/6) (Elmasri et al.), Database System Concepts (5/6) (Silberschatz et al.), Database Systems (Coronel et al.), Database Systems (4/5) (Connolly et al. ), Database Systems: Complete Book (Garcia-Molina et al.) Rice University Data Center

Fall 2014

Recall: Transaction States

Active state

Partially committed state

Committed state

Failed state

Terminated State

Recovery Manager keeps track of operations

begin_transaction, read or write, end_transaction

It is important to check whether changes introduced by transaction can be permanently applied or not

commit_transaction: signals successful end of transaction (any changes can safely be committed to database and will not be undone)

rollback (or abort): signals unsuccessful end of transaction (any changes that transaction may have applied to database must be undone)

2

How to keep track of all operations?

2



Recovery Using Log Records

The System Log

Log or Journal: The log keeps track of all transaction operations that affect the values of database items.

This information may be needed to permit recovery from transaction failures.

The log is kept on disk, so it is not affected by any type of failure except for disk or catastrophic failure.

In addition, the log is periodically backed up to archival storage (tape) to guard against such catastrophic failures.

If the system crashes, we can recover to a consistent database state by examining the log

Since the log contains record of every write operation of a transaction, it is possible to undo the effect of these write operations by tracing backward through the log and resetting items changed by a write operation of T to their old_values

Similarly, it is possible to redo the effect of the write operations by tracing forward through the log and setting all items changed by a write operations to their new_values

3

Recovery Using Log Records

The System Log (cont):

T in the following discussion refers to a unique transaction-id that is generated automatically by the system and is used to identify each transaction:

Types of log record:

[start_transaction,T]: Records that transaction T has started execution.

[write_item,T,X,old_value,new_value]: Records that transaction T has changed the value of database item X from old_value to new_value.

[read_item,T,X]: Records that transaction T has read the value of database item X.

[commit,T]: Records that transaction T has completed successfully, and affirms that its effect can be committed (recorded permanently) to the database.

[abort,T]: Records that transaction T has been aborted.

4

3



Commit versus Rollback?

Commit Point

A transaction T reaches its commit point when all its operations that access the database have been executed successfully and the effect of all the transaction operations on the database have been recorded in the log

Beyond the commit point, the transaction is said to be committed, and its effect is assumed to be permanently recorded in the database.

Roll Back

Needed for transactions that have a [start_transaction, T] entry into the log but no commit entry [commit, T] into the log

5

Transaction Schedule

A schedule (or history) S of n transactions T1, T2, , Tn is an ordering of the operations of the transactions which can be interleaved in the schedule S

For each transaction Ti that participates in S, the operations of T1 in S must appear in the same order in which they occur in T1. However, that operations from other transactions Tj can be interleaved with the operations of Ti in S.

6

4




For purpose of recovery and concurrency control, we are mainly interested in the read_item and write_item operations of the transactions We also consider the commit and abort operations

A shorthand notation for describing a schedule uses the symbols:

b begin_transaction

r read_item

w write_item

e end_transaction

c commit

a abort 7

appends a subscript the transaction id to each operation in the schedule +

Example: Sa: r1(X); r2(X); w1(X); r1(Y); w2(X); w1(Y)


Two operations in a schedule are said to conflict if they satisfy all three of the following conditions:

1. they belong to different transactions 2. they access the same item X 3. at least one of the operations is a write_item(X)

8

Example: Sa: r1(X); r2(X); w1(X); r1(Y); w2(X); w1(Y)

Conflicts:

r1(X) and w2(X) r2(X) and w1(X) w1(X) and w2(X)

5



Schedule Serializability

Serial Schedule

A schedule S is serial if, for every transaction T participating in the schedule, all the operations of T are executed consecutively in the schedule.

Otherwise, the schedule is called nonserial schedule.

9

Problem with serial schedules is that they limit concurrency by prohibiting interleaving of operations


Serial schedules are considered unacceptable in practice

However, some nonserial schedules give the correct expected result

We would like to determine which of the nonserial schedules always give a correct result and which may give erroneous results serializability of a schedule

Serializable schedule: A schedule S of n transactions is serializable if it is equivalent to some serial schedule of the same n transactions

When are two schedules considered equivalent? 10

6




Equivalent Schedules

Two schedules are called result equivalent if they produce the same final state of the database

Notion of conflict equivalence: two schedules are said to be conflict equivalent if the order of any two conflicting operations is the same in both schedules

11

Example

X=100

X=110

X=100

X=100+10 X=110 X=100 * 1.1

X=500

X=550 X=500 * 1.1

X=500

X=510 X=500+10

Two schedules that are result equivalent for the initial value X=100 but are not result equivalent in general


Conflict Equivalent

We define a schedule S to be conflict serializable if it is (conflict) equivalent to some serial schedule S

In such case, we can reorder the nonconflicting operations in S until we form the equivalent serial schedule S

12

Example

T1 T2

--- ---

--- ---

write_item(X) ---

--- read_item(X)

T1 T2

write_item(X) ---

--- read_item(X)

--- ---

--- ---

S1 S2

7




13

conflict serializable schedule

Testing for Conflict Serializability

Serialization Graph (precedence graph)

a directed graph G=(N,E) that consists of a set of nodes N = {T1, T2, , Tn} and a set of directed edges E = {e1, e2, , em}

14

8



Serialization Graph

15

Serializable Schedule (no cycles) Equivalent serial schedules: None

Serialization Graph

16

Non Serializable (cycle found): Equivalent serial schedules: T3 T1 T2

9



Concurrency Control

17

Why Concurrency Control is Needed

The Lost Update Problem

This occurs when two transactions that access the same database items

have their operations interleaved in a way that makes the value of some

database item incorrect.

18

DB Values T1 T2

X = 80

read_item(X); X = 80

X := X 5; X = 75


X := X + 10; X = 90

X = 75 write_item(X);


10




The Temporary Update (or Dirty Read) Problem

This occurs when one transaction updates a database item and then the

transaction fails for some reason

The updated item is accessed by another transaction before it is changed

back to its original value.

19

DB Values T1 T2

X = 80


X := X 5; X = 75



X := X + 10; X = 85

X := X / 0; T1 aborts



The Incorrect Summary Problem

If one transaction is calculating an aggregate summary function on a number

of records while other transactions are updating some of these records, the

aggregate function may calculate some values before they are updated and

others after they are updated.

20

DB Values T1 T2

X =

read_item(X1); X1 = 80

SUM := X1; SUM = 80


SUM := SUM+X2; SUM = 95


X1 := X1 + 5; X1 = 85

X = write_item(X1);


X3 := X3 + 5; X3 = 30

X = write_item(X3);

read_item(X3); X3= 30

SUM := SUM+X3; SUM = 125

11



Maintaining Data Consistency

When executing several transactions concurrently in the database, the consistency of the data may no longer be maintained

It is necessary for the system to control the interaction among the concurrent transactions

This is achieved through one of a variety of mechanisms called concurrency-control schemes

One commonly used scheme is the two-phase locking

21

Locking Protocols

One way to ensure isolation is to require that data items be accessed in a mutually exclusive manner

While one transaction is accessing a data item, no other transaction can modify that data item

Most common method used to implement this requirement is to allow a transaction to access a data item only if it is currently holding a lock on that item

Two modes for locking a data item Shared (read): if a transaction Ti has obtained a shared-

mode lock (denoted by S) on item Q, then Ti can read, but cannot write Q

Exclusive (write): if a transaction Ti has obtained an exclusive-mode lock (denoted by X) on item Q, then Ti can both read and write Q

22

12



Locking Protocols

Lock requests are sent to the concurrency control manager

A transaction can proceed with the operation only after the concurrency-control manager grants the lock to the transaction

The use of these two lock modes allows multiple transactions to read a data item but limits write access to just one transaction at a time

Lock-compatibility matrix

23

S X

S true false

X false false

What if a lock cannot be granted after a request is made?

Requesting transaction is made to wait until all incompatible locks held by

other transactions have been released

Pitfalls of Lock-Based Protocols Risks: Deadlocks

Although locking protocols provides to some good extent a level of serializability and consistency but has some risks

Deadlock: An impasse that may result when two (or more) transactions are each waiting for locks to be released that are held by the other

24

To handle a deadlock, one of the two transactions T1 or T2 must be rolled

back and its locks released

13



Pitfalls of Lock-Based Protocols Risks: Starvation

Starvation: selection of victims is based primarily on cost factors, it may happen that the same transaction is always picked as a victim

As a result, this transaction never completes its designated task, thus there is starvation

Possible solution: include number of rollbacks in the cost factor

Example: a transaction waiting for an exclusive-lock on a data item while there is a sequence of other transactions that are granted a shared-lock to the same data item

The transaction waiting for exclusive-lock is repeatedly rolled back de to deadlocks

Such transaction that is rolled back never completes (starvation)

25

Lock-based Protocols Two-Phase Locking Protocols

This is a protocol which ensures conflict-serializable

schedules.

Phase 1: Growing Phase

transaction may obtain locks

transaction may not release locks

Phase 2: Shrinking Phase

transaction may release locks

transaction may not obtain locks

26

14



Lock-based Protocols Two-Phase Locking Protocols

Two-phase locking does not ensure freedom from deadlocks

The problem of cascading roll-back is possible under two-phase locking.

To avoid this, follow a modified protocol can be used:

Strict two-phase locking. Here a transaction must hold all its exclusive locks till it commits/aborts.

Rigorous two-phase locking is even stricter: here all locks are held till commit/abort. In this protocol transactions can be serialized in the order in which they commit.

27

Lock Conversions

Two-phase locking with lock conversions:

First Phase:

can acquire a lock-shared on item

can acquire a lock-exclusive on item

can convert a lock-shared to a lock-exclusive (upgrade)

Second Phase:

can release a lock-shared

can release a lock-exclusive

can convert a lock-exclusive to a lock-shared (downgrade)

This protocol assures serializability. But still relies on the

programmer to insert the various locking instructions.

28

15



Implementation of Locking

A lock manager can be implemented as a separate process to which transactions send lock and unlock requests

The lock manager replies to a lock request by sending a lock grant messages (or a message asking the transaction to roll back, in case of a deadlock)

The requesting transaction waits until its request is answered

The lock manager maintains a data-structure called a lock table to record granted locks and pending requests

The lock table is usually implemented as an in-memory hash table indexed on the name of the data item being locked

29

Lock Table

30

Black rectangles indicate granted locks, white ones indicate waiting requests

Lock table also records the type of lock granted or requested

New request is added to the end of the queue of requests for the data item, and granted if it is compatible with all earlier locks

Unlock requests result in the request being deleted, and later requests are checked to see if they can now be granted

If transaction aborts, all waiting or granted requests of the transaction are deleted lock manager may keep a list of locks

held by each transaction, to implement this efficiently

16



Recovery

31

Database Recovery

Purpose of Database Recovery

To bring the database into the last consistent state, which existed prior to the failure.

To preserve transaction properties (Atomicity, Consistency, Isolation and Durability).

Example:

If the system crashes before a fund transfer transaction completes its execution, then either one or both accounts may have incorrect value. Thus, the database must be restored to the state before the transaction modified any of the accounts.

32

17



Database Recovery Failure Classification

The database may become unavailable for use due to

Transaction failure: Transactions may fail because of incorrect input, deadlock, incorrect synchronization.

System failure: System may fail because of addressing error, application error, operating system fault, RAM failure, etc.

Media failure: Disk head crash, power disruption, etc.

33

Database Recovery Transaction Log

For recovery from any type of failure data values prior to modification (BFIM - BeFore IMage) and the new value after modification (AFIM AFter IMage) are required.

These values and other information is stored in a sequential file called Transaction log. A sample log is given below. Back P and Next P point to the previous and next log records of the same transaction.

34

T ID Back P Next P Operation Data item BFIM AFIM

T1 0 1

T1 1 4

T2 0 8

T1 2 5

T1 4 7

T3 0 9

T1 5 nil

Begin

Write

W

R

R

End

BeginX

Y

M

N

X = 200

Y = 100

M = 200

N = 400

X = 100

Y = 50

M = 200

N = 400

18



Database Recovery Data Update

Immediate Update: As soon as a data item is modified in cache, the disk copy is updated.

Deferred Update: All modified data items in the cache is written either after a transaction ends its execution or after a fixed number of transactions have completed their execution.

Shadow Update: The modified version of a data item does not overwrite its disk copy but is written at a separate disk location.

In-place Update: The disk version of the data item is overwritten by the cache version.

35

Database Recovery Data Caching

Data items to be modified are first stored into database cache by the Cache Manager (CM) and after modification they are flushed (written) to the disk.

The flushing is controlled by Modified and Pin-Unpin bits.

Pin-Unpin: Instructs the operating system not to flush the data item.

Modified: Indicates the AFIM of the data item.

36

19



Database Recovery Transaction Roll-back (Undo) and Roll-Forward (Redo)

To maintain atomicity, a transactions operations are redone or undone.

Undo: Restore all BFIMs on to disk (Remove all AFIMs).

Redo: Restore all AFIMs on to disk.

Database recovery is achieved either by performing only Undos or only Redos or by a combination of the two. These operations are recorded in the log as they happen.

37

Database Recovery Write-Ahead Logging (WAL)

When in-place update (immediate or deferred) is used then log is necessary for recovery and it must be available to recovery manager. This is achieved by Write-Ahead Logging (WAL) protocol. WAL states that

For Undo: Before a data items AFIM is flushed to the database disk (overwriting the BFIM) its BFIM must be written to the log and the log must be saved on a stable store (log disk).

For Redo: Before a transaction executes its commit operation, all its AFIMs must be written to the log and the log must be saved on a stable store.

38

20



Database Recovery Checkpointing

Time to time (randomly or under some criteria) the database flushes its buffer to database disk to minimize the task of recovery. The following steps defines a checkpoint operation:

1. Suspend execution of transactions temporarily.

2. Force write modified buffer data to disk.

3. Write a [checkpoint] record to the log, save the log to disk.

4. Resume normal transaction execution.

During recovery redo or undo is required to transactions appearing after [checkpoint] record.

39

Database Recovery Steal/No-Steal and Force/No-Force

Possible ways for flushing database cache to database

disk: 1. Steal: Cache can be flushed before transaction commits.

2. No-Steal: Cache cannot be flushed before transaction commit.

3. Force: Cache is immediately flushed (forced) to disk.

4. No-Force: Cache is deferred until transaction commits

These give rise to four different ways for handling

recovery: Steal/No-Force (Undo/Redo)

Steal/Force (Undo/No-redo)

No-Steal/No-Force (Redo/No-undo)

No-Steal/Force (No-undo/No-redo) 40

21



Recovery Scheme Deferred Update (No Undo/Redo)

The data update goes as follows:

A set of transactions records their updates in the

log.

At commit point under WAL scheme these

updates are saved on database disk.

After reboot from a failure the log is used to redo

all the transactions affected by this failure. No

undo is required because no AFIM is flushed to

the disk before a transaction commits.

41

19 Concurrency & Recovery

Documents

Transcript of 19 Concurrency & Recovery