1

CSE 480: Database Systems

Lecture 23: Transaction Processing and Database Recovery

2

Online Transaction Processing Systems

Systems that need real-time support for querying and updating of databases by one or more concurrent users

Examples of OLTP:– Banking & Credit card transaction processing systems– Airline/Railway reservation systems– Trading/Brokerage systems– Online E-commerce (Amazon, Walmart, etc)

What makes the requirements for OLTP different than other

systems?– Database gets updated in real time frequently, but it must always

maintain correctness of the database state (in spite of failures and concurrent access)

3

Motivating Example 1

Transfer $100 from one bank account to another

time

Balance (Account 1)

Balance (Account 2)

Operation

$200 $100

4

Motivating Example 1

Transfer $100 from one bank account to another

time

Balance (Account 1)

Balance (Account 2)

Operation

1. Check balance of Account 1$200 $100

5

Motivating Example 1

Transfer $100 from one bank account to another

time

Balance (Account 1)

Balance (Account 2)

Operation

1. Check balance of Account 1

2. Reduce balance of Account 1 by $100

$100 $100

6

Motivating Example 1

Transfer $100 from one bank account to another

time

Balance (Account 1)

Balance (Account 2)

Operation

1. Check balance of Account 1

2. Reduce balance of Account 1 by $100

3. Check balance of Account 2

$100 $100

7

Motivating Example 1

Transfer $100 from one bank account to another

time

Balance (Account 1)

Balance (Account 2)

Operation

1. Check balance of Account 1

2. Reduce balance of Account 1 by $100

3. Check balance of Account 2

4. Increase balance of Account 2 by $100

$100 $200

Require 4 database operations

8

Motivating Example 1

Transfer $100 from one bank account to another

time

Balance (Account 1)

Balance (Account 2)

Operation

1. Check balance of Account 1

2. Reduce balance of Account 1 by $100

3. Check balance of Account 2

4. Increase balance of Account 2 by $100

$100 $100

System crash (write

operation fails)

Database is in an inconsistent state after system failure!

9

Motivating Example 2

Two students registering for the same class

Student Enrollment Database

NumEnrolled: 39

MaxEnrolled: 40

Read

Num:39Max: 40

Read

Num:39Max: 40

10

Motivating Example 2

Two students registering for a class

Student Enrollment Database

NumEnrolled: 40

MaxEnrolled: 40

Register RegisterNumEnrolled: 41

MaxEnrolled: 40

Database is in an inconsistent state (violate semantic constraint) when processing requests from multiple concurrent users!

11

Challenges of OLTP

Although your SQL code is written correctly, the database may still be in an inconsistent state after processing transactions due to

– System failures

– Concurrent processing of database operations

A consistent state of the database means it satisfies all the constraints specified in the schema as well as any other constraints (e.g., semantic constraints) on the database that should hold

12

What this chapter is about?

This chapter is about – Transactions

– DBMS support to ensure correctness of transaction processing Recovery manager – to deal with system failures Concurrency control – to process database operations requested by

multiple users

13

Transactions

A transaction is an executing program that forms a logical unit of database processing

– Examples: Bank processing – deposit/withdrawal transactions Student registration – enrolment/withdrawal transactions Airline reservation – reservation/cancellation transactions

– Each transaction consists of one or more database operations

– Example: bank deposit transaction

begin_transactionread_item(acct)acct.bal := acct.bal + amountwrite_item(acct)end_transaction

1 logical unit => 1 transaction

14

ACID Properties of Transactions

But transactions are no ordinary programs

Additional requirements are placed on the execution of transactions beyond those placed on ordinary programs

– Atomicity

– Consistency

– Isolation

– Durability

15

ACID Properties of Transactions

Atomicity– A transaction must either run to its completion or, if it is not

completed, has no effect at all on the database state Consistency

– A transaction should correctly transform the database from one consistent state to another

Isolation– A transaction should appear as though it is being executed in

isolation from other transactions– The execution of a transaction should not be interfered with by

other transactions executing concurrently Durability

– Changes applied to the database by a committed transaction must persist in the database

– These changes must never be lost because of any failure

16

ACID Properties

Ensuring consistency is the responsibility of application programmers

Ensuring atomicity, isolation, and durability properties are the responsibilities of the DBMS

– Atomicity and durability properties are enforced by the recovery subsystem of DBMS

– Isolation property is enforced by the concurrency control subsystem of DBMS (next lecture)

17

Transaction Support in MySQL

For transaction processing, make sure you use the INNODB storage engine (instead of MyISAM)

How can we tell what type of storage structure used for each table?

Mysql> show table status from database_name like ‘table_name’

How to create table with a particular storage engine?Mysql> create table tableName (id int, name char(20)) engine=innodb

How to convert from MyISAM to INNODB?Mysql> alter table tableName engine=innodb

18

MySQL Example

Client 1:Mysql> create table account (id int primary

key, balance double) engine = innodb;

Mysql> start transaction;

Mysql> insert into account values (1, 1000);

Mysql> select * from account;

+------+----------+

| id | balance |

+------+-----------+

| 1 | 1000 |

+------+-----------+

Mysql> commit;

Client 2:

Mysql> select * from account;

Empty set (0.00 sec)

Mysql> select * from account;

+------+----------+

| id | balance |

+------+-----------+

| 1 | 1000 |

+------+-----------+

19

MySQL Example (Aborted Transaction)

Client 1:Mysql> start transaction;

Mysql> insert into account values (1,1000);

Mysql> select * from account;

+------+----------+

| id | balance |

+------+-----------+

| 1 | 1000 |

+------+-----------+

Mysql> rollback;

Client 2:

Mysql> select * from account;Empty set (0.00 sec)

Mysql> select * from account; Empty set (0.00 sec)

DBMS will automatically undo

the effect of insertion

20

MySQL Example (Concurrency Control)

Client 1:Mysql> create table acct2 (id int primary key,

balance double) engine=innodb;

Mysql> start transaction;

Mysql> insert into acct2 values (1,1000);

Query OK, 1 row affected (0.00 sec)

Mysql> commit

Client 2:

Mysql> start transaction;

Mysql> select * from acct2;

Empty set (0.00 sec)

Mysql> insert into acct2 values (1,50);

(Client 2 will be kept waiting until client 1 commits or rollback)

ERROR 1062 (00000): Duplicate entry '1' for key 1

21

MySQL Example (Concurrency Control)

Client 1:Mysql> create table acct2b (id int

primary key, balance double) engine=innodb;

Mysql> start transaction;

Mysql> insert into acct2b values (1,1000);

Query OK, 1 row affected (0.00 sec)

Mysql> rollback;

Query OK, 0 row affected (0.00 sec)

Client 2:

Mysql> start transaction;

Mysql> select * from acct2b;

Empty set (0.00 sec)

Mysql> insert into acct2b values (1,500);

(Client 2 will be kept waiting until client 1 commits or rollback)

Query OK, 1 row affected (5.98 sec)

22

MySQL Example (Concurrency Control)

Client 1:Mysql> create table acct3 (id int, balance

double) engine=innodb;

Mysql> start transaction;

Mysql> insert into acct3 values (1,1000);

Query OK, 1 row affected (0.00 sec)

Mysql> select * from acct3;

+------+------------+

| id | balance |

+------+------------+

| 1 | 1000 |

+------+------------+

Client 2:

Mysql> start transaction;

Mysql> select * from acct3;

Empty set (0.00 sec)

Mysql> insert into acct3 values (1, 50);

Query OK, 1 row affected (0.00 sec)

(OK because id is not primary key)

23

MySQL Example (Concurrency Control)

Client 1:

Mysql> commit;

Mysql> select * from acct3;

+------+------------+

| id | balance |

+------+------------+

| 1 | 1000 |

+------+------------+

Client 2:Mysql> select * from acct3;

+------+------------+

| id | balance |

+------+------------+

| 1 | 50 |

+------+------------+

Mysql> select * from temp3;

+------+------------+

| id | balance |

+------+------------+

| 1 | 1000 |

| 1 | 50 |

+------+------------+

Mysql> commit;

24

Types of Failures

Computer failure or system crash (e.g., media failure) Transaction/system error (e.g., integer overflow, division

by zero, user interrupt during transaction execution) Local errors or exception conditions detected by the

transaction (e.g., insufficient balance in bank account) Concurrency control enforcement (e.g., aborted

transaction) Physical problems and catastrophes

Recovery manager of DBMS is responsible for making sure that all operations in a transaction are completed successfully and their effect recorded permanently

25

Recovery

For recovery purposes, the recovery manager of DBMS must keep track of the following operations

– BEGIN_TRANSACTION

– READ or WRITE

– END_TRANSACTION

– COMMIT_TRANSACTION This signals a successful end of the transaction so that any changes

executed by the transaction can be safely committed to the database and will not be undone

– ROLLBACK (or ABORT) This signals that the transaction has ended unsuccessfully, so that

any changes or effects that the transaction may have applied to the database must be undone.

26

Transaction State

At any point in time, a transaction is in one of the following states:

– Active state

– Partially committed state

– Committed state

– Failed state

– Terminated State

27

System Log

Mechanism for dealing with failures is the system log

A log is a sequence of records that describes database updates made by transactions

– Used to restore database to a consistent state after a failure

– Log should be stored on a different disk than the database Survives processor crash and media failure

– Log should be periodically backed up to archival storage (tape) to guard against catastrophic failures

28

System Log

Types of entries in a log record (T: transaction ID)– [start_transaction,T]: transaction T has started execution.– [write_item,T,X,old_value,new_value]: transaction T has

changed the value of database item X from old_value to new_value Old_value is called before image (BFIM) New_value is called after image (AFIM)

– [read_item,T,X]: transaction T has read the value of X.– [commit,T]: transaction T has completed successfully, and

affirms that its effect can be committed (recorded permanently) to the database.

– [abort,T]: transaction T has been aborted

If the system crashes, we can recover to a consistent database state by examining the log

29

Commit Point

A transaction reaches its commit point when – All of its database operations have been executed successfully

– Effect of all the operations has been recorded in the log

The transaction then writes an entry [commit,T] into the log

Beyond the commit point, the transaction is said to be committed, and its effect is permanently recorded in the database

30

Recovery from Transaction Aborts

When a transaction T aborts:– Scan the log backward (rollback)

– Apply the before image in each of the transaction’s update records to database items to restore them to their original state.

– Scan the log backward up to Begin_transaction for T

– Write an entry [abort, T] into the log

31

Example

Suppose transaction T2 is aborted

B1 U1 B2 U1 U2 U1 U2

B – begin transactionU – update record

End of log whenT2 is aborted

Begin rollback scan

32

Example

Suppose transaction T2 is aborted

B1 U1 B2 U1 U2 U1 U2

B – begin transactionUi – update record ofTransaction i

Undo changes made by T2

Rollback scan

33

Example

Example: Aborting transaction T2

B1 U1 B2 U1 U2 U1 U2

No need toundo changes made by T1

B – begin transactionUi – update record of transaction i

Rollback scan

34

Example

Example: Aborting transaction T2

B1 U1 B2 U1 U2 U1 U2

B – begin transactionU – update record

Undo changes made by T2

Rollback scan

35

Example

Example: Aborting transaction T2

B1 U1 B2 U1 U2 U1 U2

No need toundo changes made by T1

B – begin transactionU – update record

Rollback scan

36

Example

Example: Aborting transaction T2

B1 U1 B2 U1 U2 U1 U2

End of rollback scan when T2 is aborted

B – begin transactionU – update record

Rollback scan

37

Example

Example: Aborting transaction T2

B1 U1 B2 U1 U2 U1 U2

B – begin transactionU – update record

Add entry forabort T2 to log

A2

38

Recovery from System Crash

More complicated than rollback due to aborted transaction

After system crash, active transactions must be identified and aborted when the system recovers

– When scanning the log backwards: if the first record encountered for a transaction is an update record,

the transaction must still be active If the first record encountered for a transaction is a commit/abort

record, the transaction has already completed and thus can be ignored

39

Example

Crash

The Commit/Abort records are insufficient to identify active transactions How far back should we scan to determine the active transactions when the system crashes?

B - beginU - updateC - commitA - abort

40

Checkpointing

Need a better mechanism to identify active transactions so that the recovery process can stop backward scan

System periodically appends a checkpoint record that lists all the currently active transactions

– During recovery, system must scan backward at least to the last checkpoint record

– If all active transactions recorded in the checkpoint record has committed prior to system crash, recovery process can stop

– If some active transactions recorded in the checkpoint record has not committed prior to system crash, backward scan must continue past the checkpoint record until the begin records for such transactions are encountered

41

Example

Rollback up to here

42

Log and Database Updating

Both the log and database must be updated when a transaction modifies an item.

Which one should be updated first?– Update the log first or update the database first?

– What if system crashes when one is updated but not the other?

43

Write-Ahead Log

DBMS use a write-ahead log – Update the record in log first before applying the update to

database item

If database is updated first and system crashes before log is updated

– On recovery, database item is in the new state but there is no before image to roll it back. Transaction cannot be aborted.

If log is updated first and system crashes before log is updated

– On recovery, database item in old state and before image in log. Converting After image to Before image has no effect.