04/19/23

RecoveryRecovery

Integrity or correctness of dataIntegrity or correctness of data

Would like data to be “accurate” orWould like data to be “accurate” or “correct” at all times“correct” at all times

EMPEMP

Name

WhiteGreenGray

Age

523421

1

Integrity or consistency Integrity or consistency constraintsconstraints

Predicates data must satisfyPredicates data must satisfy Examples:Examples:

- x is key of relation R- x is key of relation R- x - x y holds in R y holds in R- Domain(x) = {Red, Blue, Green}- Domain(x) = {Red, Blue, Green}is valid index for attribute x of Ris valid index for attribute x of R- no employee should make more than- no employee should make more than twice twice

the average salarythe average salary

Definition:Definition:

Consistent state:Consistent state: satisfies all constraints satisfies all constraints Consistent DB:Consistent DB: DB in consistent state DB in consistent state

Observation:Observation: DB DB cannotcannot be consistent be consistent always!always!

Example:Example: a a11 + a + a22 +…. a +…. ann = TOT ( = TOT (constraintconstraint))

Deposit $100 in Deposit $100 in aa22: a: a22 a a22 + 100 + 100

TOT TOT TOT + 100 TOT + 100

aa22

TOTTOT

.

.50

.

.1000

.

.150

.

.1000

.

.150

.

.1100

Example: a1 + a2 +…. an = TOT (constraint)Deposit $100 in a2: a2 a2 + 100

TOT TOT + 100

Transaction: Transaction: collection of actions collection of actions

that preserve consistencythat preserve consistency

Consistent DB Consistent DB’T

How can we How can we prevent/fixprevent/fix violations? violations?

cc: due to data sharing cc: due to data sharing onlyonly recovery: due to failures recovery: due to failures onlyonly

Will not consider:Will not consider:

How to write correct transactionsHow to write correct transactions How to write correct DBMSHow to write correct DBMS Constraint checking & repairConstraint checking & repair

That is, solutions studied here do not need to That is, solutions studied here do not need to know constraintsknow constraints

Storage hierarchyStorage hierarchy

Memory Disk

x x

Operations:Operations:

Input (x): block with x Input (x): block with x memory memory Output (x): block with x Output (x): block with x disk disk

Read (x,t): do input(x) if necessary t value of x in blockWrite (x,t): do input(x) if necessary value of x in block t

Key problemKey problem Unfinished Unfinished transactiontransaction

ExampleExample Constraint: A=BConstraint: A=B

TT11: A : A A A 2 2

B B B B 2 2

TT11:: Read (A,t); t Read (A,t); t t t22Write (A,t);Write (A,t);Read (B,t); t Read (B,t); t t t22Write (B,t);Write (B,t);Output (A);Output (A);Output (B);Output (B);

A: 8B: 8

A: 8B: 8

memory disk

1616

16

failure!

TT11:: Read (A,t); t Read (A,t); t t t22 A=B A=BWrite (A,t);Write (A,t);Read (B,t); t Read (B,t); t t t22Write (B,t);Write (B,t);Output (A);Output (A);Output (B);Output (B);

A:8B:8

A:8B:8

memory disk log

Undo logging

1616

<T1, start><T1, A, 8>

<T1, commit>16 <T1, B, 8>

16

One “complication”One “complication”

Log is first written in memoryLog is first written in memory Not written to disk on every actionNot written to disk on every action

memorymemory

DBDB

LogLog

A: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8>

A: 8B: 8

16BAD STATE

# 1

One “complication”One “complication”

Log is first written in memoryLog is first written in memory Not written to disk on every actionNot written to disk on every action

memorymemory

DBDB

LogLog

A: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8><T1, commit>

A: 8B: 8

16BAD STATE

# 2

<T1, B, 8><T1, commit>

...

Undo logging rulesUndo logging rules

(1) For every action generate undo log(1) For every action generate undo log record (containing old value)record (containing old value)

(2) Before (2) Before xx is modified on disk, log is modified on disk, log records pertaining to records pertaining to xx must be must be on disk on disk (write ahead logging: WAL)(write ahead logging: WAL)

(3) Before commit is flushed to log, all(3) Before commit is flushed to log, all writes ofwrites of transaction must betransaction must be reflected reflected on diskon disk

Recovery rules:Recovery rules: Undo Undo logginglogging

(1) Let S = set of transactions with(1) Let S = set of transactions with <Ti, start> in <Ti, start> in log, but no <Ti, commit> (or <Ti, abort>) record log, but no <Ti, commit> (or <Ti, abort>) record in login log

(2) For each <Ti, X, v> in log,(2) For each <Ti, X, v> in log,

in reverse order (latest in reverse order (latest earliest) do:earliest) do:

- if Ti - if Ti S then - write (X, v) S then - write (X, v)

- output (X)- output (X)

(3) For each Ti (3) For each Ti S do S do

- write <Ti, abort> to log- write <Ti, abort> to log

What if failure during recovery?What if failure during recovery?

No problem! No problem! Undo Undo idempotentidempotent

To discuss:To discuss:

Redo loggingRedo logging Undo/redo logging, why both?Undo/redo logging, why both?

Redo loggingRedo logging (deferred (deferred modification)modification)

TT1:1: Read(A,t); t tRead(A,t); t t2; write (A,t);2; write (A,t);

Read(B,t); t tRead(B,t); t t2; write (B,t);2; write (B,t);

Output(A); Output(B) Output(A); Output(B)

A: 8B: 8

A: 8B: 8

memory DB LOG

1616

<T1, start><T1, A, 16><T1, B, 16>

<T1, commit>

output16

Redo logging rulesRedo logging rules

(1) For every action, generate redo log(1) For every action, generate redo log record (containing new value)record (containing new value)

(2) Before (2) Before anythinganything is modified on disk is modified on disk (DB),(DB), all log records for transaction thatall log records for transaction that modifmodify thingsy things (including commit) must (including commit) must be on diskbe on disk

(1) Let S = set of transactions with(1) Let S = set of transactions with<Ti, commit> in log<Ti, commit> in log

(2) For each <Ti, X, v> in log, in forward(2) For each <Ti, X, v> in log, in forward

order (earliest order (earliest latest) do: latest) do:

- if Ti - if Ti S then Write(X, v) S then Write(X, v)

Output(X) Output(X)

Recovery rules:Recovery rules: Redo Redo logginglogging

Recovery is very, very Recovery is very, very

SLOW !SLOW !Redo log:Redo log:

FirstFirst T1 wrote A,BT1 wrote A,B LastLast

RecordRecord Committed a year agoCommitted a year ago RecordRecord

(1 year ago)(1 year ago) --> STILL, Need to redo after crash!!--> STILL, Need to redo after crash!!

... ... ...

Crash

Undo Recovery – Better?Undo Recovery – Better?

The first record without commit or abort The first record without commit or abort should not be too far away from crashshould not be too far away from crash

But immediate update itself is expensiveBut immediate update itself is expensive

... ... ...

first record without commit or abort

Solution:Solution: Checkpoint Checkpoint (simple (simple version)version)

Periodically:Periodically:

(1) Do not accept new transactions(1) Do not accept new transactions

(2) Wait until all transactions finish(2) Wait until all transactions finish

(3) Flush all log records to disk (log)(3) Flush all log records to disk (log)(4) Flush all buffers to disk (DB)(4) Flush all buffers to disk (DB) (do not discard buffers) (do not discard buffers)

(5) Write “checkpoint” record on disk (log)(5) Write “checkpoint” record on disk (log)

(6) Resume transaction processing(6) Resume transaction processing

Key drawbacks:Key drawbacks:

Undo logging:Undo logging: need to update immediately, need to update immediately, increasing I/Oincreasing I/O

Redo logging:Redo logging: need to keep all modified blocks need to keep all modified blocks in memory until commitin memory until commit

Solution: Solution: undo/redo logging!undo/redo logging!

Update Update <Ti, Xid, New X val, Old X val> <Ti, Xid, New X val, Old X val>

RulesRules

Page X can be flushed before orPage X can be flushed before or after Ti after Ti commitcommit

Log record flushed before corresponding Log record flushed before corresponding updated page (WAL)updated page (WAL)

Flush Flush log log at commitat commit

SummarySummary

undoundo redoredo Undo-redoUndo-redo

commitcommit commit means all commit means all updates on diskupdates on disk

Commit means Commit means some updates may some updates may start to migrate to start to migrate to diskdisk

Commit does not Commit does not signal disk updatesignal disk update

recoveryrecovery Undo all Undo all incomplete incomplete transactionstransactions

Redo all Redo all committed committed transactionstransactions

Undo all incomplete Undo all incomplete transactions and transactions and redo all committed redo all committed transactionstransactions

WAL

SummarySummary

Consistency of dataConsistency of data One source of problems: failuresOne source of problems: failures

-- WAL WAL