CS 4604: Introducon to Database Management...
Transcript of CS 4604: Introducon to Database Management...
![Page 1: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/1.jpg)
CS4604:Introduc0ontoDatabaseManagementSystems
B.AdityaPrakashLecture#19:LoggingandRecovery2:
ARIES
![Page 2: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/2.jpg)
Motivation
§ Atomicity: – Transactions may abort (“Rollback”).
§ Durability: – What if DBMS stops running? (Causes?)
crash! ❖ Desired state after system restarts: – T1 & T3 should be durable. – T2, T4 & T5 should be aborted (effects
not seen).
T1 T2 T3 T4 T5
Abort Commit
Commit
Prakash2016 VTCS4604 2
![Page 3: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/3.jpg)
General Overview
§ Preliminaries § Write-Ahead Log - main ideas § (Shadow paging) § Write-Ahead Log: ARIES
Prakash2016 VTCS4604 3
![Page 4: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/4.jpg)
Main ideas so far:
§ Write-Ahead Log, for loss of volatile storage, § with incremental updates (STEAL, NO
FORCE) § and checkpoints § On recovery: undo uncommitted; redo
committed transactions.
Prakash2016 VTCS4604 4
![Page 5: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/5.jpg)
Today: ARIES
With full details on – fuzzy checkpoints – recovery algorithm
C. Mohan (IBM)
Prakash2016 VTCS4604 5
![Page 6: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/6.jpg)
Overview
§ Preliminaries § Write-Ahead Log - main ideas § (Shadow paging) § Write-Ahead Log: ARIES
– LSN’s – examples of normal operation & of abort – fuzzy checkpoints – recovery algo
Prakash2016 VTCS4604 6
![Page 7: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/7.jpg)
LSN
§ Log Sequence Number § every log record has an LSN § Q: Why do we need it?
Prakash2016 VTCS4604 7
![Page 8: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/8.jpg)
LSN <T1 start> <T2 start> <T4 start> <T4, A, 10, 20> <T1 commit> <T4, B, 30, 40> <T3 start> <T2 commit> <T3 commit> ~~~~ CRASH ~~~~
A1: e.g, undo T4 - it is faster, if we have a linked list of the T4 log records A2: and many other uses - see later
Prakash2016 VTCS4604 8
![Page 9: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/9.jpg)
Types of log records <T1 start> <T2 start> <T4 start> <T4, A, 10, 20> <T1 commit> <T4, B, 30, 40> <T3 start> <T2 commit> <T3 commit> ~~~~ CRASH ~~~~
Q1: Which types? A1: Q2: What format? A2:
Prakash2016 VTCS4604 9
![Page 10: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/10.jpg)
Types of log records <T1 start> <T2 start> <T4 start> <T4, A, 10, 20> <T1 commit> <T4, B, 30, 40> <T3 start> <T2 commit> <T3 commit> ~~~~ CRASH ~~~~
Q1: Which types? A1: Update, commit, ckpoint, … Q2: What format? A2: x-id, type, (old value, …)
Prakash2016 VTCS4604 10
![Page 11: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/11.jpg)
Log Records Possible log record types: § Update, Commit, Abort § Checkpoint (for log
maintenance) § Compensation Log
Records (CLRs) – for UNDO actions
§ End (end of commit or abort)
prevLSN
XID
type
length
pageID
offset
before-image
after-image
LogRecord fields:
update records only
Prakash2016 VTCS4604 11
![Page 12: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/12.jpg)
Overview
§ Preliminaries § Write-Ahead Log - main ideas § (Shadow paging) § Write-Ahead Log: ARIES
– LSN’s – examples of normal operation & of abort – fuzzy checkpoints – recovery algo
Prakash2016 VTCS4604 12
![Page 13: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/13.jpg)
Writing log records
§ We don’t want to write one record at a time – (why not?)
§ How should we buffer them?
Prakash2016 VTCS4604 13
![Page 14: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/14.jpg)
Writing log records
§ We don’t want to write one record at a time – (why not?)
§ How should we buffer them? – Batch log updates; – Un-pin a data page ONLY if all the corresponding
log records have been flushed to the log.
Prakash2016 VTCS4604 14
![Page 15: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/15.jpg)
WAL & the Log
§ Each data page contains a pageLSN. – The LSN of the most recent update to
that page. § System keeps track of flushedLSN.
– The max LSN flushed so far. § WAL: For a page i to be written must flush log at least to the point where:
pageLSNi ≤ flushedLSN pageLSN
Log records flushed to disk
“Log tail” in RAM
flushedLSN
Prakash2016 VTCS4604 15
![Page 16: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/16.jpg)
WAL & the Log
§ Can we un-pin the gray page?
pageLSN
Log records flushed to disk
“Log tail” in RAM
flushedLSN
Prakash2016 VTCS4604 16
![Page 17: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/17.jpg)
WAL & the Log
§ Can we un-pin the gray page? § A: yes
pageLSN
Log records flushed to disk
“Log tail” in RAM
flushedLSN
Prakash2016 VTCS4604 17
![Page 18: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/18.jpg)
WAL & the Log
§ Can we un-pin the blue page?
pageLSN
Log records flushed to disk
“Log tail” in RAM
flushedLSN
Prakash2016 VTCS4604 18
![Page 19: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/19.jpg)
WAL & the Log
§ Can we un-pin the blue page? § A: no
pageLSN
Log records flushed to disk
“Log tail” in RAM
flushedLSN
Prakash2016 VTCS4604 19
![Page 20: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/20.jpg)
WAL & the Log
LSNs
DB
pageLSNs
RAM
flushedLSN
pageLSN
Log records flushed to disk
“Log tail” in RAM
flushedLSN
Q: why not on disk or log?
Prakash2016 VTCS4604 20
![Page 21: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/21.jpg)
Overview
§ Preliminaries § Write-Ahead Log - main ideas § (Shadow paging) § Write-Ahead Log: ARIES
– LSN’s – examples of normal operation & of abort – fuzzy checkpoints – recovery algo
Prakash2016 VTCS4604 21
![Page 22: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/22.jpg)
Normal Execution of an Xact
§ Series of reads & writes, followed by commit or abort. – We will assume that disk write is atomic.
• In practice, additional details to deal with non-atomic writes.
§ Strict 2PL. § STEAL, NO-FORCE buffer management, with
Write-Ahead Logging.
Prakash2016 VTCS4604 22
![Page 23: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/23.jpg)
Normal execution of an Xact
§ Page ‘i’ can be written out only after the corresponding log record has been flushed
Prakash2016 VTCS4604 23
![Page 24: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/24.jpg)
Transaction Commit
§ Write commit record to log. § All log records up to Xact’s commit record
are flushed to disk.
Q: why not flush the dirty pages, too?
Prakash2016 VTCS4604 24
![Page 25: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/25.jpg)
Transaction Commit
§ Write commit record to log. § All log records up to Xact’s commit record
are flushed to disk. – Note that log flushes are sequential,
synchronous writes to disk. – Many log records per log page.
§ Commit() returns. § Write end record to log.
Prakash2016 VTCS4604 25
![Page 26: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/26.jpg)
Example LSN prevLSN tid type item old new 10 NULL T1 update X 30 40 .... 50 10 T1 update Y 22 25 ... 63 50 T1 commit ... 68 63 T1 end
dbms flushes log records + some record-keeping
Prakash2016 VTCS4604 26
![Page 27: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/27.jpg)
Overview
§ Preliminaries § Write-Ahead Log - main ideas § (Shadow paging) § Write-Ahead Log: ARIES
– LSN’s – examples of normal operation & of abort – fuzzy checkpoints – recovery algo
Prakash2016 VTCS4604 27
![Page 28: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/28.jpg)
Abort
Actually, a special case of the up-coming ‘undo’ operation,
applied to only one transaction - e.g.:
Prakash2016 VTCS4604 28
![Page 29: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/29.jpg)
Abort - Example
LSN prevLSN tid type item old new 10 NULL T2 update Y 30 40 ... 63 10 T2 abort
Prakash2016 VTCS4604 29
![Page 30: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/30.jpg)
Abort - Example
LSN prevLSN tid type item old new 10 NULL T2 update Y 30 40 ... 63 10 T2 abort ... 72 63 T2 CLR (LSN 10) ... 78 72 T2 end
compensating log record
Prakash2016 VTCS4604 30
![Page 31: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/31.jpg)
Abort - Example
LSN prevLSN tid type item old new undoNextLSN 10 NULL T2 update Y 30 40 ... 63 10 T2 abort ... 72 63 T2 CLR Y 40 30 NULL ... 78 72 T2 end
Prakash2016 VTCS4604 31
![Page 32: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/32.jpg)
CLR record - details
§ a CLR record has all the fields of an ‘update’ record
§ plus the ‘undoNextLSN’ pointer, to the next-to-be-undone LSN
Prakash2016 VTCS4604 32
![Page 33: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/33.jpg)
Abort - algorithm:
§ First, write an ‘abort’ record on log and § Play back updates, in reverse order: for each
update – write a CLR log record – restore old value
§ at end, write an ‘end’ log record
Notice: CLR records never need to be undone
Prakash2016 VTCS4604 33
![Page 34: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/34.jpg)
Overview
§ Preliminaries § Write-Ahead Log - main ideas § (Shadow paging) § Write-Ahead Log: ARIES
– LSN’s – examples of normal operation & of abort – fuzzy checkpoints – recovery algo
Prakash2016 VTCS4604 34
![Page 35: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/35.jpg)
(non-fuzzy) checkpoints
§ they have performance problems - recall from previous lecture:
Prakash2016 VTCS4604 35
![Page 36: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/36.jpg)
(non-fuzzy) checkpoints
We assumed that the DBMS: § stops all transactions, and § flushes on disk the ‘dirty
pages’ Both decisions are expensive Q: Solution?
<T1 start> ... <T1 commit> ... <T499, C, 1000, 1200> <checkpoint> <T499 commit> <T500 start> <T500, A, 200, 400> <checkpoint> <T500, B, 10, 12>
before
crash Prakash2016 VTCS4604 36
![Page 37: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/37.jpg)
(non-fuzzy) checkpoints
Q: Solution? Hint1: record state as of the
beginning of the ckpt Hint2: we need some
guarantee about which pages made it to the disk
<T1 start> ... <T1 commit> ... <T499, C, 1000, 1200> <checkpoint> <T499 commit> <T500 start> <T500, A, 200, 400> <checkpoint> <T500, B, 10, 12>
before
crash Prakash2016 VTCS4604 37
![Page 38: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/38.jpg)
checkpoints
Q: Solution? A: write on the log: § the id-s of active
transactions and § the id-s (ONLY!) of dirty
pages (rest: obviously made it to the disk!)
<T1 start> ... <T1 commit> ... <T499, C, 1000, 1200> <checkpoint> <T499 commit> <T500 start> <T500, A, 200, 400> <checkpoint> <T500, B, 10, 12>
before
crash Prakash2016 VTCS4604 38
![Page 39: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/39.jpg)
(Fuzzy) checkpoints
Specifically, write to log: – begin_checkpoint record: indicates start of ckpt – end_checkpoint record: Contains current Xact table
and dirty page table. This is a `fuzzy checkpoint’: • Other Xacts continue to run; so these tables accurate only
as of the time of the begin_checkpoint record. • No attempt to force dirty pages to disk; effectiveness of
checkpoint limited by oldest unwritten change to a dirty page.
Prakash2016 VTCS4604 39
![Page 40: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/40.jpg)
(Fuzzy) checkpoints
Specifically, write to log: – begin_checkpoint record: indicates start of ckpt – end_checkpoint record: Contains current Xact table
and dirty page table. This is a `fuzzy checkpoint’: • Other Xacts continue to run; so these tables accurate only
as of the time of the begin_checkpoint record. • No attempt to force dirty pages to disk; effectiveness of
checkpoint limited by oldest unwritten change to a dirty page.
solved both problems of non-fuzzy ckpts!!
Prakash2016 VTCS4604 40
![Page 41: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/41.jpg)
(Fuzzy) checkpoints - cont’d
And: – Store LSN of most recent chkpt record on disk
(master record) • Q: why do we need that?
Prakash2016 VTCS4604 41
![Page 42: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/42.jpg)
(Fuzzy) Checkpoints More details: Two in-memory tables: #1) Transaction Table
Q: what would you store there?
Prakash2016 VTCS4604 42
![Page 43: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/43.jpg)
(Fuzzy) Checkpoints More details: Two in-memory tables: #1) Transaction Table § One entry per currently active Xact.
– entry removed when Xact commits or aborts § Contains
– XID, – status (running/committing/aborting), and – lastLSN (most recent LSN written by Xact).
Prakash2016 VTCS4604 43
![Page 44: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/44.jpg)
(Fuzzy) Checkpoints #2) Dirty Page Table:
– One entry per dirty page currently in buffer pool. – Contains recLSN -- the LSN of the log record
which first caused the page to be dirty.
Prakash2016 VTCS4604 44
![Page 45: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/45.jpg)
Overview
§ Preliminaries § Write-Ahead Log - main ideas § (Shadow paging) § Write-Ahead Log: ARIES
– LSN’s – examples of normal operation & of abort – fuzzy checkpoints – recovery algo
Prakash2016 VTCS4604 45
![Page 46: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/46.jpg)
The Big Picture: What’s Stored Where
DB
Data pages each with a pageLSN
Xact Table lastLSN status
Dirty Page Table
recLSN flushedLSN
RAM
prevLSN XID type
length pageID
offset before-image after-image
LogRecords
LOG
master record LSN of most recent checkpoint
update CLR
undoNextLSN CLR Prakash2016 VTCS4604 46
![Page 47: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/47.jpg)
Crash Recovery: Big Picture
• Start from a checkpoint (found via master record).
• Three phases. – Analysis - Figure out which Xacts
committed since checkpoint, which failed. – REDO all actions (repeat history) – UNDO effects of failed Xacts.
Oldest log rec. of Xact active at crash
Smallest recLSN in dirty page table after Analysis
Last chkpt
CRASH
A R U Prakash2016 VTCS4604 47
![Page 48: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/48.jpg)
Crash Recovery: Big Picture
• Notice: relative ordering of A, B, C may vary!
Oldest log rec. of Xact active at crash
Smallest recLSN in dirty page table after Analysis
Last chkpt
CRASH
A R U
A
B
C
Prakash2016 VTCS4604 48
![Page 49: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/49.jpg)
Recovery: The Analysis Phase
§ Re-establish knowledge of state at checkpoint. – via transaction table and dirty page table stored in the
checkpoint
Prakash2016 VTCS4604 49
![Page 50: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/50.jpg)
Recovery: The Analysis Phase
§ Scan log forward from checkpoint. – End record: Remove Xact from Xact table. – All Other records:
• Add Xact to Xact table, with status ‘U’ (=candidate for undo)
• set lastLSN=LSN, • on commit, change Xact status to ‘C’.
– also, for Update records: If page P not in Dirty Page Table (DPT), • add P to DPT, set its recLSN=LSN.
Prakash2016 VTCS4604 50
![Page 51: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/51.jpg)
Recovery: The Analysis Phase
§ At end of Analysis: – transaction table says which xacts were active at time
of crash. – DPT says which dirty pages might not have made it to
disk
Prakash2016 VTCS4604 51
![Page 52: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/52.jpg)
Phase 2: REDO
Goal: repeat History to reconstruct state at crash: – Reapply all updates (even of aborted Xacts!), redo
CLRs. – (and try to avoid unnecessary reads and writes!)
Specifically: § Scan forward from log rec containing smallest
recLSN in DPT. Q: why start here?
Prakash2016 VTCS4604 52
![Page 53: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/53.jpg)
Phase 2: REDO (cont’d)
§ ... § For each update log record or CLR with a given
LSN, REDO the action unless: – Affected page is not in the Dirty Page Table, or – Affected page is in D.P.T., but has recLSN > LSN, or – pageLSN (in DB) ≥ LSN. (this last case requires I/O)
Prakash2016 VTCS4604 53
![Page 54: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/54.jpg)
Phase 2: REDO (cont’d)
§ ... § To REDO an action:
– Reapply logged action. – Set pageLSN to LSN. No additional logging, no
forcing!
Prakash2016 VTCS4604 54
![Page 55: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/55.jpg)
Phase 2: REDO (cont’d)
§ ... § at the end of REDO phase, write ‘end’ log
records for all xacts with status ‘C’, § and remove them from transaction table
Prakash2016 VTCS4604 55
![Page 56: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/56.jpg)
Phase 3: UNDO
Goal: Undo all transactions that were active at the time of crash (‘loser xacts’)
§ That is, all xacts with ‘U’ status on the xact
table of the Analysis phase § Process them in reverse LSN order § using the lastLSN’s to speed up traversal § and issuing CLRs
Prakash2016 VTCS4604 56
![Page 57: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/57.jpg)
Phase 3: UNDO ToUndo={lastLSNs of ‘loser’ Xacts} Repeat:
– Choose (and remove) largest LSN among ToUndo. – If this LSN is a CLR and undonextLSN==NULL
• Write an End record for this Xact. – If this LSN is a CLR, and undonextLSN != NULL
• Add undonextLSN to ToUndo – Else this LSN is an update. Undo the update, write a
CLR, add prevLSN to ToUndo. Until ToUndo is empty.
Prakash2016 VTCS4604 57
![Page 58: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/58.jpg)
Phase 3: UNDO - illustration LSN LOG
00 05 10 20 30 40
45 50 60
suppose that after end of analysis phase we have: xact table
xid status lastLSN T32 U T41 U
prevLSNs
Prakash2016 VTCS4604 58
![Page 59: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/59.jpg)
Phase 3: UNDO - illustration LSN LOG
00 05 10 20 30 40
45 50 60
suppose that after end of analysis phase we have: xact table
xid status lastLSN T32 U T41 U
undo in reverse LSN order
Prakash2016 VTCS4604 59
![Page 60: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/60.jpg)
RAM
Example of Recovery
begin_checkpoint end_checkpoint update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN 10
T1 End update: T3 writes P1 update: T2 writes P5 CRASH
LSN LOG
00 05 10 20 30 40
45 50 60
Xact Table lastLSN status
Dirty Page Table recLSN
flushedLSN ToUndo
prevLSNs
Prakash2016 VTCS4604 60
![Page 61: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/61.jpg)
Questions
§ Q1: After the Analysis phase, which are the ‘loser’ transactions?
§ Q2: UNDO phase - what will it do?
Prakash2016 VTCS4604 61
![Page 62: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/62.jpg)
Questions
§ Q1: After the Analysis phase, which are the ‘loser’ transactions?
§ A1: T2 and T3 § Q2: UNDO phase - what will it do? § A2: undo ops of LSN 60, 50, 20
Prakash2016 VTCS4604 62
![Page 63: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/63.jpg)
Example: Crash During Restart! begin_checkpoint, end_checkpoint update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN 10, T1 End update: T3 writes P1
update: T2 writes P5 CRASH, RESTART CLR: Undo T2 LSN 60 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART
LSN LOG
00,05 10 20 30 40,45 50
60 70 80,85
Xact Table lastLSN status
Dirty Page Table recLSN
flushedLSN ToUndo
RAM
Prakash2016 VTCS4604 63
![Page 64: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/64.jpg)
Example: Crash During Restart! begin_checkpoint, end_checkpoint update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN 10, T1 End update: T3 writes P1
update: T2 writes P5 CRASH, RESTART CLR: Undo T2 LSN 60 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART
LSN LOG
00,05 10 20 30 40,45 50
60 70 80,85
Xact Table lastLSN status
Dirty Page Table recLSN
flushedLSN ToUndo
undonextLSN
RAM
Prakash2016 VTCS4604 64
![Page 65: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/65.jpg)
Example: Crash During Restart! begin_checkpoint, end_checkpoint update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN 10, T1 End update: T3 writes P1
update: T2 writes P5 CRASH, RESTART CLR: Undo T2 LSN 60 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART
LSN LOG
00,05 10 20 30 40,45 50
60 70 80,85
Xact Table lastLSN status
Dirty Page Table recLSN
flushedLSN ToUndo
undonextLSN
RAM
Prakash2016 VTCS4604 65
![Page 66: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/66.jpg)
Example: Crash During Restart! begin_checkpoint, end_checkpoint update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN 10, T1 End update: T3 writes P1
update: T2 writes P5 CRASH, RESTART CLR: Undo T2 LSN 60 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART
LSN LOG
00,05 10 20 30 40,45 50
60 70 80,85
undonextLSN
RAM
Prakash2016 VTCS4604 66
![Page 67: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/67.jpg)
Questions
§ Q3: After the Analysis phase, which are the ‘loser’ transactions?
§ Q4: UNDO phase - what will it do?
Prakash2016 VTCS4604 67
![Page 68: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/68.jpg)
Questions
§ Q3: After the Analysis phase, which are the ‘loser’ transactions?
§ A3: T2 only § Q4: UNDO phase - what will it do? § A4: follow the string of prevLSN of T2,
exploiting undoNextLSN
Prakash2016 VTCS4604 68
![Page 69: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/69.jpg)
Example: Crash During Restart! begin_checkpoint, end_checkpoint update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN 10, T1 End update: T3 writes P1
update: T2 writes P5 CRASH, RESTART CLR: Undo T2 LSN 60 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART
LSN LOG
00,05 10 20 30 40,45 50
60 70 80,85
Xact Table lastLSN status
Dirty Page Table recLSN
flushedLSN ToUndo
undonextLSN
RAM
Prakash2016 VTCS4604 69
![Page 70: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/70.jpg)
Questions
§ Q5: show the log, after the recovery is finished:
Prakash2016 VTCS4604 70
![Page 71: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/71.jpg)
Example: Crash During Restart! begin_checkpoint, end_checkpoint update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN 10, T1 End update: T3 writes P1
update: T2 writes P5 CRASH, RESTART CLR: Undo T2 LSN 60 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART CLR: Undo T2 LSN 20, T2 end
LSN LOG
00,05 10 20 30 40,45 50
60 70 80,85
90, 95
Xact Table lastLSN status
Dirty Page Table recLSN
flushedLSN ToUndo
undonextLSN
RAM
Prakash2016 VTCS4604 71
![Page 72: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/72.jpg)
Additional Crash Issues
§ What happens if system crashes during Analysis? During REDO?
§ How do you limit the amount of work in REDO? – Flush asynchronously in the background.
§ How do you limit the amount of work in UNDO? – Avoid long-running Xacts.
Prakash2016 VTCS4604 72
![Page 73: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/73.jpg)
Summary of Logging/Recovery
§ Recovery Manager guarantees Atomicity & Durability.
Atomicity Consistency Isolation Durability
Prakash2016 VTCS4604 73
![Page 74: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/74.jpg)
Summary of Logging/Recovery
ARIES - main ideas: – WAL (write ahead log), STEAL/NO-
FORCE – fuzzy checkpoints (snapshot of dirty
page ids) – redo everything since the earliest dirty
page; undo ‘loser’ transactions – write CLRs when undoing, to survive
failures during restarts
let OS do its best
idempotency
Prakash2016 VTCS4604 74
![Page 75: CS 4604: Introducon to Database Management Systemscourses.cs.vt.edu/~cs4604/Spring16/lectures/lecture-19.pdf · – End record: Remove Xact from Xact table. – All Other records:](https://reader035.fdocuments.us/reader035/viewer/2022071016/5fcfb4494b2d7d07d501a17f/html5/thumbnails/75.jpg)
Summary of Logging/Recovery
Additional concepts: § LSNs identify log records; linked into
backwards chains per transaction (via prevLSN).
§ pageLSN allows comparison of data page and log records.
§ (and several other subtle concepts: undoNextLSN, recLSN etc)
Prakash2016 VTCS4604 75