Post on 05-Aug-2020
Lecture20:Transactions
Lecture20
Announcements• MostimportantlessonfromProject3!??!!
• Project2gradesshouldbeup• Letusknowofquestions
• Project4comingoutlatertoday:• Re-useyourdatabasefromProject1andaddanicewebinterfaceontoptosupporttransactions.
• Evaluationshouldbeopenpleasedonotforget
• December6th:GuestlecturebyStratis Viglas fromGoogle• ATTEND!!!
2
Lecture20
Goalsforthispairoflectures
• Transactions areaprogrammingabstractionthatenablestheDBMStohandlerecovery andconcurrency forusers.
• Application:Transactionsarecriticalforusers• Evencasualusersofdataprocessingsystems!
• Fundamentals:Thebasicsofhow TXNswork• Transactionprocessingispartofthedebatearoundnewdataprocessingsystems
• GiveyouenoughinformationtounderstandhowTXNswork,andthemainconcernswithusingthem
Lectures20&21
Lecture20:IntrotoTransactions&Logging
Lecture20
Today’sLecture
1. Transactions
2. PropertiesofTransactions:ACID
3. Logging
5
Lecture20
1.Transactions
6
Lecture20>Section1
Whatyouwilllearnaboutinthissection
1. Our“model”oftheDBMS/computer
2. Transactionsbasics
3. Motivation:Recovery&Durability
4. Motivation:Concurrency[nextlecture]
7
Lecture20>Section1
High-level:Diskvs.MainMemory
• Disk:
• Slow• Sequentialaccess
• (althoughfastsequentialreads)
• Durable• Wewillassumethatonceondisk,dataissafe!
• Cheap
8
Lecture20>Section3>Ourmodel
Platters
SpindleDisk head
Arm movement
Arm assembly
Tracks
Sector
Cylinder
• RandomAccessMemory(RAM)orMainMemory:
• Fast• Randomaccess,byteaddressable
• ~10xfasterforsequentialaccess• ~100,000xfasterforrandomaccess!
• Volatile• Datacanbelostife.g.crashoccurs,powergoesout,etc!
• Expensive• For$100,get16GBofRAMvs.2TBofdisk!
9
Lecture20>Section3>Ourmodel
High-level:Diskvs.MainMemory
Ourmodel:ThreeTypesofRegionsofMemory
1. Local: InourmodeleachprocessinaDBMShasitsownlocalmemory,whereitstoresvaluesthatonlyit“sees”
2. Global:Eachprocesscanreadfrom/writetoshareddatainmainmemory
3. Disk:Globalmemorycanreadfrom/flushtodisk
4. Log:Assumeonstablediskstorage- spansbothmainmemoryanddisk…
Local GlobalMain
Memory(RAM)
Disk
“Flushing todisk”=writingtodiskfrommainmemory
1 2
3
Lecture20>Section3>Ourmodel
Logisasequence frommainmemory->disk
4
• Keepinmindthetradeoffshereasmotivationforthemechanismsweintroduce
• Mainmemory:fastbutlimitedcapacity,volatile
• Vs.Disk:slowbutlargecapacity,durable
11
Lecture20>Section3>Ourmodel
High-level:Diskvs.MainMemory
Howdoweeffectivelyutilizeboth ensuringcertaincriticalguarantees?
Transactions
12
Lecture20>Section1>TransactionsBasics
Transactions:BasicDefinition
Atransaction(“TXN”)isasequenceofoneormoreoperations (readsorwrites)whichreflectsasinglereal-worldtransition.
START TRANSACTIONUPDATE ProductSET Price = Price – 1.99WHERE pname = ‘Gizmo’
COMMIT
Lecture20>Section1>TransactionsBasics
Intherealworld,aTXNeitherhappenedcompletelyornotatall
Transactions:BasicDefinition
Atransaction(“TXN”)isasequenceofoneormoreoperations (readsorwrites)whichreflectsasinglereal-worldtransition.
Lecture20>Section1>TransactionsBasics
Intherealworld,aTXNeitherhappenedcompletelyornotatall
Examples:
• Transfermoneybetweenaccounts
• Purchaseagroupofproducts
• Registerforaclass(eitherwaitlistorallocated)
15
TransactionsinSQL
• In“ad-hoc”SQL:• Default:eachstatement=onetransaction
• Inaprogram,multiplestatementscanbegroupedtogetherasatransaction:
Lecture20>Section1>TransactionsBasics
START TRANSACTIONUPDATE Bank SET amount = amount – 100WHERE name = ‘Bob’UPDATE Bank SET amount = amount + 100 WHERE name = ‘Joe’
COMMIT
ModelofTransactionforthisClass
Note:WeassumethattheDBMSonlyseesreadsandwritestodata
• Usermaydomuchmore
• Inrealsystems,databasesdohavemoreinfo...
Lecture20>Section1>TransactionsBasics
MotivationforTransactionsGroupinguseractions(reads&writes)intotransactionshelpswithtwogoals:
1. Recovery&Durability:KeepingtheDBMSdataconsistentanddurableinthefaceofcrashes,aborts,systemshutdowns,etc.
2. Concurrency: AchievingbetterperformancebyparallelizingTXNswithout creatinganomalies
Lecture20>Section1>Motivation
Thislecture!
Nextlecture
Motivation1.Recovery&Durability ofuserdataisessentialforreliableDBMSusage
• TheDBMSmayexperiencecrashes(e.g.poweroutages,etc.)
• IndividualTXNsmaybeaborted(e.g.bytheuser)
Lecture20>Section1>Motivation:Recovery&Durability
Idea:MakesurethatTXNsareeitherdurablystoredinfull,ornotatall;keeplogtobeableto“roll-back”TXNs
19
Protectionagainstcrashes/aborts
Client 1:INSERT INTO SmallProduct(name, price)
SELECT pname, priceFROM ProductWHERE price <= 0.99
DELETE ProductWHERE price <=0.99
Whatgoeswrong?
Crash/abort!
Lecture20>Section1>Motivation:Recovery&Durability
20
Protectionagainstcrashes/abortsClient 1:
START TRANSACTIONINSERT INTO SmallProduct(name, price)
SELECT pname, priceFROM ProductWHERE price <= 0.99
DELETE ProductWHERE price <=0.99
COMMIT OR ROLLBACK
Nowwe’dbefine!We’llseehow/whythislecture
Lecture20>Section1>Motivation:Recovery&Durability
Motivation2.Concurrent executionofuserprogramsisessentialforgoodDBMSperformance.
• Diskaccessesmaybefrequentandslow- optimizeforthroughput(#ofTXNs),tradeforlatency(timeforanyoneTXN)
• UsersshouldstillbeabletoexecuteTXNsasifinisolation andsuchthatconsistencyismaintained
Idea:HavetheDBMShandlerunningseveraluserTXNsconcurrently,inordertokeepCPUshumming…
Lecture20>Section1>Motivation:Concurrency
22
Multipleusers:singlestatements
Client 1: UPDATE ProductSET Price = Price – 1.99WHERE pname = ‘Gizmo’
Client 2: UPDATE ProductSET Price = Price*0.5WHERE pname=‘Gizmo’
Twomanagersattempttodiscountproductsconcurrently-Whatcouldgowrong?
Lecture20>Section1>Motivation:Concurrency
23
Multipleusers:singlestatementsClient 1: START TRANSACTION
UPDATE ProductSET Price = Price – 1.99WHERE pname = ‘Gizmo’
COMMIT
Client 2: START TRANSACTIONUPDATE ProductSET Price = Price*0.5WHERE pname=‘Gizmo’
COMMIT
Nowworkslikeacharm- we’llseehow/whynextlecture…
Lecture20>Section1>Motivation:Concurrency
2.PropertiesofTransactions
24
Lecture20>Section2
Whatyouwilllearnaboutinthissection
1. Atomicity
2. Consistency
3. Isolation
4. Durability
5. ACTIVITY?
25
Lecture20>Section2
26
TransactionProperties:ACID
• Atomic• Stateshowseitheralltheeffectsoftxn,ornoneofthem
• Consistent• Txn movesfromastatewhereintegrityholds,toanotherwhereintegrityholds
• Isolated• Effectoftxns isthesameastxns runningoneafteranother(ie lookslikebatchmode)
• Durable• Onceatxn hascommitted,itseffectsremaininthedatabase
ACIDcontinuestobeasourceofgreatdebate!
Lecture20>Section2
27
ACID:Atomicity
• TXN’sactivitiesareatomic:allornothing
• Intuitively:intherealworld,atransactionissomethingthatwouldeitheroccurcompletely ornotatall
• TwopossibleoutcomesforaTXN
• Itcommits:allthechangesaremade
• Itaborts:nochangesaremade
Lecture20>Section2>Atomicity
28
ACID:Consistency
• Thetablesmustalwayssatisfyuser-specifiedintegrityconstraints• Examples:
• Accountnumberisunique• Stockamountcan’tbenegative• Sumofdebitsandofcredits is0
• Howconsistencyisachieved:• Programmermakessureatxn takesaconsistentstatetoaconsistentstate• Systemmakessurethatthetxn isatomic
Lecture20>Section2>Consistency
29
ACID:Isolation
• Atransactionexecutesconcurrentlywithothertransactions
• Isolation:theeffectisasifeachtransactionexecutesinisolation oftheothers.
• E.g.Shouldnotbeabletoobservechangesfromothertransactionsduringtherun
Lecture20>Section2>Isolation
30
ACID:Durability
• TheeffectofaTXNmustcontinuetoexist(“persist”)aftertheTXN
• Andafterthewholeprogramhasterminated• Andeveniftherearepowerfailures,crashes,etc.• Andetc…
• Means:WritedatatodiskChangeonthehorizon?Non-VolatileRam(NVRam).Byteaddressable.
Lecture20>Section2>Durability
ChallengesforACIDproperties
• Inspiteoffailures:Powerfailures,butnotmediafailures
• Usersmayaborttheprogram:needto“rollbackthechanges”• Needtolog whathappened
• Manyusersexecutingconcurrently• Canbesolvedvialocking(we’llseethisnextlecture!)
Andallthiswith…Performance!!
Lecture20>Section2
Thislecture
Nextlecture
ANote:ACIDiscontentious!
• ManydebatesoverACID,bothhistoricallyand currently
• Manynewer“NoSQL”DBMSsrelaxACID
• Inturn,now“NewSQL”reintroducesACIDcompliancetoNoSQL-styleDBMSs…
Lecture20>Section2
ACIDisanextremelyimportant&successfulparadigm,butstilldebated!
Goalforthislecture:EnsuringAtomicity&Durability• Atomicity:
• TXNsshouldeitherhappencompletelyornotatall
• Ifabort/crashduringTXN,no effectsshouldbeseen
33
Lecture20>Section3>Motivation&basics
ACID
TXN1
TXN2
No changespersisted
All changespersisted
We’llfocusonhowtoaccomplishatomicity(vialogging)
Crash/abort
• Durability:• IfDBMSstopsrunning,changesduetocompletedTXNsshouldallpersist
• Juststoreonstabledisk
TheLog• Isalistofmodifications
• Logisduplexedandarchived onstablestorage.
• Canforcewrite entriestodisk• Apagegoestodisk.
• Alllogactivitieshandledtransparently theDBMS.
Assumewedon’tloseit!
Lecture20>Section3>Motivation&basics
BasicIdea:(Physical)Logging• RecordUNDOinformationforeveryupdate!
• Sequentialwritestolog• Minimalinfo(diff)writtentolog
• Thelog consistsofanorderedlistofactions• Logrecordcontains:
<XID,location,olddata,newdata>
ThisissufficienttoUNDOanytransaction!
Lecture20>Section3>Motivation&basics
Whydoweneedloggingforatomicity?• Couldn’twejustwriteTXNtodiskonly oncewholeTXNcomplete?
• Then,ifabort/crashandTXNnotcomplete,ithasnoeffect- atomicity!• Withunlimitedmemoryandtime,thiscouldwork…
• However,weneedtologpartialresultsofTXNs becauseof:• Memoryconstraints(enoughspaceforfullTXN??)• Timeconstraints(whatifoneTXNtakesverylong?)
Weneedtowritepartialresultstodisk!…Andsoweneedalog tobeabletoundo thesepartialresults!
Lecture20>Section3>Motivation&basics
3.Atomicity&DurabilityviaLogging
37
Lecture20>Section3
Whatyouwilllearnaboutinthissection
1. Logging:Ananimationofcommitprotocols
38
Lecture20>Section3
APictureofLogging
Lecture20>Section3>Loggingcommitprotocol
Apictureoflogging
DataonDisk
MainMemory
LogonDisk
LogT A=0
B=5
A=0
T:R(A),W(A)
Lecture20>Section3>Loggingcommitprotocol
Apictureoflogging
DataonDisk
MainMemory
LogonDisk
LogT A=1
B=5
A=0
T:R(A),W(A)A:0à1
Lecture20>Section3>Loggingcommitprotocol
Apictureoflogging
DataonDisk
MainMemory
LogonDisk
LogT A=1
B=5
A=0
T:R(A),W(A)A:0à1
Lecture20>Section3>Loggingcommitprotocol
Operationrecordedinloginmainmemory!
Whatisthecorrectwaytowritethisalltodisk?• We’lllookattheWrite-AheadLogging(WAL)protocol
• We’llseewhyitworksbylookingatotherprotocolswhichareincorrect!
43
Remember:Keyideaistoensuredurabilitywhilemaintainingourabilityto“undo”!
Write-AheadLogging(WAL)TXNCommitProtocol
Lecture20>Section3>Loggingcommitprotocol
TransactionCommitProcess
1. FORCEWritecommit recordtolog
2. AlllogrecordsuptolastupdatefromthisTXareFORCED
3. Commit()returns
Transactioniscommittedoncecommitlogrecordisonstablestorage
Lecture20>Section3>Loggingcommitprotocol
IncorrectCommitProtocol#1
DataonDisk
MainMemory
LogonDisk
LogT A=1
B=5
A=0
T:R(A),W(A) A:0à1
Lecture20>Section3>Loggingcommitprotocol
Let’strycommittingbefore we’vewritteneitherdataorlogtodisk…
Ifwecrashnow,isTdurable?
OK,Commit!
LostT’supdate!
IncorrectCommitProtocol#2
DataonDisk
MainMemory
LogonDisk
LogT A=1
B=5
A=0
T:R(A),W(A) A:0à1
Lecture20>Section3>Loggingcommitprotocol
Let’strycommittingafter we’vewrittendatabutbefore we’vewrittenlogtodisk…
Ifwecrashnow,isTdurable?Yes!Except…
OK,Commit!
HowdoweknowwhetherTwascommitted??
ImprovedCommitProtocol(WAL)
Lecture20>Section3>Loggingcommitprotocol
Write-aheadLogging(WAL)CommitProtocol
DataonDisk
MainMemory
LogonDisk
LogT A=1
B=5
A=0
T:R(A),W(A) A:0à1
Lecture20>Section3>Loggingcommitprotocol
Thistime,let’strycommittingafter we’vewrittenlogtodiskbutbefore we’vewrittendatatodisk…thisisWAL!
Ifwecrashnow,isTdurable?
OK,Commit!
Write-aheadLogging(WAL)CommitProtocol
DataonDisk
MainMemory
LogonDisk
T
A=0
T:R(A),W(A)
A:0à1
Lecture20>Section3>Loggingcommitprotocol
Thistime,let’strycommittingafter we’vewrittenlogtodiskbutbefore we’vewrittendatatodisk…thisisWAL!
Ifwecrashnow,isTdurable?
OK,Commit!
USETHELOG!A=1
Write-AheadLogging(WAL)
• DBusesWrite-AheadLogging(WAL) Protocol:
1. Mustforcelogrecord foranupdatebefore thecorrespondingdatapagegoestostorage
2. MustwritealllogrecordsforaTXbefore commit
Lecture20>Section3>Loggingcommitprotocol
Eachupdateislogged!Whynotreads?
à Atomicity
à Durability
LoggingSummary
• IfDBsaysTXcommits,TXeffectremains afterdatabasecrash
• DBcanundoactionsandhelpuswithatomicity
• Thisisonlyhalfthestory…
Lecture20>Section3>Loggingcommitprotocol