Using chained transactions for maximum concurrency under load (QCONSF 2010)
-
Upload
billy-newport -
Category
Technology
-
view
4.024 -
download
3
description
Transcript of Using chained transactions for maximum concurrency under load (QCONSF 2010)
![Page 1: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/1.jpg)
@billynewport 1
Thoughts on Transactions
Chained transactions for scaling ACID@billynewport
![Page 2: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/2.jpg)
@billynewport 2
AgendaExplain scenario
Implement using a single database
Explain concurrency issues under load
Implement using a sharded database
Implement using WebSphere eXtreme Scale and chained transactions
![Page 3: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/3.jpg)
@billynewport 3
ScenarioLarge eCommerce web site
Problem is order checkout
We track inventory levels for each SKU
Orders during checkout need to adjust available inventory.
![Page 4: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/4.jpg)
@billynewport 4
Shopping cart metricsMillions of SKUs
Cart size of 5 items for electronics/big ticket items
Cart size of 20 items for clothing
Expect concurrent load of 2500 checkouts per second
![Page 5: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/5.jpg)
@billynewport 5
DatabaseBegin
For each item in cartSelect for update where sku = item.skuDecrement available sku levelIf not available then rollback…Update level where sku = item.sku
Commit
Cart items randomly distributed amongst all 2m items, lots of concurrency.
Simple enough, right? All is good?
![Page 6: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/6.jpg)
@billynewport 6
Problem: cabbage patch dolls
Cabbage patch dolls are popular this fall…
![Page 7: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/7.jpg)
@billynewport 7
Database killers!The dolls cause major
concurrency problems Lots of row level locks Contention on doll rows Possible table lock escalation App server thread issues Connection pools empty Then DEATH!
They aren’t sweet and cuddly any more…
![Page 8: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/8.jpg)
@billynewport 8
Database killersWe need a way to get locks to decrement
inventory
But, we don’t want to hold the lock for very long
Bigger carts make the problem worse, all the locks held for longer
Ideally, hold locks for constant time
Any contentious items make problem worse
![Page 9: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/9.jpg)
@billynewport 9
SolutionHold lock on inventory rows for as short a time
as possible
Decouple this from size of cart.
How?
![Page 10: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/10.jpg)
@billynewport 10
Chained transactions
Programmers think of transactions in synchronous terms.Begin / Do Work / Hold locks / Commit
Easy to program, bad for concurrency.
![Page 11: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/11.jpg)
@billynewport 11
InspirationMicrosoft had COM objects with apartment
model threading.
Modern Actor support is similar. Some state with a mailbox.
BPEL supports flows with compensation
Data meets actors is a good analogySend a message (cart) to a group of actors
identified using their keys with a compensator
![Page 12: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/12.jpg)
@billynewport 12
AlternativeWe need to think asynchronously in terms of flows
with compensation Map of <SKU Key/SKU Amount> Brick:
Do{ code to reduce inventory level for SKU }
Undo{ code to increase level inventory for SKU }
Provide Map with do/undo bricks
Easy to program, great concurrency.
![Page 13: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/13.jpg)
@billynewport 13
Transactions and sharded stores
Option 1: Write transaction to one shard then spread out asynchronously
Option 2: 2 phase commit
Option 3: Chained transactions
![Page 14: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/14.jpg)
@billynewport 14
Transactions and sharded stores
Option 1: Write transaction to one shard then spread out asynchronously
Option 2: 2 phase commit
Option 3: Chained transactions
![Page 15: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/15.jpg)
@billynewport 15
Transactions and sharded stores
Option 1: Write transaction to one shard then spread out asynchronously
Option 2: 2 phase commit
Option 3: Chained transactions
![Page 16: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/16.jpg)
@billynewport 16
Implementation1PC only required
Data store supportingRow locksRow oriented data
Integrated FIFO messaging
IBM WebSphere eXtreme Scale provides these capabilities.
![Page 17: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/17.jpg)
@billynewport 17
ImplementationApplication makes map and code bricks
Submits transaction as an asynchronous job.
Uses a Future to check on job outcome.
Do blocks can trigger flow reversal if a problem occurs. Invoke undo block for each completed step
![Page 18: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/18.jpg)
@billynewport 18
MechanismLoop
Receive message for actor keyProcess itSend modified cart to next ‘sku’ using local
‘transmit q’Commit transaction
Background thread pushes messages in transmit q to the destination shards using exactly once semantics.
![Page 19: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/19.jpg)
@billynewport 19
PerformanceA single logical transaction will be slower than a
1PC DBMS implementation.
However, under concurrent load then it will deliver:Higher throughputBetter response times
Thru better contention managementEach ‘SKU’ only locked for a very short period
![Page 20: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/20.jpg)
@billynewport 20
GeneralizationThis could be thought of as a workflow engine.
But, a big difference here is that a workflow engine usually talks with a remote store.
Here: the flow state is the MESSAGE It moves around to where the data is for the next step
Using a MESSAGE for flow state rather than a database means it scales linearly.
The message ‘store’ is integrated and scales with the data store.
![Page 21: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/21.jpg)
@billynewport 21
Architecture ComparisonConventional Message oriented
FlowDB
ApplDB
MsgStor
e
BPEngine
BPEngine
BPEngin
e
FlowState
Flow Edge
= Msg
IntegratedMsg/Data store
ApplDB
Write behind
IntegratedMsg/Data store
IntegratedMsg/Data store
![Page 22: Using chained transactions for maximum concurrency under load (QCONSF 2010)](https://reader035.fdocuments.us/reader035/viewer/2022062617/54bc36d34a79597d4d8b461e/html5/thumbnails/22.jpg)
@billynewport 22
Sample implementationComing soon.
Running in lab
Working with several eCommerce customers looking to implement soon.
Soon to be published on github as sample code.