Return of the transaction king
-
Upload
ryan-knight -
Category
Technology
-
view
205 -
download
2
Transcript of Return of the transaction king
![Page 1: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/1.jpg)
Ryan Knight - @knight_cloud
Return of the Transaction King
![Page 2: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/2.jpg)
Brief History of Data
RDBMS + ACID Transactions
Atomic
Consistent
Isolated
Durable
![Page 3: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/3.jpg)
Traditional ACID Transactions
Did not Scale with the Internet
Leader w/ Read Replicas
Distributed Locks
![Page 4: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/4.jpg)
CAP Theorem Forced Choices
Global Scale or Strong Consistency?
Rise of Eventual Consistency
![Page 5: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/5.jpg)
Eventual Consistency
Really a Necessary Evil
Last Write Wins
Event Sourcing
Write now and figure it out later
![Page 6: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/6.jpg)
● DNS, IoT, Time Series, Monitoring, Web Metrics, Followers, etc.
● Modify Business to Fit - Allow for Inconsistency○ Inventory - Oversell and cancel order if
necessary○ Bank - Overdraft and charge
Where Event Sourcing Works
![Page 7: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/7.jpg)
We live in a stateful and finite world! - James Ward
But ….
![Page 8: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/8.jpg)
Where Event Sourcing BreaksWhen Order of Events Matters
Can not write from 2 hosts and guarantee ordering
Reconcile Diverging History of Events
Possible to Lose History
![Page 9: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/9.jpg)
Strong Consistency
![Page 10: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/10.jpg)
Differing Views of Strong Consistency
Write / Read Local Quorum
No Read before Write
No Compare and Swap
No Snapshot Isolation
![Page 11: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/11.jpg)
Write SkewTwo concurrent transactions each determine what they are writing based on reading a data set which overlaps what the other is writing
begriffs.com
![Page 12: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/12.jpg)
Can we Ignore Consistency?
Pushes consistency problem into the application where they are far harder to solve
Eventual Consistency really is a necessary Evil
![Page 13: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/13.jpg)
Consistency in the Application Tier
Events Sourcing w/ Command
Serialization
Serialization Points - Kafka
Consumers
What happens when they fail?
![Page 14: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/14.jpg)
Cloud Native / Microservices
Partitioning of the Application Domain and Related Data
Increased Coordination between Data
Number of Microservices Increases Number of Interactions
Exponential Growth in Complexity
![Page 15: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/15.jpg)
Cloud Native + NoSQL - ACID==========Rube Goldberg machine
![Page 16: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/16.jpg)
ACID Transactions
Atomic
Consistent / Correct
Isolated
Durable
![Page 17: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/17.jpg)
ACID TRANSACTIONS
+ EVENT SOURCING
============RETURN OF
TRANSACTIONS
![Page 18: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/18.jpg)
Modern Transactional Databases
Fauna
Google Spanner
CockroachDB
GLOBAL SCALE + STRONG CONSISTENCY
![Page 19: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/19.jpg)
Value of Transactions
Decrease Application Tier Complexity
Reduce Cognitive Overhead
Increased Developer Productivity
Increased Focus on Business Value
![Page 20: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/20.jpg)
● Deal with People's Money ● Financial Transactions● Inventory ● Stock Exchange● Auctions● Points
When do you need ACID TXN?
![Page 21: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/21.jpg)
Do we really have to sacrifice AP to get C?But there are two important caveats: first, you only need forfeit something during an actual partition, and even then there are many mitigations. Second, the actual theorem is about 100% availability, while the interesting discussion here is about the tradeoffs involved for realistic high availability.
Eric Brewer Spanner Paper
![Page 22: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/22.jpg)
Problems with CAP, and Yahoo’s little known NoSQL system - Daniel Abadi
Tradeoff between latency (L) and consistency under normal conditions
![Page 23: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/23.jpg)
Google Spanner
● Private global network along with GPS receivers and atomic clocks.
● Atomic Clocks for Serializability Guarantees● AdWords & Google Play for 5+ years● Google Cloud Platform Control Plane ● No Scheduled Downtime● Can do maintenance by moving data between nodes.
![Page 24: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/24.jpg)
CockroachDBHybrid Logical Clock similar to a vector clock for ordering of transactions
Challenges with clock skew
Waits up to 250 MS on reads
![Page 25: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/25.jpg)
What is Fauna?
Globally distributed, transactional database
Born out of Twitter and the Calvin Pape
Functional / Type-Safe Query Language
Indexes, Constraints and Views
![Page 26: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/26.jpg)
Fauna Transaction Resolution
Resolves transactions before committing to DB
Execute phase and Commit phase
All Transaction w/ in a epoch our ordered and applied as a batch
Uses a RAFT replicated log that derives a total order of all transactions.
![Page 27: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/27.jpg)
Distributed ACID Transaction Engine
Consensus protocols from Raft and Calvin Paper
Can span thousands of records and indexes
Designed for high throughput and WAN replication
Ensures multi-node and multi- datacenter correctness
![Page 28: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/28.jpg)
Security, Identity, IsolationNative Security and Access Control
Row-level security, identity, and isolation
Complete Data Isolation across datasets
![Page 29: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/29.jpg)
Quality of Service Managed Resource Scheduler
Operating System Like Design
Single Cluster for Differing and Competing Workloads
![Page 30: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/30.jpg)
Temporality
FaunaDB keeps all instances of a data, not overwriting them but creating new ones when a write is performed. This is useful especially when auditing data, verifying its evolution over time.
![Page 31: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/31.jpg)
Fauna Data RepresentationDatabase
Class(es)
Instanc(es)
Arrays and Scalar Types
Nested Data
Database
Database(s)
Class(es)
Class(es)
![Page 32: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/32.jpg)
InstancesEach record stored as an object.
All objects are instances of a specific class.
Every instance has an identifier called a ref.
An instance’s ref encodes its class along with a unique id
![Page 33: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/33.jpg)
FunctionalTransaction Scope
Not traditional begin / end transaction
Entire Function / Query Block is scope of transaction
Paradigm will feel natural to Functional Programmers
![Page 34: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/34.jpg)
Similar to Traditional RDBMS Indexes Terms and Values
Transform, cover, and order instance data
Enforce unique constraints.
Referenced explicitly in query expressions;
Optimizer does not make index applications
Indexes
![Page 35: Return of the transaction king](https://reader033.fdocuments.us/reader033/viewer/2022051521/5a6479217f8b9a2c568b4729/html5/thumbnails/35.jpg)
Example “Smart Contract”
● User Nodes w/ Simulated Account Balance
● Server Node acts as Intermediary and
Notary on Contract
● User Nodes Propose Borrowing from Peers
● Peer Accepts or Rejects Proposal
● Contract is Created between Peers Lending
Fixed Amount