Consistency Models in New Generation Databases
-
Upload
iammutex -
Category
Technology
-
view
3.407 -
download
3
description
Transcript of Consistency Models in New Generation Databases
![Page 2: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/2.jpg)
Thoughts on Transaction and
Consistency Models
![Page 3: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/3.jpg)
The database world is changingDocument Datastores, Key Value, Graph databases
![Page 4: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/4.jpg)
The database world is changingTransactional model
![Page 5: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/5.jpg)
The database world is changingFull Acid
![Page 6: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/6.jpg)
The database world is changing
![Page 7: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/7.jpg)
CAP It is impossible in the asynchronous network model to
implement a read/write data object that guarantees the following properties:• Availability• Atomic consistency in all fair executions (including those in which messages are lost).
![Page 8: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/8.jpg)
CAP It is impossible in the asynchronous network model to
implement a read/write data object that guarantees the following properties:• Availability• Atomic consistency in all fair executions (including those in which messages are lost).
Or: If the network is broken, your database won’t work
But we get to define “won’t work”
![Page 9: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/9.jpg)
Consistency Models - CAP
• Choices are Available-P (AP) or Consistent-P (CP)• Write Availability, not Read Availability, is the Main Question
• It’s not all about CAPScale, Reduced latency:
•Multi data center•Speed•Even load distribution
![Page 10: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/10.jpg)
Examples of Eventually Consistent Systems
• Eventual Consistency: •“The storage system guarantees that if no new updates are made to the object, eventually all accesses will return the last updated value”
•Examples:• DNS• Async replication (RDBMS, MongoDB)• Memcached (TTL cache)
![Page 11: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/11.jpg)
Eventual Consistency
![Page 12: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/12.jpg)
Eventual Consistency
Read(x) : 1, 2, 2, 4, 4, 4, 4 …
![Page 13: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/13.jpg)
Could we get this?
Read(x) : 1, 2, 1, 4, 2, 4, 4, 4 …
![Page 14: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/14.jpg)
Monotonic read consistency
• Appserver and slave on same box• Appserver only reads from Slave
• Eventually consistent• Guaranteed to see reads in order
• Failover ?
Prevent seeing writes out of order
![Page 15: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/15.jpg)
RYOW Consistency
Primary
Read / Has Written
Replication
Read
Secondary
• Read Your Own Writes
• Eventually consistent for readers• Immediately consistent for writers
![Page 16: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/16.jpg)
Amazon Dynamo
•R: # of servers to read from•W: # servers to get response from•N: Replication factor
• R+W>N has nice properties
![Page 17: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/17.jpg)
ExampleExample 1
R + W <= N
R = 1W = 1N = 5
Possibly Stale dataHigher availability
![Page 18: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/18.jpg)
ExampleExample 1
R + W <= N
R = 1W = 1N = 5
Possibly Stale dataHigher availability
Example 2
R + W > N
R = 2 R = 1W = 1 W = 2N = 2 N = 2
Consistent data
![Page 19: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/19.jpg)
R + W > N
If R+W > N, we can’t have both fast local reads and writes at the same time if all the data centers are equal peers
![Page 20: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/20.jpg)
Consistency levels
• Strong: W + R > N• Weak / Eventual : W + R <= N• Optimized Read: R=1, W=N• Optimized Write: W=1, R=N
![Page 21: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/21.jpg)
Multi Datacenter Strategies
• DR• Failover
• Single Region, Multi Datacenter• Useful for Strong or Eventual Consistency
• Local Reads, Remote Writes• All writes to master on WAN, EC on Slaves
• Intelligent Homing• Master copy close to user location
![Page 22: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/22.jpg)
Network Partitions
![Page 23: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/23.jpg)
Network Write Possibilities•Deny all writes
• Still Read fully consistent data • Give up write Availability
![Page 24: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/24.jpg)
Network Write Possibilities•Deny all writes
• Still Read fully consistent data • Give up write Availability
•Allow writes on one side• Failover• Possible to allow reads on other side
![Page 25: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/25.jpg)
Network Write Possibilities•Deny all writes
• Still Read fully consistent data • Give up write Availability
•Allow writes on one side• Failover• Possible to allow reads on other side
•Allow writes on both sides• Available • Give up consistency
![Page 26: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/26.jpg)
Multiple Writer Strategies• Last one wins
• Use Vector clocks to decide latest
• Insert Inserts often really means if ( !exists(x)) then set(x) exists is hard to implement in eventually consistent systems
![Page 27: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/27.jpg)
Multiple Writer Strategies• Delete op1: set( { _id : 'joe', age : 40 } } op2: delete( { _id : 'joe' } ) op3: set( { _id : 'joe', age : 33 } )
• Consider switching 2 and 3• Tombstone: Remember delete, and apply last-operation-wins
• Update update users set age=40 where _id=’joe’
However: op1: update users set age=40 where _id='joe' op2: update users set state='ca' where _id='joe'
![Page 28: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/28.jpg)
Multiple Writer Strategies• Programatic Merge
• Store operations, instead of state• Replay operations • Did you get the last operation ? • Not immediate
•Communtative operations• Conflict free? • Fold-able• Example: add, increment, decrement
![Page 29: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/29.jpg)
Trivial Network Partitions
![Page 30: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/30.jpg)
MongoDB Options
![Page 31: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/31.jpg)
MongoDB Transaction Support• Single Master / Sharded
• MongoDB Supports Atomic Operations on Single Documents• But not Rollback
• $ operators• $set, $unset, $inc, $push, $pushall, $pull, $pullall
• Update if current• find followed by update
• Find and modify
![Page 32: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/32.jpg)
MongoDB
Primary
Read / Write
![Page 33: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/33.jpg)
MongoDB
Primary
Secondary
Read / Write
Secondary for Backup
Replication
Replication
![Page 34: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/34.jpg)
MongoDB
Primary
Secondary
Read / Write
Secondary for Backup
Read Replication
Replication
![Page 35: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/35.jpg)
MongoDB
Primary
Secondary
Read / Write
Secondary for Backup
Read Replication
Replication
Replicaset
![Page 36: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/36.jpg)
Write Scalability: Sharding
write
read
ReplicaSet 1
Primary
Secondary
Secondary
ReplicaSet 2
Primary
Secondary
Secondary
ReplicaSet 3
Primary
Secondary
Secondary
key range 0 .. 30
key range 31 .. 60
key range 61 .. 100
![Page 37: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/37.jpg)
![Page 38: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/38.jpg)
Sometimes we need global state / more consistency
•Unique key constraints•User registration
•ACL changes
![Page 39: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/39.jpg)
Could it be the case that…
uptime( CP + average developer ) >= uptime( AP + average developer )
where uptime:= system is up and non-buggy?
![Page 40: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/40.jpg)
Thank You :-)@rogerb
![Page 41: Consistency Models in New Generation Databases](https://reader033.fdocuments.us/reader033/viewer/2022060110/555a3c59d8b42a83368b4cc5/html5/thumbnails/41.jpg)
@mongodb
conferences, appearances, and meetupshttp://www.10gen.com/events
http://bit.ly/mongoU Facebook | Twitter | LinkedIn
http://linkd.in/joinmongo
download at mongodb.org