The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase...
Transcript of The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase...
![Page 1: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/1.jpg)
The NoSQL Ecosystem,Relaxed Consistency, and
Snoop Dogg
Adam MarcusMIT CSAIL
[email protected] / @marcua
![Page 2: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/2.jpg)
About Me
● Social Computing + Database Systems● Easily Distracted: Wrote The NoSQL Ecosystem in
The Architecture of Open Source Applications 1
1 http://www.aosabook.org/en/nosql.html
![Page 3: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/3.jpg)
Preliminaries
● Almost always, use PostgreSQL or MySQL● Most problems can be solved by a single (large)
machine● Consider paying a DB vendor to solve problem
![Page 4: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/4.jpg)
But sometimes, need >1 machines
● Analytics on MapReduce or Column Stores● Facebook famously stores 1B+ users on ~10K
machines
![Page 5: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/5.jpg)
Scaling transactional workloads
● Google spends thousands of person-hours on Megastore/Spanner
● Everyone else jumps through hoops to stick to “relational model” and “SQL”● Partition data to avoid multinode transactions (2PC)● Avoid multirow transactions to prevent locking
![Page 6: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/6.jpg)
History, Compressed
Late 1990s Today
![Page 7: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/7.jpg)
History, Compressed
Late 1990s Today
Buy RAM
![Page 8: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/8.jpg)
History, Compressed
Late 1990s
Wrap RDBMSs
Today
Buy RAM
![Page 9: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/9.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM
![Page 10: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/10.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM
Not Only SQL
![Page 11: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/11.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM Commercialize
![Page 12: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/12.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM Commercialize
![Page 13: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/13.jpg)
The List, So You Don't Yell at Me
Cassandra
HBase
Voldemort
Riak
RedisMongoDB
HyperTable
Neo4j
HyperGraphDBDEX
InfoGrid
VertexDB
Sones
CouchDB
AllegroGraph
BerkeleyDB
Oracle NoSQLMemcacheDBTokyo Cabinet
FlockDB
![Page 14: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/14.jpg)
The List, So You Don't Yell at Me
Cassandra
HBase
Voldemort
Riak
RedisMongoDB
HyperTable
Neo4j
HyperGraphDBDEX
InfoGrid
VertexDB
Sones
CouchDB
AllegroGraph
BerkeleyDB
Oracle NoSQLMemcacheDBTokyo Cabinet
FlockDB
Marcus' Law of Databases:
The number of persistence options doubles every 1.5 years
![Page 15: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/15.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 16: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/16.jpg)
BigTable
● Google, 2006● Column store with sloppy schemas● Strong consistency● Open source: HBase
![Page 17: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/17.jpg)
Dynamo
● Amazon, 2007● Key-value store● Allows eventual consistency● Open source: Voldemort, Riak
![Page 18: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/18.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 19: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/19.jpg)
understanding NoSQL=
understanding a long series of tradeoffs
![Page 20: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/20.jpg)
Standards-compliant SQL Systems
● Relational model● Powerful query language● Transactional semantics● Predefined schemas● Strong consistency between replicas*
![Page 21: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/21.jpg)
Standards-compliant SQL Systems
● Relational model● Powerful query language● Transactional semantics● Predefined schemas● Strong consistency between replicas*
NoSQL: Maybe you don't need all of these?
![Page 22: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/22.jpg)
NoSQL Systems are a Buffet
![Page 23: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/23.jpg)
NoSQL Systems are a Buffet(of progressively larger grenades)
![Page 24: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/24.jpg)
NoSQL Systems are a Buffet(of progressively larger grenades)
● Data model● Query model● Durability● Transactional consistency● Partitioning● Replica consistency
![Page 25: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/25.jpg)
Prof. Madden will not quiz you on specific features of NoSQL systems
(i.e., don't memorize that MongoDB is a document store)
![Page 26: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/26.jpg)
Data Model
● Usually key-based...● Binary blob: Voldemort● Documents: MongoDB, CouchDB, Riak● Data structures: Redis● Column-families: HBase, Cassandra
![Page 27: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/27.jpg)
Data Model
● Usually key-based...● Binary blob: Voldemort● Documents: MongoDB, CouchDB, Riak● Data structures: Redis● Column-families: HBase, Cassandra
● ...but not always● Graph stores
![Page 28: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/28.jpg)
Query Model
● Redis: data structure-specific operations● CouchDB, Riak: MapReduce● Cassandra, MongoDB: SQL-like languages, no
joins or transactions
![Page 29: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/29.jpg)
Query Model
● Redis: data structure-specific operations● CouchDB, Riak: MapReduce● Cassandra, MongoDB: SQL-like languages, no
joins or transactions● Third-party
● High-level: PigLatin, HiveQL● Library: Cascading, Crunch● Streaming: Flume, Kafka, S4, Scribe
![Page 30: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/30.jpg)
Transactions
● Full ACID for single key● Redis: multi-key single-node transactions
![Page 31: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/31.jpg)
demo!
![Page 32: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/32.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 33: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/33.jpg)
Single-server durability
● Memory only: memcached● Single-server durability: the rest
● fsync every N seconds: most● Write-ahead logging: Cassandra, HBase, Redis, Riak● Group commit: Cassandra, HBase/HDFS
![Page 34: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/34.jpg)
When one server is not enough
● Replicate
● Partition
![Page 35: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/35.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition
![Page 36: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/36.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition● Vertical● Horizontal
![Page 37: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/37.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition● Vertical● Horizontal
Consistency/Availability
![Page 38: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/38.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition● Vertical● Horizontal Partitioning Scheme
Consistency/Availability
![Page 39: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/39.jpg)
Eventual vs. Strong Consistency: FIGHT!
![Page 40: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/40.jpg)
Eventual vs. Strong Consistency: FIGHT!
N = # replicasW = # write acknowledgementsR = # read acknowledgements
![Page 41: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/41.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Sally:100K
Sally:100K
Sally:100K
![Page 42: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/42.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 43: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/43.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 44: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/44.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 45: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/45.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
![Page 46: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/46.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
![Page 47: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/47.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
Sally's Salary?
?
![Page 48: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/48.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
Sally's Salary?
100K
![Page 49: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/49.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 50: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/50.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 51: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/51.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:100K
Sally:100K
Sally:100K
120K
Waste of time,Waste of time,Waste of time
![Page 52: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/52.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:100K
Sally:100K
Sally:100K
120K
![Page 53: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/53.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:120K
Sally:120K
Sally:120K
![Page 54: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/54.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:120K
Sally:120K
Sally:120K
Sally's Salary?
?
![Page 55: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/55.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:120K
Sally:120K
Sally:120K
Sally's Salary?
120K
![Page 56: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/56.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 57: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/57.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 58: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/58.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
![Page 59: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/59.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
![Page 60: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/60.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
Sally's Salary?
?
![Page 61: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/61.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
100K120K120K
QUORUM!
![Page 62: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/62.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
100K120K120K
![Page 63: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/63.jpg)
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
![Page 64: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/64.jpg)
N = 3, W = 3, R = 1
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
Consistent, unavailable writes on failure
![Page 65: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/65.jpg)
N = 3, W = 2, R = 2
N = 3, W = 3, R = 1
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
Consistent, unavailable writes on failure
Available, consistent with 1 failure
![Page 66: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/66.jpg)
N = 3, W = 2, R = 2
N = 3, W = 3, R = 1
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
Consistent, unavailable writes on failure
Available, consistent with 1 failure
![Page 67: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/67.jpg)
Consistency
![Page 68: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/68.jpg)
Consistency (not aCid)
![Page 69: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/69.jpg)
Consistency (not aCid)
![Page 70: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/70.jpg)
Consistency
● Strong: Appears that all replicas see all writes● Eventual: If left alone, replicas eventually converge● Weak: Replicas have different, divergent versions
![Page 71: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/71.jpg)
Consistency
● Strong: Appears that all replicas see all writes● Eventual: If left alone, replicas eventually converge● Weak: Replicas have different, divergent versions
The magicalCAP theorem
line
![Page 72: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/72.jpg)
Consistency math
![Page 73: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/73.jpg)
Consistency math
Strong: N < R + W
![Page 74: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/74.jpg)
Consistency math
Strong: N < R + W Easy: W = N Tricky: W < N
![Page 75: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/75.jpg)
Consistency math
Strong: N < R + W Easy: W = N Tricky: W < NEventual/Weak: N >= R + W
![Page 76: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/76.jpg)
Consistency Options
● BigTable (HBase): Strong● Dynamo (Voldemort/Riak/Cassandra):
tunable strong or eventual
![Page 77: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/77.jpg)
Consistency Options
● BigTable (HBase): Strong● Dynamo (Voldemort/Riak/Cassandra):
tunable strong or eventual
● ...and many others (Yahoo! PNUTs has timeline consistency)
![Page 78: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/78.jpg)
How do we getconsistent (eventually)?
Sally:120K
Sally:120K
Sally:100K
Sally:120K
Sally:120K
Sally:120K
![Page 79: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/79.jpg)
How does Dynamo go from weak to eventual consistency?
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
![Page 80: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/80.jpg)
Version w/ vector clocks
Sally:120K
Sally:100K
?
![Page 81: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/81.jpg)
How replicas get out of sync
![Page 82: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/82.jpg)
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 83: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/83.jpg)
Sally:120K
Sally:120K
Sally:100K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 84: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/84.jpg)
Sally:120K
Sally:120K
Sally:100K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 85: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/85.jpg)
Sally:120K
Sally:120K
Sally:100K
130K
N = 3, W = 1, R = 1
How replicas get out of sync
MOAR RAISE!
![Page 86: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/86.jpg)
Sally:120K
Sally:120K
Sally:130K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 87: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/87.jpg)
Sally:120K
Sally:120K
Sally:130K
N = 3, W = 1, R = 1
How replicas get out of sync
Sally Salary?
?
![Page 88: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/88.jpg)
Simple versioning doesn't work!
Sally:100K
Sally:100K
Sally:100K
Version: 20 Version: 20 Version: 20
![Page 89: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/89.jpg)
Sally:120K
Sally:120K
Sally:100K
Version: 21 Version: 21 Version: 20
Simple versioning doesn't work!
![Page 90: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/90.jpg)
Sally:120K
Sally:120K
Sally:130K
Version: 21 Version: 21 Version: 21
Simple versioning doesn't work!
![Page 91: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/91.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
![Page 92: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/92.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:100K
Sally:100K
Sally:100K
Version:<1,1,1>
Version:<1,1,1>
Version:<1,1,1>
![Page 93: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/93.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:100K
Sally:100K
Sally:100K
Version:<1,1,1>
Version:<1,1,1>
Version:<1,1,1>
120K
![Page 94: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/94.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:120K
Sally:100K
Sally:100K
Version:<2,1,1>
Version:<1,1,1>
Version:<1,1,1>
120K
![Page 95: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/95.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:120K
Sally:120K
Sally:100K
Version:<2,1,1>
Version:<2,1,1>
Version:<1,1,1>
120K
120K
![Page 96: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/96.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:120K
Sally:120K
Sally:100K
Version:<2,1,1>
Version:<2,1,1>
Version:<1,1,1>
120K
120K
![Page 97: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/97.jpg)
Read repair
● Client reads a value from multiple replicas● Compares vector clocks
● <2,1,1> = <2,1,1>
(in sync)● <2,1,1> > <1,1,1>
(update <1,1,1> before conflict)● <2,1,1> !! <1,1,2>
(conflict)
![Page 98: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/98.jpg)
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
How does Dynamo go from weak to eventual consistency?
![Page 99: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/99.jpg)
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
Proactivelystay in sync
How does Dynamo go from weak to eventual consistency?
![Page 100: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/100.jpg)
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
Proactivelystay in sync
Track replicassymmetrically
How does Dynamo go from weak to eventual consistency?
![Page 101: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/101.jpg)
Replication
● Consistency● Availability● Playing catchup
![Page 102: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/102.jpg)
Partitioning
● Consistent hashing: Voldemort, Riak● Range partitioning: HBase, MongoDB● Both: Cassandra
![Page 103: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/103.jpg)
Your partitioning scheme matters
● Partition so all requests go to one machine● Ideally, all joins happen on one machine● Example: map all GMail for a user to one machine
![Page 104: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/104.jpg)
Your partitioning scheme matters
● Partition so all requests go to one machine● Ideally, all joins happen on one machine● Example: map all GMail for a user to one machine
● Highly networked data (Facebook News) hard to partition: roughly two options● Each request floods many partitions● Denormalize data (replicate), partition by user
![Page 105: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/105.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 106: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/106.jpg)
NoSQL Use-cases
● Cassandra at Netflix● HBase at Facebook● MongoDB at Craigslist
![Page 107: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/107.jpg)
Cassandra
● BigTable data model: key column family● Dynamo sharding model: consistent hashing● Eventual or strong consistency
![Page 108: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/108.jpg)
Cassandra at Netflix
● Transitioned from Oracle● Store customer profiles, customer:movie watch
log, and detailed usage logging
Note: no multi-record locking (e.g., bank transfer)
In-datacenter: 3 replicas, per-app consistency
read/write quorum = 1, ~1ms latency
read/write quorum = 2, ~3ms latency
“Replicating Datacenter Oracle with Global Apache Cassandra on AWS” by Adrian Cockcroft http://www.slideshare.net/adrianco/migrating-netflix-from-oracle-to-global-cassandra
![Page 109: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/109.jpg)
Cassandra at Netflix
● Transitioned from Oracle● Store customer profiles, customer:movie watch
log, and detailed usage logging ● In-datacenter: 3 replicas, per-app consistency
“Replicating Datacenter Oracle with Global Apache Cassandra on AWS” by Adrian Cockcroft http://www.slideshare.net/adrianco/migrating-netflix-from-oracle-to-global-cassandra
![Page 110: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/110.jpg)
Cassandra at Netflix (cont'd)
● Benefit: async inter-datacenter replication● Benefit: no downtime for schema changes● Benefit: hooks for live backups
![Page 111: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/111.jpg)
HBase
● Data model: key column family● Sharding model: range partitioning● Strong consistency
Applications
Logging events/crawls, storing analytics
Twitter: replicate data from MySQL, Hadoop analytics
Facebook Messages
![Page 112: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/112.jpg)
HBase
● Data model: key column family● Sharding model: range partitioning● Strong consistency
● Applications● Logging events/crawls, storing analytics● Twitter: replicate data from MySQL, Hadoop analytics● Facebook Messages
![Page 113: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/113.jpg)
HBase for Facebook Messages
● Cassandra/Dynamo eventual consistency was difficult to program against
Benefit: simple consistency model
Benefit: flexible data model
Benefit: simple sharding, load balancing, replication
![Page 114: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/114.jpg)
HBase for Facebook Messages
● Cassandra/Dynamo eventual consistency was difficult to program against
● Benefit: simple consistency model● Benefit: flexible data model● Benefit: simple partitioning, load balancing,
replication
![Page 115: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/115.jpg)
MongoDB
● Document-based data model● Range-based partitioning● Consistency depends on how you use it
![Page 116: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/116.jpg)
MongoDB: Two use-cases
● Archiving at Craigslist● 2.2B historical posts, semi-structured● Relatively large blobs: avg 2KB, max > 4 MB
![Page 117: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/117.jpg)
MongoDB: Two use-cases
● Archiving at Craigslist● 2.2B historical posts, semi-structured● Relatively large blobs: avg 2KB, max > 4 MB
● Checkins at Foursquare● Geospatial indexing● Small location-based updates, partitioned on user
![Page 118: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/118.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 119: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/119.jpg)
Polyglot Persistence
● Account data in MySQL● Cache in memcached● Counters + queues in redis● Statistics in MongoDB● Event logging in HDFS/HBase/Cassandra
![Page 120: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/120.jpg)
When you see “NoSQL,” think “tradeoffs”
● Data model● Query Model● Transactions● Consistency vs. Availability● Partitioning schemes
Adam [email protected]
@marcua