London + Dublin Cassandra 2.0
-
Upload
jbellis -
Category
Technology
-
view
2.236 -
download
1
Transcript of London + Dublin Cassandra 2.0
![Page 1: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/1.jpg)
©2013 DataStax Confidential. Do not distribute without consent.
@spyced
Jonathan EllisCTO, DataStax / Project Chair, Apache Cassandra
Cassandra 2.0
![Page 2: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/2.jpg)
Five Years of Cassandra
Jul-09 May-10 Feb-11 Dec-11 Oct-12 Jul-13
0.1 0.3 0.6 0.7 1.0 1.2...
2.0
DSE
Jul-08
![Page 3: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/3.jpg)
Core values
0
20000
40000
60000
80000
0 2 4 6 8 10 12
Cassandra HBase Redis MySQL
•Massive scalablility•High performance • Reliability/Availability
![Page 4: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/4.jpg)
CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int);
CREATE INDEX ON users(state);
SELECT * FROM users WHERE state=‘Texas’ AND birth_date > 1950;
New Core Value
•Massive scalablility•High performance • Reliability/Availability• Ease of use
![Page 5: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/5.jpg)
*Key concepts?
*Data Modeling section of documentation: http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/ddl/ddl_anatomy_table_c.html
CQL delivers"Coming from a relational database background we foundthe transition to Cassandra to be very straightforward. There are afew simple key concepts one must grasp at first but ever since it'sbeen smooth sailing for us."
Boris Wolf, Comcast
![Page 6: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/6.jpg)
1.2 for Developers• CQL3• Thrift compatibility
• Collections• Data dictionary
• Auth support• Hadoop support
• Native drivers
• Tracing• Atomic batches
![Page 7: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/7.jpg)
[cassandra.yaml]authenticator: PasswordAuthenticator# DSE offers KerberosAuthenticator as well
CREATE USER robin WITH PASSWORD 'manager' SUPERUSER;
ALTER USER cassandra WITH PASSWORD 'newpassword';
LIST USERS;
DROP USER cassandra;
Authentication
![Page 8: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/8.jpg)
[cassandra.yaml]authorizer: CassandraAuthorizer
GRANT select ON audit TO jonathan;
GRANT modify ON users TO robin;
GRANT all ON ALL KEYSPACES TO lara;
Authorization
![Page 9: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/9.jpg)
Native drivers• CQL native protocol: efficient, lightweight, asynchronous• Java (GA): https://github.com/datastax/java-driver• .NET (Beta): https://github.com/datastax/csharp-driver• Python (Beta): https://github.com/datastax/python-driver• Coming soon: PHP, Ruby, others
![Page 10: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/10.jpg)
1.2 for Operators• Virtual nodes• “Dense node” support (5-10TB/machine)• JBOD improvements• Off-heap bloom filters, compression metadata• Parallel leveled compaction
![Page 11: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/11.jpg)
1.2.5+
![Page 12: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/12.jpg)
1.2.5+• ~1/2 memory usage in partition summary
![Page 13: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/13.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle
![Page 14: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/14.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle• Parallel leveled compaction
![Page 15: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/15.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle• Parallel leveled compaction• Removed cell-name bloom filters
![Page 16: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/16.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle• Parallel leveled compaction• Removed cell-name bloom filters• Thread-local allocation
![Page 17: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/17.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle• Parallel leveled compaction• Removed cell-name bloom filters• Thread-local allocation• LZ4 compression (default in 2.0)
![Page 18: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/18.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle• Parallel leveled compaction• Removed cell-name bloom filters• Thread-local allocation• LZ4 compression (default in 2.0)• (1.2.7) CQL Input/Output for Hadoop
![Page 19: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/19.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle• Parallel leveled compaction• Removed cell-name bloom filters• Thread-local allocation• LZ4 compression (default in 2.0)• (1.2.7) CQL Input/Output for Hadoop• (1.2.7) Range tombstone performance
![Page 20: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/20.jpg)
1.2.5+• ~1/2 memory usage in partition summary• Improved compaction throttle• Parallel leveled compaction• Removed cell-name bloom filters• Thread-local allocation• LZ4 compression (default in 2.0)• (1.2.7) CQL Input/Output for Hadoop• (1.2.7) Range tombstone performance• (1.2.9) Larger default LCS filesize (160MB > 5MB)
![Page 21: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/21.jpg)
Cassandra 2.0
![Page 22: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/22.jpg)
2.0• Lightweight transactions• Triggers (experimental)• Improved compaction• CQL cursors
![Page 23: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/23.jpg)
SELECT * FROM usersWHERE username = ’jbellis’
[empty resultset]
INSERT INTO users (...)VALUES (’jbellis’, ...)
Session 1SELECT * FROM usersWHERE username = ’jbellis’
[empty resultset]
INSERT INTO users (...)VALUES (’jbellis’, ...)
Session 2
Lightweight transactions: the problem
![Page 24: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/24.jpg)
Paxos• All operations are quorum-based• Each replica sends information about unfinished operations to the leader
during prepare• Paxos made Simple
![Page 25: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/25.jpg)
LWT: details• 4 round trips vs 1 for normal updates• Paxos state is durable• Immediate consistency with no leader election or failover• ConsistencyLevel.SERIAL• http://www.datastax.com/dev/blog/lightweight-transactions-in-
cassandra-2-0
![Page 26: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/26.jpg)
LWT: Use with caution• Great for 1% of your application• Eventual consistency is your friend• http://www.slideshare.net/planetcassandra/c-summit-2013-eventual-consistency-
hopeful-consistency-by-christos-kalantzis
![Page 27: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/27.jpg)
UPDATE USERS SET email = ’[email protected]’, ...WHERE username = ’jbellis’IF email = ’[email protected]’;
INSERT INTO USERS (username, email, ...)VALUES (‘jbellis’, ‘[email protected]’, ... )IF NOT EXISTS;
Using LWT
![Page 28: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/28.jpg)
TriggersCREATE TRIGGER <name> ON <table> USING <classname>;
![Page 29: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/29.jpg)
Trigger implementationclass MyTrigger implements ITrigger{ public Collection<RowMutation> augment(ByteBuffer key, ColumnFamily update) { ... }}
![Page 30: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/30.jpg)
Experimental!• Relies on internal RowMutation, ColumnFamily classes• [partition] key is a ByteBuffer• Expect changes in 2.1
![Page 31: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/31.jpg)
Compaction• Single-pass, always• LCS performs STCS in L0
![Page 32: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/32.jpg)
Healthy leveled compaction
![Page 33: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/33.jpg)
Sad leveled compaction
![Page 34: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/34.jpg)
STCS in L0
![Page 35: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/35.jpg)
Cursors (before)
SELECT *FROM timelineWHERE (user_id = :last_key AND tweet_id > :last_tweet) OR token(user_id) > token(:last_key)LIMIT 100
CREATE TABLE timeline ( user_id uuid, tweet_id timeuuid, tweet_author uuid, tweet_body text, PRIMARY KEY (user_id, tweet_id));
![Page 36: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/36.jpg)
Cursors (after)SELECT *FROM timeline
![Page 37: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/37.jpg)
Misc. performance improvements
![Page 38: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/38.jpg)
Misc. performance improvements• Tracking statistics on clustered columns allows eliminating unnecessary
sstables from the read path
![Page 39: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/39.jpg)
Misc. performance improvements• Tracking statistics on clustered columns allows eliminating unnecessary
sstables from the read path • New half-synchronous, half-asynchronous Thrift server based on LMAX
Disruptor
![Page 40: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/40.jpg)
Misc. performance improvements• Tracking statistics on clustered columns allows eliminating unnecessary
sstables from the read path • New half-synchronous, half-asynchronous Thrift server based on LMAX
Disruptor • Faster partition index lookups and cache reads by improving performance
of off-heap memory
![Page 41: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/41.jpg)
Misc. performance improvements• Tracking statistics on clustered columns allows eliminating unnecessary
sstables from the read path • New half-synchronous, half-asynchronous Thrift server based on LMAX
Disruptor • Faster partition index lookups and cache reads by improving performance
of off-heap memory• Faster reads of compressed data by switching from CRC32 to Adler
checksums
![Page 42: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/42.jpg)
Misc. performance improvements• Tracking statistics on clustered columns allows eliminating unnecessary
sstables from the read path • New half-synchronous, half-asynchronous Thrift server based on LMAX
Disruptor • Faster partition index lookups and cache reads by improving performance
of off-heap memory• Faster reads of compressed data by switching from CRC32 to Adler
checksums• JEMalloc support for off-heap allocation
![Page 43: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/43.jpg)
Spring cleaning
![Page 44: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/44.jpg)
Spring cleaning• Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema
![Page 45: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/45.jpg)
Spring cleaning• Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema • The potentially dangerous countPendingHints JMX call has been replaced
by a Hints Created metric
![Page 46: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/46.jpg)
Spring cleaning• Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema • The potentially dangerous countPendingHints JMX call has been replaced
by a Hints Created metric• The on-heap partition cache (“row cache”) has been removed
![Page 47: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/47.jpg)
Spring cleaning• Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema • The potentially dangerous countPendingHints JMX call has been replaced
by a Hints Created metric• The on-heap partition cache (“row cache”) has been removed• Vnodes are on by default
![Page 48: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/48.jpg)
Spring cleaning• Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema • The potentially dangerous countPendingHints JMX call has been replaced
by a Hints Created metric• The on-heap partition cache (“row cache”) has been removed• Vnodes are on by default
• the old token range bisection code for non-vnode clusters is gone
![Page 49: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/49.jpg)
Spring cleaning• Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema • The potentially dangerous countPendingHints JMX call has been replaced
by a Hints Created metric• The on-heap partition cache (“row cache”) has been removed• Vnodes are on by default
• the old token range bisection code for non-vnode clusters is gone• Removed emergency memory pressure valve logic
![Page 50: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/50.jpg)
Operational concerns
![Page 51: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/51.jpg)
Operational concerns• Java7 is now required!
![Page 52: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/52.jpg)
Operational concerns• Java7 is now required! • Leveled compaction level information has been moved into sstable
metadata
![Page 53: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/53.jpg)
Operational concerns• Java7 is now required! • Leveled compaction level information has been moved into sstable
metadata• Kernel page cache skipping has been removed in favor of optional row
preheating (preheat_kernel_page_cache)
![Page 54: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/54.jpg)
Operational concerns• Java7 is now required! • Leveled compaction level information has been moved into sstable
metadata• Kernel page cache skipping has been removed in favor of optional row
preheating (preheat_kernel_page_cache)• Streaming has been rewritten to be more transparent and robust.
![Page 55: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/55.jpg)
Operational concerns• Java7 is now required! • Leveled compaction level information has been moved into sstable
metadata• Kernel page cache skipping has been removed in favor of optional row
preheating (preheat_kernel_page_cache)• Streaming has been rewritten to be more transparent and robust.• Streaming support for old-version sstables
![Page 56: London + Dublin Cassandra 2.0](https://reader034.fdocuments.us/reader034/viewer/2022042714/555c270dd8b42a0b418b4d84/html5/thumbnails/56.jpg)
©2013 DataStax Confidential. Do not distribute without consent. 31