Time series with Apache Cassandra - Long version

©2013 DataStax Confidential. Do not distribute without consent.

@PatrickMcFadin

Patrick McFadinChief Evangelist

Time Series with Apache Cassandra

1

Quick intro to Cassandra• Shared nothing •Masterless peer-to-peer • Based on Dynamo

Scaling• Add nodes to scale •Millions Ops/s Cassandra HBase Redis MySQL

THRO

UG

HPU

T O

PS/S

EC)

Uptime• Built to replicate • Resilient to failure • Always on

NONE

Easy to use• CQL is a familiar syntax • Friendly to programmers • Paxos for locking

CREATE TABLE users (! username varchar,! firstname varchar,! lastname varchar,! email list<varchar>,! password varchar,! created_date timestamp,! PRIMARY KEY (username)!);

INSERT INTO users (username, firstname, lastname, ! email, password, created_date)!VALUES ('pmcfadin','Patrick','McFadin',! ['[email protected]'],'ba27e03fd95e507daf2937c937d499ab',! '2011-06-20 13:50:00');!

INSERT INTO users (username, firstname, ! lastname, email, password, created_date)!VALUES ('pmcfadin','Patrick','McFadin',! ['[email protected]'],! 'ba27e03fd95e507daf2937c937d499ab',! '2011-06-20 13:50:00')!IF NOT EXISTS;

mailto:[email protected]

mailto:[email protected]

Time series in production• It’s all about “What’s happening” • Data is the new currency

Stack Driver• AWS and Rackspace monitoring • Quick indexes • Batch rollup results

MyDrive•Moved from Mongo to Cassandra • Queue processing • Bound at the storing data

“One thing that is not at all obvious from the graph is that the system was also under massively heavier strain after the switch to Cassandra because of additional bulk processing going on in the background.” - Karl Matthias, MyDrive

Paddy Power• Real-time product and pricing •Much like stock tickers • Active-active across two data

centers

“Specifically for Cassandra and Datastax, the ability to process time-series data is something that lots of companies have done in the past, not something that we were very aware of, and that was one of the reasons why we chose this as the first use case for Cassandra.” - John Turner, Paddy Power

Internet Of Things• 15B devices by 2015 • 40B devices by 2020!

Why Cassandra for Time Series

ScalesResilientGood data modelEfficient Storage Model

What about that?

Example 1: Weather Station•Weather station collects data • Cassandra stores in sequence • Application reads in sequence

Use case

• Store data per weather station • Store time series in order: first to last

• Get all data for one weather station • Get data for a single date and time • Get data for a range of dates and times

Needed Queries

Data Model to support queries

Data Model•Weather Station Id and Time

are unique • Store as many as needed

CREATE TABLE temperature ( weatherstation_id text, event_time timestamp, temperature text, PRIMARY KEY (weatherstation_id,event_time) );

INSERT INTO temperature(weatherstation_id,event_time,temperature) VALUES ('1234ABCD','2013-04-03 07:01:00','72F'); !INSERT INTO temperature(weatherstation_id,event_time,temperature) VALUES ('1234ABCD','2013-04-03 07:02:00','73F'); !INSERT INTO temperature(weatherstation_id,event_time,temperature) VALUES ('1234ABCD','2013-04-03 07:03:00','73F'); !INSERT INTO temperature(weatherstation_id,event_time,temperature) VALUES ('1234ABCD','2013-04-03 07:04:00','74F');

Storage Model - Logical View

2013-04-03 07:01:00

72F

2013-04-03 07:02:00

73F

2013-04-03 07:03:00

73F

SELECT weatherstation_id,event_time,temperature FROM temperature WHERE weatherstation_id='1234ABCD';

1234ABCD

1234ABCD

1234ABCD

weatherstation_id event_time temperature

2013-04-03 07:04:00

74F1234ABCD

Storage Model - Disk Layout

2013-04-03 07:01:00

72F

2013-04-03 07:02:00

73F

2013-04-03 07:03:00

73F1234ABCD

2013-04-03 07:04:00

74F

SELECT weatherstation_id,event_time,temperature FROM temperature WHERE weatherstation_id='1234ABCD';

Merged, Sorted and Stored Sequentially

2013-04-03 07:05:00 !!74F

2013-04-03 07:06:00 !!75F

Query patterns• Range queries • “Slice” operation on disk

SELECT weatherstation_id,event_time,temperature FROM temperature WHERE weatherstation_id='1234ABCD' AND event_time >= '2013-04-03 07:01:00' AND event_time <= '2013-04-03 07:04:00';

2013-04-03 07:01:00

72F

2013-04-03 07:02:00

73F

2013-04-03 07:03:00

73F1234ABCD

2013-04-03 07:04:00

74F

2013-04-03 07:05:00 !!74F

2013-04-03 07:06:00 !!75F

Single seek on disk

Query patterns• Range queries • “Slice” operation on disk

SELECT weatherstation_id,event_time,temperature FROM temperature WHERE weatherstation_id='1234ABCD' AND event_time >= '2013-04-03 07:01:00' AND event_time <= '2013-04-03 07:04:00';

2013-04-03 07:01:00

72F

2013-04-03 07:02:00

73F

2013-04-03 07:03:00

73F

1234ABCD

2013-04-03 07:04:00

74F

weatherstation_id event_time temperature

1234ABCD

1234ABCD

1234ABCD

Programmers like this

Sorted by event_time

Additional help on the storage engine

SSTable seeks• Each read minimum

1 seek • Cache and bloom

filter help minimize

Total seek time = Disk Latency * number of seeks

The key to speed

Use the first part of the primary key to get the node (data localization)

Minimize seeks for SStables (Bloom Filter,Key Cache up-to-date)

Find the data fast in the SSTable (Indexes)

Min/Max Value Hint•New since 2.0 • Range index on primary key values per SSTable •Minimizes seeks on range data

CASSANDRA-5514 if you are interested in details

SELECT temperature FROM event_time,temperature WHERE weatherstation_id='1234ABCD' AND event_time => '2013-04-03 07:01:00' AND event_time =< '2013-04-03 07:04:00';

Row Key: 1234ABCD Min event_time: 2013-04-01 00:00:00 Max event_time: 2013-04-04 23:59:59



?

This one

Ingestion models• Apache Kafka • Apache Flume • Storm • Spark Streaming • Custom Applications

Apache Kafka

Your totally!killer!application

Kafka + Storm• Kafka provides reliable queuing • Storm processes (rollups, counts) • Cassandra stores at the same speed • Storm lookup on Cassandra

Apache KafkaApache Storm

Queue Process Store

Flume• Source accepts data • Channel buffers data • Sink processes and stores • Popular for log processing

Sink

Channel

SourceApplication

Load Balancer

Syslog

Dealing with data at speed• 1 million writes per second? • 1 insert every microsecond • Collisions?

• Primary Key determines node placement • Random partitioning • Special data type - TimeUUID

Your totally!killer!application weatherstation_id='1234ABCD'

weatherstation_id='5678EFGH'

How does data replicate?

Primary key determines placement*

Partitioning

jim age: 36 car: camaro gender: M

carol age: 37 car: subaru gender: F

johnny age:12 gender: M

suzy age:10 gender: F

jim

carol

johnny

suzy

PK

5e02739678...

a9a0198010...

f4eb27cea7...

78b421309e...

MD5 Hash

MD5* hash operation yields a 128-bit number for keys of any size.

Key Hashing

Node A

Node D Node C

Node B

The Token Ring

jim 5e02739678...

carol a9a0198010...

johnny f4eb27cea7...

suzy 78b421309e...

Start EndA 0xc000000000..1 0x0000000000..0

B 0x0000000000..1 0x4000000000..0

C 0x4000000000..1 0x8000000000..0

D 0x8000000000..1 0xc000000000..0

Node A

Node D Node C

Node B

carol a9a0198010...

Replication

Node A

Node D Node C

Node B

carol a9a0198010...

ReplicationReplication factor = 3

Consistency is a different topic for later

TimeUUID

• Also known as a Version 1 UUID • Sortable • Reversible

Timestamp to Microsecond + UUID = TimeUUID

04d580b0-9412-11e3-baa8-0800200c9a66 Wednesday, February 12, 2014 6:18:06 PM GMT

http://www.famkruithof.net/uuid/uuidgen

=

Example 2: Financial Transactions• Trading of stocks •When did they happen? •Massive speeds and volumes

“Sirca, a non-profit university consortium based in Sydney, is the world’s biggest broker of financial data, ingesting into its database 2million pieces of information a second from every major trading exchange.”*

* http://www.theage.com.au/it-pro/business-it/help-poverty-theres-an-app-for-that-20140120-hv948.html

http://www.theage.com.au/it-pro/business-it/help-poverty-theres-an-app-for-that-20140120-hv948.html

Use case

• Store data per symbol and date • Store time series in reverse order: last to first •Make sure every transaction is unique

• Get all trades for symbol and day • Get trade for a single date and time • Get last 10 trades for symbol and date

Needed Queries

Data Model to support queries

Data Model

• date is int of days since epoch • timeuuid keeps it unique • Reverse the times for later

queries

CREATE TABLE stock_ticks ( symbol text, date int, trade timeuuid, trade_details text, PRIMARY KEY ((symbol, date), trade) ) WITH CLUSTERING ORDER BY (trade DESC);

INSERT INTO stock_ticks(symbol, date, trade, trade_details) VALUES (‘NFLX’,340,04d580b0-1431-1e33-baf8-0833200c98a6,'BUY:2000'); !INSERT INTO stock_ticks(symbol, date, trade, trade_details) VALUES (‘NFLX’,340,05d580b0-6472-1ef3-a3a8-0430200c9a66,'BUY:300'); !INSERT INTO stock_ticks(symbol, date, trade, trade_details) VALUES (‘NFLX’,340,02d580b0-9412-d223-55a8-0976200c9a25,'SELL:450'); !INSERT INTO stock_ticks(symbol, date, trade, trade_details) VALUES (‘NFLX’,340,08d580b0-4482-11e3-5fd3-3421200c9a65,'SELL:3000');

Storage Model - Logical View

08d580b0-4482-11e3-5fd3-3421200c9a65

SELL:3000

02d580b0-9412-d223-55a8-0976200c9a25

SELL:450

05d580b0-6472-1ef3-a3a8-0430200c9a66

BUY:300

SELECT trade,trade_details FROM stock_ticks WHERE symbol =‘NFLX’ AND date=‘340’;

NFLX:340

NFLX:340

NFLX:340

symbol:date trade trade_details

04d580b0-1431-1e33-baf8-0833200c98a6

BUY:2000NFLX:340

Last thing inserted

First thing inserted

04d580b0-1431-1e33-baf8-0833200c98a6

05d580b0-6472-1ef3-a3a8-0430200c9a66

02d580b0-9412-d223-55a8

BUY:2000BUY:300

08d580b0-4482-11e3-5fd3-3421200c9a65

SELL:3000 SELL:450

Storage Model - Disk Layout

NFLX:340

Order is from last trade to first

SELECT trade,trade_details FROM stock_ticks WHERE symbol =‘NFLX’ AND date=‘340’;

04d580b0-1431-1e33-baf8-0833200c98a6

05d580b0-6472-1ef3-a3a8-0430200c9a66

02d580b0-9412-d223-55a8-0976200c9a25

Query patterns• Limit queries • Get last X trades

From here

SELECT trade,trade_details FROM stock_ticks WHERE symbol =‘NFLX’ AND date=‘340’ LIMIT 3;

BUY:2000BUY:300

08d580b0-4482-11e3-5fd3-3421200c9a65

SELL:3000 SELL:450NFLX:340

to here

Query patterns

Reverse sorted by trade Last 3 trades

08d580b0-4482-11e3-5fd3-3421200c9a65

SELL:3000

02d580b0-9412-d223-55a8-0976200c9a25

SELL:450

05d580b0-6472-1ef3-a3a8-0430200c9a66

BUY:300

NFLX:340

NFLX:340

NFLX:340

symbol:date trade trade_details

• Limit queries • Get last X trades

SELECT trade,trade_details FROM stock_ticks WHERE symbol =‘NFLX’ AND date=‘340’ LIMIT 3;

Way more examples

• 5 minute interviews • Use cases • Free training!

!www.planetcassandra.org

Thank You!

Follow me for more updates all the time: @PatrickMcFadin

Time series with Apache Cassandra - Long version

Technology

Transcript of Time series with Apache Cassandra - Long version