Using Event-Driven Architectures with Cassandra

@petabridge Petabridge.com

Event-Driven Architectures with Cassandra

Aaron Stannard, CTO & Cofounder PetabridgeDataStax MVP 2015

http://twitter.com/petabridge

http://petabridge.com/



Crash Course in EDA





Successful Product

30,000,000devices permonth

500 mbs of eventdata per second

100,000,000db transactionsper day

30 servers





Struggling Business

12 months ofrunway

3 engineers

0engineersw/prior distributedsystems experience





A Radical New Product1. IntegrateMarkedUpinto your application.

Your Installer

Notification Client

Your ApplicationAnalytics SDK

2. Users install your appandMarkedUp collects events.

3. Create campaigns basedon events and user-data.

4.MarkedUp delivers personalized, brandedmessages to users instantly via push,based on their in-app activity.





Allow our customers to start conversations with specific types of users.





Had to be done in real-time for best results.





Prototype: HTTP + Database





(Read after Write)





TheoryUSER

HTTP LOAD BALANCER

ASP.NETMVC(IIS)

CASSANDRACLUSTER

Events 0...3

0

0

1

0 10,1

2

20,1,2

3

30,1,2,3

MESSAGE





Reality

HTTP LOAD BALANCER

ASP.NETMVC(IIS)

CASSANDRACLUSTER

USER

Events 0...3

0 3

0 ??3??

1 2

1??2 ??





Breakthrough!

USER

HTTP LOAD BALANCER

Events 0...3

0 12 3

MESSAGE





I had STATEFUL, REACTIVE, STREAM PROCESSING

problem!





Implementation: HTTP -> EDA -> C*

C* ClusterPrivateVPCZone

HTTP Load Balancer

HTTP Load Balancer

API

TargetingSystem

Win32 MailboxService

CQL Writes

CQL R/W

CQL Reads

Akka.NETremoting(TCP +protobufs)





Why Do We Care About EDA?





Databases Aren’t Magical

HTTP LOAD BALANCER

ASP.NETMVC(IIS)

CASSANDRACLUSTER

USER

Events 0...3

0 3

0 ??3??

1 2

1??2 ??





Event-Driven Architecture Concepts





Key Terms• Event – a significant change in

“state”• Message – a notification of an event• Emitter – detect, gather, transfer

events• Sinks – react to events immediately• Channels – conduit between

emitters and sinks





Goals & Benefits• Extreme decoupling

• Easily distributed

• Inherently asynchronous and concurrent





EDA Styles• Simple Event Processing

• Event Stream Processing

• Complex Event Processing (CEP)





Event Stream Processing w/ FSMs





State A

State B

State C

Event 0

Stay

Event 1

Transition

Event 2

Transition

Event 3

REACT





FSM0

C*

C*

C*

C* C*

Events 0..2

Transition

Flush

FSM 0 State

FSM 0 State FSM 0 State





C*

C*

C*

C* C*

FSM 0 State

FSM 0 State FSM 0 State

FSM0

Crash

FSM0

RecreateFSMRequestState

ReplayState





CQL SchemaCREATE TABLE IF NOT EXISTS FSMState ( persistence_id text, state_id int, state_data blob, PRIMARY KEY (persistence_id, state_id))





Event Sourcing with C*





C*

RoomMessageHistoryActor

Message 1

Message 2

Message 3

Chat History forUser X

1. Build history,persist to store

History

Message 1Message 2Message 3

2. Requesthistory

3. Deliver historyas materialized view

4. Present to user





C*

RoomMessageHistoryActor

1. Startsafter application

restart

2. Replays storedmessages from

C*Message 1

Message 2

Message 3

History

Message 1Message 2Message 3

3. Rebuildsin-memory

history





EDA + C* Tips and Tricks• Large number of small sinks works

best• Define simple, reusable journals and

snapshots• C* == durable backup, App = single

source of truth • TTL everywhere your business

domain supports it





Have questions? Ask us!





http://petabridge.com/datastax

Using Event-Driven Architectures with Cassandra

Technology

Transcript of Using Event-Driven Architectures with Cassandra