Post on 11-Feb-2017
@petabridge Petabridge.com
Event-Driven Architectures with Cassandra
Aaron Stannard, CTO & Cofounder PetabridgeDataStax MVP 2015
@petabridge Petabridge.com
Crash Course in EDA
@petabridge Petabridge.com
Successful Product
30,000,000devices permonth
500 mbs of eventdata per second
100,000,000db transactionsper day
30 servers
@petabridge Petabridge.com
Struggling Business
12 months ofrunway
3 engineers
0engineersw/prior distributedsystems experience
@petabridge Petabridge.com
A Radical New Product1. IntegrateMarkedUpinto your application.
Your Installer
Notification Client
Your ApplicationAnalytics SDK
2. Users install your appandMarkedUp collects events.
3. Create campaigns basedon events and user-data.
4.MarkedUp delivers personalized, brandedmessages to users instantly via push,based on their in-app activity.
@petabridge Petabridge.com
Allow our customers to start conversations with specific types of users.
@petabridge Petabridge.com
Had to be done in real-time for best results.
@petabridge Petabridge.com
Prototype: HTTP + Database
@petabridge Petabridge.com
(Read after Write)
@petabridge Petabridge.com
TheoryUSER
HTTP LOAD BALANCER
ASP.NETMVC(IIS)
CASSANDRACLUSTER
Events 0...3
0
0
1
0 10,1
2
20,1,2
3
30,1,2,3
MESSAGE
@petabridge Petabridge.com
Reality
HTTP LOAD BALANCER
ASP.NETMVC(IIS)
CASSANDRACLUSTER
USER
Events 0...3
0 3
0 ??3??
1 2
1??2 ??
@petabridge Petabridge.com
Breakthrough!
USER
HTTP LOAD BALANCER
Events 0...3
0 12 3
MESSAGE
@petabridge Petabridge.com
I had STATEFUL, REACTIVE, STREAM PROCESSING
problem!
@petabridge Petabridge.com
Implementation: HTTP -> EDA -> C*
C* ClusterPrivateVPCZone
HTTP Load Balancer
HTTP Load Balancer
API
TargetingSystem
Win32 MailboxService
CQL Writes
CQL R/W
CQL Reads
Akka.NETremoting(TCP +protobufs)
@petabridge Petabridge.com
Why Do We Care About EDA?
@petabridge Petabridge.com
Databases Aren’t Magical
HTTP LOAD BALANCER
ASP.NETMVC(IIS)
CASSANDRACLUSTER
USER
Events 0...3
0 3
0 ??3??
1 2
1??2 ??
@petabridge Petabridge.com
Event-Driven Architecture Concepts
@petabridge Petabridge.com
Key Terms• Event – a significant change in
“state”• Message – a notification of an event• Emitter – detect, gather, transfer
events• Sinks – react to events immediately• Channels – conduit between
emitters and sinks
@petabridge Petabridge.com
Goals & Benefits• Extreme decoupling
• Easily distributed
• Inherently asynchronous and concurrent
@petabridge Petabridge.com
EDA Styles• Simple Event Processing
• Event Stream Processing
• Complex Event Processing (CEP)
@petabridge Petabridge.com
Event Stream Processing w/ FSMs
@petabridge Petabridge.com
State A
State B
State C
Event 0
Stay
Event 1
Transition
Event 2
Transition
Event 3
REACT
@petabridge Petabridge.com
FSM0
C*
C*
C*
C* C*
Events 0..2
Transition
Flush
FSM 0 State
FSM 0 State FSM 0 State
@petabridge Petabridge.com
C*
C*
C*
C* C*
FSM 0 State
FSM 0 State FSM 0 State
FSM0
Crash
FSM0
RecreateFSMRequestState
ReplayState
@petabridge Petabridge.com
CQL SchemaCREATE TABLE IF NOT EXISTS FSMState ( persistence_id text, state_id int, state_data blob, PRIMARY KEY (persistence_id, state_id))
@petabridge Petabridge.com
Event Sourcing with C*
@petabridge Petabridge.com
C*
RoomMessageHistoryActor
Message 1
Message 2
Message 3
Chat History forUser X
1. Build history,persist to store
History
Message 1Message 2Message 3
2. Requesthistory
3. Deliver historyas materialized view
4. Present to user
@petabridge Petabridge.com
C*
RoomMessageHistoryActor
1. Startsafter application
restart
2. Replays storedmessages from
C*Message 1
Message 2
Message 3
History
Message 1Message 2Message 3
3. Rebuildsin-memory
history
@petabridge Petabridge.com
EDA + C* Tips and Tricks• Large number of small sinks works
best• Define simple, reusable journals and
snapshots• C* == durable backup, App = single
source of truth • TTL everywhere your business
domain supports it
@petabridge Petabridge.com
Have questions? Ask us!
http://petabridge.com/