Reactive Development: Commands, Actors and Events. Oh My!!
-
Upload
david-hoerster -
Category
Software
-
view
440 -
download
0
Transcript of Reactive Development: Commands, Actors and Events. Oh My!!
Reactive DevelopmentCommands, Actors and Events – Oh My!
About Me• 6-Time .NET (Visual C#) MVP (April 2011)• Sr. Solutions Architect at Confluence• One of the organizers for Pittsburgh TechFest (
http://pghtechfest.com)• Organizer of Pittsburgh Reactive Dev Group (
http://meetup.com/reactive)• Past President of Pittsburgh .NET Users Group• Twitter - @DavidHoerster• Blog – http://blog.agileways.com
Agenda• What are we building?• Messages• What is CQRS?• …and isn’t it dead?
• Build a simple handler• Using Actors Instead of Procedural Domains• Communicating Back to the Client• Resilient Processing
ScheduleStart End Topic
8:30 AM 8:45 AM Intro / What are we building
8:45 AM 10:00 AM Part 1: CQRS / Messages / Lab 1 (Building a Handler)
10:00 AM 10:15 AM Break
10:15 AM 11:45 AM Part 2: Actors / Mini Labs
11:45 AM 12:30 PM Lunch!!
12:30 PM 2:00 PM Part 3: CQRS + Actors / Lab 2 (Actors in Handler; Projections)
2:00 PM 2:15 PM Break
2:15 PM 3:00 PM Lab 3 – End to End Communication (Real-Time Updates)
3:00 PM 3:45 PM Lab 4 – Events; Lab 5 – Querying Events (Restoring Actors from Events; Querying EventStore)
3:45 PM 4:00 PM Wrap-Up
What Are We Building?• An application that processes multiple files• Each file contains all of the plays from the 2015 baseball season for one
HOME team• The files are CSV
• Each row of the file will be sent as a message• A back end process will subscribe to the message and process the data
• Game data, batter data, pitcher data• Results will be published to a topic• Another service will subscribe to that message and push updates to a
simple HTML page via SignalR
Baseball Play Processor System
Producer
Producer
Producer
Topic
Consumer
Topic
API Gateway
ConsumerProjections
(Mongo)Events
(EventStore)Ad Hoc Query
Retro Files
Retro Files• RetroSheet.org files listing every play for every home game in 2015
(per team)• CSV files• 97 fields – we only use a handful• http://www.retrosheet.org/datause.txt
• Feel free to use more fields!!!
Producer and Consumer• Separate processes• Producer reads in Retro CSV files• Pushes each row into a message “topic”• Consumer subscribes to the new messages on the topic• Handles the row accordingly• Consumer will eventually use actors to process data• Run multiple producers, consume additional fields, create new actors,
create new projections!!!
Message “Topics”• Using a RethinkDB table per topic• Somewhat of a workaround for local development• EXE is a beta preview for Windows, so not ready for prime time
• RethinkDB allows clients to subscribe to changes• Acts like a topic• Multiple subscribers
• C# client is a port of the official Java client
MongoDB• Document database• Holds projections for batters and pitchers• And any other projections you’d like to create
• Using 3.X version with WiredTiger• RoboMongo 0.9.X supports WiredTiger (finally!)
• MongoDB C# client is official client
API Gateway• OWIN self-hosted Web API service• Subscribes to changes for batters• Exposes simple API for retrieving batter projection from MongoDB• Pushes changes from topic to client via SignalR• Only pushes to subscribed “group” – not broadcasting out all changes
• Create more pages and APIs!!!
Web Client• Stupid simple web site• One static HTML page• Uses jQuery for simple interaction• Uses SignalR JS library to subscribe to hub for batter changes• As new batter is queried, that is the “group” that is subscribed to• Feel free to create new pages, enhance look n’ feel, add more data!!!
EventStore• Used to hold events for players• Each player id is a new stream• Stream consists of 0 to many events• Can be of different types
• Events are used to restore state of player actors (batters for now) and also used for ad hoc querying later• Feel free to add new types of events (pitcher events???) and add
more data!!!
Ad Hoc Querying• Queries the EventStore for basic batter information• Average and Homers hit for each count (balls/strikes)• Some overall stats
• Uses EventStore C# client (official)• Create new queries, make it a web page, try to slice and dice data
differently!!!
Perspectives on Procedural
Real World Scenario
File File File
Join
Op
Join/End
Real World Scenario• Actually looks like this:
• Multiple sources
• Several transforms
• Several joins
• Configurable
Messages as Part of Application Design
CQRS• Command Query Responsibility Segregation• Coined by Greg Young
• Evolution of CQS (Command Query Separation)• Coined by Bertrand Meyer
• Both revolve around the separation of writes (command) and reads (query)• CQS is more at the unit level• CQRS is at the bounded context level, or so
Messages• The core of CQRS Messages• Communication from the service layer to the domain via commands• Command handlers could also update the underlying repository
• As actions happen, events are raised• Event handlers could issue other commands• Event handlers could update the underlying repository
Messages• Regardless of how it’s implemented, communication between
application parts is via messages• Not only helps communicate intent of the action• “PublishWidget”, “ExpireWorkItem”, “UpdateDefinition”, “DefinitionUpdated”
• But allows for remote handling of messages• Queue, REST payload, etc.
Messages are key to being reactive
So?• So how does this affect my system design?
• Let’s consider some procedural pitfalls
Word Counter Example• Simple program to count the occurrences of words in a document
• Print out the top 25 words and their counts at the end
• Can be accomplished easily via procedural code
Word Counter Example
Word Counter Example• What if you wanted to spread out the counting load?
• What if you wanted to scale out to other nodes/processes/machines?
• Fault tolerance, concurrent actions, etc.?
Building Concurrent Apps• In word counter, may need to spawn some threads
• Lock some critical sections, etc.
• The system becomes responsible for knowing everything going on• Very Proactive (not Reactive!) system
Side note: reactive vs. proactive• Isn’t proactive a good thing?
• A system taking upon itself to check state is proactive• It’s doing too much
• A reactive system reacts to events that occur• It’s doing just the right amount
Side note: reactive vs. proactive• Isn’t proactive a good thing?
• A system taking upon itself to check state is proactive• It’s doing too much
• A reactive system reacts to events that occur• It’s doing just the right amount
System
EvtEvt
System
EvtEvt
Proactive
Reactive
What is CQRS?
CQRS Background• Command Query Responsibility Segregation
• Coined by Greg Young
• Heavily influenced by Domain Driven Design (DDD)• Evolution of Meyer’s CQS (Command Query Separation)• Separation of writes (command) and reads (query)
• CQS is more at the unit level• CQRS is at a higher level bounded context / service
What is CQRS?• Simply: separate paths for reads and writes
• Communicate via messages
• Commands and Events• “CreateOrder” and “OrderCreated”
• Reads are against a thin read model (preferably optimized)
CQRS Visualized: Basic
Client
Backend
Commands Queries
CQRS Visualized
Controller
Handler
Domain
Persistence / Read Model
Read LayerCommandSide
QuerySide
CQRS Extended• Can move from direct references to queued
• Decouples handlers
• Increased isolation of areas
• Leads to single message causing several changes• Raising of events and subscriptions to events
CQRSVisualized
Controller
Handler
Domain
Persistence
Read Layer
Handler
Com
man
d / E
vent
Topi
cs Read Model
CommandSide
QuerySide
CQRS Handler
CQRS Handler
Benefits• Message Based
• Segregation of Responsibilities – Separate Paths
• Smaller, simpler classes• Albeit more classes
• Focused Approach to Domain• Move away from anemic domain models
CQRS Principles• Separate Paths for Command and Queries
• Message-Based
• Async Friendly
• Thin read layer
• Don’t fear lots of small classes
Is CQRS Dead?
<rant>
"CQRS is Hard"• Misunderstandings
• Has to be async• Has to have queueing• Need a separate read model• Need to have Event Sourcing
• Perceived as overly prescriptive, but little prescription
• Confusing• Command Handler vs. Event Handler vs. Domain• Command vs. Event
Has CQRS “Failed”?
• Focus on CQRS as architecture over principles• Search for prescriptive guidance
• Infrastructure/Implementation debates• Should domains have getters??
• “…there are times a getter can be pragmatic. The trick is learning where to use them.” – Greg Young (DDD/CQRS Group)
• “This list has its heads up its own [butts] about infrastructure. What business problems are you working on?” -- Greg Young (DDD/CQRS Group)
</rant>
Popular Architecture Topics• Microservices
• Event driven
• Distributed computing
• NoSQL
• Async
But Isn’t CQRS…?
Message Driven
Responsive Read Model
Isolated HandlersAsync Friendly
Group Handlers into Clusterable Groups
NoSQL Friendly (Flattened Read Models)
CQRS Can Help Drive Reactive Apps
Lab: Building a Simple Handler
Lab Info• Scaffold: Lab1-Handler
• Create a Producer EXE (service) that adds Plays to a RethinkDB table
• Create a Consumer EXE (service) that subscribes to the “plays” RethinkDB table for changes
• Use the Shared project for common entities and messages
• In RethinkDB, make sure you:• Create a database named “baseball” and a table named “plays”• To clear the table, go to Data Explorer and issue this command:
r.db("baseball").table("plays").delete();
Lab: Using RethinkDB• What is and Why
RethinkDB?• Document database• Can join tables/collections
• Big feature is subscribing to changes• Inserting into a table is
just adding a POCO
Lab: Using RethinkDB for Subscriptions• RunChanges is
blocking• Use Reactive
Extensions to subscribe to changes in the feed• “include_initial” is
an OptArg to indicate you want all rows to start with along with changes.
CQRS Evolved – CQRS + Actors
CQRS & the Reactive Manifesto
• Message Based• Core of CQRS
• Responsive• Commands are ack/nack• Queries are (can be) against an optimized read model
• Resillient• Not core to CQRS Implementation
• Elastic• Not core to CQRS Implementation
CQRS + Reactive• Resillient and Elastic core to
Reactive
• CQRS’ core is Message-Based and Responsive
• Combining the two is powerful
Actor ModelCQRS lacks prescriptive guidance
• Actor Model provides a message-based pattern• Provides prescriptive guidance• Makes for more generalized implementation
• Actor Model is driving more reactive-based development
Actor Model• Actor Model is a system made of small units of concurrent
computation• Formulated in the early 70’s
• Each unit is an actor
• Communicates to each other via messages
• Actors can have children, which they can supervise
What is an Actor?• Lightweight class that encapsulates state and behavior
• State can be persisted (Akka.Persistence)• Behavior can be switched (Become/Unbecome)
• Has a mailbox to receive messages• Ordered delivery by default• Queue
• Can create child actors• Has a supervisory strategy for child actors
• How to handle misbehaving children
• Are always thread safe• Have a lifecycle
• Started, stopped, restarted
Actor Hierarchy
baseball
gameCoordinator
gameInfo-x gameInfo-yplayerSupervisor
batter-abatter-bbatter-c
c-01 c-11 c-32
Akka: Resiliency• Actors supervise their children
• When they misbehave, they are notified
• Can punish (fail) all children, or tell the misbehaving on to try again
• Protects rest of the system
Akka.NET• Akka.NET is an open source framework• Actor Model for .NET
• Port of Akka (for JVM/Scala)
• http://getakka.net
• NuGet – searching for “akka.net”, shows up low in the list
Starting with Akka.NET• Akka.NET is a hierarchical system
• Root of the system is ActorSystem
• Expensive to instantiate – create and hold!
• ActorSystem can then be used to create Actors
Creating an Actor• An Actor has a unique address• Can be accessed/communicated to like a URI
• Call ActorOf off ActorSystem, provide it a name• URI (actor path) is created hierarchically
• Actor Path = akka://myName/user/announcer
Creating Children• Best to call ActorOf to get an IActorRef
• Create a static Props creator
• Actor path is relative to parent’s actor path
• akka://myDemo/user/countChocula/{childName}
• When creating a child, good idea to name it• So you can reference it later!
Sending a Message• Messages are basis of actor communication• Should be IMMUTABLE!!!
• “Tell” an actor a message• Async call
Immutable!!
Receiving A Message• Create your Actor
• Derive from ReceiveActor
• In constructor, register your message subscriptions
• Looks similar to a CQRS • Handler
Receiving A Message• Each actor has a mailbox
• Messages are received in the actor’s mailbox (queue)
• Actor is notified that there’s a message• It receives the message• Takes it out of the mailbox• Next one is queued up
• Ordered delivery by default• Other mailboxes (like Priority) that are out-of-order
Actor
Mai
lbox
Msg
Async• Actors have messages delivered to them in order via a Mailbox• As message is received, handed to Actor to process• Messages in mailbox queue up• As a result, async processing of messages isn’t possible• Receive<> (async msg => { … }) doesn’t work
• If you need to perform async tasks within Receive• PipeTo(Self) usually does the trick• Re-queues message back on Mailbox• When delivered, actor resumes
Actor
Mai
lbox
Msg
PipeTo
Lab: Breaking Bad ActorsAnd other simple actor projects
Lab Info• Scaffold: A1-HelloAkka
• Simple Two Actor Project
• Try including another actor or two (and another message!)
Lab Info• Scaffold: A2-ActorTimeoutTest
• Show how an actor can commit “suicide”
• Uses a Scheduler
• Try to implement this another way• Instead of parent issuing the Poison Pill, have the Child do it• Override AroundReceive()
CQRS + Actors
Elements of CQRS in Actor Model• Message-based
• Potentially async
• Focused code
• Actor System is your Command Side
CQRS + Actors• Combining the principles of CQRS and the patterns of the Actor Model
• Message-based communication with concurrent processing
CQRS + Actors: Messages• Messages in an Actor System include both Commands and Events
• Still encapsulate intent
• Still should still contain information necessary to process/handle command or event
• Basically, no big change here
CQRS + Actors: Messages
Lab: Actor Word Counter
Lab Info• Scaffold: A3-ActorWordCounter
• Notice how actors can call each other (and themselves)
• Update messages to be more like Commands and Events
• CHALLENGE: Can you use the RoundRobinPool???
CQRS + Actors – Handlers and Services
CQRS + Actors: Handlers• Handlers begin to encapsulate your
Actor System
• Generally a single Actor System per handler
• Have several actors at top level of hierarchy to handle initial messages
• Domain can be injected into the Actor System via Create()
Handler
Client
QueuePersistenceOther Handler
Direct call or via queue
CQRS + Actors: Handlers
CQRS + Actors: Handlers
CQRS + Actors: Handlers• Handlers act as the entry point• Web API• Service subscribed to queue
• CQRS IHandle<T> for those messages “exposed”
• Actor System is called by the CQRS Handler
• Actor System has many other “internal” actors
CQRS + Actors: Services
Word Counter Handler
super
Count Write Stats
A B G H Y Z
Word Counter Handler
CQRS + Actors: Services
super
Count
A B
Word Writer Handler
super
Write
G H
Word Stats Handler
super
Stats
Y Z
Message Bus
CQRS + Actors + Services
Service A Service B Service C
Message Bus
DW / DL / Read Model
Cmds / Events Cmds / Events
API Gateway SvcCommands Queries
RM RMRM
Queries (?)
Service AService AService A
CQRS + Actors + Services + Cluster (Akka.Cluster)
Service A Service B Service C
Message Bus
DW / DL / Read Model
Cmds / Events Cmds / Events
API Gateway SvcCommands Queries
RM RMRM
Queries (?)
CQRS + Actors + Services• Actors act as the workers for your handlers
• Services encapsulate handler into a bounded context
• CQRS is the messaging and communication pattern within system
• Using these three helps build killer reactive applications
Lab: Distributed Reactive Apps
Lab Info• Scaffold: Lab2-GameActor
• The Domain\Actors and Messages folders have been excluded from the project• Include them if you want to run things• Study them if you want to write your own
Lab Info• RethinkDB Setup• Database named: “baseball”• Tables named: “plays” and “batterStat”• To clear these tables, issue the command from Data Explorer
r.db("baseball").table("batterStat").delete();r.db("baseball").table("plays").delete();
• MongoDB shouldn’t need setup – database and collections should be created• Database is “baseball2015” and collections are “batter” and “pitcher”
Lab Info• Add a new collection to MongoDB’s baseball2015 database via
mongoimport• Command line• Bin\mongoimport /d baseball2015 /c players /headerline /type csv “data\players.csv”
• This will allow you to view players’ real names instead of just IDs (e.g. Andrew McCutchen vs. mccua001)
Lab Info: Actor Model/user
gameEventCoord
gameSupervisor
gameActor 1 gameActor 2
teamSupervisor
teamActor 1 teamActor 2
batterSupervisor
pitcherSupervisor
pitcherActor 1 pitcherActor 2
batterActor 1 batterActor 2
Lab: Communicating Back
Lab Info• Scaffold: Lab3-Display
• This lab subscribes to a topic/table in RethinkDB for batter events• Events are pushed to a web JavaScript client via SignalR• Web API uses SignalR groups to only send subscribed players to clients
(instead of all updates)
• Domain\Actors and Messages are excluded. Use if you’d like• UI-Api\Hubs and UI-Api\Utils are excluded. These set up SignalR Hubs and
Group Subscriptions. Use if you’d like.
Lab: End to End
Lab Info• Scaffold: Lab4-Events
• Nothing excluded• Try to add additional information to events or add pitcher events to
EventStore• Add a pitcher.html page, similar to default.html, to query on pitcher
information
Lab Info• Scaffold: Lab5-EventStoreSample
• Use a populated EventStore to query• Slice events differently• Create different types of queries
• Also, try to incorporate this into a web page instead of a console app• Show Batter Count stats on a web page, even in real time via SignalR
updates!!!
Thank you!