Post on 21-Jan-2018
1
Building Event Driven Services with Stateful Streams
Ben Stopford @benstopford
2
Microservice are an evolution
3
Distributed Applications
4
Apply the Single Responsibility Principal
5
Service-Based Application
Orders Service
Basket Service
Payment Service
Fulfillment Service
Stock Service
6
Release together
Orders Service
Validation Service
Payment Service
Fulfillment Service
Stock Service
7
Microservices are Independently Deployable
8
Independently Deployable
Orders Service
Validation Service
Payment Service
Fulfillment Service
Stock Service
9
Service can have different release schedules
Services Released Together
Services Released Independently
Stock
Orders
Payment
Stock
Orders
Payment
10
Allows us to scale
11
Scale in people terms
12
Microservices
• Single Responsibility • Fine grained • Modularized • Independently deployable • Run by different teams
13
When it comes to data, microservices promote independence.
14
Stateful services have their own DB
Orders Service
Validation Service
Payment Service
Fulfillment Service
Stock Service
15
This makes sense
16 Database
Data on inside
Data on outside
Interface amplifies
data
Databases provide a rich point of coupling
17
Service A Service B
Service C
So microservices avoid shared databases
18
But most business services share the same core stream of facts.
Catalog Authorisation
19
Many databases leads to many data islands
Orders Service
Validation Service
Payment Service
Fulfillment Service
Stock Service
20
Difficult to compose a joined up view
21
Services Evolution
1.0 - SOA 2.0 - Microservices 3.0 - What’s next???
22
How do we get to 3.0?
• Loose Coupling • Data Enabled • Event Driven • Operational Transparency
23
How do we get to 3.0?
• Loose Coupling • Data Enabled • Event Driven • Operational Transparency
24
The synchronous world of request response protocols leads to tight, point-to-point
couplings.
25
Coupling
A measure of the assumptions two services make about one another when they exchange information.
Frank Laymann
26
Change to one service => many dependencies to consider
CHANGE
27
Leverage Receiver Driven Flow Control
28
UI Orders Service
Stock Service
Payment Provider
Maybe order more stock
Request-Response Model
29
UI Orders Service
Stock Service
Repricing Service
Payment Provider
Maybe order more stock
New services must be called explicitly
30
UI Orders Service
Stock Service
Payment Provider
Maybe order more stock
Event Driven Model with Broker
31
UI Orders Service
Stock Service
Payment Provider
Maybe order more stock
Flow Control is Receiver Driven: Pluggable
Repricing Service
32
Events + Brokers Decouple Architectures
33
How do we get to 3.0?
• Loose Coupling • Data Enabled • Event Driven • Operational Transparency
34
Useful Grid
Orders Service
Payments Service
Customers Service
We need to join different datasets
35
Useful Grid
Orders Service
Payments Service
Customers Service
Need a better way to handle data the data services expose
36
How do we get to 3.0?
• Loose Coupling • Data Enabled • Event Driven • Operational Transparency
37
Useful Grid
Orders Payments
Customers
Why Poll when we can Push
38
Useful Grid
Orders Service
Payments Service
Customers Service
Streaming leads to being both event driven and data enabled
Event Driven
Data Enabled
39
How do we get to 3.0?
• Loose Coupling • Data Enabled • Event Driven • Operational Transparency
40
Distributed Murder Mysteries
41
Ledger Service
Loan Validation
Shared Narrative
Fraud Service
History Service
Account Service Account
History
Login Service
Loan Service
A Journal of Service Interactions
42
How do we get to 3.0?
• Loose Coupling • Data Enabled • Event Driven • Operational Transparency
43
Tenet 1: Events and Asynchronous processes
better model the way businesses work
44
Tenet 2: Services need to design for data-on-the-
outside
45
Tenet 3: The measure of a ‘good’ architecture is
it’s ability to evolve
46
Steps to Streaming Services
47
1. Take Responsibility for the past and evolve
48
Stay Simple. Take Responsibility for the past
Browser
Webserver
49
Evolve Forwards
Browser
Webserver Orders Service
50
2. Raise events. Don’t talk to services.
51
Events => Decouple Events == Facts
52
Raise events. Don’t talk to services
Browser
Webserver Orders Service
53
Order Requested Order
Received
Browser
Webserver
Orders Service
Raise events. Don’t talk to services
54
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Raise events. Don’t talk to services
55
3. Use Kafka as a Service Backbone
56
KAFKA
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Use Kafka as a Backbone for Events
57
KAFKA
- Highly available - Linearly scalable - Load balance consumers - HA consumers
58
4. Use CDC to evolve away from legacy
59 KAFKA
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Evolve away from Legacy
60 KAFKA
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Use the Database as a ‘Seam’
Connect
Products
61
5. Make use of Schemas
62 KAFKA
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Use Schemas to validate Backwards Compatibility
Connect
Products
Schema Registry
63
6. Use the Single Writer Principal
64 KAFKA
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Apply the single writer principal
Connect
Products
Schema Registry
Order Completed
65
Single Writer Principal
- Creates local consistency points in the absence of Global Consistency
- Makes schema upgrades easier to manage.
66
7. Embrace Multi-tenancy
67 KAFKA
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Apply Bandwidth Limits to services
Connect
Products
Schema Registry
Order Completed
100Mb/s
68
8. Store Datasets in the Log
69
Messaging that Remembers
Orders Customers
Payments Stock
70
Rewind and Replay
Rewind
Replay Orders Service
71 KAFKA
Order Requested Order
Validated Order
Received
Browser
Webserver
Orders Service
Utilize Whole Datasets
Connect
Products
Schema Registry
Order Completed Repricing
72
Compacted Topic: Delete superceded messages that share the same key
K1 K1 K1 K2 K2
K2 K1
V1
V1
V2 V3
V2 V4 V3
Compacted Topics
73
Orders Customers
Payments Stock
Single, Shared Source of Truth
Data-on-the-Outside
74
But how do you query a log?
75
9. Use Materialized Views to Query the Log
76
Materialized View
Create via a query (select * from Orders Where Region == ‘USA’)
Base Table
Materialized Views in a DB
77
Insert data Materialized View
Create via a query (select * from Orders Where Region == ‘USA’)
Query is rerun on data as it arrives
Base Table
Materialized Views in a DB
78
In practice often more complex
select Order.region, count(order.quantity)
from Orders, Product, Customer where Product.group = ‘Houshold’ and Customer.type = external group by Order.region
Orders Product Customer
View
Read Optimized
79
Same problem but with services
select Order.region, count(order.quantity)
from Orders, Product, Customer where Product.group = ‘Houshold’ and Customer.type = external group by Order.region
My Service View
Read Optimized
Orders Service
Product Service
Customer Service
80
Use Materialized Views, but turned inside out!
81
Or Embed Views with the Streams API
Insert data
Kafka Streams
Business Logic
KAFKA
Orders Service
Query in DSL
82
On startup: Rewind and Replay
Rewind
Replay
Orders Service
83
Data Services: Distributed Materialized View
Kafka
Queryable State API
Service
KStreams View
84
Create Views with Connect
DB of choice
Kafka Connect
API Orders Service
Context Boundary
Kafka
85
10. Move Data to Code
86
High Throughput Data Movement
Service
Service
Service
Service instance 1
Service instance 2
Service instance 3
Service instance 4
Kafka Cluster (many machines)
87
88
Connect
Order Requested Order
Validated Order
Completed Order
Received
Products
Browser
Webserver
Schema Registry
Orders Service Stock
Stock
Materialize Stock ‘View’ Inside Service
KAFKA
89
If messaging Remembers: Views don’t have to!
90
11. Take only the data you need today
91
Connect
Order Requested Order
Validated Order
Completed Order
Received
Products
Browser
Webserver
Schema Registry
Orders Service Stock
Stock
Take only the data you need!
KAFKA
92
Take only the data you need
{“Stock Inventory”: {“Id”: “Foo1234”,“Vendor”: “Foo Industries”,“Description”: “This is a …”, “Delivery Category”: “ND”,“Stock Status”: [
“Items in Stock”: 53,“Items on Order”: 0
]…etc…}
93
Data Movement
Be realistic: • Network is no longer the bottleneck • Indexing is:
• In memory indexes help • Keep datasets focussed
94
12. Use the log instead as a ‘database’ (for data-on-the-inside)
95
Connect
Order Requested Order
Validated Order
Completed Order
Received
Products
Browser
Webserver
Schema Registry
Orders Service
Reserved Stocks
Stock
Stock
Reserved Stocks
Save Internal State to the Log (Event Sourcing)
KAFKA
96
Append state as events
Service
Append
KAFKA
97
Rewind and Replay on startup
Rewind
Replay Service
KAFKA
98
Connect
Order Requested Order
Validated Order
Completed Order
Received
Products
Browser
Webserver
Schema Registry
Orders Service
Reserved Stocks
Stock
Stock
Reserved Stocks
Order Service Loads Reserved Stocks on Startup
KAFKA
99
14. Use Transactions to tie Communication & State together
100
OrderRequested (IPad)
2a. Order Validated
2c. Offset Commit 2b. IPad Reserved
Internal State: Stock = 17 Reservations = 2
Tie Events & State with Transactions
101
Connect
TRANSACTION
Order Requested Order
Validated Order
Completed Order
Received
Products
Browser
Webserver
Schema Registry
Orders Service
Reserved Stocks
Stock
Stock
Reserved Stocks
Stay Simple. Take Responsibility for the past
KAFKA
102
15. Compose Services as a Streaming Pipeline
103
(1) Stateless Processor
e.g. - Filter orders by region - Convert to local domain model - Simple rule
104
(2) Stateless, Data Enabled Processor
Similar to star schema • Facts are stream • Dimensions are GKTables
e.g. Enrich Orders with Customer Info & Account details
Stream
GKTable
GKTable
Stream-Table Join
105
(3) Gates
e.g. rules engines or event correlation When Order & Payment then …
Stream 1
Window
Window Stream-Stream Join
Stream 2
106
(4) Stateful Processor
e.g. - Average order amount by region
- Posting Engine (reverse replace of previous position)
State store backed up to Kafka
107
(5) Stream-Aside Pattern
public static void main(…){…}
JVM
108
(7) Sidecar Pattern
109
Combine them: 1. Stateless
2. Data enriched
3. Gates
4. Stateful processors
5. Stream-aside
6. Sidecar
110
Group services into bounded contexts
111
Kafka
Kafka
Independently deployable subsystem
112
Richer Example
113
All order logic, stateless service
Orders-by-Customer view (KStreams + Queryable State API)
UI Service Stream
Maintenance & Monitoring
Highly available, stateful service
UI uses Orders-by-Customer View directly
via Queryable State
History Service pushes data to a local Elastic Search Instance
Orders Service
Derived View
UI joins data Tables & Streams
Fulfillment Service Analytics
Orders Product Custo-mers KTables
KTables KTables
Schemas
114
So…
115
Good architectures have little to do with this:
116
It’s about how systems evolves over time
117
Request driven isn’t enough
• High coupling • Hard to handle
async flows • Hard to move and
join datasets.
118
Services need to be data enabled
119
The decoupling effects of the broker make the architecture pluggable and data enabled
Orders Customers
Payments Stock
120
...with a single source of truth
Orders Customers
Payments Stock
121
...and many service specific views
Orders Customers
Payments Stock
122
… which react to events as they occur
Orders Customers
Payments Stock
123
Microservices enabled by a streaming platform
• Loose coupling • Data Enabled • Event Driven • Operational transparency
124
How do we get to 3.0?
• Loose coupling • Embrace data • Event Driven • Operational transparency
125
Thank You @benstopford
https://www.confluent.io/blog/tag/microservices/
126
Tips and Tricks
• Take only the data you need today • Limit the scope of request response interfaces • Build small services. Build big ones too. • Don’t fear the network. Fear the index. • The Sync-Async divide
• Pure event driven with event streams and views. • Map request response to event driven (Leverage CQRS)
• Too fine-grained services. • Too much data • KStreams rebalancing • Need for automation