Building Event Driven Services with Stateful Streams

1

Building Event Driven Services with Stateful Streams

Ben Stopford @benstopford

2

Microservice are an evolution

3

Distributed Applications

4

Apply the Single Responsibility Principal

5

Service-Based Application

Orders Service

Basket Service

Payment Service

Fulfillment Service

Stock Service

6

Release together

Orders Service

Validation Service

Payment Service

Fulfillment Service

Stock Service

7

Microservices are Independently Deployable

8

Independently Deployable

Orders Service

Validation Service

Payment Service

Fulfillment Service

Stock Service

9

Service can have different release schedules

Services Released Together

Services Released Independently

Stock

Orders

Payment

Stock

Orders

Payment

10

Allows us to scale

11

Scale in people terms

12

Microservices

•  Single Responsibility •  Fine grained •  Modularized •  Independently deployable •  Run by different teams

13

When it comes to data, microservices promote independence.

14

Stateful services have their own DB

Orders Service

Validation Service

Payment Service

Fulfillment Service

Stock Service

15

This makes sense

16 Database

Data on inside

Data on outside

Interface amplifies

data

Databases provide a rich point of coupling

17

Service A Service B

Service C

So microservices avoid shared databases

18

But most business services share the same core stream of facts.

Catalog Authorisation

19

Many databases leads to many data islands

Orders Service

Validation Service

Payment Service

Fulfillment Service

Stock Service

20

Difficult to compose a joined up view

21

Services Evolution

1.0 - SOA 2.0 - Microservices 3.0 - What’s next???

22

How do we get to 3.0?

•  Loose Coupling •  Data Enabled •  Event Driven •  Operational Transparency

23



24

The synchronous world of request response protocols leads to tight, point-to-point

couplings.

25

Coupling

A measure of the assumptions two services make about one another when they exchange information.

Frank Laymann

26

Change to one service => many dependencies to consider

CHANGE

27

Leverage Receiver Driven Flow Control

28

UI Orders Service

Stock Service

Payment Provider

Maybe order more stock

Request-Response Model

29

UI Orders Service

Stock Service

Repricing Service

Payment Provider


New services must be called explicitly

30

UI Orders Service

Stock Service

Payment Provider


Event Driven Model with Broker

31

UI Orders Service

Stock Service

Payment Provider


Flow Control is Receiver Driven: Pluggable

Repricing Service

32

Events + Brokers Decouple Architectures

33



34

Useful Grid

Orders Service

Payments Service

Customers Service

We need to join different datasets

35

Useful Grid

Orders Service

Payments Service

Customers Service

Need a better way to handle data the data services expose

36



37

Useful Grid

Orders Payments

Customers

Why Poll when we can Push

38

Useful Grid

Orders Service

Payments Service

Customers Service

Streaming leads to being both event driven and data enabled

Event Driven

Data Enabled

39



40

Distributed Murder Mysteries

41

Ledger Service

Loan Validation

Shared Narrative

Fraud Service

History Service

Account Service Account

History

Login Service

Loan Service

A Journal of Service Interactions

42



43

Tenet 1: Events and Asynchronous processes

better model the way businesses work

44

Tenet 2: Services need to design for data-on-the-

outside

45

Tenet 3: The measure of a ‘good’ architecture is

it’s ability to evolve

46

Steps to Streaming Services

47

1. Take Responsibility for the past and evolve

48

Stay Simple. Take Responsibility for the past

Browser

Webserver

49

Evolve Forwards

Browser

Webserver Orders Service

50

2. Raise events. Don’t talk to services.

51

Events => Decouple Events == Facts

52

Raise events. Don’t talk to services

Browser

Webserver Orders Service

53

Order Requested Order

Received

Browser

Webserver

Orders Service


54


Validated Order

Received

Browser

Webserver

Orders Service


55

3. Use Kafka as a Service Backbone

56

KAFKA


Validated Order

Received

Browser

Webserver

Orders Service

Use Kafka as a Backbone for Events

57

KAFKA

- Highly available - Linearly scalable - Load balance consumers - HA consumers

58

4. Use CDC to evolve away from legacy

59 KAFKA


Validated Order

Received

Browser

Webserver

Orders Service

Evolve away from Legacy

60 KAFKA


Validated Order

Received

Browser

Webserver

Orders Service

Use the Database as a ‘Seam’

Connect

Products

61

5. Make use of Schemas

62 KAFKA


Validated Order

Received

Browser

Webserver

Orders Service

Use Schemas to validate Backwards Compatibility

Connect

Products

Schema Registry

63

6. Use the Single Writer Principal

64 KAFKA


Validated Order

Received

Browser

Webserver

Orders Service

Apply the single writer principal

Connect

Products

Schema Registry

Order Completed

65

Single Writer Principal

-  Creates local consistency points in the absence of Global Consistency

-  Makes schema upgrades easier to manage.

66

7. Embrace Multi-tenancy

67 KAFKA


Validated Order

Received

Browser

Webserver

Orders Service

Apply Bandwidth Limits to services

Connect

Products

Schema Registry

Order Completed

100Mb/s

68

8. Store Datasets in the Log

69

Messaging that Remembers

Orders Customers

Payments Stock

70

Rewind and Replay

Rewind

Replay Orders Service

71 KAFKA


Validated Order

Received

Browser

Webserver

Orders Service

Utilize Whole Datasets

Connect

Products

Schema Registry

Order Completed Repricing

72

Compacted Topic: Delete superceded messages that share the same key

K1 K1 K1 K2 K2

K2 K1

V1

V1

V2 V3

V2 V4 V3

Compacted Topics

73

Orders Customers

Payments Stock

Single, Shared Source of Truth

Data-on-the-Outside

74

But how do you query a log?

75

9. Use Materialized Views to Query the Log

76

Materialized View

Create via a query (select * from Orders Where Region == ‘USA’)

Base Table

Materialized Views in a DB

77

Insert data Materialized View

Create via a query (select * from Orders Where Region == ‘USA’)

Query is rerun on data as it arrives

Base Table

Materialized Views in a DB

78

In practice often more complex

select Order.region, count(order.quantity)

from Orders, Product, Customer where Product.group = ‘Houshold’ and Customer.type = external group by Order.region

Orders Product Customer

View

Read Optimized

79

Same problem but with services

select Order.region, count(order.quantity)

from Orders, Product, Customer where Product.group = ‘Houshold’ and Customer.type = external group by Order.region

My Service View

Read Optimized

Orders Service

Product Service

Customer Service

80

Use Materialized Views, but turned inside out!

81

Or Embed Views with the Streams API

Insert data

Kafka Streams

Business Logic

KAFKA

Orders Service

Query in DSL

82

On startup: Rewind and Replay

Rewind

Replay

Orders Service

83

Data Services: Distributed Materialized View

Kafka

Queryable State API

Service

KStreams View

84

Create Views with Connect

DB of choice

Kafka Connect

API Orders Service

Context Boundary

Kafka

85

10. Move Data to Code

86

High Throughput Data Movement

Service

Service

Service

Service instance 1

Service instance 2

Service instance 3

Service instance 4

Kafka Cluster (many machines)

88

Connect


Validated Order

Completed Order

Received

Products

Browser

Webserver

Schema Registry

Orders Service Stock

Stock

Materialize Stock ‘View’ Inside Service

KAFKA

89

If messaging Remembers: Views don’t have to!

90

11. Take only the data you need today

91

Connect


Validated Order

Completed Order

Received

Products

Browser

Webserver

Schema Registry

Orders Service Stock

Stock

Take only the data you need!

KAFKA

92

Take only the data you need

{“Stock Inventory”: {“Id”: “Foo1234”,“Vendor”: “Foo Industries”,“Description”: “This is a …”, “Delivery Category”: “ND”,“Stock Status”: [

“Items in Stock”: 53,“Items on Order”: 0

]…etc…}

93

Data Movement

Be realistic: •  Network is no longer the bottleneck •  Indexing is:

•  In memory indexes help •  Keep datasets focussed

94

12. Use the log instead as a ‘database’ (for data-on-the-inside)

95

Connect


Validated Order

Completed Order

Received

Products

Browser

Webserver

Schema Registry

Orders Service

Reserved Stocks

Stock

Stock

Reserved Stocks

Save Internal State to the Log (Event Sourcing)

KAFKA

96

Append state as events

Service

Append

KAFKA

97

Rewind and Replay on startup

Rewind

Replay Service

KAFKA

98

Connect


Validated Order

Completed Order

Received

Products

Browser

Webserver

Schema Registry

Orders Service

Reserved Stocks

Stock

Stock

Reserved Stocks

Order Service Loads Reserved Stocks on Startup

KAFKA

99

14. Use Transactions to tie Communication & State together

100

OrderRequested (IPad)

2a. Order Validated

2c. Offset Commit 2b. IPad Reserved

Internal State: Stock = 17 Reservations = 2

Tie Events & State with Transactions

101

Connect

TRANSACTION


Validated Order

Completed Order

Received

Products

Browser

Webserver

Schema Registry

Orders Service

Reserved Stocks

Stock

Stock

Reserved Stocks

Stay Simple. Take Responsibility for the past

KAFKA

102

15. Compose Services as a Streaming Pipeline

103

(1) Stateless Processor

e.g. - Filter orders by region -  Convert to local domain model -  Simple rule

104

(2) Stateless, Data Enabled Processor

Similar to star schema •  Facts are stream •  Dimensions are GKTables

e.g. Enrich Orders with Customer Info & Account details

Stream

GKTable

GKTable

Stream-Table Join

105

(3) Gates

e.g. rules engines or event correlation When Order & Payment then …

Stream 1

Window

Window Stream-Stream Join

Stream 2

106

(4) Stateful Processor

e.g. - Average order amount by region

- Posting Engine (reverse replace of previous position)

State store backed up to Kafka

107

(5) Stream-Aside Pattern

public static void main(…){…}

JVM

108

(7) Sidecar Pattern

109

Combine them: 1.  Stateless

2.  Data enriched

3.  Gates

4.  Stateful processors

5.  Stream-aside

6.  Sidecar

110

Group services into bounded contexts

111

Kafka

Kafka

Independently deployable subsystem

112

Richer Example

113

All order logic, stateless service

Orders-by-Customer view (KStreams + Queryable State API)

UI Service Stream

Maintenance & Monitoring

Highly available, stateful service

UI uses Orders-by-Customer View directly

via Queryable State

History Service pushes data to a local Elastic Search Instance

Orders Service

Derived View

UI joins data Tables & Streams

Fulfillment Service Analytics

Orders Product Custo-mers KTables

KTables KTables

Schemas

114

So…

115

Good architectures have little to do with this:

116

It’s about how systems evolves over time

117

Request driven isn’t enough

•  High coupling •  Hard to handle

async flows •  Hard to move and

join datasets.

118

Services need to be data enabled

119

The decoupling effects of the broker make the architecture pluggable and data enabled

Orders Customers

Payments Stock

120

...with a single source of truth

Orders Customers

Payments Stock

121

...and many service specific views

Orders Customers

Payments Stock

122

… which react to events as they occur

Orders Customers

Payments Stock

123

Microservices enabled by a streaming platform

•  Loose coupling •  Data Enabled •  Event Driven •  Operational transparency

124


•  Loose coupling •  Embrace data •  Event Driven •  Operational transparency

125

Thank You @benstopford

https://www.confluent.io/blog/tag/microservices/

126

Tips and Tricks

•  Take only the data you need today •  Limit the scope of request response interfaces •  Build small services. Build big ones too. •  Don’t fear the network. Fear the index. •  The Sync-Async divide

•  Pure event driven with event streams and views. •  Map request response to event driven (Leverage CQRS)

•  Too fine-grained services. •  Too much data •  KStreams rebalancing •  Need for automation

Building Event Driven Services with Stateful Streams

Technology

Transcript of Building Event Driven Services with Stateful Streams