Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox
-
Upload
opencredo -
Category
Technology
-
view
258 -
download
3
Transcript of Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox
ConcursusEvent Sourcing Evolved
Introductions
Dominic Fox
Twitter: @dynamic_proxy
Email: [email protected]
Github: http://github.com/poetix
Concursus Github: http://github.com/opencredo/concursus
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Agenda
"I have told people over and over and over again, don't write a
CQRS framework…I can basically guarantee you that it will be
abandonware within one year, like every other one has become.”
- Greg Young, inventor of the term “CQRS”
Why Concursus?
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
From Presence to Presents
From Presence to Presents
Aggregate services together into a single container or integrate
them through a single enterprise service bus...
Manage all of our data through a single relational database
schema with global constraints...
Use distributed locks and transactions to make a distributed
system behave as if it were one single system with a global
transactional semantics...
From Presence to Presents
From Presence to Presents
From Presence to Presents
From Presence to Presents“Here the Microservice can become an escape route from reality. Within each Microservice, we can
live on a safe island of determinism and strong consistency — an island where we can live happily
under the illusion that time and the present is absolute.
However, as soon as we exit the boundary of the Microservice we enter a wild ocean of non-
determinism—the world of distributed systems, which is a very different world. You have probably
heard that building distributed systems is hard. It is. That being said it is also the world that gives
us solutions for resilience, elasticity, isolation amongst others. At this point, what we need to do is
not to run back to the monolith, but instead learn how to apply and use the right set of principles,
abstractions and tools in order to manage it.”
- Jonas Bonér, Reactive Microservices Architecture, p. 28-29
From Presence to Presents
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
“Write First, Reason Later”
“Write First, Reason Later”
“Write First, Reason Later”
Scene from Endgame, by Samuel Beckett
“Write First, Reason Later”
“Write First, Reason Later”
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Three Processing Schedules
1.Transient
2.Durable
3.Persistent
Three Processing Schedules
1.Transient
2.Durable
3.Persistent
Three Processing Schedules
1.Transient
2.Durable
3.Persistent
Three Processing Schedules
1.Transient - what happens
2.Durable - what’s happening
3.Persistent - what happened
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Domain Model: Events
aggregateType: lightbulbaggregateId: 69016fb5-1d69-4a34-910b-f8ff5c702ad9
eventTimestamp: 2016-03-31T10:31:17.981Zparameters: { “wattage”: 60 }
Domain Model: Events
aggregateType: lightbulbaggregateId: 69016fb5-1d69-4a34-910b-f8ff5c702ad9
eventTimestamp: 2016-03-31T10:36:42.171Zparameters: { “location”: “hallway”}
Domain Model: Events
aggregateType: lightbulbaggregateId: 69016fb5-1d69-4a34-910b-f8ff5c702ad9
eventTimestamp: 2016-03-31T10:36:42.171ZprocessingTimestamp: 2016-03-31T10:36:48.3904Zparameters: { “location”: “hallway”}
Domain Model: Events
Domain Model: Summary
Every Event occurs to an Aggregate, identified by its type and id.
Every Event has an eventTimestamp, generated by the source of the event.
An Event History is a log of Events, ordered by eventTimestamp, with an additional processingTimestampwhich records when the Event was captured.
Network
Event sources
Event processors
Events arrive:• Partitioned• Interleaved• Out-of-order
Processing Model: Ordering
Log is:• Partitioned by aggregate id• Ordered by event timestamp
Processing Model: Ordering
CREATE TABLE IF NOT EXISTS concursus.Event (aggregateType text,aggregateId text,eventTimestamp timestamp,streamId text,processingId timeuuid,name text,version text,parameters map<text, text>,characteristics int,PRIMARY KEY((aggregateType, aggregateId), eventTimestamp,
streamId)) WITH CLUSTERING ORDER BY (eventTimestamp DESC);
Cassandra Schema
CassandraEvent Store
RabbitMQ Topic
DownstreamprocessingLog
events
Publish events
Cassandra & AMQP
CassandraEvent Store
RabbitMQ Topic
Downstreamprocessing
out-of-order events
ordered query results
Cassandra & AMQP
CassandraEvent Store
Kafka Topic
Downstreamprocessing
Event store listener
Publish events
Log events
Cassandra & Kafka
Processing Model: Summary
Events arrive partitioned, interleaved and out-of-order.
Events are sorted into event histories by aggregate type and id.
Events are sorted within event histories by event timestamp, not processing timestamp.
Event consumers need to take into account the possibility that an event history may be incomplete at the time it is read – consider using a watermark to give incoming events time to “settle”.
Programming Model: Core Metaphor
Consumer<Event>
Programming Model: Core Metaphor
You give me a Consumer<Event>, and I send Events to it one at a time:
Emitting Events
I implement Consumer<Event>, and handle Events that are sent to me.
Handling Events
Event-handling middleware is a chain of Consumer<Event>s that transforms, routes, persists and dispatches events. A single event submitted to this chain may be:
■ Serialised to JSON
■ Written to a message queue topic
■ Retrieved from the topic and deserialised
■ Persisted to an event store (e.g. Cassandra)
■ Published to an event handler which maintains a query-optimised view of part of the system
■ Published to an event handler which maintains an index of aggregates by event property values (e.g. lightbulbs by wattage)
Event-Handling Middleware
Java 8 Mapping
Java 8 Mapping
Java 8 Mapping
Querying and Replaying
Querying and Collecting
Kotlin Mapping
Kotlin Mapping
Kotlin Mapping
■ Event type and middleware implementing Consumer<Event>provide basic mechanics for creating, processing and handling events.
■ Java 8 mappings provide a convenient, type-safe method-mapping wrapper around these mechanisms.
■ Kotlin mappings provide an alternative wrapper based on immutable classes.
■ Other mappings are possible – Scala, Clojure, Java with POJOs…
Programming Model: Summary
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Future Directions
1. Stable Kafka integration
Future Directions
1. Stable Kafka integration
2. Avro integration
Future Directions
1. Stable Kafka integration
2. Avro integration
3. End-to-end async programming model
Questions?