Microservices in Clojure

36
Lucas Cavalcanti @lucascs Microservices in Clojure

Transcript of Microservices in Clojure

Page 1: Microservices in Clojure

Lucas Cavalcanti@lucascs

Microservices in Clojure

Page 2: Microservices in Clojure

ContextMicroservices

~80 Clojure services ~60 engineers

~10 teams 3.5 years old

Page 3: Microservices in Clojure

OOP Objects, the mainstream abstraction

Image @ http://www.eduardopires.net.br/2015/01/solid-teoria-e-pratica/

Page 4: Microservices in Clojure

What about Functional Programming?

SÃO PAULO, BRASIL

Page 5: Microservices in Clojure

TABLE OF CONTENTS

Immutability Components Pure Functions Schemas Ports and Adapters

SÃO PAULO, BRASIL

Page 6: Microservices in Clojure

Immutability

SOUTHEAST BRAZIL REGION FROM SPACE

Page 7: Microservices in Clojure

Immutability Definition

“If I’m given a value, it’s guaranteed that it won’t ever change”

Page 8: Microservices in Clojure

Technology choices Immutability

Page 9: Microservices in Clojure

Clojure Immutability

All default data structures are immutable: -Maps, Lists, Sets -Records

Mutability is explicit: atoms/refs @, dynamic vars *…*

Page 10: Microservices in Clojure

Datomic Immutability

Datomic stores the changes/transactions, not just the data -append only -db as a value -everything is data (transaction, schema, entities)

Page 11: Microservices in Clojure

Kafka Immutability

Persistent Queues/Topics -each consumer has its offset -ability to replay messages

Page 12: Microservices in Clojure

AWS + Docker Immutability

Ability to spin machines with a given image/configuration -Each build generates a docker image -Each deploy spins a new machine with the new

version -As soon as the new version is healthy, old version is

killed. (blue-green deployment)

Page 13: Microservices in Clojure

Components

SOUTHEAST BRAZIL REGION FROM SPACE

Page 14: Microservices in Clojure

Components https://github.com/stuartsierra/component

(defprotocol Database (query [this query-str]))(defrecord SomeDatabase [config-1 config-2 other-components] component/Lifecycle (start [this] (assoc this :connection (connect! config-1 config-2 other-components))) (stop [this] (release! (:connection this)) (dissoc this :connection)) Database (query [this query-str] (do-query! (:connection this) query-str)))

Page 15: Microservices in Clojure

System map Components

{:database #SomeDatabase{...} :http-client #HttpClient{...} :kafka #Kafka{...} :auth #AuthCredentials{...} ...}

-Created at startup -Entrypoints (e.g http server or kafka consumers) have access to all

components the business flows need -dependencies of a given flow are threaded from the entry point until

the end, one by one if possible -Thus no static access to system map! (e.g via a global atom) -Any resemblance to objects and classes is just coincidence ;)

Page 16: Microservices in Clojure

Pure functions

SOUTHEAST BRAZIL REGION FROM SPACE

Page 17: Microservices in Clojure

Pure functions Definition

"Given the same inputs, it will always produce the same output"

Page 18: Microservices in Clojure

Simplicity Pure functions

-easier to reason about, fewer moving pieces -easier to test, less need for mocking values -parallelizable by default, no need for locks or STMs

Page 19: Microservices in Clojure

Datomic Pure functions

-Datomic’s db as a value allows us to consider a function that queries the database as a pure function -db is a snapshot of the database at a certain point in time. -So, querying the same db instance will always produce the same result

Page 20: Microservices in Clojure

Impure functions Pure functions

-functions that produce side effects should be marked as such. We use `!` at the end. -split code which handles and transforms data from code that handles side effects -should be moved to the borders of the flow, if possible -Consider returning a future/promise like value, so side effect results can be composed (e.g with manifold or finagle)

https://github.com/ztellman/manifoldhttps://github.com/twitter/finagle

Page 21: Microservices in Clojure

Schema/Spec

SOUTHEAST BRAZIL REGION FROM SPACE

Page 22: Microservices in Clojure

Schema Legacy

Majority of our code base was written before clojure.spec existed, so I’ll be talking about the Schema library instead. Most principles apply to clojure.spec as well.

Page 23: Microservices in Clojure

Schema/Spec Documentation

-Clojure doesn’t force you to write types -parameter names are not enough -declaring types helps a lot when glancing at the function -values can be verified against a schema

Page 24: Microservices in Clojure

Function declaration Schema/spec

-All pure functions declare schemas for parameters and return value -All impure functions declare for parameters and don’t declare output type if it’s not relevant. -Validated at runtime in dev/test environments, on every function call -Validation is off on production.

Page 25: Microservices in Clojure

Wire formats Schema/Spec

-Internal schemas are your domain models -Wire schemas are how you expose data to other services/clients -If they are different, you can evolve internal schemas without breaking clients -Need an adapter layer -wire schemas are always validated on entry/exit points, specially in production -single repository for all wire schemas (for all 60+ services) -caveat: this repository has a really high churn. Beware

Page 26: Microservices in Clojure

Growing Schemas Spec-ulation

Please watch Rich Hickey’s talk at Clojure Conj 2016 Spec-ulation:

https://www.youtube.com/watch?v=oyLBGkS5ICk

Page 27: Microservices in Clojure

Ports and Adapters (a.k.a Hexagonal Architecture)

SOUTHEAST BRAZIL REGION FROM SPACE

Page 28: Microservices in Clojure

Ports and Adapters Definition

Core logic is independent to how we can call it (yellow) A port is an entry-point of the application (blue) An adapter is the bridge between a port and the core logic (red)

http://www.dossier-andreas.net/software_architecture/ports_and_adapters.htmlhttp://alistair.cockburn.us/Hexagonal+architecture

Page 29: Microservices in Clojure

Ports and Adapters (Nubank version) Extended Definition

Pure business logic (green) Controller logic wires the flow between the ports (yellow) A port is an entry-point of the application (blue) An adapter is the bridge between a port and the core logic (red)

Page 30: Microservices in Clojure

Ports (Components) Ports and Adapters

-Ports are initialised at startup -Each port has a corresponding component -Serializes data to a transport format (e.g JSON, Transit) -Usually library code shared by all services -Tested via integration tests

HTTP

Kafka

Datomic

File Storage

Metrics

E-mail

Page 31: Microservices in Clojure

Adapters (Diplomat) Ports and Adapters

-Adapters are the interface to ports

-Contain HTTP and Kafka consumer handlers

-Adapt wire schema to internal schema

-Calls and is called by controller functions

-Tested with fake versions of the port components, or mocks

HTTP

Kafka

Datomic

File Storage

Metrics

E-mail

Page 32: Microservices in Clojure

Controllers Ports and Adapters

-Controllers wires the flow between entry-point and the side effects

-Only deals with internal schemas

-Delegates business logic to pure functions

-Composes side effect results -Tested mostly with mocks

HTTP

Kafka

Datomic

File Storage

Metrics

E-mail

Page 33: Microservices in Clojure

Business Logic Ports and Adapters

-Handles and transforms immutable data

-Pure functions -Best place to enforce invariants and type checks (e.g using clojure.spec)

-Can be tested using generative testing

-Should be the largest part of the application

HTTP

Kafka

Datomic

File Storage

Metrics

E-mail

Page 34: Microservices in Clojure

Microservices Ports and Adapters

-Each service follows about the same design

-Services communicate with each other using one of the ports (e.g HTTP or Kafka)

-Services DON’T share databases

-HTTP responses contain hypermedia, so we can replace a service without having to change clients

-Tested with end to end tests, with all services deployed

Page 35: Microservices in Clojure

Clojure is simple

Keep your design simple

Keep your architecture simple

SÃO PAULO, BRASIL

Page 36: Microservices in Clojure

Lucas Cavalcanti@lucascs

Thank you