Post on 21-Jan-2018
tl;dr
• We are moving from complicated systems to complex systems
• Architecture is becoming more about technical leadership
• We must encode all requirements into a continuous delivery pipeline
16/10/2017 @danielbryantuk
@danielbryantuk
• Independent Technical Consultant, CTO at SpectoLabs
• Architecture, DevOps, Java, microservices, cloud, containers
• Continuous Delivery (CI/CD) advocate
• Leading change through technology and teams
16/10/2017 @danielbryantuk
bit.ly/2jWDSF7
Velocity (with stability) is key to business success
16/10/2017 @danielbryantuk
Adrian Cockcroft (2015) www.slideshare.net/Indicee/cloud-trends-2015-pdf
“Continuous delivery is achieved when stability and speed can satisfy business demand.
Discontinuous delivery occurs when stability and speed are insufficient.”
- Steve Smith (@AgileSteveSmith)
16/10/2017 @danielbryantuk
16/10/2017 @danielbryantuk
Simple(Sense, Categorise, Respond)
Complicated(Sense, Analyse, Respond)
Complex(Probe, Sense, Respond)
1990sMonolithsSingle languageIn-house hardware (servers, SAN, networks)Manual config and scriptingOptimise for Stability (MTBF)Specialist staff/departments
2010sMicroservices, functions, SaaS-all-the-thingsPolyglot languagesCloud and containers (Datacenter as a Computer)Software-Defined EverythingOptimise for innovation (and MTTR)Business teams (“FinDev”, SRE and Platform Team)
2000sMonoliths, Coarse-grained SOA, SaaSFrontend/backend language“Co-lo” or private datacentersConfiguration managementOptimise for Recovery (MTTR)Generalist teams (Full Stack and “DevOps”)
Chaotic(Act, Sense, Respond)
From monolith to…
16/10/2017 @danielbryantuk
UI / Biz / Repo
Monolith
Domains
Modules, components,frameworks, libraries
http://scs-architecture.org/
16/10/2017 @danielbryantuk
Self Contained Systems (SCS)
Microservices
Function-as-a-Service“Serverless”
16/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance, control
Domain modelling, SRP, evolution
Function (in/out),rapid evolution
Organisation Implemented and maintained (typically) by single team
Implemented by different org units. Maintenance done elsewhere
Services implemented and owned by product teams
Implemented by pioneers (hipsters?)
Deployment Monolithic deployment Monolithic orchestration of multiple services
Services deployed individually
Functions deployed individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process communication
None RPC or messaging, typically via middleware (ESB/MQ)
RPC via dumb pipesand smart endpoints, messaging/events
Events
Pioneers / stewards
Organisations, community or individuals
Enterprises and Vendors
Community and highperf organisations
Vendors/community
Core ArchitecturalConstraints
Language and framework Canonical domain model, standards
Interoperability Cost (Gbs/ms)
16/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance, control
Domain modelling, SRP, evolution
Function (in/out),rapid evolution
Organisation Implemented and maintained (typically) by single team
Implemented by different org units. Maintenance done elsewhere
Services implemented and owned by product teams
Implemented by pioneers (hipsters?)
Deployment Monolithic deployment Monolithic orchestration of multiple services
Services deployed individually
Functions deployed individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process communication
None RPC or messaging, typically via middleware (ESB/MQ)
RPC via dumb pipesand smart endpoints, messaging/events
Events
Pioneers / stewards
Organisations, community or individuals
Enterprises and Vendors
Community and highperf organisations
Vendors/community
Core ArchitecturalConstraints
Language and framework Canonical domain model, standards
Interoperability Cost (Gbs/ms)
Architecture fundamentals
• Coupling• ”Components have little or no knowledge of other components”
• Interfaces
• Schema
• Cohesion • “Degree to which the elements within a component belong together”
• Single responsibility principle
• Separation of concerns
16/10/2017 @danielbryantuk
Coupling, Cohesion and Continuous Delivery
• Design• Cohesion makes reasoning easy
• Loose coupling reduces impact
• Build, unit and integration• Cohesion limits dependencies
• Loose coupling allows simulation
• End-to-end tests• Cohesion/coupling orchestration
• Deployment• Cohesion minimises number of
components in play
• Coupling leads to “monoliths”
• Observability• Cohesive is easier to understand
• High coupling typically leads to large blast radius and “murder mystery” style debugging
16/10/2017 @danielbryantuk
Working locally
16/10/2017 @danielbryantuk
https://news.ycombinator.com/item?id=13960107https://opencredo.com/working-locally-with-microservices/https://www.datawire.io/telepresence/ | https://hoverfly.io/
Testing: Core concepts
16/10/2017 @danielbryantuk
/lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/martinfowler.com/bliki/TestPyramid.html
Testing: Core concepts
16/10/2017 @danielbryantuk
martinfowler.com/articles/microservice-testing/#testing-progress-3
Functional: Outside-in
16/10/2017 @danielbryantuk
https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/http://www.thucydides.info/docs/serenity/
Talking of (service) contracts…
• APIs are service contracts
• Consumer-driven Contracts
• martinfowler.com/articles/consumerDrivenContracts.html
16/10/2017 @danielbryantuk
Talking of (service) contracts…
16/10/2017 @danielbryantukdocs.pact.io
cloud.spring.io/spring-cloud-contract
Talking of (messaging) contracts…
• What about messaging?
• Message schema are an API
• www.infoq.com/presentations/contracts-streaming-microservices
16/10/2017 @danielbryantuk
Talking of (messaging) contracts…
16/10/2017 @danielbryantuk
www.infoq.com/presentations/contracts-streaming-microservices
docs.confluent.io/current/schema-registry/docs/maven-plugin.html
Contract verification
• Is it worth the cost?
• Tradeoff: Trust and comms• Startups / SMEs
• Enterprises…
• My (anecdotal) experience
• Orchestration vs choreography
• Orchestration:• Asserting state (output)
• Choreography• Asserting behaviour (flow)
• Contracts are part of this!
16/10/2017 @danielbryantuk
Testing NFRs in the build pipeline
• Architecture quality• SonarQube / Code Climate
• Performance and Load testing • Gatling / Locust / Bees with m’guns
• Security testing • OWASP Dependency check / Bdd-security
• Docker Bench for Security / CoreOS Clair
16/10/2017 @danielbryantuk
Delaying NFRs to the ‘Last Responsible Moment’
Newsflash!
Sometimes the
last responsible moment
is up-front
Modern architectures don’t necessarily make this easier
16/10/2017 @danielbryantuk
Separating deployment and release
• Modern architectures demand independent deploy and release
• Feature flags
• Shadow traffic
• Dark launching
16/10/2017 @danielbryantuk
blog.twitter.com/engineering/en_us/a/2015/diffy-testing-services-without-writing-tests.html
www.youtube.com/watch?v=lqRQYEHAtpk
Modern architectures provide flexibility
• Routing between multiple service versions• Service meshes, Istio, Linkerd etc
• Serverless allows multiple environments to co-exist
16/10/2017 @danielbryantuk
16/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance, control
Domain modelling, SRP, evolution
Function (in/out),rapid evolution
Organisation Implemented and maintained (typically) by single team
Implemented by different org units. Maintenance done elsewhere
Services implemented and owned by product teams
Implemented by pioneers (hipsters?)
Deployment Monolithic deployment Monolithic orchestration of multiple services
Services deployed individually
Functions deployed individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process communication
None RPC or messaging, typically via middleware (ESB/MQ)
RPC via dumb pipes/smart endpoints, messaging/events
Events
Pioneers / stewards
Organisations, community or individuals
Enterprises and Vendors
Community and highperf organisations
Vendors/community
Core ArchitecturalConstraints
Language and framework Canonical domain model, standards
Interoperability Cost
16/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Cohesion and coupling enforced at modules
CD focuses on end-to-end functionality
Provide goals and “best practice” examples
Cohesion and coupling enforced at
service level
CD focuses on service integrity
Provide objectives and standards
Cohesion and coupling enforced at
service API level
CD focuses on service interaction
Provide principles and guidelines
Cohesion and coupling enforced at
function API level
CD focuses on service output/state
Provide values and ”rules of
engagement”
In conclusion…
• We are moving from complicated systems to complex systems• Design and test with coupling and cohesion in mind
• Architecture is becoming more about technical leadership• Recognise your situation and influence accordingly
• We must encode all requirements into a continuous delivery pipeline• Both functional and nonfunctional requirements
16/10/2017 @danielbryantuk
Thanks for listening…
Twitter: @danielbryantuk
Email: daniel.bryant@tai-dev.co.uk
Writing: https://www.infoq.com/profile/Daniel-Bryant
Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM
16/10/2017 @danielbryantuk
bit.ly/2jWDSF7
Coming soon!