Continuous Delivery Patterns for Modern Java Architectures · Continuous Delivery Patterns for...

Continuous Delivery Patterns for Modern Java Architectures

Daniel Bryant

@danielbryantuk

Architecture: Expectations versus reality

11/04/2018 @danielbryantuk

“DevOps”

tl;dr

• We are moving from complicated systems to complex systems

• Architecture is becoming more about technical leadership

• We must encode all requirements into a continuous delivery pipeline


@danielbryantuk

• Independent Technical Consultant, Product Architect at Datawire

• Architecture, DevOps, Java, microservices, cloud, containers

• Continuous Delivery (CI/CD) advocate

• Leading change through technology and teams


bit.ly/2jWDSF7

http://bit.ly/2jWDSF7

Setting the Scene


Continuous Delivery

• Produce valuable and robust software in short cycles

• Optimising for feedback and learning

• Not (necessarily) Continuous Deployment


Velocity (with stability) is key to business success

“Continuous delivery is achieved when stability and speed can satisfy business demand.

Discontinuous delivery occurs when stability and speed are insufficient.”

- Steve Smith (@SteveSmithCD)



Feedback: - Was our hypothesis proven?- How can we improve biz,

architecture and ops?

Let’s bring in some containers (or additional packaging)


What are our core challenges with modern Java app architectures?



Multiple services/pipelines


Gated application deployment

1.

2.

3.


Independent service deployment

Architecture 101


From monolith to…


UI / Biz / Repo

Monolith

Domains

Modules, components,frameworks, libraries

http://scs-architecture.org/

http://scs-architecture.org/


Self Contained Systems (SCS)

Microservices

Function-as-a-Service“Serverless”


Monoliths SOA Microservices / SCS FaaS / Serverless

Scope Project Enterprise Product Feature (or glue?)

Focus Swiss Army Knife Reuse, governance, control

Domain modelling, SRP, evolution

Function (in/out),rapid evolution

Organisation Implemented and maintained (typically) by single team

Implemented by different org units. Maintenance done elsewhere

Services implemented and owned by product teams

Implemented by pioneers (hipsters?)

Deployment Monolithic deployment Monolithic orchestration of multiple services

Services deployed individually

Functions deployed individually

Management None Centralised Distributed Chaotic / Orchestrated

Inter-process communication

None RPC or messaging, typically via middleware (ESB/MQ)

RPC via dumb pipesand smart endpoints, messaging/events

Events

Pioneers / stewards

Organisations, community or individuals

Enterprises and Vendors

Community and highperf organisations

Vendors/community

Core ArchitecturalConstraints

Language and framework Canonical domain model, standards

Interoperability Cost (Gbs/ms)

Architecture fundamentals

• Coupling• ”Components have little or no knowledge of other components”

• Interfaces

• Schema

• Cohesion • “Degree to which the elements within a component belong together”

• Single reason to change

• Separation of concerns


Coupling, Cohesion and Continuous Delivery

• Design• Cohesion makes reasoning easy

• Loose coupling reduces impact

• Build, unit and integration• Cohesion limits dependencies

• Loose coupling allows simulation

• End-to-end tests• Cohesion/coupling orchestration

• Deployment• Cohesion minimises number of

components in play

• Coupling leads to “monoliths”

• Observability• Cohesive is easier to understand

• High coupling typically leads to large blast radius and “murder mystery” style debugging


Testing 101


Testing: Core concepts


/lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/martinfowler.com/bliki/TestPyramid.html

http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/

https://martinfowler.com/bliki/TestPyramid.html

Testing: Core concepts


martinfowler.com/articles/microservice-testing/#testing-progress-3 martinfowler.com/articles/practical-test-pyramid.html

https://martinfowler.com/articles/microservice-testing/#testing-progress-3

https://martinfowler.com/articles/practical-test-pyramid.html

Testing: Core (microservice) concepts


https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16

https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16

Testing microservice integration


Functional testing: End-to-end


Functional Testing: Outside-in


https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/http://www.thucydides.info/docs/serenity/

https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/

http://www.thucydides.info/docs/serenity/

Functional


Talking of contracts…


Talking of (service) contracts…

• APIs are service contracts

• Consumer-driven Contracts

• martinfowler.com/articles/consumerDrivenContracts.html


https://martinfowler.com/articles/consumerDrivenContracts.html

Talking of (service) contracts…

11/04/2018 @danielbryantukdocs.pact.io

cloud.spring.io/spring-cloud-contractgithub.com/spring-cloud-samples/spring-cloud-contract-samples

https://docs.pact.io/

https://cloud.spring.io/spring-cloud-contract/

https://github.com/spring-cloud-samples/spring-cloud-contract-samples

Talking of (messaging) contracts…

• What about messaging?

• Message schema are an API

• www.infoq.com/presentations/contracts-streaming-microservices


https://www.infoq.com/presentations/contracts-streaming-microservices

Talking of (messaging) contracts…


www.infoq.com/presentations/contracts-streaming-microservices

docs.confluent.io/current/schema-registry/docs/maven-plugin.html

https://www.infoq.com/presentations/contracts-streaming-microservices

https://docs.confluent.io/current/schema-registry/docs/maven-plugin.html

Contract verification

• Is it worth the cost?

• Tradeoff: Trust/comms vs time• Startups / SMEs

• Enterprises…

• My (anecdotal) experience

• Choreography vs orchestration



https://netflix.github.io/conductor/http://rockscript.io/https://nifi.apache.org/docs.htmlhttps://aws.amazon.com/documentation/step-functions/http://camel.apache.org/https://blog.bernd-ruecker.com/orchestrating-azure-functions-using-bpmn-and-camunda-a-case-study-ff71264cfad6

http://rockscript.io/


https://nifi.apache.org/docs.html




Contract verification

• Is it worth the cost?

• Tradeoff: Trust and comms• Startups / SMEs

• Enterprises…

• My (anecdotal) experience

• Choreography vs orchestration

• Choreography• Verifying behaviour (interactions)

• Contracts are part of this!

• London school TDD

• Orchestration• Verify state (output)

• Lint/validate orchestration DSL

• Chicago school TDD


Measure what matters



Feedback: - Was our hypothesis proven?- How can we improve biz,

architecture and ops?

Visibility for the business


Testing NFRs in the build pipeline

• Architecture quality• SonarQube / Code Climate

• Performance and Load testing • Gatling / Locust / Bees with m’guns

• Security testing • OWASP Dependency check / bdd-security

• Docker Bench for Security / CoreOS Clair


Architectural Visibility


Unit Testing Architecture


https://www.archunit.org/

https://www.archunit.org/

Security Visibility: Basic Java Code Scanning


Security Visibility: Static Package Scanning


https://github.com/arminc/clair-scanner

https://github.com/arminc/clair-scanner

Serverless security

“Since the OS is unreachable, attackers will shift their attention to the areas that remain exposed – and first amongst those would

be the application itself.”

Responsibility for addressing vulnerable app libraries falls to you – the function developer.


https://www.infoq.com/articles/serverless-security

https://www.infoq.com/articles/serverless-security

Security Visibility: Dependencies


www.owasp.org/index.php/OWASP_Dependency_Check

https://www.owasp.org/index.php/OWASP_Dependency_Check

Delaying NFRs to the ‘Last Responsible Moment’

Newsflash!

Sometimes the

last responsible moment

is up-front

Modern architectures don’t necessarily make this easier


Wrapping Up




Scope Project Enterprise Product Feature (or glue?)

Focus Swiss Army Knife Reuse, governance, control

Domain modelling, SRP, evolution

Function (in/out),rapid evolution

Organisation Implemented and maintained (typically) by single team

Implemented by different org units. Maintenance done elsewhere

Services implemented and owned by product teams

Implemented by pioneers (hipsters?)

Deployment Monolithic deployment Monolithic orchestration of multiple services

Services deployed individually

Functions deployed individually

Management None Centralised Distributed Chaotic / Orchestrated

Inter-process communication

None RPC or messaging, typically via middleware (ESB/MQ)

RPC via dumb pipes/smart endpoints, messaging/events

Events

Pioneers / stewards

Organisations, community or individuals

Enterprises and Vendors

Community and highperf organisations

Vendors/community

Core ArchitecturalConstraints

Language and framework Canonical domain model, standards

Interoperability Cost



Cohesion and coupling enforced at modules

CD focuses on end-to-end functionality

Provide goals and “best practice” examples

Cohesion and coupling enforced at

service level

CD focuses on service integrity

Provide objectives and standards


service API level

CD focuses on service interaction

Provide principles and guidelines


function API level

CD focuses on service output/state

Provide principles and guidelines

In conclusion…

• We are moving from complicated systems to complex systems• Design and test with coupling and cohesion in mind

• Architecture is becoming more about technical leadership• Recognise your situation and influence accordingly

• We must encode all requirements into a continuous delivery pipeline• Both functional and nonfunctional requirements


Bedtime reading…


blog.christianposta.com/microservices/low-risk-monolith-to-microservice-evolution-part-iii/

http://blog.christianposta.com/microservices/low-risk-monolith-to-microservice-evolution-part-iii/

Thanks for listening…

Twitter: @danielbryantuk

Email: [email protected]

Writing: https://www.infoq.com/profile/Daniel-Bryant

Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM


bit.ly/2jWDSF7

Coming soon!

mailto:https://twitter.com/danielbryantuk

mailto:[email protected]

https://www.infoq.com/profile/Daniel-Bryant

https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM

http://bit.ly/2jWDSF7

Continuous Delivery Patterns for Modern Java Architectures · Continuous Delivery Patterns for...

Documents

Transcript of Continuous Delivery Patterns for Modern Java Architectures · Continuous Delivery Patterns for...