How to break apart a monolithic system safely

without destroying your teamMatthew Skelton, Skelton Thatcher Consulting

@matthewpskelton

07 November 2016, Amsterdam, NL - #velocityconf

TodayCognitive load for teams

‘Monolith’Code Forensics

Team-first boundariesMonolith-splitting recipe

For now, let’s forget:

MicroservicesCQRS / Event Sourcing

Queues(Architectural changes)

Team-first digital transformation30+ organisations

UK, US, EU, India, China

How to break apart a monolithic system safely without destroying your

team

Safer, more rapid changes to software systems

(Business Agility)

A ‘team-first’ approach to software subsystem

boundaries

TEAM

TEAM

capabilitiesappetite & aptitude

understanding

responsibilities

(assumption)

the team is stable, slowly changing, and long-lived

#NoProjects

Conway’s Law

“organizations which design systems ... are constrained to

produce designs which are copies of the communication structures of

these organizations”

– Mel Conway, 1968http://www.melconway.com/Home/Conways_Law.html

Front-end developers

Back-end developers

‘Reverse Conway’Tobbe Gyllebring (@drunkcod)

homomorphic force

(#Conway #Yawnoc)

HT @allankellynet

(same) (shape)

Cognitive load for teams

Cognitive load

the total amount of mental effort being used in

the working memory(see Sweller, 1988)

Cognitive load

Intrinsic Extraneous (Irrelevant )

Germane (Relevant)

‘Hacking Your Head’: Jo Pearce

See http://www.slideshare.net/JoPearce5/hacking-your-head-managing-information-overload-45-mix

@jdpearce

We have SCIENCE!

Science since 1988 (!)• Driskell et al, 1999 ‘Does Stress Lead to a Loss of Team Perspective?’ Group Dynamics:

Theory, Research, and Practice 3, no. 4 (1999): 291.

• Fan et al, 2010 ‘Learning HMM-Based Cognitive Load Models for Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119.

• Ilgen & Hollenbeck, 1993 ‘Effective Team Performance under Stress and Normal Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993.

• Johnston et al, 2002 ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002.

• Sweller, John, 1994 ‘Cognitive Load Theory, Learning Difficulty, and Instructional Design’. Learning and Instruction 4 (1994): 295–312.

• Sweller, John, 1988. ‘Cognitive Load during Problem Solving: Effects on Learning’. Cognitive Science 12, no. 2 (1988): 257–285.

“stress impacts teamperformance … by narrowing or weakening the team-level

perspective required foreffective team behavior.”

– Driskell et al, 1999Group Dynamics: Theory, Research, and Practice 1999, Vol. 3, No. 4,291-302

(not just ‘pop’ science!)

‘Monolith’

monolith

μόνο λίθος“single stone”

Reiner Flassig - CC BY-SA 2.0 de - Wikipedia

“Don’t start with a monolithwhen your goal is a

microservices architecture”

– Stefan Tilkov, innoQ

http://martinfowler.com/articles/dont-start-monolith.html

“Start monolithic and extract”

– Tammer Saleh, Pivotal

https://www.infoq.com/presentations/cloud-anti-patterns

A ‘team-first’ approach to software subsystem

boundaries

Types of software monoliths•Application monolith•Joined at the DB•Monolithic build (rebuild everything)•Monolithic releases (coupled)•Monolithic thinking (standardisation)

Application monolith

Single block of codeDeployed as a unit

Joined at the DB

Difficult to change separately (but not impossible)Risk is (probably) elevated

Chris Collyer, http://www.stone-circles.org.uk/stone/pentreifan.htm

Monolithic builds

One gigantic CI build just to get a new version of any component

Monolithic releases

Smaller components bundled together into a ‘release’

Monolithic thinking

‘One-size-fits-all’ for teamsAssumption that minimising variation is A Good Thing

Dangers of splitting a monolith•Reduced domain consistency•Data duplication (unintentional)•Additional operational complexity due to distributed system and async messaging•Degraded UX across the product

Splitting a monolith

Reiner Flassig - CC BY-SA 2.0 de - Wikipedia

Choose the right technique for splitting Understand the nature of the monolith

‘Fracture planes’ for code•Business domain bounded context•Regulatory compliance•Change cadence•Risk•Performance isolation•User personas•Team location

‘Fracture planes’ for code

Ask: Could we consume this thing as a service?

‘Fracture planes’ for code

Ask: Could we provide this thing as a service?

Code Forensics

ForensicsYour Code as a Crime SceneAdam Tornhill

Adam TornhillCode, Crime, Complexity: Analyzing software with forensic psychologyAdam TornhillTEDxTrondheim

youtube.com/watch?v=qJ_hplxTYJw

‘Code Maat’ tool

Adam Tornhill, http://www.adamtornhill.com/articles/crimescene/codeascrimescene.htm

Code City plus Code Maat forensics

Beware of badly-named subsystems

"information-poor abstract names are magnets for extra [unwanted] responsibilities"

– Adam Tornhillp.185, Your Code as a Crime Scene

Team-first boundaries

DevOpsTopologies.com

Team types

Component team

Platform / ’substrate’ team

Supporting / ‘productivity’ team

Product/Feature team

Team typesde

vops

topo

logies.com Platform / ’substrate’ team

Product/Feature team

Team typesde

vops

topo

logies.com

Component team

Platform / ’substrate’ team

Product/Feature team

Team typesde

vops

topo

logies.com

Component team

Platform / ’substrate’ team

Product/Feature team

Supporting / ‘productivity’ team

Code repositoriesAlign repositories to subsystem boundaries

Avoid monolithic-y repos like TFS** Don’t get me started on TFS, grrr…

Code repositories

Repo 1 Build Test Deploy Run

Repo 2 Build Test Deploy Run

Repo 3 Build Test Deploy Run

“You can use a monorepo only if your organisation has published a

scientific paper on Computer Science. Otherwise, use one repo

per deployable runnable thing.”

– Matthew SkeltonLondonCD meetup group, 11 Oct 2016

Find natural or available ‘fracture planes’ for splitting

a monolith

(with the team in mind)

Industry experienceExamples from recent work with clients

Different monolithsDifferent ‘fracture planes’

Software for R&D teamsCustomers: over 80% of top 20

global pharma companies

2014:Monolithic: builds, releasesPharma GxP: traceabilityChanges slow and tricky

Fracture plane: business domain

Fracture plane: technology

Compose with packages Logging for insights

Technology splitsImproved traceability

Major UK broadcaster (TV + online)Increasing digital delivery of content

2014:Wide mix of technologies

Need for visibility of changesNeed for rapid turnaround

Fracture planes: read-only, location & cadence

Split off Reporting (read-only)Split on change cadence

Increased change throughput

UK’s second largest credit reference agency

Customers: major banks + large orgs

2014:6-weekly monolithic releaseOutage-sensitive customers

Fracture planes: risk & business domain

Split releases based on riskAlign teams to value streamsMore rapid release cadence

Software for Internal CommsCustomers: FTSE 100 & Fortune 1000

2015:Successful system (monolithic)

ISO 27001 complianceAim to expand the org (x N)

Fracture plane: business domain

Optimise split for team engagement & personas

Minimise ISO 27001 footprintAided organisation expansion

When not to split a monolith•‘Heritage’ ERP system (‘cloudified’)•No native Unit Test framework•20-30 min startup times•VMs need 56GB RAM (yes)•CI builds take 50 mins

Monolith-splitting recipeTried and tested!

Monolith-splitting recipe

1. Instrument the monolith – logging 2. Grok data flows and fault responses3. Align teams to available segments

4. Split off segments one-by-one

Instrument the monolith

Instrument the monolith

Instrument the monolith

search by event

Event ID

{Delivered,InTransit,Arrived}

transaction trace

Correlation ID

612999958…

Technical

Domain

public enum EventID

{

// Badly-initialised logging data

NotSet = 0,

// An unrecognised event has occurred

UnexpectedError = 10000,

ApplicationStarted = 20000,

ApplicationShutdownNoticeReceived = 20001,

PageGenerationStarted = 30000,

PageGenerationCompleted = 30001,

MessageQueued = 40000,

MessagePeeked = 40001,

BasketItemAdded = 60001,

BasketItemRemoved = 60002,

CreditCardDetailsSubmitted = 70001,

// ...

}

BasketItemAdded = 60001

BasketItemRemoved = 60002

Instrument the monolith

Correlation ID Logs

Event ID

use logging as a channel/vector to

make distributed systems more testable

use logging as a channel/vector to

make distributed systems more testable

Grok data flows and fault responses

Grok data flows and fault responses

Correlation ID

Event ID

Unexpected collaborating subsystems

Undetected fault condition

Grok data flows and fault responses

Correlation ID

Event ID

Adjust subsystem boundaries

Fix poor fault responses

runbooktemplate.info

Run Book dialogue sheets

Align teams to available segments

Align teams to available segments

Align teams to available segmentsMap to business domain

Align teams to available segmentsIdentify likely components or ‘platform’ elements

Split off segments one-by-one

Split off segments one-by-one

Split off segments one-by-oneSeparate:- Builds- Infrastructure- Deployments- Versions- Lifecycle

Team needs / responsibilities / capabilities come first

use logging as a channel/vector to

make distributed systems more testable

Invest in Build & Release Engineering

Monolith-splitting recipe *

1. Instrument the monolith – logging 2. Grok data flows and fault responses3. Align teams to available segments

4. Split off segments one-by-one(5. Invest in Build & Release Engineering)

* The simplistic version

Monolith-splitting recipe *

1. Instrument 2. Grok behaviour

3. Align teams4. Split off segments

5. Invest in Build & Release* The simplistic version

Monolith-splitting recipe *

* The simplistic version

1. Instrument 2. Grok behaviour

3. Align teams4. Split off segments

5. Invest in Build & Release

Determine monolith type(Apply ‘Code Forensics’)

Find ‘fracture planes’Assess cognitive load for teams

Use the monolith-splitting recipe

How to break apart a monolithic system safely

without destroying your team

Further material

Books & articles• Working Effectively with Legacy Code, by Michael Feathers• Building Microservices by Sam Newman (O’Reilly, 2015)• Managing Cognitive Load for Team Learning by Jo Pearce

http://12devsofxmas.co.uk/2015/12/day-3-managing-cognitive-load-for-team-learning/

• Continuous Delivery with Windows and .NET by Matthew Skelton and Chris O’Dell (O’Reilly, 2016) http://cdwithwindows.net/

• Team Guide to Software Operability by Matthew Skelton and Rob Thatcher (Skelton Thatcher Publications, 2016) http://operabilitybook.com/

• Ye Olde Monolith by Pter Pilgrim https://dzone.com/articles/ye-olde-monolith

Training• From Monolith to Microservices (online training) – Sam Newman,

author of Building Microserviceshttp://www.oreilly.com/live-training/from-monolith-to-microservices.html

• Run Book template & Run Book dialogue sheets http://runbooktemplate.info/

Talks & slides• Hacking Your Head – Managing Information Overload, by Jo

Pearce - http://www.slideshare.net/JoPearce5/hacking-your-head-managing-information-overload-45-mix /http://conferences.oreilly.com/oscon/open-source-eu/public/schedule/detail/53013

• Principles of Microservices by Sam Newman @ Devoxx 2015 https://www.youtube.com/watch?v=PFQnNFe27kU

Research papers• Driskell, James E., Eduardo Salas, and Joan Johnston. ‘Does Stress Lead to a Loss of Team

Perspective?’ Group Dynamics: Theory, Research, and Practice 3, no. 4 (1999): 291.• Fan, Xiaocong, Po-Chun Chen, and John Yen. ‘Learning HMM-Based Cognitive Load Models for

Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119. • Ilgen, Daniel R., and John R. Hollenbeck. ‘Effective Team Performance under Stress and Normal

Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA284683.

• Johnston, Joan H., Stephen M. Fiore, Carol Paris, and C. A. Smith. ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA525820.

• Sweller, John. ‘Cognitive Load Theory, Learning Difficulty, and Instructional Design’. Learning and Instruction 4 (1994): 295–312.

• Sweller, John. ‘Cognitive Load during Problem Solving: Effects on Learning’. Cognitive Science 12, no. 2 (1988): 257–285.

Determine monolith type(Apply ‘Code Forensics’)

Find ‘fracture planes’Assess cognitive load for teams

Use the monolith-splitting recipe

How to break apart a monolithic system safely

without destroying your team

thank you

Matthew Skelton@matthewpskelton

skeltonthatcher.com