AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy...
Transcript of AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy...
![Page 1: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/1.jpg)
AD31 DevOps Engineering 10:00 AM
AD31 -‐ Using Component Testing for Ultra-‐Fast Builds
Presented by:
Timothy Cochran Thoughtworks
Brought to you by:
888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ [email protected] -‐ https://agiledevopswest.techwell.com/
![Page 2: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/2.jpg)
Timothy Cochran Tim Cochrancurrently works for ThoughtWorks NYC, and has been full stack developer and architect for over 15 years. Working on everything from large distributed enterprise projects to small NGO visualization apps. He recently has been helping companies with digital transformation, moving towards continuous delivery and building a DevOps culture. He also is an automated testing zealot, practices TDD almost exclusively, he builds test strategy using the right combinations of unit, E2E and functional testing for clients.
![Page 3: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/3.jpg)
11
Utilizing Component Testing for Ultra fast Builds
©ThoughtWorks 2019 Commercial in Confidence
![Page 4: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/4.jpg)
GLOBAL SOFTWARE CONSULTANCY
2©ThoughtWorks 2019 Commercial in Confidence
![Page 5: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/5.jpg)
Tech Radar
ThoughtMobile
ThoughtWorks Incorporated
1993
2000
2005
1996
2004
2006
2008
2010
2014
2016, 2017, 2018
2018
500+ ThoughtWorkers
ThoughtWorks University
ThoughtWorks was one of the first
companies to run distributed agile
projects successfully
3000+ ThoughtWorkers
5000+ThoughtWorkers
2001
free, open-source software
Top 10 contributor in open source
![Page 6: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/6.jpg)
Why Automated Testing
4
Cost of QA
Productivity
Functional & Non-Functional Problems
User Sentiment
![Page 7: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/7.jpg)
The Problem
5
![Page 8: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/8.jpg)
Software Design
6
![Page 9: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/9.jpg)
Software Design
7
![Page 10: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/10.jpg)
8
Black Box
![Page 11: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/11.jpg)
9
E2E tests
Black Box
![Page 12: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/12.jpg)
10
Download code Compile Linting / Unit Test Code Coverage Upload Artifact
2 min 5 min 10 min 20 min 5 min
Build / Test pipeline (30 mins)
Prepare E2E tests
5 min 2 hour
E2E / Regression test pipeline (2 hours 5 mins)
Deployment pipeline (15 mins)
Copy binaries Restart Services Health Checks
5 min 5 mins3 mins
Schema Changes
1 min
![Page 13: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/13.jpg)
11
Download code Compile Linting / Unit Test Code Coverage Upload Artifact
2 min 5 min 10 min 20 min 5 min
Build / Test pipeline (30 mins)
Prepare
5 min
E2E / Regression test pipeline (35 mins)
Deployment pipeline (14 mins)
Copy binaries Restart Services Health Checks
5 min 5 mins3 mins
Schema Changes
1 min
30 mins
E2E tests
![Page 14: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/14.jpg)
Anti-patterns
12
Unit test - Deploy - Regression are separate pipelines, owned by different teams
Flakey tests, Fails one in 5 tests
Build is never green, Deploying with failing tests
Environments that don’t reflect Prod, Software still breaks in production
Broken Environments, Tests aren’t run because env is down or misconfigured
![Page 15: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/15.jpg)
13
![Page 16: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/16.jpg)
UI
Service
Unit
![Page 17: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/17.jpg)
Component Testing
15
![Page 18: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/18.jpg)
Software Design
16
![Page 19: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/19.jpg)
17
![Page 20: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/20.jpg)
18
E2E tests
![Page 21: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/21.jpg)
Component Testing
19
tests
tests
tests teststests
![Page 22: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/22.jpg)
Types of Components
20
A code module
A data pipeline
Third party service
UI Component
Microservice
![Page 23: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/23.jpg)
21
What goes wrong?
Dev Prod-Like Prod
Integration with other systems
Edge Data Cases
Misunderstanding of Requirements
More edge cases
User unsatisfied
Misconfigured
NFR problems
![Page 24: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/24.jpg)
22
E2E tests Component / Service
Edge Cases
Functionality (Acceptance)
Integration between components
Critical Path
Configuration
![Page 25: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/25.jpg)
23
What is a Component?
Code
DB IO External APIs
![Page 26: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/26.jpg)
A microservice?
24
![Page 27: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/27.jpg)
25
What is a Component? A service
Code
DB IO External APIs
Service
CartService
OrderService
AddToCartService
PostReviewService
![Page 28: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/28.jpg)
Domain Model
26
![Page 29: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/29.jpg)
How to test? - Out of Process
27
Component
Test
Stub dependencies using service virtualization
![Page 30: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/30.jpg)
How to test? - In Process
28
Component
Test
Stub dependencies using service virtualization
Or use in-memory stubs
![Page 31: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/31.jpg)
29
I use subcutaneous test to mean a test that operates just under the UI of an application.
![Page 32: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/32.jpg)
Contrived Example
30
![Page 33: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/33.jpg)
Contrived Example
31
UIReact
BFFNode.JS
ChargeServiceNode.JS
OtherServiceNode.JS
http
http
![Page 34: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/34.jpg)
32
UIReact
BFFNode.JS
ChargeServiceNode.JS
OtherServiceNode.JS
http
http
Test
![Page 35: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/35.jpg)
33
~1000ms: headless, locally
![Page 36: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/36.jpg)
34
UIReact
BFFNode.JS
ChargeServiceNode.JS
OtherServiceNode.JS
http
http
Test
![Page 37: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/37.jpg)
35
~50ms: locally
![Page 38: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/38.jpg)
36
How to test?
Advantage Disadvantage
In Process: Executed through Code
Fastest, Deterministic Doesn’t tests all the code (e.g. network, controller)
Out of Process: Executed through the API
Fast, Deterministic
Forces API based ecosystem
Retests boiler plate code, network layer
![Page 39: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/39.jpg)
Unit test
Unit test
Unit test
Unit test
37
Checkout Build
2 min 3 min
3 min
Linting
3 min
Test Coverage
5 min
In-Process Tests
5 min Deploy App
7 mins
Time for dev feedback - 22 mins
5 mins
Unit test
Out of Process tests
5 mins
E2E
![Page 40: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/40.jpg)
How to find entry points?
38
Understand what your application does, what capabilities does it provide?
Where is the complexity in your domain?
Talk to your engineers and business
Domain Driven Design (DDD) techniques is good way, establish the domain events, bounded contexts, entities, behaviour, values.
![Page 41: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/41.jpg)
39
UI testing
API
![Page 42: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/42.jpg)
40
UI testing
Service virtualization
![Page 43: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/43.jpg)
41
UI testing
Component test
![Page 44: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/44.jpg)
42
UI testinghttps://blog.pragmatists.com/genuine-guide-to-testing-react-redux-applications-6f3265c11f63
![Page 45: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/45.jpg)
43
E2E tests
![Page 46: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/46.jpg)
44
Culture
Quality is everyone’s responsibility
Cross functional teams
Deliberately design your application to be testable,
Think about testing from day one
![Page 47: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! - Cochrane.pdfAnti-patterns 12 Unit test - Deploy - Regression are separate pipelines, owned by different teams Flakey tests, Fails](https://reader034.fdocuments.us/reader034/viewer/2022042303/5ece609830baae06585bbe6b/html5/thumbnails/47.jpg)