Continuous Testing in Vegas
-
Upload
jaredrrichardson -
Category
Software
-
view
77 -
download
0
Transcript of Continuous Testing in Vegas
![Page 1: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/1.jpg)
Continuous TestingJared Richardson
Agile Artisans
@JaredRichardson
GrowsMethod.com
![Page 2: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/2.jpg)
Who am I?
![Page 3: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/3.jpg)
3
Cofounder of GROWS Methodology
Author Ship It! and Career 2.0
Screencast editor PragProg.com
2nd public signatory of the Agile Manifesto
Started AgileRTP in 2007
Coach
AgileArtisans.com
Jared Richardson
![Page 4: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/4.jpg)
Who are you?
![Page 5: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/5.jpg)
What’s up with the material?
Major rewrite
Introduction
Slides on USB
On Slideshare.com
Provided to conference as well
![Page 6: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/6.jpg)
Who’s a User?
Already using continuous integration?
Continuous deployment?
Which tools?
Test first?
![Page 7: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/7.jpg)
What would you say you do here?
Roles?
Responsibilities?
Why are you here?
What are your tests for this session?
![Page 8: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/8.jpg)
Objectives
Understand Continuous Feedback in context of a healthy SDLC
Understand the Continuous Paradigm
Understand an end-to-end Continuous workflow
Understand how to scale your efforts
![Page 9: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/9.jpg)
Outline
GROWS Methodology
Continuous paradigm
Battleship!
Continuous Integration (benefits & workflow)
Continuous Testing (benefits & workflow)
Miscellaneous tips & tricks
![Page 10: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/10.jpg)
Breaks
Halfway (90 minutes)?
Each hour ?
Speak up!
![Page 11: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/11.jpg)
SDLC
![Page 12: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/12.jpg)
Idea
Verifies
Use or delete Implements
Software Development Life Cycle
![Page 13: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/13.jpg)
wikipedia.org/wiki/Waterfall_model#/media/File:Waterfall_model.svg
Traditional Approach
![Page 14: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/14.jpg)
![Page 15: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/15.jpg)
Requirements
Write Code
The Traditional Approach
Demo
Rewrite (60%)
“Discussion”
The architects generate a
document and hand it off
An isolated event, primarily worked by the
customer proxy
Arch
Code like mad!
We’re starting late!
Not what we meant!
Wrong archIgnored my docs
etcso on
![Page 16: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/16.jpg)
Demo
Write Code
Requirements
A More Agile Approach
Before significant
time and $$ is invested
Discuss
Include the Golden Triad
Dev QA
Customer
Rewrite
![Page 17: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/17.jpg)
Demo
Write Code
Architecture
Proof of Concept
Requirements Arch
![Page 18: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/18.jpg)
Why Feedback?
Feedback is good for budgets
Cheaper
Faster
More accurate
Lower risk
![Page 19: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/19.jpg)
flickr.com/photos/alancleaver/4293345631
To Err is HumanWhat’s your recovery
plan?
![Page 20: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/20.jpg)
ExperimentationLearn about software by writing software
Can’t study your way into experience
Like riding a bike
Create feedback loops
flickr.com/photos/schwar/6801765768
![Page 21: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/21.jpg)
Requirements
Rewrite
Discuss
Development
Node.js
Bootstrap
Angular.js
Pick one
SOA
Monolithic
Peer to peer
Pick one
Developersdisagree on
which architectureis best
Continue until you can demo
Developersdisagree on
which Javascripttoolkit is best
Demo
ReviseRewriteAdjust
Frequent demos discover
communication failures early
![Page 22: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/22.jpg)
Discuss
IntentionalExperimentatio
nStudy
Experiment
Learn
![Page 23: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/23.jpg)
Decide! Learn
Learn Decide!
Two Paths
![Page 24: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/24.jpg)
GROWS Methodology
![Page 25: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/25.jpg)
Directed Empiricism
![Page 26: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/26.jpg)
Why Bother?
https://www.mountaingoatsoftware.com/blog/agile-succeeds-three-times-more-often-than-waterfall
![Page 27: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/27.jpg)
AcceptanceWe’re not very good
We’re going to mess up
Mistakes are inevitable
How do we recover?
![Page 28: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/28.jpg)
ContinuousParadigm
![Page 29: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/29.jpg)
ExperimentationWe’re not very good
We’re going to mess up
Mistakes are inevitable
How do we recover?
![Page 30: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/30.jpg)
STICK ExperimentsShort
Time boxed
Informative
Concrete
Key
Specific
flickr.com/photos/peterpearson/1062279922
![Page 31: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/31.jpg)
Automated TestsAnswer Questions
Is this done?
Does it still work?
Does it work somewhere besides my machine?
This is what I’m asking you to write.
Everything is checked in.
![Page 32: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/32.jpg)
flickr.com/photos/bobowen/4501875907
Product or system
flickr.com/photos/30691679@N07/3808395213
Module/Sub
System
flickr.com/photos/kevino/1438502858
Class
![Page 33: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/33.jpg)
Feedback Loops are Fractal
flickr.com/photos/dancentury/4579906149
![Page 34: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/34.jpg)
Agility Is...
Tighter Feedback Loops
![Page 35: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/35.jpg)
Which Level?
35
![Page 36: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/36.jpg)
36
Days
Iterations
Months
Quarters
Years
Optional
Optional
![Page 37: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/37.jpg)
C*/Board
37
![Page 38: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/38.jpg)
VP
38
![Page 39: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/39.jpg)
Director Level
39
![Page 40: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/40.jpg)
Management Level
40
![Page 41: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/41.jpg)
DeveloperLevel
41
![Page 42: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/42.jpg)
flickr.com/photos/89228431@N06/11348332905
Minimize Our Dark Times
Code Change
Validation
![Page 43: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/43.jpg)
Fast feedback leads to fast
fixes
flickr.com/photos/bogenfreund/556656621
![Page 44: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/44.jpg)
Different Levels
Manual acceptance tests
Unit tests
Continuous Testing
![Page 45: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/45.jpg)
Different Levels
Good at the bottom
Great at the top
Ignore the middle
![Page 46: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/46.jpg)
Continuous Feedback
Continuous Integration
Continuous Delivery
Continuous Testing
Continuous Monitoring
![Page 47: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/47.jpg)
Continuous deployment is controversial.
Most people ... think I’m advocating
low-quality code
or an undisciplined
cowboy-coding
development process.
Eric Ries
radar.oreilly.com/2009/03/continuous-deployment-5-eas.html
![Page 48: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/48.jpg)
Language is a Tool
Use it
![Page 49: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/49.jpg)
ExerciseHow long is your SDLC (idea to verification)?
What are your SDLC feedback bottlenecks?
Write down 3 worst bottlenecks
Gather into groups of 5 to 7
Share and discuss
Keep your list!
Take ten minutes
![Page 50: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/50.jpg)
Continuous Integration
![Page 51: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/51.jpg)
Continuous Integration
Watch Code
Build product
Run tests
Publish results
![Page 52: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/52.jpg)
Pros
Fast feedbackKnown stateEasily consumed historyShines a light
52
![Page 53: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/53.jpg)
Cons
Slows you downNags you after every code changeExposes problemsShares issuesForces you to deal with problemsSmoke alarm
53flickr.com/photos/117168856@N06/12486837493
![Page 54: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/54.jpg)
54
Work
Time
Perception of Problem Accumulation
![Page 55: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/55.jpg)
55
Work
Time
Actual Problem Accumulation
![Page 56: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/56.jpg)
56
Work
Time
Continuous Problem Resolution
![Page 57: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/57.jpg)
Sidebar: How We Learn
Dreyfus Model of Skills Acquisition
57
![Page 58: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/58.jpg)
58
Start
Familiar
Working
Smooth
Teaching
Dreyfus ModelofSkillsAcquisition
Steps
Recipes
Intuition
Stuck
Principles
![Page 59: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/59.jpg)
Key Points
Skills don’t map
59dilbert.com/strip/1990-09-13
![Page 60: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/60.jpg)
Key Points
Beginners need steps
60
![Page 61: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/61.jpg)
Steps
1. Use source code management (SCM)2. Check in ~everything~3. Automate the build (script)4. Acquire a build environment (isolated)5. Install build tools (dev tools)
61
![Page 62: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/62.jpg)
Java Tooling
Publish results
![Page 63: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/63.jpg)
Continuous Integration ServersManyJenkinsCruise ControlBamboo PulseTFS
63
![Page 64: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/64.jpg)
Jenkins
Open sourceEstablishedSplit from HudsonJava applicationAny application serverSelf contained
64
![Page 65: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/65.jpg)
Example
Publish results
javac (Oracle & IBM)message brokeretc
ws ejb deploymessage broker toolssshexec (to scripts)
JUnitHP's UFTSeleniumCucumber
Web pageRSSEmail
![Page 66: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/66.jpg)
Common Scenarios
66
![Page 67: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/67.jpg)
Dependent Builds
My build depends on another team’s buildNo problem
We’ll trigger your build based on their results
67
![Page 68: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/68.jpg)
Dependent Builds
Services
Client/GUI
Data Layer
![Page 69: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/69.jpg)
Complicated Dependencies
Our software is too complicated for CI69
flick
r.com
/pho
tos/
chao
ticm
ind7
5/16
1697
2227
0/
![Page 70: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/70.jpg)
Dependent Builds
Data Layer
Data Services
Rich Client
Common Libs
Web Portal
Biz Services
![Page 71: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/71.jpg)
What’s Big?
3,000 developers300 major projects
All relatedPlatform to UI
Multiple branches
71
![Page 72: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/72.jpg)
Common Integration Build
72
![Page 73: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/73.jpg)
Chained Continuous Integration
Team 1 Team 2 Team 3
Integrationbuild
![Page 74: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/74.jpg)
Multiple Platforms
74
![Page 75: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/75.jpg)
Platform Testing
Team 1 Team 3Team 2 Team 4 Team 5
Integration Build
Platform1 Plat 3Plat 2 Plat 4 Plat 5
![Page 76: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/76.jpg)
Battleship!flickr.com
/photos/puuikibeach/4121540002
![Page 77: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/77.jpg)
Each player has
a board
flickr.com/photos/spookyam
d/6333317567
flickr.com/photos/jking89/1363983490
Place your fleet
Drop your bombs
![Page 78: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/78.jpg)
Two Waysto Play
flickr.com/photos/compacflt/22797421725
flickr.com/photos/bz3rk/2436165683
Use all your ammo quickly
Use ammo one shot at a time
![Page 79: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/79.jpg)
Round 1Start a timer
Place your fleet
Use all your “ammo”
Check with your opponent
Record hits/misses/time
Play twice
Round 2Start a timer
Place your fleet
Fire 1 round
Check with your opponent
Continue until out of ammo
Record hits/misses/time
Ships & Ammo
1 Sub (2 dot ship)
2 Destroyers (3 dots ships)
1 Battleship (4 dot ship)
1 Aircraft carrier (5 dot ship)
35 rounds to fire
![Page 80: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/80.jpg)
ResultsWhich way is faster?
Which spends less “budget”?
Which way is more effective?
Which way do you prefer?
![Page 81: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/81.jpg)
Feedback
Faster with feedback?
Slower with feedback?
Is slower bad?
![Page 82: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/82.jpg)
Efficient or Effective
![Page 83: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/83.jpg)
A B C D E F G H I J
1
2
3
4
5
6
7
8
9
10
Opponent’s Fleet
A B C D E F G H I J
1
2
3
4
5
6
7
8
9
10
My Fleet
A B C D E F G H I J
1
2
3
4
5
6
7
8
9
10
Opponent’s Fleet
A B C D E F G H I J
1
2
3
4
5
6
7
8
9
10
My Fleet
![Page 84: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/84.jpg)
A B C D E F G H I
1
2
3
4
5
6
7
8
9
A B C D E F G H I
1
2
3
4
5
6
7
8
9
A B C D E F G H I
1
2
3
4
5
6
7
8
9
A B C D E F G H I
1
2
3
4
5
6
7
8
9
A B C D E F G H I
1
2
3
4
5
6
7
8
9
A B C D E F G H I
1
2
3
4
5
6
7
8
9
A B C D E F G H I
1
2
3
4
5
6
7
8
9
A B C D E F G H I
1
2
3
4
5
6
7
8
9
Time Shots Misses Hits Sinks % hits % sinks1st All at once
2nd All at once
3rd Interactive
4th Interactive
Battleship! An exercise by Tony Brill1 2 3 4
![Page 85: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/85.jpg)
Continuous Testing
![Page 86: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/86.jpg)
Continuous Integration
Watch Code
Build product
Run tests
Publish results
![Page 87: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/87.jpg)
Continuous Testing
Watch CI
Deploy product
Integration tests
Publish results
![Page 88: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/88.jpg)
Continuous Testing
CI CT
Compile & unit test
Deploy &integration
test
![Page 89: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/89.jpg)
Watch
Build
Unit Tests
PublishContinuousIntegration
AgileArtisans.com
![Page 90: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/90.jpg)
Watch
Build
Unit Tests
Deploy to Dev
IntegrationTests
FirstLook
ContinuousTesting
AgileArtisans.com
![Page 91: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/91.jpg)
SCMDevelopers
Watch
Build
Unit Tests
Deploy to Dev
IntegrationTests
FirstLook
ContinuousTesting
On Demand Deployto QA
ManualTesting
AgileArtisans.com
![Page 92: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/92.jpg)
Continuous Testing
Build in CI (compile & unit tests)
Deploy to dev (staging)
Run integration tests
![Page 93: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/93.jpg)
What does this tell us?
If it works
![Page 94: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/94.jpg)
How?
By keeping fast feedback fastProviding slow feedback slowly
Don’t tie fast feedback to slow feedback
![Page 95: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/95.jpg)
Fast vs Slow
“Fast”Integratio
nTests
Compiles& Unit Tests
SlowIntegratio
nTests
5-10 minutes
30 to 60
4 to 8 hours
![Page 96: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/96.jpg)
What's this look like?
![Page 97: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/97.jpg)
Continuous Testing
CI CT
Compile & unit test
Deploy &integration
test
![Page 98: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/98.jpg)
Multi Platform Deploys
CICD Win7
CD Linux
CD OS X
![Page 99: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/99.jpg)
Exercise
Remember your SDLC bottlenecks? What could continuous integration solve?What could continuous testing solve?Regroup with your team and discussBring your top 3 solutions
99
![Page 100: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/100.jpg)
As We Explore a Continuous World...
100
![Page 101: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/101.jpg)
What Concepts Emerge?
101
![Page 102: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/102.jpg)
Nuke and PaveReset (completely)Start overCompletely Scripted
102
![Page 103: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/103.jpg)
BinariesNuke & PaveDeleteRecompileDeployUse
103
flickr.com/photos/x-ray_delta_one/4152356464
![Page 104: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/104.jpg)
DataNuke & PaveDelete schemaDrop tablesRecreateRestockRailsLiquibase
104
flickr.com/photos/x-ray_delta_one/4152356464
![Page 105: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/105.jpg)
Rails ActiveRecord
CREATE TABLE products (
id int(11) NOT NULL auto_increment,
name varchar(255),
PRIMARY KEY (id)
);
105
![Page 106: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/106.jpg)
Rails ActiveRecord
change_table :card_deck do |table| tablet.remove :description, :name tablet.string :first_name tablet.index :part_number tablet.rename :list_name, :last_nameend
106
![Page 107: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/107.jpg)
Rails ActiveRecordrake db:migraterake db:migrate VERSION=0rake db:reset
107
![Page 108: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/108.jpg)
ActiveRecordrake db:resetDrops schemaRecreates schema
108
![Page 109: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/109.jpg)
Operating System
Nuke & PaveVirtualizeRecreate
109
flickr.com/photos/x-ray_delta_one/4152356464
![Page 110: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/110.jpg)
Steps for a New OS
Install VirtualBox
Install Vagrant
vagrant init hashicorp/precise32
vagrant up
110
![Page 111: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/111.jpg)
Benefits
Scriptable
Reproducible environment
Solid results
Fast results
Repeatable results
![Page 112: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/112.jpg)
When?
Day one
Easiest
![Page 113: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/113.jpg)
“If you don’t do it on Day One,you probably won’t”
Andy Hunt
113
andy.pragprog.com
![Page 114: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/114.jpg)
Easy?
No
Difficult
Necessary
![Page 115: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/115.jpg)
Costs Move to Zero
Once scripted…
What changes?
![Page 116: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/116.jpg)
How do We Improve Our Feedback Loops?
![Page 117: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/117.jpg)
Developers
Devs write code
normally
SCM
Code is shared via a source code
management tool
ContinuousTesting
Providing immediate
feedback and a shared
environment for a QA
“First Look”
TriggeredDeployto QA
QA team members
trigger this deploy
ManualTesting
A stable environment for
manual and exploratory
testing
![Page 118: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/118.jpg)
SCMDevelopers
Watch
Build
Unit Tests
Deploy to Dev
IntegrationTests
FirstLook
ContinuousTesting
On Demand Deployto QA
ManualTesting
Rapid feedbackPair Dev & QA
![Page 119: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/119.jpg)
First Look• QA can see what’s been coded immediately
• Differences in opinions can be discussed quickly
• Fewer problems during the larger testing cycles
• Keeps QA & Dev working together as partners
• Minimizes feedback for the previous iteration
• Faster feedback for QA
![Page 120: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/120.jpg)
Continuous Testing
• Includes QA in the continuous feedback cycle
• Retains a traditional QA stable testing environment
• Dev & QA should both review all tests (unit & int.)
• QA requests trigger the QA env. deployment
![Page 121: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/121.jpg)
Alternatives
![Page 122: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/122.jpg)
flickr.com/photos/lightsinthedark/5810724330/
Work in a vacuum
![Page 123: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/123.jpg)
Builds
Deploys
Tests
flickr.com/photos/pmillera4/9317924474
Ruthlessly Automate
![Page 124: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/124.jpg)
Be Lazy!
You have builds scripted
CI servers running
Tests written
Things deploy
What else can you do?
flickr.com/photos/ofernandezberrios/2720567252
![Page 125: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/125.jpg)
Dev
Environments
QA
Staging
Prod
5 minutes
IntegrationTests
8:15am
8:20am
8:50am
8:25am
8:30am
8:35am
8:40am
8:45am
![Page 126: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/126.jpg)
Traditionally…
Someone needs the appNotices it’s down
URGENT!FIX!
Fire drill
flickr.com/photos/kaibara/5332191131
![Page 127: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/127.jpg)
Detect Problems
Before it’s urgentDetectNotify
Fix
![Page 128: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/128.jpg)
Wait5 minutes
IntegrationTests
Continuous
MonitoringNotify
Every Platform
![Page 129: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/129.jpg)
Feedback Loops
IdentifyReduceMinutesNot days
Never weeks
![Page 130: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/130.jpg)
Months
flickr.com/photos/parkstreetparrot/9764446493
![Page 131: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/131.jpg)
Stepsflickr.com
/photos/jalalspages/842122500
![Page 132: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/132.jpg)
Steps
Continuous Integration (aka compiles) flickr.com/photos/jalalspages/842122500
![Page 133: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/133.jpg)
Steps
Continuous Integration (aka compiles)
Test coverage numbers (aka unit tests)
flickr.com/photos/jalalspages/842122500
![Page 134: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/134.jpg)
Steps
Continuous Integration (aka compiles)
Test coverage numbers (aka unit tests)
Continuous deployment (aka deploys to dev/test)
flickr.com/photos/jalalspages/842122500
![Page 135: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/135.jpg)
Steps
Continuous Integration (aka compiles)
Test coverage numbers (aka unit tests)
Continuous deployment (aka deploys to dev/test)Continuous testing (aka integration tests)
flickr.com/photos/jalalspages/842122500
![Page 136: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/136.jpg)
Steps
Continuous Integration (aka compiles)
Test coverage numbers (aka unit tests)
Continuous deployment (aka deploys to dev/test)Continuous testing (aka integration tests)
Continuous monitoring (test against environs)
flickr.com/photos/jalalspages/842122500
![Page 137: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/137.jpg)
Culture Change
Fundamental change in development
![Page 138: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/138.jpg)
StepsAutomate builds
Continuous Integration
Add unit tests
Automate deployments
Continuous Delivery
Add integration testsflickr.com/photos/untitlism/
2547423465
![Page 139: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/139.jpg)
Your Move
![Page 140: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/140.jpg)
ExerciseAt lunch
Continuous TestingTwo 3x5 cards3 reasons for
3 barriers to adoptionGroups of 5 to 7
Brainstorm on solutions
flickr.com/photos/curtisperry/6119618990
![Page 141: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/141.jpg)
ExerciseUseful or not...
Will you pursue or not...
Continuous Integration
Continuous Testing
Continuous Monitoring
Test automation
How will these ideas help or hurt your org?
Write down your answers and pass them up
![Page 142: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/142.jpg)
![Page 144: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/144.jpg)
![Page 145: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/145.jpg)
An Addendum
![Page 146: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/146.jpg)
Test Automation
Unit tests
Acceptance Tests
![Page 147: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/147.jpg)
Unit Tests
One class
One method
Fast
Illuminating
![Page 148: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/148.jpg)
Integration Tests
High level
Lots of coverage
No detail
![Page 149: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/149.jpg)
Continuous Feedback
Large amounts of code
Small amounts of change
Tests run after every change
Requires frequent code checkins
![Page 150: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/150.jpg)
What's Better?
Unit or Integration?
150
![Page 151: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/151.jpg)
Best Is...
AutomatedReliable
Frequently executed
151
![Page 152: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/152.jpg)
Test LifecycleSet up stateUse product Verify results(Clean up)
152
![Page 153: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/153.jpg)
Set Your Teams Up
Builds work
Functionality works
Biggest risk?
Requirements
![Page 154: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/154.jpg)
Developers
Testers
Customers
The Team
![Page 155: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/155.jpg)
Grid Driven Development
Remove technical barriers
Everyone speaks “Excel”
![Page 156: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/156.jpg)
Testing Strategies
Test First is best
Defect Driven Testing
Blitzkrieg Testing
![Page 157: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/157.jpg)
Tips
Open standards...
Junit or Junit compatible
Avoid vendor lock in
Glue tools
Time matters....
Fast servers
Incremental change
![Page 158: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/158.jpg)
Script Tips
Save scripts in source code management
Modular
Basic encapsulation
One target => One function
![Page 159: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/159.jpg)
Script Targetsall depends_on clean, compile, test-unit
clean # deletes build artifacts
compile # builds the product (Ruby?)
test-unit # run your tests
deploy_locally # to your dev environment
deploy_dev (?) # to a shared dev environment
test-integration # run your integration tests
![Page 160: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/160.jpg)
Resources
JUnit JUnit.org
Jenkins Jenkins-ci.org
BDD Dannorth.net/introducing-bdd
Cucumber Cukes.info
Continuous Deployment en.wikipedia.org/wiki/Continuous_delivery
![Page 161: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/161.jpg)
Devops Resources
VagrantUp.com
GetChef.com/chef
PuppetLabs.com
Ansible.com
VirtualBox.org
![Page 162: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/162.jpg)
Data Resources
Guides.RubyOnRails.org/active_record_basics.html
Liquibase.org
FlywayDB.org
![Page 163: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/163.jpg)
Additional Content
Material for you to review later…
![Page 164: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/164.jpg)
Release Planning
Releases contain featuresHow many are ready to ship?
164
![Page 165: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/165.jpg)
Releases contain features
165
Team ReleaseDate
ReleaseTheme
# Feature
s#
StoriesStories
Complete%
Done
1
2
3
![Page 166: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/166.jpg)
Releases contain features
Team Release Features Stories StoriesComplete % Done
A 7/1 25 392 17 0.23
B 7/15 13 208 104 0.5
C 7/12 96 2,400 150 0.16
D 12/30 476 ?? 234 ?
![Page 167: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/167.jpg)
Release dateBurn Up Chart
167Time
Features
Features in the release
Requiredrate of work
![Page 168: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/168.jpg)
Will we hit our date?
168Time
Features
ACTUALrate of work
![Page 169: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/169.jpg)
Burn Up Chart
169Time
Features
ACTUALrate of work
![Page 170: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/170.jpg)
Feature Creep
170Time
FeaturesACTUAL
rate of work
![Page 171: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/171.jpg)
Observe Progress
On track
Behind
Unrealistic
171
![Page 172: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/172.jpg)
The Solution
Reduce features
DO NOT focus on increased work
172
![Page 173: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/173.jpg)
Good or Not?
173Time
Features
ACTUALrate of work
![Page 174: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/174.jpg)
What’s the Fix?
174Time
Features
How do I make the teamwork harder?
![Page 175: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/175.jpg)
175
LongerHours
Work Faster!EvenLater
More Mistakes
Lots of Debugger
Time
![Page 176: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/176.jpg)
What’s the Fix?
176Time
Features
![Page 177: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/177.jpg)
What’s the Fix?
177Time
Features
![Page 178: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/178.jpg)
What’s the Fix?
178Time
Features
![Page 179: Continuous Testing in Vegas](https://reader035.fdocuments.us/reader035/viewer/2022070516/587097851a28ab412b8b6c27/html5/thumbnails/179.jpg)
What Happens to Overloaded Planes?
179flickr.com/photos/jameski/5322886899