Agile Engineering for Managers Workshop

Agile Engineering PracticesConcepts and Usage

Agile Engineering Objectives

We want to answer the following Qs:

• What practices can Agile teams use (to handle increasing complexity)?

• Conceptually, how do they work?

• What benefits can a team derive from them?

• What are some common challenges in implementing them?

• What tools can be used to support them?

Agile Engineering Agenda

• What do we mean by Agile Engineering?• What does this mean in terms of change?• What are the relationships of engineering practices to..?

– The Manifesto– Iterative and Flow-based Processes– User Stories

• Agile Engineering Practices– Source Code Control– Pair Programming– Continuous Integration– Testing; Types of Testing, TDD/BDD, etc.– Refactoring– Continuous Delivery

• Debrief– Reminder on Change

Whatdo we mean

Agileby

Engineering?

Meet Mr. Spock

Mr. Spock develops software.

He wants to develop software his users will love.

It’s creative problem solving. (How logical.)

He wants to deploy it often so his users can love him more.

He wants tools and techniques to help him accomplish these things.

Meet Mr. Scott

Mr. Scott runs systems.

He wants the systems to run reliably.

Problem-solving almost always means bad news for him.

Users usually do not love it when he has to do problem-solving.

Fast deployments hopefully don’t mean more work and outages.

When things don’t go well…

CAPT Kirk, the CIO, shows more emotion than Mr. Spock would like.

You’d thing the Klingons had taken his son or something…

Agile engineering practices help us…

Produce quality codeProtect us from ourselves

Deploy often and reliably

Agile Alliances Practices Map

An Abbreviated HistoryLesson

XP InventedMost engineering

practices come from XP

1968: Edsger DijkstraForerunner of XP/TDD

is Constructive Approach to Dev

1996: Kent Beck

Project MercuryOrigins of Test-First

William RoyceWaterfall

1970:BDD Invented

Automated Acceptance Tests

2006: Dan North

Mikado MethodSafely Refactor

Legacy Code

Ola Ellnestam2010:

Daniel Brolund

Manifesto

2001

Whatdoes this mean

in terms ofchange?

Adopting

Introduces change for most teams

Understanding how change affects teams is important

Agile Engineering Practices

Status Quo

New Status Quo

disruption amount

disruption time

The Satir Change Curve

A detailed depiction of the Satir Change model -http://stevenmsmith.com/ar-satir-change-model/

Status Quo

New Status Quo

disruption amount

disruption time

The Satir Change Curve


Allow teams time to learn

Give teams encouragement

Expect a degradation

How does

relate

Agile

to

Engineering

theAgile Manifesto?

Manifesto for Agile Software

Development

We are uncovering better ways of developing software

by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes

and tools Working software over comprehensive

documentation Customer collaboration over

contract negotiation Responding to change over

following a plan

That is, while there is value in the items on the right, we

We follow these principles:

Our highest priority is to satisfy the customer

through early and continuous delivery

of valuable software.

Welcome changing requirements, even late in

development. Agile processes harness change for

the customer's competitive advantage.

Deliver working software frequently, from a

couple of weeks to a couple of months, with a

preference to the shorter timescale.

Business people and developers must work

together daily throughout the project.

Build projects around motivated individuals.

Give them the environment and support they need,

and trust them to get the job done.

The most efficient and effective method of

conveying information to and within a development

team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development.

The sponsors, developers, and users should be able

to maintain a constant pace indefinitely.

Continuous attention to technical excellence

and good design enhances agility.

Simplicity--the art of maximizing the amount

of work not done--is essential.

The best architectures, requirements, and designs

emerge from self-organizing teams.

At regular intervals, the team reflects on how

to become more effective, then tunes and adjusts

its behavior accordingly.

How does

relate

Agile

to

Engineering

theagile processes?

Iterative Process

Iteration Iteration

Deploy Deploy

Acceptance

& Unit

Testing

Continuous

Integration

Continuous

Integration

Acceptance

& Unit

Testing

Flow-Based (Kanban)Backlog DevAccept Tests Deploy Done

Deploy

Acceptance

Tests

Written

Continuous

Integration

Unit

Testing

Acceptance

Testing

How does

relate

Agile

to

Engineering

user stories?

User Stories

Planning [WBS]

TestingRequirements

Story as backbone

of Agile Release

through prioritized ordering of stories into backlog

through acceptance criteria of story

through combination of story and Definition of Done

User Stories

Planning [WBS]

TestingRequirements

Story as backbone

of Agile Release

through prioritized ordering of stories into backlog

through acceptance criteria of story

through combination of story and Definition of Done

CODE

TDD

What are theAgile

EngineeringPractices?

Timefor a

break…

As played by the

Dave Brubeck Quartet… Take Five

#1Source Code Control

Source Code Control

• Fundamental to almost every other technical practice

• Does Ø good w/out frequent code check-ins

• Provides the safety net for making changes

What is wrong with using a source code repository like this?

Shared File Space

ReleaseCandidate

DeclareRelease

Release

Who is this?

What considerations do you need to think about if you do it this way?

Release

DeclareRelease

Who is this?ReleaseCandidate

What considerations should we consider if developers horde code and

don’t commit it to the repository?

Release

DeclareRelease

ReleaseCandidate

X

Popular Source Code Repository Tools

Open Source Commercial

Dimensions

#2 Pair Programming

What is Pair Programming?

2 Developers, 1 Keyboard

In a classic

it’sXP sense,

Pair Programming Upside

• Better Code– Peer review baked in

– Different ways to approach need

• Faster Output– Design decisions/implications thought through

– Once people feel comfortable, throughput tends to go up

• Shared Understanding of Work– More people know code/decisions

– Higher Bus Lottery Factor

Pair Programming Flipside

• Personality Fits Are Important– Not everyone feels comfortable exposing their work– Need to avoid dominant controller– Good listening skills

• Breather Time– Need space/time to selves, particularly for introverts– Consider allowing OK to check out– Space for people to go to…

• Expect Some ‘Adjustment’– Human dynamics of learning to share space/thoughts

Possible Pairs

Dev Dev

Dev Tester

BA Dev

Dev PO/SME

Pair Poetry

We’re going to simulate two people working on the same code in the form of poetry.

There once was a man from Nantucket….

Pair PoetryRound 1:

• Pair Up; you’ll each need a pen and a piece of paper, and two story dice.

• Each of you is going to write a Limerick together; – Limericks follow an end of a sentence rhyming pattern of A-A-B-B-A– 1st, 2nd, and 5th lines are 8-9 syllables – 3rd and 4th lines are 5 syllables– They are almost always humorous, if not a little bawdy

• We’ll first write a Limerick that isn’t using pairing– Lines 1 and 3 and the first 3 words on the last line are written by the first person– Lines 2 and 4 and the last set of words on the last line by the second

• Here’s how to play:– Roll the story dice, decide who will take the one picture and the other the 2nd

picture. Decide who goes first….– Put your name on your sheet, whether you are 1st or 2nd and write your lines…– No discussion

• Once you complete your part, put your hands up; I’ll collect your sheets once I see both hands.

An Example Limerick…

There once was a young lady named Bright

Whose speed was much faster than light

She set out one day

In a relative way

And returned on the previous night.

Round 1 Debrief

• How well did the Limericks flow?– Any roughness?– Subject get off the rails?– How about that last sentence?– Did anyone mess up the rhyming?

• What did the dice represent? (Think from a coding standpoint.)

• How about each line of the poem?

• What shared mental model did you have?

Pair PoetryRound 2:

• Same pairs; you’ll each need a pen and will share a piece of paper, and two story dice.

• Again we’ll write a Limerick together; – Limericks follow an end of a sentence rhyming pattern of A-A-B-B-A– 1st, 2nd, and 5th lines are 8-9 syllables – 3rd and 4th lines are 5 syllables– They are almost always humorous, if not a little bawdy

• We’ll now write a Limerick that ‘pairs’ sequentially– Lines 1 and 3 and the first 3 words on the last line are written by the first person– Lines 2 and 4 and the last set of words by the second

• Here’s how to play:– Roll the story dice, SWAP who will take the one picture and the other the 2nd

picture. Decide who goes first….– Put your name on your sheet, whether you are 1st or 2nd and write your lines…– Take turns writing your lines passing the paper between you.

• Once both people have completed their parts, put your hands up; I’ll collect your sheets.

Round 2 Debrief

• Any improvement in your Limerick’s flow?– Any roughness?– Subject get off the rails?– How about that last sentence?– Did anyone mess up the rhyming?

• How did it feel not being able to change a prior line?


• Is working sequentially on code like this a realistic approach?

Pair PoetryRound 3:

• Same pairs; you’ll each need only ONE pen and ONE piece of paper, and two story dice.

• Again we’ll write a Limerick together; – Limericks follow an end of a sentence rhyming pattern of A-A-B-B-A– 1st, 2nd, and 5th lines are 8-9 syllables – 3rd and 4th lines are 5 syllables– They are almost always humorous, if not a little bawdy

• We’ll now write a Limerick that pairs fully– Either of you can write, the other can comment, critique, provide input, etc. – You can also REQUEST THE PEN!– Change any line/word you want as long as both people agree

• Here’s how to play:– Roll the story dice, use these two pictures when writing your Limerick

– Put both your names on your sheet and write your lines together…

• Once it is complete, put your hands up; I’ll collect your sheets.

Round 3 Debrief

• Any improvement in your Limerick’s flow?– Any roughness?

– Subject get off the rails?


• Did you finish more quickly?

• How did you feel about the quality of your limerick?

Beyond Pairs… Try Triads!

Dev Tester Dev

Dev BA Dev

Dev PO/SME Tester

Dev Tester BA

Some Tools for Remote Pairing

• RealVNC

• TeamViewer

• Windows Remote Desktop

• AeroAdmin

• Chrome Remote Desktop

• Remote Utilities

• join.me

• BeamYourScreen

Timefor a

break…

#3 ContinuousIntegration

CI Helps Provide

the Fast Feedback Loops to

Discovering Mismatches…

…in Environments

…in Understanding

Resulting in Better Reliability

How CI Works…

00101101

Valve opens on ✓-in

Success = Deploy Binary

When Implemented

ExecuteBuild Scripts

ExecuteTest

Scripts

Continuous Integrationwith

CI w/Lego, Part 1

• Split into two pairs; each pair gets a kit of Legos (either the left side or the right side), a set of build specifications for their side, and an 8x8 Lego plate

• Read the instructions and then we’ll start all at the same time

• When both the left and right side have completed their structures, we are going to integrate them

CI w/Lego, Part 2

• Stay in your two pairs; each gets a new kit of Legos (either the left side or the right side), a set of build specifications for their side, and keeps their set of plates

• Read the instructions and then we’ll start all at once; you will be working in 3 iterations

• At the end of each iteration, we are going to integrate the structures

Debrief

#4 Continuous Testing

Why Continuous?

What do we mean?

What is the benefit?

Agile Jenga :: Let’s Get Started

• Separate into pairs• Each pair takes a pair of dice (one red, one

white), and a set of Jenga blocks; these are numbered. (Some pair may get the more challenging Jenga Extreme blocks…)

• Decide who will be the developer and who will be the tester

• You will need a small sheet of paper and a pen

Agile Jenga :: Building Requirements

The Developer will build a structure; it must meet the following requirements:

• Use ALL Blocks

• It must be at least 3 “stories” tall (a story consists of at least one horizontal piece and one vertical piece)

We will build this structure in 3 rounds, with each person being the tester at least once

Agile Jenga :: Round 1

Developer• Build the structure

Tester• At the End, Run Your Tests: Roll the Dice 4 times

(match the red and black numbers rolled to the blocks)

• Blocks that match the rolls must be removed by the Developer and the tower must be rebuilt if necessary


Developer• Build 9 blocks of the structure

Tester• At the end of each 9 Block iteration, Run Your Tests: Roll the

Dice 4 times minus any previously found bugs times, but always at least once (match the red and black numbers, reroll any already pulled)

• Blocks that match the rolls must be removed by the developer and the tower must be rebuilt if necessary

Move onto the next iteration of 9 blocks to be built (4 iterations)


Tester

• At the Start, Run Your Tests: Roll the Dice 4 times and record the numbers (reroll if any have been previously removed); keep these “private”

Developer

• Select and build 9 blocks of the structure

Tester

• As the structure is built, remove any matching numbered blocks as you see them

Move onto the next iteration of 9 blocks to be built (4 iterations)

Agile Jenga- Debrief -

Benefits to Bringing Testing Forward

Release

Iteration

Day

• Finds defects quicker (like before you are ready to deploy a release!)

• Defects are a form of feedback! (too late & the feedback is painful)

Deploy

Test to Specification

Test to Failure

Product Design

Technical Design

Interaction Design

Acceptance(Functional)

Tests

xUnitTests

UITests

ExploratoryTesting

Stress Testing

Security Testing

Usability Testing

(A/B Testing)

Finite, Repeatable,Automated

Environmental, Less Automated

Diagram concept from p.76, Leading Lean Development, Mary & Tom Poppendieck, 2010

A Full Complement of Tests

5 MinutesYou have

starting now…

Take a break!

Traditional Testing Approach

Requirements &

Design Specs

Develop Code,Compile Executable

Develop Test Scripts

ExecuteTest Scripts

Error Found!

Test Expectation≠

Executable Implementation

WTFWe’re supposed to be releasing next week and you tell me this isn’t working? Get that bug fixed!

The objective is

to kill bugs quickly,

but to

AVOID them altogether!

What if…Requirements Specs = Test Specs = Acceptance Criteria

Design Specs = Test Specs = Quality Criteria

We built the right thing.

We built the thing right.

Gre

ater

Det

ail

“A Constructive ApproachTo the Problem of Program Correctness”, 1968, Edsger Wybe Dijkstra

1968How do we know our programs are correct?

1) Construct a proof showing what the program should do

2) Develop code that meets the proof

REPEAT until doneREPEAT until done

In 1968,

in fact not until the late 90s,

the tools

didn’t exist to implement Dr. Dijkstra’s vision.

Today,the Constructive Approach = Test Driven Development (TDD)

(Specs = Tests = Proofs)

Specs Tests CodeDevelopment

TestExecution

Gre

ater

Det

ail Acceptance Tests Stories/Functions

xUnit Tests Specific Detailed Methods/Functions

Mark as ready for test when completed; based on Product Owner priority

Test completed code

Write Test(s)

Execute Test to

Show Failure

Create Mock Data To

Show Passing Test(s)

Write Code to

Pass Tests

Why?Know test will show fail

Why? Tests = Specs

X Unit Tests

Recurse over this pattern!

Why?Know test will show pass (expected results)

Say you've got a method,

getName(String path)

that parses a file name from a file path.

You can write a JUnit test class to feed it a path and ensure that the results are what you expected.

What does an xUnitTest look like?

Simple example from jGuru.com

import java.io.File; import junit.framework.*; import junit.extensions.*;

public class ThingTester extends TestCase{

public ThingTester (String name) {

super (name); }

public static void main(String[] args) {

junit.textui.TestRunner.run(ThingTester.class); }

public void testGetName() throws Exception {

File myFile = new File("c:\\xxx\\yyy\\zzz.txt"); assertEquals("zzz.txt", myFile.getName());

} }

To exercise the test:

Compile & run ThingTester.

• Little Dot = Passing Test; • Assertion Failure & Stack Trace = Failure & location • Write test code BEFORE program code used to make it pass.

No problems? Add more test cases. Examples: long paths, files with spaces in the name, etc.Stop when you feel you have enough edge cases covered.

Tests should cover all critical methodsTests should cover all critical edge casesTests should cover all critical error detection

Goal ≠ 100% coverage, but Goal = 100% passing for what is covered

Understand Your Test Coverage

-in your working code!(Including your tests!)

Write Test

Execute Test to

Show Failure

Create FeatureTo

Pass Test

Why?Know test will show fail

Why? Tests = Specs

Acceptance Tests

DescribeDesired

Behavior

DescribeFeature

Test Steps

Recurse over this pattern!

Underneath… …are Unit Tests

What does an Acceptance Test1

look like?

Say you've got a feature you want to implement to compute a factorial…

You can write a feature that is the proof of this function or story (including edge cases)

You also implement “steps” that poke at the implementation. As the implementation changes the steps may change, but it is less common for the feature to change.

1aka Executable Specification

Example from tutorial at lettuce.it

Feature: Compute factorialIn order to play with LettuceAs beginnersWe'll implement factorial

Scenario: Factorial of 0Given I have the number 0When I compute its factorialThen I see the number 1

Business Users can get this!

computefactorial.feature

from lettuce import *

@step('I have the number (\d+)') def have_the_number(step, number):

world.number = int(number)

@step('I compute its factorial') def compute_its_factorial(step):

world.number = factorial(world.number)

@step('I see the number (\d+)') def check_number(step, expected):

expected = int(expected) assert world.number == expected, \

"Got %d" % world.number

computefactorialsteps.py

To exercise the test: Run Feature (duh!)

Add more code to fix problems and re-run….

computefactorial.py

def factorial(number):return 1

by definition, we know that the factorial of 0 is 1So as a mock, we created a function that just returned 1

To exercise the test: Re-run Feature (duh!)

Need more than one case of course, so let’s add more test cases; i.e. feature scenarios

Feature: Compute factorial In order to play with Lettuce As beginners We'll implement factorial

Scenario: Factorial of 0 Given I have the number 0 When I compute its factorial Then I see the number 1



} In this case my steps & asserts were OK, sometimes you have to create more

To exercise the test: Rerun Feature (duh!)

Change code to fix problems and re-run….

computefactorial.py

def factorial(number): number = int(number) if (number == 0) or (number == 1):

return 1 else:

return number


Add a few more cases so we get realistic…

Feature: Compute factorial In order to play with Lettuce As beginners We'll implement factorial

Scenario: Factorial of 0 …





}Again my steps & asserts were Ok this time, sometimes you have to create more

Change code to fix problems…

computefactorial.py

def factorial(number):number = int(number)if (number == 0) or (number == 1):

return 1else:

return number*factorial(number-1)

Acceptance Tests work best when:

• Driven off of methods (functions) or several functions• Perform below the UI level

If needed, Acceptance Tests can:

• Be driven off of the UI (usually performs a little slower; must be done this way for something like PowerBuilder or other 4GL type of client/server environment)• Can be done manually

The specification is independent of the flow or implementation. The steps that poke at the underlying code may need modification over time.

-in your working code!(Including your tests!)

How often?

For unit tests…

Every time code (which includes a test) is changed… This is probably multiple times a day and should be at a minimum daily.

For acceptance tests…

Whenever a test is completed and then the code is completed. The test may be completed on Tuesday, then code that pokes at the test will be completed as the skeletal object is completed say around Wednesday, and then perhaps Friday is when all the code for that poke is completed.

Some Testing Frameworks

Unit Testing

• pyUnit

• jUnit

• nUnit

• plUnit

• cfUnit

• FlexUnit

• Jasmine (JavaScript)

• Ruby has built in framework environment

ATDD/BDD

• Cucumber

• Lettuce

• Fitnesse

• Jbehave

• rbSpec

UI Testing

• Selenium

Take 5 Minutes

You need a break!

Test-Driven Tinkering

As an avid beach reader, I want to elevate my book a bit to protect it from the incoming wave edges so that my book remains dry when I set it down to sip on my umbrella drink.• Acceptance criteria are shown by the dimensions

indicated on the sheets of paper being passed out.– Given a book of X and Y,

When I place the book on it, Then it does not fall off.

– Given the depth of the water will be about one inch where I sit,When I place the book on it and incoming water splashes up based on the lowest dimension shown,Then my book remains dry.

Test-Driven Tinkering, Part 1

• Get a set of Tinker Toys• Objective is to build a platform can hold a book

the dimensions on the sheet of paper a height between the dimensions off the ground. (There is some tolerance as to not meeting these exactly as Tinker Toys are not the most exacting…)

• Can you build the tests first using Tinker Toy pieces and then use it to guide your design?

• Are these the simplest test you can build?• Now use these tests to build your platform and

place the book on it (performance test).

Test-Driven Tinkering, Part 2

Given the waves come in closerWhen I need to move back towards the dunesThen I need a handle when I stand up to move the book elevator back with me

• Now I want the structure’s height to be the height based on my new test (the handle I can use when I stand up).

• Can you build a structure to pass this test while staying true to the other tests?

Test-Driven Tinkering

Debrief

Test-Driven Tinkering Debrief

• What do you think the Tinker Toys represent?

• What tests were you able to construct before beginning your construction? How did they guide your design?

• When you added onto your structure, did you break any of your prior tests?

Congrats!

AND… This

to …brings us

Refactoring is…

• Modifying code to handle additional scenarios• Adding features (or in the smaller case, methods)• Changing code so that it handles additional

performance characteristics

- AND -• No prior “functions” are invalidated; all prior tests

SHOULD pass

This test fail, make test pass, improve design is known as Red-Green-Blue (Refactor) and is what allows emergent design as opposed to over-designing upfront.

Refactoring also is…

• Modifying code to remove escaped defects

- WHILE –• Ensuring no prior “functions” are invalidated;

all prior tests SHOULD pass

Think of TDD & Refactoring asTransforming Your Code into

Something More Useful

Sometimes one

toone needs

refactor

Legacy Code (oh the horror!)

#7 MikadoMethod

Mikado Method Mikado Game

Making a change to your legacy code is not unlike picking out the Mikado from the other sticks in the pile.

Removing one impacts the others.And not in a good way…

What is the biggest difference?

We canRevert

ourcode!

The Guts of the Mikado Method

• Make the change you need/want to make

• See what broke – these are pre-requisites (the next set of changes to make before doing this one)

• Visualize (add to graph)

• REVERT !!

Repeat until no breakages occur.

Enable DatabaseConcurrency

Configure High

Availability

ReimplementDB Interface

Change Connection

Pooling

Remove Redundant

Code

Change Properties

File

Isolate JDBCConnection

CreateRESTfulInterface

D i

s c

o v

e r

y R e

f a c t o r

✓ ✓

✓

✓

✓

✓ ✓

✓

Adding Tests to Legacy Code

• Add unit (perhaps acceptance) tests as defects are removed or changes are made; basically anytime you make changes to the code.

• Tests should capture what results should be displayed (Think assertions! -- Just like before)

• I riddle you this: Why should we not go and simply add tests to everything?

#9 Continuous Delivery

Building to Every Environment


ExecuteTest

Scripts


ExecuteTest

Scripts

ExecuteDeployment

Scripts

Build to Binaries; then Promote

Promote

ExecuteTest

Scripts


ExecuteTest

Scripts

ExecuteDeployment

Scripts

Build to Binaries; Deploy to all Environments

ExecuteTest

Scripts

Some Popular CI-centric Tools

Build Scripts

• Ant

• Maven

• Gradle

• nAnt

• make

Deployment/Config Tools

• Chef

• Puppet

• Buildbot

Scripting for Promotion

• bash

• c-’Shell’

• TCL

• perl

Execute via the CI Server!

Delivery meaninglessis

feedback loop

withoutsome form

Continuous

of

Types of (Post-Deployment) Feedback

User Experience• Response times• Dwell times• Heatmap of “clicks”• Flow through the site

Security• Attempted penetrations• Attempted DoS attacks

Performance• Response times• Memory usage/Page faults• Storage growth

Business Metrics• Improved cycle-

time/throughput on business process

• Data quality (errors)• Additional fees/revenue• Reduced processing costs

Debrief

Following Good Engineering Practices…

Makes for a more productive team

– Faster feedback

Faster feedback = Technical excellence

– Catch defects before release

Ø Defects = Happy CIO

Status Quo

New Status Quo

disruption amount

disruption time

So Support the Change!


Allow teams time to learn

Give teams encouragement

Expect a degradation

Use thisknowledge

…and Launch!

Agile Engineering for Managers Workshop

Technology

Transcript of Agile Engineering for Managers Workshop