Stop Getting Crushed By Business Pressure

Business Pressure

Stop Getting CRUSHED by

Janelle Kleinopenmastery.org @janellekz

http://@openmastery

, Developer, Consultant, CTO @Specialized in Statistical Process Control (SPC) and Supply Chain Optimization from Lean Manufacturing (data geek)

Continuous Delivery infrastructure, automation strategy & technical mentorship

Janelle Klein

Who Am I?

How to Measure the PAIN in Software Development

Janelle Klein

Author of “Idea Flow”

leanpub.com/ideaflow

Founder of

newiron.com

http://newiron.com

This is a HARD Problem.

What is this talk about?

“Better”“Better”

What if we could get managers and developers all pulling the same direction?

ManagersDevelopers

Quick Recap

Great Team Disciplined with Best Practices Constantly Working on Improvements+

Project FAILURE

In the Last Talk…

The Retrospective

“What are we supposed to do?!”

Our biggest problem

“Well, if we don’t understand a problem, we should

collect data.”

The Retrospective

Our biggest problem

“What data would help us understand the problem?”

“What are we supposed to do?!”

Technical Debt Mistakes

I thought the main obstacle was Technical Debt

?Most of our mistakes were in the

most well-written parts of the code.

Mistakes

We made significantly more mistakes in code that we didn’t write ourselves.

Lower Familiarity

More Mistakes=

There had to be more to the story...

Unexpected Behavior

Problem Resolved

Tracking Painful Interaction with the Code (Friction)

Troubleshooting

Progress

5 hours and 18 minutes of troubleshooting...

PAINFUL

What Causes Unexpected Behavior (likeliness)?

What Makes Troubleshooting Time-Consuming (impact)?

Semantic Mistakes

Stale Memory Mistakes

Association Mistakes

Bad Input Assumption

Tedious Change Mistakes

Copy-Edit Mistakes

Transposition Mistakes

Failed Refactor Mistakes

False Alarm

Non-Deterministic Behavior

Ambiguous Clues

Lots of Code Changes

Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

Most of the pain was caused by human factors.

What causes PAIN?




Ambiguous Clues


Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

What causes PAIN?


Semantic Mistakes





Copy-Edit Mistakes



False Alarm




Ambiguous Clues


Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

What causes PAIN?

Semantic Mistakes





Copy-Edit Mistakes



False Alarm





Ambiguous Clues


Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

What causes PAIN?

PAIN is a consequence of how we interact with the code.

Semantic Mistakes





Copy-Edit Mistakes



False Alarm

PAIN occurs during the process of understanding and extending the software

Complex(So*ware(

PAIN

Not the Code.

Optimize “Idea Flow”

My team spent tons of time working on improvements that didn’t make much difference.

We had tons of automation, but the automation didn’t catch our bugs.

My team spent tons of time working on improvements that didn’t make much difference.

We had well-modularized code,

but it was still extremely time-consuming to troubleshoot defects.

The hard part isn’t solving the problems it’s identifying the right problems to solve.

“What are the specific problems that are causing the team’s pain?”

Then I got into consulting…

The Software Rewrite Cycle

Start%Over%

Unmaintainable%So0ware%

We Start with the Best of Intentions

High Quality Code

Low Technical Debt

Easy to Maintain

Good Code Coverage

Then This Happens!

Stages of Escalating Project Risk

Product Owner: “We’ve got more important things to do.”

Deferring(Problems(

Deferring(Problems(

Painful(Releases(

Manager: “Good job everyone! Keep up that great work ethic!”


Deferring(Problems(

Painful(Releases(

Thrashing)

Manager: “We need to go faster! Let’s hire more developers.”


Deferring(Problems(

Painful(Releases(

Thrashing) Project(Meltdown(

Developer: “I give up. I don’t care anymore if the project fails.”


Our “Solution”

“We should just quit our jobs.” “Yeah, it’s hopeless.”

What are we supposed to do?

Emergent Practice

Everything I’m showing today:

My Goal: Give You Hope

RESET

“A description of the goal is not a strategy.”

-- Richard P. Rumelt

What’s wrong with our current strategy?

Our “Strategy” for Success

High Quality Code

Low Technical Debt

Easy to Maintain

Good Code Coverage

RESET“A good strategy is a specific and coherent response to—and approach for overcoming—the obstacles to progress.”


The problem is we don’t have a strategy...

What are the obstacles?

Obstacle 1: Management doesn’t care about interest payments.

Obstacle 2: Management would rather you shut up and do your job.

Obstacle 3: The Problem is outside anyone’s control.


Obstacle 1: Your manager doesn’t care about interest payments.



“Let’s rewrite the software!”

My new project: “Awesome in Disguise”

I had full control.

Continuous Delivery from day 1.

Then they announced “the plan”

My project was moved under different management…

Crazy Deadlines

Constant Urgency

Compromise Safety for Speed

Time%Pressure%

Compromise%Safety%for%

Speed%

Increase%Number%&%Severity%of%Hazards%

%

More%Pain%and%Higher%Task%Effort%

Constant'Urgency'

Cycle of Chaos High-Risk Decision Habits

I Tried to Explain “Technical Debt”

“The project is already behind schedule!!”

Manager said:

“How can you possibly justify working on anything other than the deliverables?!”

So we did what we were told.

We drank a lot.

Until we couldn’t take it anymore.

Explained the problem of Technical Debt

Business Coaching

“That doesn’t sound so bad.”

The Response:

??

??

WHAT?!

Loans are a Predictable Financial Tool

Revenue

- Cost

Profit + 10%

Increase Price?

Increase Sales?

Reduce Cost?

What makes investment decisions harder isn’t higher costs, it’s lower predictability.

Investment Strategy


But… Managers care A LOT about RISK.

The gradual loss of predictability is much scarier than the gradual increase in cost.


Obstacle 1: Your Manager doesn’t care about interest payments.

Obstacle 2: Your manager would rather you shut up and do your job.

Obstacle 3: The System is setup to fail.

Another new project…

“Don’t ask for permission, ask for forgiveness.”

“Don’t ask for permission, ask for forgiveness.”

Another new project…

Then We Got New Management!

I put together “a plan”…

“What is Janelle trying to pull?! Who does she think she is?!”

Management said (behind my back):

Get Back Inside Your Box! (or else)

Severe Violation of SOCIAL PROTOCOL

SOCIAL PROTOCOL

Never talk to your manager’s boss about a problem.

Never suggest or imply your manager can’t do their job effectively by

trying to get others to override their decisions.

Decision-making responsibilities are assigned by management and not to be questioned.

Engineers: “We’re going to CRASH!”

Manager: “What do I do? We can’t miss these deadlines.”

Then I got into Consulting…

Developers

Manager

Consultant

“We’re going to hit the wall!”

Keynote

“We better invest money in this!”

The Consulting World

The Job of a Consultant

Why do they need my help?!

Keynote

RESET

Consultants Bridge the Divide

Message comes through a “certified authority.”

Message comes in management-speak.

Obstacle 2: Your manager would rather you shut up and do your job.

Follow SOCIAL PROTOCOL

Stay (Mostly) Inside Developer Box

Communicate in Manager-Speak +


Obstacle 2: Your manager would rather you follow social protocol.

Lesson 3: The system is setup to fail.




Obstacle 2: Your manager would rather you follow social protocol.

Obstacle 3: The system is setup to fail.

The Challenge: Decision-Making is Distributed

Another Problem:The Inability to Change Direction

What if our organization was a robot?

Fire x1

Dev Team

Management

Nothing’s happening…

Pain Sensor


Fire x1

Dev Team

Management

Fire x10

Pain Sensor

Management

Dev Team

Fire x10


If the feedback loop is broken, we burn.

Pain Sensor

Learn & Adapt

Role

Decision(Type(

Required(Knowledge(

Visibility and Decisions Coupled

Visibility and Decisions De-coupled

Role A

Required(Knowledge(

Role B

Decision(Type(

The Broken Feedback Loop is Baked into the Design

Manager

Alloca&on(Decisions(

Knowledge(of(Risks(

Risk(Mgmt(Decisions(

Developer

Communication Breakdown

Broken Feedback Loop (Manager Role)

Developer Product Owner

Actual'Risks'

interacts''with'

Actual'Customers'

interacts''with'

Knowledge'of'Customers'

Trade9off'Decisions'

depend'on'depend'on'Knowledge'of'Risks'

Communication Breakdown

Broken Feedback Loop (Product Owner Role)

Now we can steer!!

So#ware(Task(

Mi.gate(the(Risks(

Product(Development(

Product(Work(Queue(

Risk(Mgmt(Work(Queue(

Product(Owner(

Product(Decisions(

Knowledge(of(Customers(

Technical(Risk(Manager(


Knowledge(of(Risks(

Dev$Team$Capacity$

Manager

Alloca.on(Decisions(

Refactor the Organizational Architecture

This is the design that typically emerges when we have:

Trust.

Role

Decision(Type(

Required(Knowledge(

Visibility and Decisions Coupled

Visibility and Decisions De-coupled

Role A

Required(Knowledge(

Role B

Decision(Type(

Obstacle 3: The system is set up to fail.

We’ve got to fix the machine(even though it’s not our responsibility)


Obstacle 1: Management doesn’t care about interest payments.

Obstacle 2: Management would rather you follow social protocol.

Obstacle 3: The system is setup to fail.

RESET“A good strategy is a specific and coherent response to—and approach for overcoming—the obstacles to progress.”


What’s the Strategy?

2. Measure the Pain

4. Become a Risk Translator

5. Refactor the Organization

3. Identify the Biggest Problems

1. Make the Decision to Lead

Leadership is not a title, it’s a choice.


2. Measure the Pain





Your manager doesn’t care that your job “feels difficult”

“In God we trust, all others bring data.” —Edwards Deming

PAIN occurs during the process of understanding and extending the software

Complex(So*ware(

PAIN

Not the Code.

Optimize “Idea Flow”

Idea Flow Mapping Tools

(Open Source, Supported GA ~June 2016)github.com/ideaflow/tools

http://ideaflow.org/tools

“Idea Flow Map”

“Friction”

“What caused the pain in this case?”

Categorize the Problems with #HashTags

#ReportingEngine

#Hibernate

#MergeHell

1. Problem A

2. Problem B

3. Problem C

Add up the Pain by Category

What’s the biggest problem to solve?

Tools Demo

Troubleshooting

Progress

Learning

7:070:00

0:00 19:52

12 year old project after all original developers left.

Case Study: Huge Mess with Great Team

70-90% of dev capacity on “friction”

The Team’s Improvement Focus: Increasing unit test coverage by 5%

Case Study: Huge Mess with Great Team

1. Test Data Generation

2. Merging Problems

3. Repairing Tests

1000 hours/month

The Biggest Problem: ~700 hours/month generating test data

18 months after a Micro-Services/Continuous Delivery rewrite.

Troubleshooting

Progress

Learning40-60% of dev capacity on “friction”

0:00 28:15

12:230:00

Case Study: From Monolith to Microservices

The Architecture Looked Good on Paper

Team A Team B Team C

Complexity Moved HereWTF?! WTF?!

We Don’t Have TIME To Fix It!

The Cost of Escalating Risk

0%

100%

Release 1 Release 2 Release 3

Troubleshooting

Progress

Learning

Percentage Capacity spent on Troubleshooting (red) and Learning (blue)

(extrapolated from samples)

0%

100%



Figure out what to do Learning is front-loaded

Troubleshooting

Progress

Learning


0%

100%



Rush Before the Deadline Validation is Deferred

Troubleshooting

Progress

Learning


0%

100%



Pain Builds Baseline friction keeps rising

Troubleshooting

Progress

Learning


0%

100%



Chaos Reigns Unpredictable work stops

fitting in the timebox

Troubleshooting

Progress

Learning


The Challenge:

How do I get my team to collect data??

Would you be willing to collect data if you knew your management would give you

dedicated time to work on the biggest problems?

1. Don’t ask for Permission

2. State your Goal "I want to make the business case to management for fixing things around here. No more chaos and working on weekends, this needs to stop. But I need data to make the case so I need everyone's help."

3. State the Plan "Here's what I'm thinking. I want to run an experiment to record data for one month on all the time we spend troubleshooting. We can look at the data together and identify our biggest problems, then I’ll write it up and present the case to management to get things fixed.”

4. Enlist the Team “Will you guys help me make this happen?”

Here’s What You Do:

1. Don’t ask for Permission

2. Make the Goal Clear to Your Team "I want to make the business case to management for fixing things around here. No more chaos and working on weekends, this needs to stop. But I need data to make the case so I need everyone's help."

3. State the Plan "Here's what I'm thinking. I want to run an experiment to record data for one month on all the time we spend troubleshooting. We can look at the data together and identify our biggest problems, then I’ll write it up and present the case to management to get things fixed.”

4. Enlist the Team “Will you guys help me make this happen?”

Here’s What You Do:


2. Measure the Pain





Add up the Pain by Category


2. Merging Problems

3. Repairing False Alarms

1000 hours/month


Friction as a % of total capacity


Friction % versus Upcoming Demand


Friction % Grouped by Familiar vs Unfamiliar



2. Measure the Pain





The Constraints

Stay (Mostly) Inside Developer Box

Communicate in Manager-Speak +

Your Job is to Repair the Broken Feedback Loop

Risk Translator

Engineering(Execution)

Management(Coordination)

Risk is the bridge language.

Manager


Knowledge(of(Risks(


Developer

RiskTranslator

Risk(Summary(

Risk Translator Role

Fits Within Developer Box

measures the time spent on:

Idea Flow

x

Troubleshooting

x

Learning

x

Rework

Quality Risk Familiarity Risk Assumption Risk

Quality Risk (Troubleshooting)

Likelihood)of))Unexpected)Behavior)

Cost)to)Troubleshoot)and)Repair)

High)Frequency)Low)Impact)

Low)Frequency)Low)Impact)

Low)Frequency)High)Impact)

PAIN)

Familiarity Risk (Learning)

Likelihood)of))working)with)Unfamiliar)

Code)

Cost)to)Learn)

High)Frequency)Easy)to)Learn)

Low)Frequency)Easy)to)Learn)

Low)Frequency)Hard)to)Learn)

PAIN)

Assumption Risk (Rework)

Likelihood)of))making)a))

Bad)Assump4on)

Cost)to)Correct)Decisions)

High)Uncertainty)Low)Delay)

Low)Uncertainty)Low)Delay)

Low)Uncertainty)High)Delay)

PAIN)

Decisions that save a few hours

Side-effects that cost several hours

Save 40 hours in direct costs(leave the toy on the stairs)

Increase chances of losing 1000 hours by 20%(tripping and falling)

Explain Problems in Terms of Risk (Gambling)

Distribution of Development Capacity

Over the long-term, probability wins.

Send “Project Visibility Updates”

Hi Larry, I know it’s really hard to stay in the loop on all the different project risks, so I wanted to send you a summarized update of some of our recent findings.

Subject: Project Visibility Update

We started collecting data during development to track where all of our time was going, and made some pretty frightening discoveries.

See attached. Let me know if you’d like to talk.

Risk Translators build

Trustby making sense.


2. Measure the Pain





Manager


Knowledge(of(Risks(


Developer

RiskTranslator

Risk(Summary(

Refactor Step 1: Risk Translator

So#ware(Task(

Mi.gate(the(Risks(


Product(Work(Queue(


Product(Owner(

Product(Decisions(


Dev$Team$Capacity$


Manager2Translator$Partnership$


Knowledge(of(Risks(

Refactor Step 2: Partnership

Now we can steer!!

So#ware(Task(

Mi.gate(the(Risks(


Product(Work(Queue(


Product(Owner(

Product(Decisions(


Technical(Risk(Manager(


Knowledge(of(Risks(

Dev$Team$Capacity$

Manager


Refactor Step 3: Owner

Option 1 Option 2

Stay the Course Change

This is Safer (less risky)

or

Make the Case for Partnership

Focus on the Risks (don’t negotiate schedule)

Key to Success:

1. Explain Why You Decided to Collect Data

Saw this talk/read this book about…

How to Measure the PAIN in Software Development

Janelle Klein

Consultant +1 Effect

(blame me)

Time%Pressure%

Compromise%Safety%for%

Speed%

Increase%Number%&%Severity%of%Hazards%

%

More%Pain%and%Higher%Task%Effort%

Constant'Urgency'

“In the book, Janelle talks about this “Cycle of Chaos”…

“As the problems build, they introduce Quality Risk…

Likelihood)of))Unexpected)Behavior)

Cost)to)Troubleshoot)and)Repair)

High)Frequency)Low)Impact)

Low)Frequency)Low)Impact)

Low)Frequency)High)Impact)

PAIN)

Likelihood of Mistakes

Cost to Recover

Quality Risk

Our application is more likely to be in a BROKEN state.

“We’re measuring the problems while we work…

Problems Measured in HOURS.

2. Here’s What We Found…

Pick your WORST offending examples.

Use lots of RED.

Save time by skipping diagnostic

tools (~80 hours)

Side-effects of Troubleshooting time (~700 hours/month)

36h 25m0:00

Troubleshooting

Progress11 hours and 15 minutes of troubleshooting...

Creating a New Customer Report

“This is a timeline that shows all the time we spend troubleshooting…

Save time by constantly rushing

(~20 hours/month)

Side-effects of 25 developers

down for 2 days (~1000 hours/month)

“When the problems build up, they have a really big impact…

“When the application is broken, these are the biggest problems in our way.

1000 hours/month


2. Merging Problems

3. Repairing False Alarms

Top Three Problems

“The deadline is coming either way…”

80% of features 100% done?100% of features 80% done?

“Here’s what we were thinking…”

3-Month Improvement Trial

Dedicated resources (1 or 2 developers)

Dev team identifies highest-leverage improvement opportunities and prioritizes with management

Continue to share Project Visibility Updates each month

“Will you help us turn this project around?”

Summary

The Biggest Cause of FAILURE in our Industry:

“Better”“Better”

We have an opportunity to do this across our industry:

ManagersDevelopers

Two Options:

Option 1

Stay the Course

Option 2

Take Responsibility

or

Let’s Do This

Here’s the Catch:

In order for us to change the status quo, we have to start working together as a community.

Let’s take responsibility,

learn how to get there,

then let’s help each other to succeed.

LEARN YOUR WAY TO AWESOME.

Free to Join Industry Peer Mentorship Network

openmastery.org

http://openmastery.org

If you’d like to see our industry start collaborating on solving these problems…

and you’re willing to Measure Your PAIN…

Let’s Make the PAIN Visible!Next Talk:

#OpenDXAn Open Standard for Measuring PAIN

(Specification for Data Collection)

Developer Experience

Community Analytics Platform

Idea Flow Mapping Tools

Team Mastery Tools

TeamJoe Sally Mark Eric

Community Analytics

Anonymized Data

(REST)

Shared Taxonomy of Patterns & Principles

(with example data)

ProjectTiger

ProjectBear

This isn’t about me.


This is about ALL OF US.

This is about Ending this BULLSHIT:


If you BELIEVE…


Carry the Torch.

Courage.


Leadership.

Empathy.

Authenticity.

Respect.

Take Responsibility


Two Options:

Option 1

Stay the Course

Option 2

or

Let’s Make the PAIN Visible!

What do you see as the biggest obstacle to success?

Discussion:

Stop Getting Crushed By Business Pressure

Software

Transcript of Stop Getting Crushed By Business Pressure