Determining IT team performanceEstimation and uncertainty drivers

TIME/COST=CAPACITY

When making estimations we are primarily looking for two metrics

SIZE

When making estimations we are primarily looking for two metrics

=

Mature

?

Uncertain

CAPACITYSIZE

TIME/COST

Typically the capacity of a team is based on two techniques

CAPACITY

Historical and observed performance data

Subjective evaluation of current performance

1.

2.or

Let’s look at an example with the variance in historical performance

Size: 3000 FPTeam: 5 HCCapacity: 15-60 FP/M

3000

2.750

2.500

2.250

2.000

1.750

1.500

1.250

1.000

750

500

250

0

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Year 1 Year 2 Year 3 Year 4

Spread

Function Points

Time

Function Points

Time

Let’s look at an example with the variance in historical performance

3000

2.750

2.500

2.250

2.000

1.750

1.500

1.250

1.000

750

500

250

0

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Year 1 Year 2 Year 3 Year 4

Done between 10-40 months

Cost between 0,4 and 1,6 million Euro

Size: 3000 FPTeam: 5 HCCapacity: 15-60 FP/M

Bad PM’sChanging teamsNo standardsNo comparison foundationNo transparencyChanges in context

There can be several reasons for this spread…

And therefore our best option… is often our best guess..

3000

2.750

2.500

2.250

2.000

1.750

1.500

1.250

1.000

750

500

250

0

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Year 1 Year 2 Year 3 Year 4

Function Points

Time

Function Points

Time

3000

2.750

2.500

2.250

2.000

1.750

1.500

1.250

1.000

750

500

250

0

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Year 1 Year 2 Year 3 Year 4

The question is… with IT being so central to our business

– is this an adequate measure?

Source: Standish CHAOS “Decision Latency Theory” report; Project Management Institute ”Improve Business Results” infographic

43% 40%26%

57% 60%74%

On budget On time Achievedfunctionality

Yes No

109122

2015 2018

Most IT projects don’t live up to expectations…

…resulting in increasing investment loss

12%

Lost investment in mUSD per 1 billion USD spent

FASTER

BETTER CHEAPER

MORE

I saw this a the “black box” problem

FASTER

BETTER

CHEAPER

MORE

=

…and the problem was that we were lacking causal thinking

LOCATIONEND-USER PARTICIPANTSDB COMPLEXITYCOMPETENCECONSOLIDATIONDB SIZEREQUIREMENT NOVELTYCOOPERATIONTOOLSUSER STORIESDOMAINMETHOD MATURITYCULTURELEGACYPERSONALITYEXTERNAL INVOLVEMENTTEAM STRUCTURETEAM SIZEMANAGEMENT INVOLVEMENTBUDGET PRESSUREBUREAUCRACYDECISION MAKER PROXIMITYFUNCTION POINTSINTERFACESSTANDARDIZATIONPROGRAMMING LANGUAGEARCHITECTUREPROJECT ESTIMATIONPROCESS DESIGNEQEND USER INVOLVEMENTDB COMPLEXITYCAPABILITIESBACKUP/RESTORESOFTWARE COMPLEXITYEDUCATIONS/TRAININGSKILLSCOGNITIVE ABILITY

COMMUNICATIONPROCUREMENTPEOPLEREUSESOFT SKILLSTESTINGBUREAUCRACY

LOCATION

END-USER PARTICIPANTS

DB COMPLEXITY

COMPETENCE

CONSOLIDATIONDB SIZE

REQUIREMENT NOVELTY

COOPERATION TOOLS

USER STORIES

DOMAIN

METHOD MATURITY

CULTURE

LEGACY

PERSONALITY

EXTERNAL INVOLVEMENT

TEAM STRUCTURE

TEAM SIZE

MANAGEMENT INVOLVEMENT

BUDGET PRESSURE

BUREAUCRACY

DECISION MAKER PROXIMITY

FUNCTION POINTS

INTERFACESSTANDARDIZATION

PROGRAMMING LANGUAGE

ARCHITECTURE

PROJECT ESTIMATION PROCESS DESIGN

CMMI

END USER INVOLVEMENT

SECURITY

CAPABILITIES

BACKUP/RESTORE

SOFTWARE COMPLEXITY

EDUCATION/TRAINING

BIG DATA

BRANDING/MARKETING

COMMUNICATION

PROCUREMENT

PEOPLE

REUSESOFT SKILLS

TESTING

BUREAUCRACY

HARDWARE

COMMUNICATION

STRATEGIES

CYBER CRIME

SW PACKAGES

OUTSOURCING

FRAUD

SOFTWARE LICENSES

TECHNOLOGY STACKS

OPERATIONAL STABILITY

LOCATION

COMPETENCE

DB SIZE

METHOD MATURITY

CULTURE

DECISION MAKER PROXIMITY

PROGRAMMING LANGUAGE

ARCHITECTURE

CAPABILITIES

CONSOLIDATION

REQUIREMENT NOVELTY

TOOLS

TEAM STRUCTURE

MANAGEMENT INVOLVEMENT

FUNCTION POINTS

END USER INVOLVEMENT

INFRASTRUCTURE

SOFTWARE COMPLEXITY

END-USER PARTICIPANTS

SECURITY

COOPERATION

TEAM SIZE

STANDARDIZATION

PROJECT ESTIMATION

BACKUP/RESTORE

BRANDING/MARKETING

PEOPLE

DOMAIN

LEGACY

SECURITY

BIG DATA

COMMUNICATION

PROCUREMENT

REUSE

SOFT SKILLS

TESTING

USER STORIES

PERSONALITY

EXT. INVOLVEMENT

BUDGET PRESSURE

INTERFACES

PROCESS DESIGN

EQ

EDUCATIONS/TRAINING

BUREAUCRACY

BRANDING/MARKETING BIG DATA

PROCUREMENT

SOFT SKILLS

PERSONALITY

LOCATION

COMPETENCE

DB SIZE

PROGRAMMING LANGUAGE

CAPABILITIES

CONSOLIDATION

REQUIREMENT NOVELTY

TOOLS

MANAGEMENT INVOLVEMENT

FUNCTION POINTS

END USER INVOLVEMENT

INFRASTRUCTURE

SOFTWARE COMPLEXITY

END-USER PARTICIPANTS

DB COMPLEXITY

COOPERATION

STANDARDIZATION

PROJECT ESTIMATION

BACKUP/RESTORE

DOMAIN

SECURITY

COMMUNICATION

REUSE

TESTING

USER STORIES

EXT. INVOLVEMENT

BUDGET PRESSURE

INTERFACES

PROCESS DESIGN

EQ

EDUCATIONS/TRAINING

METHOD MATURITY

CULTURE

DECISION MAKER PROXIMITY

ARCHITECTURE

TEAM STRUCTURE

TEAM SIZE

PEOPLE LEGACY

BUREAUCRACY

METHOD MATURITY

CULTURE

DECISION MAKER PROXIMITY

ARCHITECTURE

TEAM STRUCTURE

TEAM SIZE

LEGACYBUREAUCRACY

1-100

~ 0-20%

~ 0-49%

~ 0-32%

~ -66% - +300%

~ 0-18%

~ 0-26%

~ 0-51%

~ 0-32%

PEOPLE

We arrived at the factors mostly influencing performance

Which could be expressed in a formula

People – 1-100x effect

Effect

• The quality of your team members

Source

• Harvard Business Review

• Internal studies from Google, Facebook, and Apple

Learning indicator

Behavior assessment

The

SECRETCODEASSESSMENT

In 7N we use the 5-layer model described in Nucleon

And use the SFIA framework to map the context

Team Size (up to -48% effect)

0

0,2

0,4

0,6

0,8

1

1,2

1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930

PR

OD

UC

TIV

ITY

FAC

TOR

TEAM MEMBERS

Effect

• The number of people on your team

• More members reduce productivity

Source

• Cognitive Load Theory

• The Mythical Man-Month – Frederick Brooks

Team dynamics (-66% to +300% effect)

0

0,5

1

1,5

2

2,5

3

3,5

1 2 3 4 5 6 7 8 9 10

PR

OD

UC

TIV

ITY

FAC

TOR

COMPOUND TEAM EFFECT

Effect

• The amount that high performers lift, and poor performers drag, your team

Source

• “Sitting Near a High Performer can Make you Better at Your Job” –Housman and Minor (2017)

Decision Maker Proximity (up to -32% effect)

0

0,2

0,4

0,6

0,8

1

1,2

1 2 3 4 5 6 7 8 9 10

PR

OD

UC

TIV

ITY

FAC

TOR

DECISION MAKER PROXIMITY

Effect

• The ease with which your team can interact with its decision maker and make fast and precise decisions

Source

• Harvard Business Review

• Standish CHAOS “Decision Latency Theory” report

Bureaucracy (up to -20% effect)

0,7

0,75

0,8

0,85

0,9

0,95

1

0% 20% 40% 60% 80% 100%

PR

OD

UC

TIV

ITY

FAC

TOR

PERCENTAGE BUREAUCRACY

Effect

• The amount of time spent not working on production tasks

Source

• “Team mental models and team performance” – Lim and Klein (2006)

• “Relationships among team ability composition, team mental models, and team performance” – Edwards and Day (2006)

Architecture (up to -26% effect)

0

0,2

0,4

0,6

0,8

1

1,2

1 2 3 4 5

PR

OD

UC

TIV

ITY

FAC

TOR

ARCHITECTURE SCORE

Effect

• How well you company's enterprise architecture is documented and understood to support ease of change/ implementation and re-use.

Source

• “The Relationship between Enterprise Complexity, Business Complexity and Business Performance” – Roest (2014)

• “Familiar Metric Management” –Putnam and Myers (1995)

Legacy (up to -18% effect)

0,5

0,6

0,7

0,8

0,9

1

1,1

1 2 3 4 5 6 7 8 9 10 11

PR

OD

UC

TIV

ITY

FAC

TOR

LEGACY SCORE

Effect

• How many hidden resources you invest in maintenance of obsolete systems

Source

• Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software – David Scott Bernstein

• QSM databases

Culture (up to -32% effect)

Effect

• To which degree your team’s culture accelerates or decelerates productivity

0,5

0,6

0,7

0,8

0,9

1

1,1

1 2 3 4 5 6 7 8 9 10

PR

OD

UC

TIV

ITY

FAC

TOR

CULTURAL SCORE

Source

• Primed to Perform: How to Build the Highest Performing Cultures – Doshi and McGregor (2015)

• “The Relationship between Corporate Culture and Performance” – Dizik (2016)

Methods Maturity (up to -51% effect)

Effect

• The length of time your team has been working together

PR

OD

UC

TIV

ITY

IND

EX

MONTHS OF PRACTICAL EXPERIENCE

Source

• “Managing the Development of Large Software Systems” – Royce (1970)

• “Agile & Waterfall Methodologies – A Side-By-Side Comparison” – Base36

So what does this mean in practice?

So what does this mean in practice?

Effect

77% increase in effectivity -equivalent to a potential 195 million Euro saving

So what does this mean in practice?

Effect

25,5% increase in effectivity -equivalent to a potential 91 million Euro saving

So what does this mean in practice?

Effect

14,2% increase in effectivity -equivalent to a potential 55 million Euro saving

So what does this mean in practice?

So based on the Nucleon analysis one of the largest Scandinavian banks could look at a total saving of 341 million Euros

- With a prioritized roadmap suggested for the implementation

- And an ability to get detailed, real-time performance knowledge and better estimation and simulation capabilities

”In fact, it is not just a formula – that is the summary – it is a complex family of measurements and analysis that arecompared against best practices in a structured way to reveal all of the major, minor and micro fractures and defects in your IT organizational crystal”

Jim Ditmore, COO Danske Bank

www.nucleonformula.com

Jeppe Hedaa: jh@7n.comn

Thanks for your time