III Conferência CMMI Portugal, Keynote 1: Agile Methods and Capability Maturity: Addressing the...

Portugal

Agile Methods and Capability Maturity:

Addressing the Paradox

Professor Ian Allison

Head of School of Computing Science and Digital Media

Robert Gordon University, Aberdeen, UK

2013-10-18

2 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Introduction

3 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Introduction

4 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Introduction

5 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Introduction

6 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key Themes

• Background

• Agile – the maturity paradox?

• Case study of a high maturity organisation

• Good practices to be considered

7 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Agile Development

8 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Agile Processes

9 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Current Status of Agile

84% of organisations using agile

50% started in the last 2 years

35% of companies had distributed agile projects

Main Methodology: SCRUM [54%& 11% SCRUM/XP hybrid]

Key benefits:

• Manage change

• Improve productivity

• Improve project visibility

• Enhanced quality of software

VersionOne, 2013

10 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key techniques

0 20 40 60 80 100

Daily Standup

Iteration Planning

Unit testing

Retrospectives

Release Planning

Burndown/team estimation

Velocity

Coding Standards

VersionOne, 2013

11 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

The Problem With Agile

12 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Leading Causes of Failure

• Culture at odds with agile 12%

• External pressure to follow waterfall 11%

• Broader organisational communication problem 11%

• Lack of experience with agile 9%

• Lack of management support 6%

(18% claimed no failed projects)

VersionOne, 2013

13 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Main Concerns with Agile

• Lack of upfront planning

• Loss of management control

• Management opposition

• Lack of documentation

• Lack of predictability

• Lack of engineering discipline

• Inability to scale

• Regulatory compliance

VersionOne, 2013

14 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

How Agile Addresses the CMMI Practices in

3 Case Studies

CMMI Goal Agile Practices

Requirement management

Stories; product & sprint backlog; sprint planning; sprint reviews; planning games; information radiator; Daily meetings; on-site customer; self-organising teams; planning days; release days;

Estimates Sprint planning; product backlog; Sprint backlog; Planning games; tasks and effort estimations 1-2 weeks ahead; information radiator; planning days

Project Planning Planning games; task on information radiator; product backlog

Commitment to plan

Planning games; self-organising teams; on-site customer; reflection workshops; planning days; task cards on information radiator; release days

Project tracking

Short planning cycles; Burn down charts; project velocity; visual control; daily meetings;

Configuration Management

Collective ownership; small releases; continuous integration;

Paulk, 2001 & Pikkarainen & Mantyniemi, 2006

15 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Specific Goals – Spanish Experience

0

2

4

6

8

10

12

Garzas & Paulk, 2013

16 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

How Agile Companies Move Towards CMMI

Level 5

DTE Energy (Baker, 2005)

• Important role for SEPG

• Create practice groups: Project Management; Core

development; Process Management & Measurement

• PMO; Information office

• Education & Socialisation of the idea of CMMI 5

• Communication – messages to key audiences

Pragmatics (Cohen & Glazer, 2009)

- Metrics programme that adds value

17 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key Challenges for High Maturity

• Need for recorded evidence

• Need for project planning and control

• Self-Organizing vs Planned

• Organisational level of standardisation of process

• Project Metrics and performance prediction

• Organisational level data base-lining

18 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

High Maturity Case Study

• A CMMI level 5 Indian organisation working onshore &

offshore development

• Large enterprise with a revenue of $450m

• Study over the last year included site visits that allowed

interviews with 38 practitioners, observations and a review

of confidential agile process documents

19 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Agile History

• Started using in 2002… ad hoc use…

“so small projects here and small projects there, but since

2009 we have seen a huge uptake of agile …some

[projects] because the customer says; some because

there were a lot of problems so we recommended [agile]”

(Corporate Lead Architect)

• Then…

“the size of agile projects really increased. So instead of

having 20 or 10 members on projects, now suddenly we

have 140, you know large offshore development

programs being run in agile” (Corporate Lead Architect)

• Now 600 people … 20% of workforce on agile projects

20 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Need for Recorded Evidence

• Records required in CMMI for:

• Decision rationale

• Project review and evaluation

• It is possible to have evidence in many other forms…

21 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Need for Recorded Evidence

• Records required in CMMI for:

• Decision rationale

• Project review and evaluation

• It is possible to have evidence in many other forms…

22 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Need for Recorded Evidence – Case Study

Practice Head stated that “Sufficient evidence is required so

that “Senior Management Review and Risk Management can

be performed [so] there is no contradiction with agile”

Evidence is typically recorded on agile projects often takes

the form of:

• source code annotations (for lowest level decisions)

• email discussions

• Intranet project content management systems

• Wiki information repositories that organically evolve over

the project lifetime

23 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Need for Project Planning and Control

In CMMI it is important to have defined schedules, built from

defined requirements, upfront design and known productivity

rates

In agile approaches, evolution and responding to change

takes precedence over upfront planning.

However, the project plan that is expected in CMMI need not

be “what” but can focus on “how”.

24 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.




rates





25 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.




rates





26 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Project Planning and Control – Case Study

It was clear to the organisation that “every project needs to

have a project goal and a defined process … so, preparing

the initial project plan is not a problem” (Practice Head –

SEPG).

Partly this was because even in agile …

“in a distributed environment involving large team size, [then]

some form of upfront design is essential” (RK – Agile Coach,

Feb. 2013)

27 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Self-organising vs Planned

In CMMI:

• a consistent set of processes an organization is believed

to be able to reduce the variability between projects

• rigorous project management and control through a known

set of software engineering activities is the way this is

achieved.

In Agile:

• the emphasis is on self-organization and emergence

• product features emerge as the development progresses.

• release plan is flexible and expected to emerge based on

perceived business value.

28 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Self-organising vs Planned

As the Practice Head – SEPG put it “there is nothing in CMMI

which states that teams cannot be self-organizing.”

So the balance between planned and self-organizing was

struck by emphasizing the common elements of goal-setting

and enabling some upfront design to support the distributed

context.

29 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Need for Organisational Level

Standardisation of Processes

In SCRUM:

• it is commonplace to add engineering practices (e.g. XP)

• set of roles, practices and ceremonies defined is

rather small.

In the CMMI:

• large super-set of processes are defined;

• project teams must select practices from this

large set of process definitions.

30 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Need for Organisational Level

Standardisation of Processes – Case Study

When a project starts “we generally assign a coach to that

project” (Corporate Lead Architect)

To assess process compliance SEPG “will do monthly

reviews of the projects. And they also do audits to see if the

practices are being followed; are they following

continuous integration or are they making daily

builds” (Corporate Lead Architect)

31 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Project Metrics and Performance Prediction

• Metrics used for decision making and software

development process management to compare

between projects, spot trends, and anticipate

problems as well as for managing an individual

project.

• The data needs to be appropriate and part of the

normal process to minimize the effort of capturing

the data.

• CMMI recommends the use of a quantities model

to predict future performance. As long as the model

is specific to the project there is no contradiction.

32 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Project Metrics and Performance Prediction

– Case Study

• The company found “we faced problems in collecting data

required for level 4 KPA’s. The team felt that it was an

overhead.” (Delivery Head)

• “[The] most important aspect of complying with the metrics

needed for CMMI is to understand that traditional metrics

like, productivity and time slippage [are] not applicable …

instead we have been capturing metrics on defect,

resource utilization and on improvement.” (Practice Head –

SEPG)

33 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Organisation level data base-lining

The problem in Agile is the consistency of data between

teams to enable comparisons and trends to be seen.

Traditional data sets are not relevant for agile projects

It takes time to develop a data set from which organisations

can predict future activity

34 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Organisation level data base-lining – Case

Study

The problem is that “some projects use story points and

some projects do not use story points. They use function

points” (Corporate Lead Architect)

The “initial problem with organizational metrics was that we did not

have sufficient data points … with more agile projects getting

executed that is getting resolved.” (Practice Head – SEPG)

SEPG defined an activity based estimation model. “So for example

you are saying this is a four story point for this feature... Okay so we

are trying to define some guidelines on the basis of what you will do

with your story point estimation. But that is against your agile

concept. It should be spontaneous [within any given team]”

(Program Head)

36 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key Recommendations

• Establish key forms of record keeping techniques and

ensure they are auditable

37 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key Recommendations



• Define a project planning and control process based on

collaborative short-term planning and visual control

38 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key Recommendations





• In larger settings, use an architecturally-driven agile

approach for goal setting in large distributed agile projects

balanced with local self-organisation for devolved design

39 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key Recommendations








• SEPG to use a team approach to define a process map for

agile projects that allows local tailoring, but also to add

elements such as risk management

40 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Key Recommendations








• SEPG to use a team approach to define a process map for

agile projects that allows local tailoring, but also to add

elements such as risk management

• Develop a framework that enables the capture of data

relevant to their agile projects with consistent use of terms

41 © 2

013 C

MM

I P

ort

ug

al C

onfe

rence S

eri

es –

All

Rig

hts

reserv

ed.

Summary

• Challenges in aligning CMMI and Agile but no major

contradictions

• Agile environments can mature – and this can be achieved

through using an SEPG team approach to agreeing

practices, data, etc

• Agile in an offshore, large-scale distributed context will

naturally deviate from ‘pure’ agile – making CMMI links

more natural

• We have identified some embryonic guidelines

• Just one major case – so need to extend the collection

III Conferência CMMI Portugal, Keynote 1: Agile Methods and Capability Maturity: Addressing the...

Technology

Transcript of III Conferência CMMI Portugal, Keynote 1: Agile Methods and Capability Maturity: Addressing the...