Large public sector projects – what determines failure or success? Scrum Gathering Atlanta 2012

25.09.2014 • © PROMIS AS 1

Large public sector projects

– what determines failure or success?Remi Hansen, PROMIS AS

25.09.2014 • © PROMIS AS 2

Benefits from attending

• Learn what factors are important for succeeding with large government Scrum

projects and what pitfalls to look out for.

25.09.2014 • © PROMIS AS 3

The Presenter – Remi Hansen

• B.Sc. In SW Engineering, M. Sc. in Industrial Economics

• More than 20 years experience from

the IT Consulting business

• A few years as a system developer

• Primarily project manager and advisor / Business Consultant

on strategic projects in both private and public sector

• Some years as line manager – incl. Head of the Project Management Community in

one of Scandinavia’s largest IT consultancy groups, with more than 150 PMs with

extensive use of Scrum in their projects

• Certified Project Management Professional (PMP), PRINCE2 Practitioner,

IT Project Professional (ITPP), CSPO, ISTQB SW Testing Foundation and ITIL.

• Experience with agile projects started in 2002.

• Currently Senior PM in PROMIS, a leading provider of agile project

management services in Norway (www.promis.no)

http://www.promis.no/

25.09.2014 • © PROMIS AS 4

Background

• The Nordic model – mixed market economy, welfare states

• “It is distinguished from other welfare states with similar goals by its emphasis

on maximizing labor force participation, promoting gender equality, egalitarian

and extensive benefit levels, large magnitude of redistribution, and liberal use

of expansionary fiscal policy.” (Wikipedia)

• Public sector > 45 % of GNP

• Huge public sector spending large and complex IT systems

• Our fair share of bad projects

• Agile (Scrum) has become the standard approach for public sector projects

• Same motivation as in private sector – a fixed and stable requirements specification

will not give the desired outcome

25.09.2014 • © PROMIS AS 5

Agenda

Introduction 10 min

• Brief presentation of the three projects: 10 min

• Six main recommendations: 60 min

• Lessons learned summary: 10 min

• Q&A

25.09.2014 • © PROMIS AS 6

The case study projects

25.09.2014 • © PROMIS AS 7

Case study

• Lessons learned from 3 federal

government projects

• Over 1,000,000 hours of effort

• Not a scientific study

• 1st and 2nd hand information used

• Factual information is from public sources,

the rest is partly my personal opinion and

partly from sources listed

Photo (Flickr):

Okko Pyykkö

25.09.2014 • © PROMIS AS 8

Norwegian Public Service Pension Fund (SPK)

PERFORM project

• SPK

• Administers pension entitlements of $ 59 billion on behalf of the Norwegian state

for almost 1 million former and existing employees in the public sector

• $ 3,2 billion in pension payments (2009)

• Purpose / benefit of the project

• The pension reform

• Outdated technology platform

• New benefit type introduced

• Status: Just completed – very successful, seen as best practice in agile

implementation

• Size:

• 13 teams, 2+1 main suppliers, 175 people

• 300 epics 2500 user stories

• 12 releases over 4 years

• $ 175 Million

25.09.2014 • © PROMIS AS 9

Norwegian Public Service Pension Fund

PERFORM project

• Interesting feature

• Huge project – needed to start development before legislation was passed

• Contract and sourcing setup

• Multisourced

• T & M on Solution Description phase – the customer retained responsibility for

analysis and conceptual design

• Suppliers responsible for construction of their deliveries

• Each delivery (release) negotiated with a separate target price – i.e. target price on

user story level

• Target price 50 / 50 (no cap)

25.09.2014 • © PROMIS AS 10

Statnett

LARM Project

Statnett

• State Enterprise responsible for all high

voltage electricity transmission

and distribution in Norway

• Owns, operates and regulates

the national main grid

• Statnett co-ordinates supply and demand, and owns the main Norwegian power grid.

Photo: Statnett.no

25.09.2014 • © PROMIS AS 11

Statnett

LARM Project

LARM

• Community-critical system for ensuring the responsiveness of

the national power grid

• Status:

• Ongoing (midway)

• Challenged (overruns and conflicting interests)

• Size:

• 2 mid-sized Scrum teams initially – gradually increased staffing

• $ 20 million (supplier cost only), 3 year project


• Very specialized domain – few users, but community critical operation


• Target price with customer friendly profile (very close to fixed price contract)

• All activities and phases included in target price

• Price commitment on total scope Photo: Statnett.no

25.09.2014 • © PROMIS AS 12

The Norwegian Public Roads Administration

AUTOSYS project

The NPRA

• Responsible for the planning, construction and operation of the national and

county road networks, vehicle inspection and requirements, driver training and

licensing

Autosys

• Replacing 30 year old mainframe system

• More self-service, fewer manual processes

• 20 000 users, 100 million transactions / year

8 million vehicles and 3,5 million driver licenses

• 5 releases in 3 years (overall 5 year project)

• SOA platform

• Status: Ongoing, challenged

• Size:

• 65 on supplier side, 35 on customer side

• 4 Scrum teams

• $ 100-150 million project budget ($70 M supplier cost)

Photo: Vegvesen.no

25.09.2014 • © PROMIS AS 13

The Norwegian Public Roads Administration

AUTOSYS project


• Previous attempt to run the project terminated – generally seen as a failure outside

the NPRA (called a “scandal” by the press)

• High visibility to the public


• Target price with customer friendly profile (close to fixed price contract)

• All activities and phases included in target price

• Price commitment on total scope

25.09.2014 • © PROMIS AS 14

Case study projects summarized

• Large to very large projects in public sector

• Very different domains and parts of the state administration

• All Scrum based projects

• One successfully completed, two ongoing challenged

• All rely on IT suppliers governed by contractual obligations

• All target price contracts (PS2000 based), but with very different customer /

supplier balance

25.09.2014 • © PROMIS AS 15

Agenda

Introduction 10 min

Brief presentation of the three projects: 10 min

• Six main recommendations: 60 min


• Q&A

25.09.2014 • © PROMIS AS 16

Main recommendation areas

Six main recommendations:

1. Enterprise scale Scrum implementation

2. Soft skills and cultural aspects

3. Business value and product backlog issues

4. Integration of test and development tasks in sprint teams

5. Decision making and team interaction

6. Specific public sector challenges and contractual implications

25.09.2014 • © PROMIS AS 17

Area 1: Enterprise scale Scrum

implementation

25.09.2014 • © PROMIS AS 18

Findings

Enterprise scale Scrum implementation

What is needed to run a large Scrum project?

Wanting to be agile, but also maintaining predictability and control – different levels

of agility seen

Different approaches to scaling the projects’ Scrum implementation regarding

a) Organization, roles and responsibilities

b) Processes

c) Planning and controlling

d) Testing and approval

25.09.2014 • © PROMIS AS 19

Findings


Contracts

• Add complexity – A large supporting organization is generally not need with two

teams, but a large contract requires administrative overhead

• Customers (also in private sector) want supplier commitment to cost /

productivity / buy-in

• A large project under a contract is different from just adding some extra Scrum

teams to your in-house Scrum development

Revisited later!

25.09.2014 • © PROMIS AS 20

Recommendations


• Everybody cannot overview every aspect of the project when they get large –

division of responsibility needed

More people narrower field of responsibility and more interfaces to other

roles bigger chance of miscommunication

• More roles added needs clearly defined responsibilities

25.09.2014 • © PROMIS AS 21

Scrum Team

Scrum master

Subject Matter

Expert (Cust.)

Test specialist

(Cust.)

Technical

ArchitectDeveloper / tester

Developer / tester

Developer / tester

Developer / tester

Developer / tester

Scrum team

25.09.2014 • © PROMIS AS 22

Example organizationJoint

Steering Committee

Customer’s

Programme

Director

Supplier

Programme

Manager

Customer

Deliverables and

testing

Sub-project

Development

Scrum

team 4

Scrum

team 2

Scrum

team 1

Scrum

team 3

Supplier

Project Owner

Test

Manager

Technical

Architect

Usability

experts

Solution

Architect

Config. Mgr.

Tech. support

Test specialist

Technical tests

Test specialist

Functional tests

Training & impl.

Co-ordinator

Oracle Enterprise

Architect

Business Process

Expert

Supplier Internal SC

Customer

Internal SC

Customer

Implementation

project

Maintenance

coordinator

(from Delivery 1)

25.09.2014 • © PROMIS AS 23

Recommendations


Some relevant roles

• Project management – large financial impact, …

• Product team – instead of single Product Owner (covered later)

• Architecture / technical environments team

25.09.2014 • © PROMIS AS 24

Recommendations

b) Processes

• More defined processes are necessary in large projects

• More roles and people, more fine grained division of work

• Not the level of documentation that counts,

it’s the level of common understanding!

Photo (Flickr):

IvanWalsh.com Photo (Flickr):

Samuel Mann

25.09.2014 • © PROMIS AS 25

Recommendations

c) Planning and controlling

• A release master plan (solution roadmap) – not just a product backlog

• Control gates: Book keeping for each sprint (earned value calculations etc) –

covered later

25.09.2014 • © PROMIS AS 26

Recommendations

c) Planning and controlling – Release Master Plan

• The Release Master Plan is a tool for planning handover and implementation of

new functionality

• A release demands capacity in the organization – the must be prepared

• The Release Master Plan is a tool for the project team and forms the foundation for

all sprints together to form a predictable and acceptable delivery

• The Release Master Plan identifies the number of releases, their size and

timing

• Orchestrates deliveries and pinpoints internal and external dependencies

• Shows milestones and phases toward go-live

• Basis for budgeting

• Guiding principle: Realization of business value!

25.09.2014 • © PROMIS AS 27

Recommendations

c) Planning and controlling – Release Master Plan

Content:

• Functional target

• What epics (functional and technical) are included?

• High level priorities

• Team capacities

• The number and duration of sprints

• Planned dates for sprint demos, control gates, start and finish for acceptance

test, go-live

25.09.2014 • © PROMIS AS 28

Recommendations

d) Testing and approval

• Control gates following each sprint

• Dedicated test phase (addressed later)

25.09.2014 • © PROMIS AS 29

Recommendations

d) Testing and approval – Control gate at sprint completion

• To check if the user stories meet Definition of done, the project performs a

«control gate»

• Administrative «tail» - aftermath of one sprint spills over into the next sprint –

Should be kept as short as possible to avoid discontinuity

• With some training, this may be performed in 2-3 working days following the

sprint demo

• During these days, it is ‘all customer representatives to the pumps’

• Functional verification by Product Owner and his / her team

• Checking code quality

• Checking architectural guidelines

• Checking test documentation

• Checking other documentation

• Contractual milestone

25.09.2014 • © PROMIS AS 30

The Anatomy of the Sprint

Sprint demo Sprint X

Sprint planning Sprint X+1 starts with decomposition

Sprint backlog for Sprint X+1 ready

Control gate:

1. Evaluation of Sprint X

2. Approval of sprint plan for Sprint X+1

3. Updating the risk matrix

4. Change control

5. Repatriation of agreed outstanding issues to the

product backlog

25.09.2014 • © PROMIS AS 31

Construction phase: The control gates

Blue line: Sprint team

Red line: Product Owner

Yellow line: Verification, Control gate and Definition of Done

Construction phaseSolution description

phase

25.09.2014 • © PROMIS AS 32

Planning and controlling considerations in CG

What was the productivity of the last sprint?

How was the product quality?

How is the risk picture evolving?

What can we improve – emphasis on learning and continous improvements

25.09.2014 • © PROMIS AS 33

Recommendations

d) Testing and approval – Commercial reconciliation

• Payments are sometimes linked to progress in development

• Our clear recommendation is to avoid commercial conditions linked to the

Control Gates – commercial reconciliation should wait until Acceptance testing

is completed

Photo (Flickr):

Images_of_Money

25.09.2014 • © PROMIS AS 34


Key Learning Points

Size and contractual obligations drives the need for a

supporting project organization

Keep organization and processes on a practical level

to maintain agility

Use a Release Master Plan for planning and

communication

Execute a Control Gate process after each sprint for

control (progress / velocity, productivity, risk, etc.)

Photo (Flickr):

Horia Varlan

25.09.2014 • © PROMIS AS 35

Area 2: Soft skills and cultural aspects

25.09.2014 • © PROMIS AS 36

Findings

Soft skills and cultural aspects

• The projects differed significantly in their ability to build a common project

culture

• Contract parties not bonding together is very counterproductive – more focus

on handovers than the actual work to be done

• Project initiation takes time and should be given sufficient time

• Depending on both customer and supplier – T&M pricing is a good idea in this

phase, to ensure enough time and effort is spent on initiation

• Different cultures collided – washed away the basis for mutual understanding

and trust impeding effective interaction

• Customers adapting bad practice based on lack of experience

– Knowing when to seek external help is very useful

• Lacking the courage to take decisions severely hampers progress

25.09.2014 • © PROMIS AS 37

A clash of organizational cultures

Stereotypical public sector officer

behavior

Anticipated contribution in an agile

system development project

Work according to regulations Design future work processes based on

business value

Working with individual cases See different types of processes in

relation to simplify and standardize

Concrete work patterns "as is" – few

thoughts about future processes

Conceptual modeling of future processes –

limited knowledge of existing processes

Exact rules and all exceptions Good enough and main processes first

Collective / Agency Individual / user

Long-term production perspective Project delivery now

Don’t forget: What we expect from customer representatives

may be very different from what they do on a daily basis

– different jobs attract different people!

25.09.2014 • © PROMIS AS 38

Recommendations

Project culture and soft skills

• Invest sufficient time in creating a common project culture - focused on creating

business value

• Customer representatives in the Scrum teams is a good start!

• Take into account that cultural mismatches affect productivity

25.09.2014 • © PROMIS AS 39

Recommendations

Avoiding bad practice

• Contracts emphasis the supplier vs. customer obligations, less on the project

as one group

Avoid contractual thinking from team members on all levels – confine that to

the management level

• Appoint managers who are not afraid to take decisions

• Make sure to get external help to promote better practice – a good way to

resolve conflicts

• Use advisors to assist in resolution, not for gathering ammunition for a possible

blame game

25.09.2014 • © PROMIS AS 40

Soft skills and cultural aspects

Key Learning Points

Organizational culture aspects may influence your

project’s ability to succeed – act accordingly!

Scrum is relying on effective decision making – create

a culture to promote that

Photo (Flickr):

Horia Varlan

25.09.2014 • © PROMIS AS 41

Area 3: Business value and product backlog

issues

25.09.2014 • © PROMIS AS 42

Findings

Business value and product backlog issues

• Poor quality of the product backlog is devastating for the project

• Too large user stories impossible to remove anything from the scope

(every user story contains something critical)

• Poorly defined user stories many discussions late completion of user story

limited time for testing

• Ambiguities low productivity and many assumptions (risk)

• Decision making: real (and lasting!) prioritization is difficult for inexperienced

P.O.s

• A lot of accumulated needs in the organization

• A tendency to behave like kids in a candy store, if there are no mechanisms to

control scope

• Responsibilities for different aspects of the product backlog is a source of

misunderstanding and controversy

25.09.2014 • © PROMIS AS 43

Findings

The vicious circle of poor backlog quality

Poor quality of the sprint backlog (capacity and/or expertise)

Abundant clarifications needed - takes more time from the P.O. and Scrum

teams

Late clarifications late completion of user stories

Poor tests and low productivity (developers losing focus, multitasking unclear

tasks)

More effort needed from both the customer and the supplier into the next sprint

to sort out whether user stories are “done”

Prolonged Control Gate period

Reduced opportunity for P.O. team to contribute to the planning and launching

of the next sprint (busy doing backward-facing activities) – introducing

uncertainty in the sprint plan

Lack of commitment from the teams – not possible to deliver the sprint contract

anyway, because of the poor sprint backlog quality

25.09.2014 • © PROMIS AS 44

Findings


• The P.O. team did not work effectively – the Chief P.O. had to take all

decisions

• The customer did not meet their obligations regarding Business Analysts and

Test Specialist – partly having the wrong profiles

• The resulting lack of clarifications and scoping (acceptance criteria) gave extra

load on the P.O.

25.09.2014 • © PROMIS AS 45

Recommendations

Good product backlog practice

• The customer must control the scope!

• A well organized P.O. Team supporting the Chief P.O. is needed

• Individual P.O.s must co-ordinate across domains – creating one coherent

backlog

• Appoint a Business Analyst to each sprint team = P.O.’s delegates to the

teams will relieve the load on the P.O.

– Preferably Customer Business Rep., but in a long project even a hired

Business Analysts can add value

25.09.2014 • © PROMIS AS 46

Recommendations

Good product backlog practice

• Continuously work with grooming the backlog – invest enough capacity to

clearly define User stories, and give priority

• Setting priorities takes a lot more effort than one should expect – working

towards the line organization and avoiding rematches

• Put in place a method for assessing business value – to avoid somebody’s pet

requirements sneak past prioritization

• View the User story estimate as the budget for the task – don’t estimate the

task again with high level of freedom and creativity

• Challenge the P.O. to find a simpler solution if the scope doesn’t match the estimate

25.09.2014 • © PROMIS AS 47

Product owner team

Development team 1 Development team 2 Development team 3

Product owner team

Product owner

25.09.2014 • © PROMIS AS 48


Key Learning Points

• The quality of the backlog cannot be stressed enough –

the project success completely relies on it!

Ensure enough effort is spent grooming the backlog

• Use business value as basis for prioritization

• Set up a P.O. team as described and appoint a

Business Analyst to each scrum team

Photo (Flickr):

Horia Varlan

25.09.2014 • © PROMIS AS 49

Area 4: Integration of test and development

tasks in the sprint teams

25.09.2014 • © PROMIS AS 50

Findings

Integration of test and development tasks

• Different system test approaches taken – integration of test into development

seems to give the best result

• The customer did not participate with test specialist in the Scrum teams as

dictated by contract

• Absence of testing capacity in the teams was compensated with extra effort

from the test team

• Role confusion (became part of the development team and not a gatekeeper to dev.)

• Capacity problems in the Test team not well prepared for system test

• Poor testing in the sprints the customer had to test more in Control Gate

(after sprint demo)

• Delayed CG and sprint plan for next sprint

• More book keeping and duplication of tests at the expense of other important work

• Aftermath stealing attention from planning

• Test data issues

25.09.2014 • © PROMIS AS 51

System Hardening sprints

• 2-3 sprints after the development sprints to perform system test and complete

outstanding work (typically documentation and clean-up) – done in parallel with

solution description for the next delivery

• Is that a good idea?

There’s always some activities not completed. May give better preparation for and

smoother execution of Acceptance test

Real integration test will not be done along the way becomes a waterfall

approach, where testing is postponed (long time from programming to testing)

Risk mitigation is delayed

Very demanding for the project to do both system testing and solution design in

parallel

Probably not a good idea – include system testing in the sprints instead to

minimize time from programming to testing

25.09.2014 • © PROMIS AS 52

Recommendations


• The Test Manager should state quality requirements to development teams –

i.e. Definition of Done related to testing

• The Test Manager should have the authority to label user stories «Done» - i.e.

accepting what user stories will be demonstrated at sprint completion

• The Test Manager must have the motivation to withhold user stories not ready

for demo

• Allocate time in the coming sprint to do system tests of previous sprints delivery

– rather than waiting and doing all system testing in the end (hardening phase)

25.09.2014 • © PROMIS AS 53

Recommendations


• There should be very limited need for the customer to perform separate tests –

rather verifying the test activities undertaken by the supplier

• In traditional waterfall environments «acceptance» is a very important decision

– such customer’s need reassurance that approving a user story does not

mean they have accepted the quality of the system (just that this part of the job

is done and in accordance with expectations) – i.e. commitment from the

customer on the content

25.09.2014 • © PROMIS AS 54

Recommendations


• Integrate test into Scrum teams

• Better flow and continuity

• Close cooperation between business, developers and testers

• Appoint one in each Scrum team as Team Test Responsible

• No formal responsibilty for the product quality (still team responsibilty)

• Assignment to keep a close look at the testing activities and remind the others to do

their testing

• Planning, design, test conditions, development

and test performed collectively by the team

• Testers are no longer quality police

– instead valued members of the team

• Unit tests, integration tests, functional system

test and system integration tests

in construction sprints

25.09.2014 • © PROMIS AS 55

Test in agile development

• Tests are run continuously through the period

• Acceptance criteria written in parallel with

description of needs and solution design

• Tests written in advance of programing and

run in the sprints for completed user stories

• What can be approved, should be approved

progressively (in Control Gates)

25.09.2014 • © PROMIS AS 56

Integration of test and development tasks in the sprint teams

Key Learning Points

Maximize the testing done as part of the development

sprints

Appoint a test responsible in each sprint team

Empower the Test Manager to set process

requirements

Avoid pointless replication of tests on the Customer

side

Photo (Flickr):

Horia Varlan

25.09.2014 • © PROMIS AS 57

Area 5: Decision making and team interaction

25.09.2014 • © PROMIS AS 58

Findings

Decision making and team interaction

• It takes courage to manage an agile project – No collective decision process to

hide behind

• E.g. as Product Owner you put your head on the block – will public sector employees

feel they can gain benefits making that worthwhile?

• Demanding to say “we don’t need that documented”

• Blame game mindset introduce waste and is self-perpetuating – fight it!

• The relentless will to learn and improve is not evident in every project – those

who learn fast seems to succeed

25.09.2014 • © PROMIS AS 59

Findings


• With larger projects the teams need guidelines – don’t leave it to the teams to

sort out how the project should work

• Delegate the task, but make a central decision

• All projects used self-organizing teams – with success

• Able to manage their own work and maintain good motivation

• Retrospectives worked well in all projects

• The teams used a wide range of techniques in the retrospectives – the process

worked well

• Important to summarize recommendations and actions across teams (incl. the

customer)

• With slippage and scope challenges the teams started to take shortcuts

• Measurement primarily on velocity – pressure to take on more new functionality

• Started building technical debt and accumulated risk

• Stopped taking responsibility for cross-team issues

25.09.2014 • © PROMIS AS 60

Recommendations


• Defer decisions

• Let those best qualified take the

decision – create a culture for

delegation and trust

• Use “mini demo” to reduce risk

• Introduce friendly competition between

the teams

Photo (Flickr):

Budzlife

25.09.2014 • © PROMIS AS 61

Recommendations

Communication boosters

• Separate daily Scrums are not sufficient

for coordination

• Scrum of Scrums

• Meta Scrum

• «Town hall meetings»

• Communities of practice

• Use visualization actively

to communicate concepts, plans,

and progress – the master release plan

is a good starting point

Photo (Flickr):

jardenberg

25.09.2014 • © PROMIS AS 62


Key Learning Points

Key stakeholders must be courageous - Strive for a

culture based on trust

Stress the importance of retrospectives and pursue

improvements

Use communication boosters to ensure everybody is

on the same page

Photo (Flickr):

Horia Varlan

25.09.2014 • © PROMIS AS 64

Findings

Specific public sector challenges and contractual implications

• Public sector is engineered for cost control – «must» have a contract with

supplier obligations (a T&M project is difficult to fund in public sector budget

processes)

• Little personal benefit from taking risk and accountability

• No culture for accountability for project benefits – without financial

measurements the planned benefits may be too vague

• Two of the projects seem to be systematically underestimated – this

undermines the target price mechanism

• Sticking to unrealistic estimates is counterproductive – the supplier will not deliver as

good as it can and the customer will probably not get a very good system

• Two of the projects included the initiation phase in the target price, with

penalties on “initiation complete” milestone. The supplier thus defined itself as

finished on time, even if the delivery was not complete. That was paid dearly in

later stages.

25.09.2014 • © PROMIS AS 65

Findings


• In PERFORM the customer chose a balanced contract and took responsibility

for scope control

• Contract assignment per supplier per delivery at the end of each Solution

Description phase

• The challenged projects were more focused on cost control than having a

balanced contract

• Close to fixed price (cap on risk sharing)

• Including initiation and solution description in the fixed price

• Commitment on price for the complete scope, without any means of adjustment to

actual productivity

• This seemed to also reduce the customer’s focus on scope control – defining

that as supplier responsibility

• When overruns appeared (effectively making the price model = fixed price) we

saw a “culture of blame”

25.09.2014 • © PROMIS AS 66

Contracting price models:

• Time&Material:

• The Customer pays for all worked hours, disregarding

productivity and quality – all risk is on the Customer

• Fixed Price:

• The Customer pays the same amount, disregarding the hours

needed to produce the scope with agreed quality – all risk is

on the Supplier

• Target Price:

• Customer and Supplier share benefits or losses in an agreed

ratio – if this ratio is 50%, the risk is evenly distributed

between the parties

25.09.2014 • © PROMIS AS 68

What price model is beneficiary in agile projects?

Target price is best suited because:

• Agile is a close and mutual co-operation

between customer and supplier

• Target price with 50/50 sharing of

incentives and sanctions ensure the

customer and supplier works towards the

same goal

• Target price is suitable when requirements

are not very detailed

• Target price is suitable for projects with risk

• Target price can be based on a less

detailed basis than traditional fixed price

Price model Average overrun

Time & Material based

(4 projects)

55%

Fixed price (5 projects) 33%

Target price (7 projects) 10%

Others (2 projects) 13%

Total (18 projects) 27%

• Fixed price assumes

requirements are described in

detail and that uncertainty is

low.

• T & M is suitable when scope is

not well defined and uncertainty

is high.

25.09.2014 • © PROMIS AS 69

Why Target Price Contracts with Risk Sharing?

• The Supplier is invited to commit to estimates with a higher degree of

uncertainty than in traditional fixed price contracts

• This is paramount in agile system development projects, where one of the main

principles is to avoid waste

• In particular, effort can be reduced on initial specification, detailed design and

planning

• The Customer may produce tender documents with high level requirements

• The Suppliers produce solution descriptions, estimated with a significant

degree of uncertainty

• In this way, both parties reduce work load in the initial bidding

phases of the project

25.09.2014 • © PROMIS AS 70

Recommendations


A balanced contract is necessary to ensure commitment and quality

• Implies 50 / 50 risk sharing with no cap

• Not agile to commit to up-front estimates for several years based on very

limited knowledge – risk for the supplier and ultimately the customer

• Include mechanisms to avoid commitment to estimates over a very long time

• Negotiating target price per delivery (release)

• Other mechanisms to adjust to actual velocity and productivity

• Consider T&M for the initiation phase, to ensure no corners are cut

25.09.2014 • © PROMIS AS 71

PS2000 – a Target Price Contracting standard

• A Norwegian contracting standard in the IT industry

• A result of a research program during the years 1997 – 2000

• By 2012, used in a number of large system development projects in the

Norwegian public sector

• Also used in private sector and internationally

• Referred to in Mary and Tom Poppendieck:

’Implementing Lean Software Development:

From Concept to Cash’ Addison-Wesley 2007

http://www.amazon.com/gp/product/images/0321437381/ref=dp_image_0?ie=UTF8&n=283155&s=books

http://www.amazon.com/gp/product/images/0321437381/ref=dp_image_0?ie=UTF8&n=283155&s=books

25.09.2014 • © PROMIS AS 72

PS2000 Standard contract - English version

http://www.dataforeningen.no/it-contract-standards.146223.no.html

Requirement

Analysis Acceptance and -

Completion phase

Detailed planning Analysis and design

Testing Development

Progression

Iterative

construction phase

CG1

CGn C

CG2

PMS 0 Contract Signing

Solution

Description

PMS 1 Approved Solution

Description

PMS 2 Delivery ready

for Acceptance

PMS 3 Accepted

Delivery


25.09.2014 • © PROMIS AS 73

In summary - Advantages with PS2000 Agile

• Corresponding to the principles in Agile in a better way than alternative

contracting models

• Not unlike agreements to deliver competence and capacity (as in

Time&Material contracts), but framing in the scope, time, cost and risk

• Suitable for repeating releases in a frame agreement for new development or

application maintenance

• An incentive for the supplier to deliver more functionality within agreed time

limits, but with satisfactory quality

• The main success factor is that the parties agree on the process manging the

Product Backlog, Sprints and Control Gates, with the corresponding roles

• Change control becomes lean – a signed Product Backlog after each sprint

documents the agreed changes

25.09.2014 • © PROMIS AS 74


Key Learning Points

Agile projects can be run under a contract, given that

the contract

Is balanced

Is engineered for agile execution

Lets scope be defined as late as possible

(preferably contracting each release as a separate target

price delivery)

PS2000 Agile is such a contract standard

Photo (Flickr):

Horia Varlan

25.09.2014 • © PROMIS AS 75

Agenda

Introduction 10 min

Brief presentation of the three projects: 10 min

Six main recommendations: 60 min


• Q&A

25.09.2014 • © PROMIS AS 77

Summarizing some major issues

• Impression that all the projects would be agile – only one of them were

• Contractual constraints (~ fixed price)

• Cultural challenges – still a fixed price mindset on the customer side

• Customer not participating as actively as anticipated – both staffing and decision

making

Scrum does not necessarily make a project agile!

• Truly agile execution is possible within a contract constraint, with the right

contract

• Naive to expect Scrum / agile to solve every problem

• Even the most successful project had challenges in the first stage – but a

willingness to learn & adapt brought the project on the path of good practice

25.09.2014 • © PROMIS AS 78

Some smart moves in the PERFORM project

paving the way for success

• Gained experience with Scrum before starting the project

• Understood the value of a balanced contract

• Initiation phase on T&M price model

• Had own Scrum teams in parallel with

supplier teams

• Invested in good procedures and tools for migration

and configuration control

• Test integrated part of development

• Hired external help for agile project management

coupled with active top management involvement

• The customer took overall responsibility for all processes – the main suppliers

had responsibility for construction of their parts for every release

• A continuous focus on improvement

Truly agile execution!

Photo (Flickr):

apdk

25.09.2014 • © PROMIS AS 79

8 defining features of agile projects

1. Business value is the most important criteria for quality and direction

2. Continuous prioritization of functionality based on cost / benefit

3. Close communication between the business side and developers

4. Short iterations to delivery of executable code

5. Frequent releases to production (integrated and tested)

6. Binding decisions taken as late as possible (‘Rolling Wave Planning’)

7. Evaluation, learning and improvement along the way

8. Autonomy: Self-organizing cross-functional teams

25.09.2014 • © PROMIS AS 80

Defining features of agile projects

Feature Degree of

divergence in case

study

Covariance

between good agile

practice and

success

Business value focus High

Continuous prioritization High

Communication with business Medium-High

Short iteration – executable code Low

Frequent releases Low

Deferring decisions High

Learning Medium-High

Autonomy Low

25.09.2014 • © PROMIS AS 81

Other reflections

• Using agile methodology does not replace good craftsmanship in software

engineering

• Emphasis on architecture

• A good process to establish the product backlog

• Best practice in conceptual design / modelling and code standards

• Configuration control and build (CI)

• Estimation

• Test strategy and plans

• Agile projects facing non-agile environments (e.g. interfacing systems with

decided release plans for the coming year) is high risk – must be solved

through governance

25.09.2014 • © PROMIS AS 82

Public sector specifics?

• Wide variety of organizations, management and culture in public sector – not

possible to see the whole sector as one homogeneous group

• The recommendations given here applies equally to private sector projects

25.09.2014 • © PROMIS AS 83

References

• IT Project Professional certification curriculum - http://smidigeprosjekter.no/itpp

• Upcoming book – working title ‘Agile Contracting’

• PS2000 Standard Contract

• The Norwegian Computer Society offer a complete set of standard contracts for

software development and maintenance and management.

• The PS2000 Standard Contract (standard version) is based on iterative

development. An agile version exists with alternative set of annexes, expanded and

adjusted according to agile methodology. The terminology is mainly based on Scrum

• http://www.dataforeningen.no/it-contract-standards.146223.no.html

• Complete english version - PS2000 Agile (PS2000 Agile, Maintenance, Framework,

Service Operations) ~ 1000 USD

• PRINCE2

http://smidigeprosjekter.no/itpp


Large public sector projects – what determines failure or success? Scrum Gathering Atlanta 2012

Software

Transcript of Large public sector projects – what determines failure or success? Scrum Gathering Atlanta 2012