Good Projects Gone Bad: an Introduction to Process Maturity

Good Project Gone Bad: Planning, Managing and Delivering Complex

Technology Projects

1. Initial

2. Managed

3. Defined

4. Quantitatively Managed

5. Optimizing


Technology Projects

Michael EdsonDirector, Web and New Media Strategy

Smithsonian Institution

Nik HoneysettHead of AdministrationJ. Paul Getty Museum

AAM Annual ConferenceMonday, Apr 28, 2008

9:00 AM-10:15 AM

Introduction

NoteSee related text version of this show

with links and footnotes

“a large but unknowable proportion ofbusinesses fail pursuing

nearly perfect strategies.”

-- Peter JonesWe Tried to Warn YouBoxes and Arrows

“Good companies tell stories of success, but great companies also tell stories of past failures to avoid repeating them.”

– Christian StadlerThe 4 Principles of Enduring SuccessHarvard Business Review

The World Has Changed

“Twenty years from now we’ll look back and say this was the embryonic period.

The Web is only going to get more revolutionary”

--Tim Berners-Lee, 2006

Introduction to Process Maturity

Good Project Gone Bad: Planning, Managing and Delivering Complex Technology Projects


Smithsonian [email protected]

www.si.edu

Projects in Trouble

“As systems become increasingly complex, successful software development becomes increasingly difficult. Most major system developments are fraught with cost, schedule, and performance shortfalls. We have repeatedly reported on costs rising by millions of dollars, schedule delays of not months but years, and multibillion-dollar systems that don’t perform as envisioned.”--US General Services Administration, 1992

AAM Accreditation

Capability Maturity Model1. Initial – Processes, if they are defined at all, are ad hoc.

Successes depend on individual heroics and are generally not repeatable.

2. Managed – Basic project management practices are established and the discipline is in place to repeat earlier successes with similar projects.

3. Defined – Processes are documented and standardized and all projects use approved, tailored versions of the standard processes.

4. Quantitatively Managed – The performance of processes and the quality of end-products are managed with quantitative measurement and analysis.

5. Optimizing – Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas.

Capability Maturity Model

1. Initial

2. Managed

3. Defined


5. Optimizing

Understanding the levels

Level 1. Level 2. Level 3. Level 4. Level 5.

People Success depends on individual heroics

“Fire fighting” is a way of life

Relationships between disciplines are uncoordinated, perhaps even adversarial

Success depends on individuals

Commitments are understood and managed

People are trained

Project groups work together, perhaps as an integrated team

Training is planned and provided according to roles

Strong sense of teamwork exists within each project

Strong sense of teamwork exists across the organization.

Everyone is involved in process improvement

Processes Few stable processes exist or are used

“Just do it!”

At the individual project level, documented and stable estimating, planning and commitment processes are used

Problems are recognized and corrected as they occur

Integrated management and engineering (how things get built) processes are used across the organization

Problems are anticipated and prevented, or their impacts are minimized

Processes are quantitatively understood and stabilized

Sources of individual problems are understood and eliminated

Processes are continuously and systematically improved.

Common sources of problems are understood and eliminated

Measurement Data collection and analysis are ad hoc Planning and management data used by individual projects

Data are collected and used in all defined processes

Data are systematically shared across projects

Data definition and collection are standardized across the organization

Data are used to understand work processes quantitatively and stabilize them

Data are used to evaluate and select process improvements

Technology Introduction of new technology is risky Technology supports established, stable activities

New technologies are evaluated on a qualitative basis

New technologies are evaluated on a quantitative basis

New technologies are proactively pursued and deployed

Handout (and next slides)


People

Processes

Measurement

Technology

1 2 3 4 5


People

Processes

Measurement

Technology

1 2 3 4 5

Success depends on individual heroics


People

Processes

Measurement

Technology

1 2 3 4 5

“Fire fighting” is a way of life


People

Processes

Measurement

Technology

1 2 3 4 5

Relationships between disciplines are uncoordinated,perhaps even adversarial


People

Processes

Measurement

Technology

1 2 3 4 5

Success depends on individuals

Commitments are understoodand managed

People are trained


People

Processes

Measurement

Technology

1 2 3 4 5

Project groups work together,perhaps as an integrated team

Training is planned and providedaccording to rolesv


People

Processes

Measurement

Technology

1 2 3 4 5

Strong sense of teamworkexists within each project


People

Processes

Measurement

Technology

1 2 3 4 5

Strong sense of teamworkexists across the organization

Everyone is involved inprocess improvement


People

Processes

Measurement

Technology

1 2 3 4 5


People

Processes

Measurement

Technology

1 2 3 4 5

Few stable processes exist or are used

“Just do it!”


People

Processes

Measurement

Technology

1 2 3 4 5

At the individual project level,documented and stableestimating, planning andcommitment processes are used

Problems are recognized andcorrected as they occur


People

Processes

Measurement

Technology

1 2 3 4 5

Integrated management andengineering processes(how things get built)are used across theorganization

Problems are anticipated andprevented, or their impacts areminimized


People

Processes

Measurement

Technology

1 2 3 4 5

Processes are quantitativelyunderstood and stabilized

Sources of individual problems areunderstood and eliminated


People

Processes

Measurement

Technology

1 2 3 4 5

Processes are continuously andsystematically improved

Common sources of problems areunderstood and eliminated

Using the model


Figure outwhere youare?

1. Initial

2. Managed

3. Defined


5. Optimizing


Ratchet upgraduallyover time

1. Initial

2. Managed

3. Defined


5. Optimizing

1. Initial

2. Managed

3. Defined


5. Optimizing


Don’t skip steps


Don’t slip back!

1. Initial

2. Managed

3. Defined


5. Optimizing

1. Initial

2. Managed

3. Defined


5. Optimizing

Capability Maturity ModelPick projectsAppropriateFor yourlevel

Some Practical Ways to increase process maturity

Best Practices

Avoid Classic MistakesRank Estimated

Probability of Occurrence

Classic Mistake Action to Take

1 68% Lack of content or source-code control

Implement source-code control practices

Best Practices




1 68% Lack of content or source-code control

Implement source-code control practices

2 60% Failure to produce a design document

Produce a design, Ex Post Facto, starting week of August 25th

Best Practices




3 60% Lack of project management plan

Project plan v 1.0 completed August 20th

4 60% Failure to maintain project visibility

Project visibility addressed in project plan.

Best Practices




5 60% Feature Creep Produce a design. Prioritize feature set.

6 58% Wasted time upstream

The cow is already out of the barn on this one!

Best Practices




7 57% Reliance on heroics to complete a project

Define roles and responsibilities. Emphasize accurate estimation. Implement management controls to track progress and anticipate delays.

8 53% Friction within team

Address proactively with team members and management.

Best Practices




9 53% Failure to accurately estimate time and resources

Related to lack of design. Having a project management plan should help. Managers must ensure staff accurately defines and estimates tasks.

10 50% Failure to define requirements up front

create requirements doc Ex Post Facto.

Best Practices

Spiral Project Plan

Design

BuildTest

EvaluateThenplannext loop

START

Best Practices

Assign Roles and Responsibilities

Tasks that have clear ownersare more likely to get done

Best Practices


Managerial Roles– Sponsor

• Internal client(s) for whom we’re producing the project. Defines goals. Supervises Project Owner and provides resources and direction to Project Owner and team. Provides “head above the trees” perspective of overall effort.

Best Practices


Managerial Roles– Project Owner

• Responsible for,– high level organization and execution of project.– requirements analysis– creative brief– interface with project sponsors– team selection– high-level definition project lifecycle– monitoring and periodic reviews of content/functionality over

entire project lifecycle– Usually reports to the Project Management Team

Best Practices

Assign Roles and ResponsibilitiesMake sure every role is assigned to a person

Best Practices

Standardized ReportingBest Practices

Governance

PurposeThe purpose of this form is to provide an

overview of proposed objectives and production/maintenance lifecycles for new Web content. This form requires information needed to support the editorial decision-making process. A completed form serves as a contract between project sponsors, team members, and SAAM decision makers.

PurposeThe purpose of this form is to provide an

overview of proposed objectives and production/maintenance lifecycles for new Web content. This form requires information needed to support the editorial decision-making process. A completed form serves as a contract between project sponsors, team members, and SAAM decision makers.

Best Practices

Review process for new sites

Governance

ProcessThis section is written with the project manager/project leader in mind1. Somebody generates an idea and you take ownership of it: you are the project leader.

You discuss the idea with potential partners, team members, and SAAM management. You define a project and walk it through the approval process.

2. You discuss the idea/project at the SAAM Web Weekly, and (optionally) at the SAAM Web Quarterly.

3. If the idea passes through informal discussions you formalize the creative and management aspects of the project and fill out this form.

4. You present the project and this form to the SAAM Web Quarterly and lead a discussion. You can review simple projects via e-mail: more complex projects require a meeting of the Web Quarterly and may require several meetings.

5. The SAAM Web Quarterly approves the idea (or engages you in an iterative process of questions, comments and review) and makes a recommendation to the Director.

6. The Director approves the idea.7. You begin the next stages of planning and execution.From this point on project management is handled at a detailed level by a Project

Management Plan.








Management Plan.

Best Practices


Governance








Management Plan.








Management Plan.

What kinds of projects should use this process?It is hard to describe this categorically. We’ll be using common sense case-by-case.

Best Practices


Governance

Details1. Who will be leading this idea though the approval process?2. Who will be the project sponsor?3. Who will be the project owner?4. What other project “roles” are defined?5. What is the title of the idea?6. Please give an overview of the idea as you would pitch it to the Director and the Web

Quarterly.7. What deadlines are associated with this idea?8. What partners (internal or external) will be involved?9. Please describe the 3-year lifecycle of this idea.10. What staff resources will be required for the 3-year lifecycle?11. What financial resources will be required for the 3-year lifecycle?12. What technological resources will be required for the 3-year lifecycle?



Best Practices


Governance





Who will be leading this idea though the approval process?

Best Practices


Governance





Who will be the project sponsor?

Best Practices


Governance





What staff resources will be required for the 3-year lifecycle?

Best Practices


Governance





& etc…

Best Practices


Consequences and Phenomena

Capability Maturity Mismatch

When you and your vendor have different capability maturity levels… there can be a

disruptive shearing effect on project processes

TEAM

Capability Maturity MismatchVendors say they see• conflicting institutional voices/opinions (client doesn’t

speak with one voice)• adversarial relationships (“I don’t feel like we’re on the

same team”)• wrong people in key positions• unrealistic expectations• content-approval deadlines are not met• undefined decision-making processes• little or no measurement of key performance indicators • insufficient staffing for the task at hand• completed projects are not maintained after delivery

Governance and Control

“Just Enough”

Lightweight Frameworks• Web 2.0 Design Patterns (O’Reilly)

• The Long Tail• Data is the Next Intel Inside• Users Add Value• Network Effects by Default• Some Rights Reserved.• The Perpetual Beta• Cooperate, Don't Control• Software Above the Level of a Single Device

Real World Examples

• Handheld Multimedia Guide

Real World Examples

• Handheld Multimedia Guide• Blog

Real World Examples

• Handheld Multimedia Guide• Blog• Findability project

What you can do• Monday

– Discuss process maturity with your coworkers– Pick a process to measure and improve – Assign responsibility and go for it!

• In 90 days– Look at the CMM levels and agree on two things you can do to

eke your way up a notch– Establish a group to own the process improvement initiatives– Talk with your vendors about what you could do better– Use Roles and Responsibilities, standard reporting, classic

mistake avoidance– Learn about Web 2.0 frameworks– Measure measure measure

The Road to Success

Efficient-Development Town

YOU ARE HERE

Reference: McConnell, Steve Rapid Development, Taming Wild Software Schedules.Microsoft Press, 1996

The Road to Success



The Road to Success

Classic-Mistakes Town



The Road to Success


High-Cost/Long-Schedule Town



The Road to Success



Sometimes-Predictable-Cost-and-Schedule Town



The Road to Success




Predictable-Cost-and-Schedule Town Efficient-Development Town


The Road to Success





Specialization…


The Road to Success





Specialization…

Most organizationsare here…


The Road to Success





Specialization…

To get here, use anyeffective practicewhatsoever… BUT USE IT!


ReferencesCarroll, Sean B. 2005. Endless Forms Most Beautiful: The New Science of Evo Devo. New York : Norton, 2005.CMMI Project Team. 2007. CMMI for Acquisition, Version 1.2: Improving Processes for Acquiring Better Products and Services. Pittsburgh : Software

Engineering Institute, Carnegie Mellon, 2007.—. 2006. CMMI for Development, Version 1.2. Pittsburgh : Software Engineering Institute, Carnegie Mellon University, 2006.Edson, Michael. 2006. Data Access Strategy. Museums and the Web: Conference Proceedings. [Online] 3 1, 2006. [Cited: 4 22, 2008.]

http://www.archimuse.com/mw2006/papers/edson/edson.html.General Accounting Office. 1992. IMTEC-93-13 Mission-Critical Systems: Defense Attempting to Address Major Software Challenges. [Online] 1992. [Cited: 4

22, 2008.] http://archive.gao.gov/d36t11/148399.pdf.Hotle, Matthew. 2007. 'Just Enough Process' for Applications. ID Number: G00145561. s.l. : Gartner, Inc., 2007.—. 2007. The Little Big Application Organization: How a Big Organization Can Still Remain Small. ID Number: G00146962. s.l. : Gartner, Inc., 2007.—. 2008. The 'Seven Deadly Sins' of Application Governance. ID Number: G00155896. s.l. : Gartner, Inc., 2008.Jones, Peter. 2008. We Tried To Warn You, Part 2: Failure is a matter of timing. Boxes and Arrows. [Online] 3 26, 2008. [Cited: 4 22, 2008.]

http://www.boxesandarrows.com/view/we-tried-to-warn-you32.Keen, Andrew. 2007. The Cult of the Amateur: How today's internet is killing our culture. New York : Doubleday, 2007.Kopcho, Joanne and Hottle, Matthew. 2008. CMMI Remains the Standard for Software Process Frameworks. Article ID #G00156315. Gartner.com. [Online]

4 18, 2008. [Cited: 4 22, 2008.] http://gartner.com.Leading Change: Why Transformation Efforts Fail. Kotter, John P. 1995. 1995, Harvard Business Review; Mar/Apr95, Vol. 73 Issue 2, pp. 59 - 67.McConnell, Steve. 1996. Rapid Development: Taming Wild Software Schedules. Redmond : Microsoft Press, 1996.O'Reilly, Tim. 2005. Web 2.0: Design Patterns and Business Models for the Next Generation of Software. O'Reilly.com. [Online] 9 30, 2005. [Cited: 4 24,

2008.] http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html.2007. Universal Music CEO Doug Morris Speaks, Recording Industry in Even Deeper Shit Than We Thought. Aprapos of Nothing. [Online] 11 26, 2007. [Cited:

4 22, 2008.] http://nymag.com/daily/entertainment/2007/11/universal_music_ceo_doug_morris.html accessed 4/19/2008.2007. Universal's CEO Once Called iPod Users Thieves. Now He's Giving Songs Away. Wired.com. [Online] 11 27, 2007. [Cited: 4 22, 2008.]

http://www.wired.com/entertainment/music/magazine/15-12/mf_morris.Vivendi Board Biography of Doug Morris. [Online] [Cited: 4 22, 2008.] http://www.vivendi.com/corp/en/governance/dir_morris.php.


Technology Projects


Smithsonian Institution

Nik HoneysettHead of AdministrationJ. Paul Getty Museum

AAM Annual ConferenceMonday, Apr 28, 2008

9:00 AM-10:15 AM

Good Projects Gone Bad: an Introduction to Process Maturity

Technology

Transcript of Good Projects Gone Bad: an Introduction to Process Maturity