September 2008 CMMI Performance ResultsCMMI ® v1.1/V1.2 – SCAMPI v1.1/V1.2 Class A Appraisal...

Software Engineering InstituteCarnegie Mellon

CMMI® v1.1/V1.2 – SCAMPI v1.1/V1.2 Class A Appraisal ResultsSoftware Engineering InstituteCarnegie Mellon

Process Maturity Profile

CMMI®

SCAMPISM Class A Appraisal Results 2008 Mid-Year Update

September 2008

September 2008 Process Maturity Profile - © 2008 by Carnegie Mellon University

September 2008

We could not produce this report

without the support of the organizations

and lead appraisers who reported their

appraisal results to the SEISM.

Our many thanks for their continuing

cooperation with our data collection

and analysis efforts.

CMMI Appraisal Program

The Software Engineering Institute is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon® University

® CMMI, Capability Maturity Model and Carnegie Mellon are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University

SM SCAMPI and SEI are service marks of Carnegie Mellon University

© 2008 by Carnegie Mellon University


CMMI® v1.1/V1.2 – SCAMPI v1.1/V1.2 Class A Appraisal Results

Outline

Introduction

Current Status

Community Trends

Organizational Trends


Organizational Trends

Summary And Terms used in this Briefing

How to Report your Appraisal Results to the SEI

This briefing is based solely on the appraisals reported to the SEI. It cannot be used as a source for an exact indicator of all organizations in the world using CMMI models or appraisal methods nor can it be used for certification or verification purposes.

The SEI does not certify appraisal results or organizations. The SEI licenses SEI Partners, authorizes lead appraisers, team leaders and certified high maturity lead appraisers to conduct appraisals. Neither the SEI nor any other organization is a “certifying authority” of the results from a SCAMPI appraisal. Therefore requests to the SEI to provide information to be used for this purpose can not be fulfilled. Information provided by the SEI, such as this briefing, is to demonstrate reported results of the use of CMMI products and services only.

Organizations performing source selection or verification should consider performing an evaluation appraisal. For more information on appraisal methods and for a directory of authorized lead appraisers who can perform them, visit: http://www.sei.cmu.edu/appraisal-program/



Introduction: Purpose

Characterize the adoption of CMMI® ®

Describe results from Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) v1.1/v1.2 Class A appraisals using Capability Maturity Model® Integration(CMMI) v1.1 /v1.2 *

Encourage continued reporting of results


* Organizations previously appraised against CMMI v1.0 and which have not reappraised against v 1.1 are not included in this report

Please visit: http://www.sei.cmu.edu/appraisal-program/profile/index.html for additional information or questions you may have about this briefing before contacting the SEI directly



This briefing includes three primary sections:

1. Current Status- Snapshot of the software community based on the most

recent appraisal, since 2002, of reporting organizations

2. Community Trends- Global distribution of appraisals- Growth in the number of appraisals performed

Introduction - 2: Briefing Contents


- Growth in the number of appraisals performed - Shifts in the maturity profile over time

3. Organizational Trends- Analysis of Process Area (PA) satisfaction - Time to move up in maturity based on change in maturity

level and time between appraisals



June 2008

Section 1: Current Status

SCAMPI v1.1/v1.2 Class A appraisals conducted from the SCAMPI V1.1 April 2002 release through andreported to the SEI by

• appraisals• organizations

July 2008

3,5533,009


• organizations• participating companies• reappraised organizations• projects• Non-USA organizations

Please refer to: Terms Used in this Report on page 30

Additional charts providing different views and break down of this information will be added to this briefing

as more appraisal data is reported to the SEI over time.

2,168

451

17,398

69.9%

3,009


CMMI® v1.1/V1.2 – SCAMPI v1.1/V1.2 Class A Appraisal ResultsN

um

ber

Of

Org

an

izati

on

s

1000

1200

1400

1600

1800

Process Maturity Profile by All Reporting Organizations

32.2%

44.4%


Based on most recent appraisal of organizations

Nu

mb

er

Of

Org

an

izati

on

s

0

200

400

600

800

1000

Not Given Initial Managed Defined Quantitatively Managed

Optimizing

3009

7.9%

1.3%3.0%

11.2%



500

600

700

800

Nu

mb

er

Of

Org

an

izati

on

s

Process Maturity Profile (V1.2 Only)by All Reporting Organizations

53.1%


Based on most recent appraisal of organizations958

0

100

200

300

400


Optimizing

Nu

mb

er

Of

Org

an

izati

on

s

6.2%

1.1%

32.0%

0.7%

6.8%



Reporting Organization Categories

Contractor for

Commercial/In-house

22.1%

73.0%


Based on organizations reporting an organization category

0 500 1000 1500 2000 2500 3000

Military/Government Agency

Contractor for Military/Government

Number of Organizations

22.1%

4.9%

2983


CMMI® v1.1/V1.2 – SCAMPI v1.1/V1.2 Class A Appraisal Results%

Of

Org

an

izati

on

s W

ith

in C

ate

go

ry

Process Maturity Profile by Reporting Organization Categories

60%

70%

80%

90%

100%

Commercial/In-house = 2178

Contractor for Military/Government = 658

Military/Government Agency = 147


Based on most recent appraisal of organizations reporting an organization category

% O

f O

rgan

izati

on

s W

ith

in C

ate

go

ry

6.5%

1.0%

31.3%

46.3%

3.5%

11.4%9.1%

2.0%

32.8%

42.1%

1.8%

12.2%

23.1%

2.0%

45.6%

23.8%

0.7%

4.8%

0%

10%

20%

30%

40%

50%


Optimizing

2983



Retail Trade0.3%

Wholesale Trade0.2%

Transportation, Communication, Electric, Gas

and Sanitary Services3.0%

Finance, Insurance and Real Estate6.0%

Public Administration (Including Defense)

4.3%

Fabricated Metal Products0.2%

Primary Metal Industries0.4%

Industrial Machinery And Equipment

Business Services34.8%

Organization TypeBased on Primary Standard Industrial Classification (SIC) Code


Equipment1.1%

Electronic & Other Electric Equipment

10.2%

Instruments And Related Products

1.5%

Transportation Equipment3.6%

Other Manufacturing Industries

0.9%

Health Services1.6%

Other Services8.2%

Engineering & Management Services23.8%

Based on organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html2165

Manufacturing

17.9%

Services

68.3%



25 or fewer13.7%

201 to 300

301 to 5007.4%

501 to 10006.4%

1001 to 20003.4%

2000+2.2%

Organization SizeBased on the total number of employees within the area of the organization that was appraised

201 to 2000+

27.9%


26 to 5016.9%

51 to 7513.2%

76 to 1008.6%

101 to 20019.7%

201 to 3008.5%

Based on organizations reporting size data2968

1 to 100

52.4%

27.9%



Maturity Profile by Organization Size Based on the total number of employees within the area of the organization that was appraised

% o

f O

rgan

izati

on

s b

y S

ize

64

.8%

52

.8%

54

.5%

48

.9%

60%

70%

80%

90%

100%


Based on organizations reporting size data

% o

f O

rgan

izati

on

s b

y S

ize

8.1

%

7.0

%

6.6

% 8.6

%

9.2

%

7.5

%

5.4

% 7.9

% 11

.8%

7.8

%

1.7

%

1.0

%

1.0

%

1.2

%

2.8

%

0.5

%

1.6

%

2.9

%

42

.6%

31

.6%

25

.1%

26

.2%

24

.6%

19

.5%

11

.1%

5.9

%

4.7

%

23

.4%

47

.0%

52

.8%

54

.5%

48

.4%

44

.4% 4

8.9

%

39

.7%

38

.2%

46

.9%

0.5

%

0.6

%

2.6

%

3.1

%

3.9

% 6.0

%

5.0

%

5.8

%

2.0

%

4.7

%

1.5

%

1.8

% 5.4

% 8.6

% 11

.1% 14

.7%

20

.8%

33

.9% 3

9.2

%

35

.9%

0%

10%

20%

30%

40%

50%

25 or fewer 26 to 50 51 to 75 76 to 100 101 to 200 201 to 300 301 to 500 501 to 1000 1001 to 2000 2000+

Not Given Initial Managed Defined Quantitatively Managed Optimizing

2968



800

1000

1200

1400

1600

Disciplines Selected for Appraisals (V1.1 Only)

49.2%

SE = System Engineering SW = Software EngineeringIPPD = Integrated Product and

Process Development SS = Supplier Sourcing

Nu

mb

er

of

Ap

pra

isals

49.2%

40.2%


0

200

400

600

800

SE

/SW

SW

SE

/SW

/IPP

D/S

S

SE

/SW

/SS

SE

SE

/SW

/IPP

D

SW

/SS

SW

/IPP

D

SE

/SS

SW

/IPP

D/S

S

SE

/IPP

D/S

S

Based on appraisals reporting discipline data

Nu

mb

er

of

Ap

pra

isals

For more information about Allowable Models & Combinations, visit: http://www.sei.cmu.edu/cmmi/background/aspec.html

2.8% 2.8% 2.0% 1.4% 0.7% 0.7% 0.1% 0.1% 0.0%

2,576



Contractor for

Commercial/In-house

USA and Non-USAReporting Organization Categories

288

1890

513


Based on USA organizations and Non-USA organizations reporting an organization category

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Military/Government Agency

Contractor for Military/Government

% of Organizations

USA: 100% = 901

Non-USA: 100% = 2082

2082905

513

145

100

47



60%

70%

80%

90%

100%

USA: 100 % = 905

Non-USA: 100 % = 2104

% o

f O

rgan

izati

on

s

Maturity Profile by All Reporting USA and Non-USA Organizations


Based on USA organizations and Non-USA organizations

0%

10%

20%

30%

40%

50%


Optimizing

% o

f O

rgan

izati

on

s

905 2104

125

113

13 24

344

625

327

1010

10

80

85

252



Appraisals conducted through and reported to the SEI

• 63 Countries reported appraisals- most of the appraisals are from Asia and North America

• Appraisals were posted on PARS

June 2008

Section 2: Community Trends

2583


• Appraisals were posted on PARS- the expired ones were removed and the reappraised were

replaced by the newest appraisals- the posting rate is increasing and it is 88% for 2007 and

it is 93 % for 2008


2583



Countries where Appraisals have been Performed and Reported to the SEI


Red country name: New additions with this reporting

Argentina Australia Austria Bahrain Bangladesh Belarus Belgium Brazil

Bulgaria Canada Chile China Colombia Costa Rica Czech Republic Denmark

Dominican Republic Egypt Finland France Germany Hong Kong Hungary India

Indonesia Ireland Israel Italy Japan Korea, Republic Of Latvia Lithuania

Luxembourg Malaysia Mauritius Mexico Morocco Netherlands New Zealand Norway

Pakistan Peru Philippines Poland Portugal Romania Russia Saudi Arabia

Singapore Slovakia South Africa Spain Sweden Switzerland Taiwan Thailand

Turkey Ukraine United Arab Emirates United Kingdom United States Uruguay Viet Nam



Number of Appraisals Reported to the SEI by Continent

Nu

mb

er

of

Ap

pra

isals

1593

11871200

1400

1600

1800

2000


Based on appraisals

Nu

mb

er

of

Ap

pra

isals

3553

42

457

31

243

0

200

400

600

800

1000

Africa Asia Europe North America Oceania South America



Number of Appraisals and Maturity LevelsReported to the SEI by Country

Country

Number of

Appraisals

Maturity

Level 1

Reported

Maturity

Level 2

Reported

Maturity

Level 3

Reported

Maturity

Level 4

Reported

Maturity

Level 5

Reported Country

Number of

Appraisals

Maturity

Level 1

Reported

Maturity

Level 2

Reported

Maturity

Level 3

Reported

Maturity

Level 4

Reported

Maturity

Level 5

Reported

Argentina 57 40 10 2 3 Luxembourg 10 or fewer

Australia 30 1 8 5 2 4 Malaysia 50 19 26 5

Austria 10 or fewer Mauritius 10 or fewer

Bahrain 10 or fewer Mexico 45 1 20 17 3 4

Bangladesh 10 or fewer Morocco 10 or fewer

Belarus 10 or fewer Netherlands 10 or fewer

Belgium 10 or fewer New Zealand 10 or fewer

Brazil 88 41 35 1 8 Norway 10 or fewer

Bulgaria 10 or fewer Pakistan 24 1 17 4 1

Canada 47 1 11 20 5 3 Peru 10 or fewer

Chile 27 17 8 1 Philippines 20 2 10 7


Chile 27 17 8 1 Philippines 20 2 10 7

China 585 1 111 395 22 39 Poland 10 or fewer

Colombia 19 6 10 1 2 Portugal 10 or fewer

Costa Rica 10 or fewer Romania 10 or fewer

Czech Republic 10 or fewer Russia 10 or fewer

Denmark 10 or fewer Saudi Arabia 10 or fewer

Dominican Republic 10 or fewer Singapore 16 3 8 1 4

Egypt 31 1 14 11 2 2 Slovakia 10 or fewer

Finland 10 or fewer South Africa 10 or fewer

France 125 4 75 38 1 2 Spain 85 1 53 25 1 3

Germany 58 7 30 9 1 1 Sweden 10 or fewer

Hong Kong 15 2 9 4 Switzerland 10 or fewer

Hungary 10 or fewer Taiwan 111 1 72 35 2

India 362 1 13 156 22 158 Thailand 13 7 4 1

Indonesia 10 or fewer Turkey 11 9 2

Ireland 10 or fewer Ukraine 10 or fewer

Israel 16 3 9 2 United Arab Emirates10 or fewer

Italy 19 9 9 United Kingdom 79 3 39 26 1 2

Japan 241 17 69 103 13 15 United States 1136 26 404 394 21 118

Korea, Republic Of 112 1 35 49 11 7 Uruguay 10 or fewer

Latvia 10 or fewer Viet Nam 10 or fewer

Lithuania 10 or fewer



911

800

1000

1200

TOTAL

POSTED ON PARs

Number of SCAMPI A Appraisals Reported to SEI by Year

Nu

mb

er

of

Ap

pra

isals

839

1028


76

214

371

621

384

0

200

400

600

2002 2003 2004 2005 2006 2007 2008

Based on appraisals

Nu

mb

er

of

Ap

pra

isals

52

186

437

599

3553

410



Number of SCAMPI A Appraisals Reported to SEI by Year

Nu

mb

er

of

Ap

pra

isals 834

1006

730

848109

180

800

1000

1200

First Appraisal

Reappraisal


Based on appraisals

Nu

mb

er

of

Ap

pra

isals

52

186

437

598

51

163

394

504

730

319

1

23

43

95

93

0

200

400

600

2002 2003 2004 2005 2006 2007 2008

3553



1112

1842

2690

3009

60%

70%

80%

90%

100%

2002-2005

2006

2007

2008

Trends in the Community Maturity Profile

% o

f O

rgan

izati

on

s

Year Orgs


8.4

%

2.5

%

33.0

%

33.1

%

4.4

%

18.6

%

8.0

%

1.7

%

32.4

%

37.5

%

4.5

%

16.0

%

7.7

%

1.6

%

33.0

%

41.6

%

3.4

%

12.7

%

7.3

%

1.4

%

32.4

%

44.3

%

3.1

%

11.6

%

0%

10%

20%

30%

40%

50%

Not Given Initial Managed Defined Quantitatively ManagedOptimizing

Based on a cumulative view of the most recent appraisals of organizations up through the year indicated

% o

f O

rgan

izati

on

s



Section 3: Organizational Trends

Appraisals conducted through and reported to the SEI

• Process Area (PA) profiles- satisfaction of PAs by maturity level for organizations

appraised at levels 1 and 2

• reappraised organizations

June 2008

451

1,227


• reappraised organizations- accounting for appraisals- although some organizations conducted multiple

reappraisals, only the first and latest appraisals were used in creating the charts on pages 26


451995



OPD

OT

IPM

RSKM

ISM

DAR

OPP

QPM

OID

CAR

Fully Satisfied

Rated

Process Area Profiles - 1Organizations Appraised at Maturity Level 1

Defined

QuantitativelyManaged

Optimizing


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

REQM

PP

PMC

SAM

MA

PPQA

CM

RD

TS

PI

VER

VAL

OPF

Based on appraisals reporting a maturity level 1 rating

% of Appraisals

Managed

70



RSKM

IT

ISM

DAR

OEI

QPM

OPP

Fully Satisfied

Rated

Quantitatively Managed

Process Area Profiles - 2Organizations Appraised at Maturity Level 2


Based on appraisals reporting a maturity level 2 rating

0% 5% 10% 15% 20%

RD

TS

PI

Ver

Val

OPF

OPD

OT

IPM

Defined

% of Appraisals

1157



60%

70%

80%

90%

100%

First

Latest

Maturity Level ofFirst and Latest Appraisal

Org

an

izati

on

s


11.1

%

6.9

%

38.6

%

27.3

%

4.0

%

12.2

%

7.5

%

0.4

%

8.4

%

47.2

%

4.7

%

31.7

%

0%

10%

20%

30%

40%

50%


Optimizing

Based on reappraised organizations using their first and latest appraisal

% o

f O

rgan

izati

on

s

451



Time to Move Up

Number of months to move to nextmaturity level

100

50

75


Largest observed value that is not an outlier

75th Percentile

25th PercentileMedian

Smallest observed value that is not an outlier

30

18

0

50

All (2002 to Present)

1 to 2 2 to 3 3 to 4 4 to 515 154 28 24

2006 to Present

1 to 2 2 to 3 3 to 4 4 to 53 55 6 8

19

Time Period of Initial Appraisal

LevelOrgs

13

17 15.512.5

2002 to 2005

1 to 2 2 to 3 3 to 4 4 to 512 99 22 16

19

5

16

4

13

9

4



Summary - 1

3553 appraisals have been reported to the SEI in 76 months

The Organization Category of “Commercial/In-house” reports most of the appraisals

The highest percentage of “Commercial/In-house” reporting appraisals is from outside the USA


The highest percentage of “Military/Government Agency” reporting appraisals is from the USA

Outside the USA: China, India, Spain, Argentina, Brazil and Malaysia reported appraisals are increasing at a rapid rate



Summary - 2

For organizations that began their CMMI-based SCAMPI effort in

2002 or later, the median time to move from:

• maturity level 1 to 2 is 4 months

• maturity level 2 to 3 is 17 months

• maturity level 3 to 4 is 15.5 months






Terms Used in this Report

Company - Parent of the appraised entity

A company can be a commercial or non-commercial firm, for-profit

or not for-profit business, a research and development unit, a higher

education unit, a government agency, or branch of service, etc.

Organization - a.k.a. Appraised entity

The organization unit to which the appraisal results apply. An

appraised entity can be the entire company, a selected business


appraised entity can be the entire company, a selected business

unit, units supporting a particular product line or service, etc.

Non-USA - Appraised entity whose geographic location is not within the United

States. The parent of the appraised entity may or may not be based

within the United States.



Report your Appraisal Results to the SEI

The briefing is only possible due to the cooperation of organizations and individuals sending in their appraisal results to the SEI

In order to provide this information and service in the future, it will depend on this continued cooperation


Please visit:

http://www.sei.cmu.edu/appraisal-program/profile/index.html

for forms, information, and instructions on how to report appraisals to the SEI



Contact Information

Please visit:

http://www.sei.cmu.edu/appraisal-program/profile/index.html

and review the information provided before contacting:

SEI Customer Relations (412) 268-5800SEI FAX number (412) 268-5758


SEI FAX number (412) 268-5758

Internet [email protected]

Mailing AddressCustomer RelationsSoftware Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213-3890

September 2008 CMMI Performance ResultsCMMI ® v1.1/V1.2 – SCAMPI v1.1/V1.2 Class A Appraisal...

Documents

Transcript of September 2008 CMMI Performance ResultsCMMI ® v1.1/V1.2 – SCAMPI v1.1/V1.2 Class A Appraisal...