COSATMO/COSYSMO Workshop

University of Southern California

Center for Systems and Software Engineering

COSATMO/COSYSMO Workshop

Jim Alstad, USC-CSSE

Gan Wang, BAE Systems

Garry J Roedler, Lockheed Martin

April 30, 2014

04/29 1



Workshop Agenda

04/29 2

• Reminder: general directions for COSYSMO 3.0 development

• Delphi for SEFR and SEWR category parameter values– Overview of SEFR and SEWR– Instructions for Delphi questionnaires

• Discussion of extending COSYSMO to estimate development costs



COSATMO/COSYSMO 3.0 Direction• Several factors affecting the COSYSMO cost model

have been shown to be valuable in increasing estimation accuracy:– Reuse (simple model—SEWR)– Reuse (with SEFR)– Requirements volatility (SERV)The rating scales for these could be integrated into a comprehensive COSYSMO model.– Which should provide more accurate estimates than any of

these alone• Add additional data points exhibiting a range of

values for SEWR, SEFR, SERV• Fit a COSYSMO III model to the overall dataset

– Add variables and/or subset the data as needed

04/28 3



COSATMO/COSYSMO Extension to Total Development Costs

• Explore a model for total development cost based primarily on the COSYSMO parameters (Roedler)– Can such a model be improved by dividing development

cost into three parts: system engineering, hardware engineering, software engineering? (Alstad)

04/28 4



Workshop Agenda

04/29 5






What is this Delphi all about?• A Generalized Reuse Framework

implemented in COSYSMO– To assess not only the effort to leverage but

also the effort to invest in a development

project

• The goal is an industry-wide model for

design sensitivity analysis and product

line investment decisions– To bring back to organization to further

tailor or use

• Invite your participation to calibrate the

model coefficients, through a Delphi

• The Delphi process today…6



System Engineering For Reuse (SEFR)

• Producer’s View• Production of

reusable resources

System Engineering With

Reuse (SEWR)• Consumer’s View• Consumption of

reusable resources

Two Fundamental Reuse Processes

7



CONOPS – Reuse in a Project

% E

ffort

# of Articles in the Product Line

1 2 3 4

DFR

DWRDWR

DFR

DWR

DFR

DWR

DFR

100

0

Total Effort

Investments in Development for Reuse (DFR) are leveraged to reduce Product Line Cost

8

Total Project Effort = SEWR Effort + SEFR Effort



Generalized Reuse Framework:System Engineering for Reuse

• System Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states:– Conceptualized for Reuse (e.g., Concept of Operations

document)– Designed for Reuse (e.g., component detailed design)– Constructed for Reuse (e.g., integrated component)– Validated for Reuse (e.g., validated component)

04/29 9



Generalized Reuse Framework:System Engineering for Reuse

04/29 10



Generalized Reuse Framework:System Engineering with Reuse

• System Engineering with Reuse is project System Engineering, with reusable artifacts being brought into the product– A special case: zero reusable artifacts

• Each reusable artifact is included in one of these SEWR states of maturity:– New (i.e., not reused)– Re-implemented (through requirements & architecture)– Adapted (through detailed design)– Adopted (through implementation)– Managed (through system verification & validation)

04/29 11



Generalized Reuse Framework:System Engineering with Reuse

04/29 12



COSYSMO – Context

• COSYSMO– COnstructive SYStems Engineering Cost

MOdel – Parametric Estimate of the Systems

Engineering Effort– Covers full systems engineering lifecycle– Originally developed by Dr. Ricardo Valerdi

and Dr. Barry Boehm at USC

• Inception of COSYSMO 1.0– Valerdi, R., The Constructive Systems Engineering Cost Model (COSYSMO),

PhD Dissertation, University of Southern California, May 2005.

• Introduced the Reuse Model Extension to COSYSMO 2.0– Wang, G., Valerdi, R., Ankrum, A., Millar, C., and Roedler, G., “COSYSMO

Reuse Extension,” Proceedings of the 18th INCOSE International

Symposium, June 2008.– Fortune, J. Estimating Systems Engineering Reuse with the Constructive

Systems Engineering Cost Model (COSYSMO 2.0). Ph.D. Dissertation.

University of Southern California. December 2009– Wang, G., Valerdi, R., Fortune, J., “Reuse in Systems Engineering,” IEEE

System Journal, v4, No.3, 2010.

• Marching to COSYSMO 3.0 (work in progress…)– Fortune, J. and Valerdi, R., “Considerations for Successful Reuse in Systems

Engineering,” AIAA Space 2008, San Diego, CA, September 2008.– Wang, G. and Rice, J., “Considerations for a Generalized Reuse Framework

for System Development,” Proceedings of the 21st INCOSE International

Symposium, June 2011.– Peña, M. Quantifying the Impact of Requirements Volatility on Systems

Engineering Effort. Ph.D. Dissertation. University of Southern California.

August 2012.– Fortune, J. and Valerdi, R., “A Framework for Systems Engineering Reuse,”

Systems Engineering, 16(2), 2013.

13



COSYSMO Overview (1/2)• COSYSMO starts by computing the “size” of a

system engineering project, in units of eReq (“equivalent nominal requirements”)

• These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios.

• Each artifact is evaluated as being easy, nominal, or difficult.

• Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size:

04/29 14

Artifact Type Easy Nominal Difficult

System Req’ts 0.5 1.0 5.0

System Interfaces 1.1 2.8 6.3

System Algs 2.2 4.1 11.5

Op Scenarios 6.2 14.4 30.0



COSYSMO Overview (2/2)

• Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant.

• This results in the following equation for a COSYSMO estimate of effort in person-months:

04/29 15



Generalized Reuse Framework:SEFR Model Equations

• A SEFR estimate adjusts each artifact’s size contribution by considering its SEFR state according to this table (from the first Delphi):

04/29 16

SEFR State (Degree of Development) SEFR State Factor

Conceptualized for Reuse 36.98%

Designed for Reuse 58.02%

Constructed for Reuse 79.15%

Validated for Reuse 94.74%



Generalized Reuse Framework:SEWR Model Equations

• A SEWR estimate adjusts each artifact’s size contribution by considering its SEWR state according to this table (from the first Delphi):

04/29 17

SEWR State (Maturity) SEWR State Factor

New 100.00%

Re-Implemented 66.73%

Adapted 56.27%

Adopted 38.80%

Managed 21.70%



The Exercise – Mapping Activities to Reuse Categories

• Format of Questionnaire Spreadsheet: – 1st tab: definition of reuse

categories– 2nd tab: DWR questionnaire– 3rd tab: DFR questionnaire– 4th tab: definition of EIA-632

activities

• You do:– Fill out the cells intersecting EIA-

632 activities (rows) and the reuse categories (columns)

– Enter the percentage you believe that particular activity applies or is relevant to a particular reuse category

– Fill in a few words of your rationale for each selection

18

Reusable Artifacts Feeding into the DWR Process

==>

Syste

m C

once

pt

Syste

m D

efini

tion

Realiz

ed S

yste

m

Deplo

yed

Syste

m

Syste

m D

efini

tion

Realiz

ed S

yste

m

Deplo

yed

Syste

m

Realiz

ed S

yste

m

Deplo

yed

Syste

m

Realiz

ed S

yste

m

Deplo

yed

Syste

m

Realiz

ed S

yste

m

Deplo

yed

Syste

m

Deplo

yed

Syste

m

New Modified (Implemented) Adapted Deleted Adopted (Integrated)EIA/ANSI 632 Process EIA/ANSI 632 Task Rationale Rationale Rationale Rationale Rationale Managed Rationale

1. Product Supply 100% 100% 100% 100%

2. Product Acquisition 100% 100% 100% 100%

3. Supplier Performance 100% 100% 100% 100%

4. Process Implementation Strategy 100% 100% 100% 100%

5. Technical Effort Definition 100% 100% 100% 100%

6. Schedule and Organization 100% 100% 100% 100%

7. Technical Plans 100% 100% 100% 100%

8. Work Directives 100% 100% 100% 100%

9. Progress Against Plans and Schedules 100% 100% 100% 100%

10. Progress Against Requirements 100% 100% 100% 100%

11. Technical Reviews 100% 100% 100% 100%

12. Outcomes Management 100% 100% 100% 100%

13. Information Dissemination 100% 100% 100% 100%

14. Acquirer Requirements 100% 100% 100% 100%

15. Other Stakeholder Requirements 100% 100% 100% 100%

16. System Technical Requirements 100% 100% 100% 100%

17. Logical Solution Representations 100% 100% 100% 100%

18. Physical Solution Representations 100% 100% 100% 100%

19. Specified Requirements 100% 100% 100% 100%

20. Implementation 100% 100% 100% 100%

21. Transition to Use 100% 100% 100% 100%

22. Effectiveness Analysis 100% 100% 100% 100%

23. Tradeoff Analysis 100% 100% 100% 100%

24. Risk Analysis 100% 100% 100% 100%

25. Requirements Statements Validation 100% 100% 100% 100%

26. Acquirer Requirements Validation 100% 100% 100% 100%

27. Other Stakeholder Requirements Validation 100% 100% 100% 100%

28. System Technical Requirements Validation 100% 100% 100% 100%

29. Logical Solution Representations Validation 100% 100% 100% 100%

30. Design Solution Verification 100% 100% 100% 100%

31. End Product Verification 100% 100% 100% 100%

32. Enabling Product Readiness 100% 100% 100% 100%

33. End Products Validation 100% 100% 100% 100%

Technical Evaluation

New

Acquisition and Supply

Technical Management

System Design

Modified (Implemented) Adapted Deleted Adopted (Integrated)

Product Realization

X



Helpful Hints• When evaluate the questionnaire, frame your thinking around the

COSYSMO construct, i.e., a group of– Requirements

– Interfaces

– Algorithms

– Scenarios

• Based on your overall professional experience, not a

single or a particular program experience– Seek nominal behavior, biased toward desirable/more

successful reuse situations• The handouts have the definitions of reuse categories and EIA-632

activities

• For definition of general (engineering) terms, use INCOSE SE Handbook

19



Questions and Comments on Delphi

Contacts:

Jim AlstadUSC CSSE

Los Angeles, [email protected]

Gan WangBAE Systems

Reston, [email protected]

Garry J. RoedlerLockheed MartinPhiladelphia, PA

[email protected]

20

mailto:[email protected]





Workshop Agenda

04/29 21






Extending COSYSMO To Estimate Development Cost

• The general goal: Use a model that is very similar to COSYSMO to estimate development cost

• There is some evidence that system engineering cost has some correlation with development cost

• Where to go from here?– Especially, who has data from actual projects that they are

willing to contribute?

04/29 22

COSATMO/COSYSMO Workshop

Documents

Transcript of COSATMO/COSYSMO Workshop