System of Systems Engineering (SoSE) Cost Estimation Jo Ann Lane jolane at usc.edu Presented by...

System of SystemsEngineering (SoSE) Cost Estimation

Jo Ann Lanejolane at usc.edu

Presented by Marilee Wheaton

November 2010

2

Overview

Key definitionsCurrent SE cost model capabilitiesExtensions for complex systemsExtensions for SoSsAlternatives for “# of requirements” size driverUsing SoSE cost model to evaluate alternativesSummary

3

Net-Centric SoS Net-CentricConnectivity

What is a “System of Systems”?

Very large systems using a framework or architecture to integrate constituent systems Exhibits emergent behavior not otherwise achievable by constituent systemsSoS constituent systems (CS)

• Independently developed and managed• New or existing systems in various stages• May include multiple COTS products• Have their own purpose• Can dynamically come and go from SoS

Typical domains• Business: Enterprise-wide and cross-enterprise

integrations• Military/Crisis Response: Dynamic

communications infrastructure

Based on Mark Maier’s SoS definition [Maier, 1998]

Laboratory System

Imaging Management System

PharmacySystem

PatientManagementSystem

TelemetrySystem

Health Care Network

SoS SE Taxonomy in Order of Increasing Authority and Responsibility

Virtual [Maier, 1998]• Ad hoc

Collaborative [Maier, 1998]• No formal management at SoS level

Acknowledged [Dahmann, 2008]• SoS SE team to guide SoS SE efforts

Directed [Maier, 2008]• SoS SE efforts managed through formal

mechanisms

Net- Centric SoS Net-CentricConnectivity

Laboratory System

Imaging Management System

PharmacySystem

PatientManagementSystem

TelemetrySystem

Health Care Network

Internet

Future Combat Systems

5

SourceSelection

● ● ●

ValuationExploration Architecting Develop Operation



OperationDevelop Operation Operation Operation

System A

System B

System C

System x

LCO-typeProposal &Feasibility Info

Candidate Supplier/ Strategic Partner n

● ●●

Candidate Supplier/Strategic Partner 1

SoS-Level ValuationExploration Architecting Develop

FCR1 DCR1

Operation

OCR1

Rebaseline/Adjustment FCR1 OCR2

OCRx1

FCRB DCRB OCRB1

FCRA DCRA

FCRC DCRC OCRC1

OCRx2 OCRx3 OCRx4 OCRx5

OCRC2

OCRB2

OCRA1

Example: SoSE (Directed)

6

SourceSelection?Existing collaborative SoS?

● ● ●





System A

System B

System C

System x

Candidate Supplier/ Strategic Partner n

● ●●

Candidate Supplier/Strategic Partner 1


FCR1 DCR1

Operation

OCR1


OCRx1

FCRB DCRB OCRB1

FCRA DCRA

FCRC DCRC OCRC1


OCRC2

OCRB2

OCRA1

Example: SoSE (Acknowledged)

7

● ● ●





System A

System B

System C

System x


FCR1 DCR1

Operation

OCR1


OCRx1

FCRB DCRB OCRB1

FCRA DCRA

FCRC DCRC OCRC1


OCRC2

OCRB2

OCRA1

Example: SoSE (Collaborative)X

8

Translating capability objectives Translating capability objectives

Translating capability objectives

Addressing new requirements & options

Addressing new requirements & options

Addressingrequirements & solution options

Understanding systems & relationships(includes plans)

Understanding systems & relationships(includes plans)

Understanding systems & relationships

External Environment

Developing, evolving and maintaining SoS design/arch

Developing, evolving and maintaining SoS design/arch

Developing& evolvingSoS architecture

Assessing (actual) performance to capability objectives

Assessing (actual) performance to capability objectives

Assessing performance to capability objectives

Orchestrating upgrades to SoS



Monitoring & assessing changes



Traditional SE and SoSE Activities

Traditional SE (Defense Acquisition Guide [DoD, 2006] View)

SoSE (SoS SE Guidebook View Based onInterviews and Analysis of 18 DoD SoSs in Various Stages)

9

SoSE Compared to Traditional SE Activities: Key Challenges for SoSE

People Challenges• Business model and incentives to encourage working together at the

SoS level• Removing multiple decision making layers • Requiring accountability at the enterprise level

Process Challenges• Determining what to manage and what to leave to the CSs• Doing the necessary tradeoffs at the SoS level • Human-system integration

Technical Challenges• Commonality of data, architecture, and business strategies at the

SoS level• Evolution management • Maturity of technology

10

COCOMO Cost Model Suite Overview*

* Barry Boehm, Ricardo Valerdi, Jo Ann Lane, and Winsor Brown, “COCOMO Suite Methodology and Evolution”, CrossTalk, April 2005.

11

SoSE Cost Model Background

Early attempts to develop a “directed” SoS cost model were not successful

• Seldom start with greenfield development• Not enough directed SoSs to calibrate a cost model

SoS cost model needs• Cost estimation associated with a new capability• Cost tradeoffs to support decisions

• Example: Migrate collaborative SoS to an acknowledged SoS

12

Size Drivers

Cost DriversSEEffort

Calibration

• Number of requirements• Number of interfaces • Number of algorithms• Number of operational

scenarios

• 8 Application factors• 6 Team factors• Schedule driver

COSYSMO

Current Systems Engineering Cost Model *Capabilities

Prediction AccuracyAcademic version

• Single system cost model calibrated with data from multiple organizations:

PRED(30)=75% Local calibration versions

• Anecdotal evidence: PRED(30)=85%

* COSYSMO [Valerdi, 2005]

General Form of General Form of academicacademicCOSYSMO EquationCOSYSMO Equation Effort (person months) = [38.55 * EM * (size)1.06] / 152

where • 38.55 and 1.06 are the academicCOSYSMO calibration factors• EM is computed from cost drivers

13

COSYSMO Limitations for Complex Systems and SoSs

Limitations for complex systems and SoS• Single set of cost drivers for system does not support

definition of multiple components with different characterizations

Additional limitations for SoS• Does not address constituent system oversight effort

at SoS level • Does not address constituent system engineering

contributions to SoSE

14

Modifications for Complex Systems

Additional modifications• The academic calibration constants can be adjusted to provide more

accurate estimates by performing a local calibration• Reuse factors [Wang et al., 2008] can be added for each component

Effort (person months) = 38.55*∑EMi*(parti size/total size)*(total size)1.06/152where

38.55 and 1.06 are the academicCOSYSMO calibration factors

i ranges from 1 to the number of components within the complex system

15

Extensions for SoSs

16

Systems Engineering Requirements Categoriesfor SoSE in an Acknowledged SoS

Requirements related to SoS capabilities• Initially engineered at SoS level by SoSE team with support from

constituent system engineers for those systems impacted by the SoS capability, then allocated to constituent systems for further SE

Non-SoS requirements related to constituent system stakeholder needs• Must be monitored by SoSE team to identify changes that might

adversely impact SoS• Represents on-going volatility at the constituent system level that

is occurring in parallel with SoS capability changes

17

Key SoSE Characteristics Used to Develop COSYSMO SoS Extensions

SoSE sub-model• SoSE oversight of constituents can be characterized by using the appropriate

COSYSMO reuse factor• Other non-traditional SE activities performed by SoSE team can be handled

through COSYSMO cost factors• Two types of requirements (SoS and constituent system non-SoS requirements)

modeled together using different effort multipliers for each set*

Constituent system sub-model• Each constituent system within the SoS is independently owned and managed• Constituent system SE effort to support the SoSE team can be characterized by

including extra design effort for the SoS requirements• Two types of requirements (SoS and constituent system non-SoS requirements)

modeled together using different effort multipliers for each set or component*

* Use of multiple effort multipliers allows one to model the diseconomy of scale as the SoS becomes larger through the integration of components with different characteristics....

18

SoS Effort Calculations

SoSE Effort

SoSE Effort = 38.55*[((SoSCR/SoSTreq)*(SoSTreq)1.06 *EMSoS-CR)+ ((SoSMR/SoSTreq)*(SoSTreq)1.06 * EMSoS-MR)/152]

Where:

Total SoSE requirements = SoS Capability Requirements + SoS “Monitored” Requirements

SoS “monitored” reqs = [∑SE non-SoS requirements being addressed current upgrade cycles for all SoS constituent systems] * “Oversight Factor”

“Oversight Factor” = 5% , 10%, 15% (these values are based on the COSYSMO reuse work and expert judgment from various CSSE affiliates and the SoS SE Guidebook team)

SoS capability effort

Oversight of CSs

19

SoSE Effort Multiplier Example

2.50

20

Example Effort Multiplier for SoSE Monitoring of CS Requirements

0.47

21

SoS Effort Calculations (continued)

Single Constituent System Effort

Total single system reqsw-SoSE = SoS requirements allocated to system + SE reqs in upgrade cycle

Single system SE Effort in an Acknowledged SoS

= 38.55*[1.15*( (SoSCSalloc / CSTreqSoSE)*( CSTreqSoSE)1.06* EMCS-CRwSOSE) +

(CSnonSoS / CSTreqSoSE)*( CSTreqSoSE)1.06* EMCSnonSOS] /152

Computed for each constituent system in the SoS...Approach is recursive:

Can also model each constituent system as a complex system or SoS...


Constituent system upgrade effort

CS “tax” to support SoSE team

22

Total SoS SE Effort

• SoS effort includes • SoS capability effort and • Constituent system non-SoS effort associated with single system

enhancements• To compute SoS capability effort, subtract out the total constituent

system non-SoS effort • Approach incorporates the diseconomy of scale at the constituent

system level associated with the additional SoS capability requirements

SoS effort = SoSE effort + ∑ constituent system efforti

where i ranges from 1 to the number of constituent systems within the SoS

Using Alternative Size Drivers

SoSE Cost Model: Alternative Size Drivers

24

Size Drivers

Cost Factors

Estimated EngineeringEffort

Calibration

• Number of System Requirements• Number of System Interfaces• Number of Algorithms• Number of Operational Scenarios

• People characteristics• Process characteristics• Product characteristics

COSYSMO

SoSE Capability Effort Calculation Using Alternative Size Drivers

Constituent system (CS) effort depends upon • SoS alternative selected• CSi effort depends upon types of changes required for CSi

• New interface(s)/interface change(s)• Internal algorithm change(s)/data conversions

Size driver options• Number of requirements • Number of algorithms• Number of interfaces • Number of operational scenarios

Each size driver characterized with respect to complexityAll size drivers converted to equivalent # of nominal reqs

25

SoS effort = SoSE effort + ∑ constituent system efforti

SoSE Estimation Steps for New Capability

1. Understand/review current CS capabilities2. Identify new capability alternatives3. For each alternative, identify

• CSs that contribute to each alternative• For each contributing CS, changes needed to support alternative

• New interfaces/interface change(s)• Data element/algorithm change(s)• Capability size count(s) and associated complexity of each

4. Conduct alternative tradeoffs and finalize cost estimate for selected alternative

5. Identify CS changes required for desired architecture enhancements

6. Calculate COSYSMO effort multipliers at SoS and CS levels7. Calculate SoSE effort for alternative

26

SoSE Estimation Steps for New Capability: Focus of Discussion

1. Understand/review current CS capabilities2. Identify new capability alternatives3. For each alternative, identify

• CSs that contribute to each alternative• For each contributing CS, changes needed to support alternative

• New interfaces/interface change(s)• Data element/algorithm change(s)• Capability size count(s) and associated complexity of each

4. Conduct alternative tradeoffs and finalize cost estimate for selected alternative

5. Identify CS changes required for desired architecture enhancements

6. Calculate COSYSMO effort multipliers at SoS and CS levels7. Calculate SoSE effort for alternative

27

Primary SoS Core Elements Determining SoSE Size Drivers

28

SoS: A set or arrangement of systems that results when independent and useful systems are integrated into a larger system that delivers unique capabilities

SysML Models for Characterizing SoS/SoS Capabilities

Use cases• Characterize both CS and

SoS capabilities from the different user perspectives

Sequence diagrams• Characterize and analyze

the operational flow for an SoS capability

Object blocks• Characterize each SoS CS

and its capabilities

Interface classes• Describe each CS interface

Input/output entity classes• Express the associated data

attributes of each data item transferred over that interface

• May include units, coordinate system, reference frame, source algorithm, etc.

29

Example SoS: Regional Area Crisis Response SoS (RACRS)

30Command Control Center (CCC) Context Diagram

Mission Scenarios: Use Cases and Sequence Diagrams

CCC Interface Class andEvacuate Area I/O Entities by Actor

Using SoSE Cost Model to Evaluate Alternatives: Collaborative vs. Acknowledged SoS

34

Systems Engineering Requirements Categories

Requirements related to SoS capabilitiesa) Acknowledged SoS: Initially engineered at SoS level by SoSE team with

support from CS engineers for those systems impacted by the SoS capability, then allocated to CSs for further SE

b) Collaborative SoS: Not engineered at the SoS level, but must be engineered fully at the CS level through collaborative efforts with other CS engineers

Non-SoS requirements related to CS stakeholder needs• Must be monitored by SoSE team to identify changes that might

adversely impact SoS• Represents on-going volatility at the CS level that is occurring in

parallel with SoS capability changes

35

System Capability

Effort for a “collaborative”SoS

Effort using an “acknowledged” SoSE teamEquivalent

set of “sea-level” requirements

Conversion to COSYSMO size units

Calculations based on SoS characteristics/size and capability implementation approach using COSYSMO algorithm

Overview of SoSE Comparative Model

36

Summary of Comparative Model Effort Multipliers

EM Value* Modified Cost ParametersSoSE effort 2.50 Requirements understanding (low)

Level of service requirements (high)# of recursive levels in the design (high)Multisite coordination (low)

SoSE monitoring of CS Reqs 0.47 Technology risk (very low)Documentation (very low)Personnel/team capability (high)

Capability SE at CS level with SoSE Support 1.06 Architecture understanding (high)Level of service requirements (high)

Capability SE at CS level without SoSE Support

1.79 Requirements understanding (low)Level of service requirements (high)

SE of non-SoS reqs 0.72 Architecture understanding (high)# of recursive levels in the design (low)

* Default value: 1.0 (all cost parameters set to nominal)

37

SoSE Effort Calculations

SoSE Effort = 38.55*[((SoSCR/SoSTreq)*(SoSTreq)1.06 *EMSoS-CR)+ ((SoSMR/SoSTreq)*(SoSTreq)1.06 * EMSoS-MR)/152]

Where:

Total SoSE requirements = SoS Capability Requirements + SoS “Monitored” Requirements

SoS “monitored” reqs = [∑SE non-SoS requirements being addressed current upgrade cycles for all SoS constituent systems] * “Oversight Factor”

“Oversight Factor” = 5% , 10%, 15% (these values are based on the COSYSMO reuse work and expert judgment from various CSSE affiliates and the SoS SE Guidebook team)


Oversight of CSs

38

Single CS Effort Calculation (Acknowledged)

Total single system reqsw-SoSE = SoS requirements allocated to system + SE reqs in upgrade cycle

Effort = 38.55*[1.15*( (SoSCSalloc / CSTreqSoSE)*( CSTreqSoSE)1.06* EMCS-CRwSOSE) + (CSnonSoS / CSTreqSoSE)*( CSTreqSoSE)1.06* EMCSnonSOS] /152


CS upgrade effort

CS “tax” to support SoSE team

39

Total SoS SE Effort (Acknowledged)

• SoS effort includes • SoS capability effort and • CS non-SoS effort associated with single system enhancements

• Approach incorporates the diseconomy of scale at the CS level associated with the additional SoS capability requirements

SoS effort = SoSE effort + ∑ CS efforti

where i ranges from 1 to the number of CSs within the SoS

40

Single CS Effort Calculation (Collaborative)

Total single system reqsw-SoSE = SoS requirements + SE reqs in upgrade cycle

Effort = 38.55*[( (SoSCR / CSTreqwoSoSE)*( CSTreqwoSoSE)1.06* EMCS-CRnSoSE) + (CSnonSoS / CSTreqwoSoSE)*( CSTreqwoSoSE)1.06* EMCSnonSOS] /152

SoS capability effort w/o SoSE support

CS upgrade effort

41

Range of SoS Complexity Factor Values

SoSE Model Parameter

Description Range of Values

SoS Size Number of constituent systems within the SoS

2-200

SoS Capability Size Number of equivalent nominal requirements as defined by COSYSMO

1-1000

Constituent System Volatility

Number of non-SoS changes being implemented in each constituent system in parallel with SoS capability changes

0-2000

Scope of SoS Capability

Number of constituent systems that must be changed to support capability

One to SoS Size (total number of constituents systems within the SoS)

SoSE Oversight Factor Oversight adjustment factor to capture SoSE effort associated with monitoring constituent system non-SoS changes

5%, 10%, and 15%

42

Model Results

Relative Cost of Collaborative and Acknowledged SoSECapability Affects Half of the Systems

System Volatility = 100 Reqs and SoS Capability = 100 Reqs

-300.00

0.00

300.00

600.00

900.00

1200.00

1500.00

1800.00

0 50 100 150 200 250

Number of Systems

Sav

ing

s (P

erso

n M

on

ths)

OSF 5%

OSF 10%

OSF 15%



-200.00

0.00

200.00

400.00

600.00

800.00

0 50 100 150 200 250

Number of Systems

Sa

vin

gs

(P

ers

on

Mo

nth

s)

OSF 5%

OSF 10%

OSF 15%

Scenario 1 (SoS Size Varies) Scenario 2 (SoS Size Varies)

Each graph shows for each OSF value: (SoSE effort + ∑Acknowledged CSi effort*) – (∑Collaborative CSi effort *)

* CS effort is the sum of the SoS capability effort and the non-SoS requirements effort

0

Cost savings with SoSE Team

Extra cost of SoSE TeamP

erso

n M

on

ths

43

Model Results (continued)

Relative Cost of Collaborative and Acknowledged SoSECapability Affects One-Fourth of the Systems


-400.00

0.00

400.00

800.00

1200.00

1600.00

2000.00

0 50 100 150 200 250

Number of Systems

Sa

vin

gs

(P

ers

on

M

on

ths

) OSF 5%

OSF 10%

OSF 15%



-200.00

-100.00

0.00

100.00

200.00

300.00

400.00

0 50 100 150 200 250

Number of Systems

Sa

vin

gs

(P

ers

on

Mo

nth

s)

OSF 5%

OSF 10%

OSF 15%





-15000.00

-10000.00

-5000.00

0.00

0 50 100 150 200 250

Number of Systems

Sav

ing

s (P

erso

n M

on

ths)

OSF 5%

OSF 10%

OSF 15%

Relative Cost of Collaborative and Acknowledged SoSECapability Affects All of the Systems


-10000.00

-8000.00

-6000.00

-4000.00

-2000.00

0.00

2000.00

0 50 100 150 200 250

Number of Systems

Sav

ing

s (P

erso

n M

on

ths)

OSF 5%

OSF 10%

OSF 15%

44


Scenario 7-a (SoS Size = 10) Scenario 7-b (SoS Size = 100)

Relative Cost of Collaborative and Acknowledged SoSESoS Capability Scope Varies


-1500.00

-1000.00

-500.00

0.00

500.00

1000.00

1500.00

0 1 2 3 4 5 6 7 8 9 10 11 12

Number of Systems Affected by Capability

Sa

vin

gs

(P

ers

on

Mo

nth

s)

OSF 5%

OSF 10%

OSF 15%



-5000.00

0.00

5000.00

10000.00

15000.00

20000.00

25000.00

0 20 40 60 80 100 120


Sav

ing

s (P

erso

n

Mo

nth

s) OSF 5%

OSF 10%

OSF 15%

Scenario 8-a (SoS Size = 10) Scenario 8-b (SoS Size = 100)


System Volatility = None and SoS Capability = 1000 Reqs

-1000.00

-500.00

0.00

500.00

1000.00

1500.00

0 1 2 3 4 5 6 7 8 9 10 11 12


Sav

ing

s (P

erso

n M

on

ths)

OSF 5%

OSF 10%

OSF 15%

Relative Cost of Collaborative and Acknowledged SoSESoS Capability Scopre Varies


0.00

5000.00

10000.00

15000.00

20000.00

25000.00

0 20 40 60 80 100 120

Number of SYstems Affected by Capability

Sav

ings

(Per

son

Mon

ths)

OSF 5%

OSF 10%

OSF 15%

45


Scenario 9 (SoS Size = 10) Scenario 10 (SoS Size = 5)


System Volatility = 1000 and SoS Capability = 1 Req

-300.00

-200.00

-100.00

0.00

100.00

0 1 2 3 4 5 6 7 8 9 10 11 12


Savin

gs (

Pers

on

Mo

nth

s)

OSF 5%

OSF 10%

OSF 15%

Relative Cost of Collaborative and Acknowledged SoSESoS Size = 5 SoS Capability Scope Varies


-1000.00

-500.00

0.00

500.00

0 1 2 3 4 5 6


Savin

gs (

Pers

on

Mo

nth

s)

OSF 5%

OSF 10%

OSF 15%

Scenario 11 (SoS Size = 5) Scenario 12 (SoS Size = 5)



-800.00

-600.00

-400.00

-200.00

0.00

200.00

0 1 2 3 4 5 6


Sav

ings

(Per

son

Mon

ths)

OSF 5%

OSF 10%

OSF 15%


System Volatility = 1000 and SoS Capability = 1 Req

-160.00

-120.00

-80.00

-40.00

0.00

0 1 2 3 4 5 6


Sa

vin

gs

(P

ers

on

Mo

nth

s)

OSF 5%

OSF 10%

OSF 15%

46

Summary

Presented approach for extending COSYSMO cost model to estimate systems engineering effort for

• Complex systems• Systems of systems

Additional accuracy improvements can be provided through

• Local calibrations of the COSYSMO constants• Incorporation of reuse factors [Wang, et al., 2008]

Examples provided for• Using alternative size drivers• Showing how cost model can be used to evaluate alternatives

47

Acknowledgements

The author would like to acknowledge • The pioneering work done by Dr. Ricardo

Valerdi in the development of the initial COSYSMO cost model upon which this research effort is based

• The research support received from Stevens Institute of Technology and the International Council on Systems Engineering (INCOSE) Foundation through the 2007 INCOSE Foundation/Stevens Doctoral Award

48

References1. Dahmann, J. and K. Baldwin. 2008. Understanding the current state of US defense systems of

systems and the implications for systems engineering. Proceedings of the IEEE Systems Conference, April 7-10, in Montreal, Canada.

2. Department of Defense. 2008. Systems engineering guide for system of systems, version 1.0.

3. Maier, M. 1998. Architecting principles for systems-of-systems. Systems Engineering 1, no. 4: 267-284.

4. Valerdi, R. 2005. Constructive systems engineering cost model. PhD. Dissertation, University of Southern California.

5. Valerdi, R. and M. Wheaton. 2005. ANSI/EIA 632 as a standardized WBS for COSYSMO, AIAA-2005-7373, Proceedings of the AIAA 5th Aviation, Technology, Integration, and Operations Conference, Arlington, Virginia.

6. Wang, G., R. Valerdi, A. Ankrum, C. Millar, and G. Roedler. 2008. COSYSMO reuse extension, Proceedings of the 18th Annual International Symposium of INCOSE, The Netherlands.

System of Systems Engineering (SoSE) Cost Estimation Jo Ann Lane jolane at usc.edu Presented by...

Documents

Transcript of System of Systems Engineering (SoSE) Cost Estimation Jo Ann Lane jolane at usc.edu Presented by...