2-5 November 2010 University of Southern California Los Angeles CA Systems Engineering Desk...
-
Upload
myron-reed -
Category
Documents
-
view
216 -
download
0
description
Transcript of 2-5 November 2010 University of Southern California Los Angeles CA Systems Engineering Desk...
2-5 November 2010
University of Southern CaliforniaLos Angeles CA
Systems Engineering Desk Reference
An Aid for Cost Estimators
Sherry StukesJet Propulsion Laboratory/
California Institute of Technology
Integrated Ground Data Systems
Ground Software Systems Engineering
4800 Oak Grove Drive MS 301-225
Pasadena CA818.393.7517 (o)805.402.8664 (c)
25th International Forum on COCOMO and
Systems Software Cost Modeling
Henry ApgarPresident
MCR Technologies LLC390 N Sepulveda Blvd
Suite 1050El Segundo CA
424.218.1616 (o)805.402.4232 (c)[email protected]
Copyright 2010. All rights reserved.
BackgroundSSCAG* Systems Subgroup Desk
Reference productRepresents industry “Best Practices”Useful to Cost EstimatorsOrganized into six sections
ContributorsNASA (JPL, LaRC, MSFC)MCR TechnologiesUSCDesign for Value
*SSCAG (Space Systems Cost Analysis Group) is an International working group comprised of member organizations that develop estimating products for the space industry. SSCAG currently has four active subgroups: Hardware, Software, Risk, and Systems, supported by members from industry, Government, and the academic community.
SAICUSAF (SMC, AFCAA)RaytheonModel Vendors (Galorath,
PRICE Systems)
2
Desk Reference Overview System Engineering Desk Reference
ContentDocument OverviewHow to Predict and Evaluate Systems Engineering and SoS
CostsDefinitionsRules of Thumb“Top 10”ToolsLessons Learned and Estimating Examples
SourcesRedStar LibraryConstellation ProgramModel vendor researchUniversity researchContractor organizations 3
Defining our TermsSystem of Systems (SoS) is a
collection of task-oriented or dedicated systems that pool their resources and capabilities together to obtain a new, more complex, 'meta-system' which offers more functionality and performance than simply the sum of the constituent systems.”
4
Assemblies
Subsystems
Systems
SOS
System Engineering can be considered to include the pure engineering efforts to ensure that a number of subsidiary elements function together properly but also project/program management, integration and test, missions assurance and ground support elements sometimes called “SEPM” or Systems Engineering and Project/Program. The definition can be adapted to lower levels, including subsystems and assemblies.
SoS HierarchyThe hierarchal aspect of
SoS is reflected in the fact that depending on how one defines system, almost any integration activity can be tagged as SoS
An example of SoS hierarchy is the crew launch system for the NASA Constellation Program. System of Systems” consists of
the Ares I launch vehicle system and the Orion crew capsule system
These systems themselves are comprised of multiple systems 5* Reference – materials submitted by
Andy Prince, NASA Marshall Space Flight Center.
Rules of Thumb Model Development
More parameters increase the explanatory power of the model, but too many parameters make the model too complex to use and difficult to calibrate.
Break the problem and analysis into phases over time; the right amount of granularity is important.
Let available data drive the application boundaries of the model.
Design the rating scale according to the phenomenon being modeled.
Some system characteristics are more likely to be cost penalties than cost savings.
Model Calibration All calibrations are local. Calibrations fix chronic errors in over- or underestimation. Be skeptical of data that you did not collect. For every parameter in the model, 5 data points are
required for the calibration. Don’t do more analysis than the data is worth. You need less data than you think, you have more data
than you think. Model Usage
A model is not reality. All models are wrong, but some of them are useful. Begin with the end in mind.
Requirements are king. Not all requirements are created equal. Reuse is not free. Operational Scenarios may come first, but
requirements will ultimately describe the system.
Don't double dip. Find your sea level. Nominal is the norm. If you're estimating a large project, personnel
capability is Nominal. Most of your off-Nominal cost drivers should
match your last project. If you're going to sin, sin consistently. Use a combination of models to estimate total
system cost. Avoid overlap between models. Estimate using multiple methods (analogy,
parametric, etc.). Estimation
Estimate early and often. Experts all disagree forever. Bound the options
they are given to evaluate.People are generally optimistic.
6
“Top 10” ToolsVersion SE Defined LCC
PhaseLevels WBS
DefinedSE Est. Method
Price ES System Engineering Process A-F
Subsystem, System
User Defined Imbedded Algorithm
Price True Planner
System Engineering A-D SoS EIA/ANSI 632 WBS
Academic - COSYSMO algorithm
SEER -H System Engineering and Integration A-F
Subsystem, System
User Defined Imbedded Algorithm
NAFCOM System Engineering & Integration A-E System Yes -
ConfigurableCER by mission type
SSCM07 System Engineering (w/PM) C,D System Yes – Defined CER
USCM Systems Engineering (w/in Program Level Cost)
A-D System Yes – Configurable
CER by Spacecraft type
NICM (both models)
Systems Engineering (estimated but not def.)
B,C,D thru L+30 System Yes - Defined CER in two
forms
COCOMOSoftware Engineering in the System Context (Waterfall WBS)
Waterfall and Mbase
SystemSoftware
Waterfall WBS and Mbase WBS
COCOMO algorithm
COSYSMOfor SE
Systems Engineering Effort A-D
System, Systems Eng
EIA/ANSI 632 WBS
academic COSYSMO algorithm
COSYSMOfor SoSE
Systems of Systems Engineering Effort C-E SoS DoD SEGuide
for SoSacademic COSYSMO algorithm
7
Cost Factors - NASA SOS Cost ModelingSystem-of-Systems Cost =
0.07411*(DDTE$)0.9993*(1.09585)M/LV
where DDTE$ = Total DDTE* Cost, in 2006$, M/LV = Manned or Launch Vehicle (Yes = 1, No
= 0)
Good Quality MetricsR2 = 93.1%SPE = 33.2%
System-of-Systems Actual Vs. Estimated
$0
$500
$1,000
$1,500
$2,000
$2,500
$3,000
$3,500
$0 $500 $1,000 $1,500 $2,000 $2,500 $3,000 $3,500
Actual (2006 $M)
Estim
ated
(200
6 $M
)
8*DDTE – Design Development Test and
Evaluation
9
Estimate Example – NASA
Factor-based approach - derived from real data (c. Smart, SAIC)
WBS Number
WBS Element
WBS Title WBS Description / Notes
1
SO
S
System of Systems Level Systems Engineering, Integration and Test and Top Level Program Management of Architecture
2 Space Segment Assets in orbit or interplanetary
2.1 Crew Expolration Vehicle (CEV)
2.1.3 Command Module (CM)2.1.3.2 CM PMP CM Prime Mission Product
(HW + SW)Orion Integration, Assy., & C/OOrion SE&IOrion PMOrion STOOrion GSE
2.1.4 Service Module (SM)2.1.4.1 SM SE IT PM Systems Engineering,
Integration and Test and Top Level Program Management of SM
2.1.4.2 SM PMP SM Prime Mission Product (HW + SW)
2.2 International Space Station2.2.1 ISS Communications2.2.2 TDRS Space Segment additional satellite(s) to
support CxP ISS mission3 Launch Segment
3.1 CLV3.1.1 First Stage SRM3.1.2 Upper Stage Improvements for
communicationg with ground and TDRSS.
3.1.2.2 Upper Stage Avioncis Software
3.1.2.1 Upper Stage Avionics Hardware
Comms to/from AF ROCC
Upper Stage Integration, Assy., and C/OUpper Stage SE&IUpper Stage PMUpper Stage STOUpper Stage GSE
4 Mission/Ground Ops4.1 Ground Stations Provides -S-band and UHF
air-to- ground4.1.1 Launch Head MILA/PDL = Merritt Island &
Ponce de Leon; Merritt has 14 antennae; Ponce de Leon has 3 antennae; both are dedicated to shuttle/CxP missions.
4.1.2 Wallops Island, VA 5m and 8m S-Band tracking; not dedicated site
4.1.3 New Boston, NH4.1.4 Argentia, Newfoundland
4.2 Deep Space Network (DSN) Stations
Canberra, Goldstone, Madrid
4.3 Range Safety Backuo provided by DoD ground-based radars
SP
AC
EM
ISS
ION
/GR
OU
ND
OP
SLA
UN
CH
Systems of Systems Cost
Total DDT&E Cost % of DDTE
Apollo $1,654.1 $18,449.0 8.97%Atlas II $234.6 $3,040.4 7.71%Brilliant Pebbles $77.8 $1,100.2 7.07%HST $113.3 $1,592.6 7.12%ISS $3,646.4 $36,027.4 10.12%Peacekeeper $265.6 $10,006.7 2.65%Pioneer Venus $24.9 $306.3 8.13%Saturn IB $325.1 $3,776.7 8.61%Saturn V $603.9 $10,333.9 5.84%Shuttle $597.8 $18,152.6 3.29%Skylab $334.1 $4,127.7 8.09%Titan IV $516.5 $5,178.0 9.97%
9* Reference – data collected and analyzed by Dr. Christian Smart, SAIC, under contract to NASA Marshall Space Flight Center.
RecommendationsCarefully consider the applicationConsider how SoS differs from traditional engineering systems
and how this affects the estimatorSupporting platforms are operationally, geographic, and managerial
independent, as well as network-centricNew acquisition concepts means we need new CERs, factors, and cost
driversImmaturity of concept means little cost data is currently available
SOS cost drivers are unique and require considerations beyond traditional systems estimating
Review available research and papersCurrent research by USC, DAU, SEI, NASA, Cranfield University
(UK)Available papers from USC, MIT, IEEE, INCOSE
10
Lessons LearnedInconsistent parameter definitions
Between models (need to adopt INCOSE standards and develop a mapping scheme)
Between historic data (collected before standards were established) and new CERs
Inconsistent WBS applicationsAre we double-counting the SOS costs by applying factors at each platform
level?Inconsistent platform applications
Do we use the same factors for hardware systems as well as software systems? Are space platform factors different from air and ground platform factors? Are manned-space platform factors different from unmanned-space platform factors?
Might be more useful if, in the near term, we rely on databases (and factors) rather than on statistical CERs
11
Publication ScheduleHighly motivated, volunteer workforceHosted in a collaborative work areaPeriodic “tag-up” teleconsCurrent Schedule
Review 1 – 31 October 2010Review 2 – 20 November 2010Materials to Editor – 30 November 2010Complete DR first draft – 31 December 2010
Seeking volunteer reviewersPublished Desk Reference will be available in
April 2011!12