Implementing a COTS System Design Using Raytheon IPDS and Axiomatic Design
By
Carl McGaha
A MASTER OF ENGINEERING REPORT
Submitted to the College of Engineering at
Texas Tech University in
Partial Fulfillment of
The Requirements for the
Degree of
MASTER OF ENGINEERING
Approved
______________________________________ Dr. A. Ertas
______________________________________ Dr. T. Maxwell
______________________________________ Dr. M. Tanik
______________________________________ Dr. E.W. Kiesling
October 15, 2005
B-1
ACKNOWLEDGEMENTS
I would like to thank all of the Texas Tech staff and guest speakers who contributed to this
program. It was obvious that many of the instructors and guest speakers put forth efforts well beyond
what would have been required or expected. I would also like to express my gratitude to Raytheon for
allowing me to participate in this program.
B-2
TABLE OF CONTENTS
ACKNOWLEDGEMENTS......................................................................................................1 DISCLAIMER ..........................................................................................................................4
ABSTRACT ..............................................................................................................................5 LIST OF FIGURES ..................................................................................................................6
CHAPTER I INTRODUCTION ....................................................................................................................7
CHAPTER II BACKGROUND .....................................................................................................................10 2.1 IPD and Raytheon IPDS......................................................................................................10
2.1.1 The Ten Tenets of IPDS..........................................................................................10 2.2 COTS..................................................................................................................................11 CHAPTER III THE DEFINITION OF COTS ...............................................................................................12 3.1 Non Trivial .........................................................................................................................12 CHAPTER IV BUSINESS STRATEGY EXECUTION ................................................................................13 4.1 Gate 4 - Proposal Review....................................................................................................13 4.2 Who Is The “Customer”? ....................................................................................................13
4.2 COTS vs. Custom Trade ............................................................................................13 CHAPTER V PROJECT PLANNING, MANAGEMENT AND CONTROL .............................................15 5.1 Gate 5 - Startup Review ......................................................................................................15 5.2 Development Cycle Model..................................................................................................15 CHAPTER VI REQUIREMENTS AND ARCHITECTURE DEVELOPMENT.........................................19 6.1 Gate 6 - System Functional Review (Requirements Review) ...............................................19 6.2 Axiomatic Design ...............................................................................................................19
6.2.1 Basic Principles ......................................................................................................19 6.2.2 An Axiomatic Design Tool .....................................................................................24 6.2.3 A Simple COTS System Design..............................................................................26
CHAPTER VII PRODUCT DESIGN AND DEVELOPMENT ......................................................................50 7.1 Gate 7 - Preliminary Design Review ...................................................................................50 7.2 Gate 8 - Critical Design Review..........................................................................................53 CHAPTER VIII SYSTEM INTEGRATION, VERIFICATION, AND VALIDATION..................................54 8.1 Gate 9 - Test/Ship Readiness Review ..................................................................................54 CHAPTER IX PRODUCTION AND DEPLOYMENT .................................................................................55
B-3
9.1 COTS "Production".............................................................................................................55 9.2 Gate 10 - Production Readiness Review ..............................................................................55 CHAPTER X OPERATIONS AND SUPPORT............................................................................................56 10.1 Remote Support ................................................................................................................56 CHAPTER XI PROGRAM CLOSURE .........................................................................................................57 11.1 Gate 11 - Transition and Closure Review ..........................................................................57 CHAPTER XII SUMMARY AND CONCLUSIONS ......................................................................................58 12.1 Summary and Conclusions ................................................................................................58
12.1.1 COTS & IPDS ......................................................................................................58 12.1.1 COTS & Axiomatic Design...................................................................................58
REFERENCES .......................................................................................................................59
B-4
DISCLAIMER
The opinions expressed in this report are strictly those of the author and are not necessarily those
of Raytheon, Texas Tech University, nor any U.S. Government agency. While it is my sincere belief that
the recommendations made in this report will result in a tangible benefit to programs that apply them, I
make no guarantee of program success or profit.
B-5
ABSTRACT
The purpose of this report is to create a set of recommended processes and strategies that can be
incorporated into Raytheon IPDS to result in a more efficient implementation of systems developed with
a majority of COTS (Commercial Off The Shelf) equipment and software. The information included
comes from the basic sources:
1) The Texas Tech Masters of Engineering curriculum (Incorporate course material appropriate
for COTS system development)
2) 25 years of personal experience (Captured lessons learned from having seen it done wrong
and right many times over)
3) Raytheon IPDS
4) Other sources, as identified.
B-6
LIST OF FIGURES
Figure 1. Raytheon IPDS, Top Level View. 8
Figure 2. Simple COTS Based “Spiral” Development Model 17
Figure 3. Simple COTS Based “Waterfall” Development Model 18
Figure 4. “Coupling” in Axiomatic Design 21
Figure 5. “Zig-Zagging” in Axiomatic Design 22
Figure 6. The “Optimizer” Sorting Algorithm 24
Figure 7. Simple COTS Based System Diagram 27
Figure 8. Simple Initial Design Matrix 28
Figure 9. Initial Data Entry Into Tool 29
Figure 10. Data Entry After Combining Rows & Columns 31
Figure 11. First Pass 32
Figure 12. Second Pass 34
Figure 13. Third Pass 36
Figure 14. Third Pass 38
Figure 15. Fourth Pass 40
Figure 16. Fifth Pass 42
Figure 17. Fifth Pass 43
Figure 18. Final Matrix 46
B-7
CHAPTER I INTRODUCTION
The first chapter of this paper focuses on the definition of "COTS". The order of the remaining
chapters follows the top level organization of IPDS, shown in Figure 1, with one exception. In order to
follow a more chronological flow, the Project Planning Management and Control stage is addressed
twice; once in chapter 5 for Startup (Gate 5), and again in Chapter 11 for Transition & Closure (Gate 11).
The Project Management and Control stage of IPDS spans the entire project, but this report focuses on the
Stage checkpoints; referred to in IPDS terminology as Gates. Gates 1-3 are not discussed; for two
reasons:
1) These gates require very little input from Engineering
2) There should be no separate issues for COTS in these early gates.
In each chapter, the special strategies, processes, and considerations for COTS development
related to that stage of IPDS will be identified and discussed.
B-8
Program
Execution
Gates 5 -11
Business Business
Decision Decision
Gates 1Gates 1 --44
2 - Project Planning, Management and Control
3 -
Requirements
andArchitecture
Development
5 -
System Integration,Verification
and Validation
4 -Product Design
and
Development
Planning 6 - Production and Deployment
Planning 7 - Operations and Support
1 -
Business Strategy
Planning/Execution
Strategic
PlanningCapture/Proposal Development
1 2 3 4
5
6
7 8
109
11
= GATE
Opportunity Review (Interest/No Interest)
Win Strategy Review (Pursue/No Pursue)
Pre-Proposal Readiness Review (Bid/No Bid)
Proposal Review (Bid Approval)
Internal Preliminary
Design Review
Transition and
Closure ReviewProject Start -Up Review
Internal System Functional Review
Internal Critical
Design ReviewInternal Test/Ship
Readiness Review
Internal Production
Readiness Review
Program
Execution
Gates 5 -11
Business Business
Decision Decision
Gates 1Gates 1 --44
2 - Project Planning, Management and Control
3 -
Requirements
andArchitecture
Development
5 -
System Integration,Verification
and Validation
4 -Product Design
and
Development
Planning 6 - Production and DeploymentPlanning 6 - Production and Deployment
Planning 7 - Operations and SupportPlanning 7 - Operations and Support
1 -
Business Strategy
Planning/Execution
Strategic
PlanningCapture/Proposal Development
1 -
Business Strategy
Planning/Execution
Strategic
PlanningCapture/Proposal Development
1 2 3 4
5
6
7 8
109
11
= GATE
Opportunity Review (Interest/No Interest)
Win Strategy Review (Pursue/No Pursue)
Pre-Proposal Readiness Review (Bid/No Bid)
Proposal Review (Bid Approval)
Internal Preliminary
Design Review
Transition and
Closure ReviewProject Start -Up Review
Internal System Functional Review
Internal Critical
Design ReviewInternal Test/Ship
Readiness Review
Internal Production
Readiness Review
Figure 1. Raytheon IPDS, Top Level View.
B-9
I will not attempt to connect information in this report to lower levels of the Raytheon IPDS
structure for two reasons:
1) To keep the report at a non-company-proprietary level
2) Because a new release of IPDS is in work that will be changing a lot of the structure below the top
level. This version will likely be released before my report is complete, so direct linkage to lower
levels of the current version would not be of any practical use.
B-10
CHAPTER II BACKGROUND
2.1 IPD and Raytheon IP DS
IPD (Integrated Produce Development) is a process framework originated in MILSTD-499
(USAF) 17 July 1969, encompassing the elements of TQM (Total Quality Management). A version of
IPD adopted by Raytheon is IPDP (Integrated Product Development Process). This is the detailed
process that when followed implements the best practices of Legacy Raytheon and various acquired
legacy companies. IPDS (Integrated Product development Process), is the policy umbrella that requires
use of IPDP on all Raytheon programs and projects. For the remainder of this paper, I will refer only to
IPDS; which has become standard Raytheon terminology.
2.1.1 The Ten Tenet s of IP DS
The Ten Tenets of IPDS (Texas Tech University Masters Program, Systems Engineering Principles, ME 5354) 1. Customer Focus
2. Integrated Management Tools.
(Identify and Satisfy Customer Needs, Better, Seamless Planning and Tracking of Resources, Cost,
and Faster, Cheaper Schedule)
3. Concurrent Development of Product and Processes
4. Proactive Identification and Management of Risk
(Product Focus & Process Understanding Management of Technical, Cost, and Schedule Risk)
5. Event Driven Scheduling
6. Early and Continuous Life Cycle Planning
(Incrementally Demonstrate Product Maturity Maximize Flexibility for Optimization and Use of
Alternative Approaches Prior to Beginning Follow-on Activities)
B-11
7. Multi-Disciplinary Teamwork and Communication
(Challenge Requirements and Use Commercial Specs and Standards)
8. The Right People, At The Right Place, At The Right Time
9. Empowerment
10. Encourage Robust Design and Improved Process Capability
(At Lowest Possible Level, Resources Allocated)D
2.2 COTS
COTS (Commercial Of The Shelf) refers not only to pieces of equipment, but to a trend by
government customers towards use of COTS equipment vs. custom design. For the remainder of this
paper, when I refer to COTS, I will be referring to equipment. If I am referring to the use of COTS in a
program, I will use the term “COTS Program”.
B-12
CHAPTER III The Definition of COTS
3.1 Non Trivial
The purist definition: "Commercial Off The Shelf" means simply that it is not COTS unless it is
"on the shelf" at the time the order is placed. This is frequently not the case. Many times, an acquisition
activity will define COTS as simplistically as "Not Built In House"; and an item designated "COTS" will
go through a full design cycle at a subcontractor. On the other hand, high-dollar computer equipment is
rarely "stocked"; so it is likely that a lot of COTS will not start getting built until after an order is placed.
The definition this paper will use is that an item is COTS if it has already been built and delivered prior to
placing the order. Other cases where purchased equipment is often called COTS will be discussed, and
identified as non-COTS or as a special variety of COTS. One such special case would be First Article
COTS. In some respects, this should be treated the same as custom. A typical COTS based system
development will consist of a combination of COTS items that really are "off the shelf", COTS items that
have been built before, but are not currently "stocked", and items that are nominally "COTS" but have
never been built before. It is important to know up front:
1) What your management thinks "COTS" means
2) What your customer thinks "COTS" means
3) What you think "COTS" means
All three may be different. The point is not to define a “right” definition of COTS. Any definition that
you, your management, and your customer all agree to is “right”; but don’t just assume your idea is the
same as everyone else’s.
B-13
CHAPTER IV Business Strategy Execution
4 .1 Gat e 4 - P roposal Review
This is the initial gate that requires significant input from Engineering. There are a number of
special considerations for a proposal for a system incorporating a large amount of COTS. Do not skip
any normal proposal review steps for an “All COTS” design.
4.2 Who Is The “Custom er”?
This is not obvious on many government contracts. It is not unusual for there to be an End User
separate from the organization providing the funding. So you must balance Customer "needs" vs. End
User "needs" vs. Customer requirements vs. End User requirements. If you do this right, there will
probably be a couple of iterations of customer SOW. If the Customer is not the End User; the Customer
probably does not have a full understanding of the End User's real needs. (Government agencies are not
known for their outstanding communications ability.)
4.2 COTS vs. Cust om Trade
A comparison of the complete cost of COTS is rarely done. A typical trade goes something like:
Catalog Cost of COTS < Total Cost of Custom; therefore, use COTS
The normally forgotten or underestimated list:
Procurement Cost
COTS procurement has a cost associated with it, in some cases a higher cost than custom
HW; especially if procurement specifications or modifications are required.
Development Monitoring Cost (Pseudo-COTS)
This is an area where it is especially important to make sure there is a clear agreement
between Management, Customers, and Engineering on what “COTS” is. If an item is being
built to specification for the first time, the only cost delta from custom will be in labor rates,
B-14
or cutting corners. The former can be a legitimate cost savings; the latter can end up being
more expensive.
O&M Cost
Be sure to consider the total lifecycle cost of COTS vs. Custom; and not just the initial
purchase or development cost. COTS with poor reliability can be cheaper to buy, but end up
being more expensive in the long run.
Customer Encouraged
Is the customer specifying, or encouraging, an all COTS system? If so, be sure you
know what his definition of "COTS" is, what he is expecting to save on cost & schedule; and
what he is willing to sacrifice in quality and performance. The higher risk is when the
customer is encouraging a particular COTS purchase, but not actually putting his desire in the
contract or SOW (Statement of Work). In that case, you can end up being the bad guy for
specifying COTS; when you were just doing what the customer asked you to.
Specification Cost (Pseudo-COTS)
When the customer’s definition of COTS is “not built here”; you may find yourself in the
position of having to write design specs for “COTS”. Be sure to include that cost in your
comparison of the cost of COTS vs. custom. This is not as arbitrary as it sounds. Custom
HW built on the same Cost Plus contract as a large systems integration is also subject to the
over-all Cost Plus contract. This means that a custom item purchased on a subcontract can be
Fixed Price, while the identical item, built per the identical specs and process, but built on the
base contract; is Cost Plus. The end result is lower cost risk to the customer by defining
COTS as “not built by the prime contractor”.
B-15
CHAPTER V Project Planning, Management and Control
5 .1 Gat e 5 - S tartup Review
In general, all IPDS checkpoints required for a custom design should be included for a COTS
design. The argument is often used, “it is so simple, why do I need a Program Management Plan?”. The
answer is that if it is that simple, your PMP will be very short, maybe just a single PowerPoint slide. Be
careful to not allow this sort of “logic” to cut steps out. Each element normally required for a startup
review should be addressed. If a business area has multiple small projects, all following the same general
pattern; create a set of plans and process documents that can be applied to them all. Additional COTS
Checkpoints for a Startup Review:
1) Is “COTS” for this project clearly defined?
2) Are there clear guidelines for COTS vs. custom decisions?
5.2 Development Cycle Model
The development cycle model for a COTS program (or any other), should be clearly identified at
the Startup Review. The development cycle with a majority of COTS is nearly always a "spiral" type
model, whether the program planners admit it or not. Plan for it up front or you will lose a lot of the
COTS savings. I have lost count how many times I have had to explain at a preliminary design review
why equipment was already on order before the design was complete. What was really happening was
that we were defining requirements and procuring equipment on a "spiral" model; but customer and
management expectations were that reviews would be defined and held per a traditional "waterfall"
model. This creates a no-win situation. You either execute per an undocumented "spiral" model, which
makes you appear to be executing design and development steps out of sequence; or you force the
development to fit a traditional "waterfall" model, and fail to deliver the cost and schedule savings that
are expected from a mostly-COTS solution. Figure 2 is a typical COTS/Custom Development model;
with all COTS HW, some COTS SW, and some custom SW. Figure 3 is a “waterfall” version of the
B-16
same model, with some more detail shown. It is not a true “waterfall” model, because after the first
spiral, Requirements, Design, and Development can all be proceeding in parallel.
B-17
sin(B)= x/R
sin(B)*R= x
cos(B)= y/R
cos(B)*R= y
Gate 5
Gate 6
Gate 7
Gate 8
Gate 5a
Gate 6a
Gate 7a
Gate 8a
Gate 5c
Gate 6c
Gate 7c
Gate 8c
Gate 9
Gate 10
Gate 11
Figure 2. Simple COTS Based “Spiral” Development Model
B-18
Spiral # Gate Gate Title Activity
A Gate 5 Start-Up Gate Review Review Program Plans &
Gate Checklists for Spiral A
A Develop Requirements
A Gate 6 System Functional
(Requirements and
Concept) Gate
Review
Review requirements for SW
Development Environment,
including preliminary
requirements for deliverable
HW & SW, definitized to the
point necessary to
completely define the
development environment.
A Preliminary Design
A Gate 7 Preliminary Design
(Architecture) Gate
Review
Review Development
Environment Architecture
Design
A Development Environment
Detailed Design
A Program Planning for Spiral
B
A Gate 8 Critical Design
(Detailed Design and
Build Readiness)
Gate Review
Review Development
Environment Detailed
Design.
A Place order for Development
Environment COTS HW &
SW
Spiral # Gate Gate Title
B Gate 5a Start-Up Gate Review Review Program Plans &
Gate Checklists for Spiral B.
Review Status of Spiral A to
ensure it is far enough along
for Spiral B to begin.
B Finish Development of
Deliverable HW
Requirements
B Gate 6a System Functional
(Requirements and
Concept) Gate
Review
Review Deliverable HW
Requirements
B Preliminary Design
B Gate 7b Preliminary Design
(Architecture) Gate
Review
Review Deliverable HW
Architecture Design
B Deliverable HW Detailed
Design
B Program Planning for Spiral
C
B Gate 8b Critical Design
(Detailed Design and
Build Readiness)
Gate Review
Review Deliverable HW
Detailed Design.
B Place Order for Deliverable
COTS HW & SW
Spiral # Gate Gate Title
C Gate 5c Start-Up Gate Review Review Program Plans &
Gate Checklists for Spiral C.
Review Status of Spiral B to
ensure it is far enough along
for Spiral C to begin.
C Develop Deliverable Custom
SW Requirements
C Gate 6c System Functional
(Requirements and
Concept) Gate
Review
Review Deliverable SW
Requirements
C Preliminary Design
C Gate 7c Preliminary Design
(Architecture) Gate
Review
Review Deliverable Custom
SW Preliminary Design
C Receive COTS HW & SW for
Development Environment
C Gate 8c Critical Design
(Detailed Design and
Build Readiness)
Gate Review
Review Deliverable Custom
SW Detailed Design.
C Develop Custom Software
C Receive COTS HW & SW for
Deliverable HW
C Test Custom SW in
Deliverable System (Factory
Acceptance Testing)
C Gate 10 Production
Readiness Gate
Review
Is the program prepared to
support delivery and
operation without the
equipment that is being
delivered? This is also where
you would review production
planning and resources for
additional "Builds"; which
could be Software only.
C Gate 9 Test/Ship Readiness
Gate Review
Review Factory Acceptance
Test Results, Go/No Go
Decision for Ship to
Customer
C Test Delivered System
C Gate 11 Transition and
Closure Gate Review
Disposition Development HW
& SW, Contract Closout
Figure 3. Simple COTS Based “Waterfall” Development Model
A typical COTS program; three spirals:
Spiral A: Development Environment Development
Sprial B: Deliverable COTS HW & SW Development
Spiral C: Deliverable Custom SW Development
B-19
CHAPTER VI Requirements and Architecture Development
6 .1 Gat e 6 - System F unct ional Review (Requirem ent s Rev iew)
A COTS program will probably have multiple Functional Requirements Reviews; one for each
spiral. The example COTS Program shown in the Chapter 5 development model (Figures 2 and 3) has 3
Requirements reviews (Gates 6, 6a, and 6b). Additional checkpoint to consider for a COTS program:
1) Have requirements been adequately defined for suppliers & subcontractors?
6.2 Axiom ati c Design
Certain Axiomatic Design Principles are very valuable for COTS based requirements definition;
even if you don't buy the whole "Axiomatic Design" premise. (It is important to note that some the
concepts are useful, even if you don't buy the over-all premise; because "Axiomatic Design" is currently
far from universally accepted.)
6.2.1 Basi c Pr inciples
Claims Made by Nam Suh Regarding Axiomatic Design
• “A general theory for system design is presented”
• “The theory is applicable to … large systems, software systems, organizations…”
• “The flow diagram … can be used for many different tasks: design, construction, operation,
modification, … maintenance … diagnosis …, and for archival documentation.”
• “Design axioms were found to improve all designs without exceptions or counter-examples…
When counter-examples or exceptions are proposed, the author always found flaws in the
arguments.”
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
B-20
What are Axioms?
• “Primitive propositions whose truth is known immediately without the use of deduction”
- Cambridge Dictionary of Philosophy
• “A statement that is stipulated to be true for the purpose of constructing a theory”
- Harper Collins Dictionary of Mathematics
• “Fundamental truths that are always observed to be valid and for which there are no
counterexamples or exceptions”
- Suh, The Principles of Design
• “Axioms are posited as accepted truths, and a system of logic is built around them”
– Hazelrigg, “An Axiomatic Framework for Engineering Design”
Benefit of Separate Domains
• Customer Needs are stated in the customer’s language
• Functional Requirements and Constraints are determined to satisfy Customer Needs
• “The FRs must be determined in a solution neutral environment” (or, in other words, say
“what” not “how”)
– BAD = the adhesive should not peel
– BETTER = the attachment should hold under the following loading conditions
• Keeping FRs in solution neutral terms prevents inadvertent “lock in” to specific modes of
solution
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
B-21
Suh’s Design Axioms
• The Independence Axiom
– Maintain the independence of the functional requirements.
• The Information Axiom
– Minimize the information content.
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
The second axiom is just another way of saying that, all other things being equal, simpler is
better; common knowledge for any experienced engineer. This paper will focus on application of the first
Axiom to COTS integration.
An “uncoupled” design is a design in which no requirement has any coupling to any other
requirement. A “decoupled” design is one in which there is some coupling, but in which the matrix can
be sorted to lower triangular form. (See Figure 4)
Note that the decoupled design can be either a lower or upper triangular matrix. This examples
paper uses lower triangular form, because that sets the order of COTS component selection as left to right.
Using an upper triangular form would work equally well, but the order of component selection would be
right to left.
!!!
"
#
$$$
%
&
X
X
X
!!!
"
#
$$$
%
&
XXX
XX
X
!!!
"
#
$$$
%
&
XXX
XXX
XXX
An Uncoupled A Decoupled A Coupled Design Design Design
Figure 4. “Coupling” in Axiomatic Design
B-22
Functional Coupling vs Physical Coupling
Just because a single physical entity carries out multiple functions it does not imply functional
coupling!
Benefits of Uncoupling
• Simpler operation
• More transparent design
• Simpler to change the design
• More parallelism in the design process
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
The concept of "Zig-Zagging" is mentioned here for completeness. It not addressed in this paper
with respect to COTS integration, because the nature of COTS integration typically restricts the designer
to only top level components.
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
Figure 5. “Zig-Zagging” in Axiomatic Design
B-23
The Supposedly Dire Consequences of Coupling
• “.. when several functional requirements must be satisfied, designers must develop designs that
have a diagonal or triangular matrix”
• “For a product to be manufacturable, the design matrix of a product, [A] … times the design
matrix for the manufacturing process, [B] … must yield either a diagonal or triangular matrix.
Consequently, when … either [A] or [B] represents a coupled design, the product cannot be
manufactured.” Theorem 9 (Design for Manufacturability)
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
For COTS Systems, the consequences are slightly different. The components are already
manufactured, so the above statement is not completely applicable. The consequences of a highly
coupled COTS design are that there is no clear order of component definition. That is, if you define the
Operating System first, then the Processor; you might have to go back and pick a different operating
system, because the Processor you selected will not run the Operating System you selected. In practice,
due to restocking charges and other costs of "going back", what happens is that you end up being
unnecessarily constrained in your choice of (for example) Processor, by your previous choice of
Operating System. Application of axiomatic design maximizes your options for COTS selection, and
ensures that those choices only need to be made one time.
The second Axiom is applicable to COTS integration too; but not in the same way as to a custom
design. Application of the second Axiom means that, all other things being equal, choose the COTS that
satisfies all of the requirements, with as few additional "features" that do not satisfy any requirements as
possible.
B-24
6.2.2 An Axiom ati c Design Tool
The Axiomatic design concept applied to COTS development in this paper is the use of Design
Parameter vs. Requirements matrices to analyze requirement coupling. The single most labor intensive
part of this analysis seems to be sorting the matrix to Lower Triangular form. To this end; an Excel tool
has been created to facilitate this “sorting” activity. The “Optimizer” macro assigns weights to each row
and column of the matrix based on the number of ones in a row or column, and the number of ones in
each row or column that intersects a one. For example:
Row
Sums
1 1 2
1 1 1 3
1 1 1 3
1 1 2
1 1 1 1 4
1 1
Column
Sums3 2 1 3 3 1
Figure 6. The “Optimizer” Sorting Algorithm
The weight assigned to the highlighted column is 2 + 3. In addition, each of the column and row
sums is weighted proportionately to how far it is from the lower left hand corner of the matrix. The
matrix is then sorted by the weighted row sums and column sums. After each sort, the macro tests to see
if there are any ones left above the diagonal, and if there are, it sorts again. Since the second level of
weighting changes depending on position; the sort result tends to push the ones towards the lower left.
The macro monitors the running average of the number of ones above the diagonal, and if it does not
change after a fixed number of passes, the macro exits with a warning that a solution may not exist.
B-25
(Unfortunately, I have been unable to find or invent a method of determining in advance whether a matrix
with no obvious error conditions can be sorted to lower triangular form.)
This particular implementation of the macro and worksheet will sort up to a 25 x 25 matrix; but
the algorithms have been tested up to a 100 x 100, and are still fully functional. But Excel runs extremely
slow with a fully populated 100 x 100, and the method used to identify identical rows and columns
sometimes will give false errors, due to the internal resolution of Microsoft Excel (15 significant figures)
The “Optimizer” also provides some basic error checking, as well as the ability to identify highly
coupled row/column elements. Errors that are currently specifically flagged in the macro:
- Two or More Rows or Columns Are Identical; Or A Row or Column Has No Coupling
- There Are Not Enough Coupled Parameters & Requirements to Generate a Lower Triangular
Matrix of This Array Size
- There Are Too Many Coupled Parameters & Requirements to Generate a Lower Triangular
Matrix of This Array Size
- This Is Not A Square Matrix
- This matrix is too dense, with too many rows or columns containing the same number of
elements to be sorted into lower triangular form
Where appropriate, errors are highlighted in yellow. For example, duplicate rows are
highlighted; too many coupled elements are not.
B-26
6.2.3 A Simple COTS Syst em Design
The system shown in Figure 7 is intended to be quite generic, to illustrate the principles of
Axiomatic design that I am recommending to apply to COTS integration. Figure 8 is the initial design
matrix, created by identifying couplings (dependencies) between Design Parameters and Functional
Requirements.
B-27
Figure 7. Simple COTS Based System Diagram
Memory Cards
Mass Storage
Signal Processors
Sensors
Processor(s)
Operating System
Application Software
User/Operator
Man Machine Interface
Hardware, Including
Keyboard(s)
Monitor(s)
B-28
(1
) Pro
cessor(
s)
(2)
Mem
ory
Card
(s)
(3)
Mass S
tora
ge
(4)
Sig
nal Pro
cessor(
s)
(5)
Sensors
(6)
Opera
ting S
yste
m
(7)
Application S
oft
ware
(8)
Cabin
et(
s)
(9)
Monitor(
s)
(10)
Keyboard
s
(11)
Oth
er
User
MM
IH H
W
(12)
Furn
iture
(1) Processing Speed 1 0 0 0 0 1 0 0 0 0 0 0
(2) Disk Read/Write Speed 1 0 1 0 0 1 0 0 0 0 0 0
(3) Memory Read/Write Speed 1 1 0 0 0 1 0 0 0 0 0 0
(4) Sensor Functionality 0 0 0 1 1 0 1 0 0 0 0 0
(5) Physical Space 1 0 1 1 0 0 0 1 1 0 0 1
(6) Electrical Power 1 1 1 1 0 0 0 0 1 0 0 0
(7) Operator Input/Output 0 0 0 0 0 0 1 0 1 1 1 0
(8) Disk Capacity 1 0 1 0 0 1 0 0 0 0 0 0
(9) Memory Capacity 1 1 0 0 0 1 0 0 0 0 0 0
(10) Application SW Functionality 1 1 1 1 1 1 1 0 0 0 0 0
(11) Ergonomics 0 0 0 0 0 0 0 1 1 1 1 1
(12) Operating System Functionality 1 1 1 0 0 1 0 0 0 0 0 0
Figure 8. Simple Initial Design Matrix
B-29
Note the two items included that are not on the diagram; Cabinet(s), and Furniture. In order to
determine that the design is sufficiently decoupled to simultaneously satisfy all requirements, this matrix
must be rearranged, by whole columns and rows, to fit into lower triangular format. Figure 9 is the initial
data entry into the Excel tool.
(1
) P
rocessor(
s)
(2)
Mem
ory
Card
(s)
(3)
Ma
ss S
tora
ge
(4)
Sig
na
l P
roce
sso
r(s)
(5)
Sensors
(6)
Op
era
tin
g S
yste
m
(7)
Applic
ation S
oftw
are
(8)
Cabin
et(
s)
(9)
Monitor(
s)
(10)
Keyboard
s
(11)
Oth
er
User
MM
IH H
W
(12)
Furn
iture
(1) Processing Speed x x
(2) Disk Read/Write Speed x x x
(3) Memory Read/Write Speed x x x
(4) Sensor Functionality x x x
(5) Physical Space x x x x x x
(6) Electrical Power x x x x x
(7) Operator Input/Output x x x x
(8) Disk Capacity x x x
(9) Memory Capacity x x x
(10) Application SW Functionality x x x x x x x
(11) Ergonomics x x x x x
(12) Operating System Functionality x x x x
Figure 9. Initial Data Entry Into Tool
B-30
The yellow highlighted rows and columns indicate duplicates. If a row is duplicated, that means
that for the purposes of resolving the matrix to lower triangular form, those two requirements can be
combined. If it does not make sense to combine the requirements, or if an equal number of design
parameters cannot also be combined; you must either identify an additional coupling between one of the
requirements and one of the design parameters; or eliminate that requirement and one design parameter.
In this case, the duplicates are:
First Duplicate Rows:
(2) Disk Read/Write Speed and (8) Disk Capacity
Combining these two makes sense, since it is unlikely that any design parameter choice that
affects one of these would not also affect the other.
Second Duplicate Rows:
(3) Memory Read/Write Speed and (9) Memory Capacity
Again, it makes sense to combine these two in this context.
First Duplicate Columns:
(8) Cabinets and (12) Furniture
Cabinets and Furniture design parameters may also be combined.
Second Duplicate Columns:
(10) Keyboards and (11) Other MMIH HW
It also makes sense to combine these; in fact, a bit more thought before creating the matrix might
have put both of these in the same grouping to begin with.
Since all the combinations make sense, and there are an equal number of requirements and
design parameters that need to be combined; do it. Figure 10 shows the result.
B-31
(1) P
roce
ssor
(s)
(2) M
emor
y C
ard(
s)
(3) M
ass
Sto
rage
(4) S
igna
l Pro
cess
or(s
)
(5) S
enso
rs
(6) O
pera
ting
Syst
em
(7) A
pplic
atio
n So
ftwar
e
(8) C
abin
et(s
) & F
urni
ture
(9) M
onito
r(s)
(10)
MM
IH H
W, I
nclu
ding
Key
boar
d(s)
(1) Processing Speed x x (2) Disk Read/Write Speed & Capacity x x x
(3) Memory Read/Write Speed & Capacity x x x (4) Sensor Functionality x x x
(5) Physical Space x x x x x (6) Electrical Power x x x x x
(7) Operator Input/Output x x x (10) Application SW Functionality x x x x x x x
(11) Ergonomics x x x (12) Operating System Functionality x x x x
Figure 10. Data Entry After Combining Rows & Columns
B-32
This set seems to work; now the tool does not detect any immediate problems. Figure 11 shows
the result of running the tool the first time.
(1)
Pro
cessor(
s)
(6)
Op
era
tin
g S
yste
m
(2)
Mem
ory
Card
(s)
(3)
Ma
ss S
tora
ge
(4)
Sig
na
l P
roce
sso
r(s)
(9)
Monitor(
s)
(7)
Applic
ation S
oftw
are
(5)
Sensors
(10)
MM
IH H
W, In
clu
din
g K
eyboard
(s)
(8)
Cabin
et(
s)
& F
urn
iture
(1) Processing Speed x x
(3) Memory Read/Write Speed & Capacity x x x
(2) Disk Read/Write Speed & Capacity x x x
(12) Operating System Functionality x x x x
(6) Electrical Power x x x x x
(4) Sensor Functionality x x x
(10) Application SW Functionality x x x x x x x
(7) Operator Input/Output x x x
(11) Ergonomics x x x
(5) Physical Space x x x x x Figure 11. First Pass
B-33
The tool sorts the matrix based on row and column weights, which in this case, is not good enough to get
it into lower triangular form. This means couplings are going to have to be identified that can be
removed. Couplings can be removed by new data, assumptions, or constraints; all of which needs to be
documented. My first cut is going to be to create a constraint that all equipment must fit within the
cabinets & furniture. This is not particularly onerous, as in general, all equipment will fit within properly
selected cabinets & furniture anyway. Creating this Constraint will allow removal of the following
couplings:
(1) Processor(s) to (5) Physical Space
(3) Mass Storage to (5) Physical Space
(4) Signal Processor(s) to (5) Physical Space
(9) Monitor(s) to (5) Physical Space
Figure 12 shows the result of removing these couplings and running the tool again.
B-34
(1) P
roce
ssor
(s)
(6) O
pera
ting
Syst
em
(8) C
abin
et(s
) & F
urni
ture
(2) M
emor
y C
ard(
s)
(3) M
ass
Sto
rage
(9) M
onito
r(s)
(10)
MM
IH H
W, I
nclu
ding
Key
boar
d(s)
(7) A
pplic
atio
n So
ftwar
e
(4) S
igna
l Pro
cess
or(s
)
(5) S
enso
rs
(1) Processing Speed x x (5) Physical Space x
(3) Memory Read/Write Speed & Capacity x x x (2) Disk Read/Write Speed & Capacity x x x
(12) Operating System Functionality x x x x (11) Ergonomics x x x
(7) Operator Input/Output x x x (6) Electrical Power x x x x x
(4) Sensor Functionality x x x (10) Application SW Functionality x x x x x x x
Figure 12. Second Pass
B-35
It is very important to document all added constraints and removed couplings. Once the matrix is in
lower triangular form, it will probably be possible to put some of the couplings back in, and remove the
corresponding constraints. Next, note that there are no columns with only element; and there must be
one, or there is no solution. Look at each column with two elements, and see which one makes the most
sense to decouple. The only one that makes sense is to try to decouple:
(5) Sensors to (10) Application SW Functionality
The added constraint is that the pool of possible application SW is reduced to only that which is
supported by the pool of possible sensors. Figure 13 shows the result of removing that coupling and
running the tool again.
B-36
(1)
Pro
cessor(
s)
(6)
Op
era
tin
g S
yste
m
(8)
Cabin
et(
s)
& F
urn
iture
(2)
Mem
ory
Card
(s)
(3)
Ma
ss S
tora
ge
(4)
Sig
na
l P
roce
sso
r(s)
(9)
Monitor(
s)
(7)
Applic
ation S
oftw
are
(5)
Sensors
(10)
MM
IH H
W, In
clu
din
g K
eyboard
(s)
(1) Processing Speed x x
(5) Physical Space x
(3) Memory Read/Write Speed & Capacity x x x
(2) Disk Read/Write Speed & Capacity x x x
(12) Operating System Functionality x x x x
(6) Electrical Power x x x x x
(10) Application SW Functionality x x x x x x
(4) Sensor Functionality x x x
(11) Ergonomics x x x
(7) Operator Input/Output x x x Figure 13. Third Pass
B-37
Next, remove four couplings by altering two requirements:
The requirements for Mass Storage Capacity and Memory Capacity
are both increased to the level that will support all possible
Processors and Operating Systems.
This constraint removes the following couplings:
(1) Processor(s) to (3) Memory Read/Write Speed & Capacity
(1) Processor(s) to (2) Disk Read/Write Speed & Capacity
(6) Operating System to (2) Disk Read/Write Speed & Capacity
(6) Operating System to (3) Memory Read/Write Speed & Capacity
Figure 14 shows the result of removing these couplings and running the tool again.
B-38
(2
) M
em
ory
Card
(s)
(3)
Ma
ss S
tora
ge
(1)
Pro
cessor(
s)
(6)
Op
era
tin
g S
yste
m
(8)
Cabin
et(
s)
& F
urn
iture
(4)
Sig
na
l P
roce
sso
r(s)
(9)
Monitor(
s)
(7)
Applic
ation S
oftw
are
(5)
Sensors
(10)
MM
IH H
W, In
clu
din
g K
eyboard
(s)
(3) Memory Read/Write Speed & Capacity x
(2) Disk Read/Write Speed & Capacity x
(1) Processing Speed x x
(5) Physical Space x
(12) Operating System Functionality x x x x
(6) Electrical Power x x x x x
(10) Application SW Functionality x x x x x x
(4) Sensor Functionality x x x
(11) Ergonomics x x x
(7) Operator Input/Output x x x Figure 14. Third Pass
B-39
Next, constrain application SW to support all Operating Systems. This removes the coupling:
(6) Operating System to (10) Application SW Functionality
Next remove the coupling between:
(2) Memory Card(s) to (6) Electrical Power
By making the constraint that memory cards must not cause the processor to require more AC Power.
(That is, the processor AC power is already sized for the maximum number of memory cards possible.
(Which is likely to be the case anyway, so is not a serious constraint) Running the tool again, shows no
change other than these couplings.
Next make the constraint that all available processors must be fast enough to support all available
Operating Systems.
This removes the coupling:
(6) Operating System to (1) Processing Speed
And the constraint that any commercial monitor will satisfy the operator input/output requirement, which
removes the coupling:
(9) Monitors to (7) Operator Input/Output
Figure 15 shows the result of running the tool with these couplings removed. Now only three remain
above the diagonal.
B-40
(2)
Mem
ory
Card
(s)
(3)
Ma
ss S
tora
ge
(1)
Pro
cessor(
s)
(8)
Cabin
et(
s)
& F
urn
iture
(4)
Sig
na
l P
roce
sso
r(s)
(9)
Monitor(
s)
(10)
MM
IH H
W, In
clu
din
g K
eyboard
(s)
(7)
Applic
ation S
oftw
are
(5)
Sensors
(6)
Op
era
tin
g S
yste
m
(3) Memory Read/Write Speed & Capacity x
(2) Disk Read/Write Speed & Capacity x
(1) Processing Speed x
(5) Physical Space x
(6) Electrical Power x x x x
(11) Ergonomics x x x
(7) Operator Input/Output x x
(10) Application SW Functionality x x x x x
(4) Sensor Functionality x x x
(12) Operating System Functionality x x x x Figure 15. Fourth Pass
Next, remove the coupling between Application SW and Sensor Functionality. This constrains the
possible sensors to those that do not require control or configuration by the application SW:
B-41
(7) Application Software to (4) Sensor Functionality
Now we also realize that constrained the Mass Storage and Memory Capacity requirements to support all
Operating Systems and Processors; we missed two other couplings that should have been removed:
(2) Memory Card(s) to (12) Operating System Functionality
(3) Mass Storage to (12) Operating System Functionality
Figure 16 shows the result of running the tool after removing these couplings.
B-42
(2)
Mem
ory
Card
(s)
(3)
Ma
ss S
tora
ge
(1)
Pro
cessor(
s)
(8)
Cabin
et(
s)
& F
urn
iture
(4)
Sig
na
l P
roce
sso
r(s)
(7)
Applic
ation S
oftw
are
(10)
MM
IH H
W, In
clu
din
g K
eyboard
(s)
(5)
Sensors
(9)
Monitor(
s)
(6)
Op
era
tin
g S
yste
m
(3) Memory Read/Write Speed & Capacity x
(2) Disk Read/Write Speed & Capacity x
(1) Processing Speed x
(5) Physical Space x
(10) Application SW Functionality x x x x x
(7) Operator Input/Output x x
(4) Sensor Functionality x x
(11) Ergonomics x x x
(12) Operating System Functionality x x
(6) Electrical Power x x x x Figure 16. Fifth Pass
B-43
This looks worse; but remember the tool is sorting based on weighted sums, so even though there are
more couplings above the diagonal, over-all, the matrix is closer to lower triangular form. Next, add the
constraint that Operator Input/Output will be whatever it can be with Application Software selected based
on other requirements. That constraint removes the coupling:
(7) Application Software to (7) Operator Input/Output
Figure 17 shows the result. Note that even though it looked like we took a step backwards last time,
removing this coupling allows the matrix to sort totally to lower triangular form.
(2)
Mem
ory
Card
(s)
(3)
Ma
ss S
tora
ge
(1)
Pro
cessor(
s)
(8)
Cabin
et(
s)
& F
urn
iture
(10)
MM
IH H
W, In
clu
din
g K
eyboard
(s)
(9)
Monitor(
s)
(4)
Sig
na
l P
roce
sso
r(s)
(6)
Op
era
tin
g S
yste
m
(5)
Sensors
(7)
Applic
ation S
oftw
are
(3) Memory Read/Write Speed & Capacity x
(2) Disk Read/Write Speed & Capacity x
(1) Processing Speed x
(5) Physical Space x
(7) Operator Input/Output x
(11) Ergonomics x x x
(6) Electrical Power x x x x
(12) Operating System Functionality x x
(4) Sensor Functionality x x
(10) Application SW Functionality x x x x x Figure 17. Fifth Pass
B-44
Now look at the form of the matrix, and see if any of the couplings that were removed earlier are now
below the diagonal. Any that are can be put back in, and the corresponding constraints removed. The
following couplings can be put back in:
(1) Processor(s) to (5) Physical Space
(3) Mass Storage to (5) Physical Space
(6) Operating System to (10) Application SW Functionality
(2) Memory Card(s) to (6) Electrical Power
(2) Memory Card(s) to (12) Operating System Functionality
(3) Mass Storage to (12) Operating System Functionality
Putting these couplings back in allows removal or modification of the following constraints:
1) All equipment must fit within the cabinets & furniture
2) The memory capacity requirement will support all possible Processors & Operating Systems
3) The Mass Storage capacity requirement will support all possible Processors & Operating
Systems
4) Installing Memory Card(s) will not cause the processor to draw more power than it’s nominal
rating
B-45
The remaining couplings that are above the diagonal are:
(4) Signal Processor(s) to (5) Physical Space
(9) Monitor(s) to (5) Physical Space
(5) Sensors to (10) Application SW Functionality
(1) Processor(s) to (3) Memory Read/Write Speed & Capacity
(1) Processor(s) to (2) Disk Read/Write Speed & Capacity
(6) Operating System to (2) Disk Read/Write Speed & Capacity
(6) Operating System to (3) Memory Read/Write Speed & Capacity
(6) Operating System to (1) Processing Speed
(9) Monitors to (7) Operator Input/Output
(7) Application Software to (4) Sensor Functionality
(7) Application Software to (7) Operator Input/Output
Now try adding each of these back in, and re-running the tool. This time, choosing the couplings to try
adding back in based on the constraints you would most like to remove; because each one that is added
back in limits how many more can be added and still allow sorting to lower triangular form. The
following Couplings could be sorted back in:
(1) Processor(s) to (3) Memory Read/Write Speed & Capacity
(1) Processor(s) to (2) Disk Read/Write Speed & Capacity
(7) Application Software to (4) Sensor Functionality
B-46
Which allows removal or modification of the following additional constraint:
1) Sensors may not require control or configuration by the application SW
The final matrix is shown in Figure 18:
(1)
Pro
cessor(
s)
(3)
Ma
ss S
tora
ge
(2)
Mem
ory
Card
(s)
(10)
MM
IH H
W, In
clu
din
g K
eyboard
(s)
(6)
Op
era
tin
g S
yste
m
(8)
Cabin
et(
s)
& F
urn
iture
(9)
Monitor(
s)
(4)
Sig
na
l P
roce
sso
r(s)
(7)
Applic
ation S
oftw
are
(5)
Sensors
(1) Processing Speed x
(2) Disk Read/Write Speed & Capacity x x
(3) Memory Read/Write Speed & Capacity x x
(7) Operator Input/Output x
(12) Operating System Functionality x x x x
(5) Physical Space x x x
(11) Ergonomics x x x
(6) Electrical Power x x x x x
(10) Application SW Functionality x x x x x x
(4) Sensor Functionality x x x Figure 18. Final Matrix
B-47
The couplings that were removed and could not be put back in:
(4) Signal Processor(s) to (5) Physical Space
(9) Monitor(s) to (5) Physical Space
(5) Sensors to (10) Application SW Functionality
(6) Operating System to (2) Disk Read/Write Speed & Capacity
(6) Operating System to (3) Memory Read/Write Speed & Capacity
(6) Operating System to (10) Application SW Functionality
(6) Operating System to (1) Processing Speed
(9) Monitors to (7) Operator Input/Output
(7) Application Software to (7) Operator Input/Output
Which leaves the following final constraints:
1) All equipment must fit within the cabinets & furniture, except Processors and Mass Storage)
2) Mass Storage Speed and Capacity requirements must support all Operating Systems
3) Memory Speed and Capacity requirements must support all Operating Systems
4) All possible Application SW will Support all Operating Systems
5) All possible processor speeds will support all possible Operating Systems
6) Any commercial monitor will support the Operator Input/Output requirement
7) Sensors will not require Application SW control or configuration
8) Operator Input/Output requirements will be modified as necessary to support whatever
Application SW is selected
B-48
The final selection order resulting from this analysis is:
1) (1) Processor(s)
2) (3) Mass Storage
3) (2) Memory Card(s)
4) (10) MMIH HW, Including Keyboard(s)
5) (6) Operating System
6) (8) Cabinet(s) & Furniture
7) (9) Monitor(s)
8) (4) Signal Processor(s)
9) (7) Application Software
10) (5) Sensors
B-49
The order at the top would not surprise anyone with COTS integration experience. But it might come as
somewhat of a surprise that Cabinets & Furniture should be selected before COTS Application SW. It is
important at this point to note that this is not necessarily the only possible order; but it is an order that is
guaranteed to not cause any late changes to the COTS selection process driven by prior selections. For
example, the selection of Application Software will not force a different choice of Processor, Mass
Storage, or Memory Card(s). It might be possible that Cabinets & Furniture could be selected last, and
still not cause any reselection. If that became an issue, one could try manually re-sorting the matrix to see
if it is possible to put Cabinets & Furniture on the bottom row while still maintaining a lower triangular
matrix. In this case, it is not readily apparent that such a rearrangement is possible, so you would just
have to resign yourself to defining Cabinets & Furniture first. (Or look at eliminating even more
coupling)
This process is somewhat iterative; but not nearly as costly as restocking and reordering previously
purchased equipment, and it correctly executed, it guarantees that will not happen.
B-50
CHAPTER VII Product Design and Development
7 .1 Gat e 7 - P re l iminary Design Rev iew
“It’s just COTS”
Those are the words that are usually heard when a manager or engineer is about to skip over part of the
normal engineering discipline because COTS is being used vs. a full-up custom design. But consider:
Engineering
The process of devising a system, component, or process to meet desired needs. It is a decision-
making process (often iterative) in which the basic sciences, mathematics, and engineering
sciences are applied to convert resources optimally to meet a stated objective.
– Accreditation Board for Engineering and Technology
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
This is a good definition of “engineering”; and nothing in this definition (or any other credible variation)
will give you a reason to exclude selection of COTS components, or configuring them into a deliverable
system. Much of the same rigor and discipline is needed for a COTS effort as for a custom design. A
major difficulty is that there is an expectation by the customer and by management that a COTS design
will result in significant cost and schedule savings. This can be true; but you don’t get the savings by
cutting engineering. (See Section 4.1) There may even be cases, when you consider the entire lifecycle
cost, where a custom design will actually cost less. (Gasp!)
B-51
Systems Engineering integrates all the disciplines and specialty groups into a team effort forming a
structured development process that proceeds from concept to production to operation. Systems
Engineering considers both the business and the technical needs of all customers with the goal of
providing a quality product that meets the user needs.
According to INCOSE, the basic Systems
Engineering process tasks are:
1) Define the System Objectives
2) Establish the Functionality
3) Establish the Performance Requirements
4) Evolve Design and Operation Concepts
5) Select a Baseline
6) Verify that the Baseline Meets Requirements
7) Validate that the Baseline Satisfies the User
8) Iterate the Process through Lower Levels
(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm
The only thing on this list that might possibly be totally cut without adding all of the same risks you
would incur by cutting it from a custom design would be item 8). (But even this one might be applicable
for a large “System of Systems” type effort, so it should not be thrown out without thinking) The savings
for using COTS vs. custom designs should generally not be expected on the Systems Engineering side;
but on the Hardware and Software Development side. In fact, you can expect some additional cost in 1),
2), 3), and 4); because it is highly likely that all three of these tasks will have to be iterated with the
customer, as COTS options and capabilities become clearer as applied to the task at hand. And even in
the development area, the savings won’t come from cutting engineering discipline; it comes from labor
and material savings. The only additional checkpoint for a COTS design is to make certain no
B-52
preliminary design steps are skipped because “it’s just COTS”. If a step is trivial for COTS, then
performing it and documenting it is also trivial; so do it. This is another case where a typical COTS
project would have multiple PDRs. The example COTS project in Chapter 5 has 3; Gates 7, 7a, and 7b.
B-53
7.2 Gat e 8 - Cr it i cal Des ign Review
Detailed design is an area where one could reasonably expect to see some significant COTS
savings in a COTS development vs. a custom development. Where in a full up custom development, you
would now be designing “black boxes” whose functionality was defined at the Preliminary Design level;
for COTS many or all of these boxes may now be purchased items. The best philosophy is to not skip
any design steps. Again, skip no steps due do a predominately COTS implementation. If the use of
COTS makes a step trivial; it should not take much time to just do it; vs. skipping over it. This is another
case where a typical COTS project would have multiple CDRs. The example COTS project in Chapter 5
has 3; Gates 8, 8a, and 8b.
B-54
CHAPTER VIII System Integration, Verification, and Validation
8 .1 Gat e 9 - Test /Sh ip Read iness Review
A typical COTS program will only have one “formal” Test/Ship Readiness review. But holding
internal equivalents to Gate 9 at the end of each spiral is a good idea. These could be just quick sit-down
meetings with all stakeholders to ensure that everything is set to continue to the next spiral. The “formal”
review is the one the customer must buy off on before proceeding to formal testing. (Note that formal
testing may be performed before delivery, after delivery, or both; depending on the nature of the system
being delivered and customer requirements.) There are no additional checkpoints for COTS; but it is
important here too that none of the normal steps are skipped because “It’s just COTS”. It still has to meet
requirements, and if it does not get tested, how do you know it does? The example COTS development in
Chapter 5 only has a single “formal” Gate 9. Whether to have Readiness reviews between each spiral
should be established at Gate 5.
B-55
CHAPTER IX Production and Deployment
9 .1 COTS "Product ion"
This phase is normally deemed "not applicable" to COTS, and tailored out. That is a big mistake.
Unless you are paying a subcontractor for a turn-key job, you are still going to have to deal with some
sort of "production", and you will certainly have to deal with "deployment". A significant twist is that for
a majority COTS program, this gate should come before Gate 9; as there is generally no "prototype", and
most customers expect the first developed system to be a deliverable. This presents a different set of
problems in itself. For many custom designs, prototype equipment remains at the factory for
enhancement, or use in solving problems reported from the field. But for most COTS implementations;
this is not an option. Even if the development system was procured separately, and remains at the factory;
thanks to Moore’s Law, that equipment will quickly become obsolete. So most COTS developments do
not have the advantage of having duplicate equipment to work field maintenance problems. This is
another key reason why careful consideration should be given to total life cycle when making a COTS vs.
custom decision.
9.2 Gat e 10 - Product ion Read iness Review
This review should not be skipped; but it could be combined with Gate 9. A key COTS
checkpoint for this review:
Is the program prepared (or required) to support DR work-off, Operation, Maintenance, etc.; with the
equipment being delivered no longer at the factory?
The example COTS integration program in Chapter 5 has one Gate 10, just prior to Gate 9.
B-56
CHAPTER X Operations and Support
10.1 Rem ote Support
All of the same issues exist with COTS maintenance of COTS equipment as with custom
equipment. In fact, once fielded, the user can only tell that the equipment meets its requirements or not,
and is reliable, or not; COTS vs. Custom becomes irrelevant. The key difference from the standpoint of
factory support is that you are going to have to rely heavily on the suppliers you purchased the equipment
from; which could end up increasing the maintenance cost. (A major reason the total lifecycle cost should
be considered when making the initial COTS/Custom trade.)
B-57
CHAPTER XI Program Closure
11.1 Gate 11 - Transit ion and Closure Review
Shutdown of a COTS program is another area where considerable savings can be achieved.
There is typically no large investment in design tools, hence very little cost in disposition of tools no
longer needed. A critical area to address in a COTS system is:
Does the O&M contractor have all of the technical information, and all of the current vendor
contact information that they will need to maintain and operate the system?
Tech manuals and maintenance manuals that are “automatically” generated by following the
custom design processes may not exist for COTS equipment.
B-58
CHAPTER XII SUMMARY AND CONCLUSIONS
12.1 Summary and Conc lusions
12.1.1 COTS & IPDS
Heavy use of COTS equipment and software in place of custom designs can save the customer
schedule and cost. But it is important to make sure the savings are coming from the use of the COTS, and
not from cutting corners in the design process. Cutting those same corners would save exactly the same
amount of time in a custom design; and in a COTS design, cutting those corners adds exactly the same
amount of risk to the program it would add in a custom design. Key elements of IPDS to retain:
1) Gate Reviews – There is no good reason to “tailor out” any of the IPDS Gates just because
you are running a “COTS” project. Combine them, reduce the formality, shorten them, etc.;
but address every element of every gate.
2) Design Discipline – If the design activity (Requirements development, test, etc.) is trivial; the
amount of time to execute the process will also be trivial; so do it.
12.1.1 COTS & Axiomat ic Design
All the concepts advanced by all the advocates of Axiomatic Design do not have to be embraced,
or even accepted, to see a significant benefit by applying the “Independence Axiom” to COTS design. I
have seen no other way to generate a sequence of defining COTS elements that is guaranteed to result in
no re-specification and re-procurement of some equipment mid-way through a COTS integration project.
The small amount of requirement analysis time spent is well worth the assurance it provides.
B-59
REFERENCES
(TBD) Ertas, A. 1999, “How to deal with Complexity,” Journal of Integrated Design and Process Science, Vol. 5, No. 2, pp.23-36. Hubka, V. and Eder, W.E., Theory of Technical Systems, 1998, Berlin, Springer-Verlag, 2nd Ed. (Ref. MIT OpenCourseWare, Introduction, Systems Engineering)
http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm Tech University Masters Program, Systems Engineering Principles, ME 5354
Top Related