Implementing a COTS System Design Using Raytheon IPDS and ...

Implementing a COTS System Design Using Raytheon IPDS and Axiomatic Design

By

Carl McGaha

A MASTER OF ENGINEERING REPORT

Submitted to the College of Engineering at

Texas Tech University in

Partial Fulfillment of

The Requirements for the

Degree of

MASTER OF ENGINEERING

Approved

______________________________________ Dr. A. Ertas

______________________________________ Dr. T. Maxwell

______________________________________ Dr. M. Tanik

______________________________________ Dr. E.W. Kiesling

October 15, 2005

B-1

ACKNOWLEDGEMENTS

I would like to thank all of the Texas Tech staff and guest speakers who contributed to this

program. It was obvious that many of the instructors and guest speakers put forth efforts well beyond

what would have been required or expected. I would also like to express my gratitude to Raytheon for

allowing me to participate in this program.

B-2

TABLE OF CONTENTS

ACKNOWLEDGEMENTS......................................................................................................1 DISCLAIMER ..........................................................................................................................4

ABSTRACT ..............................................................................................................................5 LIST OF FIGURES ..................................................................................................................6

CHAPTER I INTRODUCTION ....................................................................................................................7

CHAPTER II BACKGROUND .....................................................................................................................10 2.1 IPD and Raytheon IPDS......................................................................................................10

2.1.1 The Ten Tenets of IPDS..........................................................................................10 2.2 COTS..................................................................................................................................11 CHAPTER III THE DEFINITION OF COTS ...............................................................................................12 3.1 Non Trivial .........................................................................................................................12 CHAPTER IV BUSINESS STRATEGY EXECUTION ................................................................................13 4.1 Gate 4 - Proposal Review....................................................................................................13 4.2 Who Is The “Customer”? ....................................................................................................13

4.2 COTS vs. Custom Trade ............................................................................................13 CHAPTER V PROJECT PLANNING, MANAGEMENT AND CONTROL .............................................15 5.1 Gate 5 - Startup Review ......................................................................................................15 5.2 Development Cycle Model..................................................................................................15 CHAPTER VI REQUIREMENTS AND ARCHITECTURE DEVELOPMENT.........................................19 6.1 Gate 6 - System Functional Review (Requirements Review) ...............................................19 6.2 Axiomatic Design ...............................................................................................................19

6.2.1 Basic Principles ......................................................................................................19 6.2.2 An Axiomatic Design Tool .....................................................................................24 6.2.3 A Simple COTS System Design..............................................................................26

CHAPTER VII PRODUCT DESIGN AND DEVELOPMENT ......................................................................50 7.1 Gate 7 - Preliminary Design Review ...................................................................................50 7.2 Gate 8 - Critical Design Review..........................................................................................53 CHAPTER VIII SYSTEM INTEGRATION, VERIFICATION, AND VALIDATION..................................54 8.1 Gate 9 - Test/Ship Readiness Review ..................................................................................54 CHAPTER IX PRODUCTION AND DEPLOYMENT .................................................................................55

B-3

9.1 COTS "Production".............................................................................................................55 9.2 Gate 10 - Production Readiness Review ..............................................................................55 CHAPTER X OPERATIONS AND SUPPORT............................................................................................56 10.1 Remote Support ................................................................................................................56 CHAPTER XI PROGRAM CLOSURE .........................................................................................................57 11.1 Gate 11 - Transition and Closure Review ..........................................................................57 CHAPTER XII SUMMARY AND CONCLUSIONS ......................................................................................58 12.1 Summary and Conclusions ................................................................................................58

12.1.1 COTS & IPDS ......................................................................................................58 12.1.1 COTS & Axiomatic Design...................................................................................58

REFERENCES .......................................................................................................................59

B-4

DISCLAIMER

The opinions expressed in this report are strictly those of the author and are not necessarily those

of Raytheon, Texas Tech University, nor any U.S. Government agency. While it is my sincere belief that

the recommendations made in this report will result in a tangible benefit to programs that apply them, I

make no guarantee of program success or profit.

B-5

ABSTRACT

The purpose of this report is to create a set of recommended processes and strategies that can be

incorporated into Raytheon IPDS to result in a more efficient implementation of systems developed with

a majority of COTS (Commercial Off The Shelf) equipment and software. The information included

comes from the basic sources:

1) The Texas Tech Masters of Engineering curriculum (Incorporate course material appropriate

for COTS system development)

2) 25 years of personal experience (Captured lessons learned from having seen it done wrong

and right many times over)

3) Raytheon IPDS

4) Other sources, as identified.

B-6

LIST OF FIGURES

Figure 1. Raytheon IPDS, Top Level View. 8

Figure 2. Simple COTS Based “Spiral” Development Model 17

Figure 3. Simple COTS Based “Waterfall” Development Model 18

Figure 4. “Coupling” in Axiomatic Design 21

Figure 5. “Zig-Zagging” in Axiomatic Design 22

Figure 6. The “Optimizer” Sorting Algorithm 24

Figure 7. Simple COTS Based System Diagram 27

Figure 8. Simple Initial Design Matrix 28

Figure 9. Initial Data Entry Into Tool 29

Figure 10. Data Entry After Combining Rows & Columns 31

Figure 11. First Pass 32

Figure 12. Second Pass 34

Figure 13. Third Pass 36

Figure 14. Third Pass 38

Figure 15. Fourth Pass 40

Figure 16. Fifth Pass 42

Figure 17. Fifth Pass 43

Figure 18. Final Matrix 46

B-7

CHAPTER I INTRODUCTION

The first chapter of this paper focuses on the definition of "COTS". The order of the remaining

chapters follows the top level organization of IPDS, shown in Figure 1, with one exception. In order to

follow a more chronological flow, the Project Planning Management and Control stage is addressed

twice; once in chapter 5 for Startup (Gate 5), and again in Chapter 11 for Transition & Closure (Gate 11).

The Project Management and Control stage of IPDS spans the entire project, but this report focuses on the

Stage checkpoints; referred to in IPDS terminology as Gates. Gates 1-3 are not discussed; for two

reasons:

1) These gates require very little input from Engineering

2) There should be no separate issues for COTS in these early gates.

In each chapter, the special strategies, processes, and considerations for COTS development

related to that stage of IPDS will be identified and discussed.

B-8

Program

Execution

Gates 5 -11

Business Business

Decision Decision

Gates 1Gates 1 --44

2 - Project Planning, Management and Control

3 -

Requirements

andArchitecture

Development

5 -

System Integration,Verification

and Validation

4 -Product Design

and

Development

Planning 6 - Production and Deployment

Planning 7 - Operations and Support

1 -

Business Strategy

Planning/Execution

Strategic

PlanningCapture/Proposal Development

1 2 3 4

5

6

7 8

109

11

= GATE

Opportunity Review (Interest/No Interest)

Win Strategy Review (Pursue/No Pursue)

Pre-Proposal Readiness Review (Bid/No Bid)

Proposal Review (Bid Approval)

Internal Preliminary

Design Review

Transition and

Closure ReviewProject Start -Up Review

Internal System Functional Review

Internal Critical

Design ReviewInternal Test/Ship

Readiness Review

Internal Production

Readiness Review

Program

Execution

Gates 5 -11

Business Business

Decision Decision

Gates 1Gates 1 --44

2 - Project Planning, Management and Control

3 -

Requirements

andArchitecture

Development

5 -

System Integration,Verification

and Validation

4 -Product Design

and

Development

Planning 6 - Production and DeploymentPlanning 6 - Production and Deployment

Planning 7 - Operations and SupportPlanning 7 - Operations and Support

1 -

Business Strategy

Planning/Execution

Strategic


1 -

Business Strategy

Planning/Execution

Strategic


1 2 3 4

5

6

7 8

109

11

= GATE

Opportunity Review (Interest/No Interest)

Win Strategy Review (Pursue/No Pursue)

Pre-Proposal Readiness Review (Bid/No Bid)

Proposal Review (Bid Approval)

Internal Preliminary

Design Review

Transition and

Closure ReviewProject Start -Up Review

Internal System Functional Review

Internal Critical

Design ReviewInternal Test/Ship

Readiness Review

Internal Production

Readiness Review

Figure 1. Raytheon IPDS, Top Level View.

B-9

I will not attempt to connect information in this report to lower levels of the Raytheon IPDS

structure for two reasons:

1) To keep the report at a non-company-proprietary level

2) Because a new release of IPDS is in work that will be changing a lot of the structure below the top

level. This version will likely be released before my report is complete, so direct linkage to lower

levels of the current version would not be of any practical use.

B-10

CHAPTER II BACKGROUND

2.1 IPD and Raytheon IP DS

IPD (Integrated Produce Development) is a process framework originated in MILSTD-499

(USAF) 17 July 1969, encompassing the elements of TQM (Total Quality Management). A version of

IPD adopted by Raytheon is IPDP (Integrated Product Development Process). This is the detailed

process that when followed implements the best practices of Legacy Raytheon and various acquired

legacy companies. IPDS (Integrated Product development Process), is the policy umbrella that requires

use of IPDP on all Raytheon programs and projects. For the remainder of this paper, I will refer only to

IPDS; which has become standard Raytheon terminology.

2.1.1 The Ten Tenet s of IP DS

The Ten Tenets of IPDS (Texas Tech University Masters Program, Systems Engineering Principles, ME 5354) 1. Customer Focus

2. Integrated Management Tools.

(Identify and Satisfy Customer Needs, Better, Seamless Planning and Tracking of Resources, Cost,

and Faster, Cheaper Schedule)

3. Concurrent Development of Product and Processes

4. Proactive Identification and Management of Risk

(Product Focus & Process Understanding Management of Technical, Cost, and Schedule Risk)

5. Event Driven Scheduling

6. Early and Continuous Life Cycle Planning

(Incrementally Demonstrate Product Maturity Maximize Flexibility for Optimization and Use of

Alternative Approaches Prior to Beginning Follow-on Activities)

B-11

7. Multi-Disciplinary Teamwork and Communication

(Challenge Requirements and Use Commercial Specs and Standards)

8. The Right People, At The Right Place, At The Right Time

9. Empowerment

10. Encourage Robust Design and Improved Process Capability

(At Lowest Possible Level, Resources Allocated)D

2.2 COTS

COTS (Commercial Of The Shelf) refers not only to pieces of equipment, but to a trend by

government customers towards use of COTS equipment vs. custom design. For the remainder of this

paper, when I refer to COTS, I will be referring to equipment. If I am referring to the use of COTS in a

program, I will use the term “COTS Program”.

B-12

CHAPTER III The Definition of COTS

3.1 Non Trivial

The purist definition: "Commercial Off The Shelf" means simply that it is not COTS unless it is

"on the shelf" at the time the order is placed. This is frequently not the case. Many times, an acquisition

activity will define COTS as simplistically as "Not Built In House"; and an item designated "COTS" will

go through a full design cycle at a subcontractor. On the other hand, high-dollar computer equipment is

rarely "stocked"; so it is likely that a lot of COTS will not start getting built until after an order is placed.

The definition this paper will use is that an item is COTS if it has already been built and delivered prior to

placing the order. Other cases where purchased equipment is often called COTS will be discussed, and

identified as non-COTS or as a special variety of COTS. One such special case would be First Article

COTS. In some respects, this should be treated the same as custom. A typical COTS based system

development will consist of a combination of COTS items that really are "off the shelf", COTS items that

have been built before, but are not currently "stocked", and items that are nominally "COTS" but have

never been built before. It is important to know up front:

1) What your management thinks "COTS" means

2) What your customer thinks "COTS" means

3) What you think "COTS" means

All three may be different. The point is not to define a “right” definition of COTS. Any definition that

you, your management, and your customer all agree to is “right”; but don’t just assume your idea is the

same as everyone else’s.

B-13

CHAPTER IV Business Strategy Execution

4 .1 Gat e 4 - P roposal Review

This is the initial gate that requires significant input from Engineering. There are a number of

special considerations for a proposal for a system incorporating a large amount of COTS. Do not skip

any normal proposal review steps for an “All COTS” design.

4.2 Who Is The “Custom er”?

This is not obvious on many government contracts. It is not unusual for there to be an End User

separate from the organization providing the funding. So you must balance Customer "needs" vs. End

User "needs" vs. Customer requirements vs. End User requirements. If you do this right, there will

probably be a couple of iterations of customer SOW. If the Customer is not the End User; the Customer

probably does not have a full understanding of the End User's real needs. (Government agencies are not

known for their outstanding communications ability.)

4.2 COTS vs. Cust om Trade

A comparison of the complete cost of COTS is rarely done. A typical trade goes something like:

Catalog Cost of COTS < Total Cost of Custom; therefore, use COTS

The normally forgotten or underestimated list:

Procurement Cost

COTS procurement has a cost associated with it, in some cases a higher cost than custom

HW; especially if procurement specifications or modifications are required.

Development Monitoring Cost (Pseudo-COTS)

This is an area where it is especially important to make sure there is a clear agreement

between Management, Customers, and Engineering on what “COTS” is. If an item is being

built to specification for the first time, the only cost delta from custom will be in labor rates,

B-14

or cutting corners. The former can be a legitimate cost savings; the latter can end up being

more expensive.

O&M Cost

Be sure to consider the total lifecycle cost of COTS vs. Custom; and not just the initial

purchase or development cost. COTS with poor reliability can be cheaper to buy, but end up

being more expensive in the long run.

Customer Encouraged

Is the customer specifying, or encouraging, an all COTS system? If so, be sure you

know what his definition of "COTS" is, what he is expecting to save on cost & schedule; and

what he is willing to sacrifice in quality and performance. The higher risk is when the

customer is encouraging a particular COTS purchase, but not actually putting his desire in the

contract or SOW (Statement of Work). In that case, you can end up being the bad guy for

specifying COTS; when you were just doing what the customer asked you to.

Specification Cost (Pseudo-COTS)

When the customer’s definition of COTS is “not built here”; you may find yourself in the

position of having to write design specs for “COTS”. Be sure to include that cost in your

comparison of the cost of COTS vs. custom. This is not as arbitrary as it sounds. Custom

HW built on the same Cost Plus contract as a large systems integration is also subject to the

over-all Cost Plus contract. This means that a custom item purchased on a subcontract can be

Fixed Price, while the identical item, built per the identical specs and process, but built on the

base contract; is Cost Plus. The end result is lower cost risk to the customer by defining

COTS as “not built by the prime contractor”.

B-15

CHAPTER V Project Planning, Management and Control

5 .1 Gat e 5 - S tartup Review

In general, all IPDS checkpoints required for a custom design should be included for a COTS

design. The argument is often used, “it is so simple, why do I need a Program Management Plan?”. The

answer is that if it is that simple, your PMP will be very short, maybe just a single PowerPoint slide. Be

careful to not allow this sort of “logic” to cut steps out. Each element normally required for a startup

review should be addressed. If a business area has multiple small projects, all following the same general

pattern; create a set of plans and process documents that can be applied to them all. Additional COTS

Checkpoints for a Startup Review:

1) Is “COTS” for this project clearly defined?

2) Are there clear guidelines for COTS vs. custom decisions?

5.2 Development Cycle Model

The development cycle model for a COTS program (or any other), should be clearly identified at

the Startup Review. The development cycle with a majority of COTS is nearly always a "spiral" type

model, whether the program planners admit it or not. Plan for it up front or you will lose a lot of the

COTS savings. I have lost count how many times I have had to explain at a preliminary design review

why equipment was already on order before the design was complete. What was really happening was

that we were defining requirements and procuring equipment on a "spiral" model; but customer and

management expectations were that reviews would be defined and held per a traditional "waterfall"

model. This creates a no-win situation. You either execute per an undocumented "spiral" model, which

makes you appear to be executing design and development steps out of sequence; or you force the

development to fit a traditional "waterfall" model, and fail to deliver the cost and schedule savings that

are expected from a mostly-COTS solution. Figure 2 is a typical COTS/Custom Development model;

with all COTS HW, some COTS SW, and some custom SW. Figure 3 is a “waterfall” version of the

B-16

same model, with some more detail shown. It is not a true “waterfall” model, because after the first

spiral, Requirements, Design, and Development can all be proceeding in parallel.

B-17

sin(B)= x/R

sin(B)*R= x

cos(B)= y/R

cos(B)*R= y

Gate 5

Gate 6

Gate 7

Gate 8

Gate 5a

Gate 6a

Gate 7a

Gate 8a

Gate 5c

Gate 6c

Gate 7c

Gate 8c

Gate 9

Gate 10

Gate 11

Figure 2. Simple COTS Based “Spiral” Development Model

B-18

Spiral # Gate Gate Title Activity

A Gate 5 Start-Up Gate Review Review Program Plans &

Gate Checklists for Spiral A

A Develop Requirements

A Gate 6 System Functional

(Requirements and

Concept) Gate

Review

Review requirements for SW

Development Environment,

including preliminary

requirements for deliverable

HW & SW, definitized to the

point necessary to

completely define the

development environment.

A Preliminary Design

A Gate 7 Preliminary Design

(Architecture) Gate

Review

Review Development

Environment Architecture

Design

A Development Environment

Detailed Design

A Program Planning for Spiral

B

A Gate 8 Critical Design

(Detailed Design and

Build Readiness)

Gate Review

Review Development

Environment Detailed

Design.

A Place order for Development

Environment COTS HW &

SW

Spiral # Gate Gate Title

B Gate 5a Start-Up Gate Review Review Program Plans &

Gate Checklists for Spiral B.

Review Status of Spiral A to

ensure it is far enough along

for Spiral B to begin.

B Finish Development of

Deliverable HW

Requirements

B Gate 6a System Functional

(Requirements and

Concept) Gate

Review

Review Deliverable HW

Requirements

B Preliminary Design

B Gate 7b Preliminary Design

(Architecture) Gate

Review


Architecture Design

B Deliverable HW Detailed

Design

B Program Planning for Spiral

C

B Gate 8b Critical Design


Build Readiness)

Gate Review


Detailed Design.

B Place Order for Deliverable

COTS HW & SW

Spiral # Gate Gate Title

C Gate 5c Start-Up Gate Review Review Program Plans &

Gate Checklists for Spiral C.

Review Status of Spiral B to

ensure it is far enough along

for Spiral C to begin.

C Develop Deliverable Custom

SW Requirements

C Gate 6c System Functional

(Requirements and

Concept) Gate

Review

Review Deliverable SW

Requirements

C Preliminary Design

C Gate 7c Preliminary Design

(Architecture) Gate

Review

Review Deliverable Custom

SW Preliminary Design

C Receive COTS HW & SW for

Development Environment

C Gate 8c Critical Design


Build Readiness)

Gate Review

Review Deliverable Custom

SW Detailed Design.

C Develop Custom Software

C Receive COTS HW & SW for

Deliverable HW

C Test Custom SW in

Deliverable System (Factory

Acceptance Testing)

C Gate 10 Production

Readiness Gate

Review

Is the program prepared to

support delivery and

operation without the

equipment that is being

delivered? This is also where

you would review production

planning and resources for

additional "Builds"; which

could be Software only.

C Gate 9 Test/Ship Readiness

Gate Review

Review Factory Acceptance

Test Results, Go/No Go

Decision for Ship to

Customer

C Test Delivered System

C Gate 11 Transition and

Closure Gate Review

Disposition Development HW

& SW, Contract Closout

Figure 3. Simple COTS Based “Waterfall” Development Model

A typical COTS program; three spirals:

Spiral A: Development Environment Development

Sprial B: Deliverable COTS HW & SW Development

Spiral C: Deliverable Custom SW Development

B-19

CHAPTER VI Requirements and Architecture Development

6 .1 Gat e 6 - System F unct ional Review (Requirem ent s Rev iew)

A COTS program will probably have multiple Functional Requirements Reviews; one for each

spiral. The example COTS Program shown in the Chapter 5 development model (Figures 2 and 3) has 3

Requirements reviews (Gates 6, 6a, and 6b). Additional checkpoint to consider for a COTS program:

1) Have requirements been adequately defined for suppliers & subcontractors?

6.2 Axiom ati c Design

Certain Axiomatic Design Principles are very valuable for COTS based requirements definition;

even if you don't buy the whole "Axiomatic Design" premise. (It is important to note that some the

concepts are useful, even if you don't buy the over-all premise; because "Axiomatic Design" is currently

far from universally accepted.)

6.2.1 Basi c Pr inciples

Claims Made by Nam Suh Regarding Axiomatic Design

• “A general theory for system design is presented”

• “The theory is applicable to … large systems, software systems, organizations…”

• “The flow diagram … can be used for many different tasks: design, construction, operation,

modification, … maintenance … diagnosis …, and for archival documentation.”

• “Design axioms were found to improve all designs without exceptions or counter-examples…

When counter-examples or exceptions are proposed, the author always found flaws in the

arguments.”

(Ref. MIT OpenCourseWare, Introduction, Systems Engineering)

http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm

B-20

What are Axioms?

• “Primitive propositions whose truth is known immediately without the use of deduction”

- Cambridge Dictionary of Philosophy

• “A statement that is stipulated to be true for the purpose of constructing a theory”

- Harper Collins Dictionary of Mathematics

• “Fundamental truths that are always observed to be valid and for which there are no

counterexamples or exceptions”

- Suh, The Principles of Design

• “Axioms are posited as accepted truths, and a system of logic is built around them”

– Hazelrigg, “An Axiomatic Framework for Engineering Design”

Benefit of Separate Domains

• Customer Needs are stated in the customer’s language

• Functional Requirements and Constraints are determined to satisfy Customer Needs

• “The FRs must be determined in a solution neutral environment” (or, in other words, say

“what” not “how”)

– BAD = the adhesive should not peel

– BETTER = the attachment should hold under the following loading conditions

• Keeping FRs in solution neutral terms prevents inadvertent “lock in” to specific modes of

solution



B-21

Suh’s Design Axioms

• The Independence Axiom

– Maintain the independence of the functional requirements.

• The Information Axiom

– Minimize the information content.



The second axiom is just another way of saying that, all other things being equal, simpler is

better; common knowledge for any experienced engineer. This paper will focus on application of the first

Axiom to COTS integration.

An “uncoupled” design is a design in which no requirement has any coupling to any other

requirement. A “decoupled” design is one in which there is some coupling, but in which the matrix can

be sorted to lower triangular form. (See Figure 4)

Note that the decoupled design can be either a lower or upper triangular matrix. This examples

paper uses lower triangular form, because that sets the order of COTS component selection as left to right.

Using an upper triangular form would work equally well, but the order of component selection would be

right to left.

!!!

"

#

$$$

%

&

X

X

X

!!!

"

#

$$$

%

&

XXX

XX

X

!!!

"

#

$$$

%

&

XXX

XXX

XXX

An Uncoupled A Decoupled A Coupled Design Design Design

Figure 4. “Coupling” in Axiomatic Design

B-22

Functional Coupling vs Physical Coupling

Just because a single physical entity carries out multiple functions it does not imply functional

coupling!

Benefits of Uncoupling

• Simpler operation

• More transparent design

• Simpler to change the design

• More parallelism in the design process



The concept of "Zig-Zagging" is mentioned here for completeness. It not addressed in this paper

with respect to COTS integration, because the nature of COTS integration typically restricts the designer

to only top level components.



Figure 5. “Zig-Zagging” in Axiomatic Design

B-23

The Supposedly Dire Consequences of Coupling

• “.. when several functional requirements must be satisfied, designers must develop designs that

have a diagonal or triangular matrix”

• “For a product to be manufacturable, the design matrix of a product, [A] … times the design

matrix for the manufacturing process, [B] … must yield either a diagonal or triangular matrix.

Consequently, when … either [A] or [B] represents a coupled design, the product cannot be

manufactured.” Theorem 9 (Design for Manufacturability)



For COTS Systems, the consequences are slightly different. The components are already

manufactured, so the above statement is not completely applicable. The consequences of a highly

coupled COTS design are that there is no clear order of component definition. That is, if you define the

Operating System first, then the Processor; you might have to go back and pick a different operating

system, because the Processor you selected will not run the Operating System you selected. In practice,

due to restocking charges and other costs of "going back", what happens is that you end up being

unnecessarily constrained in your choice of (for example) Processor, by your previous choice of

Operating System. Application of axiomatic design maximizes your options for COTS selection, and

ensures that those choices only need to be made one time.

The second Axiom is applicable to COTS integration too; but not in the same way as to a custom

design. Application of the second Axiom means that, all other things being equal, choose the COTS that

satisfies all of the requirements, with as few additional "features" that do not satisfy any requirements as

possible.

B-24

6.2.2 An Axiom ati c Design Tool

The Axiomatic design concept applied to COTS development in this paper is the use of Design

Parameter vs. Requirements matrices to analyze requirement coupling. The single most labor intensive

part of this analysis seems to be sorting the matrix to Lower Triangular form. To this end; an Excel tool

has been created to facilitate this “sorting” activity. The “Optimizer” macro assigns weights to each row

and column of the matrix based on the number of ones in a row or column, and the number of ones in

each row or column that intersects a one. For example:

Row

Sums

1 1 2

1 1 1 3

1 1 1 3

1 1 2

1 1 1 1 4

1 1

Column

Sums3 2 1 3 3 1

Figure 6. The “Optimizer” Sorting Algorithm

The weight assigned to the highlighted column is 2 + 3. In addition, each of the column and row

sums is weighted proportionately to how far it is from the lower left hand corner of the matrix. The

matrix is then sorted by the weighted row sums and column sums. After each sort, the macro tests to see

if there are any ones left above the diagonal, and if there are, it sorts again. Since the second level of

weighting changes depending on position; the sort result tends to push the ones towards the lower left.

The macro monitors the running average of the number of ones above the diagonal, and if it does not

change after a fixed number of passes, the macro exits with a warning that a solution may not exist.

B-25

(Unfortunately, I have been unable to find or invent a method of determining in advance whether a matrix

with no obvious error conditions can be sorted to lower triangular form.)

This particular implementation of the macro and worksheet will sort up to a 25 x 25 matrix; but

the algorithms have been tested up to a 100 x 100, and are still fully functional. But Excel runs extremely

slow with a fully populated 100 x 100, and the method used to identify identical rows and columns

sometimes will give false errors, due to the internal resolution of Microsoft Excel (15 significant figures)

The “Optimizer” also provides some basic error checking, as well as the ability to identify highly

coupled row/column elements. Errors that are currently specifically flagged in the macro:

- Two or More Rows or Columns Are Identical; Or A Row or Column Has No Coupling

- There Are Not Enough Coupled Parameters & Requirements to Generate a Lower Triangular

Matrix of This Array Size

- There Are Too Many Coupled Parameters & Requirements to Generate a Lower Triangular

Matrix of This Array Size

- This Is Not A Square Matrix

- This matrix is too dense, with too many rows or columns containing the same number of

elements to be sorted into lower triangular form

Where appropriate, errors are highlighted in yellow. For example, duplicate rows are

highlighted; too many coupled elements are not.

B-26

6.2.3 A Simple COTS Syst em Design

The system shown in Figure 7 is intended to be quite generic, to illustrate the principles of

Axiomatic design that I am recommending to apply to COTS integration. Figure 8 is the initial design

matrix, created by identifying couplings (dependencies) between Design Parameters and Functional

Requirements.

B-27

Figure 7. Simple COTS Based System Diagram

Memory Cards

Mass Storage

Signal Processors

Sensors

Processor(s)

Operating System

Application Software

User/Operator

Man Machine Interface

Hardware, Including

Keyboard(s)

Monitor(s)

B-28

(1

) Pro

cessor(

s)

(2)

Mem

ory

Card

(s)

(3)

Mass S

tora

ge

(4)

Sig

nal Pro

cessor(

s)

(5)

Sensors

(6)

Opera

ting S

yste

m

(7)

Application S

oft

ware

(8)

Cabin

et(

s)

(9)

Monitor(

s)

(10)

Keyboard

s

(11)

Oth

er

User

MM

IH H

W

(12)

Furn

iture

(1) Processing Speed 1 0 0 0 0 1 0 0 0 0 0 0

(2) Disk Read/Write Speed 1 0 1 0 0 1 0 0 0 0 0 0

(3) Memory Read/Write Speed 1 1 0 0 0 1 0 0 0 0 0 0

(4) Sensor Functionality 0 0 0 1 1 0 1 0 0 0 0 0

(5) Physical Space 1 0 1 1 0 0 0 1 1 0 0 1

(6) Electrical Power 1 1 1 1 0 0 0 0 1 0 0 0

(7) Operator Input/Output 0 0 0 0 0 0 1 0 1 1 1 0

(8) Disk Capacity 1 0 1 0 0 1 0 0 0 0 0 0

(9) Memory Capacity 1 1 0 0 0 1 0 0 0 0 0 0

(10) Application SW Functionality 1 1 1 1 1 1 1 0 0 0 0 0

(11) Ergonomics 0 0 0 0 0 0 0 1 1 1 1 1

(12) Operating System Functionality 1 1 1 0 0 1 0 0 0 0 0 0

Figure 8. Simple Initial Design Matrix

B-29

Note the two items included that are not on the diagram; Cabinet(s), and Furniture. In order to

determine that the design is sufficiently decoupled to simultaneously satisfy all requirements, this matrix

must be rearranged, by whole columns and rows, to fit into lower triangular format. Figure 9 is the initial

data entry into the Excel tool.

(1

) P

rocessor(

s)

(2)

Mem

ory

Card

(s)

(3)

Ma

ss S

tora

ge

(4)

Sig

na

l P

roce

sso

r(s)

(5)

Sensors

(6)

Op

era

tin

g S

yste

m

(7)

Applic

ation S

oftw

are

(8)

Cabin

et(

s)

(9)

Monitor(

s)

(10)

Keyboard

s

(11)

Oth

er

User

MM

IH H

W

(12)

Furn

iture

(1) Processing Speed x x

(2) Disk Read/Write Speed x x x

(3) Memory Read/Write Speed x x x

(4) Sensor Functionality x x x

(5) Physical Space x x x x x x

(6) Electrical Power x x x x x

(7) Operator Input/Output x x x x

(8) Disk Capacity x x x

(9) Memory Capacity x x x

(10) Application SW Functionality x x x x x x x

(11) Ergonomics x x x x x

(12) Operating System Functionality x x x x

Figure 9. Initial Data Entry Into Tool

B-30

The yellow highlighted rows and columns indicate duplicates. If a row is duplicated, that means

that for the purposes of resolving the matrix to lower triangular form, those two requirements can be

combined. If it does not make sense to combine the requirements, or if an equal number of design

parameters cannot also be combined; you must either identify an additional coupling between one of the

requirements and one of the design parameters; or eliminate that requirement and one design parameter.

In this case, the duplicates are:

First Duplicate Rows:

(2) Disk Read/Write Speed and (8) Disk Capacity

Combining these two makes sense, since it is unlikely that any design parameter choice that

affects one of these would not also affect the other.

Second Duplicate Rows:

(3) Memory Read/Write Speed and (9) Memory Capacity

Again, it makes sense to combine these two in this context.

First Duplicate Columns:

(8) Cabinets and (12) Furniture

Cabinets and Furniture design parameters may also be combined.

Second Duplicate Columns:

(10) Keyboards and (11) Other MMIH HW

It also makes sense to combine these; in fact, a bit more thought before creating the matrix might

have put both of these in the same grouping to begin with.

Since all the combinations make sense, and there are an equal number of requirements and

design parameters that need to be combined; do it. Figure 10 shows the result.

B-31

(1) P

roce

ssor

(s)

(2) M

emor

y C

ard(

s)

(3) M

ass

Sto

rage

(4) S

igna

l Pro

cess

or(s

)

(5) S

enso

rs

(6) O

pera

ting

Syst

em

(7) A

pplic

atio

n So

ftwar

e

(8) C

abin

et(s

) & F

urni

ture

(9) M

onito

r(s)

(10)

MM

IH H

W, I

nclu

ding

Key

boar

d(s)

(1) Processing Speed x x (2) Disk Read/Write Speed & Capacity x x x

(3) Memory Read/Write Speed & Capacity x x x (4) Sensor Functionality x x x

(5) Physical Space x x x x x (6) Electrical Power x x x x x

(7) Operator Input/Output x x x (10) Application SW Functionality x x x x x x x

(11) Ergonomics x x x (12) Operating System Functionality x x x x

Figure 10. Data Entry After Combining Rows & Columns

B-32

This set seems to work; now the tool does not detect any immediate problems. Figure 11 shows

the result of running the tool the first time.

(1)

Pro

cessor(

s)

(6)

Op

era

tin

g S

yste

m

(2)

Mem

ory

Card

(s)

(3)

Ma

ss S

tora

ge

(4)

Sig

na

l P

roce

sso

r(s)

(9)

Monitor(

s)

(7)

Applic

ation S

oftw

are

(5)

Sensors

(10)

MM

IH H

W, In

clu

din

g K

eyboard

(s)

(8)

Cabin

et(

s)

& F

urn

iture


(3) Memory Read/Write Speed & Capacity x x x

(2) Disk Read/Write Speed & Capacity x x x




(10) Application SW Functionality x x x x x x x

(7) Operator Input/Output x x x

(11) Ergonomics x x x

(5) Physical Space x x x x x Figure 11. First Pass

B-33

The tool sorts the matrix based on row and column weights, which in this case, is not good enough to get

it into lower triangular form. This means couplings are going to have to be identified that can be

removed. Couplings can be removed by new data, assumptions, or constraints; all of which needs to be

documented. My first cut is going to be to create a constraint that all equipment must fit within the

cabinets & furniture. This is not particularly onerous, as in general, all equipment will fit within properly

selected cabinets & furniture anyway. Creating this Constraint will allow removal of the following

couplings:

(1) Processor(s) to (5) Physical Space

(3) Mass Storage to (5) Physical Space

(4) Signal Processor(s) to (5) Physical Space

(9) Monitor(s) to (5) Physical Space

Figure 12 shows the result of removing these couplings and running the tool again.

B-34

(1) P

roce

ssor

(s)

(6) O

pera

ting

Syst

em

(8) C

abin

et(s

) & F

urni

ture

(2) M

emor

y C

ard(

s)

(3) M

ass

Sto

rage

(9) M

onito

r(s)

(10)

MM

IH H

W, I

nclu

ding

Key

boar

d(s)

(7) A

pplic

atio

n So

ftwar

e

(4) S

igna

l Pro

cess

or(s

)

(5) S

enso

rs

(1) Processing Speed x x (5) Physical Space x

(3) Memory Read/Write Speed & Capacity x x x (2) Disk Read/Write Speed & Capacity x x x

(12) Operating System Functionality x x x x (11) Ergonomics x x x

(7) Operator Input/Output x x x (6) Electrical Power x x x x x

(4) Sensor Functionality x x x (10) Application SW Functionality x x x x x x x

Figure 12. Second Pass

B-35

It is very important to document all added constraints and removed couplings. Once the matrix is in

lower triangular form, it will probably be possible to put some of the couplings back in, and remove the

corresponding constraints. Next, note that there are no columns with only element; and there must be

one, or there is no solution. Look at each column with two elements, and see which one makes the most

sense to decouple. The only one that makes sense is to try to decouple:

(5) Sensors to (10) Application SW Functionality

The added constraint is that the pool of possible application SW is reduced to only that which is

supported by the pool of possible sensors. Figure 13 shows the result of removing that coupling and

running the tool again.

B-36

(1)

Pro

cessor(

s)

(6)

Op

era

tin

g S

yste

m

(8)

Cabin

et(

s)

& F

urn

iture

(2)

Mem

ory

Card

(s)

(3)

Ma

ss S

tora

ge

(4)

Sig

na

l P

roce

sso

r(s)

(9)

Monitor(

s)

(7)

Applic

ation S

oftw

are

(5)

Sensors

(10)

MM

IH H

W, In

clu

din

g K

eyboard

(s)


(5) Physical Space x

(3) Memory Read/Write Speed & Capacity x x x

(2) Disk Read/Write Speed & Capacity x x x



(10) Application SW Functionality x x x x x x



(7) Operator Input/Output x x x Figure 13. Third Pass

B-37

Next, remove four couplings by altering two requirements:

The requirements for Mass Storage Capacity and Memory Capacity

are both increased to the level that will support all possible

Processors and Operating Systems.

This constraint removes the following couplings:

(1) Processor(s) to (3) Memory Read/Write Speed & Capacity

(1) Processor(s) to (2) Disk Read/Write Speed & Capacity

(6) Operating System to (2) Disk Read/Write Speed & Capacity

(6) Operating System to (3) Memory Read/Write Speed & Capacity

Figure 14 shows the result of removing these couplings and running the tool again.

B-38

(2

) M

em

ory

Card

(s)

(3)

Ma

ss S

tora

ge

(1)

Pro

cessor(

s)

(6)

Op

era

tin

g S

yste

m

(8)

Cabin

et(

s)

& F

urn

iture

(4)

Sig

na

l P

roce

sso

r(s)

(9)

Monitor(

s)

(7)

Applic

ation S

oftw

are

(5)

Sensors

(10)

MM

IH H

W, In

clu

din

g K

eyboard

(s)

(3) Memory Read/Write Speed & Capacity x

(2) Disk Read/Write Speed & Capacity x








(7) Operator Input/Output x x x Figure 14. Third Pass

B-39

Next, constrain application SW to support all Operating Systems. This removes the coupling:

(6) Operating System to (10) Application SW Functionality

Next remove the coupling between:

(2) Memory Card(s) to (6) Electrical Power

By making the constraint that memory cards must not cause the processor to require more AC Power.

(That is, the processor AC power is already sized for the maximum number of memory cards possible.

(Which is likely to be the case anyway, so is not a serious constraint) Running the tool again, shows no

change other than these couplings.

Next make the constraint that all available processors must be fast enough to support all available

Operating Systems.

This removes the coupling:

(6) Operating System to (1) Processing Speed

And the constraint that any commercial monitor will satisfy the operator input/output requirement, which

removes the coupling:

(9) Monitors to (7) Operator Input/Output

Figure 15 shows the result of running the tool with these couplings removed. Now only three remain

above the diagonal.

B-40

(2)

Mem

ory

Card

(s)

(3)

Ma

ss S

tora

ge

(1)

Pro

cessor(

s)

(8)

Cabin

et(

s)

& F

urn

iture

(4)

Sig

na

l P

roce

sso

r(s)

(9)

Monitor(

s)

(10)

MM

IH H

W, In

clu

din

g K

eyboard

(s)

(7)

Applic

ation S

oftw

are

(5)

Sensors

(6)

Op

era

tin

g S

yste

m



(1) Processing Speed x


(6) Electrical Power x x x x


(7) Operator Input/Output x x

(10) Application SW Functionality x x x x x


(12) Operating System Functionality x x x x Figure 15. Fourth Pass

Next, remove the coupling between Application SW and Sensor Functionality. This constrains the

possible sensors to those that do not require control or configuration by the application SW:

B-41

(7) Application Software to (4) Sensor Functionality

Now we also realize that constrained the Mass Storage and Memory Capacity requirements to support all

Operating Systems and Processors; we missed two other couplings that should have been removed:

(2) Memory Card(s) to (12) Operating System Functionality

(3) Mass Storage to (12) Operating System Functionality

Figure 16 shows the result of running the tool after removing these couplings.

B-42

(2)

Mem

ory

Card

(s)

(3)

Ma

ss S

tora

ge

(1)

Pro

cessor(

s)

(8)

Cabin

et(

s)

& F

urn

iture

(4)

Sig

na

l P

roce

sso

r(s)

(7)

Applic

ation S

oftw

are

(10)

MM

IH H

W, In

clu

din

g K

eyboard

(s)

(5)

Sensors

(9)

Monitor(

s)

(6)

Op

era

tin

g S

yste

m





(10) Application SW Functionality x x x x x

(7) Operator Input/Output x x

(4) Sensor Functionality x x


(12) Operating System Functionality x x

(6) Electrical Power x x x x Figure 16. Fifth Pass

B-43

This looks worse; but remember the tool is sorting based on weighted sums, so even though there are

more couplings above the diagonal, over-all, the matrix is closer to lower triangular form. Next, add the

constraint that Operator Input/Output will be whatever it can be with Application Software selected based

on other requirements. That constraint removes the coupling:

(7) Application Software to (7) Operator Input/Output

Figure 17 shows the result. Note that even though it looked like we took a step backwards last time,

removing this coupling allows the matrix to sort totally to lower triangular form.

(2)

Mem

ory

Card

(s)

(3)

Ma

ss S

tora

ge

(1)

Pro

cessor(

s)

(8)

Cabin

et(

s)

& F

urn

iture

(10)

MM

IH H

W, In

clu

din

g K

eyboard

(s)

(9)

Monitor(

s)

(4)

Sig

na

l P

roce

sso

r(s)

(6)

Op

era

tin

g S

yste

m

(5)

Sensors

(7)

Applic

ation S

oftw

are





(7) Operator Input/Output x


(6) Electrical Power x x x x

(12) Operating System Functionality x x

(4) Sensor Functionality x x

(10) Application SW Functionality x x x x x Figure 17. Fifth Pass

B-44

Now look at the form of the matrix, and see if any of the couplings that were removed earlier are now

below the diagonal. Any that are can be put back in, and the corresponding constraints removed. The

following couplings can be put back in:

(1) Processor(s) to (5) Physical Space

(3) Mass Storage to (5) Physical Space


(2) Memory Card(s) to (6) Electrical Power

(2) Memory Card(s) to (12) Operating System Functionality

(3) Mass Storage to (12) Operating System Functionality

Putting these couplings back in allows removal or modification of the following constraints:

1) All equipment must fit within the cabinets & furniture

2) The memory capacity requirement will support all possible Processors & Operating Systems

3) The Mass Storage capacity requirement will support all possible Processors & Operating

Systems

4) Installing Memory Card(s) will not cause the processor to draw more power than it’s nominal

rating

B-45

The remaining couplings that are above the diagonal are:












Now try adding each of these back in, and re-running the tool. This time, choosing the couplings to try

adding back in based on the constraints you would most like to remove; because each one that is added

back in limits how many more can be added and still allow sorting to lower triangular form. The

following Couplings could be sorted back in:




B-46

Which allows removal or modification of the following additional constraint:

1) Sensors may not require control or configuration by the application SW

The final matrix is shown in Figure 18:

(1)

Pro

cessor(

s)

(3)

Ma

ss S

tora

ge

(2)

Mem

ory

Card

(s)

(10)

MM

IH H

W, In

clu

din

g K

eyboard

(s)

(6)

Op

era

tin

g S

yste

m

(8)

Cabin

et(

s)

& F

urn

iture

(9)

Monitor(

s)

(4)

Sig

na

l P

roce

sso

r(s)

(7)

Applic

ation S

oftw

are

(5)

Sensors


(2) Disk Read/Write Speed & Capacity x x

(3) Memory Read/Write Speed & Capacity x x

(7) Operator Input/Output x


(5) Physical Space x x x




(4) Sensor Functionality x x x Figure 18. Final Matrix

B-47

The couplings that were removed and could not be put back in:










Which leaves the following final constraints:

1) All equipment must fit within the cabinets & furniture, except Processors and Mass Storage)

2) Mass Storage Speed and Capacity requirements must support all Operating Systems

3) Memory Speed and Capacity requirements must support all Operating Systems

4) All possible Application SW will Support all Operating Systems

5) All possible processor speeds will support all possible Operating Systems

6) Any commercial monitor will support the Operator Input/Output requirement

7) Sensors will not require Application SW control or configuration

8) Operator Input/Output requirements will be modified as necessary to support whatever

Application SW is selected

B-48

The final selection order resulting from this analysis is:

1) (1) Processor(s)

2) (3) Mass Storage

3) (2) Memory Card(s)

4) (10) MMIH HW, Including Keyboard(s)

5) (6) Operating System

6) (8) Cabinet(s) & Furniture

7) (9) Monitor(s)

8) (4) Signal Processor(s)

9) (7) Application Software

10) (5) Sensors

B-49

The order at the top would not surprise anyone with COTS integration experience. But it might come as

somewhat of a surprise that Cabinets & Furniture should be selected before COTS Application SW. It is

important at this point to note that this is not necessarily the only possible order; but it is an order that is

guaranteed to not cause any late changes to the COTS selection process driven by prior selections. For

example, the selection of Application Software will not force a different choice of Processor, Mass

Storage, or Memory Card(s). It might be possible that Cabinets & Furniture could be selected last, and

still not cause any reselection. If that became an issue, one could try manually re-sorting the matrix to see

if it is possible to put Cabinets & Furniture on the bottom row while still maintaining a lower triangular

matrix. In this case, it is not readily apparent that such a rearrangement is possible, so you would just

have to resign yourself to defining Cabinets & Furniture first. (Or look at eliminating even more

coupling)

This process is somewhat iterative; but not nearly as costly as restocking and reordering previously

purchased equipment, and it correctly executed, it guarantees that will not happen.

B-50

CHAPTER VII Product Design and Development

7 .1 Gat e 7 - P re l iminary Design Rev iew

“It’s just COTS”

Those are the words that are usually heard when a manager or engineer is about to skip over part of the

normal engineering discipline because COTS is being used vs. a full-up custom design. But consider:

Engineering

The process of devising a system, component, or process to meet desired needs. It is a decision-

making process (often iterative) in which the basic sciences, mathematics, and engineering

sciences are applied to convert resources optimally to meet a stated objective.

– Accreditation Board for Engineering and Technology



This is a good definition of “engineering”; and nothing in this definition (or any other credible variation)

will give you a reason to exclude selection of COTS components, or configuring them into a deliverable

system. Much of the same rigor and discipline is needed for a COTS effort as for a custom design. A

major difficulty is that there is an expectation by the customer and by management that a COTS design

will result in significant cost and schedule savings. This can be true; but you don’t get the savings by

cutting engineering. (See Section 4.1) There may even be cases, when you consider the entire lifecycle

cost, where a custom design will actually cost less. (Gasp!)

B-51

Systems Engineering integrates all the disciplines and specialty groups into a team effort forming a

structured development process that proceeds from concept to production to operation. Systems

Engineering considers both the business and the technical needs of all customers with the goal of

providing a quality product that meets the user needs.

According to INCOSE, the basic Systems

Engineering process tasks are:

1) Define the System Objectives

2) Establish the Functionality

3) Establish the Performance Requirements

4) Evolve Design and Operation Concepts

5) Select a Baseline

6) Verify that the Baseline Meets Requirements

7) Validate that the Baseline Satisfies the User

8) Iterate the Process through Lower Levels



The only thing on this list that might possibly be totally cut without adding all of the same risks you

would incur by cutting it from a custom design would be item 8). (But even this one might be applicable

for a large “System of Systems” type effort, so it should not be thrown out without thinking) The savings

for using COTS vs. custom designs should generally not be expected on the Systems Engineering side;

but on the Hardware and Software Development side. In fact, you can expect some additional cost in 1),

2), 3), and 4); because it is highly likely that all three of these tasks will have to be iterated with the

customer, as COTS options and capabilities become clearer as applied to the task at hand. And even in

the development area, the savings won’t come from cutting engineering discipline; it comes from labor

and material savings. The only additional checkpoint for a COTS design is to make certain no

B-52

preliminary design steps are skipped because “it’s just COTS”. If a step is trivial for COTS, then

performing it and documenting it is also trivial; so do it. This is another case where a typical COTS

project would have multiple PDRs. The example COTS project in Chapter 5 has 3; Gates 7, 7a, and 7b.

B-53

7.2 Gat e 8 - Cr it i cal Des ign Review

Detailed design is an area where one could reasonably expect to see some significant COTS

savings in a COTS development vs. a custom development. Where in a full up custom development, you

would now be designing “black boxes” whose functionality was defined at the Preliminary Design level;

for COTS many or all of these boxes may now be purchased items. The best philosophy is to not skip

any design steps. Again, skip no steps due do a predominately COTS implementation. If the use of

COTS makes a step trivial; it should not take much time to just do it; vs. skipping over it. This is another

case where a typical COTS project would have multiple CDRs. The example COTS project in Chapter 5

has 3; Gates 8, 8a, and 8b.

B-54

CHAPTER VIII System Integration, Verification, and Validation

8 .1 Gat e 9 - Test /Sh ip Read iness Review

A typical COTS program will only have one “formal” Test/Ship Readiness review. But holding

internal equivalents to Gate 9 at the end of each spiral is a good idea. These could be just quick sit-down

meetings with all stakeholders to ensure that everything is set to continue to the next spiral. The “formal”

review is the one the customer must buy off on before proceeding to formal testing. (Note that formal

testing may be performed before delivery, after delivery, or both; depending on the nature of the system

being delivered and customer requirements.) There are no additional checkpoints for COTS; but it is

important here too that none of the normal steps are skipped because “It’s just COTS”. It still has to meet

requirements, and if it does not get tested, how do you know it does? The example COTS development in

Chapter 5 only has a single “formal” Gate 9. Whether to have Readiness reviews between each spiral

should be established at Gate 5.

B-55

CHAPTER IX Production and Deployment

9 .1 COTS "Product ion"

This phase is normally deemed "not applicable" to COTS, and tailored out. That is a big mistake.

Unless you are paying a subcontractor for a turn-key job, you are still going to have to deal with some

sort of "production", and you will certainly have to deal with "deployment". A significant twist is that for

a majority COTS program, this gate should come before Gate 9; as there is generally no "prototype", and

most customers expect the first developed system to be a deliverable. This presents a different set of

problems in itself. For many custom designs, prototype equipment remains at the factory for

enhancement, or use in solving problems reported from the field. But for most COTS implementations;

this is not an option. Even if the development system was procured separately, and remains at the factory;

thanks to Moore’s Law, that equipment will quickly become obsolete. So most COTS developments do

not have the advantage of having duplicate equipment to work field maintenance problems. This is

another key reason why careful consideration should be given to total life cycle when making a COTS vs.

custom decision.

9.2 Gat e 10 - Product ion Read iness Review

This review should not be skipped; but it could be combined with Gate 9. A key COTS

checkpoint for this review:

Is the program prepared (or required) to support DR work-off, Operation, Maintenance, etc.; with the

equipment being delivered no longer at the factory?

The example COTS integration program in Chapter 5 has one Gate 10, just prior to Gate 9.

B-56

CHAPTER X Operations and Support

10.1 Rem ote Support

All of the same issues exist with COTS maintenance of COTS equipment as with custom

equipment. In fact, once fielded, the user can only tell that the equipment meets its requirements or not,

and is reliable, or not; COTS vs. Custom becomes irrelevant. The key difference from the standpoint of

factory support is that you are going to have to rely heavily on the suppliers you purchased the equipment

from; which could end up increasing the maintenance cost. (A major reason the total lifecycle cost should

be considered when making the initial COTS/Custom trade.)

B-57

CHAPTER XI Program Closure

11.1 Gate 11 - Transit ion and Closure Review

Shutdown of a COTS program is another area where considerable savings can be achieved.

There is typically no large investment in design tools, hence very little cost in disposition of tools no

longer needed. A critical area to address in a COTS system is:

Does the O&M contractor have all of the technical information, and all of the current vendor

contact information that they will need to maintain and operate the system?

Tech manuals and maintenance manuals that are “automatically” generated by following the

custom design processes may not exist for COTS equipment.

B-58

CHAPTER XII SUMMARY AND CONCLUSIONS

12.1 Summary and Conc lusions

12.1.1 COTS & IPDS

Heavy use of COTS equipment and software in place of custom designs can save the customer

schedule and cost. But it is important to make sure the savings are coming from the use of the COTS, and

not from cutting corners in the design process. Cutting those same corners would save exactly the same

amount of time in a custom design; and in a COTS design, cutting those corners adds exactly the same

amount of risk to the program it would add in a custom design. Key elements of IPDS to retain:

1) Gate Reviews – There is no good reason to “tailor out” any of the IPDS Gates just because

you are running a “COTS” project. Combine them, reduce the formality, shorten them, etc.;

but address every element of every gate.

2) Design Discipline – If the design activity (Requirements development, test, etc.) is trivial; the

amount of time to execute the process will also be trivial; so do it.

12.1.1 COTS & Axiomat ic Design

All the concepts advanced by all the advocates of Axiomatic Design do not have to be embraced,

or even accepted, to see a significant benefit by applying the “Independence Axiom” to COTS design. I

have seen no other way to generate a sequence of defining COTS elements that is guaranteed to result in

no re-specification and re-procurement of some equipment mid-way through a COTS integration project.

The small amount of requirement analysis time spent is well worth the assurance it provides.

B-59

REFERENCES

(TBD) Ertas, A. 1999, “How to deal with Complexity,” Journal of Integrated Design and Process Science, Vol. 5, No. 2, pp.23-36. Hubka, V. and Eder, W.E., Theory of Technical Systems, 1998, Berlin, Springer-Verlag, 2nd Ed. (Ref. MIT OpenCourseWare, Introduction, Systems Engineering)

http://ocw.mit.edu/OcwWeb/Engineering-Systems-Division/ESD-33Summer2004/LectureNotes/index.htm Tech University Masters Program, Systems Engineering Principles, ME 5354

Implementing a COTS System Design Using Raytheon IPDS and ...

Documents

Transcript of Implementing a COTS System Design Using Raytheon IPDS and ...