NASA Test and Evaluation Master Plan

SDM 5004 System Engineering and Project Management

Multi-satellite Operations Control Center (MSOCC)

Communications Switching System (MCSS)

Test and Evaluation Master Plan (TEMP)

Revision 01

(SDM5004-TEMP-MSOCC-0000)

May 2011

Division of Engineering & Technology Management (D-ETM)

National University of Singapore

Faculty of Engineering

Block EA, #05-34 (Building EA, Level 5, Room 34)

9 Engineering Drive 1

Singapore 117576

ii

SDM5004-TEMP-MSOCC-0000 REV 01

WBS: 00

SDM 5004 System Engineering and Project Management

MULTI-SATELLITE OPERATIONS CONTROL CENTER (MSOCC)

COMMUNICATIONS SWITCHING SYSTEM (MCSS)

TEST AND EVALUATION MASTER PLAN (TEMP)

Revision 01

May 2011

Submitted by:

02 May 2011 A0000660B

Aditya Nugroho

Project Manager

Date Matriculation Number

Approved by:

May 2011

Dr. Joseph E. Kasser

Director Operational Test

and Evaluation

Date

iii

RECORD OF CHANGES

*A - ADDED M - MODIFIED D – DELETED

VERSION

NUMBER DATE

NUMBER OF

FIGURE, TABLE

OR

PARAGRAPH

A*

M

D

TITLE OR BRIEF

DESCRIPTION AUTHOR

00 18.04.2011 TEMP Paragraph M

D

Knowledge component in the

TEMP shall appear in the

footnotes.

Dr. Joseph E.

Kasser

iv

REQUIREMENT TRACEABILITY MATRIX

ID SUMMARY OF REQUIREMENT TYPE SECTION

01 An introduction to Project Management and

Systems Engineering Written Section 4 Page 17

02 Modern project management Written Section 4 Page 17

03 Project Selection and Strategy Written Section 3 Page 11

04 Project Organisation Written Section 3 Page 8

05 Project Planning Written Section 4 Page 17

06 Scheduling Written Section 4 Page 20

07 Cost Estimating Written Section 4 Page 30

09 Performance Monitoring Written Section 4 Page 35

10 Project Leadership Written Section 3 Page 11

11 Project Teams Written Section 3 Page 12

Section 4 Page 39

12 Constraint Management Written Section 4 0

14 Project Audit and Closeout Written Section 4 Page 43

Lessons Learned Written Section 5 Page 46

Footnotes the knowledge component Written Appropriate Section

v

Table of Contents

RECORD OF CHANGES ............................................................................................................. III

REQUIREMENT TRACEABILITY MATRIX ........................................................................... IV

1. INTRODUCTION ................................................................................................................. 1

1.1 Purpose ............................................................................................................................. 1

1.2 Scope ................................................................................................................................ 1

1.3 Assumptions ..................................................................................................................... 1

1.4 References ........................................................................................................................ 2

1.5 Definitions ........................................................................................................................ 3

2. DESCRIPTION ..................................................................................................................... 4

2.1 MCSS Mission Description .............................................................................................. 4

2.2 MCSS System Description ............................................................................................... 4

3. PROGRAM SUMMARY ..................................................................................................... 6

3.1 Test and Evaluation Objectives ........................................................................................ 6

3.2 Test and Evaluation Methodology ................................................................................... 7

3.3 Management Organization and Responsibilities .............................................................. 8

3.3.1 External Interfaces ..................................................................................................... 8

3.3.2 Internal Interfaces ...................................................................................................... 9

3.3.3 Project Team Roles and Responsibilities ................................................................ 11

3.3.4 Documentation ........................................................................................................ 13

3.4 Resources Summary ....................................................................................................... 14

3.4.1 Integrated Test Master Schedule and Budget Summary ......................................... 14

3.4.2 Staff Resource Summary ......................................................................................... 15

3.5 Test Location and Equipment ........................................................................................ 16

4. TEST AND EVALUATION PROCESS PLAN ................................................................ 17

4.1 Work Planning ................................................................................................................ 17

4.1.1 Test and Evaluation Work Breakdown Structure (WBS) ....................................... 17

4.1.2 T&E Schedule Allocation Plan ............................................................................... 20

4.1.3 T&E Staff Resources Allocation Plan ..................................................................... 27

4.1.4 Budget Allocation Plan ........................................................................................... 30

4.2 Test and Evaluation Control Plan ................................................................................... 35

4.2.1 Budget and Schedule Control .................................................................................. 36

4.2.2 Quality Control ........................................................................................................ 37

4.2.3 Reporting and Communication ............................................................................... 39

4.3 Risk Management Plan ................................................................................................... 40

4.3.1 Risk Identification ................................................................................................... 40

4.3.2 Risk Mitigation ........................................................................................................ 40

4.4 Project Closeout Plan ..................................................................................................... 43

4.4.1 Post Performance Analysis ...................................................................................... 43

4.4.2 Project Closeout Schedule Allocation ..................................................................... 43

4.4.3 Project Closeout Critical Path ................................................................................. 43

4.4.4 Project Closeout Resource Allocation ..................................................................... 44

4.4.5 Project Closeout Budget Allocation ........................................................................ 45

vi

5. LESSON LEARNED ........................................................................................................... 46

6. BIBLIOGRAPHY ............................................................................................................... 48

7. APPENDIX 1. WORK PACKAGES DESCRIPTION .................................................... 50

APPENDIX A: PRELIMINARY DESIGN (SRR-PDR) .............................................................. 50

APPENDIX B: DETAILED DESIGN (PDR-IRR) ...................................................................... 60

APPENDIX C: DEVELOPMENT AND INTEGRATION (IRR-TRR) ....................................... 71

APPENDIX D: QUALIFICATION AND INSTALLATION (TRR-DRR) ................................. 82

APPENDIX E: PROJECT CLOSEOUT ....................................................................................... 92

8. APPENDIX 2. RISKS FACTORS CATEGORIZATION............................................... 98

vii

List of Tables and Figures

Table 1. 1 T&E Assumptions .......................................................................................................... 1

Table 1. 2 Document Reference ...................................................................................................... 2

Table 3. 1 Requirements Verification Matrix ................................................................................. 6

Table 3. 2 T&E Document Deliverables ......................................................................................... 7

Table 3. 3 Roles and Responsibilities of organization team for T&E activities ........................... 12

Table 3. 4 Basic rate of project team cost ..................................................................................... 13

Table 3. 5 T&E Documentation .................................................................................................... 13

Table 3. 6 Total budget allocations ............................................................................................... 14

Table 3. 7 Staff allocations per development phase ...................................................................... 15

Table 4. 1 WBS Dictionary of T&E activities .............................................................................. 19

Table 4. 2 Budget allocation Preliminary Design Phase ............................................................... 31

Table 4. 3 Budget allocation Detailed Design Phase .................................................................... 32

Table 4. 4 Budget allocation Detailed Design Phase .................................................................... 33

Table 4. 5 Budget allocation Qualification and Installation Phase ............................................... 34

Table 4. 6 T&E Management Control System .............................................................................. 36

Table 4. 7 Sub lower level work packages budget and schedule control ...................................... 37

Table 4. 8 Work packages depict the quality control process ....................................................... 38

Table 4. 9 Communication matrix ................................................................................................. 39

Table 4. 10 Work packages of risk management plan .................................................................. 40

Table 4. 11 SRR-PDR Initial Risk Identification and Mitigation ................................................. 41

Table 4. 12 PDR-IRR Initial Risk Identification and Mitigation .................................................. 41

Table 4. 13 IRR-TRR Initial Risk Identification and Mitigation .................................................. 42

Table 4. 14 TRR-DRR Initial Risk Identification and Mitigation ................................................ 42

Table 4. 15 Budget allocation Project Closeout Phase .................................................................. 45

Figure 2. 1 Switch configuration ..................................................................................................... 4

Figure 3. 1 MCSS T&E Vee Model ................................................................................................ 8

Figure 3. 2 MCSSRP External Interfaces ........................................................................................ 9

Figure 3. 3 MCSSRP Matrix Organization Structure ................................................................... 10

Figure 3. 4 MCSSRP Organization Structure ............................................................................... 10

Figure 3. 5 Integrated T&E Master Schedule ............................................................................... 14

Figure 3. 6 Summary T&E Manpower Cost ................................................................................. 15

Figure 4. 1 Work Breakdown Structure for T&E activities .......................................................... 18 Figure 4. 2 Preliminary Design T&E schedule ............................................................................. 20 Figure 4. 3 Detailed Design T&E schedule ................................................................................... 21 Figure 4. 4 Development and Integration T&E schedule .............................................................. 21 Figure 4. 5 Qualification and Installation T&E schedule .............................................................. 22 Figure 4. 6 PERT-CPM Preliminary Design. ................................................................................ 23 Figure 4. 7 PERT-CPM Detailed Design ...................................................................................... 24 Figure 4. 8 PERT-CPM Development and Integration ................................................................. 25

viii

Figure 4. 9 PERT-CPM Qualification and Installation ................................................................. 26 Figure 4. 10 Resource allocation Preliminary Design Phase ........................................................ 27 Figure 4. 11 Resource allocation Detailed Design Phase .............................................................. 28 Figure 4. 12 Resource allocation Development and Integration Phase ......................................... 29 Figure 4. 13 Resource allocation Qualification and Installation Phase ......................................... 30 Figure 4. 14 Budget accumulative Preliminary Design Phase ...................................................... 32 Figure 4. 15 Budget accumulative Detailed Design Phase ........................................................... 33 Figure 4. 16 Budget accumulative Development and Integration Phase ...................................... 34 Figure 4. 17 Budget accumulative Qualification and Installation Phase ...................................... 35 Figure 4. 18 Project closeout schedule .......................................................................................... 43 Figure 4. 19 PERT-CPM Project Closeout Phase ......................................................................... 44 Figure 4. 20 Resource allocation of Project Closeout Phase ......................................................... 44 Figure 4. 21 Budget accumulative Project Closeout Phase ........................................................... 45

1

1. INTRODUCTION

1.1 Purpose

Test and Evaluation Master Plan (TEMP) describes the program Test and Evaluation (T&E)

objectives, requirements, general methodology (test flow and description of each T&E phase),

responsibilities, and scheduling of test phases for the the Multi Satellite Operations Control

Center (MSOCC) Communications Switching System (MCSS) switch upgrade. This TEMP is

a program-level management planning document for all MCSS T&E activities and as a guide

for developing T&E plans. In the MCSS Systems Engineering Management Plan (SEMP) that

is described the TEMP is subordinate to the program SEMP1.

1.2 Scope

The process described in this TEMP applies to engineering activities associated with the

design, development, and operation of the MCSS over the life cycle of the program. These

activities include the testing of manufactured items. Activities related to items on the T&E

activities will comply with requirements specified in the MCSS System Requirements

Document (SRD) Section 5. Test and evaluation plans which specify actions to be taken in

implementing test and evaluation at the project level, will be prepared by project manager.

1.3 Assumptions

This section lists assumptions that are specific to the test planning.

Table 1. 1 T&E Assumptions

Assumptions ID Description

T&E

Budgeting

BA.01

BA.02

• The project manager will be responsible for reviewing all

cost/budget changes for this project.

• Sufficient capital will be available to fund the MCSSRP.

1 Test and evaluation is an integral part of the systems engineering process. Key aspects of the systems

engineering process, more fully described in the SEMP, are discussed in this TEMP to illustrate how T&E

supports the overall systems engineering process. Bell, David.W and Brown, David.C. Systems Engineering and

Test and Evaluation—The Integrated Process. ITEA Journal 2010; 31: 57–62

2

T&E

Schedule

TA.01

TA.02

TA.03

• The critical path method (CPM) schedule and milestone list that

details the timeframes and deadlines for the completion of T&E

activities are given based on fixed time estimation.

• No resource limitations for T&E (i.e, resource personnel) and all the

resources currently available for this project.

• The project will be conducted with resources from a matrixed

organization structure.

T&E Staff SA.01

SA.02

• Personnel will be properly trained on the tools and techniques

needed to support this effort.

• Upon completion of each T&E activities in project phase, staff will

transition to the next phase of the project, return to their previous

work on a full time basis.

Testing QA.01

QA.02

• Testing environments will be stable during execute the system test

• Users will commit adequate resources to User Acceptance Testing

1.4 References

This section lists all the documents referenced in the TEMP.

Table 1. 2 Document Reference

Doc

No Document Name

1

An Enhanced Framework for the Management of Information Technology Projects, PPTO-

TM-002 Project Plan Template, Chief Information Officer Branch, Treasury Board of

Canada Secretariat January 2000.

2 MSOCC MCSS Systems Engineering Management Plan (SEMP). Presented in Session 8.

3 Defence Acquisition University Press. Test and Evaluation Management Guide 5th Edition.

FORT BELVOIR, VA, January 2005.

4 NASA/GSFC, Multi-Satellite Operations Control Center (MSOCC) Communications

Switching System (MCSS) System Requirements Document, 511-4SRD/0489.

5 Processes for Engineering a System, Electronic Industries Alliance (EIA) Standard 632,

ANSI/EIA-632-1998.

6

Systems Engineering – System Life Cycle Processes, International Organization for

Standardization (ISO)/International Electrotechnical Commission (IEC) 15288, ISO/IEC

15288:2002

3

1.5 Definitions

This section contains definitions for any specific terminology or acronyms used in this plan.

ACRONYM DESCRIPTION

CI Configuration Item

COTS Commercially available Off-The-Shelf

DOD Department of Defense

DRR Delivery Readiness Review

GSFC Goddard Space Flight Center

IRR Implementation Readiness Review

ISO International Organization for Standardization

MCSS MSOCC Communications Switching System

MCSSRP MCSS Replacement Project

MSOCC Multi Satellite Operations Control Center

NASCOM NASA Communications Network

NASA National Aeronautics and Space Administration

PDR Preliminary Design Review

PM Project Manager

QA Quality Assurance

RAM Reliability, Availability and Maintainability

SEMP System Engineering Management Plan

SOW Statement Of Work

SRD System Requirement Document

SRR System Requirement Review

T&E Test and Evaluation

TEMP Test and Evaluation Master Plan

TRR Test Readiness Review

V&V Verification and Validation

4

2. DESCRIPTION

2.1 MCSS Mission Description

The MSOCC uses a switching system to route special synchronous RS-422-A type digital

data within the MSOCC and between the National Aeronautics and Space Administration

(NASA) Communications Network (Nascom) and the computer equipment within MSOCC.

The mission is to replace the data switch that routed signals from multiple low earth orbit

(LEO) satellites to data processing computers without loss of LEO satellite scientific data2.

2.2 MCSS System Description

The need were recognized for a single switch to replace the three existing switches with at

least twice the capability of the three switches combined. The switch will be controlled by the

DOCS, will handle data at rates of up to 6.312 Mbps, and will have a capacity of 255 full-

duplex connections (see Figure 2.1). The new switch system will henceforth be referred to as

MCSS. The MCSS function requirements are grouped into 4 subsystems which are:

1) Switch Matrix

2) Switch Control

3) Timing Generator and

4) Test And Monitoring.

Figure 2. 1 Switch configuration

2 Kasser, J. E. and Mirchandani, C. J., ―The MSOCC Data Switch Replacement: A Case Study in Elicitating and

Elucidating Requirements‖, proceedings of the 15th International Symposium of the International Council on

Systems Engineering (INCOSE), Rochester, NY, 2005. Page 2-3

Replace

5

From the result of the preliminary discussions with potential switch vendors have shown that

many of the MCSS requirements can be met by COTS switch available in the market. The

requirements not met by the purchase would be taken care of by inhouse development.

Therefore, the decision is to purchase a standard industry switch which will be modified to

meet the design requirements. The Timing Generator subsystem and Test and Monitoring

subsystem will be developed in-house.

6

3. PROGRAM SUMMARY

3.1 Test and Evaluation Objectives

Test and evaluation is an integral part of the systems engineering process that is used to attain

system objectives. T&E objectives have been derived to support the systems engineering

process. Two general objectives of the T&E program for MCSS are:

(1) to support system design and development, and

(2) to verify compliance with requirements3.

To ensure conformance with requirements contained in the System Requirements Documents

(SRD), following Table 3.1 described the example of Requirements Verification Matrix for

each subsystems to be performed during test execution.

Table 3. 1 Requirements Verification Matrix

Req No Description Priority WBS

Reference

5.2 SWITCH MATRIX

5.2.1 The MCSS shall contain 255 ports

High Q101, Q201 5.2.2 The MCSS shall switch binary digital signals

5.2.3 The MCSS shall switch serial signals

5.2.4 This subsystem shall switch Data and Timing as signal pairs

5.3 SWITCH CONTROL

5.3.1 This subsystem shall be the MCSS operator’s control interface

High Q301

5.3.2 This subsystem shall require the use of passwords to control, assign or

modify switch connections.

5.3.3 This subsystem shall perform a self test on the MCSS upon power up

5.3.4 This subsystem shall provide the capacity to allow the control interface to

select the active switch control unit

5.4 TIMING GENERATOR

5.4.1 This subsystem shall contain identical primary and backup Terminal Timing

signal generators High Q302

5.4.2 This subsystem shall distribute Terminal Timing signals to a minimum of

255 ports

5.5 TEST AND MONITORING

5.5.1 This subsystem shall not affect the signals being monitored

High Q303 5.5.2 This subsystem shall contain at least 10 NBG’s

5.5.3 The NGB shall be able to generate predefined valid Nascom blocks at the

timing frequencies identified

3 In the MCSS T&E integration, verification and validation will takes place over the systems engineering

lifecycle to show that the systems meets the objectives of each phase are satisfied. INCOSE., ―System

Engineering Handbook Version 2.0‖, Seattle, 2000

7

3.2 Test and Evaluation Methodology

The approach used to conduct T&E is based on achieving the T&E objectives described

above.Test approach defined in the MCSS SRD will involve Development Team and

Transition Team. Operations personnel, the Development Team and Transition Team will be

involved in testing the MCSS at MSOCC.

Test procedures will develop to perform the test in four categories; factory acceptance test,

site acceptance test, integration acceptance test and final acceptance test. Thus, each of T&E

activity will deliver the documents as described in Table 3.2.

Table 3. 2 T&E Document Deliverables

Phase ID Document Deliverables Due Date

Preliminary

Design F100 Test Plan Document 31 January 2011

Detailed

Design F200 Test Procedures Document

28 February

2011

Development

and Integration F300 Final Subsystem Test Report 30 March 2011

Qualification

and Installation F400

Final Test Acceptance Report Document

(Verification and Validation) 30 April 2011

Closeout F500 Project Final Report 14 May 2011

MCSS T&E will start from the subsystems development until the subystem installation is

replace the current MSOCC switching system. The T&E activity except the Purchased Switch

Test is interdependence and can be illustrated by using a Vee model4 in the project lifecycle

as described in Figure 3.1.

As we bought the COTS Industry Switch from the market, those purchase switch should have

immediately test for immediate feedback result and will be served as independent testing. The

4 The Vee model illustrate here is only for the purpose the interdependence testing; not all of the phases in the

Vee-model (especially validation phase) according to the MCSS System Engineering Management Plan would

be partof this document. Vee model was adapted from 1204 Test and Evaluation lecture notes and modified for

MCSS from Forsberg, K., Mooz, H. and Cotterman, H., Visualising Project Management, John Wiley & Sons,

Inc., 3rd edition, 2005. Page 145

8

black box in the Vee model indicated that the phase is already tested in the beginning and will

not be incorporate in this TEMP.

Figure 3. 1 MCSS T&E Vee Model

The Test Pass and Fail criteria for each T&E activity will be develop according to the MCSS

System Requirement Document in Section 4. This Test Pass and Fail criteria will be

explained more detail in the Test Plan work packages following IEEE 829-1998 Standard for

Software Test Documentation5.

3.3 Management Organization and Responsibilities

3.3.1 External Interfaces

This section describe the organizational between the project and external entities6.The

external interfaces for the MCSSRP project would be the members of the project committee

(Computer Sciences Corporation), MCSSRP project manager, Ford Aerospace

(subcontractor) and NASA Goddard Space Flight Center who is the stakeholder for this

project.

5 MCSS T&E Work Packages will adapted the Test Plan format which contain: How the testing will be done,

Who will do it, What will be tested, How long it will take and What the test coverage. IEEE 829-1998 Standard

for Software Test Documentation. 6 With identifying project external interfaces it would covers the significance of stakeholders to project success.

Forsberg, K., Mooz, H. and Cotterman, H., Visualising Project Management, John Wiley & Sons, Inc., 3rd

edition, 2005. Page 12

Identify User Needs

and Test Concept

Document

Develop Test and

Evaluation Master Plan

Expand performance

specifications into

―design-to‖ Subsystem

Evolve ―design-to‖ into

documentation and

implementation plan

Validate components

against ―build-to‖

documentation

Assemble configuration

item and subsystem and

perform verification

Integrate system and

verifify to performance

specifications

Demonstarate and

validate system

capability

Fabricate, Assemble

and Code to ―build-to‖

documentation

Subsystem

Test Plan and

Procedure

Integration

Test Plan and

Procedure

Acceptance

Test Plan and

Procedure

Requirement

specification

System

specification

Preliminary

design

Detailed

design

Handover

Acceptance

test

Integration

Test

Subsystem

Test

9

Figure 3. 2 MCSSRP External Interfaces

3.3.2 Internal Interfaces

This section describe the internal structure of the MCSS T&E project organization to include

the interfaces among the units of the development team. The matrix organization7 will be use

to create succesful implementation of the T&E activities and provide efficient project

execution environment with emphasis on the functionality of each discipline8.

The organizational interfaces between the MCSS T&E project and organizational entities that

provide supporting processes, such as quality assurance, and verification and validation, will

be specified in support T&E and system engineering process. Organizational charts in Figure

3.3 and Figure 3.4 described the lines of authority, responsibility, and communication within

the project.

7 As MCCSRP is considered medium tech project with low risk involved, matrix organization would be

appropriate for describe the internal structure of the project. Shenhar, A. J. and Bonen, Z., ―The New Taxonomy

of Systems: Toward an Adaptive Systems Engineering Framework‖, IEEE Transactions on Systems, Man, and

Cybernetics – Part A: Systems and Humans, Vol. 27 (1997), no. 2, 137 – 145 8 Under the matrix organization, all project tasks in WBS are assigned to each discipline through work packages.

This is what we will applied in this TEMP. Ichiro Koshijima and Tomio Umeda. Human Resource Allocation in

Project Management. Management Science Approach.

http://www.sba.muohio.edu/abas/2001/brussels/Koshijima_Koshijima-Umeda.pdf

MCSSRP

Project

Manager

Vendor of

MCSS

switches

External

Environment

Project Team

Major

stakeholder –

NASA - GSFC

Project Sponsor –

CSC Exec. Mgt

Sub-contractor -

Ford Aerospace


10

Figure 3. 3 MCSSRP Matrix Organization Structure

Figure 3. 4 MCSSRP Organization Structure

Executive Management

Build & Development Department

MCSSRP Project Manager

Test & Evaluation Department

Configuration Department

MCSS System and Development Team

MCSS Test and Evaluation Team

MCSS Configuration Team

Quality Assurance Department

MCSS Quality Assurance Team

Administrative Office

MCSSRP Project Manager

MCSS Test and Evaluation Team

MCSS Configuration Team

Quality Assurance Team

Configuration Item Test Team

Integration Test Team

Delivery Team

MCSS System and Development Team

Requirements Team

Design Team

Implementation Team

LibrarianQuality Assurance

Engineer

• System engineer

• Architecting eng• Software eng• Hardware eng

• System eng• Software eng• Hardware eng

• System engineer• CM Specialist

• Logistic engineer

• System eng• Senior Test

Engineer• Test Engineer

11

3.3.2.1 Project Manager (PM)9

1) Internal responsibilities:

a. Approve all cost and schedule-related planning documents.

b. Approve all organizational work and WBS.

c. Control of all organizational resources.

d. Monitor and report all project performance

e. Evaluate trade-offs prior to major decisions and approve any deviations to approved

plans.

f. Monitor and assure that all corrective actions are completed in the specified time.

2) External responsibilities: The PM is the principal point of contact for maintaining

liaison with the Computer Science Corporation, coordinating and supporting meetings,

conducting program reviews, approving all deliverables and managing all subcontractor

(Ford Aerospace) activity.

3.3.3 Project Team Roles and Responsibilities

This section identify the organizational units or project team that are responsible for T&E

processes and activities. The Maturity Model10

will be use for assessment of competency in

allocating the resources in each work activity. Next table describe the roles and

responsibilities and basic unit rate for each team organization who is responsibles for T&E

activities.

9 Project Manager has roles in three different arena; the customer’s, executive management’s, and the project

team’s. Project Manager is like a conductor of orchestra, providing the guidance for all project teams to achieve

common goal. Forsberg, K., Mooz, H. and Cotterman, H., Visualising Project Management, John Wiley & Sons,

Inc., 3rd edition, 2005. Page181 – 195. 10 The Maturity Model will helps to allocate the human resources more efficient in the MCSSRP project based

on assesment of Type of Engineers (Type I, Type II, Type III, Type IV, and Type V).Joseph E. Kasser and Moti

Frank, 2010. A Maturity Model for the Competency of Systems Engineers. Proceedings of the 20th International

Symposium of the INCOSE, Chicago, IL.

Table 3. 3 Roles and Responsibilities of organization team for T&E activities

Position Roles and Responsibilities Staff and Resources Code Type of

Engineers WBS Activities Status Organization Unit

MCSSRP Project

Manager

Provides testing management oversight.

Responsibilities:

provide technical direction

acquire appropriate resources

provide management reporting

Project Manager (R1) Type V

M101, M102, M201,

M202, M301, M302,

M401, M402, M403

Full Time MCSSRP Project

Manager

MCSS Test and

Evaluation (T&E)

Team

An organizational team, responsible for CI

and Integration T&E, who reports to the

Project Manager .

Responsibilities:

execute tests, log results

recover from errors, document change

requests

System Engineer (R2)

Senior System Engineer (R3)

Senior Test Engineer (R4)

Test Engineer (R5)

Logistic Engineer (R6)

Type II

Type III

Type V

Q101, Q102, Q103,

Q104, Q201, Q202,

Q203, Q204, Q205,

Q301, Q302, Q303,

Q304, Q305, Q401,

Q402, Q403, Q404

Full Time

Part Time

Test and Evaluation

Department

Configuration

Item Test Team Engineering staff responsible for CI Testing.



Type III

Type V

Q101, Q103, Q203,

Q205, Q302, Q303 Part Time

Configuration

Department

Integration Test

Team

Engineering staff responsible for

CI/Hardware Configuration Item (HWCI)

Testing.


Test Engineer (R5)

Type V

Type III Q204, Q304, Q305 Full Time

Test and Evaluation

Department

Delivery Team Engineering staff responsible for on-site

deliveries, training, and testing.

Test Engineer (R5)


Type III

Type II

Q401, Q402, Q403,

Q404

Full Time

Part Time

Test and Evaluation

Department

Quality Assurance

(QA) Team

An organizational team, responsible for QA

functions, that reports to the Project

Manager.

Responsibilities:

decide on the scope of the Q/A testing in

agreement with Project Manager

verification and validation

QA Engineer (R7) Type III

Q101, Q103, Q203,

Q205, Q302, Q303,

Q204, Q304, Q305,

Q401, Q402, Q403,

Q404

Full-Time Quality Assurance

Department

Table 3. 4 Basic rate of project team cost

R1 Project Manager (R1) V $500

R2 System Engineer (R2) III $250

R3 Senior System Engineer (R3) V $350

R4 Senior Test Engineer (R4) V $350

R5 Test Engineer (R5) III $250

R6 Logistic Engineer (R6) II $200

R7 QA Engineer (R7) III $150

Resource

CodeLabor Category Type

Unit Rate/Day

(8 hours)

3.3.4 Documentation

The MCSS TEMP is the top-level test and evaluation planning document. Project manager

will be responsible for developing test and evaluation plan. Documents11

will be produced by

using microsoft word and spread sheet. Test and evaluation plan documents will identify the

following:

Table 3. 5 T&E Documentation

Documents Section

The methodology to be used for implementing test and evaluation. Section 3 Page 7

The organizational structure and responsibilities for implementing T&E. Section 3 Page 8

An integrated T&E schedule depicting major project milestones and T&E

activities in support of those milestones. Section 4 Page 17

An basic work packages of each of the major T&E phases including

identification of the subsystem being tested, test objective, and scope of

the test.

Appendix 1 Page 50-97

A summary of the T&E resources required for execution of the T&E

program. Section 3 Page 14-15

Documentation to be developed and maintained, including traceability and

conformance verification matrices. Section 3 Page 6

11 Good engineering documents are critical to the cost effectiveness of systems engineering. Reducing the cost of

preparing effective documents is an approach that applied in any organization and will reduce the cost of the

SDLC (systems development life cycle). Kasser, J. E., ―Improving the Systems Engineering Documentation

Production Process‖, proceedings of the 5th Annual International Symposium of the INCOSE, 1995, pages 6-9.

14

3.4 Resources Summary

This section provide a summary of the schedule, budget and staff resources for the T&E

activities.

3.4.1 Integrated Test Master Schedule and Budget Summary

Master Schedule as depicted in Figure 3.5. Budget allocations are contained in Table 3.6 and

Figure 3.5. The supporting Work Breakdown Structure (WBS), cost, schedule, and staffing

requirements for the each increment of the Master Schedule are contained in Section 4.

Week

WBS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

100

200

300

400

500

January February March April May

SRR PDR IRR TRR DRR Closeout

Figure 3. 5 Integrated T&E Master Schedule

Table 3. 6 Total budget allocations

1 Labor Cost Unit Rate/Day Working Days Subtotal

Project Manager $500 52 $26,000

System Engineer $250 8 $2,000

Senior System Engineer $350 10 $3,500

Senior Test Engineer $350 12 $4,200

Test Engineer $250 88 $22,000

Logistic Engineer $200 3 $600

QA Engineer $150 44 $6,600

$64,900

2 Material Cost Features Quantity

Nascom Block Generator (NBG) MCSS 10 $10,000

Block Error Detector (BED) MCSS 10 $10,000

Oscilloscope MCSS 2 $14,800

Test Rigs Test and Monitoring 5 $5,000

288 Ports Switch Matrix Switch Matrix 1 $360,000

$399,800

$464,700

$46,470

Total Budget

Fixed Fee Contract (10% )

Total Labor Cost

Total Material Cost

15

SRR-PDR PDR-IRR IRR-TRR TRR-DRRProject

Closeout

Cost $12,950 $12,800 $19,050 $15,100 $5,000

Cumulative Cost $12,950 $25,750 $44,800 $59,900 $64,900

$0

$10,000

$20,000

$30,000

$40,000

$50,000

$60,000

$70,000

MCSS T&E Manpower Cost

Figure 3. 6 Summary T&E Manpower Cost

The T&E activities total budget cost is $464,700 and fixed fee contract $46,470

3.4.2 Staff Resource Summary

Required staffing allocations by development phase are defined in Table 3.7. The table

provides guidance for staff allocations for the overall T&E activities.

Table 3. 7 Staff allocations per development phase

Labor CategoryPreliminary

Design

Detailed

Design

Development

and

Integration

Qualification

and

Installation

Closeout

Project Manager 1 1 1 1 1

System Engineer 1 0 0 1 0

Senior System Engineer 1 1 1 1 0

Senior Test Engineer 1 3 0 0 0

Test Engineer 2 2 6 2 0

Logistic Engineer 0 0 0 1 0

QA Engineer 1 1 3 1 0

Subtotals 7 8 11 7 1

16

3.5 Test Location and Equipment

The T&E activity other than Site Acceptance Test will be performed in in–house

Development Lab. The Site Acceptance Test is done at the first delivered location (EDF). The

in-house Development Lab has a sufficient facility and equipment for running the required

Test. However some of the equipment is not available and purchased independently in System

Requirement Phase. The complete listing of the Equipment used in the MCSS T&E is given

below.

Frequency counters, serial data analyzer, logic analyzer

Oscilloscope

Block Error Detector (BED)

Nascom Block Generator (NBG)

Test Rigs

Software test programs

17

4. Test and Evaluation Process Plan

4.1 Work Planning

This section specify specify the work activities, schedule, resources, and budget details for the

MCSS T&E activities. Work packages for each work activity, necessary resources, estimated

duration, work products to be produced, acceptance criteria for the work products, and

predecessor and successor T&E work activities are contained in Appendix 1.

4.1.1 Test and Evaluation Work Breakdown Structure (WBS)

System Life Cycle Processes ISO/IEC 15288 would be adopted in support of WBS T&E

activities. Detail of the phases will describe in next section. The MCSS project phases after

SRR is divided into 5 phases, they are:

1) Preliminary Design (SRR-PDR)

2) Detailed Design (PDR-CDR)

3) Development and Integration (CDR-IRR)

4) Qualification and Installation (IRR-DRR)

5) Project close out phase

The WBS will use the Top-Down approach12

which identifies the needed tasks, resource

allocations, and cost estimates. Figure 4.1 described the WBS of T&E and coding scheme.

4.1.1.1 Work Breakdown Structure Dictionary

To better understand the nature of the work required to satisfy each element, a complete WBS

dictionary13

will be use in MCSS T&E activities. Table 4.1 described the WBS dictionary of

this project.

12 Top-Down approach of work breakdown structure (WBS) will be use to explained the test planning from the

highest level into the lowest level for each phase. By using WBS the Project Manager could identifying the

implications of WBS to budgeting, scheduling, risk assessment, and cost collection during planning of MCSS

T&E activities. Forsberg, K., Mooz, H. and Cotterman, H., Visualising Project Management, John Wiley &

Sons, Inc., 3rd edition, 2005. 13 By using WBS dictionary it will help clarify the distinctions between WBS elements. Practice Standard for

Work Breakdown Structures (Second Edition), published by the Project Management Institute, ISBN

1933890134, page 8

Figure 4. 1 Work Breakdown Structure for T&E activities

MCSS T&E

100

Preliminary Design

Management

M101-Technical direction and

acquire resources

M102-Risk identification and

management

Testing

Q101 Execute purchased switch

test

Q102 Prepare modified switch matrix test plan

Q103 Prepare integration test plan

and procedure

Q104 Introductory training

200

Detailed Design

Management

M201 Technical direction and management

reporting

M202 Risk management

Testing

Q201 Execute modified switch

matrix test

Q202 Prepare modified Switch Control test plan

Q203 Prepare Timing Generator

test plan

Q204 Prepare Test and Monitoring test

plan

Q205 MCSS Subsystem design

RAM analysis

300

Development and Integration

Management

M301 Technical direction and management

reporting


Testing

Q301 Execute Switch Control test

Q302 Execute Timing Generator

test

Q303 Execute Test and Monitoring test

Q304 Finalize whole subsystem integration test

Q305 MCSS integration test

(verified)

400

Qualification and Installation

Management

M401 Handing over documentation


M403 Test and Evaluation

completion review

Testing

Q401 Final MCSS subsystem

operational test

Q402 Final MCSS subsystem factory

acceptance test

Q403 Final MCSS subsystem site acceptance test

Q404 Final MCSS subsystem customer

acceptance test

500

Project closeout

Management

M501 Perform client closeout

M502 Perform organizational

closeout

M503 Conduct subcontractor

closeout

M504 Write the project final report

M505 Conduct team closeout

Table 4. 1 WBS Dictionary of T&E activities

WBS ID Work Package Name Description Milestones

M101Technical direction and

acquire resources

To facilitate a smooth running of T&E activities including required

resources at SRR-PDRSRR-PDR

M102Risk identification and

management

This element include how to identify and conduct risk management at

each T&E activities at SRR-PDRSRR-PDR

Q101Execute purchased Switch

testTo test whether Purchased Switch is meet the requirements MCSS SRD SRR-PDR

Q102Prepare modified Swith

Matrix test planPlanning the Modified Switch Matrix Test Plan SRR-PDR

Q103Prepare integration test

plan and procedureTo provide a MCSS integration test pland and procedure for the tester SRR-PDR

Q104Implement Introductory

training

To train the manpower who will performed in the T&E with the MCSS

system to maintain the Test Quality SRR-PDR


management reporting

To facilitate a smooth running of T&E activities including management

report at PDR-IRRPDR-IRR

M202 Risk managementThis element include how to conduct risk management at each T&E

activities at PDR-IRRPDR-IRR

Q201Execute modified Switch

Matrix test

To test whether the Modified Switch Matrix is meet the requirements in

System requirements Document Section 5PDR-IRR

Q202Prepare modified Switch

Control test planPlanning the Modified Switch Control Test Plan PDR-IRR

Q203Prepare Timing Generator

test planPlanning the Modified Timing Generator Test Plan PDR-IRR

Q204Prepare Test and

Monitoring test planPlanning the Modified Test and Monitoring Test Plan PDR-IRR

Q205MCSS Subsystem RAM

analysis

Assurance that design maturity is sufficient to begin detailed design and

review the status including achieved resultsPDR-IRR



To facilitate a smooth running of T&E activities including management

report at IRR-TRRIRR-TRR


activities at IRR-TRRIRR-TRR

Q301 Execute Switch Control testTo test whether the Modified Switch Control is meet the requirements in

System requirements Document Section 5IRR-TRR

Q302Execute Timing Generator

test

To test whether the Modified Timing Generator is meet the requirements

in System requirements Document Section 5IRR-TRR

Q303Execute Test and

Monitoring test

To test whether the Modified Test and Monitoring Test is meet the

requirements in System requirements Document Section 5IRR-TRR

Q304Finalize whole subsystem

integration testCombined 4 separate review subsystems integration test IRR-TRR

Q305MCSS subsystem

integration test (verified)

Verified combined 4 separate review subsystems integration test meet

the requirementsIRR-TRR

M401Handing over

documentation

The purpose of this document is to assist a smooth handover for the

customerTRR-DRR


activities at TRR-DRRTRR-DRR

M403Test and Evaluation

completion reviewTo review project completion and identify any lesson learnt TRR-DRR

Q401Final MCSS subsystem

operational test

A Factory Acceptance Test witnessed by the transition test to test

whether each subsystems of the integrated MCSS is meet the requirement

in SRD section 5 and can operate as expected by customer

TRR-DRR


factory acceptance test

Review the result of the test documented in the Factory Acceptance Test

Report for verification and validation of the MCSSTRR-DRR

Q403Final MCSS subsystem site

acceptance test

To ensure reliable performance of MCSS subsystem and minimum impact

to the MSOCCTRR-DRR


customer acceptance test

To test fully integrated MCSS after being installed but before Final

Acceptance Test by NASA to be considered fully operationalTRR-DRR

20

4.1.2 T&E Schedule Allocation Plan

This section provide scheduling relationships among MCSS T&E work activities. Techniques

will be use for depicting schedule relationships in activities include activity Gantt charts,

critical path networks14

, and PERT chart15

. Bottom-Up approach16

will be use to estimate the

project time. Thus the schedule plan will follow one iteration of the Waterfall system and

system development lifecycle (SDLC) in T&E activities and fit within a timeline of 16 weeks.

Following section describe of the schedule plan fot each phase.

4.1.2.1 Preliminary Design (SRR-PDR)

The schedule for SRR-PDR phase will be start on 01 January 2011. This phase will cover the

technical direction for T&E activities, risk identification, test plans, resources and

introductory training.

TASK

IDACTIVITY 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5


acquire resources


management


test


Matrix test plan


plan and procedure

Q104Implement introductory

training

Week 1 Week 2 Week 3 Week 4

Figure 4. 2 Preliminary Design T&E schedule

14 A good project network is very useful to determine the critical path and slack time, and a good start to project

resource scheduling. Larson,Eriik W, and Gray,Clifford F.,Project Management, McGraw-Hill ., 5th Edition,

2011. Page 188 15 PERT chart can shows the dependencies between the various tasks and activities unlike Gantt Chart. Howard

Eisner, Essentials of Project and Systems Engineering Management, John Wiley & Sons, Inc., 2nd Edition,

2002, Page 91 - 122 16 The Bottom-up (micro) approach is suitable in MCSSRP project, where the cost and time is important

Larson,Eriik W, and Gray,Clifford F.,Project Management, McGraw-Hill ., 5th Edition, 2011. Page 188

21

4.1.2.2 Detailed Design (PDR-IRR)

The schedule for PDR-IRR phase will be start on 01 February 2011. This phase will cover the

management reporting (budget and schedule review), Reliability Avalilability and

Maintainability subsystem designs, test plan and procedures, and execute subsystem test.

TASK

IDACTIVITY 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5





Matrix test


Control test plan


test plan


Monitoring test plan


analysis


Figure 4. 3 Detailed Design T&E schedule

4.1.2.3 Development and Integration (IRR-TRR)

The schedule for PDR-IRR phase will be start on 01 March 2011. This phase will cover the

management reporting (budget and schedule review), risk management, final design, execute

susbystem test and verified subsystem integration test.

TASK

IDACTIVITY 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5




Q301Execute Switch Control

test


test


Monitoring test


integration test

Q305MCSS subsystem



Figure 4. 4 Development and Integration T&E schedule

22

4.1.2.4 Qualification and Installation (TRR-DRR)

The schedule for PDR-IRR phase will be start on 01 April 2011. This phase will cover the

handing over documentation, risk management, final MCSS subystem test results and delivery

final MCSS subsystem to the customer.

TASK

IDACTIVITY 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

M401Handing over

documentation



completion review


operational test




site acceptance test




Figure 4. 5 Qualification and Installation T&E schedule

4.1.2.5 Critical Path Preliminary Design (SRR-PDR)

The T&E Critical Path (CP) 17

during SRR-PDR phase is the longest path of the expected time of the T&E activity. The time metrics is a

week which is 5 working days. Critical Path for SRR-PDR phase = M101+Q101+Q1012+Q103+Q104 = 5+4+3+3+5 = 20 Days

Figure 4. 6 PERT-CPM Preliminary Design.

17 By using the Critical Path analysis, the Project Manager will easy to identify the minimum length of time needed to complete a project and reducing project risk.

Larson,Eriik W, and Gray,Clifford F.,Project Management, McGraw-Hill ., 5th Edition, 2011. Chapter 6-Developing Project Plan.

24

4.1.2.6 Critical Path Detailed Design (PDR-IRR)

The T&E Critical Path (CP) during PDR-IRR phase is the longest path of the expected time of the T&E activity. The time metrics is a week

which is 5 working days. Critical Path for PDR-IRR phase = M201+Q201+Q203+Q205 = 7+4+3+4 = 18 Days

Figure 4. 7 PERT-CPM Detailed Design

25

4.1.2.7 Critical Path Development and Integration (IRR-TRR)

The T&E Critical Path (CP) during SRR-PDR phase is the longest path of the expected time of the T&E activity. The time metrics is a

week which is 5 working days. Critical Path for IRR-TRR phase = M301+Q302+Q304+Q305 = 7+5+3+5 = 20 Days

Figure 4. 8 PERT-CPM Development and Integration

26

4.1.2.8 Critical Path Qualification and Installation (TRR-DRR)

The T&E Critical Path (CP) during SRR-PDR phase is the longest path of the expected time of the T&E activity. The time metrics is a

week which is 5 working days. Critical Path for TRR-DRR phase = M403+Q401+Q402+Q403 = 5+4+4+4 = 20 Days

Figure 4. 9 PERT-CPM Qualification and Installation

4.1.3 T&E Staff Resources Allocation Plan

This section provide a detailed itemization of the resources allocated to each major work

activity in the MCSS T&E work breakdown structure. Resources18

include the numbers and

required skill levels of personnel for each work activity. Required staffing allocations by

development phase are defined in the following section. The tables and figures represents

number of resource required to complete each work package. It shows the types of personnel

by Maturity Model and project roles.

4.1.3.1 Preliminary Design Phase (SRR-PDR)

TASK

IDACTIVITY RES DUR ES LF TS 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5


acquire resources1 5 0 5 0 1 1 1 1 1


management1 2 0 2 0 1 1

Q101Execute purchased

Switch test3 4 5 9 0 3 3 3 3


Matrix test plan1 3 9 12 0 1 1 1


plan and procedure1 3 12 15 0 1 1 1


training1 5 15 20 0 1 1 1 1 1

2 2 1 1 1 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1


Total Resource Load

0 1 2 3

Project Manager (R1)




Test Engineer (R5)


QA Engineer (R7)

SRR-PDR Staff Allocation

0

1

2

3

4

I II III IV V

Engineers Type

Figure 4. 10 Resource allocation Preliminary Design Phase

18 In this MCSS T&E activities, scheduling would allocates resources personnel to jobs (tasks) in order to

minimize total lateness. Project Management Institute A Guide to the Project Management Body of Knowledge,

Third Edition, 2004 Project Management Institute, Inc. ISBN 193069945X

28

4.1.3.2 Detailed Design Phase (PDR-IRR)

TASK



management reporting1 7 0 7 0 1 1 1 1 1 1 1

M202 Risk management 1 3 4 7 0 1 1 1


Matrix test3 5 7 12 0 3 3 3 3 3


Control test plan1 3 12 15 0 1 1 1


test plan1 3 12 15 0 1 1 1


Monitoring test plan1 3 12 15 0 1 1 1


analysis1 4 15 19 0 1 1 1 1

1 1 1 1 2 2 2 3 3 3 3 3 3 3 3 1 1 1 1


Total Resource Load

0 1 2 3 4





Test Engineer (R5)


QA Engineer (R7)

PDR-IRR Staff Allocation

0

1

2

3

4

5

6

I II III IV V

Engineers Type

Figure 4. 11 Resource allocation Detailed Design Phase

29

4.1.3.3 Development and Integration Phase (IRR-TRR)

TASK



management reporting1 7 0 7 0 1 1 1 1 1 1 1



test3 5 7 12 0 3 3 3 3 3


test3 5 7 12 0 3 3 3 3 3


Monitoring test3 5 7 12 0 3 3 3 3 3


integration test1 3 12 15 0 1 1 1

Q305MCSS subsystem

integration test (verified)3 5 15 20 0 3 3 3 3 3

1 1 1 1 2 2 2 9 9 9 9 9 1 1 1 3 3 3 3 3

Week 10 Week 11 Week 12

Total Resource Load

Week 9

0 1 2 3 4 5 6 7





Test Engineer (R5)


QA Engineer (R7)

IRR-TRR Staff Allocation

0

1

2

3

4

5

6

7

8

9

10

I II III IV V

Engineers Type

Figure 4. 12 Resource allocation Development and Integration Phase

30


TASK


M401Handing over

documentation1 3 0 3 0 1 1 1



completion review1 5 0 5 0 1 1 1 1 1


operational test3 4 5 9 0 3 3 3 3


factory acceptance test3 4 9 13 0 3 3 3 3


site acceptance test3 4 13 17 0 3 3 3 3


customer acceptance test1 3 17 20 0 1 1 1

3 3 3 1 1 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1


Total Resource Load

0 1 2 3 4 5 6 7





Test Engineer (R5)


QA Engineer (R7)

IRR-TRR Staff Allocation

0

1

2

3

4

5

6

7

8

9

10

I II III IV V

Engineers Type

Figure 4. 13 Resource allocation Qualification and Installation Phase

4.1.4 Budget Allocation Plan

This section provides a detailed breakdown of necessary resource budgets for each of the

major work activities in the work breakdown structure. The activity budget will estimated the

cost for activity personnel where the type of cost is price contract19

. The estimating project

19 In this MCSS, as there is low risk involved (using mostly existing technology), the cost contract for personnel

could be awarded as a fixed price contracts. Howard Eisner, Essentials of Project and Systems Engineering

Management, John Wiley & Sons, Inc., 2nd Edition, 2002, Page 91 - 122

31

cost will use the Bottom-Up approach20

21

. Required budget allocations by development phase

are defined in the following section.

4.1.4.1 Preliminary Design Phase (SRR-PDR)

Below is a listing work packages from SRR-PDR. The amount allocated for each is

determined by the daily rate (8 hours), and the number of people assigned to each work

package . Total manpower cost from is $12.950.

Table 4. 2 Budget allocation Preliminary Design Phase

Code Activities Description

100 Preliminary Design Phase Engineers Type Number Unit

Rate/day


acquire resources$6,000 $6,000

M101.1 Resources plan for T&E Project Manager Type V 1 $500 5 $2,500

M101.2 Cost and schedule plan Project Manager Type V 1 $500 5 $2,500

M101.3 Subsystem test direction Project Manager Type V 1 $500 2 $1,000


managementProject Manager Type V 1 $500 2 $1,000 $1,000 $7,000

1) Test Engineer Type III 2 $250 4 $2,000

2) QA Engineer Type III 1 $150 4 $600

Q102Prepare modified Swith Matrix

test plan

Senior Test

EngineerType V 1 $350 3 $1,050 $1,050 $10,650

Q103Prepare integration test plan

and procedure

Senior System

EngineerType V 1 $350 3 $1,050 $1,050 $11,700


trainingSystems Engineer Type III 1 $250 5 $1,250 $1,250 $12,950

Sub-total $12,950 $12,950

Manpower

cost

High level

sub-total

cost

Cumulative

cost

Manpower resourcesWorking

days

$2,600 $9,600Q101 Execute purchased Switch test

20 In the MCSSRP project, every phase or milestones have applied WBS packages. Which component of WBS

packages has contain, budget, schedule, resource. Bottom-Up approach can serve as a check on cost elements in

the WBS by rolling up the work packages and associated cost accounts to major deliverables at the work package

level. Larson,Eriik W, and Gray,Clifford F.,Project Management, McGraw-Hill ., 5th Edition, 2011. Chapter 5.

Page 126-155 21 In addition Bottom-Up approach could provide accurate estimate of the cost during product life cycle. Hari,

A., Shoval, S. and Kasser, J. E., ―Conceptual Design to Cost: A new systems engineering tool‖, proceedings of

the 18th International Symposium of the INCOSE, Utrecht, Holland, 2008.

32

M101 M102 Q101 Q102 Q103 Q104

High level sub-total cost $6,000 $1,000 $2,600 $1,050 $1,050 $1,250

Cumulative cost $6,000 $7,000 $9,600 $10,650 $11,700 $12,950

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,000

Preliminary DesignPhase

Figure 4. 14 Budget accumulative Preliminary Design Phase

4.1.4.2 Detailed Design Phase (PDR-IRR)

Below is a listing work packages from PDR-IRR. The amount allocated for each is

determined by the daily rate and the number of people assigned to each work package . Total

manpower cost is $12.800.

Table 4. 3 Budget allocation Detailed Design Phase


200 Detailed Design Phase Engineers Type Number Unit

Rate/day


management reporting$3,500 $3,500

M201.1 Resource plan review Project Manager Type V 1 $500 3 $1,500

M201.2 Budget and schedule review Project Manager Type V 1 $500 2 $1,000

M201.3 Subsystem integration test

directionProject Manager Type V 1 $500 2 $1,000

M202 Risk management Project Manager Type V 1 $500 3 $1,500 $1,500 $5,000




Control test plan

Senior Test

EngineerType V 1 $350 3 $1,050 $1,050 $9,300

Q203Prepare Timing Generator test

plan

Senior Test

EngineerType V 1 $350 3 $1,050 $1,050 $10,350

Q204Prepare Test and Monitoring

test plan

Senior Test

EngineerType V 1 $350 3 $1,050 $1,050 $11,400


analysis

Senior System

EngineerType V 1 $350 4 $1,400 $1,400 $12,800

Sub-total $12,800 $12,800

Cumulative

cost

$8,250$3,250


days

Manpower

cost

High level

sub-total

cost

Q201Execute modified Switch Matrix

test

33

M201 M202 Q201 Q202 Q203 Q204 Q205

High level sub-total cost $3,500 $1,500 $3,250 $1,050 $1,050 $1,050 $1,400

Cumulative cost $3,500 $5,000 $8,250 $9,300 $10,350 $11,400 $12,800

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,000

Detailed Design Phase

Figure 4. 15 Budget accumulative Detailed Design Phase

4.1.4.3 Development and Integration Phase (IRR-TRR)

Below is a listing work packages from IRR-TRR. The amount allocated for each is



Table 4. 4 Budget allocation Detailed Design Phase


300Development and Integration

PhaseEngineers Type Number

Unit

Rate/day


management reporting$3,500 $3,500

M301.1 Resource plan review Project Manager Type V 1 $500 3 $1,500


M301.3 MCSS subsystem integration

test direction (verified)Project Manager Type V 1 $500 2 $1,000








Q304

Prepare finalize whole

subsystem integration test

procedure

Senior System

EngineerType V 1 $350 3 $1,050 $1,050 $15,800



Sub-total $19,050 $19,050

$19,050

Cumulative

cost

$8,250

$11,500

$14,750

$3,250

Working

days

Manpower resourcesManpower

cost

High level

sub-total

cost

$3,250

$3,250

$3,250Q305MCSS subsystem integration

test (verified)

Q301 Execute Switch Control test

Q302 Execute Timing Generator test

Q303Execute Test and Monitoring

test

34

M301 M302 Q301 Q302 Q303 Q304 Q305

High level sub-total cost $3,500 $1,500 $3,250 $3,250 $3,250 $1,050 $3,250

Cumulative cost $3,500 $5,000 $8,250 $11,500 $14,750 $15,800 $19,050

$0

$5,000

$10,000

$15,000

$20,000

$25,000

Development and Integration Phase

Figure 4. 16 Budget accumulative Development and Integration Phase


Below is a listing work packages from TRR-DRR. The amount allocated for each is



Table 4. 5 Budget allocation Qualification and Installation Phase


400Qualification and Installation

PhaseEngineers Type Number

Unit

Rate/day

M401 Handing over documentation System Engineer Type III 1 $250 3 $750 $750 $750



completion review$2,500 $4,750


M403.2 Project completed review Project Manager Type V 1 $500 2 $1,000







Logistic Engineer Type II 1 $200 3 $600 $600 $13,150



Sub-total $15,100 $15,100

$7,350

$9,950

$12,550

Cumulative

cost

$2,600

$2,600

$2,600

Q401

Q402

Q403

Final MCSS subsystem

operational test

Final MCSS subsystem factory

acceptance test

Final MCSS subsystem site

acceptance test


days

Manpower

cost

High level

sub-total

cost

Final MCSS subsystem

customer acceptance testQ404

$1,950 $15,100

35

M401 M402 M403 Q401 Q402 Q403 Q404

High level sub-total cost $750 $1,500 $2,500 $2,600 $2,600 $2,600 $2,550

Cumulative cost $750 $2,250 $4,750 $7,350 $9,950 $12,550 $15,100

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,000

$16,000

Qualification and Installation Phase

Figure 4. 17 Budget accumulative Qualification and Installation Phase

4.2 Test and Evaluation Control Plan

This section will specify the metrics, and control procedures necessary to measure the

requirements, schedule, budget, and the quality of work processes and work products22

.

The control plan will place at each project milestones23

. Project Manager, will develop

detailed plans based on established work assignments for T&E activities. The Management

Control System (MCS)24

tools will be used to document and track the work against major

program milestones. Table 4-6 identifies the primary tools that support T&E control plan.

22 The purpose of the Control function is to provide adequate visibility into actual project progress to allow

effective management action when the project performance deviates significantly from the plans. Forsberg, K.,

Mooz, H. and Cotterman, H., Visualising Project Management, John Wiley & Sons, Inc., 3rd edition, 2005.

Chapter 14 Project Control Page 254 – 277 23 Project control process should be in place at all level of the project. So that any deviation from planned events

can be quickly corrected. Kezsbom, D. S., Schilling, D. L . and Edward, K.A., Dynamic Project Management. A

practical guide for Managers and engineers, John Willey and Sons, New York, 1989. 24 Management control system is an integrated technique for collecting and using information to evaluate

performance. The MCS tools, supported by the WBS, cost, schedule, requirements and quality to be reviewed at

any T&E activities phase. Horngren, C., Sundem, G. and Stratton, W., 2005. Introduction to Management

Accounting, New Jersey, Pearson. Page 404.

36

Table 4. 6 T&E Management Control System

Tool Description SRR-PDR PDR-IRR IRR-TRR TRR-DRR

Bottom-Up Cost Estimating x x x x

Bottom-Up Time Estimate x x x x

Earned Value

Analysis

Schedule

Controlx x x x

Earned Value

AnalysisBudget Control x x x x

Verification

and ValidationQuality Control x x x x

Microsoft

Word

Reporting and

Communicationx x x x

Microsoft

WordDocumentation x x x x

4.2.1 Budget and Schedule Control

Project Manager is responsible for monitoring cost and schedule for the activities and for

estimating cost to complete and schedule completion on a monthly basis. The PM shall take

immediate corrective actions necessary to minimize project deviation from cost and schedule

baselines.

Using Earned Value Analysis (EVA)25

for each product component, the PM generates

variance analysis reports by comparing Actual Cost of Work Performed (ACWP), Budgeted

Cost of Work Scheduled (BCWS), Budgeted Cost of Work Performed (BCWP), and Estimate

at Completion. In addition, the Critical Path Method (CPM) will be used to control the

activities most crucial to completion of the project on-schedule.

The critical path illustrated in the section 4.1.2.5 shall receive special attention with respect to

completion on schedule26

. Following Table 4.7 sub lower level work packages depict the

budget and schedule control performed by Project Manager.

25 Earned Value Analysis (EVA) is used to assess the current cost situation as a function of performance to date.

Three cumulative cost curves (budgeted cost of work scheduled – BCWS, budgeted cost of work performed –

BCWP, Actual cost of work performed – ACWP) are shown from project initiation time to current reporting

time. HE: Eisner, H., Essentials of Project and Systems Engineering Management, 3rd Edition Page103 26 Failure to complete these activities within their allotted time will cause slippage of the entire schedule. HE:

Eisner, H., Essentials of Project and Systems Engineering Management, 3rd Edition Page103

37

Table 4. 7 Sub lower level work packages budget and schedule control


M101.2Cost and schedule

plan

To plan estimating cost tocomplete and schedule completion for the

T&E activitiesSRR-PDR


review

To monitor cost and schedule for the activities and for estimating cost

to complete and schedule completion on a monthly basis at PDR-IRRPDR-IRR


review


to complete and schedule completion on a monthly basis at IRR-TRRIRR-TRR


review


to complete and schedule completion on a monthly basis at TRR-DRRTRR-DRR

4.2.2 Quality Control

This section specifies the mechanisms to be used to measure and control the quality of the

work processes and the resulting work products. Verification and validation27

will be use as

quality control mechanisms.

The Quality Assurance team under the direction of the QA Manager provides the Project

Manager with the assurance that all quality and control requirements are being accomplished.

In performing these duties, the QA Manager monitors adherence to all processes, procedures,

and plans through the delegation of responsibilities to the QA Team.

Following Table 4.8 work packages depict the quality control mechanism performed by

Quality Assurance team.

27 Verification: proof of compliance with specifications ―Was the solution built right?‖ Validation: proof that the

user(s) is satisfied ―Was the right solution built?‖ This Is a whole life-cycle process - V & V must be applied at

each stage in the system process. Forsberg,K.,Mooz,H., and Cotterman,H., Visualising Project Management,

John Wiley & Sons,Inc., 3rd Edition, 2005. Pages 361

Table 4. 8 Work packages depict the quality control process

WBS ID Work Package Name Verification and Validation DescriptionVerification

TechniquesValidation Techniques

Responsible

TeamMilestones

Q101Execute purchased

Switch test

Verification and Validation of preliminary design to ensure it meet

requirementsAnalysis

Phase reviews with end user to ensure

the user satisfaction is achieved

Quality

Assurance teamSRR-PDR


Matrix test

Verification and Validation of MCSS to ensure the specifications

and requirements of the subsystems entities are metTesting



Quality

Assurance teamPDR-IRR


test





Quality

Assurance teamIRR-TRR


test





Quality



Monitoring test





Quality


Q305MCSS subsystem


Verification and Validation of MCSS to ensure the requirements of

the integration of subsystems are metTesting



Quality



operational test

Verification and Validation of MCSS integrated system to ensure

the system requirements are metDemonstration



Quality

Assurance teamTRR-DRR







Quality








Quality








Quality


4.2.3 Reporting and Communication

This section specifies the reporting mechanisms and information flows to be used in

communicating the status of the project. The methods and techniques of communication will

be specified in this section.

Reporting Plan: The PM provides monthly progress reports to the stakeholders summarizing

work completed, a financial report, issues, and meetings.

Communication Plan28

: The PM holds progress reviews at least every week for the purpose

of monitoring internal progress towards milestones, reviewing problems encountered,

presenting proposed resolutions, presenting near-term plans, reviewing risks and mitigation

plans, and presenting a financial review using Earned Value data (performance report).

During project team meeting, STALL29

approach is adopted to regulate matters.

Table 4. 9 Communication matrix

Communication

Type Objective Frequency Audience Owner Deliverables

WBS

Reference

Project Team

Meetings

Review status of the

project Weekly Project Team

Project

Manager

Minutes

Meeting

M101,M201

M301,M403

Technical

design meeting

Discuss technical problems

raised in the

implementation

As needed Project Team Project

Manager

Minutes

Meeting M201,M301

Project status

meetings

Reports on the status of the

project to the management Monthly

Executive

Management

Project

Manager

M101,M201

M301,M403

Project status

reports

Report of the status of the

project (activities progress) Monthly

Project

Sponsor, Sub

contractor

Project

Manager

Project

Status

reports

M101,M201

M301,M403

Project closeout Report of the final project Once

Project

Sponsor, Sub

contractor,

Executive

Management,

Project Team

Project

Manager

Final project

report

Signed

acceptance

M501,M502

M503,M504

M505

28 To make project communication effective, we go directly to the most likely problem solver, regardless of

organization structure. Project communication can be represented by Participants, Techniques, Environment,

Language = Communications. Project participants encompass a wide array of stakeholders. (e.g. team members,

management, policy makers, contractor, customers, users). Forsberg, K., Mooz, H. and Cotterman, H.,

Visualising Project Management, John Wiley & Sons, Inc., 3rd edition, 2005. Page 48 – 68. 29 STALL is an acronym for (Stay calm, Think, Ask questions and analyse, Listen, Listen). The key element in

bridging the communications gap between systems and software engineers is to use active listening enhanced by

―pattern matching‖ during discussions. Kasser, J. E., A Framework for Understanding Systems Engineering,

Booksurge Ltd, 2007, ISBN 978-1-4196-7315-3. Page 62-70

40

4.3 Risk Management Plan

This section specify the risk management plan30

for identifying, analyzing, and prioritizing

project risk factors. The PM, in the role of Risk Manager, analyzes risk-related measurements,

reviews individual database entries as needed at weekly status meetings with the project team

members, and holds a review of the database on monthly schedule with the sponsor and other

senior management. Table 4.10 work packages depict the risk management performed by

Project Manager team in overall life cycle phases31

.

Table 4. 10 Work packages of risk management plan


M102Risk identification

and management

This element include how to identify and conduct risk management at

each T&E activities at SRR-PDRSRR-PDR


activities at PDR-IRRPDR-IRR


activities at IRR-TRRIRR-TRR


activities at TRR-DRRTRR-DRR

4.3.1 Risk Identification

An identified risk will be categorized in the table format which will be maintained by the

project manager for the duration of the project. This table will list the potential project risks,

and the indicators that determine the rating (High, Medium, Low). The risk categorization

table will produce ―Top 10 Risks‖32

which will be reviewed at monthly project status

meetings (see Appendix II).

4.3.2 Risk Mitigation

The resolution approach, and responsibility assignment will be included in risk mitigation

efforts. Following tables shows the risk identification and mitigation which would be incur on

T&E activities at each life cycle phases.

30 The process will concemed with conducting risk management planning, identification, analysis, responses, and

monitoring and control on a project. A Guide to the Project Management Body of Knowledge (PMBOK" Guide),

Project Management Institute, Upper Darby, PA, 2004 31 Opportunity and risk is ongoing that is we have to perform the risk management activities in every phase of

MCSS T&E. Forsberg, K., Mooz, H. and Cotterman, H., Visualising Project Management, John Wiley & Sons,

Inc., 3rd edition, 2005. Page223 – 253 32 Top 10 risk factor was adapted from the most failure of IT systems development projects. Tesch, Debbie;

Kloppenborg, Timothy J; Frolick, Mark N. IT Project Risk Factors: The Project Management Professionals

Perspective. The Journal of Computer Information Systems. July 1 2007.

Table 4. 11 SRR-PDR Initial Risk Identification and Mitigation

WBS ID Work Package Name Risk ID ProbabilityRisk

ExposureMitigation Action by

Action

when

Overall

Risk


acquire resources

R7, R17,

R18Likely Low (L)

Monitor progress and accelerate agency staff training and other

support measures as needed.

Project

ManagerAs needed


test

R14, R16,

R19, R22May

Medium

(M)

Evaluate system integrator proposals for ability to address

unforeseen problems

Project

Manager

As per

plan


Matrix test plan

R12, R20,

R21, R26Probably Low (L)

Plan a staged implementation, seek additional funds, during

system integrator selection consider cost and benefits of off-the-

shelf solutions

Project

ManagerAs needed


plan and procedure

R12, R20,




shelf solutions

Project

ManagerAs needed


trainingR25 May

Medium

(M)

Arrange staffing and their training well in advance, ensure

concept of operations is realistic.

Project

ManagerAs needed

Low (L)

Table 4. 12 PDR-IRR Initial Risk Identification and Mitigation



Action

when

Overall

Risk



R7, R17,

R18Likely Low (L)



Project

ManagerAs needed


Matrix test

R14, R16,

R19, R22May

Medium

(M)


unforeseen problems

Project

Manager

As per

plan


Control test plan

R12, R20,




shelf solutions

Project

ManagerAs needed


test plan

R12, R20,




shelf solutions

Project

ManagerAs needed


Monitoring test plan

R12, R20,




shelf solutions

Project

ManagerAs needed


analysisR11 May

Medium

(M)

Evaluate system for ability to address unforeseen problems,

coordinate with agencies on plans for systems requirements

Project

Manager

As per

plan

Low (L)

42

Table 4. 13 IRR-TRR Initial Risk Identification and Mitigation



Action

when

Overall

Risk



R7, R17,

R18Likely Low (L)



Project

ManagerAs needed

Q301 Execute Switch Control testR14, R16,

R19, R22May

Medium

(M)


unforeseen problems

Project

Manager

As per

plan


test

R14, R16,

R19, R22May

Medium

(M)


unforeseen problems

Project

Manager

As per

plan


Monitoring test

R14, R16,

R19, R22May

Medium

(M)


unforeseen problems

Project

Manager

As per

plan


integration testR11 Probably

Medium

(M)

Evaluate system integrator proposals for stability and delivery

track record.

Project

Manager

As per

plan

Q305MCSS subsystem

integration test (verified)R11 Probably

Medium

(M)

Evaluate system integrator proposals for stability and delivery

track record.

Project

Manager

As per

plan

Medium

(M)

Table 4. 14 TRR-DRR Initial Risk Identification and Mitigation



Action

when

Overall

Risk

M401Handing over

documentationR8, R13 Likely Low (L) Monitor progress and other support measures as needed.

Project

ManagerAs needed


completion review

R7, R17,

R18Likely Low (L) Monitor progress and other support measures as needed.

Project

ManagerAs needed


operational test

R14, R15,

R19May

Medium

(M)


allow time for dealing with the unexpected, have a process ready

for dealing with the unexpected.

Project

Manager

As per

plan


factory acceptance testR14, R23 May

Medium

(M)


unforeseen problems, coordinate with agencies on plans for

node systems, plan backup communication links.

Project

Manager

As per

plan



R14, R15,

R19May

Medium

(M)


allow time for dealing with the unexpected, have a process ready

for dealing with the unexpected.

Project

Manager

As per

plan


customer acceptance testR4, R19 Probably

Medium

(M)


coordinate with customer to ensure requirements are met

Project

Manager

As per

plan

Medium

(M)

4.4 Project Closeout Plan33

This section describes the nature of the activities that will be used to closeout the project when

it is completed. Project Manager will be fully involved in the approval of project closeout.

4.4.1 Post Performance Analysis

Upon termination of the entire project due to the objectives being met, there are a number of

closure activities that will be performed before the project is considered closed. Project

participants will be gathered for the purpose of a Post- Performance Analysis (PPA)34

.

4.4.2 Project Closeout Schedule Allocation

The schedule for project close out will be start on 01 May 2011. This closeout cover the client

closeout, organizational closeout, subcontractor closeout, final report and team closeout.

TASK

IDACTIVITY 1 2 3 4 5 1 2 3 4 5

M501 Perform client closeout

M502Perform organizational

closeout

M503Conduct subcontractor

closeout

M504Write the project final

report

M505 Conduct team closeout

Week 17 Week 18

Figure 4. 18 Project closeout schedule

4.4.3 Project Closeout Critical Path

The time metrics is a week which is 5 working days. Critical Path for Close out phase =

M501+M502+M503+M504+M505 = 2+2+2+2+2 = 10 Days

33 In this MCSS T&E the closeout work packages to be performed during the execution phase were listed in the

work breakdown structure (WBS) section 4.1.1. The only difference between the usual work packages and the

closeout work packages is that the latter are usually short in duration because they are small in duration. Guy L.

De Furia. Project Management Recipes for Success. CRC Press, December 2008, ISBN 10: 1420078240. Page

207-208 34 The PPA allows data to be gathered about their performance and experiences so that project processes may be

tuned to improve performance on future projects. Robert T. Futrell; Donald F. Shafer; Linda I. Safer. Quality

Software Project Management. Prentice Hall, January 2002, ISBN 10: 0-13-091297-2. Page 1090-1092

44

Figure 4. 19 PERT-CPM Project Closeout Phase

4.4.4 Project Closeout Resource Allocation

Project Manager will be fully involved in handle project close out. Detailed itemization below

shows of the PM (Type V) allocated to perform major closeout activity in the MCSS T&E

work breakdown structure.35

TASK

IDACTIVITY RES DUR ES LF TS 1 2 3 4 5 1 2 3 4 5

M501 Perform client closeout 1 2 0 2 0 1 1

M502Perform organizational

closeout1 2 2 4 0 1 1


closeout1 2 4 6 0 1 1

M504Write the project final

report1 2 6 8 0 1 1

M505 Conduct team closeout 1 2 8 10 0 1 1

1 1 1 1 1 1 1 1 1 1

Week 13 Week 14

Total Resource Load

Figure 4. 20 Resource allocation of Project Closeout Phase

35 Since the Project Closeout only perform by Project Manager which is in Management stream activities

therefore, the graph of Engineers Type and Type of Labor not included in this section.

45

4.4.5 Project Closeout Budget Allocation

Total manpower cost is for perform project closeout is $5000.

Table 4. 15 Budget allocation Project Closeout Phase


500 Project Closeout Engineers Type Number Unit

Rate/day

M501 Perform client closeout Project Manager Type V 1 $500 2 $1,000 $1,000 $1,000

M502 Perform organizational closeout Project Manager Type V 1 $500 2 $1,000 $1,000 $2,000


closeoutProject Manager Type V 1 $500 2 $1,000 $1,000 $3,000

M504 Write the project final report Project Manager Type V 1 $500 2 $1,000 $1,000 $4,000

M505 Conduct team closeout Project Manager Type V 1 $500 2 $1,000 $1,000 $5,000

Sub-total $5,000 $5,000


days

Manpower

cost

High level

sub-total

cost

Cumulative

cost

M501 M502 M503 M504 M505

High level sub-total cost $1,000 $1,000 $1,000 $1,000 $1,000

Cumulative cost $1,000 $2,000 $3,000 $4,000 $5,000

$0

$1,000

$2,000

$3,000

$4,000

$5,000

$6,000

Project Closeout Phase

Figure 4. 21 Budget accumulative Project Closeout Phase

46

5. LESSON LEARNED

This section describe and discuss the lessons learned session which incorporate in the TEMP.

ID SESSIONS LESSON LEARNED

01 An introduction to Project Management

and Systems Engineering

Defined the types of activities performed in the development of

a TEMP

Defined the SPARK element as part of Project Plan to

incorporate in the TEMP

02 Modern project management

Construct work packages for the processes of TEMP

Applied the monitor progress through incorporate in the work

packages.

03 Project Selection and Strategy

Applied the five types of engineer-leaders to allocate the

human resources more efficient in the TEMP process.

Introduced the difference between a role (job) and an activity

to be performed in the TEMP

Used the Waterfall view as the basis for the work performed

between each of the 4 formal major milestones in the TEMP.

04 Project Organisation Able to applied and defended choice of matrix organization as

management structure for the MCSSRP TEMP

05 Project Planning

Able to applied the essential elements of a project plan

(SPARK) in the TEMP

Understand how to create and use a Work Breakdown

Structure (WBS) by using Top-Down approach which

identifies the needed tasks, resource allocations, and cost

estimates.

06 Scheduling

Able to developed scheduling estimates to T&E activities by

using Bottom-Up approach to estimate the project time

Able to develop the project network, the critical path and its

importance.

Applied resource allocation to schedule T&E activities

07 Cost Estimating

Able to applied the methods for estimating project times and

costs. Bottom-Up approach is used in the TEMP.

Able to applied different types of cost contracts for work

activities in the work breakdown structure. The activity budget

estimated the cost for activity personnel where the type of cost

is price contract

09 Performance Monitoring Able to applied the project control process tools to incorporate

in the work activities to mitigate and prevent project failure

10 Project Leadership Able to defined the scope of the project manager for the

TEMP.

47

11 Project Teams Able to defined the human element in the TEMP process and

managing personnel skills and competencies, conflict and

communications through perform in the work activities.

12 Constraint Management Applied risk management process through work activities to

mitigate and prevent project risks.

14 Project Audit and Closeout Able to applied project closure activities to be performed and

evaluate project personnel by using Post Perfomance Analysis

method.

In overall the TEMP documentation keep the project controlled, standardized the T&E

process and improve the communication and the understanding of the T&E process. Using

Vee model and System Approach for Test and Evaluation can prevent a big loss and improve

the system development, since the failure detected earlier, not after the production of the

MCSS system. Milestone is needed as a correction control across the project and also for

verification and validation of the MCSS system.

48

6. BIBLIOGRAPHY

1. A Guide to the Project Management Body of Knowledge (PMBOK" Guide), Project

Management Institute, Upper Darby, PA, 2004

2. Bell, David.W and Brown, David.C. Systems Engineering and Test and Evaluation—The

Integrated Process. ITEA Journal 2010

3. Eisner, H., Essentials of Project and Systems Engineering Management, 3rd

Edition,

Wiley Interscience, 2008, ISBN 978-0-470-12933-3

4. Forsberg, K., Mooz, H. and Cotterman, H., Visualising Project Management, John Wiley

& Sons, Inc., 3rd edition, 2005.

5. Gray, C., and Larson, E., Project Management, 4th Edition, McGraw H ill, 2007, ISBN

978-0-07-1228751-7.

6. Gregory S. Parnell, Patrick J. Driscoll, Dale L. Henderson (2011). Decision Making in

Systems Engineering and Management, Second Edition. John Wiley & Sons, Inc.

7. Guy L. De Furia. Project Management Recipes for Success. CRC Press, December 2008,

ISBN 10: 1420078240.

8. Hari, A., Shoval, S. and Kasser, J. E., ―Conceptual Design to Cost: A new systems

engineering tool‖, proceedings of the 18th International Symposium of the INCOSE,

Utrecht, Holland, 2008.

9. Horngren, C., Sundem, G. and Stratton, W., 2005. Introduction to Management

Accounting, New Jersey, Pearson.

10. Ichiro Koshijima and Tomio Umeda. Human Resource Allocation in Project

Management. Management Science Approach.


11. INCOSE., ―System Engineering Handbook Version 2.0‖, Seattle, 2000

12. Kasser, J. E. and Frank, M., ―A Maturity Model for the Competency of Systems

Engineers‖, proceedings of the 20th International Symposium of the INCOSE, Chicago,

IL., 2010.

13. Kasser, J. E. and Hitchins, D. K., ―Unifying the different systems engineering processes‖,

proceedings of Conference on Systems Engineering Research, Hoboken, NJ., 2010.

14. Kasser, J. E. and Mirchandani, C. J., ―The MSOCC Data Switch Replacement: A Case

Study in Elicitating and Elucidating Requirements‖, proceedings of the 15th International

Symposium of the International Council on Systems Engineering (INCOSE), Rochester,

NY, 2005.

15. Kasser, J. E., ―Improving the Systems Engineering Documentation Production Process‖,

proceedings of the 5th Annual International Symposium of the NCOSE, 1995

16. Kasser, J. E., ―The Hitchins-Kasser-Massie (HKM) Framework for Systems

Engineering‖, proceedings of the 17th International Symposium of the INCOSE, San

Diego, CA., 2007.

49

17. Kasser, J. E., A Framework for Understanding Systems Engineering, Booksurge Ltd,

2007, ISBN 978-1-4196-7315-3

18. Kasser, J. E., Applying Total Quality Management to Systems Engineering, Artech

House, Boston, 1995, Chapter 10.

19. Kasser, J. E., Frank, M. and Zhao, Y. Y., ―Assessing the competencies of systems

engineers‖, proceedings of the 7th biannual European Systems Engineering Conference

(EuSEC), Stockholm, Sweden, 2010.

20. Kasser, J. E., Hitchins, D. and Huynh, T. V., ―Reengineering Systems Engineering‖,

proceedings of the 3rd Annual Asia-Pacific Conference on Systems Engineering

(APCOSE), Singapore, 2009.

21. Kezsbom, D. S., Schilling, D. L. and Edward, K. A., Dynamic Project Management. A

practical guide for managers and engineers, John Wiley & Sons, New York, 1989.

22. Kossiakoff, A. & Sweet, W. N. (2003). Systems Engineering Principles and Practice.

Hoboken: John Wiley ISBN: 0-471-23443-5.

23. Practice Standard for Work Breakdown Structures (Second Edition), published by the

Project Management Institute, ISBN 1933890134

24. Project Management Institute A Guide to the Project Management Body of Knowledge,

Third Edition, 2004 Project Management Institute, Inc. ISBN 193069945X

25. Robert T. Futrell; Donald F. Shafer; Linda I. Safer. Quality Software Project

Management. Prentice Hall, January 2002, ISBN 10: 0-13-091297-2.

26. Shenhar, A. J. and Bonen, Z., ―The New Taxonomy of Systems: Toward an Adaptive

Systems Engineering Framework‖, IEEE Transactions on Systems, Man, and Cybernetics

– Part A: Systems and Humans, Vol. 27 (1997), no. 2, 137 – 145.

27. Tesch, Debbie; Kloppenborg, Timothy J; Frolick, Mark N. IT Project Risk Factors: The

Project Management Professionals Perspective. The Journal of Computer Information

Systems. July 1 2007.

50

7. APPENDIX 1. WORK PACKAGES DESCRIPTION

Appendix A: Preliminary Design (SRR-PDR)

Task ID 100

Name of Activity Preliminary Design (SRR-PDR)

Priority High

Narrative of activity Start of the project, after requirement has been given and up to when the

preliminary design is chosen.

Reason of activity [CONOPS] To plan for the testing & evaluation activities of the new MCSS according to

the requirement and agreed by stakeholders.

Schedule (+accuracy) 20 days (90% accuracy)

Product F100: Test Plan Document

Acceptance criteria of product Reviewed and approved during customer review meeting

Prerequisites Pre-System Requirement Review (Approved)

Resources

Personnel

R1:Project Manager (Type V) x 1 (Full Time)

R2:System Engineer (Type III) x 1 (Part Time)

R3:Senior System Engineer (Type V) x 1 (Part Time)

R4:Senior Test Engineer (Type V) x 1 (Part Time)

R5:Test Engineer (Type III) x 2 (Full Time)

R7:QA Engineer (Type III) x 1 (Full Time)

Equipment

MS Office

MS Project

Estimated cost $12,950

Accuracy of cost estimate -10%, +30% (Bottom up approach)

Risk (probability, mitigation

WP ID) Probably, Low risk, M101, Q101, Q102, Q103, Q104

Internal key milestones SRR-PDR

Decision points Plan approved by SRR meetings

Assumptions None

Lower level work packages ID M101, M102, Q101, Q102, Q103, Q104

51

Task ID M101

Name of Activity Technical direction and acquire resources

Priority High

Narrative of activity To provide a plan for activities to be conducted, assigned required resources

and provide technical direction and guidance for the T&E activities

Reason of activity [CONOPS] To facilitate a smooth running of T&E activities including required resources

at SRR-PDR


Product F101: Resources, Schedule and Cost Plan

F102: Methodology for Testing and Validation

Acceptance criteria of product Reviewed and approved during review meeting

Prerequisites None

Resources

Resources R1:Project Manager (Full Time)

Equipment

MS Office

MS Project

Estimated Cost $6000

Accuracy of Cost Estimate -10%, +30% (Bottom up approach)


WP ID) Likely and Low risk


Decision points Plan approved by Project Manager

Assumptions BA.01, BA.02, TA.01, TA.02, TA.03, SA.01, SA.02

Lower level work packages ID M101.1, M101.2, M101.3

52

Task ID M101.1

Name of Activity Resources plan for T&E

Priority High

Narrative of activity

Allocated resources to each major work activity in the T&E work breakdown

structure. Resources include the numbers and required skill levels of

personnel.

Reason of activity [CONOPS] To assure that the necessary resources are made available when required


Product F101.1 : Resources Plan Document

Acceptance criteria of product Reviewed and approved by PM

Prerequisites None

Resources

Resources

R1:Project Manager (Full Time)

Equipment MS Office

MS Project







Assumptions TA.02, TA.03, SA.01, SA.02

Lower level work packages ID None

53

Task ID M101.2

Name of Activity Cost and Schedule Plan

Priority High

Narrative of activity Plan a detailed breakdown of necessary resource budgets and schedule for

each of the major work activities in the work breakdown structure.

Reason of activity [CONOPS]

To allocate resource budgets for each of the major work activities in order to

minimize the over cost during execution of the task and to ensure the

necessary resources schedule are allocate properly.


Product F101.2 : Cost and Schedule Plan Document


Prerequisites None

Resources


Equipment

MS Office

MS Project







Assumptions BA.01, BA.02, TA.01, TA.02, TA.03


54

Task ID M101.3

Name of Activity Subsystem test direction

Priority High

Narrative of activity Planning guidance for subsystem test to be performed in testing stream work

activities.

Reason of activity [CONOPS] To provide detailed planning and direction of the test program are in order36


Product F101.3 : Subsystem Test Guidance


Prerequisites None

Resources


Equipment

MS Office

MS Project







Assumptions None


36 Subsystem direction planning is needed to encountered the faults during execution of individual subsystem test

which will have greater potential impact on overall program cost and schedule. Kossiakoff, A. & Sweet, W. N.

(2003). Systems Engineering Principles and Practice. Hoboken: John Wiley ISBN: 0-471-23443-5. Chapter 10:

Integration and Evaluation Page 292

55

Task ID M102

Name of Activity Risk identification and management

Priority High

Narrative of activity To document initial risk assesment and risk mitigation efforts at major work

activity in the T&E work breakdown structure,

Reason of activity [CONOPS] To identify and mitigate risk at major work activity in the T&E work

breakdown structure37.


Product F103: Risk Management Plan

Acceptance criteria of product Approved by Project Manager

Prerequisites None

Resources


Equipment

MS Office







Assumptions Project Manager act as a role of Risk Manager


37 Opportunity and risk is ongoing that is we have to perform the risk management activities in every phase of

MCSS T&E and depicted in the work packages. Forsberg, K., Mooz, H. and Cotterman, H., Visualising Project

Management, John Wiley & Sons, Inc., 3rd edition, 2005. Page223 – 253

56

Task ID Q101

Name of Activity Execute purchased Switch test38

Priority High


The Purchased Switch Test is performed following the Test Procedure

document. Test has to satisfy the Test Pass/Fail criteria defined in the MCSS

SRD including Verification and Validation of preliminary design to ensure it

meet requirements.

Reason of activity [CONOPS] To test whether the Purchased Switch is meet the requirements. To learn

if the Purchased Switch exceeds the system requirement.


Product F104: Purchased Switch Test Report

Acceptance criteria of product Test is performed according to the guidance provide by the PM.

Prerequisites M101.3 Subsystem test direction

Resources

Resources



Equipment

Test Rigs

288 Ports Switch Matrix




WP ID) May and Medium risk


Decision points Test approved by Project Manager

Assumptions QA.01


38 As we bought the COTS Industry Switch from the market, those purchase switch should have immediately test

for immediate feedback result and will be served as independent testing. This decision eliminated a major risk,

and determined that the requirements for the replacement switch would be feasible, and achievable within

budget. Kasser, J. E. and Mirchandani, C. J., ―The MSOCC Data Switch Replacement: A Case Study in

Elicitating and Elucidating Requirements‖, proceedings of the 15th International Symposium of the International

Council on Systems Engineering (INCOSE), Rochester, NY, 2005. Page 7

57

Task ID Q102

Name of Activity Prepare modified Swith Matrix test plan and procedure39

Priority High


Planning the Modified Switch Matrix Test. This is including the Test

Objective, Test Methodology, and Test Pass/Fail Criteria for the Modified

Switch Matrix

Reason of activity [CONOPS] To modify COTS switch components in step by step to meet the requirements

and interface of MCSS to the MSOCC


Product F105: Modified Switch Matrix Test Plan Document


Prerequisites MCSS System Requirement Document

Resources Resources

R4:Senior Test Engineer (Type V) (PartTime)




WP ID) Probably and Low risk


Decision points Approved by Project Manager

Assumptions None


39 Before designing the test equipment it is important fully to define the test procedures, so as to avoid later

redesign to achieve compatibility between test equipment and the component or subsystem under test. This again

emphasizes the importance of early and comprehensive test planning. Kossiakoff, A. & Sweet, W. N. (2003).

Systems Engineering Principles and Practice. Hoboken: John Wiley ISBN: 0-471-23443-5. Chapter 10:

Integration and Evaluation Page 283

58

Task ID Q103

Name of Activity Prepare integration test plan and procedure40

Priority High


Planning the MCSS Integration Test Plan and Procedure. This is including the

Test Objective, Test Methodology, and Test Pass/Fail Criteria for the

Purchased Switch Test.

Reason of activity [CONOPS] To provide a test plan and procedure of MCSS for the tester


Product F106: MCSS Integration Test Plan and Procedure Document

Acceptance criteria of product Test Pass/Fail criteria based on MCSS System Requirement Document and

the test procedure is according to the Test Plan.


Resources Resources R3:Senior System Engineer (Type V) (PartTime)







Assumptions None


40 Integration test plan and procedure is needed to translate requirements into a description of each test even and

to develop detailed test procedure for each event. The integration of a subsystem from its components parts is

normally a stepwise assembly and test process. Kossiakoff, A. & Sweet, W. N. (2003). Systems Engineering

Principles and Practice. Hoboken: John Wiley ISBN: 0-471-23443-5. Chapter 10: Integration and Evaluation

Page 281-288

59

Task ID Q104

Name of Activity Implement Introductory training

Priority High


The System Engineer will be introduced to the MCSS System

Requirement. The Senior Test Engineer will be trained for developing the

test plan and procedure and familiarize with the Test Equipment.

The Test Engineer and Quality Engineer will be trained to perform the

Test in MCSS system and be familiarized with the MCSS nature and

requirement.


To familiarize the manpower who will performed in the T&E with the

MCSS system, Test Methodology and Test Equipment to maintain the

Test Quality standard and to prevent defect in T&E activity


Product F106: MCSS Integration Test Plan and Procedure Document

Acceptance criteria of product Test Pass/Fail criteria based on MCSS System Requirement Document and

the test procedure is according to the Test Plan.


Resources Resources R3:System Engineer (Type III) (PartTime)







Assumptions SA.01


60

Appendix B: Detailed Design (PDR-IRR)

Task ID 200

Name of Activity Detailed Design (PDR-IRR)

Priority High


Choosing a single preliminary design which has been approved and purchase

the COTS products, the in-house engineering team will build the MCSS and

the test team will plan for the testing & evaluation of the integrated MCSS

and to test the subsystems in a stand-alone manner


To document consensus that a single preliminary design is chosen from a

range of options, document decision to move to construction phase and

moving on to construction phase as well as stand-alone testing of the

subsystems


Product F200: Test Procedures Document

Acceptance criteria of product Meet requirement and the MCSS subsystems pass the stand-alone test

requirements

Prerequisites Preliminary Design Review

Resources

Personnel



R4:Senior Test Engineer (Type V) x 1 (Part Time)



Equipment MS Office

MS Project




WP ID) Probably, Low risk, M201, Q201, Q202, Q203, Q204, Q205

Internal key milestones PDR-IRR

Decision points Plan approved by PDR meetings

Assumptions None

Lower level work packages ID M201, M202, Q201, Q202, Q203, Q204, Q205

61

Task ID M201

Name of Activity Technical direction and management reporting

Priority High

Narrative of activity Monitor progress and accelerate agency staff training and other support

measures as needed.

Reason of activity [CONOPS] To provides monthly progress reports to the stakeholders summarizing work

completed, a financial report, issues, and meetings.


Product F201: Resources, Schedule and Cost Review Report

F202: Methodology for Integration Testing


Prerequisites F101 Baseline Schedule, Cost and Resources.

Resources


Equipment

MS Office

MS Project






Decision points Approved by the Stakeholders



62

Task ID M201.1

Name of Activity Resources plan review

Priority High


Allocated resources plan to each major work activity in the T&E work

breakdown need to be review in monthly. Resources include the numbers and

required skill levels of personnel.


To review allocated resources of each major work activity which has been

performed in the T&E work breakdown structure to ensure that the resources

are available during execution of the task.


Product F201.1 : Resources Review Plan Document

Acceptance criteria of product Reviewed and approved during project meeting

Prerequisites None

Resources

Resources


Equipment MS Office

MS Project









63

Task ID M201.2

Name of Activity Cost and Schedule review

Priority High


Plan a detailed breakdown of necessary resource budgets and schedule for

each of the major work activities in the work breakdown structure need to be

review in monthly status.

Reason of activity [CONOPS] To monitor cost and schedule for the activities and for estimating cost to

complete and schedule completion on a monthly basis.


Product F201.2 : Cost and Schedule Review Document


Prerequisites None

Resources

Resources


Equipment MS Office

MS Project






Decision points Plan review approved by Project Manager



64

Task ID M201.3

Name of Activity Subsystem integration test direction

Priority High

Narrative of activity Defining guidance plans for integration and system testing

Reason of activity [CONOPS] To provide detailed planning and direction of the integration test program are

in order41


Product F201.3 : Subsystem Test Guidance


Prerequisites None

Resources


Equipment

MS Office

MS Project







Assumptions None


41 In this MCSS T&E, the PM provides guidance for successful implementation of integration subsystem testing.

The success of the integration is highly dependent on the advance planning and preparation for this effort that

was accomplished during the previous phases. Kossiakoff, A. & Sweet, W. N. (2003). Systems Engineering

Principles and Practice. Hoboken: John Wiley ISBN: 0-471-23443-5. Chapter 10: Integration and Evaluation

Page 273

65

Task ID M202

Name of Activity Risk management

Priority High

Narrative of activity To monitor risk assesment and risk mitigation efforts which would be occur

at major work activity in the T&E work breakdown structure.


breakdown structure.




Prerequisites None

Resources


Equipment

MS Office






Decision points Monitor by Project Manager



66

Task ID Q201

Name of Activity Execute modified Switch Matrix test

Priority High


The Modified Switch Matrix Test is performed following the Test

Plan document. Test has to satisfy the Test Pass/Fail criteria defined in the

MCSS SRD including Verification and Validation of MCSS to ensure the

specifications and requirements of the subsystems entities are met.

Reason of activity [CONOPS] To test whether the Modified Switch Matrix is meet the requirements in

System requirements Document Section 5.


Product F204: Modified Switch Matrix Report


Prerequisites Q102 Modified Switch Matrix Test Plan and Procedure

Resources

Resources



Equipment

Test Rigs

Oscilloscope







Assumptions QA.01


67

Task ID Q202

Name of Activity Prepare modified Switch Control test plan and procedure

Priority High


Planning the Modified Switch Control Test. This is including the Test

Objective, Test Methodology, and Test Pass/Fail Criteria for the Modified

Switch Control

Reason of activity [CONOPS] To prepare switch control components to meet the requirements and interface

of MCSS to the MSOCC


Product F205: Modified Switch Control Test Plan Document



Resources Resources








Assumptions None


68

Task ID Q203

Name of Activity Prepare Timing Generator test plan and procedure

Priority High


Planning the Timing Generator Test. This is including the Test Objective,

Test Methodology, and Test Pass/Fail Criteria for the Modified Switch

Control

Reason of activity [CONOPS] To prepare timing generator meet the requirements and interface of MCSS to

the MSOCC


Product F206: Timing Generator Test Plan Document



Resources Resources








Assumptions None


69

Task ID Q204

Name of Activity Prepare Test and Monitoring test plan and procedure

Priority High


Planning the Test and Monitoring test. This is including the Test Objective,

Test Methodology, and Test Pass/Fail Criteria for the Modified Switch

Control

Reason of activity [CONOPS] To prepare test and monitoring components meet the requirements and

interface of MCSS to the MSOCC


Product F207: Test and Monitoring Test Plan Document



Resources Resources








Assumptions None


70

Task ID Q205

Name of Activity MCSS Subsystem RAM analysis42

Priority High


The reliability data, in terms of MTBF will be evaluated for each individual

subsystem. The reliability of the system is refering to MCSS SRD.

Maintainability is defined according to MCSS SRD.

Reason of activity [CONOPS] To ensure subsystem design components meet the operational requirements

and interface of MCSS to the MSOCC


Product F208: RAM Analysis Report

Acceptance criteria of product Test is performed according to the MCSS SRD Operational Test


Resources Resources







Decision points Approved by PM

Assumptions QA.01


42 In MCSS T&E, the RAM analysis is performed to ensure the subsystem process operates properly for a

specified amount of time (design life) under stated use conditions without failure. What constitutes failure for a

component, subsystem, system, or process must be clearly defined as the item or process is developed. Gregory

S. Parnell, Patrick J. Driscoll, Dale L. Henderson (2011). Decision Making in Systems Engineering and

Management, Second Edition. John Wiley & Sons, Inc. Chapter 8 Page 228

71

Appendix C: Development and Integration (IRR-TRR)

Task ID 300

Name of Activity Development and Integration (IRR-TRR)

Priority High

Narrative of activity To install and integrate the new and tested MCSS into the existing system

and to finalize the final acceptance test plan

Reason of activity [CONOPS] To ensure that the MCSS is installed and integrated and to plan for the final

acceptance test


Product F300: Final Subsystem Test Report

Acceptance criteria of product The integrated MCSS can function and the Final acceptance test plan will test

the MCSS according to requirement and to the satisfaction of the client

Prerequisites Integration Readiness Review

Resources

Personnel





Equipment MS Office

MS Project




WP ID) May and Medium risk, M301, Q301, Q302, Q303, Q304, Q305

Internal key milestones IRR-TRR

Decision points Plan approved by IRR meetings

Assumptions None

Lower level work packages ID M301, M302, Q301, Q302, Q303, Q304, Q305

72

Task ID M301

Name of Activity Technical direction and management reporting

Priority High

Narrative of activity Monitor progress and accelerate agency staff training and other support

measures as needed.

Reason of activity [CONOPS] To provides monthly progress reports to the stakeholders summarizing work

completed, a financial report, issues, and meetings.


Product F301: Resources, Schedule and Cost Review Report

F302: Methodology for Integration Testing



Resources


Equipment

MS Office

MS Project









73

Task ID M301.1

Name of Activity Resources plan review

Priority High


Allocated resources plan to each major work activity in the T&E work

breakdown need to be review in monthly. Resources include the numbers and

required skill levels of personnel.


To review allocated resources of each major work activity which has been

performed in the T&E work breakdown structure to ensure that the resources

are available during execution of the task.


Product F301.1 : Resources Review Plan Document


Prerequisites None

Resources

Resources


Equipment MS Office

MS Project









74

Task ID M301.2


Priority High










Prerequisites None

Resources

Resources


Equipment MS Office

MS Project









75

Task ID M301.3

Name of Activity MCSS subsystem integration test direction (verified)

Priority High

Narrative of activity Defining guidance plans for integration overall subsystem testing

Reason of activity [CONOPS] To provide detailed planning and direction of the integration test program are

in order


Product F301.3 : MCSS Subsystem Test (Verified) Guidance


Prerequisites None

Resources


Equipment

MS Office

MS Project







Assumptions None


76

Task ID M302


Priority High








Prerequisites None

Resources


Equipment

MS Office









77

Task ID Q301

Name of Activity Execute modified Switch Control test

Priority High


The Modified Switch Control Test is performed following the Test




Reason of activity [CONOPS] To test whether the Modified Switch Control is meet the requirements in



Product F304: Modified Switch Control Report


Prerequisites Q202 Modified Switch Control Test Plan and Procedure

Resources

Resources



Equipment

Test Rigs

Oscilloscope







Assumptions QA.01


78

Task ID Q302

Name of Activity Execute modified Timing Generator test

Priority High


The Modified Timing Generator Test is performed following the Test




Reason of activity [CONOPS] To test whether the Modified Timing Generator is meet the requirements in



Product F305: Modified Timing Generator Report


Prerequisites Q203 Modified Timing Generator Test Plan and Procedure

Resources

Resources



Equipment

Test Rigs

Oscilloscope







Assumptions QA.01


79

Task ID Q303

Name of Activity Execute modified Test and Monitoring test

Priority High


The Modified Test and Monitoring Test is performed following the

Test Plan document. Test has to satisfy the Test Pass/Fail criteria defined in

the MCSS SRD including Verification and Validation of MCSS to ensure the


Reason of activity [CONOPS] To test whether the Test and Monitoring is meet the requirements in System

requirements Document Section 5.


Product F306: Test and Monitoring Report


Prerequisites Q204 Modified Test and Monitoring Test Plan and Procedure

Resources

Resources



Equipment

Test Rigs

Oscilloscope

NBG

BED







Assumptions QA.01


80

Task ID Q304

Name of Activity Prepare finalize the whole subsystem integration test

Priority High

Narrative of activity Finalization preparation of the whole subsystem integration test performed

following the integration Test Plan document.

Reason of activity [CONOPS] To test whether the final assembly of whole subsystem is meet the

requirements in System requirements Document Section 5.


Product F307: Assembly Subsystem Integration Test Report


Prerequisites Q103 Subsystem Integration Test Plan and Procedure

Resources Resources R3:Senior System Engineer (Type V) x 1 (Part Time)




WP ID) Probably and Medium risk



Assumptions None


81

Task ID Q305

Name of Activity Final MCSS subsystem integration test (verified)

Priority High


The Final Whole Subsystem Integration Test is performed following the

Assembly Subsystem Integration Plan document. Test has to satisfy the Test

Pass/Fail criteria defined in the MCSS SRD including Verification and

Validation of MCSS to ensure the specifications and requirements of the

subsystems entities are met.


To test whether the final assembly of whole subsystem is meet the

requirements in System requirements Document Section 5 and to ensure the

whole final assembly subsystem are verified to be test on on-site test.


Product F308: Final Assembly Subsystem Integration Test Report


Prerequisites Q304 Assembly Subsystem Integration Test

Resources

Resources



Equipment

Test Rigs

Oscilloscope

Block Error Detector

Nascom Block Generator




WP ID) Probably and Medium risk



Assumptions QA.01


82

Appendix D: Qualification and Installation (TRR-DRR)

Task ID 400

Name of Activity Qualification and Installation (TRR-DRR)

Priority High

Narrative of activity To conduct the final acceptance test for the MCSS and to prepare for the

handing over/delivery of the MCSS to the client

Reason of activity [CONOPS] To document consensus that the final product is tested, defect free and is

ready to be delivered


Product F400: Final Acceptance Test Report

Acceptance criteria of product Products meet requirements and pass the final acceptance testing

Prerequisites Test Readiness Review

Resources

Personnel


R2:System Engineer (Type V) x 1 (Part Time)


R6: Logistic Engineer (Type II) x 1 (Part Time)


Equipment

MS Office

MS Project




WP ID) May and Medium risk, M401, M403, Q401, Q402, Q403, Q404

Internal key milestones TRR-DRR

Decision points Plan approved by TRR meetings

Assumptions None

Lower level work packages ID M401, M402, M403, Q401, Q402, Q403, Q404

83

Task ID M401

Name of Activity Handing over final document

Priority High

Narrative of activity Handing over/delivery of the MCSS document to the client

Reason of activity [CONOPS] To document consensus that the final product is tested, defect free and is

ready to be delivered


Product F401: Final MCSS Document Report

Acceptance criteria of product Products meet requirements and pass the final user acceptance testing

Prerequisites None

Resources Personnel R2:System Engineer (Type V) x 1 (Part Time)

Estimated cost $750





Decision points Plan approved by TRR meetings

Assumptions None


84

Task ID M402


Priority High








Prerequisites None

Resources


Equipment

MS Office









85

Task ID M403

Name of Activity Test and Evaluation completion review

Priority High

Narrative of activity Review the test and evaluation work activities has satisfy the requirements to

indicate project status and schedule

Reason of activity [CONOPS] To ensure the test and evaluation work activities performed at the entire

system life cycle.


Product F403: Schedule and Cost Completion Review Report

F404: Completion Report



Resources


Equipment

MS Office

MS Project








Lower level work packages ID M403.1, M403.2

86

Task ID M403.1


Priority High










Prerequisites None

Resources

Resources


Equipment MS Office

MS Project









87

Task ID M403.2

Name of Activity Project completion review

Priority High


Project completion review is prepared by Project Manager. Project

completion report finalizes all project activities completed across all phases

of the T&E activities to formally complete the project and transfer the

completed as appropriate.

Reason of activity [CONOPS] To assess the project, ensure completion, and derive any lessons learned and

best practices to be applied to future projects.


Product F403.2 : Project Completion Report


Prerequisites None

Resources

Resources


Equipment

MS Office

MS Project







Assumptions SA.02


88

Task ID Q401

Name of Activity Final MCSS subsystem operational test

Priority High


A Factory Acceptance Test witnessed by the transition test to test whether

each subsystems of the integrated MCSS is meet the requirement in section

5 and can operate as expected in user demanded simulated condition

Reason of activity [CONOPS] To assess the project, ensure completion, and derive any lessons learned and

best practices to be applied to future projects.


Product F405: Final MCSS Subsystems Operational Test Report

Acceptance criteria of product

The 4 subsystem meet the requirement MCSS SRD section 5. The

subsystem can work as expected in user demanded simulated

condition.

Prerequisites Q304 Final whole subsystem integration test plan and procedure

Resources

Resources



Equipment

Test Rigs

Oscilloscope









Assumptions QA.01, QA.02


89

Task ID Q402

Name of Activity Final MCSS subsystem factory acceptance test

Priority High

Narrative of activity Review the result of the test documented in the Factory Acceptance Test

Report for verification and validation of the MCSS


Test Engineer reviewed the result of the Test to verify and validate the

MCSS being tested. With this review, an Early faulty/failure can be

detected and reported during the DRR


Product F406: Final MCSS Subsystems Factory Acceptance Test Report




condition.


Resources

Resources



Equipment

Test Rigs

Oscilloscope











90

Task ID Q403

Name of Activity Final MCSS subsystem site acceptance test

Priority High

Narrative of activity The integrated site acceptance test will occur before and after each transition

stage. The transition team will use designated milestone test procedure.

Reason of activity [CONOPS] To assured the usability of the MCSS at each of the transition stage during

system installation


Product F407: Final MCSS Subsystems Site Acceptance Test Report




condition.


Resources

Resources



Equipment

Test Rigs

Oscilloscope











91

Task ID Q404

Name of Activity Final MCSS subsystem customer acceptance test

Priority High


The fully integrated MCSS is tested. Quality Assurance Engineer reviewed

the result of the Test to verify and validate the MCSS being tested in

customer site. Logistic Engineer ensures the reliability of all subsystems and

modifies system design accordingly to achieve customer requirements.


To test the fully integrated MCSS after being installed but before Final

Acceptance Test by NASA to be considered fully operational. Review the

result of the test documented in the Factory Acceptance Test Report for

verification and validation of the MCSS in the customer site before being

handled to NASA.


Product F407: Final MCSS Subsystems Customer Acceptance Test Report




condition.


Resources

Resources



Equipment

Test Rigs

Oscilloscope











92

Appendix E: Project Closeout

Task ID 500

Name of Activity Project Closeout

Priority High


A project closeout activity is to end the project in a professional manner one

that reflects well upon the team, the organization, and especially upon the

project manager.


To assure that the activities will be conducted in an appropriate manner and

that the resources (people, money, and time) for the activities will also be

available at the end of the project.


Product F500: Project Final Report

Acceptance criteria of product Products meet requirements during the stakeholder meeting

Prerequisites Delivery Readiness Review

Resources

Personnel


Equipment

MS Office

MS Project

Estimated cost $5000



WP ID) None

Internal key milestones Closeout

Decision points Approved by Stakeholders (NASA GSFC) and Executive Management (CSC)

Assumptions None

Lower level work packages ID M501, M502, M503, M504, M505

93

Task ID M501

Name of Activity Perform client closeout

Priority High


Validate deliverable(s) submission and document that all deliverables were

accepted by the customer: If no documentation exists, write a memo to the

customer with a retained copy indicating that the deliverables were submitted

and the date of the submission.

Reason of activity [CONOPS] To ensure that all deliverables were accepted by the customer.


Product F501: Validation Deliverables of Document

Acceptance criteria of product Products accepted by customer

Prerequisites None

Resources

Personnel


Equipment

MS Office

MS Project




WP ID) None


Decision points Approved by Stakeholders

Assumptions None


94

Task ID M502

Name of Activity Perform organizational closeout

Priority High


Release the project room by sending a memo to the facilities management

office stating that the room will be vacated by a certain date, and available for

other uses. Finalize financial records and funds by preparing a finance

resolution memo. In it, indicate any outstanding invoices against the project

or invoices expected, plus the balance of money that will remain after these

invoices are paid.

Reason of activity [CONOPS] To conclude the team’s use of organizational resources and to make a final

resolution of the remaining resources.


Product F502: Organizational Closeout Document

Acceptance criteria of product Products accepted by Executive Management

Prerequisites None

Resources

Personnel


Equipment

MS Office

MS Project




WP ID) None


Decision points Approved by Executive Management

Assumptions None


95

Task ID M503

Name of Activity Conduct subcontractor closeout

Priority High


Determine whether subcontractors are finished: Document this in a memo for

record with a copy to finance so the subcontractor (Ford Aerospace) can be

paid.

Reason of activity [CONOPS] To ensure that all subcontractor responsibilities and work have been

completed.


Product F503: Subcontractor Closeout Document

Acceptance criteria of product Products accepted by Subcontractor

Prerequisites None

Resources

Personnel


Equipment

MS Office

MS Project




WP ID) None



Assumptions None


96

Task ID M504

Name of Activity Write the project final report

Priority High

Narrative of activity Write the report as the basis of a briefing. Project manager will be enhanced

by a well-organized and professional final report.

Reason of activity [CONOPS] To inform upper management of the completion of the project and to assure

these people that all aspects of the project have been properly closed.


Product F504: Project Final Report

Acceptance criteria of product Products accepted by Executive Management

Prerequisites None

Resources

Personnel R1:Project Manager (Type V) x 1 (Full Time)

Equipment MS Office

MS Project




WP ID) None



Assumptions None


97

Task ID M505

Name of Activity Conduct team closeout

Priority High


Conduct the final lessons learned session43: Something that the team did that

worked should be retained as a lesson learned because the next time the team

is faced with the same problem, it will want to use the action that was

successful in the past.


To release team members back to their functional departmentsand use the

lessons learned to get a sense of the team’s morale and shape behavior. The

result of these sessions will be to highlight, reward, and clarify desired

behavior.


Product F505: Lesson Learned Report

Acceptance criteria of product Products accepted by Project Team meetings

Prerequisites None

Resources

Personnel R1:Project Manager (Type V) x 1 (Full Time)

Equipment MS Office

MS Project




WP ID) None



Assumptions None


43 A lesson learned is an incident of experience that may be used to improve performance or repeat good

performance. Guy L. De Furia. Project Management Recipes for Success. CRC Press, December 2008, ISBN 10:

1420078240. Page 210

98

8. APPENDIX 2. RISKS FACTORS CATEGORIZATION

Risk

ID

Risk Factors and

CategoriesLow Risk (L) Medium Risk (M) High Risk (H)

R1 Project team stability

Project staff are expected to

stay with for the duration of

the project

One or more key project staff

are expected to leave before

their accountabilities are met

More than three key project

staff are expected to leave

before their accountabilities

are met

R2 Project processes

All project processes are

defined and being followed

One or more important and

complex project processes not

defined or being followed

More than three important and

complex project processes not

defined or being followed

R3 Customer experience

End-users are highly

experience in similar projects,

have specific ideas on how

needs can be met

End-users have experience

with similar project and have

needs in mind

End-users have no previous

experience with similar project,

unsure of how needs can be

met

R4 Customer acceptance

End-users accept concepts

and details of system, process

is in place for end-user

approvals

End-users accept most

concepts and details of

system, process in place for

end-user approvals

End-users do not accept any

concepts or design details of

system

R5 Budget fit

Cost Variance (CV) shows that

project is in line with or under

budget


project is exceeding budget by

1-5%


project is exceeding budget by

more than 5%

R6 Cost performance

Cost Performance Indicator

(CPI) has a value of

0.95 <= CPI <= 1.1


(CPI) has a value of 0.9 <= CPI

<= 1.2


(CPI) has a value of 0.9 >

CPI > 1.2

R7 Cost controls Well-established, in place System in place, weak in areas System lacking or non-existent

R8 Delivery commitment Stable commitment dates Some uncertain commitmentsUnstable, fluctuating

commitments

R9 Schedule fit

Schedule Variance (SV) shows

that project is in line with or

under schedule


that project is exceeding

schedule by 1-5%


that project is exceeding

schedule by more than 5%

R10 Schedule performance

Schedule Performance

Indicator (SPI) has a value of

0.95 <= SPI <= 1.1



0.9 <= SPI <= 1.2



0.9 >SPI > 1.2

R11 System testabilitySoftware requirements easy to

test, planning underway

Some requirements hard to

test, or test plans insufficient

Most or all of the

system hard to test, or no

planning being done

R12 System dependencies

Clearly defined dependencies

of the software effort and

other parts of the system

Some elements of the system

are well-understood and

planned, others are not yet

comprehended

No clear plan or schedule for

how the whole system will

come together

R13 Documents stabilityDocuments will be available on

time and will contain few errors

Some documents may be late

and contain minor errors

Little chance of getting

documents on time, many

corrections and changes

expected

Organization management factors

Customer factors

Budget/cost factors

Schedule factors

Project content factors

99

Risk

ID

Risk Factors and

CategoriesLow Risk (L) Medium Risk (M) High Risk (H)

R14 Test capability

Modular design allows for

easy coverage test planning

and execution

Modular design aids

developing test harnesses for

unit test

No modular design or ability to

easily establish test coverage

planning

R15 Expected test effort

Good estimate available,

readily fits system acceptance

process

Rough estimate of test time,

may be a bottleneck in the

process

Poor or no estimate of test

times, definite chance of

bottleneck

R16 Functionality

Defined software product

highly functional, meets all

customer needs

Defined software product has

good functionality, meets most

customer needs

Defined software product has

little functionality, many

customer needs not met

R17 Communication

Clearly communicates goals

and status between the team

and the rest of the organization

Communicates some of the

information some of the time

Rarely communicates clearly to

the team or to other who need

to be informed of team status

R18 Project manager experience

Project manager very

experienced with similar

projects

Project manager has moderate

experience or has experience

with different types of projects

Project manager has no

experience with this type of

project or is new to project

management

R19Quality assurance

approach

QA system established,

followed, effective

Procedures established, but

not well followed or effective

No QA process or established

procedures

R20Use of defined engineering

process

Development process in place,

established, effective, followed

by team

Development process

established, but not followed

or is ineffective

No formal process used

R21Change control for work

products

Formal change control process

in place, followed, effective

Change control process in

place, not followed, or is

ineffective

No change control process

used

R22 Tools availabilityTools in place, documented,

validated

Tools available, validated,

some development needed (or

minimal documentation)

Tools invalidated,

proprietary, or major

development needed, no

documentation

R23 Vendor support

Complete support

at reasonable price and in

needed time frame

Adequate support at

contracted price, reasonable

response time

Little or no

support, high cost, and/or

poor response time

R24 Staff adherence to task

Staff in place, no turnover

expected, working on assigned

tasks, few diversions and little

fire fighting

Staff available, some turnover

expected, some conflict in time

allocation, some fire fighting

Staff not available, high

turnover expected, many

members spend much time fire

fighting

R25Training of team

Training not

required, or training plan in

place and training ongoing

Training for some areas not

available or training planned

for future

No training plan or training not

readily available

R26 Team productivity

All milestones met,

deliverables on time,

productivity high

Milestones met,

some delays in deliverables,

productivity acceptable

Productivity low, milestones

not met, delays in deliverables

R27 Support personnelIn place, experienced,

sufficient in number

Missing some areas of

expertise

Significant discipline or

expertise missing

Performance factors

Development process factors

Development environment factors

Staff factors

Maintenance factors

NASA Test and Evaluation Master Plan

Documents

Transcript of NASA Test and Evaluation Master Plan