DEFENSE CONTRACT MANAGEMENT AGENCY [SITE LOCATION]

SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008

SYSTEM SURVEILLANCE REPORT (SSR)

INSTRUCTION AND TEMPLATE

JANUARY 2008

THE DCMA EVM CENTER DIRECTOR IS RESPONSIBLE FOR ISSUANCE AND

MAINTENANCE OF THIS DOCUMENT

Rev 1 Jun 08 Added FOUO to Template


I. The purpose of this document is to provide instructions and a template for the

System Surveillance Report used in conjunction with DCMA Instruction – Earned

Value Management System (EVMS) System-Level Surveillance. As stated in the

Instruction, the surveillance team will document the results of system surveillance

reviews in a written report that is signed and dated by the DCMA surveillance

team leader and supplier representative (if employing joint surveillance). The

report will be issued in accordance with the annual EVMS surveillance schedule

and should include an overall assessment of the supplier’s implementation of the

EVMS, scope of the review, personnel interviewed, and findings of deficiency or

non-compliance that resulted in CARs.

Minimum expectations for documentation to be addressed in the surveillance

report include:

Surveillance Selection Risk Matrix(s);

Guidelines or Process(es) reviewed;

PM and CAM(s) interviewed and Control Accounts examined;

Completed Guideline templates for each reviewed Guideline

o EV Templates 1-32

Completed EVMS Description Cross Reference Checklist; and

System deficiencies identified:

o Corrective Action Request(s);

o CAR(s) entered into central database;

o Supplier Corrective Action Plan in place;

o Actions taken to correct the deficiency; and

o CMO analysis for trends and systemic issues.

Sections of the report include:

Executive Summary

Scope

Surveillance Summaries

o Interview Summaries


o Independent Data Traces & Document Research (sometimes

attached to the EV Templates)

Surveillance Results

o Findings

o Corrective Action Request (CAR) Status

Guideline Compliance Summaries

II. THE EXECUTIVE SUMMARY

The SSR executive summary should, at a minimum, refer to the established

Standard Surveillance Plan (SSP), the supplier, and the members of the

surveillance team. It should include the dates of the surveillance and indicate if

any deficiencies were found in the implementation of the surveyed EVMS

Guidelines. The executive summary should also include:

An impact statement noting significant concerns raised by the surveillance

and the potential effects of those concern areas.

A reference to follow on actions or any open items regarding the

surveillance.

The attached SSR template provides example executive summary language.

II. THE SCOPE OF THE SURVEILLANCE

The surveillance scope section should include the following:

The start and finish dates of the surveillance

All guidelines and processes reviewed

The Program(s) being reviewed and the respective contract number(s)

The Customer or purchasing organization

An explanation of how the risk assessment results support the scope of

this review

An explanation of any revisions or deviations from the approval annual

EVMS surveillance schedule.

The DCMA CMO contact information

The Supplier EVM focal point’s contact information


III. SURVEILLANCE SUMMARIES

The surveillance summary section aligns with the scope of the review. It

describes the CAM discussions, the areas identified for independent data traces,

and any follow up research that was conducted. The attached SSR template

provides example surveillance summary language.

A. Interview Summaries

Each interview summary should, at a minimum:

Identify the CAM and his/her area of responsibility.

Identify the Guideline or Process area focus of each interview.

Summarize interview discussion points.

Note any findings from the interview.

Additional summary information may include, but is not limited to:

An assessment of the interview session tone, i.e., size, length, complexity,

extent of cooperation, effectiveness, etc.

Common themes or relationships to other interviews. Examples: WBS

elements, technical area, component, etc.

New issues or areas of interest identified as a direct result of the CAM

interview.

B. Independent Data Traces & Document Research

This section should identify any independent data traces or document research

done in support of the surveillance. It should discuss the research and data trace

objectives in the context of surveillance review scope. Summarize and/or attach

select exhibits that support findings or other items in the SSR as necessary.

Trace and research material must be dated and sources must be noted to ensure

the long term credibility of supporting material. These may be attached to or

associated directly with the EV templates for easy reference.

IV. SURVEILLANCE RESULTS


This section should document all surveillance findings along with their status as

of the report date. It should also provide a status of the program’s CARs. The

attached SSR template provides example findings language.

A. Findings

This section should quantify the findings discussed in the surveillance

summaries. It should contain succinct statements that communicate EVMS

implementation issues across guidelines and processes. Discuss any pending

actions or outstanding unresolved surveillance issues. Discuss systemic issue

trends.

B. Corrective Action Request (CAR) Status

List newly issued CAR(s), ongoing/existing CAR(s), and any CAR(s) that were

resolved during the reporting period. Briefly summarize the new CAR(s) and note

the affected guidelines or processes. Discuss the status of open CAR(s) and the

status of their respective corrective action plans (adequacy of CAP, estimated

completion, delays, contractor concerns, etc.) The attached SSR template

provides example CAR status language.

V. GUIDELINE COMPLIANCE SUMMARY

This section should identify how many guidelines were found to be compliant

during the surveillance. Reference and attach completed guideline compliance

summary worksheets for the guidelines included in this review. These forms can

be found at EV Templates. Also include the review of the supplier system

description as annotated in the completed EVMS Cross Reference Checklist.

http://guidebook.dcma.mil/79/EVMSCrossReferenceChecklist25July(2).doc

http://guidebook.dcma.mil/79/EVtemplates1-32.doc


TEMPLATE

DEFENSE CONTRACT MANAGEMENT AGENCY [SITE LOCATION]

SYSTEM SURVEILLANCE REPORT (SSR)

SURVEILLANCE IMPLEMENTED AT

[SUPPLIER NAME]

[SITE LOCATION]

[PROGRAM(s)/CONTRACT(s) REVIEWED]

[DATE OF THIS SSR]

FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.


I. EXECUTIVE SUMMARY

The Defense Contract Management Agency (DCMA) [CMO NAME AND

LOCATION] and members from [INCLUDE SUPPLIER IF SURVEILLANCE IS

JOINT] comprise the Earned Value Management System (EVMS) Joint

Surveillance Team (JST). Surveillance review efforts support the continuing

EVMS compliance of [SUPPLIER NAME] at [SUPPLIER LOCATION].

The Surveillance Team conducted an Earned Value Management System

(EVMS) review of the [PROGRAM NAME] program and contracts [LIST

CONTRACT NUMBERS] during the time period of [START AND FINISH DATES

OF REVIEW]. [XX] EVMS Guidelines were reviewed and [XX] deficiencies were

found in the implementation of Guidelines [LIST GUIDELINES] in relation to

process [NAME OF EVMS PROCESS].

[INSERT IMPACT STATEMENT]

{Example: The numerous noted deficiencies raise concerns regarding the

reliability and integrity of the Earned Value management processes and

practices. Additionally, the deficiencies jeopardize the veracity of the

performance measurement data. Without dependable, consistent, and

unquestionable EV application to processes, and the resulting EV data, the ability

to make prompt and informative management decisions is compromised.}

[XX] new CAR(s) was/were issued to address [LIST AREA OF CONCERN such

as “change management concerns”], [XX] level [XX] CAR(s) was/were closed,

and [XX] corrective action plan(s) regarding [INSERT SUBJECT OF CAP AND

STATUS such as “on budget authorization remains outstanding due to an

extended contractor approval process.]



II. SCOPE OF REVIEW

In accordance with the [INSERT APPROVAL DATE] EVMS standard surveillance

plan (SSP), the scope of the surveillance spanning [START AND FINISH DATES

OF REVIEW] included:

Process [NAME] and Guidelines [XX]

[PROGRAM, CONTRACT NUMBER]

[CUSTOMER NAME(S) AND WHETHER THEY PARTICIPATED OR NOT

IN THIS REVIEW]

[INSERT STATEMENT EXPLAINING HOW RISKS IDENTIFIED DURING THE

RISK ASSEMENT PROCESS RAISED CONCERN IN THE AREA OF THIS

REVIEW]

{Example: During the risk assessment process, these contracts were rated high

risk for work/budget authorization process since they are in the development

phase, have contract values at [$XX and $XX] respectively, and have high

percentages of remaining subcontract work [XX%], material [XX%], and

management reserve [XX%}. Contract [INSERT NUMBER] also was rated

moderate risk in this area due to an OTB in [DATE OF OTB]”.}

See Attachment A for Surveillance Selection Risk Matrixes for these contracts.

[INSERT PARAGRAPH EXPLAINING ANY DEVIATIONS TO THE APPROVED

ANNUAL EVMS SURVEILLANCE SCHEDULE].

Any concerns and/or questions regarding the content of this report, or the content and

tracking of subsequently issued CARs, should be addressed to the current surveillance

focal points listed below.

DCMA POC

[DCMA EVM analyst name]

[phone number]

[email contact]

[ SUPPLIER ] POC

[EVM focal point name]

[phone number]

[email contact]



III. SURVEILLANCE SUMMARIES

[XX] CAM interviews were conducted during this course of this review. [XX] areas

were identified for independent data traces, and follow-up research was

conducted on [XX] items.

A. Interview Summaries

[CAM NAME, RESPONSIBILITY]

[GUIDELINE OR PROCESS AREA FOCUS]

[SUMMARY OF INTERVIEW, DISCUSSIONS AND FINDINGS]

{Example:

John Doe, Systems Engineering software development WBS 3.2.5

Guideline 12

This surveillance interview discussion focused on level of effort work

package planning. The interview team noted that the CAM was not familiar

with the process for identifying and correcting charge errors.}

{Alternative}

[AREA OF RESPONSIBILITY]

[GUIDELINE OR PROCESS AREA FOCUS]

[LIST OF CAMs]

[SUMMAR OF INTERVIEW, DISCUSSIONS AND FINDINGS]

{Alternative Example:

Propulsion Integration, WBS: 0205

Planning, Scheduling & Budgeting

Mark Twain, Donna Karan, and Eddie Bauer

The Propulsion area discussions focused on cost and schedule integration

with the critical parts suppliers. The majority of the CAMs indicated that



multiple scheduling tools were used to status delivery dates and the

change request process was not always followed for baseline delivery

date modifications.}

B. Independent Data Traces & Document Research

[GUIDELINE]

[DATA TRACE/DOCUMENTS REVIEWED]

[SUMMARY OF FINDINGS]

{Example:

Guidelines 6, 7, and 32

Reviewed the IMS, CAPs and the MR log.

The trace objective was to identify any inconsistencies in the performance

measurement baseline and the integrated master schedule as MR was

allocated to re-planned activities. Traces indicated that the majority of MR

was allocated to planning packages where the IMS reflected significantly

lower levels of detail. Research indicated that planning packages were not

detail planned in a timely manor. The attached exhibit (Ex01_06/20XX)

documents the MR allocation to the PP and the 2 month lag in detail

planning identified by IMS progress.}

IV. SURVEILLANCE RESULTS

A. Findings

[INSERT EXPLANATION OF FINDINGS]

{Example: A continued lack of horizontal and vertical schedule integration (driven

by out of sequence task statusing), inadequate schedule logic, and the inability to

validate a critical path through program completion is the most significant issue

affecting the Scheduling process. In many cases, budgets were not properly

time-phased.



MR “harvesting” (moving remaining budget on completed/closed accounts to

MR), was evident in several control accounts. The surveillance team will continue

to research the derivative impacts of MR harvesting on the program’s revisions

and data maintenance processes.}

B. Corrective Action Requests (CAR) Status

[NEWLY ISSUED CAR(s), TITLES, REFERENCE NUMBERS]

[STATUS] {Example: The newly issued CAR affects the Work/Budget

Authorization process. The contractor has acknowledged acceptance of the CAR

and a corrective action plan is expected no later than [DATE].}

[ONGOING EXISTING CAR(s), TITLE]

[STATUS] {Example: Two of the 3 open CARs are on track for resolution by next

quarter. The corrective action plans have been implemented and metrics indicate

substantial improvement. The Budget Authorization CAP has been delayed 30

days as a result of conflicting organizational processes.}

[RESOLVED CAR(s), TITLES]

[STATUS] {Example: The system description has been revised to reflect the new

approved work authorization process. It was approved on [DATE] bringing the

work authorization process CAR issued in [DATE] to closure.}

V. GUIDELINE COMPLIANCE SUMMARY

Guidelines [XX] are found to be compliant. Guidelines [XX] are found to be non-

compliant. Research and surveillance findings indicate the program was unable

to adequately demonstrate [BRIEF LIST OF KNOWN COMPLIANCE

DEFICIENCIES].

{Example: integration of its contract line items, work authorization, planning,

scheduling, and budgeting processes with each other.}



See the completed EV guideline templates (attachment B) and EVMS Cross

Reference Checklist (attachment C) for additional detail.

__________________________________

Date:_______________________

[CMO specialist name]

DCMA EVM analyst

DCMA [site]

__________________________________

Date:_______________________

[Supplier’s representative name]

EVM Focal Point

[Supplier, location]

Attachments:

A. Surveillance Section Risk Matrices

B. EV Templates for Guidelines [XX]

C. EVMS Cross Reference Checklist


DEFENSE CONTRACT MANAGEMENT AGENCY [SITE LOCATION]

Documents

Transcript of DEFENSE CONTRACT MANAGEMENT AGENCY [SITE LOCATION]