Post on 18-Dec-2014
description
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
SYSTEM SURVEILLANCE REPORT (SSR)
INSTRUCTION AND TEMPLATE
JANUARY 2008
THE DCMA EVM CENTER DIRECTOR IS RESPONSIBLE FOR ISSUANCE AND
MAINTENANCE OF THIS DOCUMENT
Rev 1 Jun 08 Added FOUO to Template
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
I. The purpose of this document is to provide instructions and a template for the
System Surveillance Report used in conjunction with DCMA Instruction – Earned
Value Management System (EVMS) System-Level Surveillance. As stated in the
Instruction, the surveillance team will document the results of system surveillance
reviews in a written report that is signed and dated by the DCMA surveillance
team leader and supplier representative (if employing joint surveillance). The
report will be issued in accordance with the annual EVMS surveillance schedule
and should include an overall assessment of the supplier’s implementation of the
EVMS, scope of the review, personnel interviewed, and findings of deficiency or
non-compliance that resulted in CARs.
Minimum expectations for documentation to be addressed in the surveillance
report include:
Surveillance Selection Risk Matrix(s);
Guidelines or Process(es) reviewed;
PM and CAM(s) interviewed and Control Accounts examined;
Completed Guideline templates for each reviewed Guideline
o EV Templates 1-32
Completed EVMS Description Cross Reference Checklist; and
System deficiencies identified:
o Corrective Action Request(s);
o CAR(s) entered into central database;
o Supplier Corrective Action Plan in place;
o Actions taken to correct the deficiency; and
o CMO analysis for trends and systemic issues.
Sections of the report include:
Executive Summary
Scope
Surveillance Summaries
o Interview Summaries
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
o Independent Data Traces & Document Research (sometimes
attached to the EV Templates)
Surveillance Results
o Findings
o Corrective Action Request (CAR) Status
Guideline Compliance Summaries
II. THE EXECUTIVE SUMMARY
The SSR executive summary should, at a minimum, refer to the established
Standard Surveillance Plan (SSP), the supplier, and the members of the
surveillance team. It should include the dates of the surveillance and indicate if
any deficiencies were found in the implementation of the surveyed EVMS
Guidelines. The executive summary should also include:
An impact statement noting significant concerns raised by the surveillance
and the potential effects of those concern areas.
A reference to follow on actions or any open items regarding the
surveillance.
The attached SSR template provides example executive summary language.
II. THE SCOPE OF THE SURVEILLANCE
The surveillance scope section should include the following:
The start and finish dates of the surveillance
All guidelines and processes reviewed
The Program(s) being reviewed and the respective contract number(s)
The Customer or purchasing organization
An explanation of how the risk assessment results support the scope of
this review
An explanation of any revisions or deviations from the approval annual
EVMS surveillance schedule.
The DCMA CMO contact information
The Supplier EVM focal point’s contact information
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
III. SURVEILLANCE SUMMARIES
The surveillance summary section aligns with the scope of the review. It
describes the CAM discussions, the areas identified for independent data traces,
and any follow up research that was conducted. The attached SSR template
provides example surveillance summary language.
A. Interview Summaries
Each interview summary should, at a minimum:
Identify the CAM and his/her area of responsibility.
Identify the Guideline or Process area focus of each interview.
Summarize interview discussion points.
Note any findings from the interview.
Additional summary information may include, but is not limited to:
An assessment of the interview session tone, i.e., size, length, complexity,
extent of cooperation, effectiveness, etc.
Common themes or relationships to other interviews. Examples: WBS
elements, technical area, component, etc.
New issues or areas of interest identified as a direct result of the CAM
interview.
B. Independent Data Traces & Document Research
This section should identify any independent data traces or document research
done in support of the surveillance. It should discuss the research and data trace
objectives in the context of surveillance review scope. Summarize and/or attach
select exhibits that support findings or other items in the SSR as necessary.
Trace and research material must be dated and sources must be noted to ensure
the long term credibility of supporting material. These may be attached to or
associated directly with the EV templates for easy reference.
IV. SURVEILLANCE RESULTS
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
This section should document all surveillance findings along with their status as
of the report date. It should also provide a status of the program’s CARs. The
attached SSR template provides example findings language.
A. Findings
This section should quantify the findings discussed in the surveillance
summaries. It should contain succinct statements that communicate EVMS
implementation issues across guidelines and processes. Discuss any pending
actions or outstanding unresolved surveillance issues. Discuss systemic issue
trends.
B. Corrective Action Request (CAR) Status
List newly issued CAR(s), ongoing/existing CAR(s), and any CAR(s) that were
resolved during the reporting period. Briefly summarize the new CAR(s) and note
the affected guidelines or processes. Discuss the status of open CAR(s) and the
status of their respective corrective action plans (adequacy of CAP, estimated
completion, delays, contractor concerns, etc.) The attached SSR template
provides example CAR status language.
V. GUIDELINE COMPLIANCE SUMMARY
This section should identify how many guidelines were found to be compliant
during the surveillance. Reference and attach completed guideline compliance
summary worksheets for the guidelines included in this review. These forms can
be found at EV Templates. Also include the review of the supplier system
description as annotated in the completed EVMS Cross Reference Checklist.
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
TEMPLATE
DEFENSE CONTRACT MANAGEMENT AGENCY [SITE LOCATION]
SYSTEM SURVEILLANCE REPORT (SSR)
SURVEILLANCE IMPLEMENTED AT
[SUPPLIER NAME]
[SITE LOCATION]
[PROGRAM(s)/CONTRACT(s) REVIEWED]
[DATE OF THIS SSR]
FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
I. EXECUTIVE SUMMARY
The Defense Contract Management Agency (DCMA) [CMO NAME AND
LOCATION] and members from [INCLUDE SUPPLIER IF SURVEILLANCE IS
JOINT] comprise the Earned Value Management System (EVMS) Joint
Surveillance Team (JST). Surveillance review efforts support the continuing
EVMS compliance of [SUPPLIER NAME] at [SUPPLIER LOCATION].
The Surveillance Team conducted an Earned Value Management System
(EVMS) review of the [PROGRAM NAME] program and contracts [LIST
CONTRACT NUMBERS] during the time period of [START AND FINISH DATES
OF REVIEW]. [XX] EVMS Guidelines were reviewed and [XX] deficiencies were
found in the implementation of Guidelines [LIST GUIDELINES] in relation to
process [NAME OF EVMS PROCESS].
[INSERT IMPACT STATEMENT]
{Example: The numerous noted deficiencies raise concerns regarding the
reliability and integrity of the Earned Value management processes and
practices. Additionally, the deficiencies jeopardize the veracity of the
performance measurement data. Without dependable, consistent, and
unquestionable EV application to processes, and the resulting EV data, the ability
to make prompt and informative management decisions is compromised.}
[XX] new CAR(s) was/were issued to address [LIST AREA OF CONCERN such
as “change management concerns”], [XX] level [XX] CAR(s) was/were closed,
and [XX] corrective action plan(s) regarding [INSERT SUBJECT OF CAP AND
STATUS such as “on budget authorization remains outstanding due to an
extended contractor approval process.]
FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
II. SCOPE OF REVIEW
In accordance with the [INSERT APPROVAL DATE] EVMS standard surveillance
plan (SSP), the scope of the surveillance spanning [START AND FINISH DATES
OF REVIEW] included:
Process [NAME] and Guidelines [XX]
[PROGRAM, CONTRACT NUMBER]
[CUSTOMER NAME(S) AND WHETHER THEY PARTICIPATED OR NOT
IN THIS REVIEW]
[INSERT STATEMENT EXPLAINING HOW RISKS IDENTIFIED DURING THE
RISK ASSEMENT PROCESS RAISED CONCERN IN THE AREA OF THIS
REVIEW]
{Example: During the risk assessment process, these contracts were rated high
risk for work/budget authorization process since they are in the development
phase, have contract values at [$XX and $XX] respectively, and have high
percentages of remaining subcontract work [XX%], material [XX%], and
management reserve [XX%}. Contract [INSERT NUMBER] also was rated
moderate risk in this area due to an OTB in [DATE OF OTB]”.}
See Attachment A for Surveillance Selection Risk Matrixes for these contracts.
[INSERT PARAGRAPH EXPLAINING ANY DEVIATIONS TO THE APPROVED
ANNUAL EVMS SURVEILLANCE SCHEDULE].
Any concerns and/or questions regarding the content of this report, or the content and
tracking of subsequently issued CARs, should be addressed to the current surveillance
focal points listed below.
DCMA POC
[DCMA EVM analyst name]
[phone number]
[email contact]
[ SUPPLIER ] POC
[EVM focal point name]
[phone number]
[email contact]
FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
III. SURVEILLANCE SUMMARIES
[XX] CAM interviews were conducted during this course of this review. [XX] areas
were identified for independent data traces, and follow-up research was
conducted on [XX] items.
A. Interview Summaries
[CAM NAME, RESPONSIBILITY]
[GUIDELINE OR PROCESS AREA FOCUS]
[SUMMARY OF INTERVIEW, DISCUSSIONS AND FINDINGS]
{Example:
John Doe, Systems Engineering software development WBS 3.2.5
Guideline 12
This surveillance interview discussion focused on level of effort work
package planning. The interview team noted that the CAM was not familiar
with the process for identifying and correcting charge errors.}
{Alternative}
[AREA OF RESPONSIBILITY]
[GUIDELINE OR PROCESS AREA FOCUS]
[LIST OF CAMs]
[SUMMAR OF INTERVIEW, DISCUSSIONS AND FINDINGS]
{Alternative Example:
Propulsion Integration, WBS: 0205
Planning, Scheduling & Budgeting
Mark Twain, Donna Karan, and Eddie Bauer
The Propulsion area discussions focused on cost and schedule integration
with the critical parts suppliers. The majority of the CAMs indicated that
FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
multiple scheduling tools were used to status delivery dates and the
change request process was not always followed for baseline delivery
date modifications.}
B. Independent Data Traces & Document Research
[GUIDELINE]
[DATA TRACE/DOCUMENTS REVIEWED]
[SUMMARY OF FINDINGS]
{Example:
Guidelines 6, 7, and 32
Reviewed the IMS, CAPs and the MR log.
The trace objective was to identify any inconsistencies in the performance
measurement baseline and the integrated master schedule as MR was
allocated to re-planned activities. Traces indicated that the majority of MR
was allocated to planning packages where the IMS reflected significantly
lower levels of detail. Research indicated that planning packages were not
detail planned in a timely manor. The attached exhibit (Ex01_06/20XX)
documents the MR allocation to the PP and the 2 month lag in detail
planning identified by IMS progress.}
IV. SURVEILLANCE RESULTS
A. Findings
[INSERT EXPLANATION OF FINDINGS]
{Example: A continued lack of horizontal and vertical schedule integration (driven
by out of sequence task statusing), inadequate schedule logic, and the inability to
validate a critical path through program completion is the most significant issue
affecting the Scheduling process. In many cases, budgets were not properly
time-phased.
FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
MR “harvesting” (moving remaining budget on completed/closed accounts to
MR), was evident in several control accounts. The surveillance team will continue
to research the derivative impacts of MR harvesting on the program’s revisions
and data maintenance processes.}
B. Corrective Action Requests (CAR) Status
[NEWLY ISSUED CAR(s), TITLES, REFERENCE NUMBERS]
[STATUS] {Example: The newly issued CAR affects the Work/Budget
Authorization process. The contractor has acknowledged acceptance of the CAR
and a corrective action plan is expected no later than [DATE].}
[ONGOING EXISTING CAR(s), TITLE]
[STATUS] {Example: Two of the 3 open CARs are on track for resolution by next
quarter. The corrective action plans have been implemented and metrics indicate
substantial improvement. The Budget Authorization CAP has been delayed 30
days as a result of conflicting organizational processes.}
[RESOLVED CAR(s), TITLES]
[STATUS] {Example: The system description has been revised to reflect the new
approved work authorization process. It was approved on [DATE] bringing the
work authorization process CAR issued in [DATE] to closure.}
V. GUIDELINE COMPLIANCE SUMMARY
Guidelines [XX] are found to be compliant. Guidelines [XX] are found to be non-
compliant. Research and surveillance findings indicate the program was unable
to adequately demonstrate [BRIEF LIST OF KNOWN COMPLIANCE
DEFICIENCIES].
{Example: integration of its contract line items, work authorization, planning,
scheduling, and budgeting processes with each other.}
FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.
SYSTEM SURVEILLANCE REPORT INSTRUCTION/TEMPLATE JAN 2008
See the completed EV guideline templates (attachment B) and EVMS Cross
Reference Checklist (attachment C) for additional detail.
__________________________________
Date:_______________________
[CMO specialist name]
DCMA EVM analyst
DCMA [site]
__________________________________
Date:_______________________
[Supplier’s representative name]
EVM Focal Point
[Supplier, location]
Attachments:
A. Surveillance Section Risk Matrices
B. EV Templates for Guidelines [XX]
C. EVMS Cross Reference Checklist
FOR OFFICIAL USE ONLY. [INSERT NAME OF SUPPLIER] PROPRIETARY. THIS DOCUMENT CONTAINS SUPPLIER INFORMATION WHICH MAY BE PROPRIETARY AND PROTECTED BY THE TRADE SECRETS ACT OR OTHER INFORMATION PROTECTION LEGISLATION. DO NOT RELEASE UNDER FOIA WITHOUT REVIEW AND REDACTION. EXEMPTION B4 MAY APPLY.