CPEC Assessment TeamSector Jacksonville

Site Visit Just-in-Time TrainingJune 2008

Performance Excellence: Performance Excellence: Always ReadyAlways Ready and and Striving to do BetterStriving to do Better

JIT Site Visit Training Overview

• Outcomes

• Pilot Process

• Roles and Responsibilities

• Site Agenda

• Just-in-Time Training

• Site Visit Guidelines

Outcomes• Encourage management excellence for

successful mission execution by providing a framework to assess current performance, reveal opportunities for improvement, identify role model practices, and provide actionable feedback from trained, impartial examiners.

• Educate Coast Guard leaders and managers about the meaning, framework, and practical utility of the Criteria (CPEC);

Outcomes• Identify and Publicize organizational

achievements and successes, sharing role model practices for Coast Guard wide use and learning; and

• Recognize CPEC Award recipients as Coast Guard organizational role models that demonstrate excellent, sustained performance results while being good stewards of the public trust.

Unit Individual Small Cat Teams

Plenary Team

Team Leads

Team Coach

Program Manager

Submit Org Profile

Submit “What” answers

Review submitted info

Review all inputs

Fill out the Scorebook

Draft Key Factors

Create/validate interview questions and develop strategies

Determine approach for fact finding

Engage unit

Consensus on top strengths, OFIs, scores

Write Executive Summary

Review Scorebook and Exec Summary

(final report)

Present to Awards Board

Create PowerPoint outbrief

Present outbrief to unit

Develop Key Factors

Develop schedules

Approve schedules

Develop proposed

strengths/OFIs (5-8/item)

Finalize Scorebook

Review final report

Send final report to unit, Team Leader,

and OPCs

Brief CO/XO on

intermediate findings

•Provide just in

time training on process,documents, scoring, etc.

•Provide guidance and help with criteria and technical aspects of the process

•Ensure consistency of product among evaluation teams

•Assist team with logistics

•Provide initial contact with unit.

•Provide over-sight and support as needed.

•Provide continuity of program’s products

•Ensure final results are consistent with intent of the program

CPEC Pilot Process Map

• Team Leader and Assistant Team Leader

• Team Members

• Coach

• Program Representative

Roles and Responsibilities

Coordinate: • Examination Team activities • Virtual Meetings• Site visit logistics with the assistance of

Program Manager• Roles and responsibilities of Examination

Team

Team Leader and Assistant Team Leader

Ensure:• Examination process is followed• Deadlines are met • Deliverables are produced and

submitted to the unit and Program Manager (CG-0931)

• Unit understands the process• Examination Team conducts

themselves professionally

Team Leader and Assistant Team Leader

• Attend Examiner Training • Serve as a Category Lead and/or back-

up • Attend virtual meetings and site visit• Review unit documents prior to site visit• Prepare interview and examination

strategy• Complete Site Visit Strategy Worksheet

and Draft scorebook

Team Members

Provide:• Just-in-Time Training to Examination

Team• Advice and counsel for team leaders • Coaching on comment writing and

interviewing to Examination team members

• Review of final comments and scores• Assistance with logistics during site

visit

Coach

Ensure:• Overall examination process is followed• CPEC is properly applied to the unit

being examined• Serve in an unobtrusive (stealth) way

Coach

Provide:•Liaison with unit’s leadership team •Logistics coordination for site visit•Funding for site visit travel and meeting rooms•Sounding board to unit’s leadership•Evaluation of the process

Ensure the Site Visit is carried out in a way that is beneficial to the organization, examiners and the program

Program Representative

Generic Site Visit Agenda

Monday Senior Leadership Assessment

Team Time & Interviews

Tuesday Interviews

Deployment Check

Wednesday Interviews

Deployment Check

Consensus Meeting

Thursday Finalize Scorebook

Scoring

Friday Finalize Scorebook

Outbrief

Daily Meetings

Purpose:

• Review major findings for each category

• Discuss items to be addressed the next day

• Gauge progress and provide assistance as needed

• For CAP JAX will be held daily at 1900 at the Residence Inn.

Daily Deliverables

Sunday Finalized Game Plan

Monday Finalized Team Game Plan

Tuesday Updated Site Visit Worksheet/Scorebook

Wednesday Consensus on Top Strengths and OFIs

Thursday Finalized Scorebooks including scores

Friday Executive Summary

Power Point Out Brief

Completed Scorebook

Just-In-Time Training

• Site Visit Worksheet

• ADLI

• Results

• Code of Conduct

• *Scoring

• *Final Scorebook

*To be completed at team meeting Wednesday

ADLIApproach – Deployment – Learning -Integration

Four factors used to evaluate process items.

Process items are Categories 1 – 6

ADLIApproach The methods used to accomplish the process The appropriateness of the methods to the item requirements The effectiveness of the use of the methods The degree to which the approach is repeatable and based on reliable

data and information

ADLIDeployment The approach is applied in addressing Item requirements

relevant and important to the unit The approach is applied consistently The approach is used by all appropriate work units

ADLILearning Refining the approach through cycles of evaluation and improvement Encouraging breakthrough change to the approach through

innovation Sharing refinements and innovations with other relevant work units

and processes in the unit

ADLIIntegration The approach is aligned with the units needs identified in the

Organizational Profile and other process items The measures, information, and improvement systems are

complementary across processes and work units The plans, processes, results, analyses, learning, and actions are

harmonized across processes and work units to support unit-wide goals

Site Visit Worksheets (SVW)SVW are the initial version of the Scorebook. Two versions will be required for each Category.

One will include notes, interviewees, strategies, and findings.

The second will be the finalized version with only the top 5-8 strengths and OFIs for each item.

How to Complete SVWs

At the end of each day… Review notes from interviews, observations, and/or

documents Discuss MAJOR Findings & Conclusions w/the team during

the evening meeting Summarize & document your Findings on the SVW, identify

Strengths and OFIs If an SVW contains relevant Criteria areas to address that are

still open and a strength or opportunity cannot be concluded, ensure that you make the team leaders aware in order to schedule appropriate interviews the next day

How to Complete SVWs

Findings: Based on your strategy, what did you learn? Capture specific information (e.g., documents reviewed,

interviews conducted, specific responses to questions, actual data observed)

SUMMARIZE findings with enough detail that an outsider could benefit from reading Include specific Examples from interviews, observations,

documents Share with other category leads related findings

ENSURE THAT FINDINGS ARE CRITERIA- & KEY FACTOR-RELEVANT. THIS WILL HELP YOU BOTH TO DETERMINE AND DEMONSTRATE WHICH AREAS ARE STRENGTHS and WHICH ARE OPPORTUNITIES

Results

Understanding charts, data, and

comparisons

Understanding the Charts

Ask the person to explain the chart to you; show

the chart in question. Be sure you understand the

reply; if you’re unsure, ask for clarification.

Understanding the Data

Ask about where the results data come from and how they are calculated. Don’t make assumptions.

Understanding Comparisons

Ask about where the comparative data come from and the calculation. Don’t make assumptions.

LeTCI

Levels – Trends – Comparisons –

Integration

The four factors used to evaluate results (Category 7)

LeTCILevels

Current level of performance

Trends

Rate of performance improvements or the sustainability of good performance

Breadth of performance results

LeTCIComparisons

Performance relative to appropriate comparisons, such as competitors or

similar units

Performance relative to benchmarks or industry leaders

LeTCIIntegration Results measures address important customers, product and service, market, process,

and action performance requirements identified in the Org Profile and in Process Items Results include valid indicators of future performance Results are harmonized across processes and work units to support unit-wide goals

Final Scorebook

Wednesday Afternoon• Finalize all input into your SVWs.• As a small team, finalize 5-8 strengths and OFIs for

each item in your scorebook. • Be prepared to share these comments with the team

on Wednesday afternoon.

Wednesday/Thursday• Category discussion – Category Lead will lead

discussion reviewing each item• Recommend Item Scoring• Revise/Finalize Scorebook Item Worksheets and• Team Leader facilitates session to develop Key

Themes for Executive Summary and Executive Out Brief

• Finalize Executive Out Brief

Final Scorebook Process Steps

Purpose / Background How to fill out

Role Model Practice Worksheets

Purpose / Background Discussion

Site Visit Guidelines

Wrap-up / Review

Questions and Answers

“Chartmanship”

Do your best to understand the information. Do not comment of charting techniques or try to create your own trends or charts from the data provided.

Questions or Concerns

• Team Leader: George Yacus

• Assistant Team Lead: Peter Stinson

• Process Coach: Charlie Coiro

• Program Representative: David King