TPB Internal Review ProcessLSS Project
Vision to Action
2
• Governor McCrory’s Vision of Creating an Efficient and Effective State Government that Focuses on Customer Service Led to;
• Secretary Tata’s Initiative to Implement the Lean Six Sigma Methodology Across DOT by;
• Placing the Governance Office in the Lead Role of Directing Lean Six Sigma Projects, and by
• Training DOT Staff to Lead Process Improvement Projects to Create an Efficient and Effective Agency with a Focus on Customer Service!
Lean Six Sigma – What Is It?
3
Lean Six Sigma Is A Business Improvement Methodology that Integrates Tools and Techniques from Lean Manufacturing and Six Sigma.
Lean
4
• The Lean Methodology Incorporates Process Analysis and Value Stream Analysis to Identify and Eliminate Waste in Processes. The Aim of Lean is:• Eliminating Non-Value Added Steps in the Process• Reducing Process Complexity• Improving Process Flow
• Examples: Defects, Waiting & Unnecessary motion
Six Sigma (6)
5
• Six Sigma is a Methodology and a Management System that Utilizes a Defined Process and Tools to Reduce Process Variation and Eliminate Defects.
• It Is a Top-Down Approach• Critical Improvement Efforts are Aligned with Key
Business Goals• Problem Solving is Data-Driven
Lean and 6 Together – A Powerful Combination for Improvement
6
• Lean and Six Sigma Are Complementary
• By Reducing Waste AND Reducing Variation We Can Achieve Better Quality and Greater Speed, Contributing to Lower Cost
DMAIC
7
5 Distinct Phases
• D – Define the Project, the Process and the Opportunity for Improvement
• M – Measure the Process to Understand Process Performance
• A – Analyze the Process to Identify Root Causes of Current Performance
• I – Improve the Process to Achieve Desired Gains
• C – Control the Process to Maintain the Gains
8
Project Participants
Champion / Sponsor – Susan Pullium, Director of Strategic
Planning
Project Leads – Earlene Thomas, TPBDoug Cox, Governance Office
Team Members – Brendan Merithew, TPB Suzette Morales, TPB Daniel Sellers, TPB Hemal Shah, TPB
9
Process Map
10
The Project Charter
Problem Statement/Mission:
• TPB's internal review process for approval of draft CTP maps, problem statements and reports is required to be completed within six weeks.
• During the previous twelve months the target was met only 21% of the time.
• Missing these target requirements results in delays in product delivery, missed PDA milestones, and poor customer service.
11
Voice of the Customer
Who is the customer?
TPB staff:
Project engineers (~30)
Planning group supervisors (6)
Unit heads (2)
Branch manager (1)
12
Voice of the Customer
What did we learn from them?
Project Engineers – Experiences:• Project Engineers do not always receive needed feedback• Comments/feedback from supervisors can be inconsistent• Expectations from supervisors are not always clear
Project engineers:• Offer lowest ratings in the consistency of feedback received
13
Voice of the CustomerWhat did we learn from them? (continued)
Supervisors – Experiences:• Reports are frequently not up to standards (spelling, formatting)• Information and data in report is not always accurate/justified• Submission needs to follow prescribed preferences/guidelines
Supervisors:• Offer lowest ratings regarding completeness and consistency of
products submitted for review• 83% of supervisors say the number of errors in a submitted
product was substantial
14
Measurement System AnalysisCycle Time MSAStart and end data collected from electronic records maintained by planning group supervisors, unit heads, and the branch manager of CTPs submitted for review.• End dates for the review/approval process maintained by the branch manager.• Start dates for the review/approval process maintained by the planning group
supervisors are not as reliable.• Some CTPs did not have a start date.• Some CTPs had a start date that could not be verified.• Only those CTPs with a reliable start date were included in the study.
Number of IterationsData collected from electronic version of CTPs. Reviews conducted and maintained at supervisory level indicates for each CTP:
• Number of iterations in review process• Total number of errors (formatting versus others)
15
Data AnalysisDescriptive Statistics of Time in Process
Count (number of products) 37Mean (number of days) 113Stdev 81.81Range 285Minimum 825th Percentile (Q1) 45.550th Percentile (Median) 7775th Percentile (Q3) 182Maximum 293
• Only 8 of 37 products met the target of six weeks for review/approval (42 days).• The average number of days required for review/approval is 113 days.
Target Cycle Time (42 days)
Average Cycle Time (113 days)
1 Sample t-Test
Test InformationH0: Mean (Mu) = 42 daysHa: Mean (Mu) Greater Than 42 days
Count 37Mean 113p-value (1-sided) 0.0000LC (1-sided, 95%) 90.291
• Reject the null hypothesis (p < .05)
16
Data AnalysisComparison of Planning Groups/Units in Meeting Target
• Data suggest that delays are not related to any particular group or unit• Data suggest that product type is not related to delays in the review/approval
process.• Reducing the number of errors in a product will require fewer iterations, thus
reducing the number of days in the review/approval process.
Review of Process Flow Map• A review of the process flow map shows that each time a supervisor/unit
head/branch manager does not approve the product it is returned to the project engineer for rework. These defects in the process are non-value added steps that contribute to the review/approval process taking longer than the target of 42 days.
17
Root CausesTraining/Experience Lack of experience/competency Competency level using GIS Training occurs at the beginning and not along the way Lack of technical training for the development of problem statements Supervisors who have never done a CTP lack project engineer point of view Relatively few reviews per year, leading to lack of consistency Attempting to develop a product from a sample or previous product Conflicting/unclear goals with problem statements – major vs. minor Strategic Highway Corridor (SHC)
Inconsistency of Supervisory Levels Varying supervisory style/direction Supervisors and unit heads contradict each other Inconsistent/incompatible directions at management levels Supervisor checklist does not match comments given Lack of implicit rules
Staffing Turnover Staff on leave Personality issues
Delays Scheduling issues – day-to-day delays and/or long term absences Knowing reports/maps are not immediately needed, resulting in not pressing the issue Excessive queues of reports without review Lag time between submission, review, revisions
Performance Performance issues Product not thoroughly reviewed prior to submittal Accuracy of the product/data/analysis
Rework Simple comment/point that causes considerable rework Having to reformat document when changes are made Constant changes Have to redo some parts that were once just part of the appendix Finding new problems in second and third review when they were previously not a
problem
Communication Communication of comments Standards/procedures/format have not been forwarded Poor management of changes in standards Not knowing proper format to submit product (hard or soft) Not sure if project engineer is grandfathered for process update When does the clock start – alterations in the process impact the timeline Finding reviewers in their office to consult or challenge comments
Workload Workload requires that revisions/reviews be reprioritized Managing multiple incoming reviews Competing priorities One person is not enough Travel Is six weeks for review long enough
Size of Reports Reports are too lengthy, takes too much time to review CTPs have a large number of problem statements, maps, inserts, appendices
Printer and Software Printing/assembling paper reports Printer not working properly Loss of work for unknown reasons Compatibility of software and printer
Missing Data/Information Missing information/request for additional information Lack of data Transferring local knowledge to supervisors/unit heads
All Others During review, study areas require change Process improvement initiatives – implementation Documenting exceptions or unique features with CTP Inconsistency among agencies and their knowledge base School delays and closings
18
Pilot the SolutionDescription of the Solution:• A peer review team that will review CTP products prior to being submitted
will be piloted.• The purpose of the pilot is to determine if the proposed solution will result in
shortening the review/approval cycle time by reducing the number of errors contained in the product and, thus, the number of iterations.
Method:• Three products will be reviewed independently by the peer review team and
the group supervisor.• The peer review team and the group supervisor concurrently review the
product and quantifies the number of errors detected.• The results of the three product reviews will be compared to determine if the
peer review detects an equal or greater number of errors than the group supervisors.
19
Effectiveness of the Pilot
Pilot Demonstrated:• Peer Review detected 50 – 110% more errors than during the supervisory level
review
Inferences:• Having products reviewed prior to submittal for supervisory review/approval
allows for errors to be detected and corrected in advance• Detecting common errors prior to supervisor review allows supervisors to focus
more on specific details instead of general structure• Product reviews can potentially be completed within the target time of 42 days
20
Overview of what to expect:High-level review of CTP Products
Looking for basic errors – NOT rewriting, researching or editing. Don’t need to be an expert on the CTP area.
Meet as a group to briefly discuss commentsGo over merged comments. Opportunity to discuss CTP standards and style.
Mandatory participation, peer comments do NOT affect performance review
Project Engineer makes edits accordinglyPE edits document on own time. It is up to the PE whether or not to implement comments.
Not being evaluated on quantity or type of comments. This is a “free” quality control check prior to submitting project to supervisor.
21
Implement the Solution – How make it Work
• Starting July 1, 2015 Peer Review of all CTP Products is
mandatory to every engineer in the Branch, it is aligned with
the start of the new NCVIP cycle
• Products for review: CTP Maps, PS and CTP Report
• Procedure for the Peer Review process published
• CTP tracking tool, schedules and milestones are updated
accordingly
• Training on process
22
Peer Review Team• Mandatory 4 person teams assignments –
tracked by Staff Engineer
• Branch wide balanced participation –
at least one representative from each
(not from the same group as PE) :
Eastern planning
Western planning
Non planning (forecaster or modeler or systems planner)
• Selection based on the Random Numbers
• Mixed levels teams: Contributing, Journey, Advanced
• Tracking posted on the S drive S:\Shared\TPB Temporary\Peer Review
23
Time Requirements and Scheduling
• Total estimated time at about 8 man hours
• Product must be submitted to Staff Engineer by noon on Wednesday
• Team assigned and products emailed to the team by EOB Friday
• Peer review team provides comments by EOB Friday of the assigned week.
• During the assigned review week PE schedules in person meeting with the
team to review comments.
• All conflicts should be brought to the attention
of the Staff Engineer immediately
24
Peer Review Procedurehttps://connect.ncdot.gov/projects/planning/TransPlanManuals/CTP_PeerReview.pdf
• PE submits products electronically (MS Word format for text and PDF for maps) to
TPB Staff Engineer prior to NOON Wednesday each week.
• TPB Staff Engineer selects/assigns products to the Peer Review team and uploads
all the products to the TPB Teamsite Docs library on Inside NCDOT
• TPB Staff Engineer notifies PE, Team Members, and all the supervisors about the
assignment same week by EOB Friday
• Team completes individual reviews on the Teamsite by EOB Friday.
• During the assigned review week PE schedules in person meeting with the team to
review comments.
• PE downloads final document from the Teamsite to his/her own computer
25
Peer Review Procedure (continued)https://connect.ncdot.gov/projects/planning/TransPlanManuals/CTP_PeerReview.pdf
• Comments should be made using the Comment feature for text in MS Word and
Sticky Notes feature in Acrobat for pdf maps
• During the team meeting with PE on the scheduled date, peer review
comments are reviewed/discussed.
26
Benefits
• Improve internal review times• Improve customer service• Reduce labor costs• Improve consistency of reports and reviews• Meet standards before supervisory submission• Learn from peers• Allow other groups to have a better understanding of
CTPs
27
Review Expectations/Criteria• Review maps for completeness/display• Maps and Problem Statements (PS) Match• PE follows PS Standards• PE follows Quick Tips• PE uses correct grammar • References in report match actual figures, tables, etc.• Learning opportunity for PE and Review Team
28
What NOT to Expect
• Performance Evaluation• Thorough review of technical analysis• Detailed Local Knowledge• Validation of Information• Catch Everything• Peer review to replace supervisory review• Approval of Maps and Reports by Peer
Review team• Full consensus on all comments
29
Example of Maps Peer Review
Paper Copies: • Step 1: Print maps and write comments on the maps• Step 2: Scan/Email maps with comments using copier to send to PE
ORAdobe Reader:• Step 1: Open the map in Adobe Reader• Step 2: Click on Comment tool to select “Add Sticky Note” • Step 4: Click on the map and write comment on the sticky note.• Step 5: Save and email map with comments to PE
30
Step: 1Step: 2
Step: 3
Step: 4
31
• Step 1: Open Internet Explorer (Recommended)• Step 2: Go to Transportation Planning Branch Team site on Inside NCDOThttps://inside.ncdot.gov/Teams/transportationplanningbranch/pages/default.aspx• Step 3: Open the file listed for Peer Review in Edit mode.• Step 4: Suggest edits and/or make comments.
Example of Report / PS Peer Review
Step: 3
32
• Multiple people can edit the word document at the same time on the TPB Team site, due to the SharePoint capabilities.
• All the changes/ edits / comments made will be automatically saved in real time.
• To see the changes made to the word document Open word document > Go to Review tab. Click on Show Markup Choose Reviewers to display a list with each Reviewer's name
Utilizing Share Point
33
34
Conclusions
• Opportunity to catch worst mistakes upfront
• Bring uniformity to branch CTP products
• Learn from each other
• Understand what supervisors go through
Questions?