COAA Major Projects Benchmarking Associates Training
University of CalgarySchulich School of Engineering
Dr. Elias IkpeJatinder Kumar
Agenda• COAA Benchmarking History and Staff
• COAA Performance Assessment System (PAS)
• Code of Conduct and Confidentiality Policy
• Assessing Project Performance
• Industry Partners’ Participation in Benchmarking
• Industrial Projects Data and Metrics
• Pipeline Projects Data and Metrics
• Well Site Projects and Well Pad Projects Data & Metrics
What Is Project Benchmarking And Why Should I Care?
“Benchmarking is the process of gathering information aboutother companies in your industry to compare your performanceagainst and to use to set goals.” (F. John Reh)
COAA Benchmarking History
Phase I (2006 – 2009)Task 1: Metric and report definition.
Task 2: Program data collection instrument and report.
Task 3: Conduct onsite training and commence data collection.
Task 4: Collect and validate data, perform analysis, develop an Alberta data set for benchmarking, and prepare project and company aggregate reports.
Task 5: Prepare Alberta Report documenting performance and quantifying the impacts of local factors on productivity and performance metrics.
Phase II (2010 – 2014)Task 1: Development of new metrics (Pipelines, Well Sites, SAGD).
Task 2: System development (Performance Assessment System)
Task 3: Alberta report #2.
COAA Benchmarking Committee
CII / U. of Calgary Benchmarking & Metrics Staff Questionnaire development & management
BA training
Database management, administration, & security
Implementation assistance
Analysis & reporting
Coordination
Staff and Functions
Dr. Elias IkpeAccount Manager (University of Calgary)
(587) [email protected]
Jatinder KumarGraduate Student (University of Calgary)
(403) [email protected]
Dr. Stephen MulvaAssociate Director (CII Benchmarking & Metrics)
(512) [email protected]
Getting Help…
COAA Benchmarking Performance Assessment System (PAS)
COAA Performance Assessment System
• COAA PAL Website: https://coaa.construction-institute.org/NextGen/
Useful Information
• For security and performance reasons, a logged in user that has not been active for a period of 1 hour is timed out.
• Data entered into a browser page is not recorded until the “Save” button is clicked.
Session Timeout
• Internet Explorer 5.0 or later• An Internet connection, 56K or faster• Java and JavaScript enabled• Cookies on• Adobe Acrobat Reader (PDF file)• Flash MX Player 10.2
Browser and Connection
Requirements
• Company data are considered confidential.• Data can be used to support CII/COAA benchmarking, research, and related academic activities only if the confidentiality of companies submitting the data is protected.• Access to data is limited to CII and U. of Calgary staff and authorized researchers only.• All persons with access to COAA data must sign confidentiality agreements and abide by COAA confidentiality policies.• When data are provided in support of research activities, all confidential identifiers will be removed and only essential subsets of data will be provided.• All data published and/or presented must reflect the aggregate of at least 10 projects from 3 separate companies.• Reports and data files containing only individual project or company data are considered confidential will not be published or provided to researchers.• In cases where a disproportionate amount of data are provided by a single company, CII/COAA will suppress publication of results until the data set is sufficiently large to mitigate confidentiality or bias concerns.
COAA Benchmarking Code of Conduct and Confidentiality Policy
Note: each quartile contains ¼ of the Number of Projects
• Cost• Schedule• Safety• Changes• Rework
Current Metrics
75th Percentile
50th Percentile
25th Percentile
Mean
Median
Category(n = 20 )
Displayof
MetricRespondent:
Cost Category :Project Type :
Location:Industry Group:Project Nature:
-0.15
-0.10
-0.05
0.00
0.05
0.10
0.15
Number of projects
BetterBetter
• EngineeringProductivity
• ConstructionProductivity
Assessing Project Performance
Budget Factor - Contractors
0.50
0.75
1.00
1.25
1.50
1.75
2.00
Cost Growth - Owners
-0.60
-0.40
-0.20
0.00
0.20
0.40
0.60
Median (-3%)
“Best in Class” (-19%)
Median (0.98)
“Best in Class” (0.83)
OWNER CONTRACTOR
COST GROWTH BUDGET FACTOR
Explaining Statistics – Cost Example
Explaining Statistics
Mean = arithmetic average
Median = mid-point of allprojects (is equivalent to the Second Quartile Q2)
Inter Quartile Range (IQR)refers to the range betweenthe First Quartile (Q1) andthe Third Quartile (Q3)
Outlier Projects are notIncluded in statistics
What Do Different Quartiles Mean?
Good
Poor4Q
3Q2Q
1Q
11% Improvement (4th to 3rd Quartile)
• 26% Improvement (4th to 1st Quartile)
Key Report
Benchmarking Information Output
Benchmarking Key Report
Data Miner
Benchmarking Information Output
Data Miner Features(Metrics – Parameters – Filters – Variables)
CompanyBenchmarking
Associate
Project Manager
Project Manager
AccountManager
(CII)
AssociateDirector (CII)
SystemsAnalyst (CII)
ResearchEngineer (CII)
Industry Partners’ Participation in Benchmarking
CompanyBenchmarking
ManagerCOAA
AccountManager(UofC)
Major Responsibilities
Project Manager
Benchmarking Associate
Benchmarking Manager
Board of Advisors BOA
BMMAN
BMASSOC
BMPM BMPM
BMASSOC
BMPM
• Company level benchmarking people hierarchy
Role of Benchmarking Manager• Benchmarking Manager is the key person of
an organization to manage their benchmarking efforts with COAA/CII
• Set up the benchmarking goal• Set up the benchmarking hierarchy for internal
benchmarking purpose (optional)• Setup the project profiles (optional)• Coordinate the people participating in
benchmarking• Manage the projects/database
Role of Benchmarking Associate• Company’s trained knowledge resource for
COAA/CII Benchmarking
• Facilitates selection of project representatives
• Expedites completion of questionnaire
• Verifies completeness and accuracy of data
• Assists company with the use of COAA/CII Benchmarking products
Role of BenchmarkingProject Manager
• Works with project teams
• Collects and reviews data for selected projects
• Submits projects to a Benchmarking Associate
COAA Benchmarking ProcessProject
ManagersBenchmarking
AssociateCompany
LeadershipCOAA/CII/UofC Staff, Account Mgr/Committee
Benchmarking Manager
Commit to Benchmarking
Coordination & Attend COAA/CII Training
Make Decision / Commitment to Benchmark as
Basis for Improvement
Commit to Project Benchmarking & Improvement
Develop/Improve Metrics, Processes, Policies &
Procedures
Develop Data Collection & Reporting Tools
Perform Validation Checks, Run Analysis & Generate
Reports
Provide Input to Research & Implementation
Conduct Training & Feedback Sessions
Validate Questionnaires
ReleaseQuestionnaire to
COAACII
Initiate Questionnaire During Project Execution
Complete & Submit Questionnaire at Project Close‐out
Review and Act on Interim COAA/CII On‐line Recommendations
Managing the benchmarking efforts with COAA/CII
Select Projects for Analysis & Preload/Initiate Project in CII Database
Perform Self‐Analysis & Develop Improvement Plan
COAA Benchmarking Framework
Level 1 Level 2 Level 3
Upstream(Oil Exploration/
Production)
Oil Sands SAGD
Cogeneration
Central Plant Processing Facilities
Pad and Gathering
Oil Sands Mining/ Extraction
Oil Sands Mining
Central Plant Processing Facilities
DownstreamOil Sands Upgrading
Naptha Hydrotreater Unit
Hydrogen Plant
Oil Refining Utilities and Offsite
Natural Gas Natural Gas Processing
Pipelines
Well Sites / Well Pads
• Cost
• Schedule
• Safety
• Change
• Rework
Procure Forms
5
5
Mobilize
10
3
Layout Exc 1
15
4
Forms Conc 1
30
3
Set BasePlate1
35
1
Layout Exc 2
20
4
Forms Conc 2
40
3
Set Base Plate 2
50
1
Layout Exc 3
25
4Forms Conc 3
45
3
Set Base Plate 3
55
1
SS0
SS2
FS2
FS2
SS2
FS2
Procure Forms
5
5
Procure Forms
5
5
Mobilize
10
3
Mobilize
10
3
Layout Exc 1
15
4
Layout Exc 1
15
4
Forms Conc 1
30
3
Forms Conc 1
30
3
Set BasePlate1
35
1
Set BasePlate1
35
1
Layout Exc 2
20
4
Layout Exc 2
20
4
Forms Conc 2
40
3
Forms Conc 2
40
3
Set Base Plate 2
50
1
Set Base Plate 2
50
1
Layout Exc 3
25
4
Layout Exc 3
25
4Forms Conc 3
45
3Forms Conc 3
45
3
Set Base Plate 3
55
1
Set Base Plate 3
55
1
SS0
SS2
FS2
FS2
SS2
FS2
• EngineeringProductivity
(Design Hours/IFC Quantities)
• ConstructionProductivity
(Work-Hours/Installed Quantities)
Industrial Projects Data and Metrics
COAA Consensus Benchmarking Definitions
Aligning Phase Definitions
COAA Phase Definitions
Detailed Engineering
Single Project
Concept Adopted &
Formal Project Team
Established
Project Authorization
Contract Award to
Engineering Firm
Release of all Drawings & Specs
Development of Procurement Plan for Major
Equipment
All Major Equipment Delivered to Site
Mechanical Completion
Mechanical Completion
Custody Transfer
Beginning of Continuous Substantial
Construction Activity
Procurement
Construction
Start-up
Front End Planning
Benchmarking Phase Definitions, Activities & Cost Elements
Project Phase Start/Stop Typical Activities & Products
Typical Cost Elements
Front End PlanningTypical Participants:• Owner Personnel• Planning Consultants• Constructability
Consultant• Alliance / Partner
Start: Single Project Concept Adopted, Formal Project Team Established
Stop: Project Authorization
• Options Analysis• Life-cycle Cost Analysis• Project Execution Plan• Appropriation Submittal
Pkg• P&IDs and Site Layout• Project Scoping• Procurement Plan• Arch. Rendering
• Owner Planning Team Personnel Expenses
• Consultant Fees & Expenses • Environmental Permitting
Costs• Project Manager / Construction
Manager Fees• Licensor Costs
Detail EngineeringTypical Participants:• Owner Personnel• Design Contractor• Constructability Expert• Alliance / Partner
Start: Contract Award to Engineering Firm
Stop: Release of All Approved Drawings & Specs for Construction (or last package for fast-track)
• Drawing & Spec Prep• Bill of Material Prep• Procurement Status• Sequence of Operations• Technical Review• Definitive Cost Estimate
• Owner Project Management Personnel
• Designer Fees• Project Manager / Construction
Manager Fees
Benchmarking Phase Definitions, Activities & Cost Elements
Project Phase Start/Stop Typical Activities & Products
Typical Cost Elements
ProcurementTypical Participants:• Owner Personnel• Design Contractor• Alliance / Partner
Start: Development ofProcurement Planfor MajorEquipment
Stop: All MajorEquipmentDelivered to site
• Supplier Qualification• Supplier Inquiries• Bid Analysis• Purchasing• Engineered Equipment• Transportation• Supplier QA/QC
• Owner PM Personnel• Project/Const. Manager Fees• Procurement & Expediting
Personnel • Engineered Equipment• Transportation• Shop QA/QC
ConstructionTypical Participants:• Owner Personnel• Design Contractor
(Inspection)• Construction Contractor
& Subcontractors
Start: Beginning of Continuous Substantial Construction Activity
Stop: Mechanical Completion
• Set Up Trailers• Site Preparation• Procurement of Bulks• Issue Subcontracts• Construction Plan for
Methods/Sequencing• Build Facility & Install
Engineered Equipment• Complete Punchlist• Demobilize Construction
Equipment
• Owner PM Personnel• Project/Const. Manager Fees• Building Permits• Inspection QA/QC• Construction Labour,
Equipment & Supplies• Bulk Materials• Construction Equipment• Contractor Management
Personnel• Warranties
Project Phase Start/Stop Typical Activities & Products
Typical Cost Elements
Start-up/CommissioningTypical Participants:• Owner Personnel• Design Contractor• Construction Contractor• Training Consultant• Equipment Suppliers
Start: MechanicalCompletion
Stop: Custody Transferto User/Operator
(steady stateoperation)
• Testing Systems• Training Operators• Documenting Results• Introduce Feedstocks
& Obtain First Product• Hand-off to User/
Operator• Operating System• Functional Facility• Warranty Work
• Owner PM Personnel• PM/Const. Manager Fees• Consultant Fees & Expenses• Operator Training Expenses• Wasted Feedstocks• Supplier Fees
Benchmarking Phase Definitions, Activities & Cost Elements
Industrial Projects Data and Metrics
COAA Questionnaire
Benchmarking Cost
Questionnaire – Reporting Cost Data
Procurement Phase CostsCost of Major Equipment including process and mechanical
Cost Definitions
Direct & Indirect Construction Cost
•See next slides for the tables
•The page numbers refer to the questionnaire printout
Direct & Indirect LabourTable(Construction Productivity Table)
Cost Performance Metrics– Project Level
Project Cost Growth Total Project Cost – Budget at Authorization*Budget at Authorization*
Delta Cost Growth | Project Cost Growth |
Project Budget Factor Total Project CostBudget at Authorization* + Approved Changes
Delta Budget Factor | 1- Budget Factor |
* Owner Definition; for Contractor Budget at Contract Award
Phase Cost Factor (Owner only)- Front End Planning- Detailed Engineering- Procurement- Construction- Start-up
Phase CostTotal Project Cost
Phase Cost Growth (Owner only)- Front End Planning- Detailed Engineering- Procurement- Construction- Start-up
Phase Cost – Phase Budget at AuthorizationPhase Budget at Authorization
Cost Performance Metrics– Phase Level
Benchmarking Schedule
Questionnaire –Reporting Schedule Data
Project Schedule Growth
Total Project Duration* – Predicted Duration at AuthorizationPredicted Duration at Authorization
Delta Schedule Growth | Project Schedule Growth |
Project Schedule Factor Total Project Duration*
Predicted Duration at Authorization + Approved Changes
Delta Schedule Factor | 1- Schedule Factor |
*Owner Definition Beginning of detailed engineering through start-up*Contractor Definition Duration for scope of work
Procure Forms
5
5
Mobilize
10
3
Layout Exc 1
15
4
Forms Conc 1
30
3
Set BasePlate1
35
1
Layout Exc 2
20
4
Forms Conc 2
40
3
Set Base Plate 2
50
1
Layout Exc 3
25
4Forms Conc 3
45
3
Set Base Plate 3
55
1
SS0
SS2
FS2
FS2
SS2
FS2
Procure Forms
5
5
Procure Forms
5
5
Mobilize
10
3
Mobilize
10
3
Layout Exc 1
15
4
Layout Exc 1
15
4
Forms Conc 1
30
3
Forms Conc 1
30
3
Set BasePlate1
35
1
Set BasePlate1
35
1
Layout Exc 2
20
4
Layout Exc 2
20
4
Forms Conc 2
40
3
Forms Conc 2
40
3
Set Base Plate 2
50
1
Set Base Plate 2
50
1
Layout Exc 3
25
4
Layout Exc 3
25
4Forms Conc 3
45
3
Forms Conc 3
45
3
Set Base Plate 3
55
1
Set Base Plate 3
55
1
SS0
SS2
FS2
FS2
SS2
FS2
Schedule PerformanceMetrics – Project Level
Phase Schedule Factor (Owner only)- Front End Planning- Detailed Engineering- Procurement- Construction- Start-up
Phase DurationOverall Project Duration*
Phase Schedule Growth - Front End Planning- Detailed Engineering- Procurement- Construction- Start-up
Phase Duration – Predicted Phase Duration at AuthorizationPredicted Phase Duration at Authorization
Procure Forms
5
5
Mobilize
10
3
Layout Exc1
15
4
Forms Conc1
30
3
Set BasePlate1
35
1
Layout Exc2
20
4
Forms Conc2
40
3
Set Base Plate 2
50
1
Layout Exc3
25
4Forms Conc3
45
3
Set Base Plate 3
55
1
SS0
SS2
FS2
FS2
SS2
FS2
Procure Forms
5
5
Procure Forms
5
5
Mobilize
10
3
Mobilize
10
3
Layout Exc1
15
4
Layout Exc1
15
4
Forms Conc1
30
3
Forms Conc1
30
3
Set BasePlate1
35
1
Set BasePlate1
35
1
Layout Exc2
20
4
Layout Exc2
20
4
Forms Conc2
40
3
Forms Conc2
40
3
Set Base Plate 2
50
1
Set Base Plate 2
50
1
Layout Exc3
25
4
Layout Exc3
25
4Forms Conc3
45
3
Forms Conc3
45
3
Set Base Plate 3
55
1
Set Base Plate 3
55
1
SS0
SS2
FS2
FS2
SS2
FS2
* Includes Front End Planning Front End Planning through Start-up
Schedule PerformanceMetrics – Phase Level
Benchmarking Safety
Questionnaire –Reporting Safety Data
Safety Metrics
Benchmarking Changes
Questionnaire – ReportingCost and Schedule Changes
Change Metrics
Benchmarking Rework
Questionnaire – ReportingCost and Schedule Rework
Rework Metrics
Benchmarking Project Impacts and Outcomes
Questionnaire –Reporting Impacts / Outcomes
Benchmarking Best Practices
COAA/CII Best Practices
1.Front End Planning2.Project Risk Assessment3.Team Building4.Alignment during Front End Planning5.Design for Maintainability6.Constructability7.Materials Management8.Change Management9.Zero Accident Techniques10.Quality Management11.Automation/Integration Tech12.Planning for Startup13.Prefabrication/ Preassembly/ Modularization14.Workface Planning
Example – Reporting Best Practice Use Information
The COAA Productivity Benchmarking Program provides:
• A statistically credible resource for member companies to support productivity improvement through external benchmarking.
• Industry research to increase the understanding of engineering and construction productivity drivers and constraints.
COAA Productivity Objectives
Benchmarking Engineering Productivity
01020304050607080
Proj A Proj B Proj C Proj D
WH/Dwg
Company Average = 58
Traditional Engineering Productivity Measurement
Piping Engineering Productivity
00.10.20.30.40.50.60.70.80.9
1
Proj A Proj B Proj C Proj D
WH/LF
Company Average = .74
Piping Engineering Productivity
COAA Engineering Productivity Measurement
• Quantity Based Metrics• Focus on Significant Activities• Appropriate Level of Detail• Only Direct Work-Hours• Common Definitions for Direct/Indirect Accounts• Actual Productivity
Raw Productivity =Actual Direct Engineering Wk-hrs
IFC Quantity
COAA Engineering Productivity Metrics
Direct
Indirect
• Document Control• Reproduction Graphics• Project Management• Project Controls
• (cost/schedule/estimating)• Project Engineer• Secretary/clerk• Procurement
• (supply management)• Construction Support
• (test package support, commissioning, etc.)
• Quality Assurance• Accounting• Legal
• Discipline Engineer• Designer• Technician
COAA Engineering
Hours
By Discipline– Concrete– Structural Steel– Equipment– Piping– Electrical– Instrumentation
COAA Engineering Productivity
COAA Engineering (Concrete) Productivity Metrics
Questionnaire – Reporting Engineering Productivity
Piping Example
Benchmarking Construction Productivity
• Focus on Significant Activities• Appropriate Level of Detail• Only Direct Work-Hours• Common Definitions for Direct/Indirect Accounts• Actual Productivity
Raw Productivity =Actual Direct Construction Wk-hrs
Quantity Installed
Construction Productivity Metrics
Consensus Work HourDefinitions
(See Next Slide)
• Direct Craft Labor• Foreman • General Foreman• Load and Haul• Oilers • Operating Engineer• Safety Meetings• Scaffolding• Truck Drivers Direct
Direct
Indirect
Direct & Indirect LabourTable
By Craft– Concrete– Structural Steel– Equipment– Piping– Electrical– Instrumentation– Insulation– Scaffolding– Module Installation
COAA Construction Productivity
Level I Level II Level III Level IV
Project Total Concrete
Total SlabsOn-GradeElevated Slabs/On DeckArea Paving
Total Foundations
< 5 CM5~20 CM21~50 CM>50 CM
Concrete Structures NA
Construction Productivity Levels(Concrete)
Questionnaire – Reporting Construction Productivity
Piping Example
PipingCalculation of Weighted-Average Diameter The following equation is provided for purposes of calculating the weighted-average diameter of piping for each of the material classifications.
weighted-average diameter
where:x = is the length of pipe (in metres)d = is the diameter of the pipe (in mm)i = is the summation index for pipe diameter for a given materialn = is the summation limit which is equal to the number ofdifferent pipe diameters for a given material type.
Productivity Metrics Calculations
Pipeline ProjectsData and Metrics
COAA Benchmarking Framework
Level 1 Level 2 Level 3
Upstream(Oil Exploration/
Production)
Oil Sands SAGD
Cogeneration
Central Plant Processing Facilities
Pad and Gathering
Oil Sands Mining/ Extraction
Oil Sands Mining
Central Plant Processing Facilities
DownstreamOil Sands Upgrading
Naptha Hydrotreater Unit
Hydrogen Plant
Oil Refining Utilities and Offsite
Natural Gas Natural Gas Processing
Pipelines
Well Sites / Well Pads
Questionnaire – Reporting Pipeline Data
Pipeline Metrics
Project Cost per Spread Total Project Cost*Number of Spreads*
Pipe Cost Factor Pipe Cost*Total Installed Cost (TIC)*
... additional metrics … additional calculations
Current Pipeline Research efforts
Revise existing metrics and develop new metrics
for Pipeline Projects
* Owner Definition; for Contractor Budget at Contract Award
Well Site Projects andWell Pad ProjectsData and Metrics
COAA Benchmarking Framework
Level 1 Level 2 Level 3
Upstream(Oil Exploration/
Production)
Oil Sands SAGD
Cogeneration
Central Plant Processing Facilities
Pad and Gathering
Oil Sands Mining/ Extraction
Oil Sands Mining
Central Plant Processing Facilities
DownstreamOil Sands Upgrading
Naptha Hydrotreater Unit
Hydrogen Plant
Oil Refining Utilities and Offsite
Natural Gas Natural Gas Processing
Pipelines
Well Sites / Well Pads
Questionnaire – Reporting Well Site Data
Well Site Metrics
Cost versus Flow Rate Total Installed Cost (TIC)*Design Flow Rate*
Schedule versus Flow Rate
Total Project Duration*Design Flow Rate*
… additional metrics … additional calculations
Current Well Site Research efforts
Revise existing metrics and develop new metricsfor Well Site Projects and
Well Pad Projects
* Owner Definition; for Contractor Budget at Contract Award
Current Database Research Efforts• Review addition of Facility Capacity data• Project data templates (for Project Teams)• Develop new metrics for Industrial Projects,
especially SAGD Projects• Develop new Reports• Develop new Data Mining capabilities• Review Indirect Costs• Review Commissioning/Start Up Costs• Review Modularization• Review Quality of data input and output• … to be continuedNote: we will continue to work with our Industry Partners in these Research Areas
Questions
Comments
Discussion
Dr. Elias IkpeAccount Manager (University of Calgary)
(587) [email protected]
Jatinder KumarGraduate Student (University of Calgary)
(403) [email protected]
Dr. Stephen MulvaAssociate Director (CII Benchmarking & Metrics)
(512) [email protected]
Thank youContact us
Top Related