Benchmarkingin the Projects Arena

By Larry DysertAACE International Cascade Section

AACE International – January 2004 – Portland, OR

Agenda

Definitions Benchmarking Process in General Benchmarking Process for Projects Project Benchmarking Organizations Practices vs. Metrics Findings & Trends

AACE International – January 2004 – Portland, OR

Benchmarking

AACE International – January 2004 – Portland, OR

Benchmark

A surveyor’s mark made on apermanent landmark of known positionand altitude - Webster’s New World Dictionary

A standard point of reference inmeasuring or judging quality, value, etc.- Webster’s New World Dictionary

AACE International – January 2004 – Portland, OR

Benchmarking

The practice of being humble enough toadmit that someone else is better atsomething, and being wise enough totry to learn how to match and evensurpass them at it - InternationalBenchmarking Clearinghouse

AACE International – January 2004 – Portland, OR

Benchmarking

The search for industry best practicesthat lead to superior performance -Robert Camp

The process of identifying and learningfrom best practices anywhere in theworld: A powerful tool in the quest forcontinuous improvement - IndependentProject Analysis

AACE International – January 2004 – Portland, OR

Benchmarking

The search for industry best practicesthat lead to superior performance -Robert Camp

The process of identifying and learningfrom best practices anywhere in theworld: A powerful tool in the quest forcontinuous improvement - IndependentProject Analysis

AACE International – January 2004 – Portland, OR

Benchmarking

Benchmarking can be applied to allfacets of a business.

Benchmarking implies measurement: Business function metrics Business practices

Benchmarking goes beyond traditionalcompetitive analysis

AACE International – January 2004 – Portland, OR

Benchmarking

Benchmarking challenges the currentway of doing business by bringing innew ideas and practices

Benchmarking is an objective-settingprocess

Effective benchmarking is a continuousprocess

AACE International – January 2004 – Portland, OR

Benchmarking

The origins of benchmarking are rootedin the basic competitive analysis andtotal quality management practiceswidespread in business

Benchmarking has passed the test oftime as a useful process and costimprovement technique

AACE International – January 2004 – Portland, OR

Approaches to Benchmarking

Competitive benchmarking

Functional benchmarking

Internal benchmarking

AACE International – January 2004 – Portland, OR

Competitive Benchmarking

Studies product designs, processcapabilities, or administrative methodsused by business competitors

Competitors may not employ best-in-class practices

Competitors can be reluctant partnersfor benchmarking

AACE International – January 2004 – Portland, OR

Functional Benchmarking

Benchmarking studies performed withnon-competitors

Attempts to find the secrets of anindustry leader’s success

Functional benchmarking relies oncooperation from best-in-class leaders

AACE International – January 2004 – Portland, OR

Internal Benchmarking

Attempts to find study partners withinsame organization

Fewer barriers to establish acooperative atmosphere

AACE International – January 2004 – Portland, OR

Benchmarking

Basic steps to benchmarking Know your operation Study the industry leaders and competitors Incorporate the best Gain superiority

AACE International – January 2004 – Portland, OR

Benchmarking Code of Conduct(International Benchmarking Clearinghouse)

Keep it legal Demonstrate willingness to share same level

of information you are requesting Respect confidentiality Don’t refer without permission - utilize

company benchmarking representativeswhen possible

Be prepared at all contacts withbenchmarking partners

AACE International – January 2004 – Portland, OR

Project Benchmarking Several studies in early 1980’s identified

similar trends in project execution: Reduction in project execution capabilities Lack of formal project execution training

programs Lack of business expertise in project teams Engineering practices and project

management practices not being applied

AACE International – January 2004 – Portland, OR

Project Benchmarking

Project benchmarking organizations: Construction Industry Institute Independent Project Analysis, Inc.

AACE International – January 2004 – Portland, OR

Construction Industry Institute 92 Companies 45 Owner Companies 47 Contractors

Variety of Industries Heavy Industrial Light IndustrialManufacturing Buildings Infrastructure

AACE International – January 2004 – Portland, OR

Construction Industry Institute

Objectives: Provide industry with performance metricsMeasure use of “best practices” on projects Quantify the value of utilizing “best

practices” Educate the industry in benchmarking

practices and opportunities forimprovement

AACE International – January 2004 – Portland, OR

Construction Industry Institute Best Practices Pre-Project Planning Team Building Constructability Safety Design/Information Technology Project Change Management Strategic Alliances

AACE International – January 2004 – Portland, OR

Construction Industry Institute

Performance Metrics Cost Schedule Safety Changes Rework

AACE International – January 2004 – Portland, OR

Independent Project Analysis

Industry Benchmarking Conference

Voluntary association of owner firms in theprocess industry

Divided into Upstream & Downstreamprocess groups

AACE International – January 2004 – Portland, OR

Independent Project Analysis

ObjectivesMeasure and compare project performance Identify and share practices that drive

excellence Conduct research into new project

practices

AACE International – January 2004 – Portland, OR

Independent Project Analysis

Best Practices Front-End Loading (FEL) Use of New Technology Use of Value Improving Practices Integrated Teams Safety

AACE International – January 2004 – Portland, OR

Independent Project Analysis

Performance Metrics Cost Schedule (Cycle Time) Operability Safety

AACE International – January 2004 – Portland, OR

Practices vs. Metrics

Benchmarking is the understanding ofpractices.

Metrics are used to quantify the effect ofpractices.

AACE International – January 2004 – Portland, OR

Practices vs. Metrics

Metrics have value beyond the processof benchmarking

Project cost and schedule metrics canimprove asset evaluation and conceptdevelopment

AACE International – January 2004 – Portland, OR

Practices vs. Metrics Metrics collection can be used for: Validation tool in estimating Strategic tool in estimating Setting project goals (“should” costs, target

schedules) Assess internal metrics versus industry

norms Support calibration of internal company

tools and databases Improve understanding of cost or schedule

drivers for value engineering analysis

AACE International – January 2004 – Portland, OR

Practices vs. Metrics

To use benchmarking effectively, it isimportant to stress practices in additionto metrics

Ensure that “best practices” areidentified, and utilized. Performancegains (as measured by metrics) shouldfollow

Sample Metrics

Sample Metrics

Sample Metrics

The Search for Best Practices

AACE International – January 2004 – Portland, OR

Findings & Trends

Cost risk is highly correlated with projectdefinition Good project definition results in smaller

cost deviations

Good project definition results insignificantly less variability

Good project definition results in more costeffective projects

Results of GoodProject Definition

0.90

0.95

1.00

1.05

1.10

1.15

1.20

3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9

FEL Index

Perf

orm

ance

Inde

x

SCREENINGPOORFAIRGOODBEST

Cost

Schedule

Best Practices for Project Control

Achieving Cost/ScheduleEffectiveness

Achieving Cost/SchedulePredictability

Project Control Best Practices Owner cost specialist validates cost

estimate Detail physical progressing Frequent and detailed progress status

reporting Actual cost data captured in an owner

database Benchmarking/Metric Analysis systems

in place

AACE International – January 2004 – Portland, OR

Findings & Trends

Overall, project costs are becomingmore predictable Absolute cost deviation is decreasing

Median % deviation is close to zero

Variability is still relatively high

AACE International – January 2004 – Portland, OR

Findings & Trends

Cost effectiveness is improving veryslightly Cost predictability does not yield cost

effectiveness

Progress in developmental projects islimited

Cost engineering is not playing a largeenough role in project selection

AACE International – January 2004 – Portland, OR

Findings & Trends

Schedules are becoming moreimportant Schedules are getting faster

Schedule slip is declining

Contributes to lack of improvement in costeffectiveness

Reasons forNon-Competitive Results

Non-competitive target setting Corporate culture requires underruns Lack of benchmarks/metrics Previous projects overran cost or schedule

Turnover in key personnel Lack of team continuity Lose contact with contractors Team stretched too thin

Reasons forNon-Competitive Results

Business Issues Changes in product characteristics Cash flow delays

Contractor Issues Inexperience at site/company Poor change management Unqualified contractors Lack of contractor cooperation

Reasons forNon-Competitive Results

Practices Are Not As Good As TheyAppear Funding with inadequate project definition VIP’s not effective Thorough schedule analysis not done Links to other projects not incorporated into

plan Emphasis on saying we’re the best instead

of being the best

AACE International – January 2004 – Portland, OR

Benchmarking Reality Check

“The government are very keen on amassingstatistics. They collect them, add them, raisethem to the nth power, take the cube root,and prepare wonderful diagrams. But youmust never forget that every one of thesefigures comes in the first instance from thevillage watchman who just puts down what hedamn pleases.” - Sir Josiah Stamp (1869-1919)

AACE International – January 2004 – Portland, OR

Summary Benchmarking is a positive, pro-active

approach to change project processes toachieve superior performance

Benchmarking, by its nature, challenges thecurrent methods of doing business

Benchmarking is an objective-setting process

Benchmarking should be a continual, long-term process (and requires managementcommitment)