The Accuracy of Pre-tender Building Cost Estimates in Australia.
-
Upload
barbara-kruzic -
Category
Documents
-
view
197 -
download
1
Transcript of The Accuracy of Pre-tender Building Cost Estimates in Australia.
PLEASE SCROLL DOWN FOR ARTICLE
This article was downloaded by: [HEAL-Link Consortium]On: 4 January 2011Access details: Access Details: [subscription number 786636552]Publisher RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Construction Management and EconomicsPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t713664979
The accuracy of pre-tender building cost estimates in AustraliaAjibade Ayodeji Aibinua; Thomas Pascoa
a Faculty of Architecture, Building and Planning, University of Melbourne, Parkville, Melbourne, 3010Australia
To cite this Article Aibinu, Ajibade Ayodeji and Pasco, Thomas(2008) 'The accuracy of pre-tender building cost estimatesin Australia', Construction Management and Economics, 26: 12, 1257 — 1269To link to this Article: DOI: 10.1080/01446190802527514URL: http://dx.doi.org/10.1080/01446190802527514
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.
The accuracy of pre-tender building cost estimates inAustralia
AJIBADE AYODEJI AIBINU* and THOMAS PASCO
Faculty of Architecture, Building and Planning, University of Melbourne, Parkville, Melbourne, 3010 Australia
Received 22 January 2008; accepted 4 October 2008
A pre-tender building cost estimate is an important piece of information when making decisions at the project
planning and design stage. The important project characteristics influencing the accuracy of pre-tender
building cost estimates are examined and practical improvement for increasing the accuracy of estimates are
considered. A quantitative approach is used to address the research problem. Analysis of data from 56 projects
and from a postal questionnaire survey of 102 quantity surveying firms suggests that the accuracy of pre-tender
building cost estimates varies according to project size and principal structural material. When eight identified
project characteristics are controlled in a multiple regression analysis, the accuracy of estimates is influenced by
project size. The estimates of smaller projects are more biased than the estimates of larger projects. It was
discovered that pre-tender building costs are more often overestimated than are underestimated. Overestimated
forecasts are incorrect by a larger amount than underestimated forecasts. Data analysis also revealed that the
accuracy of pre-tender building cost estimates has not improved over time. The majority of the respondents are
somewhat dissatisfied with the accuracy of estimates in the industry. Probability estimation and simulation of
past estimates, reducing quantity surveying and cost engineering skill turnover, incorporating market
sentiments into estimates, early involvement of the quantity surveyor at the brief stage, and proper
documentation of experience gained in the estimation of projects should help firms increase the accuracy of
estimates for new projects.
Keywords: Australia, estimating accuracy, pre-tender estimates, quantity surveying, tendering.
Introduction
Pre-tender building cost estimates are susceptible to
inaccuracies (bias) because they are often prepared
within a limited timeframe, and without finalized
project scope. Pursuing an underestimated project
can lead to project failure. On the other hand,
overestimation of a project at the pre-tender stage can
lead to a viable project being dropped or re-tendered
when there is no bid close enough to permit project
award. Bias in the estimate of a project may arise from
two sources, namely, bias associated with the project
itself (will be the same regardless of the estimator) and
bias associated with the estimating technique used and
the environment (which would change depending on
the estimator). The only known published works
relating to accuracy of cost forecasts in an Australian
context are Mills (1997), which compared prediction of
building price movement by quantity surveyors with the
Australian Bureau of Statistics’ actual building price
movement, and Bromilow et al. (1988), which analysed
the variance between contract sum and final contract
sum. The objectives of this study are:
(1) to explore the frequency and size of inaccuracy
in pre-tender building cost estimates (i.e. the
variance between pre-tender cost estimate and
contract sum—accepted tender) using
Australian data;
(2) to explore project characteristics influencing the
accuracy of pre-tender building cost estimates;
(3) to assess whether the accuracy of pre-tender
building cost estimates has improved over time;
(4) to investigate what firms are doing to improve
the accuracy of cost estimates in practice, and
in that regard, evaluate the effectiveness of the
improvement methods.*Author for correspondence. E-mail: [email protected]
Construction Management and Economics (December 2008) 26, 1257–1269
Construction Management and EconomicsISSN 0144-6193 print/ISSN 1466-433X online # 2008 Taylor & Francis
http://www.tandf.co.uk/journalsDOI: 10.1080/01446190802527514
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
It is important to study the accuracy of building cost
estimates because a large part of a quantity surveyor’s
and cost engineer’s role in the construction industry is
to provide certainty of cost to clients. Thus the
knowledge developed in this study should help quantity
surveyors, estimators and cost engineers so that they
are aware of how their cost forecasts have performed
over time and what project characteristics need special
attention during cost estimation; and what practices
could increase the accuracy of their estimates.
Theoretical framework and scope of study
What is accuracy of an estimate?
Pre-tender cost estimation (or early stage cost
estimation) is the forecasting of the cost of a project
during the planning and design stage (Serpell, 2005).
Skitmore (1991) describes the accuracy of early stage
estimation as comprising two aspects, namely, bias
and consistency of the estimate when compared with
the contract or accepted tender price. Bias is
concerned with ‘the average of differences between
actual tender price and forecast’ while consistency of
estimates is concerned with ‘the degree of variation
around the average’.
Factors affecting estimating accuracy
An overview of previous studies suggests that a large
number of factors may influence the accuracy of an
estimate. Gunner and Skitmore (1999a) reviewed
previous studies and summarized the factors as
follows: building function, type of contract, condi-
tions of contract, contract sum, price intensity,
contract period, number of bidders, good/bad years,
procurement basis, project sector (public, private or
joint), number of priced items and number of
drawings. Gunner and Skitmore (1999a) analysed
the estimates of 181 projects in Singapore. They
found that a majority of the factors influenced the
accuracy of estimates. Using data from 42 projects in
Singapore Ling and Boo (2001) found similar results
when they compared five variables against Gunner
and Skitmore’s (1999a) work. Skitmore and Picken
(2000) studied the effect that four independent
factors (building type, project size, project sector
and year) had on estimating accuracy. They tested
the four factors using data from 217 projects in the
United States of America. They found that bias in the
estimate of the projects is influenced by project size
and year, while consistency in the estimates is
influenced by project type, size and year. In a study
of 67 process industry construction projects around
the world, Trost and Oberlender (2003) identified 45
factors contributing to the accuracy of early stage
estimates. They summarized the factors into 11
orthogonal elements. Of the 11 factors, the five most
important include: process design, team experience
and cost information, time allowed to prepare
estimates, site requirements, and bidding and labour
climate. All these studies suggest that there are a large
number of variables that may substantially influence
the accuracy of an early stage estimate.
According to Gunner (1997) the factors influencing
accuracy of estimates are intercorrelated so that the
true bias of one factor could be masked by one or
more factors. For example, Gunner and Skitmore
(1999b) theorize that ‘Price Intensity alone is both
necessary and sufficient to account for systematic bias
(inaccuracy) in building price forecasting’. Price
intensity is the total cost of a building divided by
the gross floor area. Price intensity theory states that
buildings with low unit rates (cost/m2 gross floor
area) would tend to be overestimated, while those
with high unit rates would tend to be underestimated.
In a study of 89 construction projects in Hong Kong,
Skitmore and Drew (2003) support the price intensity
theory.
In another study, Skitmore and Picken (2000)
using data from 217 projects in the United States
found that ‘year’ was the underlying variable respon-
sible for the bias and inconsistency in cost estimates,
after partialling out confounding effects of the four
factors put forward. The finding contrasts Gunner
and Skitmore’s (1999b) ‘price intensity’ theory.
However, their result supports Gunner’s (1997)
theory which states that intercorrelations among
variables cause confounding effects. It also supports
Gunner and Skitmore (1999a) in their suggestion that
a single underlying variable is the cause of bias and
consistency seen in estimates.
Study hypotheses
Based on a review of past studies, the following
hypotheses are set out:
Hypothesis 1: Systematic bias and inconsistency in pre-
tender building cost estimates are influenced by project
size (measured by project value, number of storeys and
gross floor area), location, project type, procurement
route, project sector, price intensity and principal
structural material.
Hypothesis 2: The accuracy of pre-tender building cost
estimates has not improved over time.
Hypothesis 3: Quantity surveying firms agree on the most
effective ways they believe will improve the accuracy of
pre-tender building cost estimates.
1258 Aibinu and Pasco
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
Research method
Data collection
Two methods of data collection were used to achieve
the study objectives. In order to address objectives 1, 2
and 3, data were collected from the files of 56
construction projects completed between 1999 and
2007. The data were obtained from the office of a
quantity surveying firm in Australia. Information
obtained in respect of each project included: project
value, the number of floors, gross floor area (GFA),
project location (central business district, metropolitan,
or rural), project type (residential, industrial or
commercial), procurement route used, project sector
(public, private or joint), price intensity (measured by
ratio of project value and gross floor area), and
principal structural material used (steel, concrete or
timber).
The projects were selected by a simple random
sampling process. A list of projects completed from
1999 to 2007 was drawn. Projects that were not
suitable for analysis, owing to incorrect job type or
lack of early stage estimate, were discarded. Thereafter,
each project was assigned a serial number sequentially
from one. Random numbers were then generated using
the Microsoft Excel program. The process yielded 85
random numbers. Numbers that were repeated were
deleted the second time they appeared. The process
produced 56 random numbers. The projects with serial
numbers corresponding with the 56 random numbers
were selected for data collection and analysis. As the
researchers were allowed first-hand access to all data,
no data were selected on the recommendation of the
quantity surveyor responsible for the estimates. Thus
there was no bias in the data collection.
In order to achieve objective 4, a structured ques-
tionnaire was designed for data collection. It comprised
questions regarding the profile of the respondents, the
profile of their company, the respondents’ satisfaction
with the current level of estimate accuracy in the
industry, and the views of the respondents regarding
acceptable level of estimate accuracy. The respondents
were also asked to rate 12 methods that could be used
for improving the accuracy of estimates. Depending on
the nature of the question, respondents were asked to
indicate their answers on a five-point Likert scale or a
categorical scale.
The questionnaires were mailed to 102 quantity
surveying firms in July/August 2007. The firms were
randomly selected from the list of about 166 firms
maintained by the Australian Institute of Quantity
Surveyors (AIQS) (AIQS, 2003).
Data analysis, results and discussion
Response rate and characteristic of sample from
questionnaire
The questionnaire survey yielded a response rate of
41%. Figure 1 shows the geographical distribution of
Figure 1 Geographical distribution of the survey and responses
Key
ACT – Act Capital Territory; NSW – New South Wales; NT – Northern Territory;
QLD – Queensland; VIC – Victoria; SA – South Australia; TAS – Tasmania; WA – Western Australia
Building cost estimates 1259
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
the responses. 29% of the responding firms have fewer
than 6 technical staff, 42% have between 6 and 15
technical staff, 22% have between 16 and 25 staff, and
7% have 26 or more technical staff.
90% of the questionnaires were completed by either
a director or an associate of the firms while 10%
were completed by a senior quantity surveyor or a
quantity surveyor. 81% of them have had over 15
years’ experience, 13% have between 10 and 15
years’ experience, and 6% have fewer than 10 years’
experience. Altogether, the data have come from the
highest echelon of quantity surveying professionals in
Australia, and their responses can be confidently relied
upon.
Respondents’ tolerance and satisfaction with
estimate accuracy in the industry
None of the respondents indicated that they were very
satisfied with the current level of estimate accuracy.
66% indicated they were very dissatisfied, dissatisfied
or neither satisfied nor dissatisfied, while 34% indicated
that they were satisfied. When early stage estimate is
compared with the lowest tender (contract sum), 24%
of respondents believed that a tolerance limit of within
¡5% is acceptable, 54% nominated a tolerance of
¡10%, 20% nominated a tolerance of ¡20%, and 2%
nominated a tolerance of ¡30%. Also, 70% of
respondents indicated that the accuracy of early stage
estimates hadn’t improved at all, very slightly or slightly
over the past 15 years while 25% believed that it has
improved almost adequately or adequately. 5% did not
respond.
Preliminary analysis of data from past
projects
Treatment of data
Data obtained from the 56 projects were analysed by
project size (measured by project value, gross floor area
and number of storeys), location, procurement route,
project type, principal structural material and price
intensity. Time of estimate was controlled by transform-
ing the pre-tender estimate and contract sum of each
project to the December 2006 price using the building
price index published by Rawlinsons (2006, 2007). Also,
differences in estimating processes and approach were
understood to have been controlled because projects
analysed were undertaken in the same company under
the same quality assurance procedure. Project sector
(whether private or public or joint) was also excluded
from the analysis because the 56 projects did not provide
a large enough spread of data to enable statistical analysis
of the impact of project sector. There were too few
samples of projects procured by the private sector. There
were also too few samples of projects procured jointly by
the public and private sectors.
Frequency and size of inaccuracies in pre-tender
building cost forecasts
Percentage cost overestimate or underestimate (esti-
mate error or bias) were estimated for each project by
using the following expression:
Estimate bias~
pre tender cost estimate{accepted tender sum
accepted tender sum|100
The mean estimate bias was also computed for the 56
projects using the following expression:
Mean estimate bias xð Þ~P
x
n
where x5estimate bias; n5number of projects.
A positive value of estimate bias implies an over-
estimation of cost while a negative value implies an
underestimation of cost. The analysis shows that in
about 7 out of every 10 projects, cost forecasts were
overestimated while in about 3 out of every 10 projects
cost forecasts were underestimated. This implies that
pre-tender cost forecasts were more often overesti-
mated than were underestimated. Bias in overestimated
costs ranges from +0.97% to +31.88% with a mean of
+10% while bias in underestimated cost ranges from
22.21% to 219.83% with a mean of 29%.
A one-sample t-test (Levine et al., 2005) was used to
test the following hypotheses: (1) The mean bias in
overestimated costs (+10%) is not different from zero;
(2) The mean bias in underestimated costs (29%) is
not different from zero. The results show that the mean
bias in overestimated costs (+10%) is significantly
different from 0% (p50.000, standard deviation57%)
which implies that the overestimated forecasts are truly
biased and not by chance. Similarly, the mean bias in
underestimated costs (29%) is significantly different
from zero (p50.000, standard deviation55%) which
also implies that underestimated forecasts are truly
biased and not by chance.
Analysis shows that the cost estimates for the 56
projects are generally biased and were overestimated
(mean524.29%) (see Table 1). Again, one-sample t-test
analysis shows that the mean (+4.29%) is significantly
different from zero (p50.004, standard devia-
tion510.61%) meaning that the estimates are biased
overall. However, when one considers the fact that early
stage estimates are prepared with little information, the
overall mean estimate bias of 24.29% may be acceptable.
1260 Aibinu and Pasco
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
In order to examine whether the error of under-
estimation is the same as the error of overestimation,
the Mann–Whitney test (non-parametric) (Levine et al,
2005) was used instead of a two-sample t-test because
normality of data could not be demonstrated. The test
shows that the mean bias in underestimated costs
(29%) is not the same as the mean bias in over-
estimated costs (+10%) (they are statistically and
significantly different: p50.000). This means that
pre-tender cost forecasts that were overestimated are
incorrect by a larger margin than pre-tender cost
forecasts that were underestimated.
Standard deviation (S) was computed for the 56
projects using the expression:
S~
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP
x{xð Þ2.
n
r
where x5estimate bias; x̄5mean estimate bias;
n5number of projects.
Thereafter, the consistency in the estimates was
determined by calculating the coefficient of variation
(CV) as follows:
CV~standard deviation
mean estimate error|100
The standard deviation and CV were also determined for
projects in the different groups of the eight factors
examined (Table 2). Coefficient of variation is a measure
of predictability of estimate bias. Large coefficient of
variation implies that estimate bias is volatile and
unpredictable. By visual inspection of the results
(Table 2), the estimates for the 56 projects (put together)
are inconsistent with a coefficient of variation of 10.17%.
It is assumed that a double digit coefficient of variation is
large. Thus the risk of estimation bias is not small. It also
suggests that firms have little control over the propensity
that estimates would be biased.
Table 1 Preliminary analysis of 56 projects
Project factors broken
into sub-groups
Number of
projects
Mean error (%)
(estimate bias)
Standard
deviation (%)
Coefficient of variation (%)
(estimate consistency)
*Project value ($)
1–5 000 000 20 8.95% 10.85% 9.96%
5 000 001–10 000 000 16 1.38% 7.79% 7.68%
10 000 000+ 20 1.95% 11.16% 10.95%
**GFA (m2)
1–3000 19 5.32% 12.32% 11.70%
3001–10 000 19 5.68% 8.54% 8.08%
Above 10 000 18 1.72% 10.76% 10.58%
Number of storeys
1–2 storeys 29 9.31% 9.32% 8.53%
3–7 storeys 17 21.59% 8.76% 8.90%
8+ storeys 10 20.30% 10.53% 10.56%
Location
***CBD 10 20.90% 12.71% 12.83%
Metropolitan 34 4.82% 9.42% 8.99%
Rural 12 7.08% 11.41% 10.66%
Procurement route
Traditional (lump sum) 38 2.66% 11.22% 10.93%
Design & construct 18 7.72% 8.48% 7.87%
Project type
Residential 18 2.17% 7.84% 7.67%
Industrial 9 8.11% 10.93% 10.11%
Commercial 29 4.41% 11.92% 11.42%
Principal structural material
Timber 9 9.78% 10.84% 9.87%
Steel 14 7.29% 7.41% 6.90%
Concrete 29 0.69% 10.54% 10.47%
Price intensity ($/m2 GFA)
1–1200 20 7.55% 8.63% 8.02%
1201–1700 18 3.00% 11.54% 11.20%
Above 1700 18 1.94% 11.30% 11.08%
Total 56 4.29% 10.61% 10.17%
Notes: *Project value is measured in Australian dollars ($); **GFA5gross floor area; ***CBD5central business district.
Building cost estimates 1261
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
Projects estimate bias and consistency across
project factors
Estimate bias and estimate consistency for each group
in the eight project factors were examined. The mean
estimate bias and coefficient variation (estimate
consistency) were also determined for the projects in
each group of the factors shown in Table 1. The
results (Table 1) show that estimates of projects in the
three project value categories tend to be overesti-
mated. The estimates of projects with lower project
value are more biased than estimates of projects with
larger value. The same trend is observed for estimates
of projects in the different ‘GFA’ and ‘number of
storeys’ categories. Thus the estimates of smaller
projects were more biased than the estimates of larger
projects. The result for ‘GFA’ is consistent with
Skitmore and Drew (2003) and Ling and Boo (2001)
which both found that the estimates of projects of
smaller gross floor area tends to be more biased than
the estimates of projects of larger gross floor area.
However, for ‘project value’, the result is inconsistent
with Skitmore and Drew (2003) where estimates of
projects with a smaller value (less than $60m) are less
biased than estimates of projects with a higher value
(over $60m).
Further, the cost estimates for traditionally procured
projects tend to be more accurate than the cost
estimates of those procured using design and construct
method. Cost estimates of residential projects tend to
be the least biased, followed by estimates of commercial
and industrial projects. The findings could be adduced
to experience; for example, the quantity surveying firm
from which the sample projects were obtained has
handled many traditionally procured projects and
residential and commercial projects. This could be
the reason why the estimates for those projects are less
biased. Projects that were procured with design and
construct method, as well as industrial projects are less
common in the firm’s experience. This could explain
why estimates for those projects are more biased.
The preliminary analysis (Table 1) also suggests that
cost estimates of projects using concrete as the
principal structural material are by far the least biased
(overestimated by 0.7%), followed by steel with an
average overestimate of 7.2%. Estimates of projects
using timber as their primary structural material tend to
be the most biased, with an average overestimate of
9.8%. Again, this result could be explained by
experience. The sample projects indicate that the firm
from which the data were drawn has handled fewer
projects using timber when compared to projects
constructed with steel and concrete.
With regard to price intensity, the trend seems to be
negatively related. Estimates of projects with low $/m2
GFA tend to be more biased than estimates of projects
with high $/m2 GFA. This is contrary to Gunner and
Skitmore’s (1999b) price intensity theory. However,
because this is a preliminary analysis, it is not conclusive.
Turning to estimate consistency, Table 1 suggests that
the estimates of projects with lower value tend to be more
consistent than the estimates of projects with higher
value. ‘Number of storeys’ follows the same trend. The
estimates of projects with lower number of storeys are
more consistent than the estimates of projects with higher
number of storeys. In contrast, ‘GFA’ shows an opposite
trend such that the estimates of projects having lower
GFA are most inconsistent when compared with the
estimates of projects with larger GFA.
For location, the estimates of projects located in the
central business district (CBD) are the least consistent
when compared with the estimates of projects in the
metropolitan and rural areas. The estimates of projects
procured with traditional methods are less consistent
when compared with the estimates of design and
construct projects. Also, residential project estimates
are more consistent than industrial project estimates
while industrial project estimates are more consistent
than the estimates of commercial projects. The analysis
also revealed that the estimates of projects constructed
with concrete as the principal structural material are the
least consistent followed by the estimates of projects
Table 2 Result of ANOVA and Levene’s test for homogeneity of variance*
ANOVA (for estimate bias) Levene’s test (for estimate consistency)
Project factor F R Test statistic RProject value 3.27 0.046* 1.006 0.372
Gross floor area 0.77 0.466 1.202 0.309
No. of storeys 11.86 0.001* 1.387 0.244
Project location 1.70 0.193 0.123 0.885
Procurement route 2.88 0.096 0.577 0.451
Project type 0.94 0.395 0.921 0.404
Structural material 3.96 0.026* 0.677 0.570
Price intensity 1.55 0.222 0.729 0.487
Notes: *Significant relationships taken at the 5% level.
1262 Aibinu and Pasco
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
constructed with timber and steel. The estimates of
projects with higher price intensity tend to be more
inconsistent than the estimates of projects with lower
price intensity.
Put together, the preliminary analysis suggests that
the estimates of small projects are more biased but are
more consistent than the estimates of large projects.
This means that bias in the estimates of smaller projects
is larger and is consistent, whereas bias in the estimates
of large projects is smaller but inconsistent.
Hypotheses testing
The impact of project factors on estimate bias and
estimate consistency
The data from the 56 projects were analysed more closely
to investigate the trends uncovered by the preliminary
analysis (Table 1). Analysis of variance (ANOVA)
method was used to compare the mean estimate bias of
projects in the different groups of each of the eight
factors. The aim was to investigate whether bias in the
estimate of the projects varies according to each of the
eight factors in Table 1. The results (Table 2) show that
there are significant differences between the mean
estimate bias for projects in the three categories of
‘project value’ (F53.27, p50.046). Also, there are
differences in the mean estimate bias for projects in the
three categories of ‘number of storeys’ (F511.86,
p50.001). Similarly, the mean estimate bias of projects
in the three categories of ‘principal structural material’
are statistically different (F53.96, p50.026).
The ANOVA results indicate that estimate bias
varies according to project value, number of storeys
and type of principal structural material used.
After significant differences in the mean estimate
were found, the Tukey–Kramer procedure (Levine
et al., 2005) was used to determine which groups are
different. The Tukey–Kramer procedure enabled
simultaneous examination of comparisons between all
pairs of groups (Levine et al., 2005) for each of the
three significant factors (project value, number of
storeys and principal structural material). The analysis
revealed the following differences:
N There is a significant difference between the
mean estimate bias of projects having a value of
‘$1–$5 000 000’ and those having value of
‘$5 000 001 and above’. All the other pairwise
differences are small enough that they may be
due to chance.
N The mean estimate bias of projects with ‘1–2
storeys’ significantly differs from the mean estimate
bias of projects with ‘3 or more storeys’. Similar to
project value, all the other pairwise differences are
small enough that they may be due to chance.
N There is a significant difference between the
mean estimate bias of projects constructed with
timber and those constructed with concrete. As
before, all the other pairwise differences are small
enough that they may be due to chance.
Similar to the trend observed in the preliminary analysis
(Table 1), the estimates of smaller projects tend to be
more biased than the estimates of larger projects. An
explanation could lie in the delegation of estimating
work. Typically, estimates of smaller projects are under-
taken by junior quantity surveyors, while estimates of
larger and more complex projects are undertaken by
more experienced staff; when junior staff are involved,
they are supervised by senior quantity surveyors.
Levene’s test (Brown et al., 1974) was used to
investigate whether the consistency in the estimates
varies according to the eight factors. Levene’s test
enabled analysis of the homogeneity of variance
(Conover et al., 1981; Levine et al., 2005). The result
(Table 2) shows that the consistency in the estimates
did not vary according to any of the eight factors
examined (i.e. the variances were found to be homo-
geneous across the groups of each of the eight project
factors).
Regression modelling of project factors
influencing estimate bias
Project factors influencing estimate bias were further
investigated using the traditional multiple linear regres-
sion technique with the help of the Statistical Package
for Social Sciences software (SPSS). The independent/
predictor variables are the eight factors listed in Table 1
and the dependent variable is estimate bias (Y). The
factors were entered stepwise. Gross floor area (GFA),
project value and price intensity were included in the
regression model as continuous variables while other
factors were included as categorical variables according
to the groupings shown in Table 1. Thus the multiple
regression model developed to determine the impact of
the factors may be represented as follows:
Y~b0zb1GFAzb2ProjectValuezb3Price Intensity
zb4CBDzb5Metropolitanzb6Rural
zb7Traditionalzb8D&Czb9Residential
zb10Industrialzb11Commercialzb12Timber
zb13Steelzb14Concretezb151{2Storeys
zb163{7Storeyszb178Storeys and aboveze
where: Y5estimate bias;
Building cost estimates 1263
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
b05is the Y intercept—it represents the average
estimate bias (Y) when all the independent factors in
the model are zero;
b1 to b17 are the slopes of Y associated with each
independent factor when other factors are held
constant;
e5random error in Y.
It is assumed the relationship between Y and the
independent variables can be approximated by a linear
model which provides best fit estimates of the model
parameters by minimizing the error of the model
(Draper and Smith, 1981).
The regression coefficients b0, b1, b2…b17 are
unknown parameters. The hypothesis of interest is
that: b15b25b3…5b1750. The predictive performance
of multiple linear regression may be judged by the value
of adjusted coefficient of determination (R2 adjusted).
The results are summarized in Table 3.
From the results (Table 3), bias in estimates of
projects is influenced by project value and number of
storeys. About 29% of the variation in the estimate bias
of projects can be attributed to project value and
number of storeys. Thus we may conclude that
significant variation in early stage estimate bias can be
explained by project size. The final regression equation
may be written as:
Y~0:5811{0:0318 project valueð Þ{0:0906
3{7storeysð Þ{0:0124 8storeys and aboveð Þ
The negative sign of the regression slopes (i.e.
20.0318, 20.0906 and 20.0124) implies that project
cost estimate bias tends to decrease as project size
increases. This further reinforces the findings from the
preliminary analysis (Table 1) and the Tukey–Kramer
procedure which suggest that estimates of smaller
projects tend to be more biased than estimates of larger
projects. As earlier suggested, this may be explained by
experience of estimators.
Lowe and Skitmore (2001) stated that estimators
prefer the use of individual data and experience.
Akintoye and Fitzgerald (2000) found that the three
main methods of cost estimating are: (1) standard
estimating procedure in which construction costs are
initially found and allowances for overheads and profit
are added; (2) comparisons with past projects based on
personal experience; and (3) comparisons with past
projects based on documented facts. They noted that
these three models are ‘experience based models’. Thus
there are reasons to believe that the higher level of bias
observed in smaller projects might be because such
estimates are prepared by junior and less experienced
staff.
Overall effects of price intensity and type of
principal structural material
In the multiple regression analysis process, price
intensity was entered first to remove any confounding
effects. However, price intensity had no effect in the
final model. Thus the data did not support the price
intensity theory (Gunner and Skitmore, 1999b).
The ANOVA analysis shows that estimate bias varies
according to the type of principal structural material
used (Table 2). However, when put together with the
other project factors in the multiple regression model,
‘principal structural material’ is not a significant
predictor of estimate bias. Perhaps the effect was
masked by project size (confounding effect) as postu-
lated by Gunner (1997).
Has pre-tender building cost estimate accuracy
improved over time?
A scatter plot was used to test Hypothesis 2 which states
that the accuracy of pre-tender cost estimate has not
improved over time. The ratio of estimate bias and project
value (bias ratio) was determined for each project.
Thereafter, the bias ratios of the 56 projects were arranged
chronologically according to the time that the estimates
were undertaken starting from 1999 to 2007. This yielded
a time series dataset. A scatter plot of the time series data
was then constructed (see Figure 2). Similarly, consis-
tency ratios were determined for the 56 projects and were
Table 3 Results of multiple regression analysis
Variable b Coefficient Standard error t value P value R2 (adjusted)
Project valuea 20.0318 0.1316 22.41 0.019 0.2838
Number of storeys
(3–7 storeys)
20.0906 0.0285 23.19 0.002 F58.27 p50.0001
Number of storeys
(8 storeys and above)
20.0124 0.0478 20.26 0.797
Constant 0.5811 0.2028 2.86 0.006
Notes: Variables are significant at the 5% level (P,0.05). a Continuous data Log transformed in order to normalize data.
1264 Aibinu and Pasco
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
arranged according to the time the estimates were
undertaken. A scatter plot of the time series data (for
consistency ratio) was constructed (Figure 3).
If the accuracy of estimates has improved over time,
the scatter plots should indicate a downward trend
towards zero on the Y axis as we move from 1999 to 2007
on the X axis in Figures 2 and 3. However, the scatter
plots suggest that the year of project estimate has no
effect on estimate bias and estimate consistency. The
trend lines are insignificant. Estimate bias is in the same
order of magnitude as it was in 1999, 2000, 2001 and all
through to 2007. The errors are random and incon-
sistent. Flyvbjerg et al. (2002) found similar results in a
study of the differences between estimated cost and
actual costs of 258 transportation infrastructure projects.
We may conclude that the accuracy of pre-tender
building cost estimates has not improved over time.
There may be four possible explanations for these results:
(1) Estimates of new projects are based on historical
cost data from past projects. Thus inaccuracies
are transmitted to new estimates over time.
Figure 2 Scatter plot of bias ratio vs. time
Figure 3 Scatter plot of consistency of estimate vs. time
Building cost estimates 1265
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
(2) Firms do not monitor the performance of their
estimates in terms of accuracy and so are not
aware of any inconsistent error trend.
Knowledge of the trend in estimates error and
inconsistency in estimates should help firms
modify their estimating policy in reaction to the
observed inaccuracies and inconsistencies
(based on Morrison, 1984 findings).
(3) Estimating expertise and skills developed by
firms based on experience with past estimates
are lost, and so reflect the lack of reduction in
estimate bias over time. In Australia, a possible
explanation is that there is a shortage and high
turnover of quantity surveying and cost engi-
neering skills (reflected by the rating points
allocated to the profession by Australia’s
Department of Employment and Workplace
Relations (DEWR, 2008) Skilled Occupation
Lists for immigration purpose). Skill shortage
and turnover may affect the accuracy of
estimates when one considers that financial
management and cost engineering processes
involve assumed knowledge developed over
time. New employees, if recruited, will take
time to reach their full effectiveness.
(4) Other than technical factors such as the skill of
the estimating team and their experience, team
expertise, estimating techniques, or inadequate
data, human-related factors such as the esti-
mator’s attitude might significantly influence
the accuracy of estimates. For instance, esti-
mators are predisposed to increase their esti-
mate (overestimate) when prices are rising—
‘pessimism bias’; and when prices are falling
estimators are predisposed to reduce their
estimates—‘optimism bias’ (Mills, 1997). This
might be responsible for the lack of trend in the
errors (estimate bias) observed and the lack of
consistency in the estimates over time. Also the
inconsistencies in the estimates might be as a
result of wide variance in human judgement
suggesting that the human factor is critical
when attempting to increase the accuracy of
estimates.
Improving the accuracy of estimates
To address objective 3 of this study, the respondents
were presented with 12 methods that could be used to
improve the accuracy of estimates. The methods were
identified from the literature—particularly Ling and
Boo (2001). Respondents were asked to rank the
effectiveness of each method on a scale of 1 to 5
(where 15least effective and 55most effective). The
Relative Effectiveness Index (REI) for each method was
determined using the expression (adapted from
Kometa et al., 1994):
REI~A
B|C
where A5total score; B5highest response options;
C5total number of survey responses.
The REI was then used to rank the methods. From
the results (Table 4), Australian quantity surveyors
perceive ‘ensuring sufficient information is available at
the time of estimating’ as the most effective method of
improving estimating accuracy (first), followed by
‘increased cost planning and control during the design
phase’ (second) and ‘checking all assumptions with
clients and consultants during the estimating period’
(third).
The result is similar to that of Ling and Boo (2001),
who found that the most effective methods of improv-
ing accuracy according to Singapore quantity surveyors
were (a) ensuring design information is sufficient and
available for estimate preparation (M3) which ranked
first in this study; (b) checking all assumptions when
preparing the estimate (M4) which ranked third; and
(c) providing a realistic timeframe for estimating
activity (M6) which ranked seventh. The similarities
indicate an international sentiment regarding methods
of improving the accuracy of estimates. Simulation,
probability and utility function is considered by
respondents as the least effective method of improving
the accuracy of estimates (ranked twelfth).
Test of agreement among quantity surveyors
Fleiss’ kappa statistical measurement (k) (Fleiss, 1971)
was used to ascertain whether quantity surveyors in
Australia agree with the ranking of the 12 methods of
improving the accuracy of estimates. Fleiss’ kappa
measurement is a variant of Cohen’s kappa statistical
measure of inter-rater reliability. Cohen’s kappa is
suitable where there are only two raters whereas Fleiss’
kappa can help researchers to assess the reliability of
agreement between more than one number of raters on
a number of items. If a number of raters assign
numerical rating to a number of items then Fleiss’
kappa statistical measurement will give a measure of
how consistent the ratings are. It is a measure of the
degree of agreement that can be expected above chance
(Fleiss, 1971). Fleiss’ kappa statistical measurement
(k) can be expressed as:
k~P{Pe
1{Pe
where: 1{Pe is the degree of agreement that is
1266 Aibinu and Pasco
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
attainable above chance; P{Pe is the degree of
agreement actually achieved above chance.
The k statistics can take the values between 0 and 1.
According to Landis and Koch (1977) agreement
between respondents is Almost Perfect if k50.81–1.00;
Moderate if k50.41–0.60; Fair if k50.21–0.40; Slight if
k50.0–0.20 and Poor if k,0.00. In this study, the
measurement k is estimated at 0.045 indicating that
quantity surveyors only slightly agree with each other on
the relative effectiveness of the 12 methods of improving
the accuracy of estimates. A possible explanation may be
that respondents make use of the different methods to
achieve the same result. A one-tail t-test reinforces this
conclusion (Table 4). The mean effectiveness for all the
methods is statistically above the midpoint of 3 on a
Likert scale of 1 to 5 (p,0.05) indicating that quantity
surveyors consider all the methods as effective means of
improving the accuracy of cost estimates.
Reducing estimate inaccuracies in practice
Further, the respondents were asked to list internal
review mechanisms in place in their company aimed at
improving the accuracy of pre-tender building cost
estimates. Overall, 33 of 41 survey respondents (80%)
listed at least one review mechanism used in their firm.
In total, 77 review mechanisms were listed and content
analysis was performed on the 77 items. Similar
mechanisms were grouped together and named by the
researchers. Figure 4 shows the frequency of 13
mechanisms identified from the content analysis.
Overwhelmingly, ‘benchmarking’ is the most popular
method used by firms to improve the accuracy of
estimates, followed by ‘internal peer review’ and ‘com-
munication with the market’ which both have relatively
smaller frequencies when compared with ‘benchmark-
ing’. This suggests that estimates are largely based on
cost data from past projects rather than on internal peer
review and market research. This reinforces the explana-
tion that errors in estimates are transmitted from past
projects to new projects. This could be responsible for
the lack of improvement in estimate bias over time.
The content analysis revealed that the use of
computer estimating software (M5) is not frequently
mentioned as a method to improve the accuracy of
estimates (mentioned by only three out of 41 firms—
7%). Also, identification and incorporation of future
market trends into estimates (M4) was mentioned only
seven times. These results also agree with the ranking of
the effectiveness of 12 methods of improving the
accuracy of estimates (Table 4) which shows that
incorporation of market sentiments into estimates using
simulation, probability and utility function (IM8) is the
least effective method of improving the accuracy of
estimates according to the respondents. Probability
estimation or simulation to predict future cost trends or
to extrapolate new estimates from past estimates could
reduce bias in cost estimates for new projects.
However, the results suggest that it is scarcely used.
This indicates that there is potentially low uptake of
computerized statistical techniques such as cost mod-
elling in the industry.
Table 4 Effectiveness of mechanisms for improving estimating accuracy
Improvement
method
Total score Mean One-tailed t-test* (t.3) REI Rank
t-value p-value
IM1 157 4.13 6.85 0.0000 0.826 5
IM2 167 4.18 8.23 0.0000 0.835 4
IM3 180 4.62 14.98 0.0000 0.923 1
IM4 172 4.20 7.81 0.0000 0.839 3
IM5 146 3.65 4.11 0.0001 0.730 10
IM6 155 3.88 6.49 0.0000 0.775 7
IM7 134 3.44 2.74 0.0047 0.687 11
IM8 132 3.38 2.07 0.0230 0.660 12
IM9 149 3.82 5.14 0.0000 0.745 8
IM10 169 4.33 9.27 0.0000 0.845 2
IM11 145 3.82 6.57 0.0000 0.744 9
IM12 153 4.03 7.15 0.0000 0.785 6
Notes: REI5Relative Effectiveness Indices of improvement methods. *One-tailed t-test of mean. IM15Ensure proper design documentation.IM25Establish effective communication and co-ordination between members of the project team. IM35Ensure sufficient information isavailable for estimating. IM45Check all assumptions with clients and consultants. IM55Establish formal feedback for design and estimatingactivities. IM65Provide a realistic timeframe for estimating activity. IM75Use a more rigorous method of estimating. IM85Incorporate marketsentiments and economic conditions into the estimate by way of simulations, probability and utility functions. IM95Incorporate other marketsentiments and economic conditions into the estimate. IM105Increase cost planning and control activities during the design stage.IM115Improve methods of selection, adjustments and application of cost data. IM125Update cost database with new cost analyses and providefeedback for improving estimate accuracy.
Building cost estimates 1267
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
Conclusion
Data analysis revealed that although bias in pre-tender
building cost estimates varies according to project size and
principal structural material used, when all factors are
controlled and combined in a multiple regression analysis,
bias in estimates of projects is significantly influenced by
project size. Estimates of smaller projects tend to be more
biased than estimates of larger projects. None of the eight
project factors studied significantly contributed to the
level of consistency observed in project estimates. Thus
Hypothesis 1 is partly supported.
Firms need to pay greater attention to smaller or less
complex projects. When cost estimates of less complex
projects are undertaken by less experienced members of
staff, they should be reviewed more rigorously by senior
and experienced quantity surveyors.
The preliminary analysis suggests that bias in the
estimates of large projects is smaller but the estimates
are inconsistent. As modern projects are becoming
more complex, previously used estimating techniques
may be inadequate and may not be as effective as they
were previously. Quantity surveyors and cost engineers
need to use suitable estimating techniques if cost
certainty on projects is to be assured.
This study supported Hypothesis 2 which states that
the accuracy of pre-tender building cost estimates have
not improved over time. Quantity surveyors only
slightly agree on the techniques for improving the
accuracy of estimates. Thus Hypothesis 3 is not
supported. Although firms are using benchmarking of
estimates of previous projects to improve the accuracy
on new projects (36% of all responses), the effective-
ness of such an approach would depend on how often
and how accurately cost databases are updated to
incorporate market sentiments and economic condi-
tions. It would also depend on firms understanding the
size and trend of inaccuracies in their past estimates,
the factors influencing the inaccuracies observed and
incorporating such knowledge into new estimates. The
use of probability estimation and simulation is a way
forward in this regard. However, there appears to be a
low uptake of this approach according to the respon-
dents. There is need to create awareness of the benefit
of computerized statistical techniques such as cost
modelling.
Figure 4 Frequency of mechanisms used by firms for improving the accuracy of estimates
Key& M1 Benchmarking& M2 Internal Peer review& M3 Communication with the market& M4 Identify and incorporate future market trends into the estimate& M5 Use of computer estimating software& M6 Bulk Checking / Self Checking procedures& M7 Ensuring proper communication / information flow on the project& M8 Use of external price information& M9 Review with final costs on projects& M10 Comparisons with received tenders for future estimates& M11 Elemental review& M12 Internal Quality Assurance procedures& M13 Identification of project specific needs or risks
1268 Aibinu and Pasco
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011
Firms need to find means of retaining the knowledge
and experience gained on estimates of past projects.
Rigorous cost analysis and documented feedback from
estimates of projects could help firms to pass on the
knowledge gained when estimating new projects. Also,
firms need to find ways of retaining their staff.
Further, quantity surveyors and cost engineers need
to be directly involved during the client briefing at the
project inception in order that they might adequately
understand a client’s requirements, rather than depend
on relayed information from the project manager or
architect. In order to reap and maximize the benefits of
cost engineering and quantity surveying skills, clients
need to appoint quantity surveyors and cost engineers
from project inception.
This is one of the few studies on this subject in an
Australian context. Its contribution is in the approach
used, which involves analysis of real life data and survey
data from professionals across Australia. The limita-
tions are acknowledged. The number of projects used
for the empirical analysis (56) was not large enough
hence small sample groups resulted when projects were
divided and analysed according to the eight project
factors. This placed restrictions on the ability to detect
higher significant effects. However, the results provide
a plausible description of the size and the pattern in the
accuracy of estimates. The research approach and the
step-by-step analysis can serve as a model for others
who may wish to conduct similar studies elsewhere on
this topic and could facilitate international comparison.
References
AIQS (2003) Find Member, QS Firms, and Best Practice
Charter, available at www.aiqs.com.au (accessed June
2007).
Akintoye, A. and Fitzgerald, E. (2000) A survey of current
cost estimating practices in the UK. Construction
Management and Economics, 18, 161–72.
Bromilow, F.J., Hinds, M.F. and Moody, N.F. (1988) The
Time and Cost Performance of Building Contracts 1976–1986,
Australian Institute of Quantity Surveyors, Canberra.
Brown, M.B., Forsythe, A.B. and Alan, B. (1974) Robust
tests for equality of variances. Journal of the American
Statistical Association, 69, 364–7.
Conover, W.J., Johnson, M.E. and Johnson, M.M. (1981) A
comparative study of tests for homogeneity of variances,
with applications to the outer continental shelf bidding
data. Technometrics, 23(4), 351–61.
DEWR (Department of Employment and Workplace
Relations) (2008) Skilled Occupation Lists (2008), avail-
able at www.immi.gov.au/allforms/pdf/1121i.pdf (accessed
30 May 2008).
Draper, N.R. and Smith, H. (1981) Applied Regression
Analysis, 2nd edn, Wiley, New York.
Fleiss, J.L. (1971) Measuring nominal scale agreement
among many raters. Psychological Bulletin, 76(5), 378–82.
Flyvbjerg, B., Holm, M.S. and Buhl, S. (2002)
Underestimating works in public works projects: error or
lie? Journal of the American Planning Association, 68(3),
279–95.
Gunner, J. (1997) Accuracy of construction pre-bid forecasts,
PhD thesis, University of Salford.
Gunner, J. and Skitmore, M. (1999a) Comparative analysis of
pre-bid forecasting of building prices based on Singapore
data. Construction Management and Economics, 17, 635–46.
Gunner, J. and Skitmore, M. (1999b) Pre-bid building price
forecasting accuracy: price intensity theory. Engi-
neering, Construction and Architectural Management, 6(3),
267–75.
Kometa, S.T., Olomolaiye, P.O. and Harris, F.C. (1994)
Attributes of UK construction clients influencing project
consultants’ performance. Construction Management and
Economics, 12(5), 433–43.
Landis, J. and Koch, G. (1977) The measurement of observer
agreement for categorical data. Biometrics, 33(1), 159–74.
Levine, D.M., Stephan, D., Krehbiel, T.C. and
Berenson, M.L. (2005) Statistics for Managers, 4th edn,
Prentice Hall, Upper Saddle River, NJ.
Ling, Y.Y. and Boo, J.H.S. (2001) Improving the accuracy of
approximate estimates of building estimates. Building
Research & Information, 29(4), 312–18.
Lowe, D. and Skitmore, R.M. (2001) Human effects in
construction contract price forecasting: experience and
experimental learning styles. Journal of Construction
Engineering and Management, 127(6), 485–93.
Mills, A. (1997) Analysis of the accuracy of building price
forecasts. Australian Institute of Quantity Surveyors Refereed
Journal, 1(2), 31–8.
Morrison, N. (1984) The accuracy of quantity surveyors’ cost
estimating. Construction Management and Economics, 2,
57–75.
Rawlinsons (2006) Australian Construction Handbook 2006,
Rawlinsons, Perth.
Rawlinsons (2007) Australian Construction Handbook 2007,
Rawlinsons, Perth.
Serpell, A.F. (2005) Improving conceptual cost estimating
performance. AACE International Transactions, EST.13,
EST.13.1–13.6.
Skitmore, M. (1991) Early Stage Construction Price
Forecasting: a Review of Performance, RICS Occasional
Paper, London.
Skitmore, M. and Drew, D. (2003) The analysis of pre-tender
building price forecasting performance: a case study.
Engineering, Construction and Architectural Management,
10(1), 36–42.
Skitmore, R.M. and Picken, D. (2000) The accuracy of pre-
tender building price forecasts: an analysis of USA data.
Australian Institute of Quantity Surveyors Refereed Journal,
4(1), 33–9.
Trost, S.M. and Oberlender, G.D. (2003) Predicting
accuracy of early cost estimates using factor analysis and
multivariate analysis. Journal of Construction Engineering and
Management, 129(2), 198–204.
Building cost estimates 1269
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011