Automation in Construction 18 (2009) 702–713

Contents lists available at ScienceDirect

Automation in Construction

j ourna l homepage: www.e lsev ie r.com/ locate /autcon

A fuzzy DEA–Neural approach to measuring design service performance inPCM projects

Ching-Hwang Wang a,⁎, Chin-Chang Chuang a,b, Chia-Chang Tsai a

a Department of Construction Engineering, National Taiwan University of Science & Technology, Taiwanb Department of Construction Management, Tungnan University, Taiwan

⁎ Corresponding author.E-mail address: C.H.Wang@mail.ntust.edu.tw (C.-H.

0926-5805/$ – see front matter © 2009 Elsevier B.V. Adoi:10.1016/j.autcon.2009.02.005

a b s t r a c t

a r t i c l e i n f o

Article history:Accepted 17 February 2009

Keywords:PCMService performanceEvaluationEffectiveness analysisNeural network

In Professional Construction Management (PCM), service performance in the design stage most affects theoverall project results. However, currently it is hard to quantify the results and make a proper determinationof the quality of the PCM design service because the evaluation mechanism and procedures have not beencompletely implemented yet. To deal with the problem, this study investigates owners' opinions and therebyestablishes the representative performance indicators of PCM design service. An effectiveness analysismethod and a neural network method are successfully combined, so as to construct an evaluation approachof PCM service performance in the design stage. This approach can offer a listing of rank and level of designservice performance in PCM projects, and further provides a simple appraisal table, so that the owner caneffectively measure the performance of PCM design service. Finally, through the case study, this approach isverified to be reasonable and acceptable in practice. The approach can be an effective tool when measuringthe PCM design service performance and can help owners to evaluate and choose PCM consultants in thedesign stage.

© 2009 Elsevier B.V. All rights reserved.

1. Introduction

Professional ConstructionManagement (PCM) is one of the serviceindustries of public construction projects in Taiwan. The concept ofPCM was introduced to Taiwan by the Bechtel Corporation (U.S.A.) inthe early 1980s, when this corporation completed the first PCM case:the Taipei World Trade Center. However, until July 1999, thegovernmental departments seldom adopted PCM contracts becauseof the lack of legal guidelines. Since the Taiwan Government started tocarry out PCM-related regulations, government entities have begun toinclude PCM budgeting on a legal basis for public constructionprojects [1].

Public construction projects in Taiwan are generally divided intotwo stages: design and construction, and they are usually executedseparately. The ideas and viewpoints of design consultants andconstruction contractors are often different, and their rights andliabilities are also often unclear. Thus, the owners are in great need ofPCM to act as a representative of the owner to audit the design and/orany type of work relating to construction. But in fact, the governmententities usually have their own official construction departments forsupervising contractors and their construction projects, but do nothave departments for planning and design. Thus owners must heavilyrely on PCM to assist them in the design stage.

Wang).

ll rights reserved.

Although applications of PCM design service in public constructionprojects have receivedwide attention from project owners, it has beendifficult to assert what the exact benefits of the PCM design serviceare. PCM consultants can offer a great deal of assistance to the ownersin theory, but in fact, the evaluation mechanism for the PCM designservice is still not complete at the present stage. Owners pay anadvisory fee for the PCM service in the design stage, but it is hard totell whether the expected goals of the project are achieved.

Currently, no matter how good or bad the PCM design service is,the PCM consultant still charges a fee in accordance with the progressof the project. It is hard to quantify the results and make a properdetermination of the quality of the PCM design service. Therefore, it isimportant to develop an evaluation mechanism for PCM serviceperformance in the design stage.

With regard to the related research concerning project serviceperformance, Tatum [2] first indicated that the indicators forevaluating the potential of PCM, consist of five service-related itemsin a building's lifecycle. The selection, the evaluation, and theimprovement suggestions for PCM service are applied to actual casestudies.

Holt, Olomolaiye, and Harris [3] adopted questionnaires and expertinterviews to establish fourteen performance indicators, and he usedthe following three techniques to analyze the performance from theseindicators: the Analytic Hierarchy Process (AHP), the fuzzy AHPanalysis, and the fuzzy Multi-Attribute Decision-Making (MADM)analysis.

703C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

After surveying 122 owners on design–build projects through thequestionnaires, Molenaar and Songer [4] revealed that the achieve-ments of projects were determined from the following factors: projectperiod, budgeting, project complexity, vendor experience, vendorteam, and the owner requirements. From these, they established fiveperformance indicators, namely: scheduling, budgeting, conformanceto expectations, administrative burden, and overall user satisfaction.Molenaar and Songer then used a regression analysis technique togenerate a model for vendor selection.

Chua, Kog, and Loh [5] looked into the key factors in engineeringprojects and through the use of theAnalyticalHierarchy Process (AHP),they categorized sixty-seven factors into four main groups. Then theyused Back-propagation Neural Networks (BPNN) to compare informa-tion about the budget performance, schedule performance, and qualityperformance with the literature in these areas.

Arditi and Lee [6] established an approach for the qualityassessment of project service by using relationship matrices. Thecolumns in these matrices represent the owner's needs and expecta-tions and these include ten indicators: schedule control, timelyresponse, job completeness, job enthusiasm, job reliability, problemexploration, appropriate service, responsibility, communication skills,and understanding the needs of the owners. The rows represent thevendor's capabilities, which included five indicators: leadership,professional knowledge, information analysis ability, human resourcetraining, and process management.

Charnes, Cooper, and Rhodes [7] proposed the prototype of DataEnvelopment Analysis (DEA) to assess the relative efficiency ofdecision-making units (DMUs) and they named it the CCR model.DEA is one of the non-parametric methods for evaluating perfor-mance, and it includes multi-input and multi-output of variables.

Chiang, Tsai, andWang [8] evaluated the service industry, by usingthe in-depth interviewing method on top management and estab-lished seven indicators including four input variables and three outputvariables. Their methodology adopted DEA to determine the efficiencyfrontier of service performance. From this technique, null efficiencyprojects can be identified, so that related information pertaining tolow performance indicators can be provided.

Oyedele and Tham [9] conducted research to provide thearchitectural performance criteria that can be used to improve andsatisfy the owner's requirements. Based on a review of the literatureand through a survey of owners via questionnaires, factor analysis wascarried out on the criteria in the data.

Chang, Hwang, and Cheng [10] further extended the efficiencyconcept of the CCR model, and proposed the effectiveness model tomeasure the achievement degree of the objective. In this model, theinputs are fixed and the outputs are defined as the achievement of theobjective if the objective is the optimum type.

Xue, Shen, and Wang [11] measured the productivity of theconstruction industry in China by using DEA-based Malmquistproductivity indices (MPI). In this paper, the MPI is used to measurethe productivity changes of the Chinese construction industry from1997 to 2003. The DEA-based MPI approach establishes a goodinstrument for building up strategic decisions for improving theperformance of the Chinese construction industry and promoting thedevelopment of the industry among different regions.

Liu, and Ling [12] conducted research into the estimation ofmarkup costs for contractors in a changeable and uncertain construc-tion environment. In this study, a fuzzy logic-based artificial neuralnetwork (ANN) model, called the fuzzy neural network (FNN) model,is constructed to assist contractors in making decision regardingmarkup costs. The FNN model provides users with a clear explanationto prove the rationality of the estimatedmarkup costs. Moreover, withthe self-learning ability of ANN, the accuracy of the estimation of thesecosts is improved. This demonstrates that FNN is able to assistcontractors with the estimation of markup cost so that a much betterestimation can be obtained.

Cheng and Ko [13] solved various kinds of problems existing inconstruction management. Fuzzy logic, neural networks, and geneticalgorithms (GAs) have been successfully applied in constructionmanagement to solve various problems concerning uncertainty.Considering the characteristics and merits of each method, thispaper combines the above three techniques to develop an Evolu-tionary Fuzzy Neural Inference Model (EFNIM). Furthermore, thisresearch integrates the EFNIMwith an object-oriented (OO) computertechnique to develop an OO Evolutionary Fuzzy Neural InferenceSystem for solving construction management problems. This systemcould be used as an intelligent decision support system for decision-making to solve manifold construction management problems.

Moreover, in the DEA method, the performance of each case iscalculated according to optimum weights. Therefore, the weights ofindicators are different for each case. Based on the concept of theCross-Efficiency method proposed by Doyle and Green [14], a set ofgeneral weights can be obtained by averaging the optimumweights ofall cases for practical application.

By making a survey of the above-mentioned studies, it can befound that the early studies mainly contained verbal accounts,questionnaires, expert interviews and traditional statistical analyses.Others proposed various conceptual frameworks for their potentialresearch. However, these studies had only fragmentary work coveringthe development of various performance evaluation systems. Themore recent studies were presented with AHP, MADM, BPNN, SOM,and DEA method. Nonetheless, when handling weights, the AHP andMADM methods have problems, including independent hypothesisand human subjectivity among the indicators. Moreover, the dis-advantages of the BPNN or SOM methods are that they requirenumerical data and cannot guarantee the best solution. As for the DEAmethod, it is a good method when establishing an evaluationmechanism with multiple indicators and when the weights cannotbe objectively determined.

From the literature study about the evaluation of PCM serviceperformance, it is clear that the evaluation mechanism for the PCMdesign service demonstrates the following phenomena: (1) mostevaluation indicators cannot be quantified items, (2) the evaluationprocess involves many indicators, the weights of which cannot beobjectively established, and (3) the evaluation result has no regularity,so it is not possible to assume a statistical distribution functionbeforehand.

To deal with the problems mentioned above, this study concen-trates on developing a new analysis approach to evaluating theperformance of PCM design service, in which the goals of the studyare: (1) to investigate and establish representative performanceindicators for the evaluation of PCM design service, and (2) toconstruct a reasonable and useful approach for evaluating PCM serviceperformance in the design stage. The first goal may especially helpPCM consultants to pay attention to the indicators, in order to meetthe owner satisfaction. The second may assist owners in evaluatingand choosing PCM consultants. It would also be useful for PCMconsultants when measuring their own service performance.

2. Basic concept of the proposed approach

Fig. 1 shows the framework of the proposed approach, and thebasic concept of this approach is described as follows.

The identification of indicators is the primary stage of this study. Inthe course of evaluation, the evaluation result of each case changeswith differences in the indicators. Hence, the choosing of performanceindicators must possess the necessary completeness. In this study, theapproach presents a method to identify the representative indicators.

DEA is an evaluation method of multi-input and multi-outputvariables, and conforms to the seven features for a good evaluationmodel defined by Lewin and Minton [15]. Since DEA uses mathema-tical programming to solve the weights of indicators, it is not

Fig. 2. Quality framework and gaps in PCM design service.

Fig. 1. The framework of FDEA–Neural approach.

704 C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

necessary to decide on the functional relationship. Instead, it ispossible to use relative comparisons to decide on the effectivenessvalues of PCM projects. Additionally, the result obtained viamathematical programming can ensure that it is the optimal solutionin the universe. Thus, it can be considered worthwhile to apply theDEA method with the evaluation of PCM service performance in thedesign stage.

Furthermore, from the viewpoint of the owner, the serviceperformance of a PCM project is defined as the overall project results,or goal accomplishment degree of a project under a fixed PCMcontract amount. It means that the input items for DEA must be fixed,and the output outcome can be used to analyze the effectiveness of aPCM project. Also, the variables, such as “the satisfaction degree of theowner” and “enthusiasm and responsibility”, are linguistic variables,and difficult to quantify. Hence, this study introduces Fuzzy theory incombination with DEA, to develop the Fuzzy Data EnvelopmentAnalysis method (FDEA). Through the fuzzy method, linguisticinformation concerning performance indicators can be convertedinto data information and the issue of non-quantifiable indicators canbe solved.

Although the analytical result of FDEA can rank PCM projectsaccording to their effectiveness values, FDEA cannot conduct theclassification and determine the levels. Therefore, it is necessary tosupplement FDEA with a reasonable clustering method, so as toacquire avalid performance level for each PCMproject for practical use.

Pattern classification is one of the main uses for neural networks.Its target is to divide a pattern space into several decision regions,where each decision region represents a category or class. In neuralnetworks, there are many types of models that can do theclassification, such as SOM, ART, and ART-2. With its characteristicsof simplicity and practicality, the Self-Organizing Map neural network(SOM) is the most common network. ART and ART-2 further providethe feedback mechanism to adjudge the most similar cluster by thepre-defined exemplifications. In the case of this study, it is difficult topick out the representative ones because of the limited samples.Therefore, this study uses the SOM method for the modelconstruction.

However, SOM can neither objectively distinguish clusters on atopological map nor determine the levels of the clusters [16,17]. It is

considered that the rank result of FDEA is a precise solution obtainedthrough mathematical programming. Hence, if FDEA can be used as areference to revise the result cluster of SOM, then the reasonableclassification and the levels can be determined. Based on thecombination of FDEA and SOM, this study develops the fuzzy DEA–Neural approach, which can be an effective tool to measure the PCMdesign service performance and can help owners to evaluate andchoose PCM consultants in the design stage.

3. The fuzzy DEA–Neural approach

This study aims at the service performance of PCM projects in thedesign stage. It combines the methods of Fuzzy, DEA, and SOM toconstruct a new evaluation approach for PCM design serviceperformance, in which the representative PCM projects are involved.

From the statistics of the Taiwan Public Construction Commission,it can be seen that over 96% of large-scale infrastructure PCM projectsare executed by the following six major engineering consultant firms:China Engineering Consultants Inc., Sino-tech Engineering Consul-tants Inc., MAA Group Consulting engineers, T.Y. Lin InternationalGroup, Wan-Ding Engineering Consultants Inc., and Join EngineeringConsultants Inc. These six firms are acknowledged to be therepresentatives handling Taiwan PCM projects.

This study investigated all the PCM projects accomplished by thesix firms. Forty-six cases which conform to the following criteria wereselected: (1) Duration: the duration was from 1998 to 2008,(2) Characteristics: they were all public building works, and (3)Size: budget ranged between 30 and 60 million U.S. dollars. Afterdistributing the questionnaire, 42 of the 46 cases replied andcompleted the related information. Therefore, it can be concludedthat the 42 cases in this studymay cover over 90% of large-scale publicbuilding works in Taiwan. It can be said to undoubtedly represent thePCM field in Taiwan at the present stage. The construction of theproposed approach is described as follows.

3.1. Performance indicators of PCM design service

PCM consultants provide project management as well as profes-sional technical services. Consequently, PCM consultants are part ofthe service industry. To analyze the service quality of PCM projects inthe design stage, this study refers to the PZB conceptual model ofservice quality. The PZB model is a conceptual framework foranalyzing service quality proposed by Parasuraman, Zeithaml, andBerry [18,19]. Therefore the model was named by the three scholars'names as the “PZB” model. By comparing the cognition between thecustomer and the service provider, the service performance can befurther recognized. It is noted that from “expectation before service”to “perception after service” in the model, there exist five gaps

Fig. 3. Quality gaps and indicators in PCM design service.

705C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

(differences) in the process. The differences lower the service qualityof the service provider. Since the PZB model is the most mature modelfor analyzing service quality, this study uses the PZB model to analyzethe PCM service performance.

This study examines the PZB conceptual model of service quality,and then analyzes the differences in cognitive values between theowner and the PCM consultant concerning the quality of the designservice. The analysis of PCM service quality is as follows: phase one,compiling the contractual duties of the PCM from PCM-relatedliterature, PCM-related laws and regulations, PCM cases, and expertinterviews; phase two, interviewing owners, to obtain the real-worldopinions of representative PCM owners; phase three, by combiningphases one and two, and the PZB conceptual model, this study hasthen established a service quality framework and located five gaps inthe PCM design performance, as shown in Figs. 2 and 3.

With regard to the five gaps of the service quality framework, thisstudy analyzes possible reasons for their occurrence. Then, five mainperformance indicators along with their sub-indicators for PCM

Table 1Performance indicators in PCM design service.

The main indicators The sub-indicators

1. Understanding the needs ofthe owner

1. PCM phase briefing of design achievements2. PCM design service meeting3. PCM design service monthly statement

2. Establishing the servicestandards

4. Establishment of design progress plan5. Establishment of construction budget plan6. Establishment of design quality standards7. Supervision of project design documents8. Establishment of financial plan

3. Responsibility divisionand integration

9. Role and management skills of PCM consultants10. Work division and cooperation of PCM designteam11. Human resources management and educationtraining

4. Technique characteristicsand supervision control

12. Basic design verification13. Timing of design change14. Execution of value engineering analysis15. Supervision and review of design progress16. Supervision and review of construction budget17. Supervision and review of design quality

5. Communication managementand service spirit

18. Communication of design achievements19. Construction contract pre-operation20. Enthusiasm and responsibility

design service are established and shown in Table 1. They aredescribed as follows:

(1) Gap 1: The first concerns the difference between “theexpectation of the owner before design” and “the perceptionof the PCM consultant by the owner”.The reason is that “Understanding the needs of the owner” is notfully completed. The PCMconsultantmay not always have a clearunderstanding of what features imply high quality to the owner,and what service must meet the owner's needs. The PCMconsultant should submit PCM phased achievements to theowner for reference, and so provide the owner a better means tounderstand the PCM consultant. Thus, this study selects “PCMphase briefing of design achievements”, “PCM design servicemeetings”, and “PCM design service monthly statements” as thesub-indicators for “Understanding the needs of the owner”.

(2) Gap 2: The second difference is between “the perception of thePCM consultant by the owner” and “the service standards of thePCM consultant”.The reason is that “Establishing the service standards” has notbeen fully developed. The PCM consultant may not have theintegrated information of service standards for service items. It isnecessary to establish various design norms, design standards,and financial plans. Thus, this study selects “establishment ofdesign progress plan,” “establishment of construction budgetplan,” “establishment of design quality standard,” “supervision ofproject design documents,” and “establishment of financial plan”as the sub-indicators for “Establishing the service standards”.

(3) Gap 3: The third difference is between “the service standards ofthe PCM consultant” and “actual results of the PCM service”.These reasons are that “Responsibility division and integration”and “Technique characteristics and supervision control” havenot beenwell developed. It is necessary to clearly delineate theresponsibilities for the members of PCM design team. Thus, thisstudy selects “role and management skills of PCM consultants,”“work division and cooperation of PCM design team,” and“human resources management and education training,” as thesub-indicators for “Responsibility division and integration”.Furthermore, the PCM consultant should have sufficientprofessional capabilities in design, so as to ensure the designprocess to be in good progress, budget and quality. Thus, thisstudy selects “basic design verification,” “timing of designchange,” “execution of value engineering analysis,” “supervision

Table 2The evaluation table of indicators.

706 C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

and review of design progress,” “supervision and review ofconstruction budget,” and “supervision and review of designquality” as the sub-indicators for “Technique characteristics andsupervision control”.

(4) Gap 4: The fourth difference is between “overall results of thePCMdesign team” and “theperceptionof theowner after design”.The reason is that “Communication management and servicespirit” has not been fully implemented. The PCM consultantshould establish a good communication mechanism for theowner, and it is necessary to do the design service with greatenthusiasm and a sense of responsibility. Thus, this study selects“communicationof design achievements,” “construction contractpre-operation,” and “enthusiasmand responsibilities” as the sub-indicators for “Communication management and service spirit”.

(5) Gap 5: The fifth difference is between “the perception of theowner after design” and “the expectation of the owner beforedesign”.This difference causes the fifth evaluation gap in the servicequality. As regards the owners, theymost respect thefinal results.The satisfactiondegreeof the owner ismost important, and this isdetermined by the perception of the owner after design. Thus,Gap5 represents theoverall project results of PCMdesign service.The value of Gap 5 determines the satisfaction degree of theowner, and this is affected by the achievements of PCM designservice in Gap1, Gap2, Gap3 and Gap4.

3.2. FDEA performance rank

For the owners, PCM design service performance can be repre-sented by the goal accomplishment degree. Thus, referring to theeffectiveness model [10], effectiveness value is used in this study tomeasure the performance of PCM design service. In addition, thisstudy adopts Fuzzy theory to calculate the membership functions oflinguistic variables. The PCM Fuzzy Effectiveness (FE) model proposedby this study is established as follows.

If there are n PCM projects, and the evaluation scores of mperformance indicators is Zj=(Z1j, Z2j,…, Zmj), j=1,…, n, then thepossible set of n×m scores for the performance indicators can beexpressed as:

F~Z� �

=~Zj j

Xnj=1

λj ·~Zijz

~Zij;

Xnj=1

λjV1; λjz0; i = 1; :::;m; j = 1; :::;n

8<:

9=;:

ð1Þ

Of which, λj is the intensity variable or weighting factor, which isrelated to the outcomes of the indicators for the PCMservice. Then, θ is theratio of the radial distances of the effectiveness frontier and the outcomes.

The effectiveness value of the kth PCM project can be derived asfollows:

Max θk

s:t:Xnj=1

λjV1; j = 1;…;n

Xnj=1

λj ·~Zijzθk ·

~Zik; i = 1;…;m

λjz0:

ð2Þ

This study uses the upper and lower limits of the α-cut set of Fuzzytheory, to convert fuzzy membership functions into several pairs ofcrisp data. From Eq. (2), conversions for the upper and lower limits areas follows:

Max θkð ÞLα

s:t:Xnj=1

λjV1; j = 1;…;n

Xnj=1

λj ·~Zij

� �U

αz θkð ÞLα ·

~Zik

� �L

α; i = 1;…;m

λjz0:

ð3Þ

Max θkð ÞUαs:t:

Xnj=1

λjV1; j = 1;…;n

Xnj=1

λj ·~Zij

� �L

αz θkð ÞUα ·

~Zik

� �U

α; i = 1;…;m

λjz0:

ð4Þ

However, in order to ensure the distinguishing ability of the FEmodel in the multi-dimensional fuzzy space, the number of theassessed projects requires at least twice the number of performanceindicators [20]. But unfortunately, the number of PCM cases in thedesign stage is limited in Taiwan. To deal with the above shortcoming,this study develops a multi-hierarchyway for the FEmodel in FDEA. Inthis way, the twenty sub-indicators are provided as the criteria forowners' reference while evaluating the five main indicators. Thescores of five main indicators were concluded by the performance ofthe sub-indicators, and used for the input variables of the FE model soas to ensure the distinguishing ability of the FE model. The outputresult represents the effectiveness value of each PCM project. In thisway, it is possible to elevate the identification ability of the FE model.

The application program of the FE model can be written withoptimization software, such as GAMS, LINDO, and LINGO. GAMS, is

Table 4The effectiveness value and performance rank.

Project Effectiveness value Rank

Upper limit Lower limit

1 0.9041 0.7703 212 1.0000 1.0000 13 1.0000 0.9122 54 0.8889 0.7500 245 0.6667 0.5500 396 0.7778 0.6500 347 0.8889 0.7500 248 0.7778 0.6500 349 1.0000 0.9141 310 1.0000 0.9000 711 1.0000 0.9000 712 0.8889 0.7500 2413 0.7778 0.6500 3414 0.7854 0.6622 3215 0.7778 0.6500 3416 0.8087 0.6797 3117 0.6667 0.5500 3918 0.9224 0.7838 1819 0.9290 0.7891 1520 1.0000 0.9000 721 0.9224 0.7838 1822 0.9290 0.7891 1523 1.0000 0.9000 7

Fig. 4. Membership functions of semantic variables.

707C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

especially good as it has a simple and clear user interface as well as arobust and stable ability for numerical analysis. In fact, GAMS is ahigh-level language environment rather than an optimization algo-rithm. Users can easily establish, modify, and debug their optimizationmodel for input of the file first. After translating to the lower-levelmachine language, the model can be calculated by numerical analysisalgorithms, the output of which the user can employ. Therefore, the

24 0.7778 0.6500 3425 1.0000 0.9107 626 0.8889 0.7500 2427 0.8889 0.7500 2428 0.6667 0.5500 3929 0.8889 0.7500 2430 0.9041 0.7703 2131 0.6667 0.5500 3932 0.8889 0.7500 2433 0.9617 0.8203 1334 1.0000 0.9000 735 0.9041 0.7703 2136 0.7854 0.6622 3237 0.9224 0.7838 1838 0.9290 0.7891 1539 1.0000 0.9000 740 0.9617 0.8203 1341 1.0000 0.9141 342 1.0000 1.0000 1

Table 3The evaluation scores of each project.

Project The evaluation scores

Indicator 1 Indicator 2 Indicator 3 Indicator 4 Indicator 5

1 S MP S D MP2 S VS VS VS MP3 VS S VS S S4 D MP S D D5 D VD D VD VD6 D D VD VD MP7 S MP MP MP D8 MP D D VD D9 VS S VS VS VS10 VS S S S MP11 VS MP S MP S12 S MP MP D S13 MP D VD VD D14 MP D MP D D15 MP D VD VD D16 D MP D VD MP17 D VD D D D18 S S MP D MP19 D S MP D S20 VS S S MP S21 S S MP MP MP22 MP S S D MP23 S S VS MP MP24 D VD MP D D25 VS S S VS MP26 MP MP S D MP27 S D MP S S28 D VD D VD D29 S MP MP D MP30 MP MP D S S31 D VD VD D D32 MP MP D S MP33 S S S MP S34 VS S MP S S35 S MP S MP D36 MP D MP D D37 MP S MP S MP38 S S MP D S39 VS S MP S VS40 S S S S S41 VS S VS VS VS42 VS VS MP S VS

computer program of the FE model was written by GAMS rather thanby other software.

The evaluation table of the main indicators is shown in Table 2 andthe evaluation scores can be represented by the owner's satisfactiondegree. The membership functions of linguistic variables are repre-sented using multi-phase fuzzy statistics and the evaluation scores ofeach indicator are expressed in L–R type as follows: Very Satisfied(VS)={90, 100, 10, 0}, Satisfied (S)={75, 80, 5, 10}, MediocrePerformance (MP)={65, 70, 5, 5}, Dissatisfied (D)={55, 60, 5, 5},and Very Dissatisfied (VD)={0, 50, 0, 5}.

This definition of semantic variables serves as a standard forcomparing the extraction of scores. Take Very Satisfied (VS)={90,100, 10, 0} for example, where 90 and 100 respectively represent thelower and upper boundaries of membership function when member-ship degree is equal to one. Ten and zero respectively represent the leftand right extended scopes of the membership function whenmembership degree is equal to zero. It is shown in Fig. 4.

The scoring system is expressed in the following: (1) When thescore is between zero and 80 (90 minus 10), the membership of VS iszero. It means that there are no owners who feel “very satisfied” aboutPCM service when the score is lower than 80. (2) When the score isbetween 80 and 90, the membership of VS varies from zero to one. Forexample, when the score is 85, the membership of VS is 0.5. It meansthat only 50% of the owners feel “very satisfied” about PCM service

708 C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

when the score is equal to 85. (3) When the score is between 90 and100, themembership of VS is equal to one. It means that all the ownersfeel “very satisfied” about PCM service when the score is higher than90.

The evaluation scores of the five main indicators are shown inTable 3. After calculating the upper and lower limits of the forty-tworepresentative PCM projects in Taiwan, the performance rank can beestablished according to the effectiveness values or goal accomplish-ment degree. The results are shown in Table 4.

Furthermore, in the analysis of the FE model in FDEA, there are fiveslack variables derived from the five indicators. The slack variables ofthe forty-two projects are shown in Table 5. From the values of theslack variables, it can be known which indicators need improvement.If a slack value is not equal to zero, it means that the project is notrelatively effective to the indicator, and a larger value means theindicator has a larger room for improvement.

3.3. SOM performance cluster

The above-mentioned FE model can establish the performancerank of forty-two representative PCM projects, but cannot acquire avalid performance level for each project. Hence, this study further usesthe cluster function of SOM to conduct a performance pre-classifica-tion. SOM can acquire the cluster result of all the projects using theconcepts of topology and neighborhood.

Table 5Slack variables of projects in PCM design service.

Project Indicator 1 Indicator 2 Indicator 3 Indicator 4 Indicator 5

1 0.0000 0.0000 0.0000 11.6307 0.00002 0.0000 0.0000 0.0000 0.0000 0.00003 0.0000 0.0000 0.0000 17.5000 17.50004 7.2727 0.0000 0.0000 6.7199 0.00005 0.0000 6.9079 0.0000 17.5000 17.50006 10.0000 10.0000 7.9605 15.0658 0.00007 0.0000 0.0000 3.2801 5.7237 20.00008 0.0000 0.0000 6.1748 25.0658 10.00009 0.0000 0.0000 0.0000 0.0000 0.000010 0.0000 0.0000 17.5000 17.5000 27.500011 0.0000 10.0000 17.5000 27.5000 17.500012 0.0000 0.0000 3.2801 15.7237 0.000013 0.0000 0.0000 23.6748 25.0658 10.000014 0.0000 0.0000 0.0000 10.3610 9.432815 0.0000 0.0000 23.6748 25.0658 10.000016 11.0570 0.0000 0.0000 22.1933 0.000017 0.0000 6.9079 0.0000 0.0000 0.000018 0.0000 0.0000 0.0000 14.8098 8.156119 21.3777 0.0000 0.0000 15.0141 0.000020 0.0000 0.0000 17.5000 27.5000 17.500021 0.0000 0.0000 0.0000 4.8098 8.156122 5.0141 0.0000 0.0000 21.3777 0.000023 17.5000 0.0000 0.0000 27.5000 27.500024 3.6364 21.4294 0.0000 10.0000 0.000025 0.0000 0.0000 17.5000 0.0000 27.500026 5.7237 0.0000 0.0000 20.0000 3.280127 0.0000 5.7237 10.0000 0.0000 0.000028 0.0000 6.9079 0.0000 17.5000 0.000029 0.0000 0.0000 3.2801 15.7237 10.000030 10.6341 0.0000 19.0035 0.0000 0.000031 0.0000 6.9079 17.5000 0.0000 0.000032 5.7237 0.0000 20.0000 0.0000 3.280133 2.4857 0.0000 0.0000 12.4857 0.000034 0.0000 0.0000 27.5000 17.5000 17.500035 0.0000 0.0000 0.0000 10.6341 19.003536 0.0000 0.0000 0.0000 10.3610 9.432837 4.8098 0.0000 8.1561 0.0000 0.000038 1.3777 0.0000 0.0000 15.0141 0.000039 0.0000 0.0000 27.5000 17.5000 0.000040 2.4857 0.0000 0.0000 2.4857 0.000041 0.0000 0.0000 0.0000 0.0000 0.000042 0.0000 0.0000 0.0000 0.0000 0.0000Frequency 33% 19% 43% 74% 48%

In this approach, the Neural Network Toolbox in MATLAB is usedto construct the computer program of SOM. The evaluation scores ofeach indicator are used for the input variables to produce a two-dimensional topological map. Then, SOM can cluster the patternsaccording to their site distribution on the topological map. In thispaper, the cluster center is difficult to define because of the limitnumber of the cases. Therefore, this study compared and triedvarious different sizes of topological maps including 5⁎5, 6⁎6, 7⁎7and 8⁎8. By referring to the four topological maps, a more consistentcluster result can be constructed and then the accuracy of theclusters is enhanced.

After being cross-compared to the 5⁎5, 6⁎6, 7⁎7, and 8⁎8topological maps as shown in Fig. 5, the reasonable cluster resultcan be found. Thus, the forty-two representative PCM projects areinitially classified into five clusters as shown in Table 6. It canpreliminarily be seen that Projects 2, 3, 9, 10, 11, 20, 25, 34, 39, 41,and 42 are included in a cluster (Grade 1), Projects 4, 18, 19, 22, 23,26, 33, 37, 38, and 40 are another cluster (Grade 2), Projects 1, 7, 12,21, 27, 29, 30, 32 and 35 form the third cluster (Grade 3), Projects 6,8, 13, 15, and 16 form the fourth cluster (Grade 4), and Projects 5, 14,17, 24, 28, 31, and 36 form the last cluster (Grade 5). So far, SOM isstill unable to determine the levels of the performance clusterscorrectly.

3.4. FDEA–SOM performance level

If non-conformities occur when contrasting FDEAwith SOM, theremust be an appropriate method for conducting reasonable revisions.To solve this problem, the approach establishes a framework for doingFDEA–SOM contrastive analysis. In the process, SOM can bepreliminarily conducted when classifying all the projects, and FDEAcan correct the subjective classification or classification bias of SOM.The process of FDEA–SOM contrastive analysis is shown in Fig. 6 and itis executed as follows:

(1) If n projects are in the same cluster of SOM, and are ranked nextto each other in FDEA:It can be inferred that the SOM result matches the FDEA result,and the performance level can be determined just by the FDEAresult.

(2) If n projects are in the same cluster of SOM, but are not rankednext to each other in FDEA:It is possible that these projects were caused by subjectiveclassification bias of SOM. These projects should be re-clusteredaccording to the FDEA result. Another reason is that there aretoo few groups in SOM: In this situation, re-adjusting theparameters of SOM and re-conducting the SOM cluster shouldbe done, until a result consistent in both FDEA and SOMappears.

(3) An individual project is not in the SOM cluster, but is next toeach other in the FDEA rank:In the neural network method of clustering, there sometimesexists an exception. This individual project should be adjustedinto a reasonable cluster according to the FDEA rank.

In each case, the aforementioned procedure should be repeateduntil it is possible to clearly divide the clusters, and the reasonablelevels of PCM projects in the design stage can be obtained. Finally, thisapproach establishes five clusters and five levels for all the forty-twoPCM projects considered, as shown in Fig. 7.

3.5. A simple appraisal table

The fuzzy FDEA–SOM approach is calculated by a computerizedmodule. To conveniently apply the approach to practice, this studyintegrates the approach into a simple appraisal table. The simpleappraisal table allows the owner to simply evaluate the performanceof PCM design service without the use of computer programs.

Fig. 5. Performance clusters of SOM.

709C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

The simple appraisal table is established as follows: Converting Eq.(2) into the dual model Eq. (5), and the weights of the indicators canbe calculated.

Min/k

s:t: uiz0; i = 1;…;m

Xmi=1

ui~Zij − /kV0; j = 1;…;n

Xmi=1

ui~Zik = 1:

ð5Þ

ϕk: effectiveness value of the kth projectZij: ith performance indicator of the jth projectui: virtual multiplier of the ith performance indicator.

The virtualmultiplier ui represents the indicator weight, and it is theoptimum weight combination of each project. The optimum weight

Table 6Cluster results of SOM.

Grade Case

1 2, 3, 9, 10, 11, 20, 25, 34, 39, 41, 422 4, 18, 19, 22, 23, 26, 33, 37, 38, 403 1, 7, 12, 21, 27, 29, 30, 32, 354 6, 8, 13, 15, 165 5, 14, 17, 24, 28, 31, 36

combination of each project is different, but their averageweight can betaken as a general weight applicable to all projects [21]. The generalweight combination can be considered by a peer-evaluation.

It can be seen in Table 7 that the optimum weights of 42 casescalculated by FDEA in both upper and lower situations. Table 8 showsthe average of weights calculated from Table 7. For example, the upperlimit of weight (45%) for Indicator 1 in Table 8 is the average of the first

Fig. 6. The framework of FDEA–SOM contrast.

Fig. 7. Performance level of FDEA–SOM.

710 C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

column of Table 7. The other indicators are averaged in the same way.Thus, the range of the general weights which can be used for all casesare determined. This study uses the general weights as the indicatorweights of the simple appraisal table as shown in Table 9. When a newproject is assessed, the user can directly use the table without re-calculating the FDEA.

After setting the general weight combination as the weights ofindicators, the owner can score each indicator in the simple appraisaltable, as shown in Table 9.

Taking the forty-two representative PCM projects in this study, andmultiplying the evaluation scores of indicators by the aforementionedweights, then the performance score of each project can be obtained.Furthermore, by individually compiling the performance scores ofprojects at each level, the maximum andminimum of these scores canrepresent the score range of this level. If the score ranges ofneighboring levels overlap, then the center point should be taken asthe border between the two levels. These established bordersbetween levels can be used as a comparison standard for futureevaluation, as shown in Table 10.

4. Discussion

Although there have been many evaluation mechanisms in theservice industry, have these been reasonable or acceptable in practice?Currently, no matter how good or bad PCM service performance is, thePCM consultant still charges a fee in accordance with the progress ofthe project. The evaluationmechanism is still not complete at present,and therefore it is hard to quantify the achievement of PCM serviceperformance. Among the people we had interviewed, there weremany who believed that it was essential that a more effectivemechanism for evaluation measurement be implemented.

4.1. About indicators

This study aims to propose a new conceptual model of evaluationfor service performance, in which the weights of multiple indicators

must be objectively established. In the course of evaluation, it can befound that the evaluation results of these cases change withdifferences in performance indicators. We spent a great deal of timediscussing the methods of choosing the performance indicators.Twenty performance sub-indicators, which the study lists, representthe most part of PCM service quality, and these twenty control itemsmust be implemented when improving service quality.

In the process of indicators identification, this approach proposedthe five gaps of the service quality framework in the PCM designperformance. Regarding PCM consultants, they must endeavorthrough Gap 1, Gap 2, Gap 3 and Gap 4 during the process of designservice. Regarding the owners, they should most respect the finalresults. The satisfaction degree of the owner is most important, andthis is determined by the results of design service in Gap 1, Gap 2,Gap 3 and Gap 4.

4.2. About FDEA

This approach takes the owner's viewpoint when evaluating theperformance of PCM design service. Because the contract budget isfixed, the key point is on the executing results of the project. Thus, theFE model of FDEA presented by this approach is used to evaluate thegoal accomplishment degree.

The FE model can calculate the maximum and minimum values ofproject goal accomplishment. If the difference of values is larger, itmeans that the project has a greater range of variation and the projectperformance is less stable. In other words, the PCM consultant hasgreater room for improvements in this project.

In Table 4, it can be determined that Projects 5, 17, 28 and 31 areRank 39, because these four projects have the lowest values of thelower limits. The ranks of other projects are determined by theirlower limits of the effectiveness values. When the lower limit isgreater, it means that the service performance is more likely to havea better rank. For example, the upper limits of Projects 23 and 25 areequal to one, but the lower limit of Project 25 (0.9107) is greaterthan that of Project 23 (0.9000). It is indicated that Project 25

Table 7Optimum weights of indicators for each project.

Project Weights (%)

Indicator 1 Indicator 2 Indicator 3 Indicator 4 Indicator 5

Upper limit Lower limit Upper limit Lower limit Upper limit Lower limit Upper limit Lower limit Upper limit Lower limit

1 38 37 38 38 23 25 0 0 0 02 53 3 22 55 8 39 8 0 9 33 19 38 21 38 19 25 21 0 21 04 0 0 0 0 100 100 0 0 0 05 100 100 0 0 0 0 0 0 0 06 0 0 0 0 0 0 0 0 100 1007 100 100 0 0 0 0 0 0 0 08 100 100 0 0 0 0 0 0 0 09 20 0 22 43 20 29 20 0 20 29

10 47 100 13 0 13 0 13 0 13 011 53 100 12 0 12 0 12 0 12 012 100 100 0 0 0 0 0 0 0 013 100 100 0 0 0 0 0 0 0 014 38 38 38 37 23 25 0 0 0 015 100 100 0 0 0 0 0 0 0 016 0 0 45 43 27 29 0 0 27 2917 100 100 0 0 0 0 0 0 0 018 38 38 38 38 23 25 0 0 0 019 0 0 45 43 27 29 0 0 27 2920 47 100 13 0 13 0 13 0 13 021 38 38 38 38 23 25 0 0 0 022 0 0 45 43 27 29 0 0 27 2923 40 0 37 0 21 100 2 0 0 024 0 0 0 0 100 100 0 0 0 025 29 33 18 33 18 0 16 33 18 026 0 0 0 0 100 100 0 0 0 027 100 100 0 0 0 0 0 0 0 028 100 100 0 0 0 0 0 0 0 029 100 100 0 0 0 0 0 0 0 030 0 0 38 38 0 0 38 38 23 2531 100 100 0 0 0 0 0 0 0 032 0 0 0 0 0 0 100 100 0 033 0 0 45 43 27 29 0 0 27 2934 47 100 13 0 13 0 13 0 13 035 38 38 38 38 23 25 0 0 0 036 38 38 38 38 23 25 0 0 0 037 0 0 38 38 0 0 38 38 23 2538 0 0 45 43 27 29 0 0 27 2939 29 100 18 0 18 0 18 0 16 040 0 0 45 43 27 29 0 0 27 2941 20 0 22 43 20 29 20 0 20 2942 19 3 19 56 21 3 21 3 19 35

711C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

(Rank 6) shows better performance than Project 23 (Rank 7). In thisway, the performance rank of all the forty-two PCM projects can beobtained.

The FE model not only can analyze the effectiveness values, butalso can offer improvement information to the owner by slack variableanalysis. Project 6 in Table 5 is taken for example: except Indicator 5(Communication management and service spirit), the slack values ofother indicators are not zero. It means that Project 6 shows goodperformance on Indicator 5 and there still remains space forimprovements on four other indicators. In addition, the slack valueof Indicator 4 (Technique characteristics and supervision control) isthe largest. It means that the improvement of Indicator 4 should bethe most important task of the PCM consultant. Hence, the PCM

Table 8The general weights for all indicators.

Main indicators Weights (%)

Upper limit Lower limit

Indicator 1 45 42Indicator 2 20 19Indicator 3 20 19Indicator 4 8 5Indicator 5 12 10

consultant should review the indicator, strengthen the professionalcapabilities of their personnel, and establish a mature educationmechanism.

Furthermore, from the statistics of Table 5, it can be found that 74%projects showed imperfect performance on Indicator 4, also 48%projects for Indicator 5, 43% projects for Indicator 3, 33% projects forIndicator 1, and 19% projects for Indicator 2. It can be concluded thatthe capabilities of PCM consultants in Taiwan are rather inadequate onIndicator 4 (Technique characteristics and supervision control). Thelowest frequency of Indicator 2 (Establishing the service standards)

Table 9The performance table of the simple appraisal table.

Mainindicators

Weights(%)

Owner's Satisfaction satisfaction degree Score

VS S MP D VD

Indicator 1 42–45Indicator 2 19–20Indicator 3 19–20Indicator 4 5–8Indicator 5 10–12

VS — very satisfied (score=84–100); S — satisfied (score=73–84); MP — mediocreperformance (score=63–72); D — dissatisfied (score=53–62); VD — very dissatisfied(score=0–52).

Table 10The level table of the simple appraisal table.

Level Performance score(project)

Statement of level judgment

1 84–100 The achievements of PCM far surpass the owner'sexpectation

2 73–83 The achievements of PCM a little surpass the owner'sexpectation

3 64–72 The achievements of PCM just conform to the owner'sexpectation

4 54–63 The achievements of PCM are a little lower than theowner's expectation

5 0–53 The achievements of PCM are far lower than the owner'sexpectation

712 C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

indicates that most PCM firms handle the items of Indictor 2 veryeffectively.

4.3. About SOM

This study uses the SOM to obtain the initial clusters. In theanalysis of SOM, the performance indicators of forty-two PCMprojectsaremapped to the topology. By checking the distribution conditions ofpatterns on the topological map, it can be seen that the shorter thedistances between projects, then more similar is the performance ofprojects. But even from the same topology, different person mayproduce a different cluster result by using subjective judgment.

In addition, in the process of SOM cluster analysis, the integrality ofperformance classification must be considered in the forty-tworepresentative projects. Namely, these representative projects mustcomprehensively comprise the well performance ones, the mediocreperformance ones, and the poor performance ones, which aregenerally acknowledged by the owners. Thus, the result of thisapproach can correctly represent the levels of PCM service perfor-mance in the design stage.

Furthermore, in questionnaire investigation, what is satisfied, verysatisfied? and what is dissatisfied, very dissatisfied? We mustconclude a clear and rational specification. In addition, forty-tworepresentative cases must be able to fully represent the five levels inthis approach, otherwise the integrality of the evaluation databasewill not be set up correctly.

4.4. About FDEA–SOM

The function of the FDEA–SOM contrastive analysis is to furthercorrect the results of FDEA and SOM. Project 23 for example in Fig. 7,the cluster result belongs to Level 2, but the effectiveness is Rank 7,and the effectiveness value is equal to those of Projects 10, 11, 20, 34and 39. Thus, the cluster result of Project 23 is considered to be wrongdue to erroneous human judgment, and Project 23 should be revisedfrom Level 2 to Level 1. Likewise, Project 4 should be revised fromLevel 2 to Level 3 because its effectiveness is Rank 24. In summary,through SOM, thenwe can determine the clusters; through FDEA, then

Table 11Result of case study.

Mainindicators

Weights(%)

Owner's satisfaction degree Score

VS S MP D VD

Indicator 1 44 88 38.72Indicator 2 19 74 14.06Indicator 3 19 68 12.92Indicator 4 8 92 7.36Indicator 5 10 75 7.50

80.56

VS — very satisfied (score=85–100); S — satisfied (score=73–84); MP — mediocreperformance (score=63–72); D — dissatisfied (score=53–62); VD — very dissatisfied(score=0–52).

we can determine the levels of clusters; and through FDEA–SOM, wecan further establish the borders between the levels.

This study respectively conducted the time sequence and theeffectiveness rank for the projects of each PCM firm, we found the factthat the quality of PCM design service in Taiwan, improved directlywith the accumulation of PCM experience. Such a phenomenonindicates the conditions within a PCM firm: because the internalmanagement information and the related achievement reportsprovide valuable assistance, there can be effective instructions basedon experience that are produced. The subsequent PCM projects canuse the previous case information as a reference, and this processhelps the PCM firm to produce good service performance. Also,changing employees in PCM design team causes differences in servicequality, and should be avoided as much as possible.

5. Case study and verification

In practical application, this approach further uses the “X veteran'svillage reconstruction” Project as a case study.

First, the study asked the owner to fill in the satisfaction degree foreach main indicator of Project X, as shown in Table 2. The data of theowner's satisfaction degree is represented as above-mentioned. Thenthe simple appraisal table was used to calculate the performancescore. If Project X does not belong to Level 1, this project is envelopedin the established effectiveness possibility frontier. The generalweights and the orders of the original projects will not be affected.The simple appraisal table can be used directly to assess theperformance of the new project. If and only if Project X belongs toLevel 1, it is possible that this project has higher performance than anyoriginal project. It will make the move of the effectiveness possibilityfrontier. Therefore, the DEA–SOM approach should be re-executed toobtain the new frontier and the newgeneral weights, so as to establisha new simple appraisal table.

By this approach, Project X was evaluated as shown in Table 11. Theperformance score 80.56 was calculated based on the general weights.It can be judged that Project X belongs to Level 2. Project X did notsurpass the existing databank of the forty-two representative cases inany performance indicator. Thus, the effectiveness frontier did notchange, and hence it was not necessary to update the databank or re-adjust the weights.

However, it is necessary to re-execute the model if slack relatedinformation is required. From the slack variables analysis of FDEA, itcan be known which indicators need improvement, and how muchimprovement should be made. Table 12 shows the slack variables ofProject X. It can be seen that the slacks of Indicator 1 (Understandingthe needs of the owner) and Indicator 4 (Technique characteristicsand supervision control) are zero. It means that Project X showsexcellent performance on both Indicators 1 and 4. Moreover, for theother indicators, Indicator 2 (Establishing the service standards),Indicator 3 (Responsibility division and integration) and Indicator 5(Communication management and service spirit) are not zero, itmeans that these three indicators are the main reasons that affect theoverall project achievement. Also, because Indicator 3 has the mostgreat value 27.5000 than others, it means that “Responsibility divisionand integration” is the most important item to be improved. The PCMconsultant should review this indicator and strengthen the profes-sional capabilities of their personnel.

As regards the section of verification, according to the followingreports: (1) PCM phase briefing of design achievements, (2) PCMdesign service meeting, and (3) PCM design service monthly

Table 12Slacks of case study.

Project Indicator 1 Indicator 2 Indicator 3 Indicator 4 Indicator 5

X 0.0000 17.500 27.5000 0.00000 17.5000

713C.-H. Wang et al. / Automation in Construction 18 (2009) 702–713

statement, the items causing defective performance of Project X indesign stage are dug out as follows:

(1) Inadequate work division and cooperation in PCM team(Indicator 3).

(2) Inadequate execution of decisions made at design meetings(Indicator 3).

(3) Partial errors in contract management (Indicator 5).(4) Lags in progress of public pipelines demolition (Indicator 2).(5) The resolution of disputes was not quick (Indicator 3).(6) Partial errors in construction budget (Indicator 2).(7) Communications in both speaking and writing were insuffi-

cient (Indicator 5).

From these items listed above, it can further be found that majordefective causes of Project X were largely due to Indicators 3. Thesedefective reasons include: plans in progress and budget, managementskills, dispute resolutions, and communication management. Thus,these facts conform to the operation results of this approach in Table12 regarding the case study. It is verified that the approach can beapplied in actual practice as well.

6. Conclusions

This study establishes a new approach for the measurement ofPCM service performance in the design stage. This approach can offera listing rank and level of design service performance in PCM projects,and also provide improvement suggestions on the indicators of eachPCM project. The result reflects the values and opinions of PCM firmsconcerned with the management of public construction projects. Acase study is presented, demonstrating the deficiencies in PCMconsultancy for Taiwanese public engineering in the design stage.This study has made the following conclusions.

• This study establishes a service quality framework of PCM designperformance, and proposes five performance indicators in PCMdesign service. Furthermore, the study confirms the defectivereasons resulting from the five performance indicators in PCMdesign service, with a total of twenty sub-indicators.

• This study confirms which indicators in the PCM design stage arerepresentative of the service performance. Users may input thefuzzy semantic description of indicators in the evaluation table,according to their satisfaction degree with each indicator, to obtainthe corresponding effectiveness value of each PCM project. Theperformance of PCM design service is measured by the effectivenessvalue.

• This study successfully takes the advantages of Fuzzy, DEA, and SOMto propose an evaluation approach of PCM service performance inthe design stage. In this approach, it proposes the FDEA–SOMcontrastive analysis: SOM can preliminarily conduct the classifica-tion of all projects, and FDEA can revise the subjective classificationor classification bias of SOM.

• This approach cannot only resolve both the issue of non-quantifiableindicators and the weights among indicators, but it also develops amulti-hierarchy way to ensure the distinguishing ability of the PCMfuzzy effectiveness (FE) model for the insufficient number of PCMdesign cases in Taiwan. In addition, this approach further provides asimple appraisal table, so that the owner can simply measure theperformance of PCM design service without the use of computerprograms.

• The result of this study not only provides PCM consultants with clearimprovements in design service, but it can also become a referencefor promoting the performance of PCM design service in othersimilar projects. For example, it has been found that Indicator 4“Technique characteristics and supervision control” is the mostimportant course. Thus, PCM consultants should review this

indicator and strengthen the professional capabilities of theirpersonnel.

• The approach proposed by this study can be an effective tool tomeasure the PCM design service performance. It can also helpowners to evaluate and choose PCM consultants in the design stage.Foreign firms and other domestic firms can take advantage of theinformation in this study to understand the mechanism forevaluating PCM public construction projects in Taiwan.

• This study is limited to the exploration of the viewpoints of owners inPCM projects. The scope of the study is also confined to a limitednumber of representative indicators in the design of public construc-tion projects. Therefore, the practical application of this approachmaynot be able to comprehensively include every PCM project.

• In future research, the authors plan to take the viewpoint of the PCMconsultant, consider the resource inputs, and use an input–outputefficiency model to conduct follow-up research. In addition, theevaluation mechanism for the service performance of other projectscan also be addressed using this approach, except that therepresentative performance indicators should be reconfirmed by agroup of pertinent experts.

References

[1] C.H. Wang, C.C. Tsai, C.C. Chuang, PCM in Taiwan: a diagnosis knowledge base inPCM plan/design phase, ASCE Journal of Computing in Civil Engineering 21 (2)(2007) 102–111.

[2] C.B. Tatum, Evaluating PCM firm potential and performance, ASCE Journal of theConstruction Division 105 (3) (1979) 239–251.

[3] G.D. Holt, P.O. Olomolaiye, F.C. Harris, Factors influencing UK construction clients'choice of contractor, Building and Environment 30 (2) (1995) 241–248.

[4] K.R. Molenaar, A.D. Songer, Model for public sector design–build project selection,ASCE Journal of Construction Engineering and Management 124 (6) (1998)467–479.

[5] D.K.H. Chua, Y.C. Kog, P.K. Loh, Critical success factors for different projectobjectives, ASCE Journal of Construction Engineering and Management 125 (3)(1999) 142–150.

[6] D. Arditi, D.E. Lee, Assessing the corporate service quality performance of design–build contractors using quality function deployment, Journal of ConstructionManagement and Economics 21 (2) (2003) 175–185.

[7] A. Charnes, W.W. Cooper, E. Rhodes, Measuring the efficiency of decision makingunits, European Journal of Operational Research 2 (4) (1978) 429–444.

[8] W.E. Chiang, M.H. Tsai, L.S.M. Wang, A DEA evaluation of Taipei hotels, Annals ofTourism Research 31 (3) (2004) 712–715.

[9] L.O. Oyedele, K.W. Tham, Clients' assessment of architects' performance in buildingdelivery process: evidence from Nigeria, Building and Environment 42 (5) (2007)2090–2099.

[10] P.L. Chang, S.N. Hwang, W.Y. Cheng, Using Data Envelopment Analysis to measurethe achievement and change of regional development in Taiwan, Journal ofEnvironmental Management 43 (1995) 49–66.

[11] X. Xue, Q. Shen, Y. Wang, Measuring the productivity of the construction industryin China by using DEA-based Malmquist productivity indices, ASCE Journal ofConstruction Engineering and Management 134 (1) (2008) 64–71.

[12] M. Liu, Y.Y. Ling, Modeling a contractor's markup estimation, ASCE Journal ofConstruction Engineering and Management 131 (4) (2005) 391–399.

[13] M.Y. Cheng, C.H. Ko, Object-oriented evolutionary fuzzy neural inference systemfor construction management, ASCE Journal of Construction Engineering andManagement 129 (4) (2003) 461–469.

[14] J. Doyle, R. Green, Efficiency and cross-efficiency in DEA: derivations, meanings,and uses, Journal of the Operational Research Society, 45 (1994) 567–578.

[15] A.Y. Lewin, J.W. Minton, Determining organizational effectiveness: another look,and an agenda for research, Management Science 32 (5) (1986) 514–538.

[16] J. Kanges, T. Kohonen, Developments and applications of the self-organizing mapand related algorithms, Mathematics and Computers in Simulation 41 (1) (1996)3–12.

[17] J.A. Flanagan, Self-organizing in Kohonen's SOM, Neural Network 9 (7) (1996)1185–1197.

[18] Parasuraman, Zeithaml, Berry, A conceptual model of service quality and itsimplications for future research, Journal of Marketing 49 (3) (1985) 41–50.

[19] Parasuraman, Zeithaml, Berry, Communication and control processes in thedelivery of service quality, Journal of Marketing 52 (1) (1988) 35–48.

[20] B. Golany, Y. Roll, An application procedure for DEA, OMEGA: International Journalof Management Science 17 (3) (1989) 237–250.

[21] S.N. Hwang, T.Y. Chang, Using data envelopment analysis to measure hotelmanagerial efficiency change in Taiwan, Tourism Management 24 (3) (2003)357–369.