WBank Economic Evaluation

8/6/2019 WBank Economic Evaluation

1/144

-I51,5 OoWIc9&*dams EconomicDevelopmentInstitute*; 4,P of The World Bank

The Economic Evaluationof ProjectsPapers rom a Curriculum DevelopmentWorkshiop

Edited byDavid G. Davies

EDI LEARNING RESOURCES SERIES


2/144


3/144

EDI LFARNNG RESOURCES ERIES

The Economic Evaluationof Projects

Papers rom a Curriculum Development WorkshopEdited by

David G. Davies

The World BankWashington, D. C.


4/144

/ 1996The International Bank for Reconstructionand Development / THE WORLD BANK1818 H Street, N.W.Washington, D.C. 20433, U.S.A.All rights reservedManufactured in the United States of AmericaFirst printing July 1996

The Economic Development Institute (EDI) was established by the World Bank in 1955 to train officialsconcemed with development planning, policymaking, investment analysis, and project implementation inmember developing countries. At present the substance of the EDI's work emphasizes macroeconomic andsectoral economic policy analysis. Through a variety of courses, seminars, and workshops, most of which aregiven overseas in cooperation with local institutions, the EDI seeks to sharpen analytical skills used in policyanalysis and to broaden understanding of the experience of individual countries with economic development.Although the EDI's publications are designed to support its training activities, many are of interest to a muchbroader audience. EDI materials, including any findings, interpretations, and conclusions, are entirely those ofthe authors and should not be attributed in any manner to the World Bank, to its affiliated organizations, or tomembers of its Board of Executive Directors or the countries they represent.Because of the informality of this series and to make the publication available with the least possible delay, themanuscript has not been edited as fully as would be the case with a more formal document, and the World Bankaccepts no responsibility for errors. Some sources cited in this book may be informal documents that are notreadily available.The material in this publication is copyrighted. Requests for permission to reproduce portions of it should besent to the Office of the Publisher at the address shown in the copyright notice above. The World Bank encouragesdissemination of its work and will normally give permission promptly and, when the reproduction is fornoncommercial purposes, without asking a fee. Permission to copy portions for classroom use is granted throughthe Copyright Clearance Center Inc., Suite 910, Rosewood Drive, Danvers, Massachusetts 01923, U. S. A.The backlist of publications by the World Bank is shown in the annual Index of Publications, which is availablefrom Distribution Unit, Office of the Publisher, The World Bank, 1818 H Street, N.W., Washington, D.C. 20433,U.S.A., or from Publications, Banque mondiale, 66, avenue d'lena, 75116 Paris, France.At the time of writing, David G. Davies was principal economist in the Studies and Training Design Division ofthe World Bank's Economic Development Institute.

Libraryof CongressCataloging-in-PublicatIonataThe economic evaluation of projects: papers from a curriculumdevelopmentworkshop edited by DavidG. Davies.p. cm.-(EDI learning resources series, ISSN 1020-3842)Includes bibliographical references.ISBN 0-8213-3325-91, Economic development projects-Evaluation-Study and teaching-Developing countries-Congresses. 2. Community developmentpersonnel-Training of-Developing countries-Congresses.1. Davies, David G. II. Series.HC59.72.E44E28 1996338.9'1' 072-dc2O 95-19178CIP


5/144

Contents

Foreword vRenewing Efforts in Project Evaluation: Workshop Summary

DavidG. Davies1. Evaluation and the New Development Agenda 11

RobertPicciotto2. On Training in Project Evaluation 17

Henry Bruton3. Reflections on Social Project Evaluation 23

Arnold Harberger4. Project Evaluation for the Next Decade 51

Arnold Harberger5. Notes on Some Issues in Social Project Evaluation 71

Arnold Harberger6. The Roleof EDI in Project Training for Developing-Country Managers 83

WilliamWard7. The Appraisal of Investment Projects: A Training Approach 97

Glenn P. Jenkins

iii


6/144


7/144


8/144


9/144

Renewing Efforts in ProjectEvaluation:WorkshopSummaryDavid G. DaviesThe papers in this volume were contributions to EDI's Curriculum DevelopmentWorkshop on the Economic Analysis of Projects, held in Washington, D.C. on October 8-9,1992. The substance of the papers and the workshop's proceedings and conclusions havehelped EDI in its curriculum development and publications efforts. They should be ofconsiderable interest to practicing project economists, and are therefore being put into thisform for wider distribution.The workshop was prompted by a need to respond to a demand from World Bank staffand new member countries for training in project analysis. The workshop's objectiveswere to re-examine the content of the curriculum and to review the state of the art ofteaching courses. Important considerations were (a) that EDI teaching and Bank practicebe consistent and contribute to the establishment of standard practices and terminologythat people who make investment decisions in member countries can understand easily,and (b) that EDI re-establish its leadership position in devising effective and efficientpedagogical designs for teaching the subject to professionals.To achieve these objectives, EDI commissioned four papers by Arnold Harberger(ProjectEvaluation or the Next Decadeand Notes on Some Issuesin SocialProjectEvaluation),Glenn Jenkins (The Appraisalof Investment Projects: TrainingApproach), nd William Ward(The Role of EDI in Project Training for Developing-CountryManagers).Because the twopapers by Harberger were written as follow-up and companion pieces to a previous paperby the same author, the earlier paper, Reflections on SocialProject Evaluation (Harberger1985), is reprinted here. The workshop opening remarks by Robert Picciotto, directorgeneral of the Bank's Operations Evaluation Department, and the written comments onthe workshop by Henry Bruton of Williams College were valuable products of theworkshop, and are therefore included in this set of papers.The papers and opening remarks provided a substantial basis for the workshop'sdiscussions and were further enriched by contributions by, among others, John Besant-Jones, Henry Bruton, George Psacharopoulos, Joanne Salop, and Lyn Squire. While therewas considerable debate, which is sunmnarized below, a consensus emerged with respectto the substance of practical project evaluation. These conclusions are as follows:

1


10/144

2 David G. Davies

* The use of economic surplus as the sole measure in formal project evaluationmeans that a great burden is placed on separate analyses of the social andmacroeconomic contexts of the project. Some implications are that- Projects should be identified, evaluated, and considered only if they can beshown to contribute to a strategic plan or strategic vision. This substantially

reduces the amount of resources required for evaluation and links the projectto its larger macroeconomic setting.- Part of a strategic plan or vision involves establishing incentive systems that

will govern the kinds of projects that the private sector will invest in. Theseincentive systems will include fiscal incentives as well as incentives resultingfrom infrastructure investments and regulatory changes. Decisions arerequired about how the project will take these matters into account.

- As investment projects are invariably complemented by organizationalchanges, institutional development, and other activities, assurance is neededthat these complementary activitiestake place and are appropriately timed.- The recurrent costs to the government of the project should be evaluatedindependently in the context of budgetary management and planning.

* Project evaluation should be simple and understandable. This should beaccomplished by the following:- Making growth (or preferably economic surplus) the sole measure used toevaluate projects. Projectswhose objectives are, for example, to increase equityor employment, to reduce poverty, or to improve the environment are thusevaluated against their contributions or costs in terms of growth (or surplus).- Using domestic prices at the domestic price level as the numeraire.- Building the economic evaluation directly from the project's financial analysis,where possible.

* Changes in the relative prices of factors of production and final products requiremuch more attention in project evaluation than they are normally accorded. Priceprojections to get at changes in relative prices require at least as much, if not more,attention than the estimation of conversion factors or shadow prices.* Computer technologies have rendered formal risk analysis accessible to all. MonteCarlo simulations should be an important part of project evaluation, especially inconnection with price projections. In many cases, point estimates of variables cannow be replaced by probability distributions and we can state the probabilities of

accepting a bad project or of rejecting a good one.The PapersThe main points or arguments made in each of the papers are summarized in thefollowing paragraphs.


11/144

RenewingEfforts n ProjectEvaluation 3

Robert Picciotto, Evaluationand the New DevelopmentAgendaPicciotto's opening remarks essentially respond to the papers written by Harberger andJenkins. He was concerned with the narrowness of the analysis being proposed. WhilePicciotto agreed that all project economists should be skilled in economic analyticaltechniques as discussed in the Harberger papers, he believes they should also be welltrained in the use of other analytical and conceptual tools needed to deal with the majorelements in what Picciotto calls the "New Development Agenda." This agenda requiresthat individual projects be examined in the context of overall country and sector policiesand as part of the array of investment projects under consideration. In effect, this meansthat all project operations should be assessed in terms of their policy content, socialimpact, environmental consequences, effect on public finances, and institutionalsoundness.

Arnold Harberger, Reflectionson SocialProjectEvaluationThis is a reprint of an article previously published in the World Bank's Pioneers nDevelopment series. In this article Harberger summarizes his basic approach to projectevaluation. The main feature of this approach is its strict and austere adherence to thethree basic postulates of applied welfare economics: (1) Competitive demand pricemeasures the benefit of each marginal unit to the demander; (2) competitive supply pricemeasures the opportunity cost of each marginal unit from the standpoint of the suppliers(factors of production); and (3) the difference between benefits and costs is a measure ofthe benefits or costs to society as a whole. Harberger specifically rejects the use ofdistributional weights in the analysis, which is advocated in a substantial part of theliterature, largely because weights do not represent the way citizens feel aboutredistributional efforts channeled through the public sector and their use effectivelynegates postulate (3). He advocates, instead, the measurement of "basic needsexternalities."

Arnold Harberger, ProjectEvaluationfor he Next DecadeThis paper reviews the conditions that prevailed during the 1960s that contributed to theflowering of project evalution methodologies. These conditions included the introductionof greatly distorted prices as governments used quotas, and administered prices, licenses,and tariffs to channel or guide the future development of their countries. Hence thepreoccupation of project evaluation with correcting financial prices to their "true"economic or shadow prices. Harberger then makes the point that projections of the marketreal exchange rate over the life of a project are at least as important as estimates of thecurrent distortions in market prices.The article provides a rather detailed, useful outline of the steps and processes ofproject evaluation. This is followed by a set of basic exercises on project timing, dealingwith successor projects, comparing projects with different lives, problems of scale, when toreplace old assets with new ones, separable components, and dealing with risk.


12/144

4 DavidG. Davies

Arnold Harberger, Notes on SomeIssues in SocialProjectEvaluationThis paper is explicitly intended to review and extend the discussions in both of theprevious papers. He covers risk neutrality, portfolio risk and covariance, discount ratesand the shadow price of funds, and projections of changes in relative prices. The lattertopic is of particular interest given the author's admonition in his previous paper to payattention to prospective changes in relative prices over the lives of projects.

William Ward, The Roleof EDI in ProjectTrainingforDeveloping-CountryManagersThe author, who has extensive experience working with EDI and the Bank in general,specifically addresses the question of what EDIshould now be doing in project evaluation.He raises a number of issues that need to be addressed in answering this question,namely: (a) EDI tends to offer single, generic courses that are not suitable for all thevarious roles that course participants play; (b) EDI is not sufficiently concerned with theorganization and procedures of systems that might use project evaluation; (c) EDI hasneglected the technical analysis of projects in favor of financial and economic analyses; (d)EDI pays insufficient attention to important market failures as opposed to governmentfailures; (e) EDI has not integrated project analysis with policy analysis sufficiently. Forexample, an objective of project evaluation is to compensate for price distortions while animporant objective of policy analysis is to eliminate them in the first instance; (f) EDI hasyet to integrate new thinking on the role of government into project evaluation practice;(g) current Bank practice does not provide a model for training in project evaluation; and(h) many of the generic training materials EDI has developed will not satisfy specificnational needs, with the implication that EDI might consider training casewriters.

Glenn P. Jenkins, TheAppraisalof Investment Projects:A TrainingApproachJenkins's paper describes the course offered at Harvard University on project evaluation insome detail. Specifically, the course subscribes to the methodology of project evaluationprescribed by Harberger. The author provides a general discussion of the relationshipbetween financial and economic appraisal and the rationale for his choice of numeraire(domestic prices and domestic price levels). He then goes on to describe an intensiveeight-week course in project evaluation intended to produce professionals who canactually do the job. The course is characterized by a heavy use of staff, computers, andcomputerized case studies. The economics of project evaluation is closely integrated withpractical case work. An important feature of the course is the considerable amount of timespent on risk analysis using Monte Carlo simulations.

Henry Bruton, On Training n ProjectEvaluationBruton's paper is a commentary on the proceedings of the workshop. His comments arediscussed in the following section.


13/144


The Workshop's DiscussionsPicciotto's remarks, which advocated training project economists in a broad range of areasconsistent with the considerations of the "New Development Agenda," inspired the mostimportant debate of the workshop. While Harberger and Jenkins accepted the importanceof the considerations presented by Picciotto, they argued that keeping the basics of projectanalysis as simple as possible and not burdening it with a need to incorporate objectivesother than output into the analysis was extremely important. They believed that projectanalysts should learn to do these basic techniques well and be able to communicate resultseffectively to those who make investment decisions. The techniques of project economicanalysis, they continued, were not a normal part of the tool bag of economists and couldbe acquired only with special training. In any case, the economic analysis of projects is aspecialty that engineers and accountants could learn as well as economists. Moreover,Harberger and Jenkins argued, given the need for a large number of trained projectanalysts, one could not expect all of them to have the broad knowledge and skills neededto make final decisions on projects. Both Harberger and Jenkins regard project evaluatorsas resembling accountants, who systematically go about estimating economic surpluses orlosses on projects unadulterated by other considerations.Thus, one side of the debate argued for broad training based on the wide scope ofknowledge and analytical skills needed to make investment decisions, while the otherargued for giving top priority to the mastery of narrowly defined techniques of projectanalysis to assure that this was done extremely well. Essentially, Harberger and Jenkinsfear that a broad curriculum would be at the expense of project analysis.Implicitly, Harberger and Jenkins assume that there has to be a division of labor.Project analysts do a narrow technical job and provide their results to others who havedifferent skills that they bring to bear on final decisionmaking. Perhaps based onexperience, Bruton, Picciotto, and Squire, by contrast, fear that narrowly trained projectanalysts would not perform well and recommend a broader curriculum. They do notenvisage that the division of labor implied in the Harberger and Jenkins model canactually be realized.Bruton's paper, On Training in Project Evaluation, summarizes the views that heexpressed at the workshop and assesses the workshop proceedings. It also tries to find amiddle ground in the debate about the scope of the curriculum. According to Bruton, thedifferences expressed in the debate ultimately reflect differences about the sources ofeconomic growth and development, which is why he believes that agreement is sodifficult. The narrower curriculum implicitly accepts the notions prevalent in the 1960sthat growth is attributable to capital accumulation, but that attempts at rapidaccumulation lead to distortions in factor prices and the exchange rate. In the belief thatthese distortions lowered the level of investment and affected its allocation, the old schoolspecifically designed the techniques of project analysis to help project decisionmaking byestimating the social opportunity costs of productive factors (their shadow prices), andthereby eliminate the effects that distortions might otherwise have had on decisions.The broader view, as expressed by Bruton, is that capital formation is not the basicsource of productivity growth and that a wide range of other matters should be taken intoaccount when making decisions about investments in projects. According to Bruton, theconcept of a project is actually rather ambiguous when all that is important (as discussedby Picciotto) is taken into account. Moreover, he believes that the techniques of project


14/144

6 DavidG. Davies

analysis are not reliable, for example, while they require projecting the values of allcomponents in a project, their record is dismal.Besant-Jones presented data bearing on this matter. He compared Bank projections of anumber of key prices with the actual prices recorded. The differences were substantialand, what is more, seemed to be inevitable. The main lesson is that even without taking awider range of considerations into account, there will be room for considerabledisagreement on the acceptability of a project.Squire, who at the time of the workshop was the chief economist in the Bank's Europe,Middle East, and North Africa regional office, remarked that in his region, projects wereconsidered for financing only after Bank staff had shown how they would help thecountry achieve its macroeconomic and sector policy objectives and how they would takeinto account the matters discussed by Bruton, Picciotto, and Ward. Squire pointed out,however, that the Bank's current organization did not assure that procedures wereuniform and common standards were applied with respect to project analysis. Salopconfirmed Squire's view.Even given these problems, Bruton nevertheless regards the narrow view of projectanalysis proposed by Harberger and Jenkins as a critical development tool, and views theproject as the heart of the development process. However, he would begin any coursewith a module that puts project evaluation in the context of the country's developmentand its major problems. He would then put students through a Harberger/Jenkins type ofproject analysis. They would estimate net present values "neat," without taking equity,poverty alleviation, and other external economies and diseconomies into account. Theywould perform sensitivity analyses with respect to the critical variables. A decision toaccept or reject the project would be based on the "neat" calculation and the sensitivityanalysis.The next step would be to consider the project in the context of the initial module,which was concerned with the country's overall economy and problems and thegovernment's policies. An answer would be sought to the questions of how the project fitsthe economy and society at the present time, what institutions are relevant to the project,and what the general macroeconomic effects of the project are. A decision should then bemade on the project's acceptability.Workshop ConsensusThe workshop participants unanimously agreed on the appropriate techniques for theeconomic evaluation of projects as spelled out in the three papers by Harberger. Themethodology is particularly valuable not only because of its clarity and meaning, butbecause it provides results in termns that decisionmakers, many of whom are noteconomists, can understand, and because students can master it relatively easily. Theparticipants also agreed that the economic conditions that project analysts should focus onin the 1990s relate to the movement of relative prices of productive factors andcommodities. Errors in price projections are far more likely than distortions to result inmistakes about projects. (The observations by Besant-Jones and Bruton should be noted inthis context.)A number of valuable points that have direct relevance to what should be taughtemerged in the discussion of Harberger's papers. One of these was that the financialanalysis of a project should be given primacy where appropriate. Perhaps the most


15/144

Renewing Efforts in Project Evaluation 7

important points, however, were made in Bruton's paper relating to the need to evaluateprojects in their macroeconomic contexts. Bruton's proposed curriculum seemed torepresent a satisfactory compromise between the opposing arguments on the breadth ofthe curriculum. The papers and arguments by Picciotto and Ward support Bruton'scurriculum, and are themselves upheld by Squire.In terms of pedagogy, the participants generally agreed on the following additionalpoints with respect to teaching the narrower aspects of project evaluation:* The center of learning should be the computer laboratory, not the lecture hall.* Students should be taught to use a computer in the context of using a spreadsheet,preferably LOTUS 1-2-3because of its widespread use.* Students should learn to use Monte Carlo simulation software for risk analysisinstead of RiskMaster 2.0 as used at Harvard.* Some form of recognition for successful completion of the course is useful, if notnecessary, for the success of the teaching program.

A Brief History of Project Evaluation in the Bank and EDIDuring the late 1960s and early 1970s,under the presidency of Robert McNamara, theWorld Bank became the pre-eminent international institution for financing developmentprojects. At the same time, and to support the Bank, EDI became a virtual trade school forproject evaluation. Within its own ranks, the Bank required the use of project evaluationmethodologies through the formal technical specifications that it placed on projectappraisal operations and published in its Operations Manual. Its personnel policies gavepriority to hiring economists with a specialization in microeconomics. To establish themethodology as a basis for investment decisionmaking among the Bank's membercountries, EDI trained large numbers of government officials in project evaluation. Inaddition to improving overall investment decisionmaking by the govermnents of membercountries, the Bank expected that EDI's training efforts would result in improvedcommunications and cooperation among government project planners and Bank staffresponsible for project operations.Project evaluation has been promoted in the Bank and EDI because it requires thesystematic collection of relevant data and information on possible investments. It helpsgovernments to avoid wasting resources at the microeconomic level and to distinguishbetween projects that are clearly beneficial from a social perspective and those whosebenefits are doubtful. It was also expected to improve communications between agovernment's central planners and sectoral or project planners on the one hand, andbetween govemments and international financial institutions on the other hand.A number of publications reflect the contributions of Bank and EDI staff andconsultants to the development of project evaluation methodologies and to training.' Most1.Muchof the literature n the Bankand elsewhere t this time was promptedby reasoning hattoday seems simplistic: doubt was cast on the practical effectiveness of the predominantdevelopment paradigm that made a country's rate of growth dependent on its savings andinvestment ates. If countriescould notsave and investsufficientlyrom their own resources, heymnight orrowthe savingsof other countriesand investthem to enhance heir ownrates of growth,including their savings rates. This reasoning saw investmentprojects as the major sourceofgrowth, and internationallending and aid institutions,such as the World Bank, as the majorsources of other countries' savings.Naturally, planning was required to assure that aid did not


16/144

8 DavidG. Daviesnoteworthy are the works of Tinbergen and Harberger, who both taught EDI courses.Tinbergen was a resident teacher in EDI when he wrote the Design of Development(Tinbergen 1958).For three decades, Harberger has continuously served as an EDI teacherand writer and Bank consultant. His most important written contributions include anessay on applied welfare economics (Harberger 1971) and a discussion of projectevaluation (Harberger 1972).Bank staff members Squire and van der Tak (1975)advancedthe methodology of project evaluation by showing how income distributional weightscould be taken into account. Gittinger (1982) wrote his classic and extensively usedtextbook, EconomicAnalysis of Agricultural Projects,and Helmers (1979) authored ProjectPlanningand IncomeDistributionwhile they were EDI staff members.However, the Bank and EDIdid not sustain the burst of intellectual and training effortson project evaluation in the late 1960sand early 1970s.From the mid-1970sonward, whileremaining primarily a project lending institution, the Bank became preoccupied withrefitting itself intellectually and operationally to deal with problems of macroeconomicand sector management. To effect these changes, both the Bank and EDI underwent majorreorganizations during the mid-1980s.The Bank retired a large number of technical projectstaff and increased the number of macroeconomists. In the process, it lost some of itssubstantial capabilities in project operations. EDI was reorganized and restaffed to handleeconomic policy and management problems, and after 1984project evaluation courses hada low priority and were seldom offered. It was assumed that other institutions would pickup EDI's training efforts.By the early 1990s, the Bank began to suspect that its project operations requiredrenewed attention. This led to two studies, one (World Bank 1991)on the quality of theeconomic analysis used to justify projects (the Joanne Salop Report) and the other (WorldBank 1992)on the overall quality of the Bank's project operations (the Wapenhans Report).The former was prompted by evaluations of Bank operations by its own OperationsEvaluation Department, which found that assumptions justifying projects tended to beoveroptimistic, and by criticism of project evaluation in the Bank by Little and Mirrlees(1990), who found that actual project evaluation departs from the theoretical ideal. Thisstudy found that:

e There is an enormous gap between the existing guidelines and current practice;* The economic evaluation of projects in the Bank tends to be overoptimistic and toneglect downside risks-a tendency it shares with the Bank's macroeconomicforecasts;* Project economic issues have been pigeonholed into a very narrow concern withthe calculation of economic rates of return, as opposed to a more fundamentalconcern with the economics of project design and strategy and their consistencywith the broad macroeconomic and sectoral strategy.

enable governments to invest in projects that wasted resources. This model, known as theHarrod/Domar model,was central o economists' hinkingat the time.The much richerand moreinfluentialdual economymodel of Lewis (1954) nd others implies a Harrod/Domarmodel.Thecapital-outputratio that emerges from this model figured prominentlyin the formulationofdevelopmentplans. A reductionin the ratio was an objectiveof cost-benefit nalysisand projectevaluation.


17/144


The second study, the Wapenhans Report, was more far-reaching. While endorsing thefindings on project economics in the Salop Report, it found that problems in the Bank'sproject portfolio were largely embedded in the Bank's culture. Undue pressure on staff toget projects to the board led to a lack of attention to local participation in projectidentification, design, and, ultimately, in project implementation. Inadequate attentionwas paid to project supervision and to portfolio management at the country level.In addition to these studies, which required a re-examination of project evaluation, anenormous demand emerged for project evaluation training, not only by the Bank's newermembers, such as China and the countries of Eastern Europe and the former U.S.S.R., butby more long-standing members in the Middle East and South Asia.Training programs in project analysis are offered by Harvard's Institute forInternational Development (HIID) and at The University of Bradford in the UnitedKingdom. EDI currently supports training in project analysis and implementation incollaboration with its training partners in the developing world, especially in transitioneconomies.

EDI, however, had reasons to hesitate before embarking on renewed training efforts inproject evaluation. The economic and policy environments of developing countries andthe thinking about them had changed dramatically during the 1980s. The implications ofthese changes for the substance and practice of project evaluation were unclear.Furthermore, surveys and evaluations showed that with some noteworthy exceptions,prior efforts by EDI and others to establish project evaluation in governments throughtraining had not been as successful as anticipated. Indeed, both the Salop and Wapenhansreports revealed problems within the Bank in this respect. What were the reasons for thislack of progress? Finally, what do the new electronic technologies imply for the practiceand teaching of project analysis?

The papers in this volume attempt to come to grips with these questions and to provideguidance in making a renewed effort in project evaluation training.

ReferencesGittinger, J. Price. 1982. EconomicAnalysisof AgriculturalProjects,2nd ed. Baltimore, Maryland: TheJohns Hopkins University Press.Hahn, F. H., and R. C. 0. Mathews. 1964. "The Theory of Economic Growth: A Survey." EconomicJournal74, (December):779-902.Harberger, Amold. 1971."Three Basic Postulates for Applied Welfare Economics:An Essay."Journalof Economic iterature9(3).

1972."Project Evaluation." In Collected apers.London: MacMillan.1985. "Reflections on Project Evaluation." In Pioneers in Development, Second Series.Washington, D.C.: World Bank.

Helmers,F. Leslie C. H. 1979.ProjectPlanningand IncomeDistribution.London: Martinus Nijhoff.Lewis, W. Arthur. 1954."Economic Development with Unlimited Supplies of Labour." ManchesterSchool, May 22,139-91.Little, Ian, and James A. Mirrlees. 1968 Manual of Industrial ProjectAnalysis in DevelopingCountries.Vol. 2, SocialCost BenefitAnalysis.Paris:OECD.

.1974. ProjectAppraisalandPlanningfor DevelopingCountries.London: Heinemann EducationalBooks.


18/144

10 DavidG. Davies

1990. "Project Appraisal and Planning Twenty Years On," Proceedings f the World BankAnnual Conference on Development Economics, Supplement to the World Bank EconomicReview.Squire, Lyn, and Herman van der Tak. 1975.EconomicAnalysis of Projects.Washington, D.C.: WorldBank.Tinbergen, Jan. 1958. TheDesign of Development.Baltimore, Maryland: The Johns Hopkins UniversityPress.World Bank. 1991. "Economic Analysis of Projects, Phase I and Phase 11Reports." Washington, D.C.Working drafts.

. 1992. Effective Implementation:Key to DevelopmentImpact. Washington, D.C.: PortfolioManagernent Task Force.


19/144

1Evaluationand the New DevelopmentAgendaRobertPicciottoHuman welfare is the ultimate objective of development. It is, therefore, appropriate tobase the evaluation of development operations on welfare economics and to have ArnoldC. Harberger, a pioneer of this proud tradition, provide intellectual leadership to thisworkshop. We are also fortunate to have as participants Professors Glenn Jenkins, HenryBruton, and Anthony Bottomley.Equally, I wish to express appreciation to Joanne Salop, Lyn Squire, and the otherBank professionals who have joined the workshop. I look forward to an insightful andspirited debate. I can think of no professional topic of greater priority or relevance thanthe one on our agenda or of a better group to deal with it.The main theme of my remarks is that the new development agenda mandates thisgroup to bring a variety of fresh perspectives to bear on the economic evaluation ofdevelopment operations.In architecture, form follows function-and so it is in the development business. Therole of the project economist, that is, the function of economic evaluation of investmentprojects, has changed since Little-Mirrlees and Squire and van der Tak dug thefoundations of current project evaluation methodologies. The spare elegance of theirmodels is reminiscent of Bauhaus architecture.But we live today in a postmodern environment characterized by diversifieddevelopment objectives, multifaceted operations, and new technologies. The time hascome to reflect these requirements in the architecture of the economic evaluationcurriculum.Accordingly, as this workshop proceeds, it will have to be responsive to today'soperational needs and to the actual dilemmas faced by our developing membercountries.We count on the participants from academia to tap into the most recent advances ofthe social sciences. Regional staff will be called on to clarify the needs implicit in currentcountry assistance strategies. EDI's representatives will have to find ways to make thesecontributions accessible to development professionals within and outside the Bank. Insum, this workshop faces a conceptual as well as a pedagogical challenge.

11


20/144

12 RobertPicciotto

Evaluation and Portfolio QualityFrom an institutional perspective, the stakes are considerable. As a financial institutionand as a development think tank, the Bank has been doing rather well. But as the leadingdevelopment lender, it has not been performing as well as it should have. Enhancedappraisal practices are essential to help improve the Bank's lending record.Since the Operations Evaluation Department (OED) has been tracking the estimatedresults of Bank operations, the failure trend has been increasing. Whereas 15 percent ofcompleted projects approved in the mid-1970s did not perform satisfactorily, more than30 percent of those approved in the mid-1980swere rated unsatisfactory at completion.Nor can we expect an early reversal of the trend: The rate of problematic operationsunder implementation doubled between 1981and 1991.Improved evaluation is equally important for our borrowers. Although it is harderand harder to generalize about developing countries, they have on average done ratherpoorly in managing their domestic and borrowed resources. While average per capitagrowth of gross domestic product (GDP) in developing countries fluctuated within a 2.6percent range from the mid-1960s to the mid-1970s, it has been stuck below 2 percent inmost years since 1986 and even turned negative in 1990 and 1991. Lessons learnedthrough evaluation should help improve the design of economic management.

Need for a New ApproachNever since its introduction as an essential tool of development financing has cost-benefitanalysis been viewed with such indifference by academics and practitioners alike.What lies behind this dismal state of affairs? The development paradigm that ascribeda central role to planning and public investment has been discredited. Accordingly, thefocus of development economics has shifted from investment planning to policy, that is,to redefining the role of the state, adjusting economic management, and reducing publicexpenditures.The consequent demise of national agencies concerned with planning (without acommensurate increase in the professional capacity and influence of evaluation unitswithin finance ministries or audit administrations) has had an unintended consequence:a pervasive neglect of quality assurance with respect to public investment-just at a timewhen accountable and transparent decisionmaking with respect to public expenditureshas emerged as an essential factor of structural reform and effective governance.

In parallel, overall country policy-;ather than the discrete investment project-hasbecome the central organizing principle of development assistance. The shift has beenespecially pronounced in the World Bank, which, over and above its project financingrole, has been called on to exercise intellectual leadership within the developmentcommunity; to set country performance standards for donors in conjunction with theInternational Monetary Fund; and to take a special role in the financing of adjustment.

In this context, it is not surprising that Bank economists have concentrated theircreative energy on macroeconomic and sectoral policy issues.


21/144

Evaluation and the New Development Agenda 13

Measuring PerformanceThe choice of appropriate evaluation methodologies is more than a matter of conceptualelegance. According to Tom Peters, "What gets measured, gets done." And W. EdwardsDeming, the guru of manufacturing quality, has issued dire warnings about theunintended consequences of accounting conventions in the corporate world.Crossing over to development, similar warnings have come from the OED,environmental groups, and grassroots nongovernmental organizations. Their concernshave evoked a deep resonance in the development community. The rapid spread of suchconcepts as sustainability, participation, and capacity building reflects a deep uneasewith the paradigm of traditional investment analysis. Is this unease justified?

Evaluation and the Development AgendaDevelopment is readily defined as equitable and sustainable economic growth. Butenvironmentalists have challenged available measures of growth; equity is far from anunambiguous concept; and sustainability has at least three dimensions: financial,institutional, and environmental. Furthermore, operations are increasingly used asvehicles for broadly based policy reform and capacity building.New objective functions animate development operations. Therefore, the tests neededfor screening programs and projects must reflect these concerns or risk irrelevance. Yetattempts to cram sophisticated adjustments into rate-of-return calculations have provenshort-lived.Development operations are widely expected to serve poverty reduction, privatesector development, and environmental protection through improved economicgovernance and a variety of programs emphasizing human resource development, publicsector reform, efficient infrastructure creation, and effective natural resourcemanagement. We must sharpen the tools of economic evaluation to deal with theseemerging priorities.

Methodological IssuesThus economic evaluation must, first and foremost, assess the effect of the operation oneconomic growth. But given the new development agenda, analysts must assess everyoperation in terms of: its policy reform content; social impact; environmentalconseq+rences; fInancial justification (including its impact on public finances); andinstitutional soundnessOf course, not all projects can be expected to meet all these objectives simultaneously,so tradeoffs may be involved. Are there practical ways of translating a gain in onecategory into a loss for the other? How can economic evaluation deal with multipleobjective functions?A similar dilemma arises at the project design stage. All too often, excessivecomplexity in project design leads to implementation gridlock and to proiect failure. Sothere is a grnwing tension between the ambition of the development agenda (associatedwith a mu!tipicity -f objectives, each championed by one or more stakeholders withinand outside the developing country concerned) and implementability considerations,


22/144

14 Robert Picciotto

that is, the desirability of keeping project designs within the constraints of domesticcapacities. What are the implications for economic evaluation?Country FocusOne implication of this conundrum is that it may not be feasible to examine the efficiencyand responsiveness of a single operation without examining the other components of thedevelopment program. This means that, beyond the individual project, economicevaluation must concentrate on the investment portfolio. In other words, the country hasbecome a "privileged unit of account" in evaluation.

There are, of course, limits to a country-based approach:* Growing interdependence among countries may bring a global dimension to

bear on economic, social, and environmental evaluation.* The recognition of a crucial role for private enterprise has highlighted the firm as

a unit of account.* The increasing sensitivity to social factors may bring a local, if not a household,

or individual, dimension to economic evaluation, namely, in such respects asresettlement and employment effects.

Participatory DevelopmentThe advent of participatory development is an attempt to respond to this new order byadjusting the design process for development operations. According to this increasinglyinfluential conception of the development process, the preoccupation with a singlemeasure of the net present value of a given project is less relevant than the transparentconsideration of alternative schemes by stakeholders.

In such a context of bounded rationality, effective guides to "second-best" solutionscan be more useful than cumbersome optimization models animated by a single objectivefunction.

The expanded menu of objectives included in the development agenda explains whyit has been necessary to diversify the instruments of development assistance beyond theinvestment project. Structural and sectoral adjustment operations, sector investment andfinancial intermediation operations, and a wide variety of institutional development,technical assistance, and advisory services have been added to the standard tool kit of theBank. All of these instruments are subject to evaluation. How can evaluationmethodology deal with each of these instruments?

Sectoral FocusSectoral diversification must also be a consideration. The share of social sector projects isincreasing. The nature of financial sector projects is changing. Privatization, public sectormanagement, and technology development are now frequent. Freestandingenvironmental projects have emerged, and many if not most development operations aresubject to environmental impact assessments. What does cost-benefit analysis have tooffer for the evaluation of each of these projects?


23/144

Evaluation nd the New DevelopmentAgenda 15

Are the rates of return assessed in one sector comparable to those in another sector?Are the methodologies applicable to an irrigation project applicable to a health project?What kind of cost-effectiveness tests are useful for social projects? Is it appropriate to usewillingness to pay as the main determinant of benefits streams in utility projects? Areexternalities suitably taken into account in traditional rate-of-return methodologies, sayfor agriculture settlement projects? How should nonrenewable resources be handled inthe evaluation process?Factoring Out External FactorsThe factoring out of exogenous variables-global and national-is an important issue forproject evaluation. For such investigations to be of practical use, however, they must begeared toward improving the engineering of development programs and projects so thatthey can withstand the intense stresses imposed on them by the macroeconomic and theglobal environment. Any serious review of evaluation methodologies should address thisconcem. In turn, what implications do risk and uncertainty have for project design andeconomic evaluation?A Multidisciplinary VentureIn conclusion, cost-benefit analysis is solidly anchored in theory. It provides a usefuldiscipline for development project design and evaluation. But, to be relevant to the newdevelopment agenda, the framework needs enrichment by the full range of economicinquiry as well as by financial analysis and the other social sciences.

As projects become more policy laden, it will be important to assess the effectivenessof policy reform. As development instruments become more diversified, it will beimperative to trace the linkages between macro policy and microeconomics. As socialand environmental projects become more numerous, resource economics must bebrought to bear.

Improved design of institutions requires evaluaticn of operations from a public choiceperspective. Agency theory and transaction cost ecoromics may have to be tapped for theanalysis of public sector projects. Infrastructure and environmental projects may have tobe analyzed from the perspective of institutional economics.

More often than not, projects work is a struggle with the dilemmas inherent in thefinancing of (and the access to) public goods, a fertile area of recent economic inquiry.

In sum, there is no denying that development practitioners must rediscover therigorous and conceptually elegant standard cost-benefit methodology again and again. Itconstitutes the basics of the profession. It is not a question of abandoning cost-benefitanalysis. Rather, it is a question of applying cost-benefit analysis to cost-benefit analysisand being prepared to use other analytical tools and the insights of other disciplines toensure improved development effect of policy and project interventions.

The world of development has changed. More than ever, its practitioners today mustbe well versed in the basics of cost-benefit analysis. But they must understand itslimitations as well as its potential. They must be equipped with a diversified set ofanalytical and conceptual instruments to deal with a variety of decisionmaking situationsin an agile and credible fashion. To remain relevant, the EDI curriculum should constructthis tool kit. This is the challenge before this workshop.


24/144


25/144

On Training in ProjectEvaluationHenry BrutonThis report summarizes the main themes of the conference and adds various commentson the themes that seem to me relevant to the training programs in the EconomicDevelopment Institute (EDI).The conference began with a strong statement by Robert Picciotto to the effect thatproject evaluation must take place in the context of the "New Development Agenda."This requires that any project be looked at from many perspectives. Cash flows are, ofcourse, important, but so are numerous other aspects of projects. Conceptual elegance isnot the objective; rather, the aim must be to find directly usable tools that catch badprojects at the appraisal stage. The primary objective is the creation of equitable,sustainable growth, and this requires attention to policy reform, the social andenvironmental impacts of a project, and the effect on the institutions of the society. Inparticular, we must focus on poverty alleviation and employment. This is obviously avery wide view of project evaluation itself and of its purpose.Equally strongly, Arnold Harberger has urged a much narrower view. The maintheme here is that we should seek to establish a common, widely used technique thatenables us to effectively address specific, identified things. Primary, if not exclusive,attention focuses on the effect of a project on output. We need to recognize that wecannot put prices on everything, cannot take everything into account, but we can dosome things very well. In taking these actions, we should make clear exactly what wehave done, what our assumptions are, how reasonable our dates are and so forth. Thenwe will be supplying a product that the policymaker can factor into decisionmaking withconfidence, even though all questions are not answered. It was noted that in the old days,the EDI taught a culture-it must now teach how to do things.The difference between these two approaches rests on contrasting ideas about howdevelopment takes place, and indeed about the definition of development. The narrowerview seems to rest on the Arthur Lewis dual-economy model, which in turn had linkswith the growth model of R. F. Harrod. In the Lewis model all investment was to takeplace in the "modern" sector. The physical capital and technology that was to be createdwas the same as that in rich countries. This meant assuming that the capital output ratioin the developing countries was fixed. The Harrod model told us that the rate of growthof output is given by the saving rate over the capital output ratio. With the latter fixed,the higher the saving rate, the higher the rate of growth of output. This in turn meant that

17


26/144

18 Henry Bruton

it was crucial to allocate the new capital in such a way that the maximum growth wouldbe realized with a particular rate of saving. So project evaluation was born, anddevelopment became essentially a matter of getting a high rate of investment and thecorrect allocation of new capital. Factor prices did not matter because technologydominated, and the high rate of output growth would solve poverty and employmentproblems. These objectives would not need any independent attention.In the 196Useconomists realized that the strategy to achieve a high rate of capitalaccumulation had led to a range of policies that severely distorted factor prices and theexchange rate. Such distortion penalized both the level of investment and its allocation.Project evaluation then was directed at correcting these distortions by introducingshadow prices, and the focus was on the estimation of shadow prices. To get privateinvestment allocated correctly, the policy package was simple: eliminate the distortions-get prices right. Development then was achieved during a situation in which all priceswere right and the rate of investment high. Project evaluation, once all prices were right,would be largely concerned with projecting future prices of output, factors, intermediategoods, and foreign exchange. The narrow view of project evaluation remains strategic asa development tool, and the project is the heart of the development process.

The broad view, in contrast, emphasizes "welfare." Development is not simply moreoutput but includes a wide range of other components of welfare that must be taken intoaccount as investment takes place. When matters of distribution, health, environment,poverty, food security, values and culture, and so forth are given weight, the projecttakes on a much more ambiguous form, and evaluation becomes more a matter of theway the project fits into the general policy package than as an independent unit to beanalyzed separately. Evidently this is a much less precise notion than is the narrow view.

This wider view also recognizes that capital formation is not the basic source ofgrowth of output. Rather, one must give equal, if not more, attention to productivitygrowth. The evidence shows that productivity growth accounts for as much or more ofthe growth of output than does capital. It is, however, obvious that unless productivitygrowth continues regularly, the rate of return on capital will fall, and private investmentwill be weak. Thus capital formation and projects no longer are the heart of growth.Despite the vast literature on productivity growth, it is fair to say that economists do notunderstand its real source. In particular, we cannot say which investments lead to regularproductivity growth of the kind needed in developing countries, nor can we say whatprices are "right" to induce the search and learning that produces the productivitygrowth.

Perhaps the primary aspect of this wider approach is that it involves a great deal ofsearching and learning, of trial and error, and of recognizing the huge complexity of evena simple economy.

Much of the discussion at the conference seemed to reflect these different points ofview. I emphasize that the differences seem to rest on various views of developmentbecause it is this fact that makes resolution of the disputes so difficult. I now want toexplore these differences, as they emerged at the conference, in a bit more detail.

The Narrow ViewThe idea of having a rather formal, widely used technique of project evaluation thatyields an answer that is understood is an important objective. Even for those who are


27/144

Training in Project Evaluation 19

willing to accept this narrow view, there are many sources of disagreement. The mostappropriate treatment of risk is open to many questions. There are many doubts aboutthe estimation and use of shadow prices; the choice of the discount rate is bothfundamental and subject to many disputes-including whether it should be positive.There are disputes about what criterion to use in determining which projects to reject oraccept. All these questions arise even if the sole objective is increased output.Fundamental to all investment projects is, of course, the projection of all componentsrelevant to the project. John Besant-Jonespresented data that compared Bank projectionsof a number of key prices with actual prices, and the differences were generally large.Such differences seem to be inevitable. In some cases it seems possible to say that acurrent price is not sustainable and to agree on the direction that it will change. Ingeneral, however, projections are going to be wrong, in many cases drastically wrong.Calculations of expected values do not really resolve this difficulty because expectedvalues are not likely to equal actual values. Indeed, participants at the conference seemedto agree that to project more than five or so years into the future was unwise. Since manypublic investment projects are intended to last well beyond five years, it is not clear howto meet this particular problem. Thus in this narrow view different projections andvarious ideas of how to project are sure to be a source of disagreement.Therefore, it does not seem likely, even in the context of this limited view of projectevaluation, that there will be a widespread acceptance of a particular package. At thesame time there is an important issue here, and I want to come back to it later.The More General View of Project EvaluationThe fact that the narrow view is open to many difficulties does not mean that the moregeneral view is "right," and simple. Obviously it is not, and this larger view has as manyor more problems that must be addressed. To accept the view that matters of povertyrelief, employment, environment, deep social values, and so forth should be explicitlyconsidered creates the need to assign such objectives some sort of measure or weight, andno one knows exactly how to do this even conceptually. Many projects that the Banksupports now appear to have no present values or internal rates of return calculated. Thisseems to be due, in large part, to the fact that many projects are concerned with thoseaspects of the economy and society that simply do not lend themselves to these sorts ofcalculations. One may conclude that on these kinds of projects the Bank has determinedthat it can make decisions on a project's appropriateness without explicit calculations ofconventional measures. If such an assumption is valid for some projects, why is it notvalid for all of them? Perhaps all projects should be treated in this way. This seems to meto be incorrect, but there is an important message here, one which I will explore in amoment.The basic question, of course, is: How does the bank decide on projects for which nonet present value (NFV) or internal rate of return (IRR) s calculated? Lyn Squire made anumber of important points on this issue. Analysts focus on the rationale of the project,of how it fits in with the general strategy of the Bank and of the country. How does itaffect the macroeconomic balance of the country? Does the idea of the project make sensein general, given the state of the country at the moment? Another important question is:Why is the private sector not already doing the project? If these questions all areanswered in a way that suggests the project looks hopeful, then attention shifts to more


28/144

20 Henry Brutonspecific matters, such as discount rates, cost recovery, effect on balance of payments, andsensitivity to exchange and conversion rates. It seemed clear that these latter calculationswere deemed less important than were the more general issues noted previously. Clearly,in this approach the traditional model of project evaluation does not really enter into thestory. As nearly as I could ascertain, the "failure rate" of projects approved in thismanner was no greater than that of those following the more traditional approach.Such an approach puts great emphasis on general knowledge of how the economyworks, of the current state of the economy and the government, of critical bottlenecks inthe country, and so on. This kind of knowledge is very real and difficult to assignnumerical value. Someone noted that it costs one hundred or more times as much toeducate a person at the college level than at the elementary level. This piece ofinformation can be used in an illuminating way in deciding where to put the educationdollar.

ComputersBefore turning to some pedagogical and curriculum issues, I want to say something aboutcomputers. Let me emphasize that I know virtually nothing about computers and amgenerally inclined to the view that the world would be better off without them. We havethem, however, and the question is how to use them in the most productive way. I haveonly two observations. Richard Lucking emphasized that the computer was a teachinginstrument, a way to help students more completely understand exactly what they weredoing during project evaluations. For students unfamiliar with computers, thinking ofthem in this way is difficult and demands careful teaching. Baher El-Hifnawi reportedthat Harvard once had one computer for each student, but then cut back to one computerper two students. The reason was that if the students had their own computers, theywould spend much too much time on them, and not enough time on other aspects of thecourse.Both of these points are surely important, and we must find a way to exploit thecomputer without the courses becoming a course in computers rather than in projectevaluation. I suggest that it is not easy, and it is important.

Some ProposalsGiven this interpretation of the conference,what makes sense for the EDI to try to do as itplaces increasing emphasis on training in project evaluation within the Bank and invarious countries? Following are some general ideas that do not exactly add up to aformal proposal but may contain ingredients of a more complete outline:1. There should be an opening session of, say, one morning, or one day that puts theproject evaluation exercise in the context of the entire development objective. This shouldbe done in a careful, systematic way and should recognize the extent of disagreement inthe profession, the range of objectives, the state of knowledge and ignorance, and somethoughts on how our views have changed since the 1950s. I put great weight on doingthis as objectively as possible.This discussion would identify the main problems of the country at the present time.These might be employment, poverty, inequality, and the environment, but they might


29/144

Training n ProjectEvaluation 21

be something more nebulous, such as entrepreneurial shortages, lack of response of theprivate sector to price or profit incentives, and corruption. These should be specificallyidentified. In this context, it would be possible to talk about the origin of projects, animportant topic not frequently discussed.2. Then the group could work through one form of the narrow version of projectevaluation. The obvious one to use is that of Harberger, the most convincing (in myview) one available. The group could use only output as the objective, that is, nothing onpoverty, environment, or other problems. It should be fairly easy to introduce some kindof sensitivity analysis into this: What happens if the cost of foreign exchange is 5- less or5- more than it is? What happens if the discount rate is moved around a bit?The class should spell out assumptions made at every step and specifically state therationale for making them. Much of the class discussion would center on why theseassumptions, not others, were made. A few hypothetical examples should be used at theoutset, but students should be put in touch with "real" projects early. My view is thatparticipants in such a course should do their own projecting, choose discount rates, andso forth rather than assume that some other office does that. To depend on someone elsefor the projections, lets the project evaluator off the hook too easily-given the problemslisted above. Once an NPV is obtained, the assumptions specified, and the sensitivitywork done, the class will make a decision to accept or reject it.When projects do not lend themselves to this sort of treatment, the class shouldrecognize this explicitly. It may, however, be possible to gather data that are relevant tothe issue of such a project, for example, education costs, as mentioned above. It would beunfortunate, in my view, to teach that it is possible to calculate the NPV of a projectaimed at reducing, for example, infant mortality even though everyone agrees it isadvantageous to reduce infant mortality.3. The class would then study this result and its justification in the context of thematerial covered in the first session. The basic question would be one noted earlier: Howwell does the project fit the economy and society at the present time?(Let me illustrate this latter point with an example from one of Al Harberger'scomments. He mentioned that a group of economists from Cambridge had asked him tosupport their proposal of a $30 billion aid package for Russia. No information wasavailable on what the money was to be used for or how the total was determined. Herefused to lend his support to the proposal. He was surely right to refuse. Then Al said ifthe group had listed some projects, things would be different. For example, we all knowthat telephones are a necessity in a modem society, and a major aid program to providetelephones in Russia could well be accepted. This may be, but I would doubt it. Given thepresent disarray of the Russian economy, I doubt that telephones would break manybottlenecks or would fit in with the economy. I would believe this, even if the NPV of atelephone project was positive.)To do this part of the exercise well requires a lot of intuition. There are people indeveloping countries who have a lot of intuition. We need to find a way to exploit thissource of information. This is difficult. People with good intuition are hard to find, hardto get information from, but they are there. Part of the exercise at this point might includea discussion of who would have information about a project's effect and how to get themto tell what they know. Certainly this source of information should be recognized.After this is done, the class should review its decision to reject or accept. If it changes,a complete statement of why should be spelled out.


30/144

22 Henry Bruton

It may be noted explicitly that this approach does not involve putting weights ornumerical values on the environment, poverty relief, and so forth and then includingthem in the calculation of NPV. Rather, the NPV is calculated "neat," and then thedecision that emerges is studied in a larger, more inclusive context. It is studied no lessrigorously or formally, but in a different way. The class would use data but not in theexplicit way done in the conventional project evaluation form.One might also consider the difficulties arising from the fact that most projects areexpected to last longer than it makes sense to project their prices, costs, and so on. Howcan that be taken into account? I think the main policy implication of this situation is torecognize the advantages of considerable flexibility. The least-cost method may not bethe least cost if it results in an inflexible, unadaptable piece of physical or human capital.Indeed, in an era when technology is changing rapidly, flexibility and adaptability are ahigh-priority objective.4. Participants frequently mentioned "institutions" at the conference. This is a difficultnotion to define. In a course with participants from different countries, it is often effectiveto ask them to identify specific institutions in their country that seem relevant toappropriateness of a project. It is important to try to make the student be very specific.Only then does an emphasis on institutions have a great deal of empirical content andpolicy relevance.5. The last part could be a study of the general macroeconomic effects of the project.6. The final result should be a specific decision to reject or accept. It does not seemuseful for the project evaluator to say to the minister or policymaker, "Here are theeconomic aspects of the project, now you decide." Project evaluators know more thanministers are ever likely to know-and may be more objective as well. They should bewilling to accept the responsibility. It will help make them more careful.PedagogyEveryone seemed to agree that the classes should beformally conducted. Participantsshould have graded tests, graded homework, and get a final grade. It should be madeclear at the outset that it is possible to fail. The basis for this emphasis is that participantsmust be motivated-and motivated strongly-to go all out to learn the material.I agree completely with all this, but I do want to turn it around a bit. I want toconvince the participant that the material is important,vitally important, it really matters.Make this clear on the first day. Then the discussion of how to do the job makes moresense and (I would hope) provides the real motivation. The testing then would bejustified by the need to ensure that the participants understood the material, really andtruly understood it. The tests would not be simply to see who failed and who passed, butto be sure that everyone passed and understood . . . and they understood because theyrecognized that such understanding was of great relevance to their country.


31/144

3Reflectionson Social ProjectEvaluationArnold HarbergerI approach with considerable humility the task of organizing and expressing thesereflections because I do not consider my work in the field of project evaluation to becharacterized by great originality. What I have strived for over the years is betterexpressed by the word "professionalism."This statement applies at two levels. First, there is the level of that largely unsung hostof people serving in budget bureaus, planning authorities, and all types of ministries andagencies (among them the World Bank itself) all over the globe, who strive selflessly tosee to it, insofar as they can, that projects not meeting adequate standards are rejected,while those in the social interest are accepted. These honorable people (for I exclude themany who demean project evaluation by using it as a device for "justifying" whateverprojects their clients or superiors want) must literally number in the thousands. They, inmy view, form a nascent profession. To me, the task of helping them develop thisprofession lies more in distilling the knowledge, wisdom, and common sense that arealready part of our heritage than in extending the frontiers of knowledge in any deepsense. Moreover, I have always thought that in doing so one should respect the goal ofsimplicity as much as possible-seeking principles and rules that can be readilycommunicated to and understood by this heroic (and hopefully growing) band ofpractitioners, most of whom belong to professions other than economics.This brings me to the second level at which the word "professionalism" is relevant.The economics profession has been with us for a long time. The insights and wisdom ithas accumulated over more than two centuries are probably the most important sourcefrom which to derive the principles and rules to guide the new profession of socialproject evaluation. This is the star that I have tried to follow as I have dealt over the yearswith project evaluation: to distill from the huge corpus of economic science the conceptsthat were particularly relevant for the new profession, and to function as a professionalmyself (rather than as a scientist) in studying different areas of application.This same spirit motivates the present paper. In the course of it I will surely have todisagree with other professional colleagues who have dedicated their time and efforts towork in the same vineyard. But even here the spirit of the disagreement is betterdescribed by "Who has found the simplest and most convenient distillation of ourprofession's accumulated tradition?" rather than by "Who has invented what?" or even,on a given point, by "Who is right and who is wrong?"

23


32/144

24 Arnold Harberger

Three Basic Postulates of Applied Welfare EconomicsThe grand tradition of applied welfare economics, going back at least to 1844 and thedays of J. A. Dupuit (1952),can be interpreted as being based on three simple postulates:

1. Competitive demand price measures the benefit of each marginal unit to thedemander.2. Competitive supply price (or marginal cost) measures the opportunity cost ofeach marginal unit from the standpoint of the suppliers (factors of production).

3. In attempting to measure the benefits and costs to a society (or group) as awhole, one must take the difference between benefits ( + ) and costs (-).On the basis of these postulates, we can derive the classical principles of economicpolicy analysis. Among these are (1) the measure (a triangle between demand and supplycurves) of the so-called welfare or efficiency cost of a tax; (2) the traditionaldemonstration of why the exercise of monopoly power is not in the social interest; (3) thecase for the so-called optimum tariff or export tax, which shows how a nation (althoughnot the world as a whole) can benefit from exploiting its monopoly power, if any, overproducts it imports and its monopoly power, if any, over products it exports; (4) thegeneralized expression for the welfare or efficiency cost of taxation in a general-equilibrium setting with many commodities, developed independently by many authorsbut perhaps most elegantly by Hotelling (1983); 5) various rules for optimal taxation, ofwhich the so-called Ramsey rule (1927) s among the most famous; and (6) the so-calledLerner theorem (1936) showing the equivalence of a uniform import tax with a uniformexport tax under conditions of balanced trade.These examples are impressive because of the power of the ideas involved and theirsignificant place in the accepted corpus of economic policy analysis. But truly they areonly the tip of the iceberg. The tradition of what is called consumer surplus analysis hashad many distinguished representatives, among whom Alfred Marshall (1890)and A. C.

Pigou (1932)stand out as giants.The three basic postulates have sometimes been criticized as not yielding in everycircumstance "true" measures of how the utility of individuals changes when somepolicy is introduced or some other disturbance occurs. But as a basis for the actualanalysis of real-world policies and projects, the postulates have not been seriouslychallenged, let alone surpassed.Some authors have worried about cases of multiple equilibria, others about examplesin which the postulates need not always yield the same measure if a sequence of policiesis imposed in a different order (the so-called integrability condition). But I do not knowof even a single case where the presence of multiple equilibria has been identified as afactor in an important real-world policy problem. And as for the integrability condition,my favorite analogy is that we all know that the distance between two cities will varywith the route we take. But most road maps, geography books, and airlines present onlya single number for distance. The logical answer in the case of intercity distance issometimes "as the crow flies," sometimes "the shortest available route." Thecorresponding answer in applied welfare economics (when the sequencing of policysteps is an issue) is to choose the most plausible or likely sequence or, if there is none, to


33/144

Reflections n SocialProjectEvaluation 25

assume that all the policies are imposed together (in technical jargon, to assume "a radialexpansion of the vector of distortions" from the relevant starting point to the end result).From my own standpoint, I have always thought of the three basic postulates asexactly that-simple postulates on which economists have constructed a system ofmeasurement (traditional applied welfare economics). All economists know that nationalincome and gross national product are inaccurate measures of national welfare. Yet moststudies rely on them. In spite of their defects, they have performed reasonably well inmost contexts. What can be said of the three basic postulates is that they are considerablymore subtle and more refined than the rules on which national income accounting isbased-they would not, for example, make the mistake of implying that welfare fallswhen mothers voluntarily leave the labor force to take care of their homes and families.In every case that I know of where the three postulates lead to results different fromthose derived by the rules of national income accounting, the postulates (as in the aboveexample) win hands down. For the real-world problems they have been used to solve,they have proven more adequate than anything else we have. They are thus the naturalsources from which to draw the principles to guide the budding profession of socialproject evaluation.On Distortions and ExternalitiesUsing the three basic postulates makes it easy to understand the sense in which the grandtradition of economics has always looked on an undistorted and fully competitiveeconomy as an optimum. If demand price as seen by demanders in each market is equalto supply price as seen by suppliers, and competition prevails, marginal social benefits asmeasured by postulate 1 will always be equal to marginal social costs as measured bypostulate 2. This happy state no longer prevails when distortions or externalities arepresent.Distortions can take on many forms, of which the simplest to analyze are taxes andsubsidies. When these are present, marginal social benefit as measured by the price paidby demanders differs from marginal social cost. If we represent the excess of demandprice over supply price in an activity by Di and the change in the level of that activity byAXi the introduction of a new policy or project will produce net benefits or net costs,through its effects on other markets, according to whether the expression Xi Di AXi ispositive or negative.I have always thought of Di as standing for "distortion," with the term construedbroadly. That is to say, the above expression is valid generally-not only for taxes andsubsidies but also for the distortions present in more complicated public policies and forthose stemming from monopoly and monopsony situations. Externalities that vary withthe level of an activity, such as traffic congestion with the volume of traffic or smokepollution with the volume of a factory's output, can also be treated as distortions (Di)within the same simple formula.

All this is important because, as will be seen below, these distortions are preciselywhy (within the framework of the three postulates) we have to build a system of socialproject evaluation that is different from the simpler economics of a distortion-free world.In particular, the pervasive presence of important distortions is the main reason it wasnecessary-or at least exceedingly useful-to construct these concepts.


34/144

26 Arnold Harberger

On Social Opportunity Costs in a Market SettingAs pointed out previously the postulates would assert that in the absence of anydistortions in the economy, the social opportunity costs of marketed goods and serviceswould equal their market prices. When new demand for a good or service is generated(say, by a new project), there are only two sources from which to satisfy that demand:increased total supply and the crowding out (displacement) of other demanders.Postulate 2 tells us that the increased supply should be evaluated at the supply price,while postulate 1 tells us that the displaced demand should be evaluated at the demandprice. In the presence of a distortion, say, a tax, these two prices are different and thesocial opportunity cost becomes (at least in simple cases) a weighted average of them.Another way of expressing this is that demand price Pd is equal to supply price Ps plusthe distortion Di. If the social opportunity cost of one unit of the good or service is aweighted average, fi Pd + f2 ps, this can be expressed as (fi+f2) Ps-4f Di. Butf1 issimply the measure of the amount of displaced demand, so the above formula boilsdown to saying that the social opportunity cost of a unit can be expressed in the formPs +Di AXi.The above is the most rudimentary example of a general rule. Other cases are morecomplicated but come down to the same thing. A simple capital market example containstwo taxes, a capital income tax, tc, and a personal income tax, tp. There are thus threerates of return: p, the gross of tax return to investment; i (= p - tc), the market rate ofinterest; and r ( = i - tp), the after-tax return received by savers. In this market, fundsraised by a project come in part from displaced investment, which by postulate 1 has anopportunity cost of p, and in part from newly stimulated saving, which by postulate 2has an opportunity cost of r. The social opportunity cost of capital can then be expressedas fl p + f2r. But this is also equal to f1 (i + tc) + f2(i - tp), so given thatfi +-f2 = 1, it is equal to i +ftc -t2tp-that is, it is a market rate of interest adjusted fordistortions by the Li DiAXi principle.No matter how many sectors we add, the fact remains that it is substantiallyequivalent in such cases to look at social opportunity cost as a measure that is a weightedaverage of the demand prices pd of those sectors whose demand was displaced by theentry of new demand, and of supply prices ps of those sectors whose supply wasstimulated, and as a measure that is a market price (like ps in the first case or i in thesecond) adjusted by a factor based on the application of Li Di Axi. Here AXi representsprecisely the same displacements of demand and stimuli to supply incorporated in theweights of the weighted average. Perhaps the most elegant case is the representation ofthe social opportunity cost of foreign exchange in terms of a weighted average of a hostof individual terms, which reflect the different tariff treatment of many categories ofimports and the various taxes and subsidies applying to different categories of exports.For a dollar's worth (the dollar being the main foreign exchange unit for most smallcountries) of imports of good j the demand price (postulate 1) is Em + T1 , and for adollar's worth of exports of good k the supply price would be Em + Sk. Here Em is themarket exchange rate for the dollar, and Sk the subsidy per dollar of exports of k. If thereare many categories, we have weights offj and fk reflecting the fractions of a dollar newlydemanded (say, by a project) that are brought about through displacing imports of j or bystimulating an increment in exports of k. The end result of this exercise is the followingexpression for the social opportunity cost of foreign exchange:


35/144

Reflections n SocialProjectEvaluation 27

Ijfj(Em + Tj) +Zkfk(Em + Sk.This is a weighted average of the demand prices (postulate 1) of many classes ofimports and of the supply prices (postulate 2) of many classes of exports. But sinceLjfj + Zkfk = 1, the expression above can equally well be written as

Em + XjfjTj +LkfkSkThis takes the form of a market rate of exchange adjusted by the Li Di AXi principle.In all the above cases the weighted average of demand prices for displaced demandand of supply prices for newly stimulated supply (based on a quite intuitive applicationof postulates 1 and 2) is equivalent to the representation of social opportunity cost as amarket price, duly adjusted by the same weighted average of the relevant distortions(based on the Li Di AXi principle, a less direct but more subtle application of postulates 1and 2).This leads us to see that the standard weighted-average measures of socialopportunity cost can be regarded as attempts to follow an approach that (though subtleand correct) asks us to look at the effects of a given action on all the distorted activities ofthe economy, and to assemble the adjustments for distortions into convenient"packages." For the weighted-average measure, the elements in the package are thesupplies and demands in the various component parts of the given broad market. Thus,for the social opportunity cost of foreign exchange, the weighted-average measureincludes the supplies of all the various export categories and the demands for all thevarious import categories, while for the social opportunity cost of capital it wouldinclude the demands for all the various investment categories and the supplies of savingsfrom different sources (presumably with different marginal tax rates, hence differentsupply prices of their savings). Of course, in all these cases, we can lump severalcategories together if they have the same, or closely similar, distortions.But there may be times when the relevant package includes elements above andbeyond those in the weighted average. In deciding on the relevant package, one mustrecall that in determining social opportunity cost we are attempting to trace whathappens when new demand enters a market. Weighted-average methods trace out thoseconsequences just on the constituent parts of that market-displacing imports andstimulating exports, for example. These displacing and stimulating effects work throughthe real exchange rate, and one must recognize that there may be other consequences. Ihave suggested, for example, that if a rise in the real exchange rate ends up displacingsome imports of petroleum, one side effect would be reduced receipts from gasolinetaxes. I have also suggested that if there was a tax on bricks, and if the introduction ofnew demand into the capital market displaces construction, there would similarly be aside effect in the form of reduced receipts of that tax. When these effects are relativelyminor, the principle of simplicity suggests ignoring them. But when they are significant,they should be taken into account. Most important, perhaps, from the standpoint of thepresent paper, is the clear conclusion that weighted-average measures of socialopportunity cost are not in themselves the correct solution. In a market situation they willalways be a component of the correct solution, but they may at times need to besupplemented in important respects. All of this, of course, follows directly from the threebasic postulates that have guided applied welfare economics from its beginnings.


36/144

28 Arnold Harberger

SomeAdditionalObservationsLet me set the stage by pointing out an important characteristic of a well-functioningcapital market or foreign exchange market. Particularly in the latter case, one can almostguarantee that the reaction will be the same: The real exchange rate will go up by thesame amount, and the same set of displacements of imports and stimuli to exports willoccur- regardless of who is the buyer of a given amount of foreign exchange. In the caseof foreign exchange in well-organized markets, the names of the buyers and the purposesfor which they buy are not even known by the market, most purchases being made byfinancial institutions or other intermediaries. The market simply "feels" the pressure ofan added demand, and a set of market-determined reactions ensues.Some people dispute the above proposition by saying that it does not take intoaccount how the foreign exchange will be used, and that when this use is considered, thesocial opportunity cost of foreign exchange will vary from case to case according to theattributes of the use. I believe that this line of reasoning misses the whole point of theconcept of the social opportunity cost of, say, foreign exchange. The main reason forassembling certain XjDiAXi in packages is that these packages turn up with greatfrequency. When this is so, one can calculate the package solution just once and save lotsof time and effort.This, I believe, is what we accomplish when calculating the social opportunity cost offoreign exchange. To my mind, the social opportunity cost of foreign exchange (in amarket setting) has no relation to its use. To put it graphically, if an enterprise orindividual in country A enters the market, buys dollars, and then suffers a fire in whichthe dollars burn up, what is the loss to the country? This loss is measured by the forces ofimport displacement and export stimulus and is the same regardless of who was thebuyer or what intentions he might have had about the use of the now-lost dollars. Here isa perfect example of how social opportunity cost can be separated from use. Of course,one cannot neglect the use of foreign exchange in social project evaluations. But here, bythe very nature of the case, a different pattern of effects (XDiAXi)

WBank Economic Evaluation

Documents

Transcript of WBank Economic Evaluation