Return on Investment In Information Technology - …...Return on Investment In Information...

Return on InvestmentIn Information Technology:

A Guide for Managers

Anthony M. Cresswell

Center for Technology in GovernmentUniversity at Albany, SUNY

187 Wolf Road, Suite 301Albany, NY 12205

Phone: (518) 442-3892Fax: (518) 442-3886

E-mail: [email protected]

August 2004

©2004 Center for Technology in GovernmentThe Center grants permission to reprint this document provided this cover page is included.

CENTER FOR TECHNOLOGY IN GOVERNMENT—RETURN ON INVESTMENT IN INFORMATION TECHNOLOGY: A GUIDE FOR MANAGERS


Acknowledgments

This report was developed on a foundation of previous work of the Center for Technologyin Government and valuable assistance from several staff members. Sharon Dawes,

Theresa Pardo, Donna Canestraro, and Meghan Cook made important contributions to earlyversions. Natalie Helbig, Dubravka Juraga provided excellent suggestions and edits on thefinal version. Mark LaVigne helped with editing as well, and also worked with Paula Hauseron the design of the printed document. The research on costs and returns of investmentin Web site conversion, reported here as a case study, is based on the technical work ofDerek Werthmuller and James Costello, and the assistance of Carrie Schneider andYi-Jung Wu.


Executive Summary 1

Chapter One 5 ROI and the Need for Smart IT Investment Decisions

6 An approach to understanding and using ROI analysis

7 Understanding strategic objectives in ROI design

8 Defining and measuring the costs and returns from IT investments

9 Understanding the enterprise: technology in the business context

12 Understanding ROI decisions in their political and policy context

12 Political risk factors

13 Organizational risk factors

13 Business process risks

13 Technology risk factors

Chapter Two 14 Methods of ROI Analysis for IT

14 Issues of time and scale

14 Importance of business process analysis

15 Benefits of process modeling

16 Methods for understanding the business process

16 Descriptive models

18 Use of analytical or formal models

18 Types of formal modeling approaches

22 Measuring costs and returns

22 Financial metrics and government accounting

23 Measuring costs and cost-effectiveness

24 Efficiency measures

24 Impact measures

25 How time perspectives change the measurements

25 The problem of choosing the project or investment life cycle

25 Risk analysis in the public sector

Appendix A: Case 1 28 Reducing the Cost of Web Site Development and Maintenance

29 Advantages of a dynamic Web site

29 Data sources

30 Cost estimation

31 Benefits

33 Summary of costs and returns

Table of Contents


Appendix B: Case 2 35 ROI for Data Integration in Health and Human Services

35 Project goals and the context of State-Level welfare reform

35 Project rationale

35 Investment in improved welfare administration

36 Project methods

36 ROI framework

37 ROI results for the Welfare Reform Related Technology Fund

38 Potential risks in benefit estimates

Appendix C: Case 3 40 Social ROI

40 The investment philanthropy approach

40 The social return on investment method

43 Measuring value and returns

43 Limitations in the SROI approach

Appendix D 45 Additional Resources

45 Enterprise architecture

45 Stakeholder analysis

45 Business process analysis and modeling

45 Formal modeling

45 Agent based modeling

46 UML modeling

46 Workflow modeling

46 Operations research

Table of Contents


Executive Summary

1

Today more than ever, governmentdecision makers must make the most

of scarce resources and at the same timerespond to ever-increasing demands forimproved performance and new technology.These competing demands generate closescrutiny of proposals for new informationtechnology (IT) investments. What’s more,high profile IT system failures have raisedconcerns about why these investmentsso often fail to live up to expectations. Asa result, many IT investment planningprocesses now require some analysis ofthe costs and returns expected from thatproposed investment. Unfortunately, publicsector managers often lack models that canguide them through such analyses. ThisGuide is offered to help fill that gap.

The Guide provides that help by presentinga practical approach to understandingwhat ROI analysis can and cannot do.A meaningful return on investment (ROI)analysis in information technology is a littlelike saying you want to live a healthierlifestyle. Like lifestyle changes, ROIanalysis is not just a single component.Instead, it is a collection of methods, skills,tools, activities, and ideas. They can becombined and used in many different waysto assess the relative value of an investmentover time. Applying this collection in aparticular situation requires making manychoices among the ideas and methodsavailable and conducting an analysisappropriate to the decision at hand.

Different choices will produce differentresults. Therefore the Guide presents aframework of the key questions that shouldlead to an appropriate ROI analysis. It thenpresents a review of the methods andresources needed along with examplesof different approaches in detailed casestudies.

How extensive should yourROI analysis be?

Once the decision has been made to conductan ROI analysis, what should it look like?

The choice of how to conduct the analysisshould be based on four critical principlespertaining to:

� the strategic objective(s) of the ROIanalysis,

� the place (and importance) of the ITinvestment in the overall enterprisearchitecture,

� the type of analysis that should beconducted (i.e., what data and methodsof analysis are best suited to thoseobjectives), and

� how the ROI analysis fits in the overalldecision context for IT investments.

Understanding the strategicobjectives of your ROI analysis

Understanding the strategic objectives ofan ROI analysis will determine how theanalysis is ultimately done and used. Ahandful of questions like these can helpmanagers decide what the objectives ofthe analysis should be.

� Is the proposed project critical to thebusiness objectives of your agencies orgovernment?

� What are the risk factors associated with theinvestment?

� Who will be impacted—positively ornegatively—by the proposed project?

� Is an ROI necessary for approval andsupport of the proposed project?

� Is the proposed project worth theinvestment of an ROI analysis? And ifso, how detailed should it be?

CENTER FOR TECHNOLOGY IN GOVERNMENT—RETURN ON INVESTMENT IN INFORMATION TECHNOLOGY: A GUIDE FOR MANAGERS2

Answering these questions will help identifythe resources needed to conduct an ROIanalysis, which in some cases can itself bea substantial investment. Extending its levelof detail beyond what is needed for effectivedecision making is a waste of resources.Focusing on strategic objectives keepsattention on the full range of benefits to beexpected from the investment, and how tomeasure them.

Understanding the contextof your IT investment

Any IT investment project is embedded in anorganization’s technology infrastructure(enterprise architecture), relevant businessprocesses, organizational environment, andexternal relationships.

� Technology infrastructure. There aredirect costs associated with the technologyand services in which you invest, andthere will also be costs in terms of itsimpact on other technology systemsalready in place. The benefits range frommore efficient automation and workflow toimproved collection, storage, and accessto information.

� Business processes. An ROI analysismust not only account for the improve-ments to relevant business processes,but also for the costs associated withtraining staff involved in using theproposed technology system.

� Organizational environment. Othercosts and returns will be linked to theorganization, for example through alteredresource flows, performance changes,changes in work flows and internalrelationships.

� External relationships. Linkages with theexternal environment may be significantas well. Resources may be committedfrom this environment to support theproject and additional costs maybe imposed on external persons ororganizations by changes in the wayservices are delivered or other businessis conducted.

Choosing the right type ofanalysis

Choosing and using the various methods ofROI analysis requires sound knowledge andjudgment: knowledge about the methodsand judgment about how best to applythem. The methods chosen should fit theparticular questions asked of an ROIanalysis. Different questions requiredifferent measurement approaches to fitthem. In general, there are four types ofquestions that prompt or drive an ROIanalysis: financial, effectiveness, efficiency,and impact.

Financial: Can we afford this? Will it payfor itself?

An ROI analysis that answers thesequestions is based on expected savings andrevenue increases compared to the dollarcost of all expenditures on the new system.The measures are set by generally acceptedor legally mandated accounting standardsand practices that apply to the particulargovernment organization. The costs andsavings or revenue might be projected overa multi-year time span to show a paybackperiod or to estimate the present value offuture returns.

Effectiveness: How much “bang for thebuck” will we get out of this project?

The ROI analysis that will answer thesetypes of questions considers how much theinvestment contributes to achieving programgoals and producing the desired results. Itconsiders direct, indirect, and opportunitycosts. The indirect costs include such thingsas training and administration over time. Anopportunity cost could be the loss of returnor revenue you would have received hadyou chosen a different alternative. Themeasurement of returns will be expandedbeyond cost savings to include levels ofperformance relative to program or projectgoals.


When deciding how to prepare and presentan ROI analysis, therefore, it is best to takeinto account all the potential risks thatinfluence the decision process. Undertakingan ROI analysis should include attention tothe risk factors identified below.

Risk factors that can impactinvestment decision process

Politics and policy factors

� Public exposure to failure� Divided authority over decisions� Multiple stakeholders� Year-to-year budget cycles� Highly regulated procurement

processes

Organizational factors

� Complex program networks� Misalignment of (or conflicting)

internal goals� Lack of leadership support

Business process factors

� Impact on existing process� Fear of changing work assignments

Technology factors

� Rapid changes in technology� Interacting with parallel systems� Scale and complexity

Most of the risk assessment issues listedhere involve problems related to thinkingbeyond the boundaries of the project,measuring factors, or determiningprobabilities. Simply recognizing whereuncertainty and potential damage may lieis half the battle. Careful risk analysis,based on the best available data andestimates, will surely assist in ROI analysisand improve planning, even if the amountor quality of data is less than ideal.

Considering these various risk factorscan help shape the style, emphasis orpresentation strategies employed tointroduce the analysis into the decision

3

Efficiency: Is this the most we can get forthis much investment?

The ROI that tackles this question requiresinformation about the greatest possiblevalue relative to its costs. Efficiency cannotbe separated from effectiveness. It is usuallyexpressed in terms of optimizing thevalue of a return for a given cost or input.Establishing that some particular result isthe best of all possible results requireseither examining many alternatives orsimulating performance in some way thatprovides a valid picture of what is possible.This can be done for some (but not all) kindsof systems with sufficient resources anddata. However to do so can substantiallyincrease the cost and complexity of theanalysis.

Impact: Will the benefits to society (ourstate, our city, etc.) justify the overallinvestment in this project?

The analysis that answers impact questionswill be concerned with the larger social andeconomic benefits and costs of a project. Todefine and measure variables that representsocial costs or benefits requires morethan the typical economic or accountingframeworks. These measures are based oneither the specific program results desiredby an agency or on general social benefitsand improved quality of life. Though notimpossible, the breadth and complexity ofthis kind of ROI analysis is rarely found inIT investment planning.

How does the ROI analysisfit into the overall decisioncontext for IT investments?

Investment decisions in the public sector,whether they involve IT or not, necessarilytake place in a context of political andpolicy influences. No matter how solidor technically sophisticated an ROIanalysis may be, it will not likely be thesole determinant of an investment decision.


making process. Such considerations asthose listed here may also help in recruitingsupport for the conclusions of your ROIanalysis and guiding how the analysisprocess is positioned when seeking thatsupport.

Conclusion

There is no single “right” way to conduct areturn on investment analysis. Nor is there aConsumer Reports for ROI products andservices. In determining how to conduct youranalysis, the best advice is to focus on thestrategic objectives of the analysis alongwith the goals and business processes ofthe proposed project. This focus will helpguide decisions about the resources andmethods to use to conduct a sound andvaluable ROI analysis.


Chapter One: ROI and the Needfor Smart IT Investment Decisions

5

“It [ROI] will let us begin to make assessments and decisions about funding aproject or developing a new service based upon some true data. That moves you

from having emotional debates about projects to having factual discussions.”Gerry Wethington, Missouri CIO1

Government decision makers must makethe most of scarce resources and at

the same time respond to ever-increasingdemands for improved performance andnew technology. Thus the need for wiseinvestment in information technologycontinues to grow. Growing demand in theface of scarce resources generates hardquestions and close scrutiny of proposalsfor new investments. What’s more, thedismal failure record of many governmentIT investments raises legitimate concernsabout the value of these investmentsand why they so often fail to live up toexpectations, or even to work at all. As aresult, IT planning processes often include,or even require, a rigorous business caseto justify new IT investments. These includeways of assessing the costs and returns tobe expected from that investment, that is,return on investment (ROI) analysis. ThisGuide is designed to help governmentexecutives who need to design, direct,conduct, or work with the results of suchan analysis to make the most of theirinvestments in information technology.

Growing interest in assessing returns on ITinvestments has spawned wide interest inmethods of return on investment analysis.However, there is little agreement aboutbest practices or specific methods forassessing ROI. Professional publicationsand consultant white papers present quitea variety of possible approaches. As aresult, government executives and decisionmakers have difficulty choosing or designinga return on investment analysis that is bothfeasible and appropriate to their needs. Tohelp administrators and decision makers

with these choices this guide presents anoverview of the purposes and concepts ofROI along with an introduction to basicmethods and example cases. It alsoincludes links to other resources for thosewho wish to explore some subjects ingreater depth.

This guide treats ROI analysis as part ofthe overall decision making process for ITinvestment. The planners and designers ofan IT project can use ROI analysis to helppersuade decision makers to support theproject. Decision makers can use an ROIanalysis, indeed may even require one,as part of an IT investment proposal to aidthem in evaluating it. In either case, an ROIanalysis will be shaped by the situation inwhich it is designed and carried out.

Decisions about what sort of return oninvestment analysis to do, or whether todo one at all, will usually depend on avariety of factors. ROI analysis may or maynot fit the larger context of investmentdecision making. Decision situationsdriven by very short deadlines or highlyspecific policy directives may rule outextensive analysis. Scale matters as well.An elaborate ROI analysis would hardly bejustified for a small-scale, low-risk projectthat requires a fast decision. By contrast,large complex projects are typically high-risk propositions for which the added timeand cost of an extensive ROI analysis wouldbe fully justified. Even though justified, insome environments ROI analysis is notused at all in favor of best practice reviewsor benchmarking to evaluate investmentpossibilities. Current practices vary

1 “Running the Numbers,” Government Technology, June 2002, p. 23.


considerably. In Iowa, for example, thestate government has a standard frameworkfor all agencies submitting IT projectproposals, including business case andROI analysis requirements.2 The US Officeof Management and Budget has imposedsimilar requirements on federal agencies.Agencies and local governments in otherstates may develop their own internalbusiness case and ROI requirements.

Given the diversity of practice in the ITinvestment world, this guide takes aneclectic, non-prescriptive approach. Ittreats the subject of ROI analysis from thepoint of view of a curious but uncertaindecision maker. The key issues facing thisdecision maker are whether to do an ROIanalysis and, if so, how. The guide doesnot advocate engaging in ROI analysisunder all circumstances, nor does it favorany particular technique. Instead it presentsan approach to understanding a range ofpurposes and methods for ROI analysisthat can assist that decision maker tomove forward with wise and effective ITinvestment choices.

An approach tounderstanding andusing ROI analysis

Decisions about how to use ROI analysisdepend on understanding the nature ofthe methods themselves and how theyrelate to the business setting. ROI analysis

in general is a rather diverse collection ofmethods, skills, tools, activities, and ideas.They all may be useful for assessing therelative value over time of some investment.These methods are not, however, a singleformula or predetermined calculation thatwill yield a simple yes-or-no answer tothe question of how to invest. ROI is nota silver bullet. Actually designing andcarrying out any kind of ROI analysisrequires making many choices amongthe ideas and methods available andconducting an analysis appropriate to thedecision situation. Different choices willproduce different results.

Consequently, a meaningful analysisof returns on investment in informationtechnology is far easier said than done.Choices about how to conduct an ROIanalysis should be based on criticalunderstandings about:

� the strategic objective(s) of theanalysis,

� the place of the proposed ITinvestment in the overall enterprise3,

� exactly how the analysis should bedone (i.e., what data and methodsof analysis are best suited to thoseobjectives), and

� how the ROI analysis fits in the overalldecision context for IT investments.

This guide will introduce you to thesefour basic understandings and provideresources for deeper investigation of each.

2 The Iowa ROI approach can be found at http://www2.info.state.ia.us/roi/index.html. The Federal business case requirements are found in OMBCircular No. A-11.3 The enterprise can be a single agency or unit, or something as broad as the education or justice enterprise. The U.S. Chief Information OfficersCouncil defines an enterprise in terms of enterprise architectures, or “blueprints” for systematically and completely defining organizations’ current(baseline) or desired (target) environments. See A Practical Guide to Federal Enterprise Architecture, Version 1.1, Chief Information OfficersCouncil, February 2001, p. 2. More specifically, the National Association of State Chief Information Officers defines enterprise architecture as anoverall plan for designing, implementing and maintaining the infrastructure to support the enterprise’s business functions and underlying networksand systems. See Enterprise Architecture Development Tool-Kit, Version 2.0, National Association of State Chief Information Officers, July 2002,p. 243.

CENTER FOR TECHNOLOGY IN GOVERNMENT—RETURN ON INVESTMENT IN INFORMATION TECHNOLOGY: A GUIDE FOR MANAGERS 7

Understanding strategicobjectives in ROI design

Your understanding of the strategicobjectives of an ROI analysis will determinehow the analysis is ultimately done andused. The matter of strategic objectives hastwo related parts. One deals with theobjectives and context of the proposedinvestment. The second deals with theobjectives and context of the ROI work itself.An adequate understanding of the first partof the strategic objective must includeanswers to these key questions detailedbelow.

What are the programmatic and businessgoals of the proposed investment?

The value of an investment is directlyrelated to the programmatic goals andbusiness process employing the newtechnology. For example, the strategic goalof investing in a new financial managementsystem would be to improve the qualityof financial decision making and control ofresource flows, not simply to produce fasteror more complex accounting reports.Attention to the strategic objectives keepsattention focused broadly on the kinds ofbenefits sought and how to measure them,rather than narrowly on the technology.Focus on the strategic and businessobjectives will also lead to a clearerunderstanding of the full range of benefitsto be expected from the investment.

What are the needs or interests of theprimary customers and other keystakeholders with respect to productsor services involved and impacts onbusiness processes?

Stakeholders are critical players in thesuccess of new projects. Ignoring orunderestimating the importance of theirrole puts the success of an investmentat great risk and can lead to substantialunanticipated costs or missed benefits.One large organization recently developedan extensive new customer help deskapplication without the participation of the

staff who handled customer phone calls.Those staff members learned of the newsystem the day before it went live. The resultwas poor performance and a serious blow tostaff morale—lower returns and highercosts.

How extensive is the analysis to be? Whatrange of costs and returns are expected?What resources and frameworks areavailable or required for the ROI analysisitself?

An ROI analysis can itself be a substantialinvestment. Extending its scope, levelof detail, or complexity beyond what isneeded for effective decision making isa waste of resources. Attempting an ROIanalysis that is beyond the resources orcapacity of the organization will also wasteresources. Choosing an appropriate scopefor the analysis is therefore a critical partof the process. These choices determinethe details of the ROI analysis itself: kindsof data used, whether hard or soft estimateswill suffice, the kinds of projections,quantitative or qualitative data that areneeded; which financial or non-financialoutcomes (customer/user satisfaction,social or political outcomes, improvedequity) are all important. The answers tothese questions then influence the kindsof personnel, tools, and other resourcesneeded.

What are the risk factors and howmight they affect the project’s costsand results?

Consideration of risk should always bea part of the ROI analysis. Risk factorsarise from the nature of the projectitself (complexity, scale, novelty), itsorganizational setting (conflict, resourceconstraints, top-level support, timepressures), and the larger environment(political turbulence, crises, and policyshifts). Risk analysis, discussed in somedetail later in this chapter, examines thelikelihood of risk factors affecting theproject and what elements of the projector its results are likely to be affected.


4 Tom Pisello, in Infoworld, June 10, 2002, p. 47.

Defining and measuringthe costs and returns fromIT investments

If an ROI analysis were just one simplething, then there would be one simple wayto measure the costs, returns, and benefits.In practice, however, there can be manydifferent questions asked of an ROIanalysis requiring different measurementapproaches to fit those questions. Howto identify these measures and apply themto analysis are the next parts of the puzzle.

Solving that puzzle requires understandingthe differences among the questions askedof an ROI analysis. There are four differentbut related types: financial, effectiveness,efficiency, and impact questions.

Can we afford this? and Will it pay foritself?

Answering these questions requiresinformation about costs and returns in termsof the monetary value of the resources used(inputs), as measured and recorded bystandard financial factors. In its simplestform, an ROI analysis based on this kindof question would calculate the return interms of the expected savings and revenueincreases (if any) compared to the dollarcost of all expenditures on the new system.The costs, savings, and revenues might beprojected over a multi-year time span inorder to show a payback period or toestimate the present value of future returns.

8

IT OPERATING EFFICIENCYHow will the project improve IT, suchas simplifying management, reducingsupport costs, boosting security, orincreasing IT productivity?

RISKWhat are the potential risks associatedwith the project? How likely will risksimpact the implementation schedule,proposed spending, or derived targetbenefits?

COMPETITIVE IMPACTHow does the proposed project comparewith competitor’s spending plans?

ACCOUNTABILITYHow will we know when the project isa success? How will the success bemeasured (metrics and time frames)?

REQUIRED INVESTMENTHow much investment–including capitalexpense, planning and deployment,application development, and ongoingmanagement and support–will the projectrequire?

FINANCIAL BENEFITSWhat are the expected financial benefitsof the project, measured according toestablished financial metrics, includingROI, … savings, and payback period?

STRATEGIC ADVANTAGEWhat are the project’s specific businessbenefits, such as operational savings,increased availability, increased …revenue, or achievement of specific …goals?

Table 1. IT ROI Questions from the CFO to the CTO4


How much ‘bang for the buck’ will we getout of this project?

Answering effectiveness questions requiresinformation about the costs of the project inrelation to how much it contributes toachieving program goals and producingthe desired results. The metrics will bemore complex, involving unit cost or activitycost calculations. The measurement ofreturns will be expanded beyond costsavings or revenue increases to includelevels of performance relative to programor project goals.

Is this the most I can get for this muchinvestment?

Answering efficiency questions requiresinformation about whether the project willproduce the greatest possible value relativeto its costs. Efficiency questions poseserious analysis problems. Establishingthat a particular result is the best ofall possible results requires eitherexamining many alternatives or simulatingperformance in some way that gives avalid picture of what is possible. This canbe done for some kinds of systems withsufficient resources and data, but cansubstantially increase the cost andcomplexity of the analysis.

Will the benefits to society (our state orour city) justify the overall investment inthis project?

Answering impact questions requiresinformation about the larger social andeconomic benefits and costs of a project.These questions pose two tough problems.Measuring the broad social and economiccosts of an investment requires data farbeyond what typical financial systemsprovide. Measuring, or even identifyingthe full range of social and economic

benefits from some government IT projectcan be even more daunting. The idea offiguring a cost/benefit ratio is appealing,but seldom feasible. Though not impossible,the breadth and complexity of this kind ofanalysis means it is rarely found in ITinvestment planning.

The four types of measurement questionsand approaches differ in several ways.The choice of relevant metrics is one criticaldifference. Some approaches are based onstrictly financial metrics (costs or returns indollar terms), others include productionoutput metrics, such as quality of goodsor services, client or customer satisfaction.Metrics may extend to organizational factorssuch as morale or to social and politicaloutcomes. These can include impacts onquality of life, social equity, social or humancapital, or political support. Thesedifferences are summarized in Table 2(page 11).

Understanding theenterprise: technology inthe business context

This guide views any IT investment projectas embedded in its organization’senterprise architecture and in a contextwith three major elements: relevantbusiness processes, the organizationalsetting, and the external environment.5

These are illustrated in Figure 1 on page 10.The immediate context of any IT project isfound in the current business process(es).Some of the costs and returns of the projectwill be directly linked to business processes,such as training costs for staff involved orthe improved efficiency of the overallprocess resulting from project implementa-tion. Other costs and returns will be linked tothe organization, for example through

9

5 While a more detailed discussion of enterprise architecture is beyond the scope of this guide, both the mapping and understanding of anorganization’s “blueprint” are critical steps that any organization should accomplish prior to conducting effective return on investment analysis.Please see the Additional Resources section in Appendix D for several links to enterprise architecture development tools available to federal,state, and local government decision makers and ongoing enterprise architecture initiatives at the state level.


changes in resource flows, performancechanges and changes in workflow andinternal relationships. Linkages with theexternal environment may be significantas well. Resources may be committed fromthat environment to support the project,and additional costs may be imposed onexternal persons or organizations bychanges in the way services are deliveredor other business is conducted.

For example, an agency implementing EDI(electronic data interchange) to supportpurchasing transactions may be imposingcosts on vendors who wish to do businesswith the agency and have to invest indeveloping their own EDI capabilities. Theway Figure 1 below represents investmentcosts and returns as part of the samecontext is an important way of looking atROI. Neither the costs nor the benefits of anIT project begin and end at the project’sboundaries. Financial allocations to a newproject mean fewer resources somewhereelse in the organization or its environment.

10

Figure 1. The Context of IT Investment Projects

Changes in one part of a business processmay impose costs on other parts that haveto adjust activity, retrain staff, or modify othersystems. Increased efficiencies in onebusiness process can make resourcesavailable elsewhere in the organization,but may also result in changes in otherlinked processes within the enterprise.

Whether you see something as a cost orbenefit of an IT project may depend onwhose perspective you take. The expendi-ture on a new personnel managementapplication is a cost to the agency that paysfor it, but it is a benefit to the vendor, who isan external stakeholder. These links make itclear that an analysis of costs and benefitsof an investment can require attentionextending well outside the project itselfinto the organization and its environment.Business processes are the criticalconnection between the project and the restof the organization. So attention to businessprocess linkages with the project is animportant part of the overall ROI analysis.


MeasurementQuestion

Can we afford thisand will it pay foritself?

How much ‘bang forthe buck’ will we getout of this project?

Is this the most I canget for this muchinvestment?

Will the benefits tosociety (our state,our city, etc.) justifythe overall invest-ment in this project?

MeasuringCosts

Financial metrics; defined bypolicy and accepted accountingprinciples ; reporting andcontrol-oriented; standards-based or consistent; not linkedto business process; ignoresimportant cost factors; shorttime frame; data routinelycollected/repor ted

Financial and outcome/qualitymetrics; operations andmanagement oriented; definedby program and businessprocess; may or may not bestandardized; often requiresnew data collection; mayinclude organizational andmanagerial factors

Financial and organizationalmetrics; management andpolicy oriented; non-standard-ized; requires new datacollection and simulation oranalytical model; can revealhidden costs

Financial, organizational,social, individual metrics;individual and managementoriented; nonstandard; requiresnew data collection andexpanded methods; revealshidden costs; potentially longtime-frame

MeasuringReturns/Benefits

Savings as measured inaccounting categories; narrowin focus and impact; increasedrevenues, reduced total costs,acceptable payback period

Possible efficiency increases;increased output; enhancedservice/product quality;enhanced access and equity;increased customer/clientsatisfaction; increasedorganizational capability;spillovers to other programsor processes

Efficiency increases;spillovers; enhancedcapabilities; avoidance ofwasteful or suboptimalstrategies

Enhanced capabilities andopportunities; avoidingunintended consequences;enhanced equity; improvedquality of life; enhancedequity; enhanced politicalsupport

Table 2. Approaches to Cost and Return Measurement


Understanding ROIdecisions in their politicaland policy context

Investment decisions in the public sector,whether they involve IT or any otherexpenditure, take place in a context ofpolitical and policy influences. No matterhow solid or technically sophisticated anROI analysis may be, it will not likely be thesole determinant of an investment decision.It is useful in deciding how to prepare andpresent an ROI analysis, therefore, totake into account possible political andorganizational factors. Such a considerationof external factors may help shape the style,emphasis, or presentation strategiesemployed to introduce ROI analysis intodecision processes. Such considerationsare discussed below and may also help inrecruiting support for the conclusions of theanalysis and guiding how the analysisprocess is described or defended.6 All ofthese considerations can be classified asdifferent types of risks.

Political risk factors

Public exposure of failure or error.Government’s business is public business.Most new ideas are implemented in fullpublic view. Any investment-gone-wrongrisks not only dollars, but the credibilityof an agency and its leadership withlegislators, executive officials, and thepublic. So government tends to reduce risksby relying on the “tried and true.” Failurerisk can be mitigated by taking care not toover promise the benefits of new projectsand to ensure that there is adequatestrategic planning to reduce the probabilityof failure. Undue caution can also riska different kind of failure: missedopportunities for successful projects.

6 The content of this section is adapted from Sharon S. Dawes, Theresa A. Pardo, Stephanie Simon, Anthony M. Cresswell, Mark F. LaVigne,David F. Andersen, and Peter A. Bloniarz. Making Smart IT Choices: Understanding Value and Risk in Government IT Investments. Albany, NY:Center for Technology in Government, 2003. http://www.ctg.albany.edu/publications/guides/smartit2

Divided authority. Public executivesseldom have a clear line of authority overagency operations. Their decisions arecircumscribed by existing laws, budget andfinancial controls, civil service systems,political constraints, and a variety ofregulations imposed by both legislaturesand the courts. These restrictions impedemanaging the complexities of multi-milliondollar IT projects in a rapidly changingtechnical environment.

Multiple stakeholders. Stakeholderstypically have competing goals. Customers,constituents, vendors, service providers,elected officials, professional staff, andothers all have some stake in IT projects.Understanding how different choices mayaffect and be affected by each stakeholdergroup helps to prevent unexpectedproblems.

Annual budgets. Government budget cyclesincrease the uncertainty about the sizeand availability of future resources. Thisdiminishes government agencies’ abilitiesto adopt or sustain new IT innovationssuccessfully, especially those that have longdevelopment periods.

Highly regulated procurement. Regulationsin the typical competitive bidding processare ill suited for the experimentation andlearning that is often essential for successfulIT investments. While promoting integrityand fairness, procurement regulations areoften a source of problems and delays.These are especially troublesome whenagencies write requests for proposals(RFP’s) that depend on the limitedinformation they have been able to gainfrom inadequate experience and research.


Organizational risk factors

Complex program networks. Governmentprograms are connected in many complexways to other programs in the same or otheragencies, or to non-governmental entities.Sometimes the connections are explicitand formal. Often they are informal orunintended. Changing programs can haveunintended consequences for severalothers, producing additional costs andproblems for the investment project.

Misalignment of goals. Some parts ofan organization may see the goal of an ITproject in narrower, possibly conflictingterms. For example, an IT unit may becomeenamored with a database or officeautomation project because of its technicalelegance. The end users of the system,however, may want capabilities that are notcompatible with the new technology. Withoutsome goal alignment a project is on the pathto failure.

Lack of leadership support andorganizational acceptance. Topmanagement support for a technologyinitiative is critically important. Similarlysupport and acceptance throughout theorganization, especially among the peoplewho will use the technology or its products,are equally important, and often moredifficult to achieve. Understanding andenhancing support reduces or limits risks.

Business process risks

Reality of the process. IT inventions areembedded in business processes. Failureto understand and account for this realityin project design and implementationintroduces major risks. Systems are oftencreated that do not serve business needs,are too expensive for the small productivitygains they provide, or are not flexibleenough to meet changing demands.

Technology risk factors

Rapid change. Obsolescence is a risk assoon as a project chooses its technology.The pace of technological change makesit difficult for planners to keep up with thedetails of new developments and tounderstand comprehensively how eachnew technical tool works, or may besuperceded. Technology choices guidedby long term strategies and stronglinkages to business goals can mitigatemany of the risks produced by rapidlychanging technology.

Technology interactions. A basic conceptof system analysis is “you can’t do just onething.” New technologies interact with oldtechnologies and work processes. Theinteractions may enhance the value of theolder and newer technology, or interfere withboth. A careful analysis of these interactionswill identify risky situations and provideinsights for avoiding problems and errors.

Scale and complexity. Risks increasedirectly with the scale and complexity of ITprojects. Planning an incremental processof development, with careful contingencyplans, can mitigate some of these risksand will avoid problems that cannot beanticipated on the path from small to large,complex systems.


Chapter Two: Methods of ROIAnalysis for IT

Choosing and using the various methods of ROI analysis requires both goodknowledge and good judgment: knowledgeabout the methods and judgment abouthow best to apply them to a particular ITproject and its setting. This section presentsan introduction to the methods. It is intendedto impart enough knowledge so that thereader can exercise effective judgmentabout whether to use a particular method,how and to what extent it may be applied,and where to go for more completeinformation and resources.

We begin with a discussion of the issuesassociated with project time frames andscale, then present various approaches toprocess analysis and measurement. Weconclude with a brief overview of riskanalysis.

Issues of time and scale

The selection of method will depend tosome degree on the time frame andthe scale of the project and the ROIanalysis. Time frame refers to both thetime perspective for the analysis and theoverall time period of the project to beincluded in the analysis framework.

Anticipating the future. ROI analysis iscommonly used prospectively. The resultsof the analysis are intended to inform adecision about a future IT investment.For a prospective analysis, estimates ofanticipated cost and performance are basedon assumptions about the future that mayinvolve considerable uncertainty. Theresults of the analysis will depend, to alarge degree, on how accurate thoseassumptions turn out to be.

Learning from the past. A retrospective ROIanalysis will show actual performance dataabout the IT project’s costs and returns. Thelonger timeline and complexity of largerprojects can lead to substantially different

results. The analyst must take thesepossibilities into account when evaluatingthe results of pilot projects. These concernsbecome part of the risk analysis discussedin a later section.

Importance of scale. Even when there areactual data from such a pilot project, theproblem of the accuracy of assumptionsremains. There is no guarantee that costsor returns from a small pilot project willaccurately predict what happens in alarger effort. The scale of the larger effortcan itself lead to different results.

In the case study described in Appendix A(page 28), the analysis combines the twoperspectives. The developers created apilot project, which was a small part of alarger project they planned to undertake.They then used the data gathered from aretrospective analysis of the pilot projectto estimate the costs and returns that wouldresult from the full development.

Importance of businessprocess analysis

IT projects can have large and wide-rangingimpacts on business processes. Theseimpacts can occur directly in the businessprocesses involved in the project and inother related processes. Methods for theanalysis of business processes are toolarge a subject to cover in detail in thisguide, so a brief introduction to thesetechniques is included in the section onmethods below. The reasons for includingattention to these methods in ROI work isrelated to the overall strategic objective ofthe project and how it fits into the rest of theorganization.

The business process perspective concen-trates on where an IT project fits within thelarger picture of the organization’s missionor core purpose. For example, a statedepartment of transportation typically


has responsibility for ensuring the safetyof the transportation infrastructure. Thiswould usually include responsibility forinspecting bridges on state highwaysystems and contracting for repairs whereneeded. The main elements of that businessprocess includes many activities, fromdecision making about inspection policiesand schedules through contracting forrepair and construction work. The activitiescould include scheduling and performinginspections, reporting and analyzingresults, then moving into the RFI, RFPdevelopment, bidding, contracting, andproject supervision. Any change in onepart of the IT system in an interconnectedbusiness process like this one, would likelyhave important impacts on other businessprocesses. An IT project that placeselectronic sensors on bridges that sendstress data to the central office will affectmore than the inspection process. It willhave ripple effects all the way from theemployment and training of inspectors to thecosts of new construction and the public’sperception of bridge safety. These latterconcerns are all potential elements in anROI analysis.

The interconnectedness of IT projects andthe overall business process is nicelyillustrated in one recent experience witha Web project in New York City. The Citygovernment added a section on its Website reporting the results of its HealthDepartment inspections of restaurants.The IT staff did not anticipate the popularityand high demand for such information. Nordid they understand the effects that demandwould have on other linked elements of thebusiness process, which includes bothproducing and disseminating inspectionresults. The Web server was initiallyoverwhelmed by hits on the inspectioninformation soon after it was posted; wordof mouth communication rapidly spread thenews. That problem could be easily fixedwith additional server capacity. However,the high-level of public demand for thisinformation and its significant impacts on

restaurateurs put new pressures on theHealth Department. They were pressedto increase the timelines of inspectionsand to use more plain language to ensurebetter understanding of the results. Thesewere substantial cost factors directly relatedto the Web site project. A narrow focus onthe IT component of the project led theplanners to miss the effects on linkedelements of the business process. Theresults were both unanticipated costsresulting from down-stream impacts onthe business process and unanticipatedbenefits due to increased service levels tothe citizens.

The linkages in this example extend fromthe specific details of an IT project—newdata on a Web site—all the way to therelationship of the organization to itspolitical environment. These linkagesand their importance are obvious inretrospect, as are many implications forboth costs and returns. The goal of goodbusiness process and ROI analyses is toprovide the same insights and informationfor planning and decision making inadvance. How that can be done is outlinedin the description of methods below.

Benefits of process modeling

Business process models make the implicitassumptions and mental models ofindividual managers and stakeholdersmore explicit and open for discussion.They also:

� inhibit premature jumping to a solutionbecause of the way they structurethinking about a problem;

� create an externalized definition of theproblem that can serve as a focus ofdiscussion and can help to align thinkingabout what are root causes of observedproblems;

� force managers and analysts to come togrips with the precise logic or causalforces that are causing a problem;


� require attention to decide which keyvariables to measure;

� allow analysts and managers tocommunicate their reasoning effectivelyand efficiently to external audiences;

� can provide a simulation of how the fullsystem will operate within a full contextof organizational and human factors;

� push managers to see the implicationsof a limited prototype when it isexpanded to full scale operations,requiring careful attention to technical,organizational, and policy issues; and

� can include financial elements that allowexplicit exploration of costs and benefitsof proposed solutions.

Sensitivity analyses of models provideanswers to “what if ” questions about varioustypes of system functionalities and possibleorganizational and human effects. Thishelps planners anticipate issues andproblems before they are encounteredin a real world system implementation.

Methods for understandingthe business process

IT investments of any sort must be viewedas part of the business processes in theirorganizations. Both the costs and returnsof the investment are tied to how the newtechnology fits with and how it altersbusiness activities. The notion of whatconstitutes IT must be expanded to includenot only the chips, wires, and software, butalso the activities and interactions thatgenerate the costs and value that resultsfrom using the IT (i.e., the enterprisearchitecture). Failing to map andunderstand the enterprise invites a badlyflawed understanding of how the ITinvestment will work, and can be a short cutto failure. For example, the U.S. Departmentof Education recently invested millions ofdollars in a Web site for college students toapply for financial aid. The site generatedvery little use and was closed down. Itfailed because it ignored the important

role of college financial aid officers in theoverall process of students seeking andreceiving aid. The designers did notunderstand or ignored the overall businessprocess of financial aid administration.

Business process analysis tools andmethods range from simple flow andGANTT charts to sophisticated formaland mathematical modeling. Simple flowcharts are sufficient to identify the basicelements of most business processes andsome of the more obvious dependencies.How far business process modeling needsto go beyond simple diagrams dependson the questions to be answered andthe resources available for the work.

The basic components of a businessprocess are activities, flows, controls,and dependencies. Analyzing thebusiness process therefore meansidentifying the various kinds of activities,what kinds of flows they generate andsupport, and how the activities and flowsare controlled and linked to produce someparticular collection of goods or services.Flows can consist of information, persons,resources, control inputs, and workproducts moving from one activity toanother. The analysis consists of gatheringinformation about the activities and flowsin as much detail as necessary. Dependingon the kinds of modeling to be used, thisinformation can include descriptions ofactivities and flows in strictly qualitativeterms or detailed measurements ofresources used, flow metrics, and outcomemeasures.

Descriptive models

Approaches to process modeling can begrouped roughly into three levels ortypes: descriptive, analytical, anddynamic. Descriptive models are themost fundamental. They describe theelements of a business process and therelationships among them. A flow chartthat describes how travel expensereimbursements are processed, for example,might show how the request originates, how

16


it moves from one work process to another,what actions or decisions take place at eachpoint, and the elapsed time for each step.Descriptive models may involve somemeasurements, such as personnel time,expenditures, other resources used, andoutputs. The flow chart in Figure 2 belowillustrates the actual billing information flowsfor a nonprofit provider of services tomentally retarded and developmentallydisabled clients. It shows how informationabout services delivered is converted intobills submitted to the various governmentsources of funding for these client

7 For example, Medicaid billing goes to the Medicaid Management Information System (MMIS), education related billing goes to the State EducationDepartment, and so forth.

services.7 This flow chart was producedto help understand how a change in thereporting and processing technology forone of the funding agency’s would affectoperations among the providers who dealwith multiple agencies. The IT projectwas planned by one state agency, butthe planners recognized that the businessprocess involved a range of other agenciesand local practices. Therefore their analysisof the costs and potential benefits of theproject had to include attention to theoverall business process, not just whathappened within one state agency.

17

Figure 2. Example of Process Flow Chart


The level of detail used for such a model isa matter of judgment, in this case dictating amodel at an intermediate level of detail. Forsome purposes, a more highly aggregatedmodel may be sufficient, such as thefollowing.

It might be necessary to examine amuch finer level of detail, such as howadministrators review service recordsbefore approving billing in Figure 2 onpage 17. The level of detail depends onthe questions that guide the overall effort.A descriptive model influences the choiceof what is to be measured and whethermore formal modeling will be needed.

Use of analytical orformal models

Analytical or formal models go beyonddescription to represent the performanceof a business process or project in somequantitative or mathematical form. Thiskind of model has many advantages foranalysts and decision makers, along withsome substantial problems and limitations.There are at least four kinds of advantages.

� Formal models require making theimplicit explicit, and to do so in preciseand systematic ways.

� Many formal modeling methods allowexploring “what if?” questions and workwith alternative scenarios by simulatingthe results of using the model of the newsystem. The planner can thus test ideasand alternatives without risk to realsystems, operations, or data.

� The analysis yielded by the modelingcan reveal insights into the behaviorof persons or organizations that wouldotherwise be difficult or impossible todetect.

� The modeling process, if done in acollaborative way, produces a sharedunderstanding of the work amongmanagers and stakeholders that canlead to improved decisions about theproject and its ultimate implementation.

Because of these advantages, the useof these techniques for process improve-ment and project planning has grownsubstantially and spawned an industryproducing software applications, consulting,and training in one or more of thesemethods (see resources in Appendix D).

These methods are not without problemsor limitations, the most important of whichis that they raise the costs of analysis inseveral ways. These models typicallyrequire more detailed and extensiveinformation, particularly performanceand outcome measures, than simplerdescriptive methods. In addition, theconstruction and use of these modelingmethods requires considerable time andexpertise. Organizations that do not havestaff prepared to do the modeling workthemselves would have to invest either inextensive training or employing consultants.It is important to review carefully whetherthe complexity of the project justifies theinvestment in formal modeling beforetaking that course of action. The reviewof alternative methods below is intendedto help in that regard.

Types of formalmodeling approaches

Agent-based modeling. The basic idea ofagent-based modeling is that complexbehaviors of collectives (social groups,members of an organization, population)can be modeled by simple computationalrules. The rules treat the members of thecollective as autonomous agents that followsimple rules of behavior, rules that can bemodeled on a computer. What happens to


the collective is the result of the individualagents interacting with each other accordingto the rules. Changes in the rules changethe overall results, sometimes in predictableand sometimes unpredictable ways.

One early example of agent-basedmodeling in the social arena was a modelof residential segregation developed inthe late 1960’s.8 The model starts witha hypothetical city with a mixed racepopulation randomly distributed overthe residential area. Each householdeither stays or moves according to twosimple rules: a preference for living nearpersons of the same race, and decidingto move if the population in surroundingareas has too low a proportion of suchhouseholds. There is no rule involvingdislike for households of different raceor desire to avoid them. Nonetheless,the action of these rules over timeproduces a city of racially segregatedneighborhoods.

In agent-based modeling, the rules arethe model or theory. Thus they makeexplicit the assumptions and motives ofthe agents and what governs their actions.The value of the model depends on thedegree to which the rules produce resultsthat match the actions and outcomes forthe actual collective of interest. If rulesare developed that reproduce the resultsof interest, they can provide a powerfultool for understanding how those resultsoccurred and for testing ideas about howto change or improve them. This kind ofmodeling may be particularly appropriatefor information technologies that willinvolve the activities of large numbersof persons engaged in the same type ofactivity, such as large numbers of citizensseeking the same kinds of information ona Web site. A valid model of how peoplewill use a system can be a valuable tool forexploring what costs and benefits to expect.

Agent-based modeling is not for thefaint of heart, however. It is mathematical,often requiring advanced knowledge ofalgorithms and computing to implement.There are special modeling languagesfor developing the computer programsto pursue agent-based work, butthey require experienced users or aconsiderable investment in training. If therules are sufficiently complex or abstractthey may be difficult to communicate toparticipants in the planning or to keystakeholders.

Operations research and statisticalmodeling. If the factors that influence abusiness process can be identified andmeasured, modeling that process byoperations research and statisticalmethods may be feasible. These modelingapproaches use mathematical equations torepresent the business process. Thisrequires measurements of the businessprocess itself and of the factors that arethought to influence how the process works.Then the modeling equations are chosen tofit the way the significant elements of thebusiness process are conceptualized.

� If the process is seen as a series ofactivities that consume resourcesand occur in particular interdependentsequences, project description toolssuch as PERT charts or critical pathanalysis may be used.

� If the sequence of events in thebusiness process is uncertain, orif multiple paths or outcomes arepossible, probabilistic methods such asMarkov chain models may effectivelyrepresent the process.

� If the sequence or path of influenceis not known, but some measureof their general effects is available,linear modeling and regression methodsmay be useful to represent the overallprocess or predict the results of changes.

8 Thomas Schelling, (1971) “Dynamic Models of Segregation,” Journal of Mathematical Sociology 1, 143-186.


� If particular combinations of influencesare thought to work together to affectthe flow and/or performance of thebusiness process, then scenarioanalysis may be useful.

These modeling methods are similar toagent-based methods in that they requiremeasurements. They differ in that theyare based less on specific theories orassumptions about the activities thatmake up the business process. Rather,they use simplifying assumptionsabout the underlying cause and effectrelationships that drive the process. Thatmakes the models easier to use in somerespects. Little original theory building isneeded. In addition, the methods aresupported by commonly availableapplications, and training in many of theparticular methods is typically part ofmanagement education programs.Choosing among these alternatives is nota straightforward task and a reasonablyhigh level of expertise is needed. Thesame applies to interpreting the resultsand understanding the limiting assumptionsthat apply to each.

System dynamics modeling. Manybusiness processes involve feedback,that is, chains of effects in which whathappens at one point in the process hasan impact elsewhere that in turn influences(increases or decreases) the original effect.For example, a new database applicationmay produce a new kind of report that usersfind particularly valuable. Seeing onepossible new outcome prompts the usersto request additional kinds of reports thatproduce more valuable results, and morenew requests, and so forth. Any kind ofprocess that can be represented as stocksor supplies that flow from one place toanother with either reinforcing or negativefeedback can be represented using themethods of system dynamics modeling.These models can provide both a concep-tual and mathematical representation of theprocesses. The conceptual or graphicalrepresentation is usually referred to as a

causal loop model. These models can bebuilt by analysts working with the staffinvolved in the process, making use of staffknowledge and insights. The image createdis a potentially powerful tool to evokeknowledge about the process and tomove the participants to a sharedunderstanding. An example of such adiagram is shown in Figure 3 (page 21).This particular model represents a workingtheory about how work progresses. Taskscan exist in one of the four following states.

� Work to do

� Undiscovered rework

� Known rework

� Work really done

At the start of work, all tasks are in thestock of work to do. As participants performwork, correctly done tasks move to thestock of work really done, with a probabilityof 1- error rate (a fraction). The error rateis based on the idea that it is impossibleto perform all tasks correctly on the firsttry. Tasks performed incorrectly requirerework, so as work proceeds some propor-tion of the tasks enter the accumulationundiscovered rework, according to theprobability of an error occurring, errorrate. Similarly, problematic tasks canbe redone correctly (moving from theaccumulation of known rework to workreally done) or incorrectly (returning toundiscovered rework), based on the errorfraction. The concreteness of instructionsand transformability of the tasks themselvesare shown as affecting the ability to dowork right in the first place or to correcterrors. With appropriate assumptionsand values for the variables andparameters in such a model, a mathematicalrepresentation can be built that reproducesthe behavior of the process itself. Analysts,who should have appropriate expertise insystem dynamics modeling, can changeassumptions or values and explore theconsequences, yielding new insights intothe possible costs and performance of aproject.

20


Figure 3. Causal Loop Diagram of Project Management Processes

Unified Modeling Language (UML) and UseCase Modeling. UML is a programmingand modeling language that provides asystem for representing and documentingobject-oriented software development andthe business process in which it resides.9

It is particularly tailored to analyze theapplication development process andfor use by development teams in thatenvironment. The overall conceptualscheme for use of the UML includes whatis referred to as Use Case Modeling as acomponent. A Use Case is a guide forsoftware design based on a behavioral casestudy of how an application is used in anactual business process. This differs fromtraditional approaches that specify softwarerequirements in terms of technical needsand goals. For IT projects that are primarilyapplication development, UML can be avery useful and powerful tool for modeling

the project itself for management purposes,as well as for its linkages to the businessprocess. However, the UML is highlytechnical and requires considerable studyand programming experience to useeffectively.

Investment in the use of UML may bejustified, especially for large applicationdevelopment projects that involve multipleteams. As the UML documentation states:

“In the face of increasingly complexsystems, visualization and modelingbecome essential. The UML is awell-defined and widely acceptedresponse to that need. It is the visualmodeling language of choice for buildingobject–oriented and component-basedsystems.10”

9 “The Unified Modeling Language (UML) provides system architects working on object analysis and design with one consistent language forspecifying, visualizing, constructing, and documenting the artifacts of software systems, as well as for business modeling.” Object ManagementGroup, p. xi.10 Object Management Group. OMG Unified Modeling Language Specification, Version 1.3. Framingham, MA: Object Management GroupHeadquarters, 1999, p. 1-4. 13 Office of the New York State Comptroller. Accounting and Reporting Manual. Albany, NY (no date), p. 17.


Workflow modeling. The workflowmodeling approach is similar to UML butinvolves a different focus and approach.Instead of using a programming languageto model work processes, workflow methodsuse generic models of work processesand the relationships among them. Thecomponents of these generic modelsrepresent business activities, resources,dependencies, and controls. The designof the models is aimed particularly atenhancing coordination by identifyingdependencies, failure modes, andproblems of handling exceptions. Forcomplex business processes, a flow chartwill not reveal the consequences of errorsat critical points, unanticipated resourceshortages, or missing factors during somepoint in the process. Workflow models allowthese situations to be explored and provideanalysis of where processes are particularlyvulnerable to failure or waste.

Workflow modeling technologies andcommercial products may be useful toanalyze an existing business processor re-engineer one. But they are designedprimarily to develop automated workprocesses. These depend on computer-based controls, typically where the flow ofinformation or materials involves networkedlinks among the activities. However, thebasic concepts of dependencies, exceptionhandling, and error modes may be appliedregardless of the level of automationin a workflow. So the conceptual toolsof workflow analysis may contribute to aprocess model concerned with costs andreturns to a new system.

Measuring costs andreturns

Financial metrics andgovernment accounting

What is measurable within a governmentaccounting frame is set by the generallyaccepted and/or legally mandatedaccounting standards and practices thatapply to the particular governmentorganization. Definitions and structuresfor financial data are standardized anddesigned to serve specific governmentpurposes. The Office of the New York StateComptroller describes the state frameworkin these terms:

“The purpose of classifying accounts is toprovide a standard format for recordingand reporting financial transactionswhich allows comparisons to be madewith others municipalities or otherfinancial periods.”

“The classification system serves as abasis for budgeting, accounting, andreporting as well as for administrativecontrol purposes, accountability to theOffice of the State Comptroller and thegeneral public, cost accounting, and thecompilation of financial statistical dataon the state level.11”

This framework organizes the accountingdata about costs/expenditures and revenuesaccording to general organizationalprograms, functions or divisions, and byobjects of expenditure (e.g., salaries,equipment, etc.). This is sometimes calleda function-object accounting or budgetingsystem. A typical function-object accountingsystem for a state agency has a highlydetailed and structured way of definingwhat kind of financial information is to berecorded and how it is to be organized.

22

11 Office of the New York State Comptroller. Accounting and Reporting Manual. Albany, NY (no date), p. 17.


The strengths of a government accountingframework for analysis of costs andbenefits are also its weaknesses, namelystandardization and rigid structures andrules. Standards and rules make such aframe-work useful for control of finances,generating standardized reports, andoverall fiscal accountability. And if theexisting categories of accounts happento provide the particular cost and returndata needed for ROI analysis, theframe-work would be readily available anduseful. However this is seldom the case,particularly for IT investments. The full valueof an IT investment is typically a mixture ofmany types of expenditures and othercosts, the accurate accounting of which isnot possible in the traditional accountingsystem. These systems will usually recordthe costs of equipment and personnelattached directly and full-time to IT activities.But they seldom record the changes inpersonnel costs for those who use thetechnology or the investments in trainingand learning to operate new systems.Maintenance and supplies may appear ina range of separate accounts, and manymanagerial and analytical costs are verydifficult to estimate or prorate accurately.

Measuring costs andcost-effectiveness

In its most basic economic sense, a costis whatever you have to give up to getsomething else. And a return is anythinggood that you get as a result, whether thatgood is measured in financial terms orotherwise. This is, of course, a muchbroader view of costs and returns thanis found in most accounting systems.For example, an accounting system wouldnot likely include as part of the cost of anew computer system the amount of timedevoted to the purchase decision. Norwould the accounting system record thebenefits of increased staff morale from amore functional or reliable system.

A full consideration of costs requiresattention both to opportunities and toindirect costs. An opportunity cost

framework takes into account whatalternative actions or returns would bemissed or foregone as a result of aparticular investment decision. The path ofany IT project necessarily excludes otherpaths not taken. If opportunities along thoseforegone paths can be identified andvalued, they can become part of the costcalculation. If I choose to implement a newprocurement processing system, forexample, the staff time spent training forimplementing the new system is timeunavailable for training in some other areaof work skills. If a value can be assigned,this lost opportunity may be consideredpart of the overall cost calculation. Indirectcosts (sometime referred to as imputedcosts) are those that are not incurred ormeasured directly, but are calculated orestimated from other measures. Forexample, the cost of using existing networkcabling for a project could be calculatedfrom the amount of new traffic generated orfrom some other prorating factor. Similarly,apartment renters do not pay real propertytaxes directly, but the renter’s portion ofthat tax can be calculated as an indirectcomponent of rent.

These costs are often “off the books” inthe sense that typical public accountingand financial management systems donot provide this kind of cost analysis.There are no standard accountingpractices to guide the measurement ofthese costs. An agency may have anexisting standard indirect cost factor thatis used in budgeting for Federal grants orcontracts (e.g., some fixed percentageof personnel expenditures). But thesestandard factors are based on averagesover many projects and are not particularlyuseful measures for any one particularproject. Opportunity costs are evenmore problematic and are often notincluded, leading to possibly significantunderestimation of overall costs.Underestimating true costs can makean investment’s returns look more attractivethan they may really be.

Assessing effectiveness requiresidentifying the outputs of the project and


its implementation in business processes orprogram terms. That means identifyingmeaningful units of output that can berelated to the cost estimates. For example,the effectiveness of a new system to processbusiness permit applications over the Webcould be measured in terms of increasednumbers of permits processed during agiven time. Lower costs per processedpermit would be a useful cost/effectivenessindicator. Producing such a cost/effectiveness measure requires detaileddata about permit processing and a way ofassigning costs on a per permit basis.Without some enhancement, most govern-ment accounting systems do not provide thebasic data necessary for such a calculation.It may be necessary to do detailed datacollection on a sample of permit transac-tions, for example, to establish a baselineunit cost figure. A business process modelcould provide the necessary data if sodesigned. Even if such a unit cost measureis available, it may have problems, such asthe distorted assumption that all permitsare equally important or costly.

Efficiency measures

Efficiency is a way of describing theeffectiveness of a project or system inrelationship to costs (or other inputs).Efficiency cannot be separated fromeffectiveness, since using resources andfailing to achieve a desired outcome can belittle more than waste; you don’t save moneyby building half a bridge. Efficiency isusually expressed in terms of optimizing thevalue of a return for a given cost or input, oralternatively minimizing the cost for a givenvalue of result. It is possible, of course, toimprove efficiency without necessarilyachieving an optimum. As long as it ispossible to compare cost/effectiveness orreturn ratios for alternative systems ormethods it is possible to make judgmentsabout efficiency. To demonstrate an optimumresult or projection, it is necessary to havea simulation or optimization calculation toprovide the data. Some operations researchand simulations, such as linear program-ming or workflow simulations, can generate

this type of calculation if the necessarymodels and data are available. Howevermost IT projects do not have the data oranalytical resources to include thesemethods in an ROI analysis.

Impact measures

The identification of variables that make upa social cost or benefit calculation alongwith their definitions is broader than boththe economic and accounting frames. Theyare based either on the specific programresults desired by an agency or on generalsocial benefits and improved quality of life.These impact measures can come fromseveral sources.

Operational data. Some useful measures ofsocial or broader economic outcomes maybe available in the data collected duringordinary program operations. Providers ofservices for homeless persons, for example,routinely collect data about programs andactivities of clients that could be indicatorsof impact on their quality of life or progresstoward independence. Similarly, lawenforcement agencies routinely collectcrime statistics that can be useful indicatorsof neighborhood climate or quality of life.

Social and economic statistics.Government agencies collect statistics onsocial and economic indicators that arerelevant to the overall social and economicstatus of their jurisdiction. These range fromthe enormous resources of the US CensusBureau to state and regional planningagencies, and include market researchstatistics available from private sources.

Special studies and evaluations. For large,high cost projects separate data collectionactivities, such as surveys or field research,may be used to collect and analyze dataabout social and economic impacts of aproject. These efforts may be expensive andtime-consuming, but may also be the onlyway to obtain data about particularoutcomes. The case example in Appendix Con Social Return on Investment providessome examples of how this may be done.


How time perspectiveschange the measurements

Time has important effects on themeasurements and calculations. Both thecosts and returns of an IT investmentextend over time. Moreover, the costs maybe incurred long before the returns arerealized. So when estimating the moneyvalue of costs and returns, it may benecessary to take into account the effectof the time perspective on the value ofmoney. That is, a dollar in hand today isworth more than the promise of a dollarin hand a year from now. How much lessthat future dollar is worth, known as thediscount, depends on the current timevalue of money, which is usually calledthe interest or discount rate. Thus if thediscount rate is 5 percent per year, thenthe promise of a dollar a year from now isworth only 95 cents today. The farther outin the future, or the larger the discount rate,the less the present value of the promiseof that future dollar. Calculations based onthe discounted value of future returns areuseful to estimate what size of return isneeded to justify the costs of an investment.The typical methods used in this kind offinancial analysis are called Net PresentValue and Internal Rate of Returncalculations. Both methods use an initialcost and some assumed figures for a futurestream of returns, along with a reasonableassumption for the interest (or discount)rate. The Net Present Value (NPV)calculation estimates the value today ofreturns coming in the future, less the cost ofthe investment. If the NPV result is less thanthe cost, then the investment does notappear to yield a net gain. The Internal Rateof Return (IRR) calculation yields the samesort of result by a different path. Usingthe same basic information as the NPVcalculation the IRR calculation yields therate of return for the investment assuming abreak even, or NPV of zero. If the IRR is less

than the current interest rate, this indicatesthat putting the investment in an interestbearing bank account would yield a biggerfinancial payoff than investing in the project.An example of NPV and IRR calculationsare shown in Table 3 (page 26). Thisexample is based on a hypotheticalinvestment analyzed over a six year period.It shows that when the time value of futurereturns is taken into account, the totalreturns must exceed the total cost by asubstantial margin to generate a positivenet present value or internal rate of return.12

The problem of choosing theproject or investment life cycle

Notice that any calculation of this sortrequires specifying a time-frame withinwhich the value of the project orinvestment will be assessed. This timeframe is sometimes referred to as the lifecycle of the project. Any IT investment isassumed to have a limited useful life,beyond which the returns are notconsidered, or when the expected returnsfrom some new technology will lead toreplacement of the previous investment.For IT investments, the rapid change intechnology makes the choice of areasonable life cycle or planning frameworka difficult but necessary part of the analysis.

Risk analysisin the public sector

Risk analysis consists of assessing theimportance of threats and how to mitigate oreliminate them. We usually evaluate a threatin terms of how likely it is to materialize andhow much damage or cost would result if itdid materialize. We know, for example, thata large asteroid could hit the Earth anddestroy a continent—an enormous threat interms of consequences. But astronomers

12 The discount rate shown in these calculations was chosen for illustration only.


13 Calculates the net present value of an investment by using a discount rate (rate) and a series of future payments and income (values) in eachperiod (i) for n periods.

tell us that the likelihood of that happeningis very, very remote—therefore, a smallthreat in terms of probability. Because ofthe low probability, the threat is seen assmall enough that few of us (asidefrom some science fiction writers andastronomers) take it into account inday-to-day life. The same logic applies torisks in IT investment. Consequently riskanalysis can be thought of as having fourbasic steps:

� identifying the source of threats,

� assessing the extent of potentialdamage or cost to the project,

� assessing the likelihood of the threatmaterializing, and

� devising ways to reduce or eliminatethe threat (i.e., mitigate the risk).

Table 3. Net Present Value Calculation13

Year

1

2

3

4

5

6

Total

Costs

$ 100,000

$ 5,000

$ 4,000

$ 4,000

$ 4,000

$ 3,000

$ 120,000

Returns

$ 15,000

$ 20,000

$ 25,000

$ 25,000

$ 30,000

$ 35,000

$ 150,000

3.0%

$ 13,297

6.0%

Discount Rate

Net Present Value

Internal Rate of Return

Threats can be reduced by taking stepsto lessen the damage or cost that wouldoccur if the threat materializes and alsoby reducing the probability that thethreatening event or action will occur. Forexample, a project could call for using themost reliable platform for a critical databaseapplication, reducing the probability thatthe system will experience a failure. Theproject could also implement a backupsystem capable of taking over processingif the primary system does fail, reducingthe potential damage of a failure.

The overall subject of risk analysis is toolarge to treat in detail here. However, mostof the risk assessment issues describedabove involve problems of thinking beyondthe boundaries of the project, measuringfactors, or determining probabilities. Thisshould not discourage risk analysis.


The experience of those involved in ITprojects can be a rich source of intelligenceand experiential data on which to basereasonable estimates of risk potentialand problem sources. The literature onIT investment is another rich source ofanalyses of successes and failures thatprovide additional insights into risks andmitigation strategies. Simply recognizingwhere uncertainty and potential damagelie is half the battle. Careful risk analysis,based on the best available data andestimates, will surely assist in ROI analysisand improve planning, even if the amountor quality of data is less than ideal.


In order to better understand the problems and issues of ROI analysis, the Center forTechnology in Government decided to usea form of ROI analysis for one of its own ITinvestment projects. The Center’s missionto build and disseminate knowledge aboutIT uses in government usually leads toprojects with other organizations. But thiscase provided an excellent opportunity forlearning and knowledge building as partof the Center’s own internal work. TheCenter’s Web site is a critically importanttool in disseminating information andmaintaining contact with colleaguesand customers.

During the time covered by this case, theWeb site consisted of approximately 3500pages and experienced over 1000 visitorsper day. The Web site was supported bya full time Webmaster and part timemaintenance and development staff of twoprofessionals and two graduate studentswho are part of the Center’s TechnologyUnit. The Web site had been in operation forseveral years and there was a substantialbaseline of experience with developmentand maintenance using the currentarchitecture. Any investment in new Website architecture would use internal staffand resources. Therefore, it presented agood opportunity to apply ROI analysis toa realistic problem in a situation wheredata collection, analysis methods, and theresults can be examined at close range.

The purpose of the investment was tochange the Center’s Web site from a static,HTML-based architecture, to a dynamic,XML-based one. The problem to besolved was a real one, namely that theCenter’s Web site had grown in size andcomplexity to the point that maintenanceand additional development put a serious

strain on existing resources. In addition,the static site could not support newformats and capabilities the Centerwanted to use on the site.

The Center faced the decision of whetherto continue to invest additional human andtechnical resources in the existing HTML-based static architecture, or change thearchitecture into an XML-based one thathad potentially much lower maintenanceand development costs. The change wouldrequire a substantial initial investment intools and staff time to learn how to useXML and related development applications.The ROI analysis would help answer thequestion of whether the potential returnsto be obtained by the use of a dynamic(XML) Web site architecture would exceedthe costs of the conversion. The expectedreturns would consist of savings comparedto the costs of maintaining and continuingto develop the existing Web site. Returnswould also include the expanded abilitiesof staff to create enhanced Web sitecapabilities and features using the newarchitecture and application tools.

Estimating the potential cost of expandingthe staff to maintain the current Web sitearchitecture was very straightforward.However, estimating the conversion coststo a dynamic Web site was a much morecomplicated problem. A complete ROIanalysis should also include an estimateof the savings (if any) to be obtained byusing the dynamic architecture. Estimatingthe savings would require comparison withthe costs of maintaining and developingthe existing system, so estimating thesecosts was part of the design.

Appendix A: Case 1Reducing the Cost of Web SiteDevelopment and Maintenance


Advantages of a dynamicWeb site

Web sites can be created with either astatic or fully dynamic architecture. StaticWeb sites consist of HTML pages thatcombine content (words, numbers, andimages), logic (how the content ismanipulated), and presentation (colors,layout, fonts, and formatting). In HTML,there is only limited capability to manageor change the content separately from theway it is manipulated or presented.Managing and changing the materialrequires working with content and HTMLtags directly within each of the pages. Staticsites are usually easier to develop thandynamic Web sites, but can become muchmore costly to maintain and manage asthey increase in size. Dynamic Webarchitecture, in this case based on XML,provides the ability to greatly simplify themanagement, evolution, and expansionof a Web site by providing control of thelogic and presentation independent ofthe content. For example, users can bepresented with information based upon theirindividual preferences, such as larger fontsfor a visually impaired person. The site canalso be made much more interactive.Dynamic Web sites are generally moreexpensive to develop than static ones,but are cheaper to manage and maintainup-to-date content.

Data sources

The best source of data about the costsand returns for this new Web architecturecame directly from the staff ’s experience.The Center staff needed an estimateof costs and returns for the XMLconversion that they could trust as a validrepresentation of what they could expect ina full-fledged conversion, without actuallyimplementing that conversion. To do this,they chose to develop a few relativelysmall new Web-based applications aspilot projects using the new methods

and architecture. The first was a decisionmaking guide to help organizations designelectronic information access programs.The project was called “Gateways.” Theexperience from those efforts provided thenecessary information about costs andreturns to inform the larger decision.

Technology Unit staff members responsiblefor Web site development and maintenanceconsisted of three full-time personnel andtwo half-time graduate assistants. All wereindividually interviewed weekly for a periodof four months. During the interviews,members were asked to recall the work theyhad done the previous week. They wereasked to identify:

� whether tasks were done in HTML or inXML,

� the amount of time spent in production(“doing” the work) versus learning thenew application in order to do the work,

� the benefits of working with XML, and

� the drawbacks.

The data for the study also includedinformation about the operation of the Website and related activities, such as impactson routine site maintenance, changes incontent development procedures, andcoordinating work on this activity withother technical tasks.

Weekly interviews were a low cost andrelatively unobtrusive data collectionmethod. They provided enough data sothat it was not necessary to observe workdirectly or have the staff record their worktimes in activity logs. This method fit onepurpose of the case: to be a useful examplefor others contemplating ROI analyses,employing methods that would be usefulin a wide range of settings withoutrequiring highly specialized training orhigh costs. The staff involved in the workagreed to the interviews and preparedfor them as a regular weekly activity.


showed most commercial products to bevery expensive. However, after a review ofavailable tools the Technology Unit decidedon the open source (free) application,Cocoon. This tool seemed to fit the needsof CTG and the staff began working with it.

While the software was free, it requiredthe staff to learn new skills. The underlyinglogic of how the Web site operates wasalso considerably different under the newarchitecture. So the development team hadto learn to think about its tasks in a newway as well. By contrast, there were zerolearning costs for maintaining the Web sitewith the old architecture and methods.

The initial learning process was a slowone. It took approximately three months forthe staff to reach a level of skill necessaryto move from using HTML to using XML.At first, as much as 80 percent of theirtime was spent on learning the XMLapplication and related skills, and 20percent on production. After a 10-weekperiod, the time devoted to learning droppedto approximately 40 percent of the total,with the remaining 60 percent used inproduction. Three months into the effort,20 percent of staff time went to learningand 80 percent to production. The stafffound that this latter 1:4 learning-to-production ratio is consistent with thelearning-to-production ratio under HTML(prior to the switch to the new technology).

The learning took place in two stages. Inthe first stage, the team shifted its way ofthinking about Web development from theHTML to the XML architecture. The secondstage was to learn the technical details ofworking in XML. After the initial three-monthlearning stage, staff members consideredthemselves knowledgeable enough aboutthe language of XML to deal with problemsquickly. The second stage included learningto use the Cocoon development tool. Sincedocumentation was limited, they used theCocoon listserv for questioning otherswho use the program. Much of thislearning came from applying the new tools

Cost estimation

One goal of the case study was to establishan estimate of baseline costs for Web siteoperations. This was complicated by thefact that the CTG Web site is continuouslygrowing with the addition of new anddifferent types of information. Theestimated costs for maintenance of thecurrent site (at the start of the study period)were approximately 75 percent of onefull-time staff person’s effort per week.This was treated as stable over the termof the case study.

The case data also included estimates ofthe effort devoted by staff members toresearching XML (dynamic Web pagedesign), which began several monthsprior to its initial use by the TechnologyUnit staff. The director of the TechnologyUnit had been contemplating movingto XML for quite awhile. This earlydeliberation was prompted by the rate ofgrowth for maintenance and developmentefforts on the CTG Web site. The effortdevoted to this period of informal researchand deliberation was considered part ofthe initial investment

Baseline costs. The Center’s Web site isvery extensive, consisting of approximately3500 Web pages and receiving on averageover 1000 visitors per day. Information iscontinuously updated and new productsand reports are introduced frequently.Maintaining the Center’s Web site is quitean undertaking. Under the old HTMLarchitecture, a Webmaster was responsiblefor maintaining site links, updating currentmaterials, adding announcements and newmaterials, and related tasks. A full-time staffperson devoted approximately 75 percentof their effort to maintaining that site(considered nine person-months per year.)

Investment costs.The primary investmentcost for this project was staff time tolearn the software and develop the skillsnecessary to build the pilot projects. Thesoftware costs for the conversion might havebeen quite high, since initial research


to the tasks specific to the pilot projects.This included creating interactive toolsthat allow users to work with applicationson the Web site. The team’s learningprocess was focused and task-specific,the results of which could be appliedimmediately. The staff avoided exploringunneeded features and functionality ofCocoon.

Additional learning costs are reflectedin the different amounts of time neededto revise the printed version versus theelectronic version of the Gatewaysdecision making guide. The first pilotapplication developed with the XMLarchitecture was based on that guide.Initial changes to the electronic versionof the guide took up to 50 percent longerthan print-only revisions because of thetime spent learning to apply the newtechnology in the Web application. Initiallythe staff found it difficult to use XML datastructures. As a result it took considerabletime to understand basic concepts andacquire basic skills for solutions that atfirst often appeared to be quite simple.

Benefits

More efficient development of newfeatures and capabilities. Benefits wereseen almost immediately during the learningprocess. During the first three months thestaff were able to see how XML woulddirectly benefit their future work. Forexample, a separate file of interactivetools was integrated into the GatewaysGuide using established style sheetstaken from existing components of CTG’sWeb site. The new technology savedconsiderable amounts of time. Thisparticular task required only four hours ofwork instead of the estimated 16 hours itwould have taken using HTML.

More efficient content management.In HTML, the content of the page is tiedto the format, which makes it difficult (ifnot impossible) to separate ongoingdevelopment and maintenance functionsfrom content revisions. In order to update

the content of a page, some knowledge ofHTML is required. XML operates differently,separating “content” from “presentation” or“style.” Since the content and style are nottied together, the ongoing developmentand maintenance functions are separable—allowing for division of labor orspecialization. Content “owners” or“creators” work only with content whileWeb designers work with Web coding.

In XML, content changes can be managedby the content creator alone. Changesmade to the XML source file will appearin all formats such as Netscape, InternetExplorer, text-only, and PDF. Under HTMLchanging one word of content typicallyrequires changing that word in all formats.In comparison, XML saves considerabletime and ensures consistency throughoutthe site. More importantly it allows forgreater flexibility in managing content.Since changes are more easily made, itis more likely that the information will beupdated and changed as needed.

When XML is used, the Web site can bemanaged quite differently from HTML-based methods. Work in XML and Cocoonplayed to the strengths of the staff membersby allowing different components of theWeb site (content, logic, presentation)and their related tasks to be kept separate.Individual staff members can concentrateon work at which they are more proficient.Dividing the labor saves time and allowsstaff members to become “experts” inspecific tasks. In an HTML environment,all three components—content, logic, andpresentation—are all combined in theindividual Web pages. Anyone workingon the page must understand and deal withthe components together. Small changesin content could require extensive work withlogic and presentation factors and involvemuch more interaction and coordinationamong content creators and HTML workers.In the XML environment, content revisionsand expansion is separate from the othercomponents, which can be manipulatedlargely independently. Content changesrequire very little coding or programmingresources, and changing or developing new


programming capabilities is not constrainedby potential impacts on content.

Less maintenance effort. Current Web sitesbased on HTML use the Web page as theunit of composition; so a 50 page site canbe thought of as having 50 separate units ofwork. These static Web pages combine bothcontent and formatting. As a result, changesin one page have virtually no effect on theothers. For a single Web page or a Web sitewith few pages or minimal updates, workingon individual pages may be relatively simpleand affordable. However, as the size andcomplexity of a site increases, the cost ofongoing development and maintenanceincreases dramatically. It thus becomesincreasingly difficult to keep the siteconsistent and up to date. In addition, somefunctions, such as forms or databases, aredifficult or impossible to do on static sites.

In XML, code and content are easier tomaintain since code and content are storedin separate sources. XML allows for thestreamlining of ongoing development andmaintenance functions because content isstored in one place but propagated in manyplaces. Because of this architecture thereare fewer files to maintain (a 50-page sitemay contain only one content file and 3–5style files). Less time is required to maintainthe fewer files. One of the benefits ofemploying this new approach, despitethe time and resources spent on thelearning process, comes from the potential“payback” in greater functionality, easiermaintenance, and reduced ongoing costs.

Greater browser support. In XML, Webdesign is independent of browsercapabilities. Changes made to an XMLsource file will appear in all formats such asNetscape, Internet Explorer, text-only, andPDF. Under HTML changing one word ofcontent would require changing that word inall formats. XML saves considerableamounts of time. More importantly it allowsfor greater flexibility in managing content.

This lowers the cost of adjusting formatsfor presentation in different browsers. It isdifficult and time consuming to craft HTMLdocuments to work consistently in allbrowsers, since each browser interpretsHTML standards differently. However, whena page is created in XML using the Cocoonframework, it can easily be made viewablein all browsers. Staff no longer devote timetesting to ensure a page is viewable in allbrowsers, since the inclusion of an XMLbrowser parameter automatically adjustsformatting.

Greater platform support. XML standard-izes and universalizes “content” or “data”across platforms. Currently, a majority ofWeb pages are mainly constructed usingHTML. The “content” is tied to the HTMLformat. The page works on a Web browser,but may not work on other platforms such aswireless devices, cell phones, and PDAs.XML offers greater delivery functionalityacross a variety of platforms. It allows theCenter’s staff and customers the flexibilityof viewing pages of the Web site on anyWeb server architecture.

More efficient management of style andpresentation. Style and presentation of theWeb site are more efficiently managed withXML than with HTML for many of thereasons discussed above. Since the pageitself is not the unit of composition, it iseasier to maintain consistency throughoutthe site. Banners and footers, for example,are maintained as single files and can beimported to all style sheets. The separationof “content” from “presentation” allows fordivision of labor. Since Web design isindependent of browser capabilities, Webpages are viewable in all formats.


Summary of costs and returns

The relationship between investment andreturns in the Web site conversion is bestexpressed in terms of shifts in productivityand opportunity. The principal costs of theproject are reflected in the opportunitiessacrificed and the added personnel costsincurred in order to develop new skills andcapabilities. The primary cost component isopportunity costs: staff were diverted fromroutine work to learn new skills and tobegin to use the new techniques. Someadditional staff resources, in the form ofgraduate students, were used as well.The magnitude of that opportunity cost isrepresented in the comparison between thetwo charts in figures 4 and 5.

Figure 4. Cost Trends

If the project had not been started (Figure 4)there would be available resources duringthe first three months of the time-frame tomaintain routine Web development and topursue some new opportunities. In the sametime period (Figure 5), by contrast, theproject began with a much expanded Webdevelopment/learning effort that consumedall slack resources and some operationalresources. No other opportunities weretaken up and ordinary Web operations werereduced. That was a period of substantialinvestment with little return.

By months 4–5 into the project (Figure 5),sufficient learning had taken place andallowed for exploiting new opportunities.As experience and the stock of usablecomponents grew, the cost of addeddevelopment dropped rapidly and resourceswere freed up to undertake even more newopportunities.


The cost of ordinary Web operations didgrow somewhat during that period due tothe normal development of new content.However, the total cost of including newcontent and making routine maintenancechanges in the site is expected to drop,despite increasing volume, due to theincreased efficiency of the dynamicarchitecture. As efficiency in thedevelopment and operations of Web sitework increase, the opportunities forexploiting new opportunities grow. This canbe contrasted with the projections inFigure 4 that suggest the increasing costof ordinary operations under the oldarchitecture will squeeze out developmentopportunities and in time will likely exceedthe budget constraint.

This kind of opportunity-based analysis canshow positive results when relatively littlefinancial data is available, and when thevalue of new products or capabilities aredifficult to determine. There are no savingsapparent from this effort unless the analysistakes into account what it would cost todevelop these new capabilities and exploitnew opportunities under the old technology.However, the expansion of developmentopportunities over time, in contrast to thebaseline scenario, does give a reasonablebasis for estimating the returns for aninvestment such as this in its early stages.Since the projections are based on the earlystages of the work, it is possible that theunanticipated problems of increased scalecould change the results over a longer term.

Figure 5. Cost Trends


In addition to the desire to enhanceprogram efficiency, the PRWORA includedfinancial sanctions for states that fail tocomply with regulations or to achieveprogram goals. Improved informationresources and technologies were seenas ways to enhance programs, achieveefficiencies, and reduce the risk ofsanctions.

The sanctions can be considerable. If thestate did not meet the Federal requirements,future funding for related programs couldbe threatened. For example, failure tocomply with current Health InsurancePortability and Accountability Act (HIPAA)requirements can result in loss of FederalFinancial Participation (FFP) of 90 percent,and penalties up to $25,000 per person,annually, in addition to civil penalties. Thepossible sanctions linked to TANF couldamount to as much as $14.8M per fiscalyear.

Investment in improvedwelfare administration

Existing technology allowed the Stateof Iowa to assist front-line workers indetermining eligibility and benefits, tomeet some Federal reporting requirements,assist in program evaluation, and to makeinformation available for decisionsregarding program and related personnelissues. To improve these informationresources, the state allocated a little over$1 million in the 2001 fiscal year to theWelfare Reform Related Technology Fundwith the available funding under TANF.These funds were appropriated in theDepartment of Human ServicesAppropriation Bill. The IT program was100 percent funded by the TANF blockgrant funds.

Project goals and thecontext of State-Levelwelfare reform

The information reported here was done fora project designed to integrate informationresources used by Iowa agencies in theadministration of welfare programs andwelfare reform efforts. The project was aresponse to the changed requirements forthe administration of welfare programsresulting from the Personal Responsibilityand Work Opportunity Reconciliation Act(PRWORA) passed by Congress in 1996.The Act gave states welfare block grantsand increased discretion in the allocation ofthese welfare resources. Iowa, as did mostother states, responded to this major policyshift by creating its own new policies andadministrative arrangements. The projectanalyzed here, the Welfare Reform RelatedTechnology Fund, was one of thoseresponses. The fund supported systemdevelopments for the major welfareprograms in the State, such as TemporaryAssistance for Needy Families (TANF), foodstamps, and Medicaid.

Project rationale

In order to use the block grants efficientlyand to design their own human serviceprograms, Iowa emphasized accuracy ofeligibility determination, benefit distribution,service delivery, and client support.Accurate and timely information is a criticalresource for many complex decisionsrequired in administering this mixture ofbenefits and requirements. Informationinfrastructure and resources are necessaryto make accurate decisions and producehigher quality programs.

Appendix BCase 2: ROI for Data Integration inHealth and Human Services


The funding supported a variety of ITenhancements for welfare administration.They included enhancing applicationdevelopment and system programming,tracking client eligibility, increasing datastorage and collection capability, andimplementing software updates andchanges. One of the expected resultsof these investments was the capacity tosubmit accurate and timely reports thatcomply with the requirements for Federaldata reporting to the US Department ofAgriculture (food stamps) and Departmentof Health and Human Services (Medicaidand TANF). The state has to comply withHIPAA requirements as well.

The information issues are large andcomplex. The needs of each welfare clientmay involve many agencies, each with itsown personnel, mission, and policies.Each agency involved has its own ITsystems, producing many barriers toinformation sharing and integration. Noneof the staff in any single department wasable to access the complete informationabout an individual client from existingdatabases. Therefore, the investmentsof this project were aimed at creatingstandard protocols for data exchange,a data warehouse, electronic referralsystems, resources directories, andrelated applications.

Sustainable and coordinated hardwareand software development was requiredto reach these project goals and providecustomers with better quality services.The technology supported programs thatprovide benefits and/or services annuallyto approximately 20,000 families beingserved by the Family Investment Program,53,000 households receiving food stamps,and 204,165 individuals receiving Medicaidbenefits (monthly averages).

Project methods

The beginning phase of the project was asystem evaluation. The goal was to providedecision makers with a comprehensiveunderstanding of the beginning status quoof the program. This would help avoidwasting resources on system componentsthat were already functioning at a high-level.

The system evaluation was followed by asearch for existing software that could becustomized to meet project needs andgoals. The project planners then exploredthe possibility of consolidating all theinformation systems onto a singleplatform to achieve integration and commonaccess. However, the implementation ofconsolidation was judged to be too costlyand time-consuming, and so was rejectedas impractical. In order to retain the valueof existing legacy systems and infrastructureinvestments, the planners chose to usemiddleware as a more effective approach.They required the middleware software tobe based on open standards and to extendexisting IT investments. The plan alsoincluded replacing and upgradingout-of-date hardware and ensuring thatthe hardware is capable of supporting thenew applications and customized software.Continuous staff training was included asa key requirement.

ROI framework

All IT projects in Iowa agencies are requiredto prepare ROI material in a standardframework as part of any proposal fornew IT investment. The state provides aWeb site,14 supporting materials, andapplications for agencies to use whenpreparing their proposals and ROI analyses.The results reported here are taken fromthose sources.

14 http://www2.info.state.ia.us/roi/index.html

36


The Iowa ROI framework requires attentionto certain standard components for allproposals. These include the following.

Analysis requirements. Proposalsshould include:

� use of the Rapid Economic Justificationmodel15 —to understand the business,alternative solutions, cost-benefitequations, possible risks, andfinancial metrics;

� cost and benefit measurements—estimation of project costs andbenefits in some comparable unitand determination of whether thebenefits exceed the cost; and

� achievement of cost avoidance anddollar matching from state sources.

Software. Proposals should deal with allsoftware components including:

� application software;

� operating system software;

� interfaces to other internal and externalsystems; and

� standard protocols for data exchange,data warehouses, linking software tothird party service providers, electronicreferral systems.

Hardware/Facility. Proposals shoulddeal with all physical components including:

� additional platforms that accommodateinteroperable operating systems;

� adequate storage and physicalenvironments;

� adequate connectivity and bandwidth;

� logical and physical connectivity; and

� major interfaces to other systems, bothinternal and external.

15 http://www.microsoft.com/technet/treeview/default.asp?url=/technet/ittasks/plan/sysplan/wwww.asp

ROI results for the WelfareReform Related TechnologyFund

The expected benefits of the project wereto enhance organizational coordinationand reduce duplicate key-in and paper-work. Table 4 (page 38) is the financialsummary for the Welfare Reform RelatedTechnology fund for state fiscal years (SFY)2001 through 2003. Total project cost (RowA in Table 4, page 38) includes items suchas personnel, software, hardware, training,facilities, professional services, supplies,and others. Total annual project benefit forthe state (Row B) explains how much thestate has benefited from the investment ofthe technology fund by the avoidance offederal penalties. The magnitude of thereturn on investment was expected todecrease year-by-year. The likelihood andmagnitude of penalties would be largest inthe first year or two, and would diminish asthe accuracy and efficiency of the systemimproved with experience and refinements.This is a normal example of the operationof diminishing marginal productivity of aninvestment over time. At any rate, becauseof the very large impact of trying to avoidFederal penalties, the ROI percentageremains very large. In addition to thefinancial returns, the project plannersexpect benefits resulting from moreefficient, effective implementation ofchanges resulting in improved customerservice, increased program accuracy, andreadily available information for programand field staff to use in making businessdecisions.


“Avoidance benefits include $440,992food stamp penalties, $5,600,000,000potential Medicaid related losses, and$29,592,824 TANF penalties. Funding forthe TANF penalties will be needed inSFY 2002 ($14,796,412) and in SFY 2003($14,796,412) plus Federal Match forfood stamps and Medicaid in the amountof $1,264,384. There is additionalpotential for sanctions due to foodstamp error rates. The amount of thesesanctions is unknown.”

The similar footnote in the FY2002description contains essentially the sameestimates for all the other savings, but thepotential Medicaid related losses drop from$5.6 billion to $500 million. This suggeststhat estimates of this sort are subject toconsiderable uncertainty and may not bethe best basis for an investment decisionwithout additional supporting data.

Table 4. Results of the IOWA ROI Analysis for Three Project Years 16

SFY 2001 SFY 2002 SFY 2003

A) Total Project Cost $ 2,024,768 $ 1,468,324 $ 1,556,016

B) Total Annual Project Benefit for State $5,631,298,200 $529,600,000 $ 30,920,192

C) Total Annual Project Cost for State $ 760,300 $ 734,162 $ 778,008

D) Project Funds Requested % fromState 37.55% 50% 50%

E) Project Funds Requested from State:(A*D) $ 760,300 $ 734,162 $ 778,008

F) Benefit/Cost Ratio: (B/C) 7406.7 721.4 39.7

G) ROI: (B-C/ E)% 740568% 72037% 3874%

** Some numbers differ from the original report due to rounding.

Potential risks in benefitestimates

The benefit figures claimed in this analysisappear to be based on a rather optimisticscenario. The biggest return is costavoidance due to diminished Federalpenalties—a very large decrease betweenthe 2001 and 2002 fiscal years. Thesepenalty levels were very high for SFY 2001and were expected to drop by over 90percent in a single year due to theintroduction of improved informationtechnology. It is not clear from the availabledocumentation how realistic these penaltyreduction estimates were. A footnote to theROI analysis report for the SFY 2001 projectdescription reads:

16 State of Iowa Return on Investment Program, IT Project Evaluation for Department of Human Services;SFY2001, 2002, 2003;http://www2.info.state.ia.us/ROI/index.html


Such optimistic estimates to justify aproject proposal are not unusual. In order towin the resource competition, it is temptingfor agencies to assume the worst scenariofor not implementing the proposed project,compared to the best case prediction forcompleting the project. That way agenciescan show very dramatic and persuasivereturns for reviewers. Decision makershave to find a reasonable balancing pointbetween the two extremes. Relatedinformation needed to make morereasonable assumptions may not beavailable in the proposals, if it is notrequired. Hence, the evaluation process isusually problematic and critical. In this IowaHealth and Human service case, reviewersmight need more detailed information aboutthe process of savings calculation in orderto make an accurate judgment. That is,the proposers of a new investment maydeliberately skew their calculations to makea stronger case than they could otherwisejustify. Reviewers may not be able todetect such deliberate exaggerations orunreasonable assumptions unless theyhave full information about how calculationswere made. ROI calculations are productsof social, political, and economic interestthat are often in conflict with each other.17

The complexity behind the numbers andcalculation processes should always bepart of the overall decision making process.

17 William Alonso & Paul Starr (editors), The Politics of Numbers. New York: Russell Sage Foundation, 1987

Resources:

1. State of Iowa Return on InvestmentProgram, IT Project Evaluation forDepartment of Human Services;SFY2001, 2002, 2003; http://www2.info.state.ia.us/ROI/index.html

2. Welfare Reform, Information Systems,and the States, NASCIO; www.nascio.org/publications/welfare1998

3. Government Technology, Case Studies:Health and Human Services;www.govtech.net/govcenter/solcenter

4. Microsoft’s Vision for Technology inHealth and Human Resources;www.microsoft.com/business/industry/gov


Appendix C: Case 3Social ROI

The investmentphilanthropy approach

This example shows the results of anextensive effort to measure a wide rangeof social and economic outcomes ofdifferent investments in social programs,in this case by a private foundation, TheRoberts Enterprise Development Fund.The analysis of returns on the Fund’sinvestment is based on impact-typequestions. The Fund invests by makinggrants to nonprofit, community-basedorganizations in the San Francisco area forthe purpose of “creating social value.” TheFund refers to this approach as investmentphilanthropy:

“Investment Philanthropy… is concernedwith the value accrued as the result ofcharitable investments. Within thisperspective, social returns (that is,benefits to society) generated byphilanthropic investments are themeasure of an investment’s success.The critical challenge in InvestmentPhilanthropy is to compare the moneyinvested with the value it creates.18”

The Fund decided to evaluate the grantsto nonprofit organizations on the basis ofthe social value created rather than on thegoals of the grantee or the apparent meritsof the organization’s efforts. To do sorequired the development of methods todefine and measure that social value. Sincemany government programs provide fundsfor nonprofit organizations and the creationof similar concepts of social value is thegoal of most government programs, themethods in this case can be instructive.

18 Roberts Enterprise Development Fund. SROI Methodology: Analyzing the Value of Social Purpose Enterprise Within a Social Return onInvestment Framework. San Francisco: The Fund, 2001, p. 10.

The social return on investmentmethod

To implement the investment philanthropyapproach, the Fund developed methodsfor measuring the social return on itsinvestments and created an administrativemechanism to ensure that the analysis andreporting information to assess SROI wouldbe available. The method defines socialvalue in a way that can be measured andprovides measurement procedures andanalysis techniques. The Fund requires itsgrantee organizations to prepare andsubmit SROI reports that provide the resultsof these analyses to the Fund so that it canevaluate the overall value of its investments.

The SROI method uses a definition ofvalue that covers a continuum frompurely economic to purely social, withsocioeconomic value in an intermediateposition:

Economic Socioeconomic Social

Economic value is represented by afinancial return on the Fund’s investment,reported as increased revenue, asset value,etc., from the grantee’s accounting report.This financial return is defined andmeasured according to the techniquesfor accounting and demonstrating profitcreation that apply in the regular capitalmarkets—i.e., the stock markets andprivate sector accounting methods. TheSROI reports prepared by grantees includedetailed financial statements equivalentto those produced by publicly tradedcompanies in the for-profit sector.

40


$ -

$ 300

$ 3,362

$ 5,004

$ 27,300

$ 40,180

$ 2,924

$ 39,072

$102,300

$ 422

$ 4,116

$ 16,120

$ 37,620

41

organizations, reduced criminal activity,is estimated by comparing arrest andconviction rates for employees with similarsegments of the population. Reductions inarrests and convictions are monetized bycounting the savings to the society of fewerconvictions and incarcerations. An exampleof the calculations for social cost savings isshown in Table 5.

Socioeconomic value is defined by creatingmethods to assign money values to socialoutcomes wherever possible. For example,increased employment opportunity for thosewho work in grantee organizations is asocial value. Socioeconomic value isexpressed in part by the increased taxespaid by those employed, as well as byreductions in welfare costs. Another socialoutcome of employment in grantee

Public Assistance Programs*

TANF

General Assistance

Food Stamps

SSI

Social Service Programs

Food Banks

Case Management

Community Clinics

Mental Health Treatment

Housing Services (shelter, trans,housing, grp home)

Emergency Room

Legal Services

Sustance Abuse Treatment

MediCal (includes employee anddependents)

Criminal Conviction Savings

Average Social Cost SavingsPer Employee

Table 5. Calculation of Average Social Cost Savings Per19

TotalDecrease(Increase)in Annual

Visits

x Cost PerVisit/Use

=

TotalDecrease(Increase)in Annual

Cost

÷

# of TargetEmployeesResponding

toQuestion

=

AverageCost

SavingsPer TargetEmployee

NA

NA

NA

NA

1,050

980

34

222

1,650

2

4

2

10

$ 0.00

$ 15.00

$ 168.10

$ 250.20

$ 1,365.00

$ 2,363.53

$ 146.20

$ 1,953.60

$ 5,115.00

$ 23.44

$ 228.67

$ 806.00

$ 1,881.00

$ 1,327.43

$15,643.67

20

20

20

20

20

20

20

20

20

20

20

20

20

NA

NA

NA

NA

$ 26

$ 41

$ 86

$ 176

$ 62

$ 211

$1,029

$8,060

$3,761

x

x

x

x

x

x

x

x

x

=

=

=

=

=

=

=

=

=

=

=

=

=

÷

÷

÷

÷

÷

÷

÷

÷

÷

÷

÷

÷

÷

=

=

=

=

=

=

=

=

=

=

=

=

=

19 Roberts Enterprise Development Fund. SROI Methodology: Analyzing the Value of Social Purpose Enterprise Within a Social Return onInvestment Framework. San Francisco: The Fund, 2001, p.32.


Social value is defined as the outcomesfor society as a whole that are positive,but cannot legitimately be given monetaryvalue. For these measures, alternativemethods are used to generate some usefulmeasure of return in nonmonetary terms.For example, a quality of life survey is partof the data collection method for employeesand clients of the grantee organizations(see Figure 6 below).

While such a survey does not yieldmonetized values for an investment return,it does provide meaningful evidence ofsome social benefit. It has all the usuallimitations of survey research data, butcan supply evidence of social valuecreation that would not be available byother means.

Figure 6 - Survey Sample for Social Return Measurement 20

20 Roberts Enterprise Development Fund. SROI Methodology: Analyzing the Value of Social Purpose Enterprise Within a Social Return onInvestment Framework. San Francisco: The Fund, 2001, p.72.

How I Feel About My LifeStrongly Agree a Neither Agree Disagree a Strongly No

Agree Little nor Disagree Little Disagree Answer

There are a lot of people I like � � � � � �to hang out with.

I like to get together with friends � � � � � �as much as possible.

I have people in my life whoreally care about what’s � � � � � �happening to me.

If for some reason I were put injail, there are people I could call � � � � � �who would bail me out.

If I were sick or hurt and and Ineeded someone to take me to � � � � � �the hospital, I would have notrouble finding someone.

If I were hungry and had nomoney to buy food, there are � � � � � �people I know who would giveme food.

If I were in trouble and somepeople were going to try to hurt � � � � � �me, there are other people Icould get protection from.


Measuring value and returns

The SROI methodology employs six stages.The first three deal with measuring valueand the last three deal with the index ofreturn.

� Stage 1: Calculate enterprise value(uses standard accounting valuemeasures).

� Stage 2: Calculate social purposevalue (assigns monetary values tosocial outcomes).

� Stage 3: Calculate blended value

� Stage 4: Calculate enterprise index ofreturn.

� Stage 5: Calculate social purpose indexof return.

� Stage 6: Calculate blended index ofreturn.

An example of the overall results of thesecalculations is shown in Table 6 on page 44.

These return calculations and thesupporting material are key components ofthe SROI reports produced by the granteeorganizations on an annual basis. Thereports include SROI metrics, businessdata, social impact data, and provideanalysis in these areas. Report contentsinclude:

� descriptions of the social purposeenterprise,

� financial analysis of the social purposeenterprise,

� profiles of the enterprise’s targetemployee population,

� SROI metrics and analysis,

� description of the nonprofit agency andits mission, and

� key social impact findings and analysis.

Overall, the SROI Report can be viewedas a nonprofit organization stock report. Itprovides a standardized way of estimatingvalue and presenting return calculations ina clear and accessible manner.

Limitations in the SROIapproach

Some of the comparative indicatorsavailable for evaluating for-profitorganizations are not available for thisSROI approach. There are no comparableindustry ratios and analyses for thenonprofit sector, although research in thissector is growing. In addition, the logic ofmanagement in for-profit organizations isdifferent in crucial ways. Profit-maximizationstrategies are not necessarily usefulor productive in the nonprofit sector.Consequently, careful attention should begiven to assessing the appropriateness ofthe standard financial measures to thisalternative use.

It is useful to consider the Fund’sdescription of lessons learned over thecourse of applying the SROI methods.21

The SROI analysis process is resourceintensive. The average practitioner must beaware of the financial and human resourcesnecessary when conducting an SROIanalysis of their social purpose enterprise.Considerable resources were providedto support the work in the granteeorganizations.

Engaging the practitioner is essential.This is not a “top down” process. It isimperative that practitioners themselvesdrive the process of identifying andsetting the social indices by whichthey will assess the value of their life’swork and the returns generated by theinvestments they receive.

21 Roberts Enterprise Development Fund. SROI Methodology: Analyzing the Value of Social Purpose Enterprise Within a Social Return onInvestment Framework. San Francisco: The Fund, 2001, p.62


Table 6. Social Return on Investment Results 22

22 Sample SROI results accessed from Roberts Enterprise Development Fund’s Website at www.redf.org/pub_sroi.htm#methodology.

SROI Results

SROI Metrics Index of Return

Enterprise Value $411,906 0.93

Social Purpose Value $20,861,055 47.14

Blended Value $21,222,960 47.96

1999

Investment to Date $442,543

Social Purpose Results (Per Target Employee) 1999

Public Savings $15,644

New Taxes $1,815

Wage Improvement $12,097

Financial Improvement $9,849

Enterprise Financials 1998 1999 2000P

Sales $233,004 $537,789 $708,957

Gross Margin 70% 69% 65%

Operating Margin (Before S&S) 5% -4% -4%

Operating Margin (After S&S) 82% 10% 0%

Projected Values (1999 into Perpetuity)

Total Projected Investment $575,775

Total Projected Social Savings and New Taxes $22,434,361

Total Projected Social Expenses $1,573,306

Total Projected Contribution to Parent $ 0


Enterprise architecture

Enterprise Architecture Development Tool-Kit, Version 2, National Association of State ChiefInformation Officers, July 2002. https://www.nascio.org/hotIssues/EA/.

The Business Reference Model, Version 1.1, Federal Enterprise Architecture ProgramManagement Office, 2002. http://www.feapmo.gov/feaBrm.htm.

State-specific Enterprise Architecture initiativesConnecticut: http://www.doit.state.ct.us/policy/domain/index.htmKentucky: http://www.state.ky.us/kirm/arcstand.htmNorth Dakota: http://www.state.nd.us/ea/

Stakeholder analysis

Making Smart IT Choices. Center for Technology in Government. University at Albany-SUNY.http://www.ctg.albany.edu/resources/abstract/absmartit.html.

John M. Bryson. Strategic Planning for Public and Nonprofit Organizations, Rev. Ed. SanFrancisco: Jossey-Bass, 1995.

Business process analysis and modeling

Business Process Resource Center, University of Warwick, UKhttp://bprc.warwick.ac.uk/umist1.html, http://bprc.warwick.ac.uk/index.html#BPRCTOP.

The Workflow Handbook 2002. Published in association with the Workflow ManagementCoalition (WfMC) Edited by Layna Fischer. March 2002, 428 pages. ISBN 0-9703509-2-9.

Workflow And Reengineering International Association http://www.waria.com.

Workflow/BRP application vendors: http://www.waria.com/databases/wfvendors-A-L.htm.

Business Process Management Journal: http://www.emeraldinsight.com/bpmj.htm.

Formal modeling

Agent-based modeling

Center for Computational Analysis of Social and Organizational Systemshttp://www.casos.ece.cmu.edu/home_frame.html.

Gaylord, R.J. and D’Andria, L. (1998), Simulating Society: A MATHEMATICA Toolkit forModelling Socioeconomic Behavior. Springer: New York, NY.

Appendix DAdditional Resources


Ilgen, D. R., & Hulin, C. L. (Eds.) (2000). Computational modeling of behavior in organizations:The third scientific discipline. Washington, DC: American Psychological Association.

Michael J. Prietula, Kathleen M. Carley & Les Gasser (Eds.), (1998) Simulating Organizations:Computational Models of Institutions and Groups, Menlo Park, CA: AAAI Press/The MIT Press.

Thomas Schelling, “Dynamic Models of Segregation,” Journal of Mathematical Sociology 1(1971), 143-186.

Thomas Schelling, Micromotives and Macrobehavior. New York: WW Norton & Co. Inc., 1978.

UML modeling

Object Management Group. OMG Unified Modeling Language Specification, Version 1.3.Framingham, MA: OMG Headquarters, 1999.

Doug Rosenberg, Kendall Scott (Contributor). Use Case Driven Object Modeling With UML:A Practical Approach (Addison Wesley Object Technology Series). New York:Addison-Wesley, 1999.

Workflow modeling

Thomas W. Malone, et al. Tools for inventing organizations: Toward a handbook oforganizational processes. Management Science 45(3) pp 425-443, March, 1999.

MIT Sloan School Center for Coordination Science. http://ccs.mit.edu.

Workflow Management Coalition. http://www.wfmc.org.

INSEAD Workflow Site.http://www.insead.fr/CALT/Encyclopedia/ComputerSciences/Groupware/Workflow/.

Operations research

Michael Trick’s Operations Research Page. http://mat.gsia.cmu.edu/INFORMS: Home Page: Institute for Operations Research and the Management Sciences.www.informs.org/.

Operations Research Society. www.mors.org/.

MIT, Operations Research Center. http://Web.mit.edu/orc/www/.

46

Return on Investment In Information Technology - …...Return on Investment In Information...

Documents

Transcript of Return on Investment In Information Technology - …...Return on Investment In Information...