ROI Analysis 1

ROI Analysis of Software Process Improvement, Online Education, and Telecommuting

David F. Rico

ROI Analysis 2

Introduction

Return on investment or ROI is a widely used approach to measure the economic value of

an investment or portfolio of investments, capital improvements, and even organization change.

ROI is increasingly used to measure the economic value of software process improvement or

SPI, online education or asynchronous learning networks, and telecommuting or teleworking.

However, the ROI of software process improvement, online education, and telecommuting is not

well known, along with appropriate ways of estimating the economic value of these paradigms.

Software process improvement, online education, and telecommuting are major trends, which

affect the economy, industry, and society as a whole, signifying the end of the industrial age.

Therefore, this paper examines the ROI of software process improvement, online education, and

telecommuting, and the hard economic evidence behind these major transformational trends.

Additionally, this paper briefly explains return on investment, and provides an in-depth analysis

of the management challenges and solutions for the ROI of software process improvement.

ROI is the ratio of adjusted economic benefits to costs, expressed as a percentage. That is,

the economic benefits, less the costs, are divided by the costs, and multiplied by 100 percent.

ROI is a measure of the magnitude of economic benefits to costs, economic benefits returned

above costs, profits returned above expenses, or simply, the value of one or more investments.

ROI is a standard tool for analyzing the costs and benefits of a portfolio of capital improvements.

Costs, benefits, and ROI are usually monetized, and expressed in economic and monetary units.

In other words, ROI is a measure of economic value, which is expressed in a relevant currency.

Benefits, which are often based on esoteric statistical models, must be identified and monetized.

Costs must be accumulated in painstaking detail to provide the most accurate economic picture.

ROI is calculated by dividing the benefits less costs by the costs, and multiplying by 100 percent.

ROI Analysis 3

Part A—Major Trends in Technology and Information Systems

ROI of Software Process Improvement

Software process improvement is an outgrowth of the field of software engineering,

which was created circa 1968. Software engineering is the professionalization of computer

science, and is aimed at successfully managing and engineering large scale, software-intensive

systems. Software process improvement is the discipline of perfecting the processes, people, and

technology associated with the field of software engineering. There are an infinite variety of

software engineering methods, as well as an infinite variety of software process improvement

approaches. Since 1968, a small body of literature quantifying the costs, benefits, and ROI of

software process improvement methods has been gradually emerging. Usually, a management

scientist creates a new software process improvement approach, and then touts its benefits.

Rarely, are systematic benefits identified, categorized, and scientifically investigated. And, even

more seldom, are software process improvement benefits monetized, or converted to currency,

and expressed in terms of ROI. And, few studies exist, which systematically compare the ROI of

software process improvement methods, along with other quantitative and economic

performance characteristics.

One of the first major studies of the costs, benefits, and ROI of software process

improvement was performed by McGibbon (1996). McGibbon’s study identified and compared

the costs, benefits, and ROI of the software inspection process, software reuse, and the clean

room methodology. The greatest contribution of McGibbon’s study was a technique to quantify

and monetize the benefits of various software process improvement methods. He adapted the

time proven method of total life cycle cost analysis to the field of software process improvement.

In doing so, McGibbon unlocked the secrets of ROI analysis for software process improvement,

ROI Analysis 4

and opened the door to ROI analysis for a variety of past, present, and future methods. Leaning

heavily on detailed cost and benefit analyses, McGibbon’s study revealed an ROI of 7,162% for

inspections, 391% for software reuse, and 3,267% for clean room.

About the time McGibbon’s (1996) work appeared, Grady (1997) published one of the

first textbooks exhibiting a broad range of costs and benefits for software process improvement.

He identified cost reductions of 6% for product definition improvement, 7.5% for detailed design

methods, 8.5% for rapid prototyping, 8.5% for systems design improvements, 14% for

inspections, and 22.5% for software reuse. He also identified cost reductions of 3.5% for

complexity analysis, 6.5% for configuration management, 3.5% for certification processes, 4.5%

for software asset management, and 5% for program understanding.

Rico (2000) extended McGibbon’s (1996) methodology in order to evaluate a broad

range of software process improvement methods from the past, present, and future. This study

examined the costs, benefits, and ROI of PSPsm, clean room, software reuse, defect prevention,

inspections, software testing, SW-CMM®, and ISO 9001. Benefit/cost ratios included PSP—

1,290:1, Cleanroom—27:1, Reuse—3:1, Defect Prevention—75:1, Inspection—133:1, Test—

9:1, CMM—6:1, and ISO 9001—4:1.

Rico (2002) reframed earlier results using scholarly metrics and models for ROI. This

study quantified costs and benefits in greater detail, and used conservative assumptions. This

resulted in overly conservative ROI values of Inspection—3,555%, PSPsm—3,104%, TSPsm—

$1,351%, SW-CMM®—1,330%, ISO 9001—739%, and CMMI®—425%. He implored his

readers to pinpoint high ROI factors, target high ROI approaches, minimize cost incurrence,

avoid cost intensive approaches, avoid training intensive approaches, look for low cost

automated solutions, and use professional methods for analyzing ROI.

ROI Analysis 5

Rico (2003), once again, reframed his methods, this time using net present value to

temper optimistic ROI values, and added detailed break even point estimates. ROI estimates

plunged for Inspection—2,542%, PSPsm—3,217%, TSPsm—2,192%, SW-CMM®—661%, ISO

9001—158%, and CMMI®—114%. He simply said choose a simple set of metrics and models

for SPI, don’t spend a lot of money to achieve a high ROI, don’t go bankrupt using expensive

methods with a high ROI, and seek low cost automated SPI methods with a high ROI. He also

said don’t wait until you’re SW-CMM®/CMMI® Level 5 to use ROI, don’t spend too much

money measuring ROI, proactively plan for ROI, use ROI to right size a SPI program for your

organization and overhead budget, and apply metrics and models for ROI before it’s too late.

Rico (2004) formulated recommendations for future software process improvement based

on detailed cost and benefit analysis of existing software process improvement methods. The

future methods included trainingless methods, automated workflow tools, low-cost commercial

tools, non-invasive measurement, expert system workflow tools, self documenting approaches,

and boundaryless virtual teams.

The impacts of applying the ROI of software process improvement include forcing

managers to apply quantitative methods, and collect and analyze cost and benefit data. This

implicitly and explicitly forces managers to study, learn, and apply scientific management

principles. This often involves continuing education in management science. Managers are now

faced with identifying alternative approaches and principles in scientific management. More

importantly, managers are faced with comparing and contrasting the qualitative and quantitative

costs and benefits of alternative methods in scientific management. Not only must managers

learn and apply scientific management principles, but they must also study, master, and apply

quantitative decision analysis methods for cost, benefit, and return on investment analysis.

ROI Analysis 6

ROI of Online Education

Online education is currently embodied by the use of the Internet, World Wide Web, and

technology to enhance the instructional and learning interaction between teacher and student.

Online education is also known as online learning, e-learning, asynchronous learning networks,

distance education, learning technologies, online course delivery, telelearning, and other names.

Early manifestations of technology assisted learning included phonographs, radio, tape

recordings, television, videotapes, interactive video disks, mainframes, minicomputers,

workstations, personal computers, computer based training, simulations, and video games. The

objectives of technology assisted learning included cost effective educational delivery, just-in-

time training, mass educational services, and remote educational services. Objectives also

included consistency of service, quality, delivery, productivity, and optimal learning experience.

Allen and Seaman (2003) define online learning as an online course, which has at least 80% of

its course content delivered online, and consists of little or no face-to-face meetings. Studies

quantifying the costs, benefits, and ROI of online learning started gaining momentum circa 1999,

and then poured in steadily during 2000, 2001, 2002, and 2003.

According to Allen and Seaman (2003), nearly two million students took online courses

in 2002. Almost 600,000 students took all of their courses online. There are signs that the growth

in online courses may approach 20% per year. 80% of all institutions offer at least one online

course, and 34% offer online degree programs. 97% of public institutions offer at least one

online course, and 50% of those offer an online degree program. Nearly 60% of institutional

leaders believe in online education, and the same amount believes online education is equal to or

superior to traditional course delivery. These statistics indicate that the tide is turning in favor of

online education, in just a few short years. This is no surprise.

ROI Analysis 7

Though not exclusively intended for evaluation and assessment of online education,

Phillips (1997) provides an interesting framework for measuring the ROI of training programs.

Phillips’ five level ROI model for evaluating training programs consists of measuring student

reaction, student learning, student application, business impact, and return on investment.

Reaction measures participant reaction to the program and stakeholder satisfaction with the

program and the planned implementation (e.g., end-of-course questionnaire/student survey).

Learning measures skills, knowledge, or attitude changes related to the program and

implementation (e.g., in-process evaluation, quizzes, term papers, oral evaluations, presentations,

and end-of-course exam). Application measures changes in behavior on the job and specific

application and implementation on the program (e.g., surveys, interviews, focus groups, in-

process measurements, audits, and compliance analysis). Impact measures business changes

related to the program (e.g., cost, quality, productivity, cycle time metrics and models). Return

on Investment compares the monetary value of the business impacts with the costs for the

program (e.g., cost, benefit, benefit/cost ratio, and return on investment). The weakness of

Phillips’ model is that it is laborious, manual, expensive, invasive, and scientifically complex.

Bishop and SchWeber (2001) have identified four quality indicators of online education,

student support, faculty support, curriculum development and delivery, and evaluation and

assessment. Student support consists of library reference services, administrative services,

bookstore services, and information technology support. Faculty support consists of recruiting,

admissions, class size, training, and information technology support. Curriculum delivery

consists of course content analysis, curriculum delivery, critical thinking, and application of web

resources. Evaluation and assessment consists of course evaluations, program reviews,

longitudinal assessments, faculty course assessments, and alumni surveys.

ROI Analysis 8

Estabrook (2001) presented the costs and benefits of implementing an asynchronous

learning network at the University of Illinois Graduate School of Library and Information

Science from 1996 to 2002. The seven year cost amounted to $3,657,800, and the average annual

cost was $522,543. Cost categories included, faculty, adjunct faculty, faculty maintenance,

summer money, assistant dean, technical support, graduate and teaching assistants, mentors,

evaluation, equipment/software, telecommunications, travel, supplies, mail, office, clerk-typist,

admissions clerk, promotion, coordinator, special activities, academic outreach, contingency, and

campus charges for services. Quantitative benefits included 14.9% higher salaries by program

graduates, a 2,952% return on investment by students, and a program growth of 330%.

Qualitative benefits included faculty skill modernization, improved quantity and quality of

curriculums, more courses offered for dollar invested, information technology infrastructure was

modernized more frequently, and the student retention rate improved.

Wentling and Park (2002) conducted a survey of the costs and benefits of e-learning.

Some of their reported benefits included student break even points ranging from 20 to 112

students. They also reported a return on investment ranging from 228% to 3,283%.

The impacts of applying the ROI of online education consist of comparing the costs and

benefits of online learning with those of traditional methods in training and education. Managers

must determine when it is appropriate to use traditional instruction methods, online education, or

a mix of educational alternatives. Managers must now budget for these alternatives, and this may

or may not include broad based fluctuations in infrastructure budgeting and cost accounting.

Managers are faced with the decision to budget for traditional training infrastructures versus

outsourced online education. And, they are faced with establishing and applying performance

standards for hiring and retention of employees educated using traditional versus online methods.

ROI Analysis 9

ROI of Telecommuting

Telecommuting is defined as the use of information technology as a substitute for

commuting to and from work using traditional transportation. Telecommuting is a proper subset

of teleworking, which is a form of distributed or decentralized work. Teleworking or

decentralization is an alternative work style for the information age versus the industrial age,

which is characterized by agriculture, administrative bureaucracies, and manufacturing. Modern

workers can now access information remotely or electronically, and are no longer required to

commute or drive to a central location to access their tools, data, and raw materials. Teleworking

eliminates dangerous commutes, inflexible and imprecise work schedules, unproductive

meetings, work related stress, expensive office space, inability to attract top talent, and chronic

automobile pollution. Teleworking increases productivity, organizational flexibility, response

times, morale, management capability, and personal flexibility. Teleworking reduces turnover,

eliminates office space, lowers real estate costs, results in a cleaner environment, and reduces

energy consumption.

Nilles (1998) provides a broad survey of the costs and benefits of teleworking, primarily

focusing on the factors surrounding home based telecommuting. Nilles presents the costs and

benefits of teleworking, namely work or social changes, employee profitability, decreases in

pollution and energy consumption, typical cost drivers, and qualitative and quantitative benefits.

Increases in work or social changes include, liberation—320%, continuity—170%, creativity—

190%, personal life—150%, environmental influences—160%, general work life—110%, stress

avoidance—90%, interdependence—50%, visibility—50%, belonging—30%, and

apprehension—10%. Teleworking increases employee profitability by more than 30%. Annual

decreases in pounds of pollution include, 1,000 people—1,720,089, 10,000 people—17,200,893,

ROI Analysis 10

100,000 people—172,008,929, 1,000,000 people—1,720,089,286, 10,000,000 people—

17,200,892,857, and 100,000,000 people—172,008,928,571. Annual decreases in gallons of

gasoline include, 1,000 people—1,560,000, 10,000 people—15,600,000, 100,000 people—

156,000,000, 1,000,000 people—1,560,000,000, 10,000,000 people—15,600,000,000, and

100,000,000 people—156,000,000,000. The cost drivers of teleworking consist of additional

training, telecommunications equipment, computers, moving expenses, facility leasing,

construction costs, furniture, insurance, miscellaneous rentals, project administration, additional

travel, and liability. The benefit factors of teleworking consist of increased employee

effectiveness, decreased sick leave, decreased medical costs, increased organization

effectiveness, decreased turnover, reduced parking requirements, office space savings, and

recruiting effectiveness. Quantitative benefits (less costs) of teleworking consist of 1,000

people—$8,417,708, 10,000 people—$84,177,083, 100,000 people—$841,770,833, 1,000,000

people—$8,417,708,333, 10,000,000 people—$84,177,083,333, and 100,000,000 people—

$841,770,833,333.

Shafizadeh, Niemeier, Mokhtarian, and Salomon (1998) conducted a broad survey of

major studies surrounding the costs and benefits of telecommuting. Their study identified critical

factors, telecommuting cost factors, telecommuting benefit factors, annual savings, and annual

benefits. Telecommuting cost factors included, marketing/training development, evaluation,

ongoing marketing/training, latent demand realization, urban sprawl, planning,

marketing/training, equipment, internal program administration, marketing/recruitment, and

training. Other cost factors included, equipment maintenance/replacement (less salvage),

communication, decreased workplace interaction/immediate access, security of data, equipment,

software, stress to perform, communication costs, utility costs, space costs, decreased workplace

ROI Analysis 11

interaction, loss of support services, and loss of boundary between work and home.

Telecommuting benefit factors included, travel reduction (direct), emission reduction (direct),

improved highway safety, increased economic development (employment opportunities for

underemployed/mobility limited labor segments), increased neighborhood safety, and space cost

savings (office and parking). Additional benefit factors included, recruitment (access to best

talent and broader labor markets), improved retention, increased productivity, less absenteeism,

less sick leave, longer hours, fewer distractions (greater productivity per hour), improved

customer service, disaster recovery, and public relations, compliance with air quality and trip

reduction regulations, and travel time/stress savings. Still, more benefit factors included, travel

cost savings, other cost savings, personal flexibility, reduced work-related stress, ability to get

more/better work done, ability to work while mobility limited or physically distant from

workplace, and more time with family. Shafizadeh’s, Niemeier’s, Mokhtarian’s, and Salomon’s

literature survey identified annual savings of 3.5 billion gallons of gasoline per year, 3.1 billion

hours of personal time saved, 1.8 million tons of pollution avoided, and $500,000 in

transportation maintenance avoided. Annual benefits per telecommuter consisted of, vehicle

miles avoided—1,583 to 3,540, gallons of fuel saved—46 to 177, fuel cost savings—$51 to

$280, state and federal taxes avoided—$14 to $32, grams of carbon monoxide not emitted—

3,398 to 120,415, grams of nitrogen oxide not emitted—620 to 6,054, and grams of

hydrocarbons not emitted—521 to 18,166. Annual benefits per person also included, grams of

particulate matter (brake dust) not emitted—33 to 243, monetized carbon monoxide not

emitted—$1 to $90, monetized nitrogen oxide not emitted—$1 to $55, monetized hydrocarbons

not emitted—$2 to $23, monetized particulate matter (brake dust) not emitted—$5 to $34, time

in hours saved—5 to 174, and infrastructure savings—$45 to $69.

ROI Analysis 12

Shafizadeh, Mokhtarian, Niemeier, and Salomon (2000a and 2000b) also conducted a

broad survey of minor studies on the costs and benefits of telecommuting. They identified typical

cost factors, such as telecom equipment, telecom software, telecom services, telecom

maintenance, training, employees, utilities, and miscellaneous costs. Typical benefit factors

consisted of work productivity, reduced absenteeism, parking space benefits, office space

benefits, recruitment/retention, commuter cost savings, and miscellaneous user benefits. Annual

costs per telecommuter consisted of training—$68.86 to $300, equipment—$149.50 to $1,108,

software—$17.50 to $210, phone installation—$4.72 to $12.73, phone services—$9.67 to $360,

work related travel—$0.02 to $1.11, equipment maintenance—$25 to $47, furniture—$12.50 to

$157, and other costs—$18.29 to $219. Annual benefits per telecommuter included travel

savings—$36.21 to $243.39, miscellaneous savings—$80.14 to $122.55, increased

productivity—$3,815, reduced absenteeism—$200, and reduced office space—$1,440.

The impacts of applying the ROI of telecommuting include determining whether your

business or enterprise model supports the concept of teleworking. Depending on the size of your

organization, a mix of teleworking alternatives must be considered. The laws, statutes, and

governance which apply to the business model must be considered. For instance, some

enterprises require employees to be U.S. citizens. Some enterprises may or may not be able to

extend employment across international borders for privacy or proprietary purposes, protection

of intellectual capital, and competitive trade purposes. In addition, international tariff laws, trade

agreements, tax laws, and visa regulations are impacted. More importantly, industrial age

attitudes must be changed, which favor in-person employment arrangements, to ones allowing

the creation and maintenance of virtual international enterprises. Managers are now faced

mastering and applying scientific principles for managing teleworkers effectively.

ROI Analysis 13

Part B—In-Depth Challenges and Solutions to ROI of Software Process Improvement

Cost and benefit factors. One of the most pervasive challenges is identifying significant

drivers of costs and benefits. There are a myriad of software process improvement methods, and

their cost and benefit drivers vary widely. Managers often ignore costs in favor of qualitative

benefits, or consider too few cost drivers such as training fees. Managers often ignore

comprehensive cost analyses. Worse yet, benefits are poorly understood, seldomly accepted, and

ignored by all but a few scholars studying the ROI of software process improvement.

Managers must receive more education and training on how to identify and apply

significant cost and benefit factors for software process improvement. The scholarly community

must make a concerted effort to make up for the short fall of quantitative decision making

information by conducting more research and publishing the results. Additionally, managers

must be taught how to wade through the myriad of cost and benefit drivers, and isolate the ones

that will help them optimize their performance and achieve their enterprise objectives.

Analysis of alternatives. Managers almost never choose a method for software process

improvement based on its costs, benefits, and ROI. Oftentimes, methods are chosen for their

popularity, familiarity, ubiquity, and qualitative benefits. Rarely, are methods chosen because

they are the least cost, most beneficial, or because they offer the greatest ROI.

Managers must be taught to apply methods for return on investment to exploit cost and

benefit factors to accomplish their business objectives. Managers must be taught to define and

differentiate methods for software process improvement based on their costs and benefits.

Managers should select methods for software process improvement based on their business

objectives. Managers should know how to exploit software process improvement, not be

exploited by inferior methods for software process improvement.

ROI Analysis 14

Analysis of costs. Some of the most popular methods for software process improvement

are expensive, cost prohibitive, and economically unviable for small to medium enterprises.

Government organizations and large enterprises requiring supplier services continue to require

small to medium enterprises to comply with overly bureaucratic and cost intensive methods.

Managers rarely use tools to quantify costs and determine the impacts to the enterprise.

Managers should be taught to apply quantitative methods for cost analysis, in order to

determine the consequences of management decisions, particularly the selection of methods for

software process improvement. Managers should choose methods based on the economic posture

of their organizations, and not over extend their resources. Scholars are bound by the

responsibility of identifying the costs of software process improvement. Scholars are also bound

by the responsibility of not only inventing new methods for software process improvement, but

presenting a cost analysis of their methods. Together, managers will be enabled to make sound

capital improvements based on a thorough understanding of the costs involved.

Training requirements. A few emerging methods for software process improvement

require hundreds of hours and months of training at a cost of $20,000 to $50,000 per person.

Large enterprises can’t afford the costs of training, and small to medium enterprises can’t either.

Most managers intuitively shy away from training intensive methods. However, a few large

government agencies continue to commit large capital investments to training intensive methods.

Managers of small to medium enterprises don’t seem to stumble over expensive training

intensive approaches to software process improvement. Large government standards don’t come

with a price tag. Therefore, managers are vulnerable to overextending their resources when

applying these models. However, training always comes with a price tag. Managers of small to

medium enterprises aren’t as vulnerable to expensive training intensive methods. However,

ROI Analysis 15

managers of large enterprises with large budgets are vulnerable to expensive methods. All

managers should avoid expensive methods for software process improvement, and gravitate

toward highly effective inexpensive approaches.

Manual approaches. While automated tools for software process improvement have been

sought since 1970, the best methods for software process improvement are manually intensive.

That is, they hinge on the ability to change, shape, and form the behavior of human beings. In

other words, manual methods are based upon conditioning human beings to consistently apply

principles in scientific management, such as project, quality, and reliability management. In spite

of well meaning approaches, humans aren’t machines and are incapable of such consistency.

The field of software process improvement should take a lesson from the discipline of

online education. Academic institutions recognize the inexpensive power of modern information

technologies for delivering instructional content to the largest number of students at the least

possible cost. Managers of software process improvement initiatives should do likewise. They

should dismiss their skepticism toward automation, and embrace inexpensive and powerful

technologies as a substitute for expensive, manually intensive methods for software process

improvement.

Methods for decision analysis. In spite of their age, methods for decision analysis aren’t

in wide spread use. Additionally, there are dozens if not hundreds of methods for decision

analysis. While ROI is growing in popularity as a decision analysis tool for analyzing software

process improvement alternatives, ROI is still not the method of choice for such purposes. At the

other extreme, some ROI methods are too labor intensive, require years of expensive education,

and sometimes require millions of dollars to apply.

Managers must be taught to routinely apply quantitative methods for decision analysis for

ROI Analysis 16

software process improvement. Furthermore, managers should accept return on investment as a

method for quantifying the business value of alternative analysis. Managers should dismiss

preconceived notions of esoteric methods for decision analysis, and apply mainstream methods

for return on investment. Managers should apply cost analysis, benefit analysis, net present

value, benefit/cost ratio, return on investment, and break even point for evaluating alternatives to

software process improvement. Managers must also be cautious of methods for return on

investment that are more expensive than the method for software process improvement itself.

Metrics for return on investment. The base of scholarly literature on the application of

return on investment principles is sparse, hard to find, and not ubiquitous. The literature that does

exist, exhibits a broad range of disparate metrics and models for return on investment.

Oftentimes, the literature displays dozens of esoteric metrics, models, and equations. Sometimes,

scholars invent their own formulas for return on investment to exhibit their mathematical

prowess, and to patent their own unique formulas. This only confuses managers, and hinders the

application of return on investment for software process improvement.

Scholars and management scientists must take responsibility for creating an industry

standard set of metrics and models for return on investment of software process improvement.

There are just too many metrics, models, and methods from which to choose. Managers are far

too challenged to keep pace with rapidly changing information technologies. They simply cannot

keep up with advances in management science. Scholars can help managers by steering them

towards cost, benefit, net present value, benefit/cost ratio, return on investment, and break even

point analysis using simple mainstream methods.

Cost of achieving return on investment. Because the most popular methods for software

process improvement are often the most expensive approaches, managers tend to believe they

ROI Analysis 17

must spend a lot of money to achieve some level of return on investment. This is often a self

defeating concept. Return on investment is the magnitude of benefits to costs. If costs are small,

return on investment is large if benefits are large. If costs are high, return on investment is

generally lower. Managers often spend too much money to achieve a negligible return on

investment.

Managers must be taught to realize that the goal is to achieve an optimal return on

investment, not achieve compliance with expensive international standards for software process

improvement. That is, they must be taught to apply methods for software process improvement

with high benefits and low costs. The scholarly community is also responsible for creating

methods for software process improvement with high benefits and low costs. Scholars have a

place of privilege and responsibility, and should teach, lead, and guide less skilled managers with

superior methods for software process improvement.

Cost of software process improvement. Today’s methods for software process

improvement are very expensive, and often range in the millions of dollars. In fact, the most

popular method for software process improvement ranges from five to fifteen million dollars to

apply. Software process improvement is very difficult, and can require up to five attempts before

achieving a single instance of success. So, their costs often manifest themselves in multiple

applications of the methods. These methods are cost prohibitive for small to medium size

enterprises, and may very well place large enterprises at risk of financial instability.

Managers must be aware that blindly adopting a method for software process

improvement, in spite of its cost, may place their enterprises at risk, instead of strengthening

them. Managers should apply inexpensive versus expensive methods for software process

improvement. There are classes of methods, which are inexpensive, yet highly effective.

ROI Analysis 18

Cost of automation. There are a few popular tools to support software process

improvement, and they are growing in popularity. Automated tools became very popular in the

1970s, and once again in the 1980s. Early tools were hindered by lack of maturity, and the

practical limitations of primitive information technologies. Later tools were hindered by over

selling of their capabilities, and limited capability to add value to software process improvement.

However, there are a few good tools, which are popular among medium to large enterprises.

Oftentimes, these tools cost hundreds of thousands, and even millions of dollars to purchase and

apply. Managers many times believe that large capital investments in expensive tools must be

made, in order to achieve some level of business value.

Managers should be taught to realize that automation is vastly superior to manual

methods for software process improvement, and there are oftentimes inexpensive solutions.

Managers must realize that expensive tools are sometimes unnecessary, and inferior to less

expensive solutions. They must realize the goal is to minimize expense and maximize benefit.

Difficulty of return on investment. Return on investment requires the application of

metrics and models. Software measurement is not in wide spread use by the general body of

managers. Metrics and models are usually relegated to a few well placed scholars, management

consultants, and high maturity organizations. Worse yet, statistical analysis is not considered a

necessary component of the most popular approaches to software process improvement. So,

managers don’t generally apply return on investment, relegate it to the field of scientific study,

and justify ignoring it because mainstream methods don’t require its application.

Managers should be conditioned to apply quantitative methods for software process

improvement, especially metrics and models for software process improvement. Managers must

be taught to use quantitative methods for everyday decision making. They must overcome the

ROI Analysis 19

myths that metrics and models are only for special circumstances, such as large, expensive, and

mission critical systems. Managers must learn to apply metrics in all circumstances.

Cost of applying return on investment. Return on investment belongs to the discipline of

applying metrics and models. Software managers believe that software measurement requires the

application of advanced tools in statistics. And, of course, software managers believe software

measurement requires years of data collection, data points, and impeccable statistical

justification. Scholarly methods for determining return on investment also require large

investments in capital improvements to apply. Therefore, managers believe large amounts of

money must be invested in order to apply methods for return on investment.

Managers must learn that return on investment can be applied to software process

improvement early, easily, and inexpensively. They must overcome the myths that return on

investment is only for the scientific community. They must also realize it doesn’t require large

capital investments, and that they can perform early, top down estimates without years of data.

Timeliness of return on investment. Return on investment is oftentimes applied when it’s

too late to optimize costs and benefits. In other words, managers often spend millions of dollars

on software process improvement, and then they want to justify their expenditures after-the-fact

by looking for return on investment. Again, return on investment is the magnitude of benefits to

costs. If a manager spends too much money, return on investment may oftentimes be negligible.

Besides, return on investment is not always about minimizing costs, but maximizing benefits.

Since managers don’t apply return on investment, they oftentimes end up with little or no

benefits. So, if managers garner few benefits and spend a lot of money, then return on investment

is sure to be negligible.

Managers must be taught to proactively plan for, and apply, return on investment for

ROI Analysis 20

software process improvement. They must choose methods for software process improvement

that are inexpensive and result in high benefits before they begin their initiatives. They must not

simply apply expensive methods with negligible return on investment, and hope for high benefits

after-the-fact. They must reverse their understanding of how to apply return on investment.

Strategic planning and return on investment. Return on investment requires the

application of cost and benefit data. Aren’t costs and benefits the crux of strategic planning? Isn’t

strategic planning all about maximizing value or benefits associated with a manager’s spending

dollar? Therefore, shouldn’t managers be attempting to get the largest number of benefits for the

least amount of cost? Oftentimes, managers devote their energies to getting the largest possible

budget, and then spending it on expensive methods for software process improvement with

negligible return on investment. Managers rarely set quantitative enterprise objectives for

optimizing benefits and minimizing costs. Managers of small to medium enterprises often spend

too much money on software process improvement. And, managers of large enterprises spend

even more money with little or no benefit.

Likewise, managers should use return on investment to right size a software process

improvement program for their enterprises and economic characteristics. They should form

enterprise strategic plans based on quantitative objectives to maximize benefits, while

minimizing expenses. Enterprises of all sizes should seek low-cost, yet highly effective methods

for software process improvement to achieve their peak operating efficiency. Small enterprises

should not over extend their resources. And, large enterprises should seek to achieve the greatest

number of benefits for the least cost. Large enterprises should no longer be satisfied to spend

their money frivolously, just because they have money to spend. Instead, managers of enterprises

of all shapes and sizes should use return on investment to optimize their capital expenditures.

ROI Analysis 21

References

Allen, I. E., & Seaman J. (2003). Sizing the opportunity: The quality and extent of online

education in the united states, 2002 and 2003. Needham, MA: The Sloan Consortium.

Bishop, T., & SchWeber, C. (2001). Linking quality and cost. In Elements of quality

online education, volume 3 in the sloan-c series. Needham, MA: The Sloan Consortium.

Estabrook, L. S. (2001). Rethinking cost-benefit models of distance learning. In Elements

of quality online education, volume 3 in the sloan-c series. Needham, MA: The Sloan

Consortium.

Grady, R. B. (1997). Successful software process improvement. Saddle River, NH:

Prentice Hall.

McGibbon, T. (1996). A business case for software process improvement (Contract

Number F30602-92-C-0158). Rome, NY: Air Force Research Laboratory—Information

Directorate (AFRL/IF), Data and Analysis Center for Software (DACS).

Nilles, J. M. (1998). Managing telework: Strategies for managing the virtual workforce.

New York, NY: John Wiley & Sons.

Phillips, J. J. (1997). Return on investment in training and performance improvement

programs. Houston, TX: Gulf Publishing Company.

Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement

(SPI) strategies (Contract Number SP0700-98-D-4000). Rome, New York: Air Force Research

Laboratory—Information Directorate (AFRL/IF), Data and Analysis Center for Software

(DACS).

Rico, D. F. (2002), How to estimate ROI for inspections, PSPsm, TSPsm, SW-CMM®, ISO

9001, and CMMI®. DoD Software Tech News, 5(4), 23-31.

ROI Analysis 22

Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers

and software engineers. Boca Raton, FL: J. Ross Publishing.

Rico, D. F. (n.d./2003). Practical metrics and models for return on investment [WWW

document]. URL http://davidfrico.com/rico03.pdf

Shafizadeh, K. R., Mokhtarian, P. L., Niemeier, D. A., & Salomon, I. (2000a). The costs

and benefits of telecommuting: An evaluation of micro-scale studies and other promotional

literature (UCB-ITS-PRR-2000-13). Davis, California: University of California Davis, Institute

of Transportation Studies.

Shafizadeh, K. R., Mokhtarian, P. L., Niemeier, D. A., & Salomon, I. (2000b). The costs

and benefits of home-based telecommuting (UCB-ITS-PRR-2000-20). Davis, California:

University of California Davis, Institute of Transportation Studies.

Shafizadeh, K. R., Niemeier, D. A., Mokhtarian, P. L., & Salomon, I. (1998). The costs

and benefits of telecommuting: An evaluation of macro-scale literature (UCB-ITS-PWP-98-5).

Davis, California: University of California Davis, Institute of Transportation Studies.

Wentling, T. L., & Park, J. (2002). Cost analysis of e-learning: A case study of a

university program. 2002 Academy of Human Resource Development Annual Conference

(AHRD ’02), Hawaii.