105

CHAPTER 5

ERP PROJECT LIFECYCLE –KEY PLANNING PHASE

ISSUES

5.1 INTRODUCTION

The research framework described in chapter 3, uses the Parr and

Shanks (2000) PPM model in Figure 3.1(a) and identifies planning,

implementation and enhancement as the three important phases in the

lifecycle of an ERP project. This chapter identifies some of the critical issues

involved in the planning phase and their influence on the final outcome of the

project. The key issues involved during this phase are (a) scope finalization

(b) ERP selection decision, (b) implementation partner selection (d) core team

selection and (d) steering committee formation. This chapter addresses the

first three of these which are the most crucial at this stage. Each of these add

to complexity and impact eventual project outcome.

Complexity of ERP projects has its genesis in the dynamics of the

implementation process. This is explored in this chapter using system

dynamics (SD) tools. The core paradigm behind SD is that the eventual path a

system takes is guided by the way in which elements of the system interact

with each other. This interaction results in patterns of system behavior that

may be counter intuitive and difficult to determine through a linear thought

process. An attempt is made to explore some of the critical dynamics and

draw inferences to guide project management practice.

This chapter illustrates the use of Analytical Hierarchy Process

(AHP) for the ERP selection decision. An example of one such exercise and

106

problems faced are described. Suggestions for improving the process and

extending it to other selection decisions are also given.

This chapter also suggests an improved methodology for scope

development and goal setting. An approach called the “Single Goal Set”

(SGS) method is explained. This method is expected to greatly improve scope

development and thereby positively impact the final outcome of an ERP

project.

5.2 THE DYNAMICS OF ERP IMPLEMENTATION

ERP implementation issues arise from the inherent complexity of

the ERP artifact in the context of its implementation. In Chapter 1 the concept

of “reciprocal dependencies” (Williams 2004) has been introduced. This

refers to the complexity that arises due to each element of the process being

an output or an input or both of another process. Such complexity makes it

difficult to assess how changing one element would affect another element

separated in time or geography. It becomes impossible to assess how the

overall system would react to a change in one or the other of the causal

factors because of the intertwined relationships between them. Sometimes the

system response to an act could be contrary to what one would intuitively

assume it to be.

An example of one such counter intuitive effect , in software

development, is what is known Brooks law which states that adding

manpower to a late software project actually delays the project rather than

speed it up (Brooks 1975). ERP projects, on account of reciprocal

dependencies, also exhibit such behaviour which runs counter to intuitive

wisdom.

One method that helps in the understanding of these complex

relationships is System Dynamics (SD) a way of thinking that was pioneered

by Forester (1961) and developed to include all manners of business problems

107

by Sterman ( 2000) and others. System Dynamics is “concerned with the

behaviour of complex systems …..(and).. is grounded in the theory of non

linear dynamics and feedback control in mathematics, physics and

engineering” (Sterman 2000 pp.4-5). Abdel- Hamid and Madnick (1991)

model a real life software project, the NASA-DE-A project using SD tools

and simulate the occurrence of the Brook’s phenomena. They show that

adding manpower does not always cause a negative outcome and that it

depends on the stage of the project.

System Dynamics offers the method of causal loop diagrams

(CLDs) an important and useful tool for representing the feedback structure of

systems. Positive feedback loops that move the situation forward are known

as reinforcing loops and negative feedback loops that hold a situation from

moving forward are known as balancing loops. The basic premise of SD is

that all systems are made up of a large number of reinforcing and balancing

loops that together impact the final direction of the system. Carefully

conceptualizing these loops and tracing their direction helps in the

understanding of complex systems. Such tools are needed as without these the

human mind is incapable of judging the movement of a complex system

beyond the second or third interconnected loop. SD methods help in modeling

large systems with innumerable loops.

Akkermans and Van Helden (2002) identify a two such loops and

illustrate vicious and virtuous cycles in case study of an ERP implementation

(Figure 5.1). The first reinforcing loop identified is the communications –

collaboration loop. Communication reinforces collaboration –poor

communication therefore inhibits collaboration. They opine that in the first

instance communications were restricted to the consultants and a “core group”

which resulted in weaker interdepartmental communication leading to poor

interdepartmental coordination. This is a vicious cycle as mistrust increases

inhibiting communication. Left unaddressed this would spiral out of control

resulting in a failed project.

108

This was halted through a countermeasure of a change in the

composition of the implementing team. Consultants who were purely

technical, were replaced by trained colleagues who were able to communicate

more effectively with users. This resulted in better communication and

consequently better collaboration. The reinforcing loop was thus turned from

a vicious one to a virtuous one. The second reinforcing loop identified by

Akkermans and Van Helden (2002) is the expectations management loop. The

researchers do not elaborate on this loop in their paper. We believe this is an

important loop as it has ramifications across the implementation process.

Figure 5.1 illustrates the CLD developed by Akkermans and Van

Helden.

Figure 5.1 Reinforcing loops in ERP implementation

(Source: Akkermans and Van Helden 2000)

109

There are several important intertwined loops in the process of an

ERP system getting implemented in an organization. CSF literature captures

these as events that occur at points of time. Causal loop representation helps

us to see the manner in which these CSFs interact and influence each other

and impact the outcome of the project.

Figure 5.2 gives an example of some of these interrelationships. We

identify some important loops: the expectations confirmation loop: This is a

balancing loop or negative loop. All negative loops have a goal –or the

desired state of the system (Sterman 2000 pp.155). The goal of this loop in

our case is the project scope. If it is higher expectations would be more, the

perceived gap in performance would be greater leading to dissatisfaction. This

impacts the Usage loop: If satisfaction is high usage is more and tendency to

revert to legacy systems is reduced. If satisfaction is low tendency to revert to

legacy systems is high and efforts to correct the system deficiencies is low. So

one sees the impact of this in both the corrective actions as well as in the

legacy systems influence loops. We also identify a cost loop. This is a

balancing loop with the budgeted cost as the system goal. With increased top

management support , additional resources could be sanctioned to close gaps .

This leads to a cost overrun which negatively impacts top management

support. Top management support also has influence on the use of legacy.

With greater top management support and advocacy for the ERP project the

tendency to revert to legacy systems is diminished.

In summary we see that SD offers us an effective tool to translate

our mental maps into CLDs. This facilitates a better understanding of how

interactions between factors, events and goals finally impact outcome. SD has

another central idea called the stock and flow structure and a programming

110

methodology that allows us to simulate the effect of changes in parameters on

system response. Stocks are accumulations and flows are what cause

accumulation or attrition of stock.

In this case, Satisfaction, Top Management Support , etc., are

stocks that keep changing on the basis of the flow of information regarding

goals attainment, modules completion , cost overrun , etc. There are inflows

as well as outflows- if gaps are reduced this information increases the stock of

top management support for the project. If costs increases, this information

depletes the stock of top management support. This powerful methodology

helps us model the impact of changes on key stocks.

When we started out this research we hoped to be able to model the

implementation process using SD and use the model to test the PVI scale. We

did not succeed in our attempt as we were not able to resolve the issue of units

of measure that could be used to model the implementation. SD requires that

there is unity of measure across the system. For example if Satisfaction was

the stock that we were tracking, what would be the flow – obviously

Satisfaction /unit time. How do we then model each of the causal factors to

yield the influence on this flow. We found this task daunting and restricted the

use of SD tools to understand the dynamics of the implementation rather than

model the whole process. We found that other researchers also followed the

same – not modeling the whole process but using SD analysis to understand

the dynamics . A case in point is the Akkermans and Van Helden study

(2002) that studies the virtuous and vicious cycles in ERP implementations.

Our objective in this chapter is to highlight the essential complexity

of an ERP implementation process.

111

Project

deadline goal

Actual Gap

+

Tendency torevert tolegacy

+

TopManagement

support

-

legacysystem

influence -

-

Corrective actions loop

Legay Systems Influence Loop

Cost

Overrun

-

Cost Loop

Perceived

gap

Satisfaction

-

-

Expectations

+

Efforts to CloseGap

-

-

Additional Resources

Sanctioned

+

+

PROJECT

SCOPE

+

+

BUDGETED

COST

+

Expectations - Confirmation Loop

+

Increased usage+

-

Usage Loop

-

Figure 5.2 Causal Loop Diagram – ERP assimilation process

112

5.3 ERP SCOPE FINALIZATION

We have seen that project scope is an important element that

contributes to the dynamics of an implementation. One of the key activities

during the planning stage of an ERP project is scope finalization. The

eventual success of an implementation is to a large extent dependent on the

effectiveness of this process.

A lower scope translates into an easier project deadline, which

results in a quicker implementation, lower costs and resultant support from

all. This impacts the expectations-confirmations loop, is one of the key

balancing loops in an ERP implementation project. A lower gap between

expectations and actual also reduces the tendency of users to resist the ERP

and revert to the legacy processes. This in turn positively impacts the efforts

to close whatever expectation gaps that may exist.

Scope finalization therefore becomes important. Right definition of

the scope has been identified by several researchers as an important CSF for

ERP success as is shown in Chapter 2 , Tables 2.2 (a) and (b). The case study

conducted as part of this research and described in Chapter 4 also showed that

the initial scope of the ERP project has a strong relationship to the success of

the project. In the case of ERP 1 the scope was too ambitious ; in ERP 2 the

scope was very simple and it led to a successful implementation.

The standard methodology for finalization of project scope begins

with defining the business vision of the enterprise. Following this the key

information needs for meeting this business goal are identified. A gap analysis

is conducted to ascertain the current status of the MIS in the company and the

existing gaps with reference to the information needs. Normally in

organizations this process is followed by a detailed requirements gathering

exercise.

113

5.3.1 Scope Finalization - Traditional Methods

Traditionally scope finalization in ERP projects draws its methods

from requirements engineering (RE) used in software development. This

involves identifying stakeholders, eliciting their needs and then documenting

these in a form that is communicable for subsequent implementation. While

the ERP scope finalization is similar, there are key differences that must be

noted. A major difference in the case of ERP is that the objective of the

process is to understand the business needs of the organization so as to “fit”

the standard processes of an ERP rather than to create a module to meet these

needs as in the case of software development.

Requirements engineering has some important sub-processes.

These are: (1) requirements elicitation, involving the gathering of user/

process needs, understanding the current and proposed work flows and

documenting the work/ tasks that need to be performed; (2) requirements

agreements involving the process of consensus building when there are

conflicting requirements and (3) requirements communications which

involves wide communications of the finally agreed requirements or goal of

the ERP being implemented (Nuseibeh and Easterbrook, 2000). The final

project goal or scope document is the output of this process (Figure 5.3).

Figure 5.3 Requirement Engineering (RE)- sub-processes

114

An important area of requirements elicitation is identifying MIS

reports needed by users. In the case of an ERP the aim is to identify those

reports that are available in the ERP package and ascertaining the acceptance

of these by the users. The process also elicits new report requirements.

An area of concern in ERP implementations is the extent of

customization that results from new work flows and new reports demanded by

users (often the standard report writer in an ERP suite cannot accommodate

all report formats and customization that may be needed). This often results in

time and cost overruns and is often the cause for ERP project failures. One of

the problems in the standard method of requirements elicitation is that when

asked users tend to give a lot of requirements and reports –whether really

needed or not. Users see this as an opportunity to ask for their complete “wish

lists”. This results in a large requirements list, often not available in the

standard list of modules /reports of the ERP, requiring several man months of

work to deliver.

To illustrate a case in point: in an ERP implementation where the

researcher was personally involved the RE phase identified 55 new reports for

the materials module (note: 200 odd standard reports were already available

in the module). The implementing team spent 6 additional man months to

develop these. After three years during an audit (prior to a version change

exercise) it was found that out of the 255 (200 standard reports plus 55 new

reports made) that were made available only 10 to 15 reports were being used

regularly, another 5 to 6 were used sparingly and the balance not used at all.

This highlights the need to be very careful in agreeing to user requests –often

these are of arguable utility and contribute to the phenomena of “scope

creep”.

In conclusion the current system needs a relook. It is important to

have some means of prioritizing requirements and ensuring that only the

115

most essential is taken up during implementation. Cost and time overruns are

often the result of unrealistic over ambitious goals, cumbersome work

arounds and unnecessary reports. The following section suggests a new

paradigm for defining the scope of an ERP implementation.

5.3.2 Scope Finalization – A New Paradigm

In the earlier section we have identified that inappropriately defined

scope can adversely affect the final outcome of an ERP project. We also

showed that the current system of requirements gathering could result in a

very large and often unachievable project scope which would lead to negative

project outcome. Therefore there is a need to look at a different method of

scope finalization.

We looked at the other mega projects to see if there could be

lessons learnt. A typical mega project is a civil project like the construction of

a bridge. Or consider project like a mountaineering expedition to conquer

Mount Everest. These are mega projects with multiple linkages, complex

tasks to be carried out and different stakeholder expectations. Yet a majority

is considered successful – despite slippages.

Did Mount Everest get conquered? The answer is –yes despite

major “overruns” on the way –of time, money and lives. Then why was this

mega project involving a challenging goal, large teams, several intermediate

milestones and unpredictable extraneous circumstances considered successful

despite intermediate failures. We would like to postulate that this is because

the super ordinate goal of reaching the peak was achieved

Let us take another mega project - building the Concorde. Was

there a slippage of time, cost and non-achievement of several expectations?

Without doubt … volumes have been written about all of these. But was the

116

project “successful” again, without doubt - yes. The grace of the Concorde in

flight removes all doubt from the mind.

What then has made these mega projects successful even though

there have been slippages in deliverables along the way? The answer probably

is that the achievement of the overarching goal of “scaling Everest” or the

“Concorde flying” negates all else. It does not matter how much cost overrun

there was, or what was not achieved, or even how many lives were lost in the

attempt. The achievement of the single overarching goal gives the project the

label of successful. Subsequent milestones are achieved easily.

This does not mean that cost or time overruns are not important.

Once the key overarching goal is achieved, the project is successful in the

minds of most stake holders. The team approaches the subsequent stages of

the project with a positive success led mindset. All other subordinate goals

then get achieved in the subsequent iterations. Thus all goals get achieved in

time.

Our study of several ERP project implementations suggests that a

major cause of project failures is “unmet expectations” due to unrealistic

goals. This is vindicated by the findings of the original The Chaos Report of

the Standish Corporation (Standish 2004). In this report, “Clear statement of

requirements”, “Clear vision and objectives”, “Realistic expectations” and

“Ownership” together account for nearly 30% of the factors of success of

projects. All these factors, collectively, represent user expectations, which

need to be met.

Based on this insight a new approach for goal finalization is suggested. The

approach would focus on a single overarching goal for the ERP project –a

goal that is understood and appreciated and “owned” by all the entities in an

117

organization. The suggested methodology is christened as the Single Goal

Implementation Method (SGIM) (Venugopal 2005).

While ideally this methodology requires one overarching goal for

the whole enterprise, in practice it may not be really be possible to have or

obtain consensus for only one single goal. There are several systems and sub-

systems in an organization. Therefore, while keeping to the spirit of the

methodology, a refinement in the single goal concept may be necessary. In

practice, therefore, the methodology would be deployed as follows:

The core process of the enterprise is identified. A Single goal

(which could be called the Summit goal) for this process is determined. In

addition, two or three more vital processes are also defined and goals for these

are also identified. The Summit Goal and the two or three vital process goals

become the Single Goal Set (SGS) for the ERP project implementation

(Figure 5.4).

Figure 5.4 Singe Goal Set – a revised paradigm (Venugopal 2005)

118

This set of goals, the single goal set (SGS), serves as the centering

force for the project –like a gyroscope, always keeping the project from

“spiraling” out of control. The implementation of the project would be

iterative. The first iteration would be aimed at achieving the SGS. Subsequent

iterations would seek to achieve the other subordinate goals. The process

therefore incorporates the agility and simplicity of the “agile methods”, while

ensuring an overall discipline of goal orientation as is the essence of the

planned methodologies. It is agile in that it is simple and iterative. It is

disciplined, in that, there is an overarching goal for the implementation

understood and accepted by all. The steps in the implementation would

consist of the following:

1. The core process of an organization is identified.

2. The other Key processes (ideally not more than two more) are

identified.

3. The team searches for a Single Goal for the core process (The

Summit Goal) and goals for each of key processes. (Ideally

there should be not more than three goals for the entire

implementation). These become the SGS for the

implementation program.

4. Other Subordinate Goals are identified. (these are for the

other processes and are not focused on in the first iteration of

the implementation)

5. The project champion and the core team intensely work on

building consensus around this set of goals for the project.

This is the key change management task of the

implementation.

119

6. The internal communication in the organization stresses that

the IT project is only for achieving stated Singe Goal Set

(SGS).

7. The implementation team focuses only on achievement of

SGS goals – subordinate goals are not focused on at all in the

first iteration.

8. When the SGS have been achieved the project is declared

successful.

9. The gaps in the subordinate goals are identified and the

organizational impact assessed.

10. The subsequent iterations are carried out to bridge gaps and

achieve subordinate goals.

This revised approach to ERP goal setting is expected to improve

the perception of the users regarding the success of the project as the

publicized key goals of the project are achieved. As a consequence the biggest

problem of implementations, i.e., “managing user expectations” is taken care

of. This would result in an overall “feel good” factor about the project, which

would give added impetus to the subsequent iterations to complete the other

goals.

There could be an argument that the gains from such an approach

are only psychological and that the organization would finally wake up to the

fact that the only gains were a false “feel good” factor? This is not so because

this methodology essentially optimizes resource allocation. Time, money, and

efforts are all first spent on realizing the most important goals of the business.

Only after these are achieved are any of the subordinate goals taken up for

implementation. In the traditional methodology often, this sharp focus is

lacking, with the result that by the end of the project the budget gets spent

120

with none of the goals being “fully” met leading to dissatisfaction. Any

further budget allocated also gets drawn reluctantly into the sinking pit that is

the project without any sign of the organization being able to come out of it.

This is classic project “escalation” described in Chapter 1.

Is the approach too simplistic? Can there really be one Summit

Goal for the entire organization. We believe it is possible to find such a goal.

In fact most organizations do have one powerful driver (or goal) that led them

to the ERP initiative in the first place as evidenced by our research. During

the scoping process, deluged by the large set of benefits promised by the

implementation partners and ERP vendors this goal gets diffused. The

expectations of the company are raised to impossible levels which are

generally not met leading to dissatisfaction.

To validate this a survey was carried out among five very diverse

companies, who had recently implemented ERP projects. A few key users

were contacted in each company and asked just one question “ What was the

main driver for the company to go for the ERP?” We found that each

company had a specific driver for ERP. In one case, it was to integrate with

the parent company’s ERP after a merger (the parent was using SAP while the

unit was on Oracle Applications) . In another case it was driven by the fact

that the present legacy system’s RDBMS was going out of support (the legacy

system was on Ingres and this was going out of support in a year’s time). In

yet another case the prime driver was the need to conform to the new

accounting standards of book closure within 2 days of the last day of every

month (a consistent internal audit comment to the board was that the account

finalization was taking too much time –sometimes as much as 3 months. The

board had mandated that this be corrected). In another case the prime need

was to integrate the remote offices to get real time inventory information. (a

management audit had shown that the inventory turn-round ratio was adverse

121

- 6 versus the desired level of 10; unaccounted inventory in the remote

locations was seen as the culprit).

It is clear from the above that companies had an overriding reason,

or overarching goal, for venturing on a large IT project like an ERP

implementation. The core of the suggested methodology is to capture this goal

and keep it in the forefront of the implementation as a Summit Goal.

A criticism of this methodology could be that this would lead to

sub-optimization. For example for meeting the SGS additional and redundant

data capture may be needed; or it may be decided to continue with a legacy

application for a sub-process. All these are sub optimizations. This criticism is

justified as there is some sub optimization happening in this method. However

we believe that these are planned sub-optimizations - something akin to

planned redundancy creation in a normalized table to achieve higher retrieval

efficiency. The crux is that the most important drivers for the implementation

are met and therefore the business needs are not compromised. Any “local”

sub optimization is acceptable if true global optimization is achieved by

meeting the most pressing needs of the enterprise.

In an actual implementation situation all such “sub-optimizations”

should be tabled and their impact on the key deliverables of cost, quality and

delivery critically examined. If justified, from a cost-benefit perspective, they

should be taken up for resolution through work-arounds or customizations as

is appropriate.

5.4 ERP SELECTION

ERP selection is an important decision during the planning phase of

an implementation. Several researchers have identified ERP selection as an

important CSF (Al Mashari et al 2003, Hong and Kim 2002, Somers and

Nelson 2001, Umble et al 2004). Various methods have been used for this the

122

most common being the scoring and ranking method (Lucas and Moore

1976). This is simple and easy to use. However it is fraught with error as

weights are generally assigned arbitrarily often based on practitioner wisdom.

There is the danger that, lacking a logical basis for assigning weights, these

could be adjusted to “force” decisions to favour one or the other choice. The

method is seemingly scientific when in reality it is not.

Mathematical optimization and multi-criteria decision analysis such

as goal programming, and nonlinear programming have been applied for

software selection (Santhanam and Kyparisis 1995, 1996). A combination of

analytic network process (ANP) and a 0–1 goal-programming model has been

tried for IS project selection (Lee and Kim 2000). However practical

applicability of these methods outside of a research setting is doubtful. This is

because it becomes difficult for line managers to appreciate the methods and

take time out to participate in the process. Moreover several of the attributes

are not easily quantifiable and attempts to do so sometimes only adds “noise”

to the system.

In summary the simpler methods like scoring and ranking are too

simplistic and mathematical optimization too complex to be of practical use.

A technique that uses the intuitive simplicity of ranking with the rigor of

mathematical optimization is the Analytical Hierarchy Process (AHP)

introduced by Saaty (2008). In the following section this process is used to

select between various ERP packages based on a systematic evaluation of

decision criteria by experts.

5.4.1 The Analytic Hierarchy Process (AHP)

AHP is designed to solve complex multi criteria decision problems.

It involves structuring multiple choice criteria into a hierarchy, assessing the

relative importance of these criteria, comparing alternatives for each, and

123

determining an overall ranking of the alternatives. The output of AHP is a

prioritized ranking of alternatives based on overall preferences expressed by

decision makers. AHP helps capture both subjective and objective evaluation

measures.

AHP is useful when decision-making is complex and unstructured.

AHP splits the overall problem into smaller evaluations between multiple

alternatives and finally through its methodology combines these to provide

the global decision. It also provides a mechanism for checking consistency of

the evaluation responses.

Since its introduction, AHP has been used in myriad situations and

is one of the most widely used multiple criteria decision-making (MCDM)

tools. These include applications of AHP in planning, selecting a best

alternative, resource allocations, resolving conflict, optimization etc.

The AHP method involves three phases: decomposition,

comparative judgments and synthesis of priorities (Wei et al 2005).

Decomposition involves developing the fundamental-objective hierarchy

which clearly outlines key objectives that decision makers seek to optimize

and breaks the same down to its attributes. Comparative judgments involve

assigning of priorities to attributes through the pair-wise comparison method

of AHP using a nine point scale. The final stage involves use of the AHP

algorithm to develop the priority matrix. Finally mathematically obtained

priorities are checked for consistency using the consistency testing

methodology developed by Saaty (1980).

5.4.2 Procedure for ERP Selection

The following stepwise procedure has already been referred to

in Chapter 3 – we repeat the same in this chapter for ease of reference

(Figure 5.5).

124

Figure 5.5 AHP Methodology

5.4.3 Application of AHP Methodology for ERP Selection

The first step in applying the AHP methodology is developing

primary secondary and tertiary hierarchies of decision criteria. The AHP

methodology is applied to arrive at priority matrices for each criteria set.

These are then combined to obtain the priority for the final decision. Figure

5.6 gives a two level criteria set that could be used to choose between

competing ERPs.

125

Figure 5.6 Objective hierarchies –ERP selection

This process is to be carried out iteratively with the relevant

management team of the company for each set of criteria in the hierarchy.

While the actual process to develop this can be complex, as part of this

research, a simple adaptation this methodology was tried out in a unit

proposing to implement an ERP. The results are presented in the following

sections.

The firm chosen was a medium scale manufacturing unit proposing

to implement an ERP system. The choice was between alternative ERP

126

packages, Oracle Applications SAP R/3, J.D. Edwards and RAMCO Marshal.

The evaluation team consisted of the IT head, the finance head who was

leading the ERP team, two functional users and an ERP consultant. The

following criteria for ERP selection used by Wei et al (2005) was considered

appropriate and adopted.

Table 5.1 Evaluation criteria

Criteria Label Factors to be considered by the respondent*

Price

Maintenance cost

Consultant expense

Training cost

Total cost TC

Implantation cost

6 – 9 monthsImplementation time IT

Project management ability

Module completion

Function – fitnessFunctionality Fu

Security

Ease of operationUser – friendliness UF

Ease of learning

Upgrade ability

Ease of integrationFlexibility Fl

Ease on in-house development

StabilityReliability R

Recovery ability

Customization C Minimum customization

*Note: The factors mentioned above were provided to the respondent to help them

understand our definition of the criteria being compared. They are not secondary criteria.

In a focus group setting the evaluation team was asked to do pair

wise comparisons between two of the criteria taken at a time. Criteria,

relevant to the firm in question, to be considered for making the comparison

were first arrived at by the group. Factors that define the criteria were also

arrived at. This is to make sure that all respondents understand the criteria in

the same manner. Comparison was made using the 9 point Saaty (1980) scale

given in Table 5.2.

127

Table 5.2 The Saaty rating scale

Intensity of

importanceDefinition Explanation

1 Equal importance Two factors contribute equally to the objective

3 Somewhat more

important

Experience and judgment slightly favour one

over the other.

5 Much more

important

Experience and judgment strongly favour one

over the other.

7 Very much more

important

Experience and judgment very strongly favour

one over the other. Its importance is

demonstrated in practice.

9 Absolutely more

important.

The evidence favoring one over the other is of

the highest possible validity.

2,4,6,8 Intermediate

values

When compromise is needed

(Source: Saaty 2008)

Each criterion under evaluation is to be rated against every other

criterion one pair taken at a time. The relative importance of one criterion

over the other is marked indicating the degree of importance using the

modified Saaty scale Table 5.3. For example if the criterion Total Cost is

considered “somewhat more important” than say User Friendliness the

number 3 on the left of the scale would be marked as shown in Table 5.3. If

Implementation Time on the other hand is “equally important” as Total cost

then 1 on the right of the scale would be marked on the relevant row. The

scale thus makes the marking quite easy.

The rating was carried out collectively for all the seven selection

criteria identified in Table 5.1. The process took time as the team struggled to

ensure that there was logical consistency in rating. It is sometimes difficult to

ensure this: for example if A >B and B> C then A has to be >C. We used a

focus group approach to arrive at group consensus. Saaty suggests use of the

geometric mean of individual ratings. However as suggested by an examiner

of this thesis, group decision aggregation methods could have been attempted.

The AHP method also gives a method to test this consistency – this is

explained in the following sections.

128

Interdependence of criteria – a discussion

In MCDM techniques interdependence of variables is considered a

problem. This is especially so in techniques like regression where the primary

interest of the researcher is in knowing the effect of unit change in an

independent variable on the dependent variable. In a selection technique like

AHP the primary interest of the researcher is in using the criteria to choose

between alternatives – so interdependence of criteria if any poses less of a

problem as long as there are no complex interrelationships between

alternatives and as long as AHP algorithm per se is not affected by the

presence of some interdependence.

Furthermore in social sciences research (as opposed to the natural

sciences) some amount of interdependence is inevitable. Tests like

discriminant analysis help identify such interdependence and techniques like

factor analysis can be used to identify and bring interdependent variables

under one group.

In the case of AHP the issue is one of selection. If criteria are

indeed interdependent, the Saaty scale would bring out this interdependence.

For example if criteria A and criteria C are highly interdependent , then the

respondent would rate them as of “ equal importance” and give it a value of 1

on the Saaty scale.

The issue of interdependence in selection can also be handled

using another more general form of the AHP technique - the Analytical

Network Process(ANP). In the AHP, each element in the hierarchy is

considered to be independent of all the others—the decision criteria are

considered to be independent of one another, and the alternatives are

considered to be independent of the decision criteria and of each other. ANP

129

does not require independence among elements, so it can be used as an

effective tool in these cases.

To illustrate this, consider the decision of choosing between ERP

packages . The decision maker wants to base his decision on only three

factors: Cost, Functionality and Customization needs. Both the AHP and

ANP can be used to provide useful frameworks to use in making his decision.

The AHP as we have seen assumes independence of these criteria.

However it can be argued that Cost and Functionality are not truly

independent. Also higher functionality may mean lesser customization hence

these are also not independent. ANP would allow consideration of these

interdependences.

While AHP considers the decision making to be a hierarchical

process , top-down from the goal- ANP models it as one that is cyclical with

each node having a relationship with another node.

The steps for gathering the responses are the same using the Saaty

scale or any other appropriate means to obtain pair wise comparisons. The

computation is slightly different. An illustration of the ANP method is

provided in Appendix 7.

130

Table 5.3 Adapted scale for ERP criteria evaluation

S.No. Factor Rating Factor

1 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Customization

2 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Functionality

3 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 User-friendliness

4 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility

5 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability

6 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time

7 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Functionality

8 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 User-friendliness

9 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility

10 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability

11 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time

12 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 User-friendliness

13 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility

14 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability

15 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time

16 User-friendliness 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility

17 User-friendliness 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability

18 User-friendliness 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Implementation Time

19 Flexibility 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability

20 Flexibility 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time

21 Reliability 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time

131

The next step in the AHP process is creation of the reciprocal

matrix based on the ratings. For example in the cases above against Cost V/s

User Friendliness a number 5 would be recorded and against User

Friendliness versus Cost the reciprocal 1/5 would be recorded. In this manner

the reciprocal matrix is created. The actual reciprocal matrix generated for the

case under evaluation is given in Table 5.4 (detailed AHP workings are in

Appendix 6).

Table 5.4 Reciprocal matrix

TC C Fu UF Fl R IT

TC 1.0000 5.0000 5.0000 3.0000 2.0000 3.0000 1.0000

C 0.2000 1.0000 3.0000 0.1667 1.0000 0.2500 2.0000

Fu 0.2000 0.3333 1.0000 4.0000 2.0000 1.0000 3.0000

UF 0.3333 6.0000 0.2500 1.0000 3.0000 1.0000 3.0000

Fl 0.5000 1.0000 0.5000 0.3333 1.0000 1.0000 3.0000

R 0.3333 4.0000 1.0000 1.0000 1.0000 1.0000 2.0000

IT 1.0000 0.5000 0.3333 0.3333 0.3333 0.5000 1.0000

SUM 3.567 17.833 11.083 9.833 10.333 7.750 15.000

The next step in the AHP methodology is creation of the

normalized pair wise comparison matrix. This is done by dividing each

element in the matrix by its column sum. The eigen vector of each row is

computed. A common method is to take the nth

root of the product of items in

a row where the number of items in a row is n.

132

Table 5.5 Normalized matrix

TC C F UF F R IT Eigen Vector* Priori

ty

TC 0.280 0.280 0.451 0.305 0.194 0.387 0.067 0.246 0.31

C 0.056 0.056 0.271 0.017 0.097 0.032 0.133 0.067 0.08

Fu 0.056 0.019 0.090 0.407 0.194 0.129 0.200 0.110 0.14

UF 0.093 0.336 0.023 0.102 0.290 0.129 0.200 0.127 0.16

Fl 0.140 0.056 0.045 0.034 0.097 0.129 0.200 0.084 0.10

R 0.093 0.224 0.090 0.102 0.097 0.129 0.133 0.118 0.15

IT 0.280 0.028 0.030 0.034 0.032 0.065 0.067 0.053 0.07

Sum 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.805 1.00

*Note: The Eigen vector can be approximated by several methods – one of

the methods is taking the geometric mean (nth

root of the product of cell

items) and normalized by dividing by sum of the roots (Coyle 2004) For

example for the first row TC, the geometric mean is given by (0.280x 0.280 x

0.451 x 0.305 x 0.194 x0.387 x 0.067)1/7

= 0.246. Similarly the means are

computed for each of the rows. When normalized by dividing by the sum of

roots 0.805 the first element of the relative Priority Vector, 0.31 is obtained.

The priority vector is the relative importance of the criteria in the

ERP selection decision. To illustrate in the case above Total Cost (TC) the

importance is 0.31 (31%) as compared to Functionality (FU) which has an

importance of 0.132 (13%). The sum of all priorities would total to 1.

The final stage is to calculate a Consistency Ratio (CR) to measure

how consistent the judgments are relative to a sample of purely random

judgments (Coyle 2004). The process of this evaluation is explained in the

following section.

133

Consistency Test

For calculating CR we first need to calculate max which is the

maximum Eigen value of the matrix. This is approximated by the weighted

total of the reciprocal matrix column totals using the relevant priority vector

as a multiplicand for assigning weights. The max in this case is given by:

X = 8.840

CR is given by the equation:

C.R. = Consistency Ratio = (C.I.) / (R.I.) (5.1)

where C.I. = Consistency Index = ( max –n) / (n –1) and R.I. = Random Index

which is described below. For each matrix of size n, Saaty’s team generated

random matrices and computed their mean C.I. value and called it the

Random Index. These values are shown in the Table 5.6.

Table 5.6 Random Index (RI) table

n 1 2 3 4 5 6 7 8 9 10

Random

Index(RI)0 0 .58 .90 1.12 1.24 1.32 1.41 1.45 1.49

(Source: Saaty 1980)

A value of CI less than or equal to 10 % of the table value is

acceptable. Larger values indicate that the judgements are no better than

random and therefore not to be considered consistent. The decision maker is

134

advised to revisit the judgement process to attempt to reduce the

inconsistencies.

We next evaluate the data in Table 5.5 for consistency:

The consistency check is carried out using an Excel AHP

calculator. The outputs are as given below (See Appendix 6):

Table 5.7 AHP calculations-final outputs

Constructs max CI CR Priorities

TC 0.31

C 0.08

Fu 0.14

UF 0.16

Fl 0.10

R 0.15

IT

8.840 0.3066 0.2322

0.07

The consistency ratio comes to 0.23. For a matrix n=7 the CI

should be < 10% of 1.32 or <0.132. In the present case the value is 0.3066

which is greater than the guideline given by Saaty for a consistent judgement.

This indicates that there are logical inconsistencies in the responses given by

the experts.

Table 5.7 gives the priorities of various decision as applicable for

the firm in question. For example in this case Total Cost (TC) comes out as

being significantly more important while Functionality (Fu) and Flexibility

(Fi) are relatively lower down on priority. This priority matrix would be

different for different decision environments and represents the collective

judgement of the decision makers on the relative importance of decision

criteria.

135

Checking against alternative choices

Next the decision alternatives are evaluated against the same

decision criteria. This is done by comparing how well each of the alternative

ERPs performs against each of the decision criteria. This is not easy to do as it

requires a knowledge of multiple ERP functionalities. It is usually done by

studying literature, discussing with vendors, discussing with implementation

consultants and talking to other firms that have made similar evaluations.

As is evident this would result in seven different relative

performance matrices one for each of the seven decision criteria used above.

For each of the matrices AHP calculations are to be followed to arrive at the

final Product v/s decision criteria performance matrix. For this matrix the

eigen vector representing the priorities can be calculated and the consistency

check worked out.

The AHP method requires that the decision criteria matrix must

first meet the consistency criteria before it can be relied upon for the rest of

the calculation. As our decision criteria judgements fail on this count the rest

of the calculation is for academic interest only. However it is presented here

for illustrative purposes.

Comparison between ERPs

The comparison is made between four ERP systems SAP, Oracle

Applications, J.D. Edwards and RAMCO Marshal. In this case, three experts

from ERP implementation organizations were requested to rate the

performance of each of these ERPs against the seven decision criteria. Saaty

recommends the use of the geometric mean of individual rater scores (Saaty

2008 pp 95). In this case however we used a consensus building approach

where the experts discussed and came to a conclusion on the rating. Table 5.8

136

gives the resulting reciprocal matrix for Total Cost (TC) comparison between

the four products. In the case of Total Cost a reverse scale is used with the

lower number being the more preferred.

Table 5.8 ERP Comparison table –criteria: Total Cost (TC)

SAP Oracle Ramco J.D Edwards

SAP 1.00 1.00 0.11 0.20

Oracle 1.00 1.00 0.11 0.20

Ramco 9.00 9.00 1.00 9.00

J.D. Edwards 5.00 5.00 0.11 1.00

Sum 16.00 16.00 1.33 10.40

The rest of the AHP calculation proceeded as described earlier

resulting in a final comparison matrix with CI and CR as given in Table 5.9.

Table 5.9AHP results –TC

ERPs A. max CI CR

SAP 4.0443305

ORACLE 4.0443305

RAMCO

4.35772 0.11924 0.1325

5.1359431

J.D.EDWARDS 4.2062908

Similarly each of the decision criteria was rated by the experts and

the final table of priorities for the ERPs was computed (Table 5.10).

137

Table 5.10 ERP selection: final priorities

TC C F UF F R IT Overall priority

SAP 0.057 0.055 0.664 0.380 0.063 0.665 0.047 0.292

Oracle 0.057 0.099 0.152 0.380 0.063 0.202 0.127 0.150

Ramco 0.685 0.577 0.049 0.178 0.618 0.041 0.502 0.386

J.D. Edwards 0.201 0.268 0.135 0.062 0.255 0.092 0.324 0.172

5.4.4 Discussions

The decision matrix 5.10 shows that RAMCO is the preferred ERP

system for this firm followed by SAP. It must be pointed out that this

judgment in no way reflects either a real evaluation between ERP products

nor the researcher’s personal view of which ERP is better. It is just an

illustrative case of a judgment made by a set of experts in a firm considering

the priorities of that firm at that point of time.

Any judgment is contextual. At a point of time in a specific context

one or the other decision criteria gains or falls in importance. The final

ranking between alternatives would reflect this. In another context with a

different priority a different ranking would emerge. In this case cost was of

primary importance and this has translated into RAMCO coming out as the

preferred choice. If another parameter, say “functionality” was a priority

another choice would have emerged.

As is seen in our case the initial evaluation of the priorities fails on

the consistency test. This would mean that subsequent calculations are

suspect. We tried to analyze why inconsistency creeps in. Our conclusions are

as follows:

1. It becomes difficult to avoid logical inconsistency when

parameters under comparison are more than 3 or 4. As a test

138

case when we asked the experts to evaluate against just 3

criteria-Cost, User-friendliness and Customization, we found

that CR was within the 0.1 stipulated by Saaty. As alternatives

go up evaluation becomes less consistent.

2. When parameters are many, time needed to do detailed

comparisons increases. By the time the rater reaches the end

he/she is quite bored and the rating then becomes arbitrary.

3. Our choice of Cost as one of the evaluation criteria was

probably erroneous (as pointed out by one of the thesis

examiners). Respondents would almost always give preference

to Cost thus vitiating the exercise. A better approach would

have been to use the Benefits (B), Opportunities (O), Cost (C),

Risks ( R) framework suggested by Saaty.

4. The Saaty scale of 9 choices is also probably too complex to

give consistent results. It becomes difficult to distinguish

between “absolutely more important” and “very much more

important”.

As a guideline for practice we would suggest the use of the AHP

technique as one of the methods for guiding for the final decision. It does give

a priority matrix which has a mathematical basis and can be better than a

simple scoring sheet. To avoid the problem of the rater getting tired or bored,

it may be worthwhile to do the rating two or three criteria at a time over two

or three sittings. This would probable give better results.

As a guideline for research we would suggest future research

experiments with a simpler 5 point scale. The Saaty algorithm would work

just as well and the overall process would probably be better.