Abstract— Decision Support Systems (DSS) are very important in clinical engineering (CE). The Analytic Hierarchy Process (AHP) is one of such systems that has been extensively and successfully used in many areas, including CE. In this paper, we provide an overview of different DSS and explain AHP in details. At the end, a case study using AHP for taking medical equipment scrapping/ retirement decision is presented.

I. INTRODUCTION N the busy environment of any healthcare facility, decisions have to be made all the time. Most decision

problems fall into the category of multi-attribute / criteria decision problems, i.e. decisions involving a finite number of alternatives and a finite number of criteria upon which such alternatives are evaluated. Having an organized, structured way of evaluating alternatives or different courses of action available as a solution to the decision problem is crucial to produce reasonable, justified, and unbiased decisions.

The selection of a DSS for a certain problem depends on several criteria, including the complexity of the problem, the simplicity of the method, the type of data available (quantitative or qualitative or both), the expertise of the decision makers etc.

Some work has been done to evaluate different DSS. Peniwati suggests 16 criteria for such evaluation and draws a comparison between them [1].

In this paper some of the most common multi-criteria DSS will be described, followed by a detailed description of one of them; AHP, and then an example of using AHP to implement equipment scrapping decision is presented.

II. LITERATURE REVIEW Multi-criteria DSS vary in complexity from simple,

elementary methods to sophisticated ones. Follows is a brief description of some of these methods.

A. Pros and Cons Analysis

In this method the pros, i.e. positive aspects, of each alternative are compared against its cons, i.e. negative aspects, such that the alternative whose pros are more and stronger that its cons is the preferred one. This method depends on qualitative comparisons and is suitable for simple decisions with few criteria and alternatives.

B. Kepner-Tregoe (K-T) Decision Analysis This method depends on the judgments/ assessments of a

Manuscript received September 26, 2010

Asmaa Ahmed Kamel is with the Department of Systems & Biomedical Engineering, Faculty of Engineering, Cairo University, Giza, Egypt (e-mail: asmaaakamel@hotmail.com).

Bassel Sobhi Tawfik is the head of the Department of Systems & Biomedical Engineering, Faculty of Engineering, Cairo University, Giza, Egypt.

group of experts, where the less the quality of the data, the larger the group required [2].

The most important criterion is identified and given a score of 10 and then the rest of criteria are evaluated relative to it, where less important criteria can be given scores down to 1. In the same manner, the alternatives are evaluated relative to each other against the previously weighted criteria and given scores from 1 to 10. The final score of an alternative is determined by multiplying its score under each criterion by the score of that criterion and adding for all criteria. The alternative with the highest score is the most preferred.

C. Cost-benefit Analysis This method is used when the primary criterion for

making the decision is money or cost of a given alternative vs. its benefit, such that finally the alternative with the largest net present value is preferred. Note that, all other methods treat cost like any criterion.

D. Multi-Attribute Utility Theory (MAUT) This method uses “utility” or preference functions to

transform criteria from different scales into a common, dimensionless scale with values from 0 to 1; where for each criterion a utility function is created [3]. Utility functions can be derived from studies or statistics related to the type of alternatives involved. Once these functions are determined, they are used to convert the alternatives raw data, whether subjective or objective, into a dimensionless utility score. The criteria are weighted according to their importance, then the alternatives normalized utility scores are multiplied by these weights and added for all criteria, such that the preferred alternative has the highest score.

E. Analytic Hierarchy Process (AHP) Using AHP, decisions involving both qualitative &

quantitative data can be effectively made. AHP was developed by T.L. Saaty in the 1970s [4] and has been used and also refined extensively since then.

In general, AHP applications can be split into the main categories of choice/selection; a best alternative is selected among a group, prioritization/ evaluation; a combination of alternatives are selected instead of one, benchmarking; different processes/entities are compared with one another or with the best of breed and rated accordingly.

In healthcare, AHP was used for multiple applications. A university hospital used it to help patients take decisions regarding undertaking a prostate cancer screening test [5]. It was also used for the selection of residents for a five-year general surgery program at a major metropolitan hospital [6], where in the study the correlation of the results of the AHP and the traditional scoring system were found to be statistically valid. AHP was used to select neonatal ventilators to be used in a planned expansion of a neonatal

Decision Support Systems in Clinical Engineering Asmaa Ahmed Kamel, Bassel Sobhi Tawfik

I

intensive care unit [7]. The use of AHP is such multidisciplinary decision was found satisfactory and the authors conclude that it should be used for future healthcare technology assessment decisions.

Besides healthcare, in 2001 AHP was used to determine the best relocation site for the earthquake that devastated the Turkish city Adapazari. Also in 1998, the British Airways used it choose the entertainment system vendor for its entire fleet of airplanes. Xerox Corporation has used it to allocate close to a billion dollars to its research projects [8].

1) Structure In AHP, the decision problem is decomposed into a

number of small problems and structured in the form of a hierarchy; with the goal of the decision at top, followed by the criteria and sub-criteria (if there are any) upon which the alternatives are evaluated. Alternatives are always found at the bottom of the hierarchy as in Fig. 1

The criteria at each level are compared in pairwise comparison matrices using the absolute scale of measurements in Table I, such that the criteria at the first level are compared against each other with respect to the goal (i.e. they are compared as to which is more important with respect to the goal) and the criteria at subsequent levels are compared with respect to the higher level parent criterion. Finally, the alternatives are compared with each other with respect to all lower level criteria (also known as covering criteria).

2) Priorities derivation methods

Different methods are available for deriving priorities or weights from the comparison matrices. A debate over the best method to be used started as early as the AHP was created and hasn’t been resolved till today.

Saaty shows that the principal eigenvector is the priority vector for a consistent matrix [9]. He calls it “the only plausible candidate for representing priorities derived from a positive reciprocal near consistent pairwise comparison matrix”.

We use the eigenvalue method because in agreement with Saaty we think that the principal eigenvector best describe the matrix properties. Also the eigenvalue method is the only method that uses the indirect estimations in a pairwise comparison matrix for the calculation of the priorities and thus the only method that takes into consideration the transitivity property of the matrices.

Fig. 1. General AHP Hierarchy

TABLE I THE FUNDAMENTAL SCALE

Intensity of Importance

Definition

Explanation

1 Equal Importance Two activities contribute equally to the objective

3 Moderate importance Experience and judgment slightly favor one activity over another

5 Strong importance Experience and judgment strongly favor one activity over another

7 Very strong or demonstrated importance

An activity is favored very strongly over another; its dominance demonstrated in practice

9 Extreme importance The evidence favoring one activity over another is of the highest possible order of affirmation

2,4,6,8 For compromise between the above values

Sometimes one needs to interpolate a compromise judgment numerically because there is no good word to describe it.

Reciprocals of above

If activity i has one of the above nonzero numbers

assigned to it when compared with activity j, then j has the

reciprocal value when compared with i

A comparison mandated by choosing the smaller element as the unit to estimate the larger one as a multiple of that unit.

1.1-1.9 For tied activities When elements are close and nearly indistinguishable; moderate is 1.3 and extreme is 1.9.

3) Different types of priorities The priorities obtained from each comparison matrix are

the local priorities, i.e. the priorities driven for a set of nodes with respect to a single criterion. Global priorities are obtained by multiplying these local priorities by priority of the parent criterion. The overall priorities for an alternative are obtained by adding its global priorities throughout the hierarchy.

4) AHP modes In general, there are two types of measurements or

comparisons, relative and absolute. In relative comparisons, alternatives are compared in pairs against a common attribute, whereas in absolute comparisons alternatives are compared against a standard in memory developed through experience.

Following this concept, AHP has two types of methods to deal with different problems, the Relative Method and the Absolute or Ratings Method.

a) The Relative Method: In this method, the criteria and alternatives are compared and priorities derived in the way described at the beginning.

b) The Ratings Method: In this method, the criteria are pairwise compared, and then rating categories are made for each covering criterion. After that these ratings are prioritized by pairwise comparing them for preference. Alternatives are then evaluated by selecting the appropriate rating category on each criterion. For example, in a decision problem to select the best job from a number of alternatives,

the rating categories for a “Job Security” criterion was High, Medium and Low. For a “Reputation” criterion, it was Excellent, Above Average, Average and Poor.

The relative method is used in case of dependence among the alternatives, whereas the absolute method is used when the alternatives are independent of each other. This is why the relative method allows the change of the ranking of the alternatives in case of the addition of a new one, while the absolute method doesn’t allow rank reversal.

Also the relative method where alternatives are compared with each other under the various criteria is more accurate, while the ratings method has the advantage of rating a large numbers of alternatives rather quickly.

5) Consistency Index A human judgement in real life situations may be

inconsistent. AHP provides a way of measuring the inconsistency of each comparison matrix judgements [4]. High values of the inconsistency ratio may call for refining the hierarchy, revising user’s judgements, collecting more data etc. before taking the decision. However, higher consistency doesn’t necessarily mean better or more accurate decisions.

6) Sensitivity Analysis Sensitivity analysis is performed to test the stability of the

final ranking of the alternatives, to make sure that no criteria are overlooked, and to detect any errors that might have occurred while rating the alternatives. In this analysis the priorities of all criteria or at least the most important ones are varied, e.g. from 0.01 to 0.99 in six steps, then alternatives new priorities are obtained, and finally the effect of varying the criteria priorities on the final ranking of the alternatives is examined.

III. CASE STUDY: EQUIPMENT SCRAPPING/ RETIREMENT DECISION

Equipment scrapping is one of the classical problems facing clinical engineers. Whether to retire a piece of equipment or not, when to retire it, and what can be done with the retired equipment are some of the questions frequently encountered while taking such a decision. The need to scrap a piece of equipment may result from equipment being unsafe, its maintenance is beyond economical repair or simply because its technology is obsolete.

A. Equipment scrapping criteria The possible criteria upon which equipment scrapping

decision can be made were examined and gathered as shown in Table II. Some of these criteria are applicable to different medical equipment types, while others may be device-specific.

The subject of our decision is nine hemodialysis machines (Fresenius 4008B, installed in a local healthcare facility) that were to be evaluated for scrapping. So the criteria applicable to hemodialysis machines were extracted from Table II and arranged in the hierarchy shown in Fig. 2.

TABLE II EQUIPMENT SCRAPPING POSSIBLE CRITERIA

Criteria Sub-criteria Definition Device age Time since installation, in terms

of years or working hours Technology status

Degree of maturity of device technology (from visionary to obsolete technology)

Performance

(this criterion is highly dependent on equipment type and thus can have more/different sub-criteria)

Equipment down-time

Mean Time Between Failures (MTBF)

The expected time between two consecutive failures for a repairable system

Accuracy/ Calibration

This sub-criterion may also descend from safety

Failures Types One possible categorization is failures that can be eliminated & those that can’t be

Safety Number of hazardous incidents

Incidents resulting in the injury/ harm of patient/operator

Device Recall The action of correcting a product-related problem or removing it from the market, as a result of being either defective or potentially harmful.

Support availability

Service support The know-how to troubleshoot a problem & diagnose the defect

Spare parts Backup equipment

i.e. if the facility has a large number of equipment doing the same function, it may be more towards scrapping the less performing ones.

Cost Spare parts Repair/ maintenance

Equipment upgradability

Upgradability can help eliminate or reduce an existing problems

Equipment utilization percentage

i.e. if the equipment utilization percentage is low and it’s already in a bad condition, then it might be scrapped (and it even needn’t be replaced)

Some criteria are inapplicable because of hemodialysis machines properties, for example “Equipment upgradability”, while others because of this study specific machines, for example “Device recall” (4008B machine is not FDA-approved) and “Service and spare parts availability” (the manufacturer still supports this model). However, other criteria were excluded because data are unavailable at the healthcare facility. Data deficiency was dealt with in a number of ways as will be explained later in the paper.

B. Model Implementation The hierarchy was implemented using the Super

Decisions Software (designed by Bill Adams and the Creative Decisions foundation) [10]. It implements the AHP and its generalization, in case of dependence and feedback between the criteria and alternatives, the Analytic Network Process.

Fig. 2. Equipmen

Fig. 3 shows was used evapriorities.

C. Problem S

Evaluating should be carpolicy of the technology fcomparison technology leequally high first think o“Technology “Technology the ratings foand Establishweight to thefinally estabfollower willreverse order.

While the given to the such that findevice, the hhave a feelingadded a fictitiunder each oidealized priomachines prio

D. SimulatioAs a result

data to creadifferences bsimulated the

1) No data incidentsHowevercriteria thcalibratiorespectivbiomedic

nt Scrapping Dec

a snap shot ofaluate the mac

Statement

a given devrried out whilinvolved heal

follower or of the prim

eader is more priority, wheof “Performa

status & Astatus” criteri

or this criteriohed”, a teche state of theblished technl arrange the. ratings were factors in fa

nally the highhigher its priog of how highious machine of the coverorities its prioorities are a fr

on Scenarios of data defici

ate new situbetween the m following scewere availab

s, calibration vr, while evaluhey were rateon limits, vely. This wacal engineerin

ision Hierarchy

f the programchines and de

ice using thee keeping in lthcare facilityleader, as th

mary criterialikely to hav

ereas a technoance & Saf

Age”. Also tion will be din are “State o

hnology leadee art, followenology, wheese ratings f

compared, mavor of scrappher the weigority for scraph or low such“H” which w

ring criteria srity equals “1

raction of this

iency, we decuations and bmachines. Toenarios: ble on the nuvalues or maiuating the med as were raand >40%

as based on wng technician

m. The ratings erive their res

e hierarchy inmind the techy, i.e. whethehis will affea, for examve the 4 criterology followerfety” and lethe ratings fifferent, for exof the Art, Maer will give ed by maturinereas a techfor importanc

more importanping the equi

ght/score of apping. Howeh final weight

was rated as thsuch that usi1” and the res“1”.

ided to add fibetter delineaowards this e

umber of hazntenance/repa

machines undeated as none,

of device what the respn at the hea

method spective

n Fig. 2 hnology er it is a ect the

mple, a ria with r might east of for the xample, aturing,

higher ng, and hnology ce in a

nce was ipment, a given ever, to t is, we

he worst ing the t of the

ictitious ate the end we

zardous air cost. er these outside

value ponsible althcare

2)

A.F

weshoobthebecanwe55

folscememothethe

B.

thesam“Dwhprithe

facility saidthese machiWe once technology-technology primary c“Technologexplained e

Simulation SFollowing theere found to ows the idea

btained by divem (fictitious tter in understn also be cone can say that .8%. Changing thllower in the enario increaeaning that a ore towards se machines che 2nd became

FI

Machin1 2 0X3 7V4 1X5 7V6 0X7 7V8 0X9 0X

10 0X

SEC

Machin1 2 0X3 7V4 7V5 1X6 0X7 7V8 0X9 0X

10 0X

Sensitivity AFrom the sene machines rame order sho

Device safety”hile the rankinimary criteriae second scena

d about the reaines. assumed the -leader healthfollower one

riteria as wgy status” cearlier.

IV. R

Scenarios Rese first scenaribe as shown

alized prioritieviding all the

machine H). tanding the re

nverted into a machine 0XF

he problem sfirst scenario

ased the totatechnology le

scrapping the hanged, where3rd etc. as sh

TABIRST SCENARIO R

ne Serial No. RH

XFE0829 V5E6787 XFE1223 V5E6795 XFE0355 V5E6786 XFE352 XFE351 XFE354

TABCOND SCENARIO

ne Serial No. RH

XFE0829 V5E6795 V5E6787 XFE1223 XFE0352 V5E6786 XFE355 XFE351 XFE354

Analysis nsitivity analysanking was f

own in Table” and “Technong varied undea priorities. Siario.

asons that cal

machines tohcare facilitye, such that twell as the riterion will

RESULTS:

sults io, the nine min Table III.

es of the mapriorities by The idealized

esults than thepercentage fo

FE0829 needs

statement froo to a technolal priorities eader healthcamachines. Al

e the 4th machown in Table

BLE III RESULTING PRIORI

Raw Priorities I0.222071 0.123941 0.100459 0.098610 0.085670 0.082898 0.081320 0.080893 0.064379 0.059761

BLE IV RESULTING PRIOR

Raw Priorities I0.169861 0.117287 0.103202 0.101344 0.092848 0.091013 0.088481 0.083401 0.078031 0.074532

sis results forfound to be s

III) under thology status” er the variatioimilar results

lled for scrapp

o be workiny and then the ranking o

ratings of be differen

machines prio. The last colchines, whichthe largest amd priorities ca

e raw prioritiesorm. For exam to be scrappe

om a technology leader inof the machare facility wilso the rankinchine became IV.

ITIES Idealized Priority

1 0.558117 0.452372 0.445047 0.385778 0.373294 0.366189 0.364265 0.289902 0.26911

RITIES Idealized Priority

1 0.690488 0.607569 0.59663 0.54661 0.535807 0.520899 0.490993 0.459383 0.43878

r the first scenstable (havinghe variation in

criteria prioron in the rest o

were obtaine

ping

ng in in a f the

the nt as

rities lumn h are mong an be s and mple, ed by

ology n this hines, ill be ng of 2nd,

y

y

nario, g the n the rities, of the d for

Fig. 3. Hierarchy implementation by the super decisions software

V. DISCUSSION: The nine machines showed relatively close final priorities;

one possible reason for this is that for the “Accuracy/Calibration”, “Technology Status”, and “Repair/Maintenance cost” criteria, all the machines had the same rating. This indicates the need to add more criteria under which the machines can score differently to have more discriminative values.

The main differences between the machines while rating them were found to be in the Equipment down-time, MTBF, and Age; this indicates that more weight might be given to these criteria.

As shown in Table IV, changing the problem statement from a technology follower to a technology leader increased the total priorities of the machines besides slightly changing their order. This means that a technology leader healthcare facility will be more towards scrapping these machines. It also suggests that the simulation of more scenarios can further discriminate the machines and provide more indicative results.

A possible scenario may assume that some of the machines are running in a private hospital and the rest in a public hospital. Regarding the “Maintenance/repair cost” criteria, for a private hospital it is more economical to replace the equipment when its maintenance/repair cost is beyond a certain ratio of the device cost (which is the right thing to do), while a public/governmental hospital will keep maintaining the machine despite the high cost because it doesn’t have the capital investment required to replace the machine. So for the first type of healthcare facilities, more weight will be given to the “Maintenance/repair cost” and the ratings threshold is more likely to be less than the 40% used in the second type.

The instability of the machines ranking under the variation of some criteria is not a good thing and reasons for this need to be further investigated.

VI. CONCLUSION We found AHP relatively easy to use for the discussed

decision problem. It provided deeper analysis than the manual methods that are usually implemented using very few criteria or even one criterion, for instance, only cost. However, care must be taken while building the decision hierarchy such that only important criteria discriminating the alternatives should be included. Also the experience and knowledge of involved decision makers is very important to

have meaningful comparisons. The lack of experience may be compensated for by having as much and as accurate data as possible; concerning the different criteria and alternatives.

When scrapping takes place, management should seek statistics showing the frequency with which each cause has contributed to the situation, in order avoid the same problems in the future. This can be derived from the ratings of the device under different criteria. Also such ratings, in addition to the criteria weights can help in determining what to do with the scrapped equipment. For example, if “Technology status” criterion had the highest weight and the equipment was rated as being of established technology, then the problem isn’t with the device but with the healthcare facility policy and so the equipment can be put to use elsewhere, for example, donated to a charity clinic.

For future work and in order to ensure the adequacy of AHP for this kind of decisions, machines of different models and from different manufacturers shall be evaluated. Also other external factors affecting hemodialysis machines condition, like quality of water from treatment units and other environmental conditions shall be incorporated in the model.

A software tool is being developed to implement the decision hierarchy and carry out the necessary mathematical operations to derive priorities and calculate alternatives final weights.

REFERENCES [1] Kirti Peniwati, “Criteria for Evaluating Group Decision-Making

Methods”. Mathematical and Computer Modelling, vol. 46, no. 7-8, pp. 935-947, October 2007.

[2] C.H. Kepner and B.B. Tregoe, “The New Rational Manager”, Princeton Research Press, Princeton, NJ, 1981.

[3] R.L. Keeney and H. Raiffa, “Decisions with Multiple Objectives: Preference and Value Tradeoffs”, John Wiley, New York, 1976.

[4] T. L. Saaty, "How to Make a Decision: The Analytic Hierarchy Process", Interfaces, vol. 24, no. 6, p. 19-43, 1994.

[5] E.B. Sloane, M. J. Liberatore, and R. L. Nydick, “Medical Decision Support Using the Analytic Hierarchy Process”, Journal of Healthcare Information Management, vol. 16, no. 4

[6] M. S. Weingarten, R. L. Nydick, F. Erlich, and M. J. Liberatore “A Pilot Study of the Use of the Analytic Hierarchy Process for the Selection of Surgery Residents.” Academic Medicine, vol. 72, no. 2, pp. 400-401, 1997.

[7] E. B. Sloane, M.J. Liberatore, R.L. Nydick, W. Luo, and Q.B. Chung, “Using the analytic hierarchy process as a clinical engineering tool to facilitate an iterative, multidisciplinary, microeconomic health technology assessment”, Computers & Operations Research, vol. 30, pp. 1447–1465, 2003.

[8] T.L. Saaty, “Decision making with the analytic hierarchy process”, Int. J. Services Sciences, vol. 1, no. 1, 2008

[9] T. L. Saaty, “Decision-making with the AHP: Why is the principal eigenvector necessary?” Proceedings of the sixth International symposium on the Analytic Hierarchy Process, Berne-Switzerland, August 2-4, 2001.

[10] Super Decisions, version no. 2.0.8, 01 Jun 2009. Available: http://www.superdecisions.com/index_tables.php3

[11] J. Tobey Clark, Healthcare Technology Replacement Planning, Clinical Engineering Handbook, ELSEVIER academic Press, ch. 42