BALLAST WATER MANAGEMENT: A COMPARATIVE ANALYSIS OF TODIM AND THOR

By L.F.A.M. Gomes1,3, C.F.S. Gomes1,2,3, L.A.D. Rangel2,3 1Ibmec/RJ, 2Universidade Federal Fluminense, 3MCDM

Research Group of Ibmec, Rio de Janeiro, Brazil

The 21st International Conference on Multiple Criteria Decision Making

13-17 June 2011, Jÿväskyla, Finland Session MCDM models developed and used in Latin America

1

Structure of the presentation

• Problem definition• Structuring of the analysis• Choice of methods• Computations• Conclusions

2

Problem definition

3

A ballast tank is a compartment within a boat or ship that holds water. A vessel may have a single ballast tank near its center or multiple ballast tanks typically on either side. A large vessel typically will have several ballast tanks including double bottom tanks, wing tanks as well as forepeak and aftpeak

tanks. Adding ballast to a vessel lowers its center of gravity, and increases the draft of the vessel. Increase draft may be required for proper propeller

immersion.The draft of a ship's hull is the vertical distance between the waterline and

the bottom of the hull), with the thickness of the hull included; in the case of not being included the draft outline would be obtained. Draft determines the

minimum depth of water a ship or boat can safely navigate. The draft can also be used to determine the weight of the cargo on board by calculating the total displacement of water and then using Archimedes’ principle. (Sources:

http://www.thefullwiki.org/Ballast_tank & http://www.thefullwiki.org/Draft_(hull))

4

Scheme of a ship’s cross section. (Source: http://www.thefullwiki.org/Ballast_tank)

5

Main ballast tanks and fuel ballast tanks. Source: http://www.maritime.org/fleetsub/appendix/pages/figa-04.htm

6

Ballast water is absolutely essential to the safe and efficient operation of modern merchant ships, providing balance and stability to unloaded ships.

Ballast is any material used to weight and/or balance an object. One example is the sandbags carried on conventional hot-air balloons, which can be

discarded to lighten the balloon's load, allowing it to ascend. Similarly, ballast water is therefore water carried by ships to ensure stability, trim and

structural integrity. Ships have carried solid ballast, in the form of rocks, sand or metal, for thousands of years. In modern times, ships use water as ballast.

However, ballast water may also pose a serious ecological, economic and health threat due to invasive aquatic species.

Although current ballast water regulations recommend minimizing the risk of introducing non-native species by exchanging ballast water in the

open ocean, this method has serious drawbacks. Even when it can be used, organisms remaining inside the ballast tanks are discharged at a

later time into ports and harbors if the exchange fails to remove all organisms

7

With the introduction of steel-hulled vessels and pumping technology, water became the ballast of choice. Water can be easily pumped in and out of ballast tanks, requires little manpower, and

as long as tanks are kept full, poses little to no stability problems. 8

Sources: http://www.providence.edu/polisci/students/megaport/ballast.htm & http://www.marinebiology.edu/Phytoplankton/ballast.htm

9

Transfer of species in ballast water started as early as shipping trade. The movement of some 3 to 12 billion tonnes of ballast water in ships

internationally each year has been responsible for the settlement of about 100 million tons of sediment. Its cleaning and the disposal of the ballast

sludge produced involve enormous costs, (approximately 30.000 € for a small bulk carrier), as well as job hazards and time. Furthermore, as the sediment cannot be removed, the freight capacity of the ship decreases with time and stability problems arise. (Source: http://www.bawapla.com/objectives.htm)

10

Besides these economic aspects, ballast water has been recognised as a major vector for the translocation of aquatic species across biogeographical boundaries.

It is estimated that as many as 10,000 alien species of plants and animals are transported per day in ships around the world. As ships travel faster and world trade grows, organisms are better able to survive the journey, using the settled sediments as a substrate, and the threat of invasive species from ballast water

increases. Thus with a reduction of sediment settlement in ballast tanks, a significantly reduced danger of alien organisms can be expected. (Source:

http://www.bawapla.com/objectives.htm)

At July 2005, eight countries (Argentina, Australia, Brazil, Finland, Maldives, The Netherlands, Spain and Syrian Arab Republic) have signed the Ballast Water Management Convention, subject to ratification . (Source: http://

www.imo.org)

11

Options being considered include: • Mechanical treatment methods such as filtration and separation.

• Physical treatment methods such as sterilization by ozone, ultra-violet light, electric currents and heat treatment.

• Designing ships that do not have structural supports and odd corners in the ballast tanks that trap organisms and ballast water.

• Chemical treatment methods such adding biocides to ballast water to kill organisms.

Various combinations of the above.

Treatment options must not interfere unduly with the safe and economical operation of the ship and must consider ship design limitations. Any control

measure that is developed must meet a number of criteria, including: •It must be safe. • It must be environmentally acceptable. • It must be cost-effective. • It must work.

(Source: http://globallast.imo.org/index.asp?page=ballastw_treatm.htm&menu=true. Here is a useful reference on BW treatment technologies:

http://www.lr.org/Images/BWT0210_tcm155-175072.pdf)

12

Unlike other forms of marine pollution, such as oil spills, where ameliorative action can be taken and from which the environment will

eventually recover, the impacts of invasive marine species are most often irreversible

In essence, ballast water is a major pathway of biological invasion throughout the world. A promising way to prevent the introduction of

bioinvaders that cause ecological and economic harm is to remove organisms from ballast water. No one method has yet been proven to remove all organisms from ballast water and therefore more research

must be conducted in this area.

13

14

Source: Australian Quarantine & Inspection Service 1993. Ballast Water Management. Ballast Water Research Series Report No. 4 AGPS Canberra.

The dead weight tonnage (DWT) is also called loaded displacement. It is equal to the difference in volume between the displacement of water by the ship

when fully loaded (loaded draft) and the displacement considering the weight of the hull, all equipment, machinery, and boilers (ballast draft). In other

words, DWT is a measure of the weight the ship is designed to carry.

15

Countries have tried to control the problem by requiring incoming ships to flush their ballast tanks in the open ocean. Governments are preparing to

impose tougher standards on discharged ballast water. In response, several companies have developed systems to kill organisms in

ballast tanks. Studies are going on in order to minimize the problem. IMO Assembly Resolution A.868(20) - “Guidelines for control and

management of ships’ ballast water to minimize the transfer of harmful aquatic organisms and pathogens”.

16

Clam life cycle Prawn life cycle

Source: http://globallast.imo.org/problem.htm

Even considering that inland water navigation in Brazil is still far from being used in its full capacity a number of invasive aquatic species have been discovered in the most

important river and sea ports of that country.

17

In Brazil, for example, it is verified that an invasion of the golden mussel “L. fortunei”, proceeding from the ships’ ballast water has ocurred. This is a native species from

Chinese rivers and arroyos and from the Asian Southeast. Only recently, for unknown reasons, it comes expanding its distribution around the world. From Bacia do Prata’s

estuary, it has expanded quickly for the superior stretches of Paraná River’s Basin, invading, mainly, the great rivers, on a speed about 240 km/year. In 2001, its presence was reported in Itaipu’s plant, and, in 2002, it was found in the hydroelectric plants of Porto Primavera and Sérgio Motta, down Paraná River, in São Paulo. The entrance of

this species in this system of rivers must have occurred through the intense navigation and transposition of boats used in the fishery. The impact of the golden mussel in Brazil has been great and has caused public health problems; clogging of tubulations, filters of hydroelectric plants and water sucking bombs; degradation of the native species; and,

also, problems related to the fishery. (Source: Management of Ballast Water in Brazil, presented at ICBWM 2008, by N.N. Pereira, R.C. Botter, M.M. Henrique)

18

Structuring of the analysis

19

A group of Brazilian of scientists and engineers comprising fifteen civilian and four military experts and including members of the Brazilian delegation to IMO’s

International Conference on Ballast Water Management for Ships was asked to demonstrate the feasibility of evaluating alternative ballast water treatment

technologies and to produce a ranking of those by using Multicriteria Analysis. The technical leadership was one of the authors of this paper; their first challenge was

to design the evaluation process.

These analysts took the experience of IMO and decided to follow the following steps:

Step 1: identify alternative ballast water management technologies according to the experience of IMO Member States;Step 2: same for evaluation criteria;Step 3: obtain consensus about the alternatives to be evaluated and about the criteria set;Step 4: identify the importance to criteria by their relative weights;Step 5: rank the alternatives by an acceptable multicriteria method;Step 6: run a sensitivity analysis;Step 7: produce a recommendation for potential users.

The multicriteria methods to be used should allow incorporating their value judgment based on their experiences as well as the experiences of IMO

Member States. The evaluation process should be taken as a learning process. Finally, it should also provide a recommendation for selecting the best ballast

water exchange and treatment methods.20

After a carefully, first screening three competitive, alternative ballast water management technologies were identified. The nineteen professionals participated full-time in that first screening, that took two weeks (in IMO). The studies in Brazil

took more then five months (before IMO). The alternatives to be analyzed were then named Management Method 1, Management Method 2, and Management Method 3. Although they were hypothetical alternatives their characteristics are quite typical of

existing technologies.

In order to apply Multicriteria Analysis to the problem the following clusters of criteria were thus identified:

• Practicability•Biological effectiveness (including pathogens) • Cost/benefit•Time frame within which the IMO standards could be practically implemented•Environmental impact of the sub-products from the use of each technology.

One week of extensive discussions lead the experts to agree on the following evaluation criteria. Those criteria were considered

exhaustive, non redundant and operational:21

22

a) Practicabilitya.1) Quantitative criteriaC1 - what ballast flow rate range is the system applicable? (m3/hour) (specify the minimum and maximum flow rate)C2 - what is the ship tonnage that the system can be applied to? (DWT) (specify the minimum and maximum tonnage)C3 - what is the additional workload on board? (man/hours)C4 - what is the highest sea state (in the Beaufort wind scale) on which the system can operate?C5 - what is the increase in tank's sediment caused by the system? (specify percentage)a.2) Questions that need to be answered by a nominal scale, subject to association to a numerical scale of intervals or by a yes/no answerC6 - does the system present any risks to the ship's crew safety or to the crew? (-3, high risk; -2, medium risk; -1, low risk; 0, no risk)C7 - does the system affect the tanks' corrosion rate? (-2, increases the rate; -1, does not increase the rate; 0, reduces the rate)C8 - does the system dispense with the need to keep chemical products on board? (Yes or No)C9 - can the system be used in short voyages (up to 12 h)? (Yes or No)C10 - can the system be operated without complete re-circulation of the ballast water? (Yes or No)C11 - is the system unaffected by incrustation that could lead to a drop in pressure and/or to a reduction in the flow rate? (Yes or No)C12 - is the system being applicable to existing ships? (Yes or No)C13 - are the ship's other functions independent from the system's operation? (Yes or No)a.2) Questions that require detailed answersC14 - does the system present any occupational hazard to the operator? Describe and quantify. (-3, high; -2, medium; -1, low; 0, no hazard)

23

b) Biological effectiveness (including pathogens)b.1) Quantitative CriteriaC15 - how effective is the system in relation to the removal, elimination and inactivation/neutralization of aquatic organisms, apart from pathogens (according to the various taxonomic groups)? (quantify in terms of percentage, size and/or concentration of organisms)C16 - same as 15 for pathogens.b.2) Questions for which the answers should be either Yes or NoC17 - does the system eliminate cysts?C18 - does the system allow the elimination of organisms when the water enters the tank?C19 - is the system adequate for the elimination of all species or life stages that may present a hazard to the environment?

c) Cost-benefitc.1) Quantitative criteriaC20 - what is the purchase cost? (US$)C21 - what is the cost of installation? (US$)C22 - what is the operational cost? (US$/ton)C23 - what is the cost variation per ship size? (US$/ton)C24 - what is the increase of fuel or oil consumption that is introduced by the use of this system on board? (percentage)

In the multicriteria analyses all criteria should be considered as having the same weight due to difficulties in achieving consensus about the

weights. The nineteen experts considered the 26 criteria relevant enough to be jointly considered in the structuring of the analysis. That family of evaluation criteria reflected the convergence of a number of

technical, environmental, and political opinions and positions.

24

d) Time frame within which the standards could be practically implementedd.1) Quantitative criteriaC25 - within which time frame could the standards be practically implemented? (no. of months)

e) Influence of the system's sub-products on the environmente.1) Question for which the answer should be either Yes or NoC26 - is the system free from generating sub-products that can have an impact on the environment?Observation: Undesirable outcomes are taken with negative values as well as those that have a negative impact with higher absolute values. According to that: i) In the criteria 3, 5, 20-24 and 25, negative values are assign for the lowest desirable feature; ii) In the criteria 8 to 13, 17, 18, 19 and 26, where the answers should be either "Yes" or "No", a value of 1 was assigned to a "Yes" answer (desirable) and a value of 0 to a "No" answer (undesirable); and iii) In the criteria 6, 7 and 14, verbal (or nominal) scales associated to a numerical scale have been created for test purposes.

One parentheses: where does politics enter into the game? One example follows:

• Relatively cheap methods exist for treating BW, one of them being the exchange for lifeless water. Lifeless water can be normally found at 20 km far from coastal areas and in places with more than 2,500 m of depth. This is a method for treatment that can be used by countries such as Brazil. It cannot be used in countries formed by many islands neither by say European countries with relatively small sea areas.

• Some countries such as Brazil would have therefore lower costs for treating BW than other countries . Vetoing the exchange for lifeless water would be something natural from these countries. Besides, it may happen that some countries would like to impose treatment technologies produced by local companies instead of accepting a wide diversity of solutions.

• The application of Multicriteria Analysis to the problem helped in providing for a much broader view of the problem. It also contributed to overpass the monocriterion perspective of pure politics.

25

On the conceptual side we followed Milan Zeleny’s thinking when he says, referring to three alternatives A, B, and I: Because the choice

between A and B is influenced by the position of I (and therefore by some of the remaining alternatives) observe that the independence of

irrelevant alternatives axiom has no place in describing human decision making. So-called “irrelevant “ alternatives are actually very relevant

and human preferences are a function of the available alternatives and change as these alternatives vary. (Zeleny, M. HSM Integrating

Knowledge, Management and Systems. Singapore: World Scientific Publishing, 2005, p. 273)

26

In the multicriteria analyses all criteria should be considered as having the same weight due to difficulties in achieving consensus about the

weights. The nineteen experts considered the 26 criteria relevant enough to be jointly considered in the analysis. That family of evaluation

criteria reflected the convergence of a number of technical, environmental, and political opinions and positions.

Three existing ballast water management technologies were considered in the analysis: Management Methods 1, 2, and 3. The evaluation matrix below

was filled by experts.

27

Criteria Alternatives

Management Method 1 Management Method 2 Management Method 3

C1 Maximum 15,000 m3/hMinimum 100 m3/h

Maximum 14,000 m3/hMinimum 200 m3/h

Maximum 13,000 m3/hMinimum 300 m3/h

C2 Maximum 450,000 DWTMinimum 450 DWT

Maximum 350,000 DWTMinimum 350 DWT

Maximum 250,000 DWTMinimum 450 DWT

C3 90 man/h 80 man/h 90 man/h

C4 7 8 10C5 10 % 12 % 5 %C6 -1 -2 -3C7 -2 -1 -3C8 1 1 0C9 1 1 0C10 1 1 0C11 0 1 1C12 0 1 1C13 0 0 1C14 0 -1 -2C15 93 % 92 % 90 %C16 90 % 88 % 91 %C17 1 0 1C18 1 0 0C19 0 1 1C20 US$ 200,000.00 US$ 210,000.00 US$ 220,000.00C21 US$ 10,000.00 US$ 21,000.00 US$ 1,000.00C22 0.02 $/ton 0.03 $/ton 0.04 $/tonC23 US$ 9 US$ 8 US$ 6C24 3 % 8 % 1 %C25 6 months 8 months 9 monthsC26 0 1 0

Choice of methods

28

Characteristics of the two methods:

• While TODIM has elements from the North American as well as the European School, THOR is based on the ELECTRE family of methods and takes care of imprecision both by fuzzy and rough set theoretical modelling

• Both are non compensatory methods

29

Although a wide variety of tools could be used, the analysts decided to use two multicriteria methods that have been conceived and developed by Brazilian researchers. For this reason TODIM and THOR were chosen as

analytical tools.

The TODIM Method:

• The TODIM (an acronym in Portuguese for Interactive and Multicriteria Decision Making) method is a discrete multicriteria method founded on Prospect Theory (PT)

• The multiattribute value function of TODIM is built in parts, with their mathematical descriptions reproducing the gain/loss function of PT. The global multiattribute value function of TODIM aggregates all measures of gains and losses over all criteria; that function is based on Tversky’s idea of an additive difference function

• Getting the weights of criteria• Choosing a reference criterion• Valuation of the alternatives in relation to each criterion, with

normalization by dividing each weight by largest weight for each criterion• Computing a matrix of relative dominance• Calculation of the measurements of the overall desirability of each

alternative• Sensitivity analysis• An Excel®-based spreadsheet for calculations and for producing graphs

30

Measure of relative dominance of each alternative i over another alternative j:

Variables and parameters of TODIM:(i,j) – dominance measurement of an alternative i in relation to another alternative jc (i, j) – measure of value of alternative I as compared against alternative j according to criterion carc – substitution rate for criterion c as compared against criterion r wic and wjc – valuations of alternatives i and j according to criterion c – attenuation factor, applicable to the segment of the value function in the negative quadrant i – overall desirability of each alternative

31

m

1cc j)(i, ),,( j)(i, ji

Gain, No gain & no loss, and Loss parts of the value function:

32

0w wif a

)w)(wa(

θ

1

0w wif 0

0w wif a

)w(wa

j)(i,Φ

jc icrc

icjcc

rc

jc ic

jc ic

crc

jcicrc

c

Other mathematical formulations that have been tested for representing the loss and gain sections of the prospect theoretical value function:

• Tanh and the shifted tanh• Cumulative PT

33

Desirability of each alternative:

34

n

j

n

j

n

j

n

j

jiji

jiji

i

1 1

1 1

),(min),(max

.),(min),(

35

THOR:

• A multicriteria decision support system based on ELECTRE methods: • weights of criteria• preference and indifference thresholds for each criterion• concordance and discordance thresholds• fuzzy and rough set theoretical resources for tackling imprecision• a module that allows a group of decision makers to reach a decision

through the exchange of views of group members, from which negotiation around the acceptable proposals starts

36

The THOR software is an implementation of the algorithm in Delphi 7.0 (Dbase in Firebird) and it allows:

• Conducting analyses of sensibility by changing weights, level of uncertainties involved, etc, etc

• Comparing the three types of ranking rules; and • Eliminating irrelevant criteria

37

38

When using THOR, 3 different situations in terms of ranking rules may be used and are shown below. S3 is less strict that S1 and S2 and therefore can

lead to more dominances of one alternative over another. S1 allows a greater number of ties that S3. S2 is an intermediate situation between S1

and S3.

- S1 :

- S2 :

- S3 :

n

jjjjjjJ

n

jjj abPabQbaRbaIbaQwbaPw

11

)|()|(

n

jjjjjJ

n

jjjj abPabQbaRbaIwbaQbaPw

11

)|() |(

n

jjjjJ

n

jjjjj abPabQbaRwbaIbaQbaPw

11

)|() |(

Some characteristics of the use of THOR:

• When using THOR the alternatives could be compared according to the three different situations, S1, S2, and S3.

• Veto criterion – the candidate ballast water treatment technology should not present any unacceptable restrictions.

• Undesirable outcomes were considered as having negative values side-by-side with outcomes having a negative impact with higher absolute values. According to that:

•In criteria 3, 5, 20-24 and 25 negative values are assigned to the lowest desirable feature

•In criteria 8 to 13, 17, 18, 19 and 26, where the answers should be either “Yes” or “No”, a value of 1 was assigned to a “Yes” answer (desirable) and a value of 0 to a “No” answer (undesirable)

• In criteria 6, 7 and 14 verbal (or nominal) scales associated to a numerical scale were created for checking purposes.

39

Some publications on TODIM, THOR and extensions:

• CHEN, F.-d.; ZHANG, X.; Kang, F.; FAN, Z.-p. and CHEN, X. A Method for Interval Multiple Attribute Decision Making With Loss Aversion. 2010 International Conference of Information Science and Management Engineering, IEEE Computer Society, 453-456, 2010.

• GOMES, L. F. A. M. ; RANGEL, L.A.D. An Application of the TODIM Method to the Multicriteria Rental Evaluation of Residential Properties. European Journal of Operational Research, v. 193, p. 204-211, 2009.

• GOMES, L. F. A. M. ; RANGEL, L.A.D.; MARANHÃO, F. J. C. Multicriteria Analysis of Natural Gas Destination in Brazil: An application of the TODIM Method . Mathematical and Computer Modelling, v. 50, p. 92-100, 2009.

• GOMES, C. F. S. ; XAVIER, L.H.; VALLE, R. Multicriteria Decision Making Applied to Waste Recycling in Brazil. Omega, v. 36, p. 395-404, 2008.

• GOMES, L.F.A.M.; LIMA, M.M.P.P. From Modelling Individual Preferences to Multicriteria Ranking of Discrete Alternatives: A Look at Prospect Theory and the Additive Difference Model. Foundations of Computing and Decision Sciences, v. 17, p. 171-184, 1992.

• GOMES, L.F.A.M.; LIMA, M.M.P.P. Todim: Basics and Application to Multicriteria Ranking of Projects with Environmental Impacts. Foundations of Computing and Decision Sciences, v. 16, p. 113-127, 1992.

40

Computations

41

In the evaluation matrix undesirable outcomes were considered with negative values as well as those that have a negative impact with higher absolute values. According to that: i) In the criteria 3, 5, 20-24 and 25,

negative values are assign for the lowest desirable feature; ii) In the criteria 8 to 13, 17, 18, 19 and 26, where the answers should be either "Yes" or "No", a value of 1 was assigned to a "Yes" answer (desirable)

and a value of 0 to a "No" answer (undesirable); and iii) In the criteria 6, 7 and 14, verbal (or nominal) scales associated to a numerical scale have

been created for test purposes.

All values in the evaluation matrix were transformed into maximization criteria and normalized for the use of the TODIM method. The THOR

method used data as shown in that matrix.

42

43

Alternatives for θ = 1.0 for θ = 5.0 for θ = 10.0Management

Method 1 1.000 1.000 1.000Management

Method 2 0.808 0.000 0.000Management

Method 2 0.000 0.210 0.425

The application of TODIM took into consideration three possible situations: (i) attenuation factor θ equal to 1.0 (less risk proneness); (ii) θ equal to 10.0

(greater risk proneness); and (iii) θ equal to 5.0 (an intermediate value between the two previous extreme situations). Results from the computations are presented in the table below. This table shows that MM1 ≻ MM2 ≻ MM3

Performance of the three alternatives computed by the TODIM method

44

TODIM for = 1

Value function for TODIM, θ = 1

-1,5

-1

-0,5

0

0,5

1

1,5

-1,5 -1 -0,5 0 0,5 1 1,5

Losses and gains

45

TODIM for = 5

Value function for TODIM, θ = 5

-1,5

-1

-0,5

0

0,5

1

1,5

-1,5 -1 -0,5 0 0,5 1 1,5

Losses and gainsLosses and gains

Value function for TODIM, θ = 5

46

TODIM for = 10

Value function for TODIM, θ = 10

-1,5

-1

-0,5

0

0,5

1

1,5

-1,5 -1 -0,5 0 0,5 1 1,5

Losses and gains

47

Criteria and alternatives in the THOR software

48An output from THOR

By using THOR it could be seen that Management Method 1 ranked higher that the two other alternatives, being slightly better than

Management Method 3. There was indeed an ordinal agreement in the outputs from TODIM and THOR when θ was made equal to 5.0 and to

10.0. Both methods then produced the following rank: MM1 MM≻ 3 MM≻ 2. When θ was equal to 1.0, however, there was a

discordance between TODIM and THOR concerning the last two alternatives: MM1 MM≻ 2 MM≻ 3 from TODIM and MM1 MM≻ 3 MM≻ 2

from THOR. Nevertheless, since this was a problem in technology choice, for practical purposes it was concluded that the two methods produced

a similar results. Taking into account the computations from TODIM for θ equal to 5.0 and to 10.0 as well as the outputs from using THOR, it was

clear that MM2 could be ignored. Therefore if a rank were to be produced only MM1 and MM3 would be in that rank. The final results is

that MM1 should be taken as the best choice.

49

Conclusions

50

The multicriteria analysis demonstrated that the choice of an overall best treatment should also be based on technical criteria and not merely on

political grounds.

It is worth noting that although the two methods rely on different bases they produced in essence quite similar results. Other applications of both TODIM and THOR have confirmed the convergence of results in

spite of the conceptual and technical differences between the two methods: while TODIM is founded on the paradigm of Prospect Theory

and data are aggregated by means of building an additive value function, THOR relies on the notion of outranking and does not take

into account the attitude of a decision maker facing risk. The fact that they produce similar results suggests that structuring a decision

problem in a comprehensive way and applying a method correctly may be at least as important as the technical characteristics of the method

per se.

51

Acknowledgements:This work was partially supported by CNPq through Research Projects

No. 310603/2009-9 and 502711/2009-4.

Thank you!

52