A Framework for Comparing Different Information Security Risk Analysis Methodologies Anita Vorster, Les Labuschagne

Ajay Kumar Iduri Sushma Sachidanand

Quang Tran Vinh

Overview

• Introduction • Information Security Risk Management

Methodologies • Criteria on which Framework is based• The Framework for Comparison • How the Framework should be used • Strengths and Weakness of this proposed

Framework• Conclusion• References

Introduction• Information security is an organization’s approach to

maintaining confidentiality, availability, integrity, nonrepudiation,accountability, authenticity and reliability of its IT systems

• Currently there are numerous risk analysis methodologies available some of which are qualitative while others are quantitative in nature

• These methodologies have a common goal of estimating the overall risk value

Introduction• An easy-to-use framework is required to compare

information security risk analysis methodologies• The best way to choose between methodologies is to

compare them, using objective, quantifiable criteria• This is where a framework for comparison is needed• If the criteria that are used are applicable to all risk

analysis methodologies, the organization can compare different methodologies objectively, and decide on the best one

Alternative Frameworks

• The framework proposed by Badenhorst

indicates whether a methodology addresses a criterion or not

• It does not use scales, or trade-offs which can aid the organization in choosing a methodology which will best meet their needs

• This shows the need for more Comparative Frameworks

Information Security Risk Management Methodologies

• The Methodologies can be broadly classified into two categories

• Quantitative Methodologies • Qualitative Methodologies

• Both qualitative and quantitative methodologies were

evaluated for developing the framework• Different risk analysis methodologies were analyzed

to determine common criteria for comparison

Qualitative Methodologies

• The qualitative methodologies considered for this framework are

• OCTAVE (Operationally Critical Threat, Asset and Vulnerability Evaluation)

• The CORAS (Construct a platform for Risk Analysis of Security Critical Systems) methodology

Quantitative Methodologies• The quantitative methodologies considered for

this framework are • ISRAM (Information Security Risk Analysis

Method)• Cost-Of-Risk Analysis (CORA) • Information Systems (IS) analysis based on a

business model • The above methodologies were chosen because

they have been well documented

OCTAVE• OCTAVE was developed at the CERT Coordination

Center (CERT/CC) [Cert Coordination Center 2003]• This approach concentrates on assets, threats and

vulnerabilities• One of the main concepts of OCTAVE is self-

direction, the people inside the organization must lead the information security risk evaluation

• An analysis team, consisting of staff from the organization's business units as well as the IT department, is responsible for leading the evaluation and recording results

OCTAVE• The OCTAVE approach has three phases, with each

broken down into processes• Each process has certain activities that must be

completed, and within each of these activities, different steps must be taken in order to achieve the desired outputs

• The final result that risk decisions can be based on is the threat profile of different assets

• Each threat profile contains information on which mitigation decisions can be based

CORAS• CORAS was developed under the Information

Society Technologies (IST) program• One of the main objectives of CORAS is to develop a

framework that exploits methods for risk analysis, semi-formal methods for object-oriented modeling, and computerized tools, for a precise, unambiguous, and efficient risk assessment of security critical systems

• The methodology is based on UML a language that uses diagrams to illustrate relationships and dependencies between users and the environment in which they work

CORAS

• During an information security risk analysis, a great deal of information is brainstormed, and during workshops and discussions, different people (users, system developers, analysts, system managers), with different expertise in different fields come together, give their opinions and share information

• A way in which all the participants can communicate efficiently and understand each other must therefore exist and a UML profile, proposed by the CORAS project, is used to achieve this

CORAS

• The framework has four main pillars, of which risk management is one. In CORAS, the final result on which decisions can be based is the UML class diagrams of each asset

ISRAM• The ISRAM methodology was developed at the

National Research Institute of Electronics and Cryptology and the Gebze Institute of Technology in Turkey

• It is marketed as a quantitative approach to risk analysis that allows for the participation of the manager and staff of the organization and a survey-based model

• Two separate and independent surveys are conducted for the two attributes of risk, namely probability and consequence

ISRAM

• ISRAM does not use techniques such as Single Occurrence Losses (SOL) or Annual Loss Expectancy (ALE), instead, the risk factor is a numerical value between 1 and 25

• This numerical value corresponds to a qualitative, high, medium or low value, and it is this qualitative value on which risk management decisions are based

CORA• International Security Technology, Inc. (IST)

developed CORA, the Cost-Of-Risk Analysis system

• The CORA risk model uses data collected about threats, functions and assets, and the vulnerabilities of the functions and assets to the threats to calculate the consequences, that is, the losses due to the occurrences of the threats

CORA• It is a methodology where the risk parameters

are expressed quantitatively and where losses are expressed in quantitative monetary terms

• CORA uses a two-step process to support risk management. Parameters for threats, functions and assets are validated and refined until the best values are determined

CORA• CORA then calculates SOL and ALE for each

of the threats identified

• It estimates a single loss value for a threat to an organization, and then multiplies this value by the frequency of the threat occurrence

IS Risk Analysis based on a Business Model

• The IS Risk Analysis Based on a Business Model was developed at the Korea Advanced Institute of Science and Technology

• This model was developed because traditional risk analysis methodologies had some limitations

• It takes an asset’s value and then not only bases the analysis on its replacement cost, but also measures the tangible asset’s value from the viewpoint of operational continuity

IS Risk Analysis

• The methodology has four stages• During this methodology, the importance level of

various business functions of the business model and the necessity level of various IS assets are determined

• Mathematical formulae are used to calculate ALE for a single threat occurrence on the organization

• The end result is a quantitative monetary value

Criteria for the Framework• Whether risk analysis is done on single assets or

groups of assets• Where in the methodology risk analysis is done• The people involved in the risk analysis• The main formulae used• Whether the results of the methodology are

relative or absolute

Criteria for the Framework

• Each criterion has a scaling• The scaling indicates the level of a criterion

based on certain trade-offs• In the end a compliance factor must be

selected to indicate how relative the criterion is to a methodology

Whether Risk Analysis is done on Single Assets or Group of Assets

• Criteria can either take on a value of 1 or 2, as follows:

• 1: if risk analysis is done on individual assets• 2: if risk analysis is done on groups of assets

• In the case of the OCTAVE and CORAS methodology, the risk analysis is done on a single asset

• If the end result is a single value for a threat scenario that can affect more than one asset, the risk analysis is done on a group of assets, such as in the case of the CORA ,ISRAM and IS Business model methodology

Where in the Methodology Risk Analysis is Done

• This criterion explains where in the methodology risk analysis takes place

• Some preparation, where values for information are estimated, must be done before risk analysis can be performed

• Some risk analysis methodologies may require more values for different information to be estimated than other methodologies

Where in the Methodology Risk Analysis is Done

• OCTAVE needs values for impact and probability• IS Risk Analysis Based on a Business Model

needs values for probability, income loss, replacement cost and relative importance of business functions

• Methodology that needs little preparation and less information is CORA

• An accurate risk analysis, more preparation means more detailed results, thus ISRAM would be a better option

Scale for this Criterion

• The scale for this criterion is based on a trade-off between time and accuracy.

• If time is most important• 1: Risk analysis done after extensive preparation• 2: Risk analysis done after some preparation• 3: Risk analysis done after little preparation

• If accuracy is most important• 1: Risk analysis done after little preparation• 2: Risk analysis done after some preparation• 3: Risk analysis done after extensive preparation

The People Involved in the Risk Analysis

• The people involved in the risk analysis can either be internal or external to the organization

• CORA uses external risk experts to perform the risk analysis, whereas OCTAVE uses internal personnel exclusively

Scale for this Criterion

• The scale for this criterion is based on a trade-off between cost and expertise

• If cost is most important, values are as follows:• 1: Risk analysis is performed by external experts• 2: Risk analysis is performed by external and internal people• 3: Risk analysis is performed by internal people

• If expertise is most important, values are as follows:• 1: Risk analysis is performed by internal people• 2: Risk analysis is performed by external and internal people• 3: Risk analysis is performed by external experts

The Main Formulae Used

• Some methodologies use mathematical formulae while others use an expected value matrix

• The main formula shows the magnitude of calculations that need to be done, thus indicating the complexity of the risk analysis

• The scale for this criterion is based on a trade-off between simplicity and accuracy

Scale for this Criterion

• If simplicity is most important, values are as follows:• 1: Risk analysis involves extensive mathematical

calculations• 2: Risk analysis involves some but simple

mathematical calculations• 3: Risk analysis involves no mathematical calculations

• If accuracy is most important, values are as follows:• 1: Risk analysis involves no mathematical calculations• 2: Risk analysis involves some but simple

mathematical calculations• 3: Risk analysis involves extensive mathematical

calculations

Whether the Results of the Methodology are Relative or Absolute

• Some methodologies produce results that are relative. This means that there is no relationship between results and they cannot be compared

• Other methodologies produce results that are absolute and can be compared

• The scale for this criterion is based on a trade-off between a methodology providing merely a ranking of risks, or a methodology that can calculate how much greater one risk is over another

Scale for this Criterion

• If merely ranking of risks is most important, values are as follows:

• 1: Results are comparable• 2: Results are not comparable

• If the differences between risks (how much greater one is over another) are most important, values are as follows:

• 1: Results are not comparable• 2: Results are comparable

The Framework for Comparison

The Main Formulae Usedin OCTAVE

• The OCTAVE methodology uses an Expected Value Matrix to determine a risk’s expected value.

• The main formula is:• Loss = Impact/consequence x Probability

The Main Formulae Usedin CORAS

• The CORAS methodology also uses the impact and probability approach.

• Loss = Impact x Probability

The Main Formulae Usedin ISRAM

The Main Formulae Usedin CORA

• ALE = Consequence x Frequency• consequence = Σn(individual SOL’s) n the

number of single occurrence losses, and • SOL = loss potential (worst case monetary value)

x vulnerability• CORA uses some, but not extensive

mathematical calculations. It gets a value of 2 for both simplicity and accuracy

The Main Formulae Usedin IS

• IS Risk Analysis Based on a Business Model uses the following:

How the Framework Should be Used

• The use of the framework is rather simplistic, which allows for use by more organizations

• Firstly the organization must identify their specific needs.

• Then, based on the scales of each criterion as defined earlier, the organization must determine values for the specific methodologies they want to compare

Strengths• Can be applied to various risk analysis

methodologies • Takes the requirements of an organization into

account• It uses scales based on different scenarios and trade-

offs• Can give an indication of which assets and people

will be needed for the risk analysis as based on the requirements of the organization

Weakness• Not taking the customization of a methodology into

account• The OCTAVE methodology can be tailored to fit

the needs of an organization• Not all processes have to be performed, which can

influence the place where risk analysis fits into the methodology

• The preparation required can therefore be reduced• The risk analysis based on the requirements of an

organization

Weakness

• The existence of other criteria, not presented by the framework

• There are many other risk analysis methodologies, such as CRAMM and there are also baselines, which cover a wider variety of information security aspects, such as the ISO 17799 framework and which can be used to define other criteria

Conclusion• Numerous methodologies are currently available and

many organizations are faced with the daunting task of determining which one to use

• The goal was to develop an easy-to-use framework that organizations can employ to compare different information security risk analysis methodologies

• The main benefit lies in the ability to eliminate the majority of methodologies that are unsuitable and to only further investigate the few that remain

References• A Framework for Comparing Different Information Security

Risk Analysis Methodologies ANITA VORSTER And LES LABUSCHANGE

• BADENHORST, K.P, ELOFF, J.H.P, AND LABUSCHAGNE, L, 1993, A comparative framework for risk analysis methods, Computers & Security

• CERT COORDINATION CENTER, 2003, The OCTAVE approach, http://www.cert.org/

• FREDRIKSEN, R, KRISTIANSEN, M, GRAN, B, AND STOLEN, K, 2001. The CORAS framework for a model-based risk management process, http://coras.sourceforge.net/documents/2002-Safecomp.pdf

• INTERNATIONAL SECURITY TECHNOLOGY Inc (IST Inc). 2000. Managing risks using CORA, PowerPoint presentation www.ist-usa.com