SINTEF ICT Networked Systems and Services 2012-06-22

SINTEF A22798- Unrestricted

Report

Conceptual Framework for the DIAMONDS Project Author(s) Gencer Erdogan, Yan Li, Ragnhild Kobro Runde, Fredrik Seehusen, Ketil Stølen

Conceptual Framework for theDIAMONDS Project

Gencer Erdogan1,2 Yan Li1,2 Ragnhild Kobro Runde1

Fredrik Seehusen2 Ketil Stølen1,2

1 Department of Informatics, University of Oslo P.O. Box 1080Blindern, N-0316 Oslo, Norway

ragnhilk@ifi.uio.no

2 Department for Networked Systems and Services, SINTEF ICTP.O. Box 124 Blindern, N-0314 Oslo, Norway

{gencer.erdogan,yan.li,fredrik.seehusen,ketil.stolen}@sintef.no

June 22, 2012

Contents1 Introduction 5

2 Risk Management 6

3 Risk Analysis 9

4 Security 11

5 Testing 13

6 Security Risk Analysis 18

7 Security Testing 20

8 Model 21

9 Model-based Security Risk Analysis 22

10 Model-based Security Testing 23

11 Test-driven Model-based Security Risk Analysis 24

12 Risk-driven Model-based Security Testing 28

4

1 IntroductionDIAMONDS is a research project addressing the combination of securitytesting and risk analysis. The main objective is to develop guidelines anda supporting framework to help businesses find a balanced approach withinthe three-dimensional space of invested effort, security testing, and risk anal-ysis. This report documents the conceptual framework for DIAMONDS byclarifying the notions of security testing, risk analysis, and related concepts,as well as defining the relations among them.

The conceptual framework offers a basis for future research in the projectby providing a common understanding of the central notions within securitytesting and security risk analysis. Our approach is to build the concep-tual framework upon established concepts from state-of-the-art. However,we also make adjustments of established notions where seen necessary orappropriate, in order to achieve a consistent framework suitable for the par-ticular target of the DIAMONDS project.

In DIAMONDS, we focus on model-based approaches to security testingand security risk analysis, where models are used as a main artifact bothduring the testing/analysis process and for documentation purposes. Inparticular we distinguish between model-based security testing (MST) andmodel-based security risk analysis (MSR). For combining MST and MSR,there are two main possibilities depending on which approach is taken asthe starting point, i.e., the main purpose of the process. We will refer tothese as risk-driven model-based security testing (RMST) and test-drivenmodel-based security risk analysis (TMSR).

The remainder of this report is organized as follows: We start by introduc-ing the overall risk management process in Section 2, before considering thebasic notions of risk analysis, security, and testing in Sections 3, 4, and 5,respectively. In Sections 6 and 7 we build on the previous sections whenpresenting the definitions for security risk analysis and security testing, re-spectively. The basic concepts for models are presented in Section 8. InSections 3-8 we use definitions from standards as much as possible, andadditionally provide our interpretation of the relationships between the var-ious concepts. The definitions of model-based security risk analysis (MSR)and model-based security testing (MST) are defined in Sections 9 and 10,respectively. Finally, we provide our proposal for test-driven model-basedsecurity risk analysis (TMSR) in Section 11, and for risk-driven model-basedsecurity testing (RMST) in Section 12.

5

2 Risk ManagementIt is necessary to introduce the overall risk management process before welook closer into risk analysis and other relevant concepts. ISO 31000 Riskmanagement - Principles and guidelines [7] is our main building block interms of defining and explaining the concept of risk management and theconcepts related to risk management. The overall risk management processshown in Figure 1 is taken from [7, p.14].

There is however one deviation: Our definition of risk estimation is equiva-lent to what ISO 31000 refers to as risk analysis. Instead we use the termrisk analysis in line with how the term is used in practice to denote the fivestep process in the middle of Figure 1 starting with establishing the contextand ending with risk treatment.

Communication

and

consultation

Monitoring

and

review

Establishing the context

Risk identification

Risk estimation

Risk evaluation

Risk treatment

Risk assessment

Figure 1: The overall risk management process (adapted from [7, p.14]).

As stated in ISO 31000 [7, p.1], all activities of an organization may in-volve risk. Organizations usually manage risk by identifying it, estimatingit and then evaluating it to see whether the risk should be modified byrisk treatment in order to satisfy the risk evaluation criteria. Through thisprocess, communication and consultation are carried out with stakeholdersto monitor and review the risk, and controls that are modifying the riskare implemented to ensure that no further risk treatment is required. Riskmanagement can be applied to an entire organization, at its many areas andlevels, at any time, and to specific functions, projects and activities. Al-though the practice of risk management has been developed over time andwithin many sectors in order to meet diverse needs, the adoption of consis-tent processes within a comprehensive framework can help to ensure thatrisk is managed effectively, efficiently and coherently across an organization.

6

• Risk ManagementRisk management refers to the coordinated activities to direct andcontrol an organization with regard to risk [7, p.2].

• Risk Management ProcessRisk management process is the systematic application of managementpolicies, procedures and practices to the activities of communicating,consulting, establishing the context, and identifying, analyzing, eval-uating, treating, monitoring and reviewing risk [7, p.3].

• Communication and ConsultationCommunication and consultation refers to the continual and iterativeprocesses that an organization conducts to provide, share or obtaininformation and to engage in dialog with stakeholders regarding themanagement of risk [7, p.3].

• Establishing the ContextEstablishing the context refers to the process of defining the exter-nal and internal parameters to be taken into account when managingrisk, and setting the scope and risk criteria for the remaining process(adapted from [7, p.3]).

• Risk AssessmentRisk assessment is the overall process of risk identification, risk esti-mation and risk evaluation(adapted from [7, p.4]).

• Risk IdentificationRisk identification is the process of finding, recognizing and describingrisks. This involves identifying sources of risk, areas of impacts, events(including changes in circumstances), their causes and their potentialconsequences. Risk identification can involve historical data, theoreti-cal analysis, informed and expert opinions, and stakeholder’s needs [7,p.4].

• Risk EstimationRisk estimation is the process of comprehending the nature of risk anddetermining the level of risk. This involves developing an understand-ing of the risk. Risk estimation provides the basis for risk evaluationand decisions on whether risks need to be treated, and on the mostappropriate risk treatment strategies and methods (adapted from [7,p.5]).

7

• Risk EvaluationRisk evaluation is the process of comparing the results of risk esti-mation with risk criteria to determine whether the risk and/or itsmagnitude is acceptable or tolerable. Risk evaluation assists in thedecision about risk treatment (adapted from [7, p.6]).

• Risk TreatmentRisk treatment is the process of modifying risk which can involve riskmitigation, risk elimination or risk prevention.(adapted from [7, p.6]).

• Risk AnalysisRisk analysis is a collective term defining the process consisting ofthe following steps: establishing the context, risk identification, riskestimation, risk evaluation and risk treatment (adapted from [7, p.14]).

• MonitoringMonitoring is the continual checking, supervising, critically observingor determining the status in order to identify change from the perfor-mance level required or expected [7, p.7].

• ReviewReview is the activity undertaken to determine the suitability, ade-quacy and effectiveness of the subject matter to achieve establishedobjectives [7, p.7].

8

3 Risk AnalysisThe conceptual model and notions defined here are based on the ISO 31000standard [7]. Figure 2 shows the conceptual model for risk analysis adoptedby the DIAMONDS project.

Figure 2: Conceptual model for risk analysis.

• RiskRisk is the combination of the consequences of an event with respectto an objective and the associated likelihood of occurrence (adaptedfrom [7, p.1]).

• ObjectiveAn objective1 is something the stakeholder is aiming towards or astrategic position it is working to attain (adapted from [12]).

• Risk SourceRisk source is an element which alone or in combination has the in-trinsic potential to give rise to risk [7, p.4].

• StakeholderStakeholder is a person or organization that can affect, be affected by,or perceive themselves to be affected by a decision or activity [7, p.4].

• EventEvent is the occurrence or change of a particular set of circumstances [7,p.4].

1Note that although the notion of objective is used in ISO 31000, it is not defined inthe standard.

9

• LikelihoodLikelihood is the chance of something happening [7, p.5].

• ConsequenceConsequence is the outcome of an event affecting objectives [7, p.5].

• Risk CriterionA risk criterion is the term of reference against which the significanceof a risk is evaluated [7, p.5].

• Risk LevelRisk level is the magnitude of a risk or combination of risks, expressedin terms of the combination of consequences and their likelihood [7,p.6].

10

4 SecurityThe terms information security, computer security and information assur-ance are frequently used interchangeably. These fields are often interrelatedand share the common goals of protecting the confidentiality, integrity andavailability of information; however, there are some subtle differences be-tween them [1].

These differences lie primarily in the approach to the subject, the method-ologies used, and the areas of concentration. Information security is mainlyconcerned with the confidentiality, integrity and availability of data regard-less of the form the data may take, such as electronic, print, or other forms.Computer security mainly focuses on ensuring the availability and correctoperation of a computer system without concern for the information storedor processed by the computer. Information assurance focuses on the reasonsfor assurance that information is protected, and is thus reasoning about in-formation security [1].

In the DIAMONDS project we use the term security in the meaning of in-formation security, where the definition of information security has beentaken from ISO/IEC 27000 Information Security Management System [6],as shown in Figure 3.

Security

Confidentiality Availability Integrity

Figure 3: Conceptual model for Security.

• SecuritySecurity refers to the preservation of confidentiality, integrity andavailability of information (adapted from [6, p.3]).

• ConfidentialityConfidentiality is the property that information is not made availableor disclosed to unauthorized individuals, entities, or processes [6, p.2].

11

• AvailabilityAvailability is the property of information being accessible and usableupon demand by an authorized entity [6, p.2].

• IntegrityIntegrity is the property of protecting the accuracy and completenessof information (adapted from [6, p.4]).

12

5 TestingIn this section, we particularly focus our conceptual clarification on softwaretesting related to the DIAMONDS project. Our primary source for the no-tion of software testing and related notions is the upcoming internationalstandard ISO/IEC 29119 Software Testing [8] defined by Software and Sys-tems Engineering Standards Committee of the IEEE Computer Society.

However, ISO/IEC 29119 is still under development, and we only have accessto a draft version of Part 2 (ISO/IEC 29119 Draft Part 2-Testing Process)at the time of writing. For related notions not found in Part 2, we use IEEE829 [5] and BS 7925-1/-2 [11], which are expected to be incorporated intoISO/IEC 29119.

As stated in ISO/IEC TR 19759 [3], testing concepts, strategies, techniques,and measures need to be integrated into a defined and controlled processwhich is run by people. The testing process provide support for testing ac-tivities as well as guidance for testing teams. It provides justified assurancethat the test objectives will be met cost-effectively. We introduce the overalltesting process (see Figure 4) defined in ISO/IEC 29119 [8] before we definethe specific testing process used in the DIAMONDS project.

Figure 4: Overall test process [8, p.7-p.22].

The testing activities that are performed during the life cycle of a softwaresystem may be grouped into a three layers test process [8, p.2], as shown inFigure 4.The aim of the organizational test process layer is to define a process for

13

the creation and maintenance of organizational test specifications, such asorganizational test policies, strategies, processes, procedures and other as-sets [8, p.2].

The aim of the test management process layer is to define processes thatcover the management of testing for a whole test project or any test phaseor test type within a test project (e.g. project test management, system testmanagement, performance test management) [8, p.2].

The aim of the dynamic test process layer is to define generic processes forperforming dynamic testing. Dynamic testing may be performed at a par-ticular phase of testing (e.g. unit, integration, system, and acceptance) orfor a particular type of testing (e.g. performance testing, security testing,and functional testing) within a test project [8, p.2].

What we refer to at the testing process in the DIAMONDS project is ba-sically what Figure 4 refers to as the dynamic test process. However, asindicated by Figure 5 we also add a test planning step capturing the testplanning of the test management process of relevance for one run of thedynamic test process.

Test Planning

Test Design &

Implementation

Test Execution

Testing Process

Test Incident

Reporting

Test Environment

Set-up &

Maintenance

Figure 5: The testing process used in DIAMONDS, adapted from [8].

14

• TestingTesting is the process of exercising the system to verify that it satisfiesspecified requirements and to detect errors (adapted from [11]).

• Test PlanningThe test planning is the process of developing the test plan. Dependingon where in the project this process is implemented this may be aproject test plan or a test plan for a specific phase, such as a system testplan, or a test plan for a specific type of testing, such as a performancetest plan (adapted from [8, p.8]).

• Test Design and ImplementationThe test design and implementation is the process of deriving the testcases and test procedures (adapted from [8, p.23]).

• Test Environment Set-up and MaintenanceThe test environment set-up and maintenance process is the processof establishing and maintaining the environment in which tests areexecuted (adapted from [8, p.27]).

• Test ExecutionThe test execution is the process of running the test procedure re-sulting from the test design and implementation process on the testenvironment established by the test environment set-up and mainte-nance process. The test execution process may need to be performeda number of times as all the available test procedures may not beexecuted in a single iteration (adapted from [8, p.28]).

• Test Incident ReportingThe test incident reporting is the process of managing the test inci-dents. This process will be entered as a result of the identificationof test failures, instances where something unusual or unexpected oc-curred during test execution, or when a retest passes (adapted from [8,p.30]).

The conceptual model for testing used in DIAMONDS is defined in Figure 6.

• Test PolicyThe test policy is a document that describes the purpose, goals, andoverall scope of the testing within the organization (adapted from [8,p.4]).

15

Test planTest policy Stakeholder

Test

requirement

Feature Test

Test result

Test procedure

System

*1..*

1..*

*

11..*

*

*

1 *

1..*1..*

*

1..*

*

1

is described by

Test failureTest incident1..*1..*

Test

environment

**

Figure 6: Conceptual model for Testing.

• Test PlanTest plan is a document describing the objective, scope, approach,resources, and schedule of intended test activities(adapted from [5,p.11]).

• Test RequirementTest requirement is a capability that must be met or possessed by thesystem (requirements may be functional or non-functional) - (adaptedfrom [11]).

• SystemA system is an interacting combination of elements that aims to accom-plish a defined objective. These include hardware, software, firmware,people, information, techniques, facilities, services, and other supportelements [3, p.2-3].

• FeatureFeature is a distinguishing characteristic of a system (includes bothfunctional and nonfunctional attributes such as performance and reusabil-ity)(adapted from [5, p.9]).

• TestTest2 is a set of inputs, execution preconditions, and expected out-comes developed for a particular objective, such as to exercise a partic-

2Note that the term test as defined here is sometimes referred to as test case in otherresources.

16

ular program path or to verify compliance with a specific requirement(adapted from [11]).

• Test ProcedureDocumentation that specifies a sequence of actions for the executionof a test [5, p.11].

• Test EnvironmentTest environment is the description of the hardware and software en-vironment in which the tests will be run, and any other software withwhich the software under test interacts (adapted from [11]).

• Test IncidentTest incident is an unplanned event occurring during testing that hasa bearing on the success of the test. Most commonly raised when atest result fails to meet expectations [11].

• Test FailureTest failure is the deviation of the software from its expected deliveryor service (adapted from [11]).

• Test ResultA test result is an actual outcome or a predicted outcome of a test(adapted from [11]).

17

6 Security Risk AnalysisLund et al. [9] classify risk analysis approaches into two main categories:

• Offensive approaches: Risk analysis concerned with balancing poten-tial gain against risk of investment loss. This kind of risk analysis ismore relevant within finance and political strategy making.

• Defensive approaches: Risk analysis concerned with protecting whatis already there.

In the context of security, the defensive approach is the one that is relevant.The conceptual model for security risk analysis is shown in Figure 7. Forthe definitions of risk, objective, risk source and event see Section 3.

ObjectiveAsset

Security

requirementThreat

Risk source

Vulnerability

*1..*Risk

Security risk

Event

Unwanted

incident

Figure 7: Conceptual model for security risk analysis.

• Security Risk AnalysisSecurity risk analysis is the process of risk analysis specialized towardssecurity.

• AssetAsset is anything that has value to the stakeholders (adopted from [6]).

• Security RequirementSecurity requirement is a specification of the required security for thesystem (adopted from [11]).

• Security RiskSecurity risk is a risk caused by a threat exploiting a vulnerability andthereby violating a security requirement.

• Unwanted IncidentUnwanted incident is an event representing a security risk.

• ThreatThreat is potential cause of an unwanted incident [6].

18

• VulnerabilityVulnerability is weakness of an asset or control that can be exploitedby a threat [6].

19

7 Security TestingBased on the notions of security and testing we define security testing asfollows:

• Security TestingSecurity testing is the process of testing specialized towards security.

To be more specific, we can understand security testing as the testing and/orevaluation of the management, operational, and technical security controlsto determine the extent to which the controls are implemented correctly,operating as intended, and producing the desired outcome with respect tomeeting the security requirements for the system or enterprise [4]. The basicsecurity concepts that need to be covered by security testing are confiden-tiality, availability, and integrity. The conceptual model for security testingis shown in Figure 8. For the definition of test requirement see Section 5.

Test

requirement

Security test

requirement

Figure 8: Conceptual model for security testing.

• Security Test RequirementA security test requirement is a test requirement specialized towardssecurity.

20

8 ModelThis section clarifies the notion of model (see Figure 9). The concepts de-scribed here are based on TOGAF (The Open Group Architecture Frame-work) [10], System Analysis and Design [13] and SWEBOK (Software En-gineering Body of Knowledge) [3].

Test model

ViewModel

Test

environment

model

System modelRisk model

Figure 9: Conceptual model for model.

• ModelModel is a representation of a subject of interest. A model provides asmaller scale, simplified, and/or abstract representation of the subjectmatter (adapted from [10]).

• ViewA view represents a related set of concerns.

• System ModelA system model represents a system.

• Risk ModelA risk model represents risks.

• Test ModelA test model represents tests.

• Test Environment ModelA test environment model represents the test environment.

21

9 Model-based Security Risk AnalysisBased on the notions of model and security risk analysis we define the termmodel-based security risk analysis as follows:

• Model-Based Security Risk AnalysisModel-based security risk analysis (MSR) is security risk analysis inwhich each step of the process includes the construction and analysisof models.

22

10 Model-based Security TestingModel-based testing is a software testing approach that relies on models ofa system under test and its environment to derive test cases. Usually, thetesting model is derived in whole or in part from a model that describes func-tional or non-functional aspects (e.g. performance, security, ergonomics) ofthe system under development [2].

• Model-based Security TestingModel-based security testing (MST) is security testing that involvesthe construction and analysis of a system model, a test model and atest environment model to derive tests.

23

11 Test-driven Model-based Security Risk Analy-sis

Test-driven model-based security risk analysis is defined as the combina-tion of security risk analysis and security testing in which security testingis carried out both before and after the security risk assessment process toserve different purposes. The first usage (i.e., testing before the security riskassessment process, as shown in Step 2 in Figure 10) supports the securityrisk analysis process by identifying potential risks, while the second usage(i.e., testing after the security risk assessment process, as shown in Step 6in Figure 10) validates security risk models based on security testing results.

• Test-driven Model-based Security Risk Analysis (TMSR)Test-driven model-based security risk analysis (TMSR) is model-basedsecurity risk analysis that use testing within the risk analysis process.

Figure 10 illustrates the process of TMSR defined as a specialization of therisk analysis process illustrated in Figure 1, with establish objective intro-duced in Step 1 and risk validation introduced in Step 7. The testing processin Step 2 and Step 6 is the testing process defined in Section 5 specializedtowards security.

The following describes the steps in the TMSR process including the inputsand outputs for each step.

Step 1: Establish Objective and ContextThe first step is the initial preparation for risk analysis in which a basicidea about objectives, scale of analysis and corresponding context are estab-lished. This can be performed by an introductory meeting with the customeron the behalf of which the analysis is conducted. The representatives of thecustomer can present their overall objectives of the analysis, security require-ments and the system they wish to have analyzed in structured meetings, toensure a common initial understanding of the analysis. After defining theoverall objectives, the rest of the analysis is planned by defining assets thatneed to be defended, risk criteria and system model.

• Input: Objective, security requirement

• Output: Assets that need to be defended, risk criteria, system model

Step 2: Testing ProcessThe purpose of the testing process used in this step is to support the securityrisk analysis process for identifying potential risks. Based on the identified

24

Step1: Establish

Objective and

Context

Step3: Risk

Identification

Step5: Risk

Evaluation

Step7: Risk

Validation and

Treatment

Security Risk Analysis

Test-driven Model-based Security Risk Analysis (TMSR)

Step2: Testing

Process

Step4: Risk

Estimation

Step6: Testing

Process

Security Testing Process

Security Testing Process

Figure 10: Test-driven Model-based Security Risk Analysis process

assets, risk criteria and system models, the relevant items of the system areselected and considered as test objectives defined in test plan. Test modelsare then built according to the test objectives and security requirements, andare in further used to generate tests. After tests have been executed, the testresult is reported and directed as an input to Step 3 for risk identification.

• Input: Assets that need to be defended, risk criteria, system model

• Output: Test result

Step 3: Risk IdentificationRisk identification is the process of finding, recognizing and describing risksin terms of the events (including changes in circumstances), their risk source

25

and potential consequences to form an incomplete risk model(e.g. a riskmodel without likelihood and consequence values). It involves a systematicidentification of threats, unwanted incidents, and vulnerabilities with respectto the identified assets and according to the security test results from Step 2.Taxonomy-based questionnaires or risk checklists may be used by projectmembers in a structured brainstorm meeting. And particular focus willbe put upon security risks that are related to software functionality andrequirements.

• Input: Assets that need to be defended, system model, test result

• Output: Incomplete risk model

Step 4: Risk EstimationRisk estimation is the process of comprehending the nature of risk and de-termining the level of risk. This involves developing an understanding of therisk by defining the likelihoods and consequences of the risks identified inStep 3. These values in combination indicate the risk level for each of theidentified risks and that further form the basis of complete risk models whichwe refer to risk model. It provides the basis for risk evaluation and deci-sions on whether risks need to be treated, and on the most appropriate risktreatment strategies and methods. The risk estimation can be conducted asa brainstorming session involving personnel with various backgrounds.

• Input: Assets that need to be defended, risk criteria, system model,incomplete risk model

• Output: Risk model

Step 5: Risk EvaluationRisk evaluation is the process of comparing the results of risk estimationwith risk criteria to determine whether the risk and/or its magnitude isacceptable or tolerable, considering assets that need to be defended. Itassists in the decision about which risks need treatment and the priority fortreatment implementation according to risk criteria.

• Input: Assets that need to be defended, risk criteria, risk model

• Output: Risk prioritized with respect to risk criteria

Step 6: Testing ProcessThe testing process used here is to support the security risk analysis processfor validating the risk models. Based on the prioritized risks, risk model andsystem model, relevant system components are analyzed and considered astest objectives in test plan. Test models are then built according to testobjectives and security requirements. Furthermore, tests are created andexecuted based on the security test models. The test result is reported anddirected as an input to Step 7.

26

• Input: Assets that need to be defended, system model, risk model,risk prioritized with respect to risk criteria

• Output: Test result

Step 7: Risk Validation and TreatmentRisk validation and treatment is the process of validating risk models madeearlier and modifying risk which can involve risk mitigation, risk eliminationor risk prevention. Security testing results from Step 6 are used to validatethe risk models. The unacceptable risks are further evaluated with possibletreatments to reduce likelihoods and negative consequences, according torisk criteria. Subsequently, the risk model will then be updated as well.

• Input:Assets that need to be defended, risk criteria, risk model, testresult

• Output: Treatment, updated risk model,

27

12 Risk-driven Model-based Security TestingRisk-driven model-based security testing is defined as the combination ofsecurity testing and security risk analysis in which security risk analysis iscarried out both before and after the test design & implementation step toserve different purpose. The first usage (i.e., security risk analysis before thetest design & implementation step, as shown in Step 2 in Figure 11) supportsthe security testing process by identifying the most important parts of thesystem, while the second usage (i.e., security risk analysis after the testdesign & implementation step, as shown in Step 5 in Figure 11) supportsthe security testing process by identifying the most important security teststhat needs to be executed.

Step1: Test

Planning

Step3: Test

Design &

Implementation

Step2: Risk

Analysis

Step6: Test

Execution

Security Risk Analysis

Risk-driven Model-based Security Testing (RMST)

Step7: Test

Incident

Reporting

Step5: Risk

Analysis

Step4: Test

Environment Set-

up & Maintenance

Security Risk Analysis

Security Testing Process

Figure 11: Risk-driven Model-based Security Testing process.

28

• Risk-driven Model-based Security TestingRisk-driven model-based security testing (RMST) is model-based se-curity testing that use risk analysis within the security testing process.

The security risk analysis process in Step 2 and Step 5 in Figure 11 isequivalent to the security risk analysis process given in Figure 10.The following describes the steps in the RMST process including the inputand output for each step.

Step 1: Test PlanningThe purpose of the test planning step is to create a test plan for one run ofthe RMST process. The test plan is established in terms of “what to test”,“how to test”, “what resources to use” and “when to test” (see Section 5for the definition of test plan). The test plan is developed together withthe stakeholder and with respect to the stakeholder’s test policy, the systemmodel and the features. Based on the test policy, the system model and thefeatures, security test requirements are defined and documented as part ofthe test plan.

• Input: Test policy, system model, feature

• Output: Test plan, security test requirement

Step 2: Risk AnalysisThe security risk analysis process in this step is carried out in order toidentify security risks of the system. The system model, the features andthe security test requirements from Step 1 are used as an input to thisstep. Based on these inputs, risk models are developed in order to identifysecurity risks, and risk criteria are defined in order to estimate, evaluate andprioritize the identified security risks.

• Input: System model, feature, security test requirement

• Output: Risk model, risk criteria, prioritized security risks with re-spect to risk criteria

Step 3: Test Design and ImplementationThe purpose of this step is to derive test models, tests and test procedures.First, the most important parts of the system are identified and prioritizedwith respect to the prioritized security risks and their underlying risk mod-els obtained in Step 2. Second, the test plan is updated according to theprioritized list of the most important parts of the system. Third, the testmodels, tests and test procedures are developed according to the prioritizedlist of the most important parts of the system. The test models are derivedin terms of interaction models with respect to the information gatheredfrom the system model, risk model, features and security test requirements.

29

Tests are derived from the test models. Test procedures are derived byordering tests according to dependencies described by pre-conditions andpost-conditions within the tests and other security test requirements. Apre-condition is an environmental and state condition which must be ful-filled before a test can be executed with a particular input value (adaptedfrom [11]). A post-condition is the predicted outcome of a test under thespecified pre-condition.

• Input: Risk model, risk criteria, prioritized security risks with re-spect to risk criteria, test plan, system model, feature, security testrequirement

• Output: Test model, test, test procedure

Step 4: Test Environment Set-up and MaintenanceThe purpose of this step is to establish and maintain the required test en-vironment. The test environment is set up with respect to the test plan,system model, test model, tests and test procedures. Additionally, main-tenance of the test environment may involve changes based on the resultsof previous tests. Where change and configuration management processesexist, changes to the test environments may be managed using these pro-cesses.

• Input: Test plan, system model, test model, test, test procedure

• Output: Test environment model

Step 5: Risk AnalysisThe security risk analysis process in this step is carried out in order toidentify security risks explored by the tests. The system model, test modeland security test requirements are used as an input to this step. Based onthese inputs, risk models are developed in order to identify security risks,and risk criteria are defined in order to estimate, evaluate and prioritize theidentified security risks.

• Input: System model, test model, security test requirement

• Output: Risk model, risk criteria, prioritized security risks with re-spect to risk criteria

Step 6: Test ExecutionThe purpose of this step is to execute the tests and the test procedures inthe test environment. First, the most important tests are identified andprioritized with respect to the prioritized security risks and their underlyingrisk models obtained in Step 5. Second, the test plan is updated accordingto the prioritized list of the most important tests. Third, the tests and thetest procedures are executed according to the prioritized list of the most

30

important tests. The test execution may be iterative according to the com-plexity, scope, and attribute of the system under test. The test model andtest environment model may be used as supporting material while executingthe tests.

• Input: Risk model, risk criteria, prioritized security risks with re-spect to risk criteria, test plan, test, test procedure, test model, testenvironment model

• Output: Test result

Step 7: Test Incident ReportingThe purpose of this step is to report security issues requiring further actionidentified as a result of test execution to the relevant stakeholders. Securitytest results are documented, analyzed and validated according to systemsecurity test requirements. If test execution is carried out iteratively, thetest incidents reporting may also be carried out iteratively.

• Input: Test result

• Output: Test result analysis

31

References[1] Information security. http://en.wikipedia.org/wiki/Information_

security, last date accessed 17.09.2011.

[2] Model based testing. http://en.wikipedia.org/wiki/Model-based_testing, last date accessed 17.09.2011.

[3] P. Bourque and R. Dupuis. Guide to the software engineering bodyof knowledge 2004 version. Technical report 19759, IEEE ComputerSociety, 2004.

[4] The Committee on National Security Systems. National InformationAssurance (IA) Glossary, CNSS Instruction No. 4009, 2010.

[5] IEEE Computer Society. IEEE 829 - Standard for Software and SystemTest Documentation, 2008.

[6] International Standards Organization. ISO 27000:2009(E), Informa-tion technology - Security techniques - Information security manage-ment systems - Overview and vocabulary, 2009.

[7] International Standards Organization. ISO 31000:2009(E), Risk man-agement – Principles and guidelines, 2009.

[8] International Standards Organization. ISO 29119 Software and systemengineering - Software Testing-Part 2 : Test process (draft), 2012.

[9] M.S. Lund, B. Solhaug, and K. Stølen. Model-Driven Risk Analysis:The CORAS Approach. Springer, 2011.

[10] The Open Group. The Open Group Architecture Framework Version9.1, 2011.

[11] Testing Standards Working Party. BS 7925-1 Vocabulary of terms insoftware testing. 1998.

[12] F. John Reh. Glossary of business management terms and abbre-viations. http://management.about.com/cs/generalmanagement/g/objective.htm, last date accessed 19.04.2012.

[13] S. Chao William. System Analysis and Design: SBC Software Archi-tecture in Practice. Lambert Academic Publishing, 2009.

32