SKELETON - ETSI€¦ · Web viewThe interaction with security testing should in general follow...

EG 203251 V 0.0.107 (2015-0431)

Methods for Testing & SpecificationsRisk-based security assessment and testing methodologies

<<

ETSI GUIDE

ReferenceDEG/MTS-202793

KeywordsAssurance, security, testing

ETSI

650 Route des LuciolesF-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 CAssociation à but non lucratif enregistrée à laSous-Préfecture de Grasse (06) N° 7803/88

Important notice

Individual copies of the present document can be downloaded from:http://www.etsi.org

The present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).

In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat.

Users of the present document should be aware that the document may be subject to revision or change of status. In-formation on the current status of this and other ETSI documents is available at http://portal.etsi.org/tb/status/status.asp

If you find errors in the present document, please send your comment to one of the following services:http://portal.etsi.org/chaircor/ETSI_support.asp

Copyright Notification

No part may be reproduced except as authorized by written permission.The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute yyyy.All rights reserved.

DECTTM, PLUGTESTSTM, UMTSTM and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.3GPPTM and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and

of the 3GPP Organizational Partners.GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.

ETSI

EG 203251 V 0.0.10 (2015-04)2

http://portal.etsi.org/tb/status/status.asp

http://www.etsi.org/

ContentsContents ...............................................................................................................................................................3

Intellectual Property Rights .................................................................................................................................4

Foreword .............................................................................................................................................................4

1 Scope .........................................................................................................................................................5

2 References .................................................................................................................................................5

2.1 Normative references ...........................................................................................................................................5

2.2 Informative references .........................................................................................................................................5

3 Definitions, symbols and abbreviations ....................................................................................................7

3.1 Definitions ...........................................................................................................................................................7

4. Overview ...................................................................................................................................................9

4.1 Security risk assessment ......................................................................................................................................9

4.2 Security testing ..................................................................................................................................................10

4.3 Risk-based security testing and test-based security risk assessment .................................................................10

5. Combining the security testing and security risk assessment workstreams ............................................12

6 Test-based activities to security risk assessment ....................................................................................15

6.1 Test-based risk identification .............................................................................................................................16

6.2 Test-based risk estimation .................................................................................................................................18

7 Risk-based activities to security testing ..................................................................................................21

7.1 Risk-based security testing basic activities ........................................................................................................21

7.2 Risk-based security test planning ......................................................................................................................22

7.3 Risk-based security test design and implementation .........................................................................................25

7.4 Risk-based test execution, analysis and summary .............................................................................................28

8. Compositional approaches and integration with the system life cycle ...................................................32

Annex <A>: A conceptual model for risk-based security testing ..............................................................35

A.1 Testing ....................................................................................................................................................35

A.2 Security Testing ......................................................................................................................................35

A.3 Risk assessment ......................................................................................................................................36

A.4 Security risk assessment .........................................................................................................................36

ETSI

EG 203251 V 0.0.10 (2015-04)3

Annex B (informative): Bibliography ...........................................................................................................38

History ...............................................................................................................................................................40

Contents ...............................................................................................................................................................3


Foreword .............................................................................................................................................................4

1 Scope .........................................................................................................................................................5

2 References .................................................................................................................................................5




3.1 Definitions ...........................................................................................................................................................7

4. Overview ...................................................................................................................................................9













8. Compositional approaches and integration with the system life cycle ...................................................33


A.1 Testing ....................................................................................................................................................36




ETSI

EG 203251 V 0.0.10 (2015-04)4


History ...............................................................................................................................................................41

Contents ...............................................................................................................................................................3


Foreword .............................................................................................................................................................4

1 Scope .........................................................................................................................................................5

2 References .................................................................................................................................................5




3.1 Definitions ...........................................................................................................................................................7

4. Overview ...................................................................................................................................................9













8. Compositional approaches to security risk assessment and testing ........................................................33


A.1 Testing ....................................................................................................................................................36




ETSI

EG 203251 V 0.0.10 (2015-04)5


History ...............................................................................................................................................................41

Contents ...............................................................................................................................................................3


Foreword .............................................................................................................................................................4

1 Scope .........................................................................................................................................................5

2 References .................................................................................................................................................5




3.1 Definitions ...........................................................................................................................................................7

4. Overview ...................................................................................................................................................9




5. Combining security testing and security risk assessment workstream ...................................................12



6.2 Test-based risk evaluation .................................................................................................................................18








A.1 Testing ....................................................................................................................................................34




ETSI

EG 203251 V 0.0.10 (2015-04)6

Annex C (informative): Bibliography ..........................................................................................................37

History ...............................................................................................................................................................39

Contents ...............................................................................................................................................................3


Foreword .............................................................................................................................................................4

1 Scope .........................................................................................................................................................5

2 References .................................................................................................................................................5




3.1 Definitions ...........................................................................................................................................................7

4. Overview ...................................................................................................................................................9




5. Combining security testing and security risk assessment processes .......................................................12





6.4 Risk-based test analysis and summary ..............................................................................................................22

7 Test-based activites to security risk assessment .....................................................................................25


7.2 Test-based risk evaluation .................................................................................................................................28


Annex <A>: A conceptual model for risk-based security testing ...............................................................32

A.1 Testing ....................................................................................................................................................32




ETSI

EG 203251 V 0.0.10 (2015-04)7

Annex C (informative): Bibliography ...........................................................................................................37

History ...............................................................................................................................................................39

ETSI

EG 203251 V 0.0.10 (2015-04)8

Intellectual Property RightsIPRs essential or potentially essential to the present document may have been declared to ETSI. The information per-taining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http://ipr.etsi.org).

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document.

ForewordThis ETSI Guide (EG) has been produced by ETSI Technical Committee Methods for Testing and Specification (MTS).

ETSI

EG 203251 V 0.0.10 (2015-04)9

1 ScopeThe present guide describes a set of methodologies that combine security risk assessment and security testing activites-activities in a systematic manner. It distinguishes between risk-based activitesactivities to security testing (risk assess-ment to improve the security testing) and test-based activitesactivities to security risk assessment (using testing to im-prove the risk assessment). The methodologies are buildbuilt upon a collection of systematically consistently aligned activities with associated rules, methods and best practices. The activitesactivities are described in such a way that they provide guidance for the relevant actors ,actors, in security testing and security risk assessementassessment processes (i.ei.e. actors in the role of a security tester, security test manager, and/or risk assessor). The activitesactivities and their level of specification are based on standardson standards like ISO 31000 and IEEE 829/29119 so that they apply for a larger number of security testing and risk assessment processes on hand.

2 ReferencesReferences are either specific (identified by date of publication and/or edition number or version number) or non-spe-cific. For specific references,onlyreferences, only the cited version applies. For non-specific references, the latest ver-sion of the referenced document (including any amendments) applies.

Referenced documents which are not found to be publicly available in the expected location might be found at http://docbox.etsi.org/Reference.

NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity.

2.1 Normative referencesThe following referenced documents are necessary for the application of the present document.

Not applicable.

2.2 Informative referencesThe following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area.

[i.1] Amland, Risk-based testing: Risk analysis fundamentals and metrics for software testing including a financial application case study. Journal of Systems and Software 53(3): 287-295 (2000)

[i.2] Brændeland, G; Refsdal, A.; Stølen, K.: Modular analysis and modelling of risk scenarios with dependencies. Journal of Systems and Software 83(10), 1995-2013 (2010)

[i.3] Broy M. and Stølen K.: Specification and Development of Interactive Systems: Focus on Streams, Interfaces and Refinement. Springer (2001)

[i.4] Herzog, P.: OSSTMM 2.1. Open-Source Security Testing Methodology Manual;Institute for Security and Open Methodologies, 2003

[i.5] IEEE: IEEE Standard for Software and System Test Documentation (IEEE 829-2008), ISBN 978-0-7381-5747-4, 2008.

[i.6] IEEE: IEEE 29119 Software and system engineering - Software Testing Part 1: Concepts and definitions, 2012.

ETSI

EG 203251 V 0.0.10 (2015-04)10

http://docbox.etsi.org/Reference

http://docbox.etsi.org/Reference

[i.7] International Standards Organization. ISO 27000:2009(E), Information technology - Security tech-niques - Information security management systems - Overview and vocabulary, 2009.

[i.8] International Standards Organization. ISO 29119 Software and system engineering - Software Testing-Part 2 : Test process (draft), 2012

[i.9] International Standards Organization. ISO 31000:2009(E), Risk management – Principles and guidelines, 2009.

[i.10] ISTQB: ISTQB Glossary of testing terms version 2.2.http://www.istqb.org/downloads/finish/20/101.html, as of date 19.03.2013.

[i.11] Mass Soldal Lund, Bjørnar Solhaug, Ketil Stølen: Model-Driven Risk Analysis, The CORAS Ap-proach, Springer Verlag Berlin Heidelberg 2011, ISBN: 978-3-642-12322-1

[i.12] Masson A., M.-L. Potet, J.Julliand, R.Tissot, G.Debois, B.Legeard, B. Chetali, F. Bouquet, E. Jaf-fuel, L. Van Aertrick, J. Andronick, A. Haddad: An access control model based testing approach for smart card applications: Results of the POSE project, JIAS, Journal of Information Assurance and Security, 5(1), 335–351 (2010)

[i.13] Michael, C. C. & Radosevich, W.: Risk-Based and Functional Security Testing; Cigital, Inc., 2005

[i.14] OMG: UML testing profile version 1.1 (formal/2012-04-01). http://www.omg.org/spec/UTP/1.1, as of date 19.03.2013

[i.15] Deliverable of the RASEN research project, RASEN Deliverable D4.3.1, http://www.rasenpro-ject.eu/deliverables.

[i.16] Redmill, F. 2004. Exploring risk-based testing and its implications: Research Articles. Softw. Test. Verif. Reliab. 14, 1 (Mar. 2004), 3-15.

[i.17] Redmill, F. 2005. Theory and practice of risk-based testing: Research Articles. Softw. Test. Verif. Reliab. 15, 1 (Mar. 2005), 3-20.

[i.18] Souza, E.; Gusmao, C. & Venancio, John Wack, Miles Tracy, M. S.: Guideline on Network Secur-ity Testing -- Recommendations of the National Institute of Standards and Technology; NIST Spe-cial Publication 800-42, 2003

[i.19] Testing Standards Working Party. BS 7925-1 Vocabulary of terms in software testing. 1998.

[i.20] Wing, J. M. A specifier's introduction to formal methods. IEEE Computer 23(9), 8,10-22,24 (1990)

[i.21] Zech, P.: Risk-Based Security Testing in Cloud Computing Environments; PhD Symposium at the Fourth IEEE International Conference on Software Testing, Verification and Validation (ICST), 2011 Trust Management (IFIPTM'2009), pages 215-233, Springer, 2009.

[i.22] Zimmermann, F.; Eschbach, R.; Kloos, J. & Bauer, T.: Risk-based Statistical Testing: A Refine-ment-based Approach to the Reliability Analysis of Safety-Critical Systems EWDC 2009: Pro-ceedings of 12th European Workshop on Dependable Computing, HAL - CCSD, 2009.

[i.23] ETSI/ETSI TS 102 165-1 (2009). Telecommunications and Internet converged Services and Proto-cols for Advanced Networking (TISPAN); Methods and protocols; Part 1: Method and proforma for Threat, Risk, Vulnerability Analysis.

[i.24] Masse, T.; O’Neil, S. & Rollins, J.; The Department of Homeland Security’s Risk Assessment Methodology: Evolution, Issues, and Options for Congress The Department of Homeland Secur-ity’s Risk Assessment Methodology, 2007

ETSI

EG 203251 V 0.0.10 (2015-04)11

[i.25] Howard, M. & Leblanc, D. E.: Writing Secure Code; Microsoft Press, 2002

[i.26] Alberts, Christopher & C., J. and Dorofee, Audrey. A. J., "OCTAVE Threat Profiles. Software Engineering Institute, Carnegie Mellon University, Criteria Version 2.0", Tech. report CMU/SEI-2001. http://www.cert.org/archive/pdf/OCTAVEthreatProfiles.pdf-TR-016. ESC-TR-2001-016, 2001.

[i.27] Jones, Jack A.: An Introduction to Factor Analysis of Information Risk (FAIR); http://www.riskmanagementinsight.com/media/docs/FAIR_introduction.pdf

[i.28] Saitta, P.; Larcom, B. & Eddington, M.: Trike v.1 Methodology Document; 2005

[i.29] James J. Cebula, L. R. Y. A Taxonomy of Operational Cyber Security Risks Carnegie Mellon, Software Engineering Institute, CERT Program, 2010

ETSI

EG 203251 V 0.0.10 (2015-04)12

3 Definitions, symbols and abbreviations

3.1 DefinitionsFor the purposes of the present document, the following terms and definitions apply:

Asset anything that has value to stakeholders, its business operation and its continuity.

Consequence: outcome of an event affecting objectives [i.9].

NOTE: See ISO 31000:2009(E) [i.9].

Event: occurrence or change of a particular set of circumstances [i.9].

NOTE: See ISO 31000:2009(E) [i.9].

Likelihood: chance of something happening [i.9].

NOTE: See ISO 31000:2009(E) [i.9].

Objective: something the stakeholder is aiming towards or a strategic position it is working to attain.

Risk: combination of the consequences of an event with respect to an objective and the associated likelihood of occur-rence (adapted from ISO 31000:2009(E)).

NOTE: See ISO 31000:2009(E)[i.9].

Risk Criterion: term of reference against which the significance of a risk is evaluated [i.9].

NOTE: See ISO 31000:2009(E) [i.9].

Risk Level: magnitude of a risk or combination of risks, expressed in terms of the combination of consequences and their likelihood [i.9].

NOTE: See ISO 31000:2009(E) [i.9].

Risk Source: element which alone or in combination has the intrinsic potential to give rise to risk[risk [i.9].

NOTE: See ISO 31000:2009(E)[i.9].

Security Requirement: specification of the required security for the system (adopted from[from [i.19]).

Security Risk: risk caused by a threat exploiting a vulnerability and thereby violating a security requirement.

Security Risk Assessment: process of risk assesmentassessment specialized towards security.

Stakeholder: person or organization that can affect, be affected by, or perceive themselves to be affected by a decision or activity[activity [i.9].

NOTE: See ISO 31000:2009(E)[i.9].

Test case: set of preconditions, inputs (including actions, where applicable), and expected results, developed to determ-ine whether or not the covered part of the test item has been implemented correctly.

NOTE: See IEEE 29119 [i.6]

Test completion criteria: set of generic and specific conditions, agreed upon with the stakeholders, for permitting a testing process or a testing sub process to be completed.

ETSI

EG 203251 V 0.0.10 (2015-04)13

Test condition: testable aspect of the test item (i.e. a component or system), such as a function, transaction, feature, quality attribute, or structural element identified as a basis for testing.


Test item: work product (e.g. system, software item, requirements document, design specification, user guide) that is an object of testing.


Test coverage item: attribute or combination of attributes to be exercised by a test case that is derived from one or more test conditions by using a test design technique.


Test log: recording which tests cases were run, who ran them, in what order, and whether each test passed or failed.

NOTE: See IEEE 29119 [i.6] and IEEE 829 [i.5]

Test incident: event occurring during testing that requires investigation (ISTQB[ISTQB [i.14]).

NOTE: See IEEE 29119 [i.6] and IEEE 829 [i.5]

Test incident report: detailed description for any unexpected incident or test that failed

NOTE: See IEEE 829[i.5], IEEE 29119 [i.6]

Test plan: detailed description of test objectives to be achieved and the means and schedule for achieving them, organ-ized to coordinate testing activities for some test item or set of test items.


Test procedure: sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap up activities post execution.


Test result: indication of whether or not a specific test case has passed or failed, i.e. if the actual result corresponds to the expected result or if deviations were observed [i.6].

Test (design) technique: compilation of activities, concepts, processes, and patterns used to identify test conditions for a test item, derive corresponding test coverage items, and subsequently derive or select test cases.


Threat: potential cause of an unwanted incident [i.7].

Vulnerability: weakness of an asset or control that can be exploited by a threat [i.7].

ETSI

EG 203251 V 0.0.10 (2015-04)14

4. OverviewThis guide introduces methodologies and their underlying activities that are dedicated to support companies and organ-izations in undertaking security assessments for large scale, networked systems. The methodologies cover security as-sessments on different level of abstraction and from different perspectives. Security risk assessment by itself can be applied with different goals in mind. Legal risk assessment especially addresses security threats in a legal context and under consideration of legal consequences. Security risk assessment specifically deals with the concise assessment of security threats, their estimated probabilities and their estimated consequences for a set of technical or business related assets. Finally, compliance assessment and security testing can be used to actually examine the target under assessment, i.e. an organization or system, for compliance issues or vulnerabilities.

While security testing is considered to especially discover technical issues to security, security risk and compliance as-sessment discover high level issues that especially address legal or business related consequences. However,T the main intention behind this document is, that the systematic integration of activitesactivities that cover aspect from security testing, and security risk assessment provide added value to the overall goal in assessing the security of large scale, net-worked system. In the same manner that the high level perspective that is taken by security risk assessment can provide guidance (i.e. by helping focus on the relevant aspects) to the activities carried out during compliance assessment or the more technical oriented security testing, testing or compliance check can provide factual feedback on the actual quality characteristics of a system and thus allow for improving the overall assessment of the system. Integrating and inter-weaving the activities from both sides, thus a systematic integration and completion of risk assessment results with se-curity testing results allows for a more precise, focused and dynamic assessment of systems, processes and other targets. A risk-based approach to security testing will focus the testing resources, on the areas, which are most likely to cause concern. Such a process involves identifying the areas of high risk within the target’s compliance and quality setting and building and prioritizing the security testing program around these risks. A test-based risk assessment, on the other hand side, is able to verify the assumptions on risk factors with tangible measurements and test results and thus allows to provide a concise feedback whether the properties of the target under assessment are really met.

4.1 Security risk assessmentSecurity risk assessment is typically an interativeiterative process hatthat analyses the potential threats to a system in order to calculate the likelihood of their occurrence. The risk assessment comprises the identification of assets, threats and vulnerabilities as well as the identification and propagation of risk treatments (i.e. security controls and other counter measures). Risk itself is considered a metric that indicates the combination of the consequences of an unwanted incident with respect to an asset and the associated likelyhoodlikelihood or estimated frequency of occurrence.

From a process point of view risk assessment is considered as the overall process of risk identification, risk estimation and risk evaluation.

Risk identification is a set of activities dedicated to finding, recognizing and describing risks. This involves identifying sources of risk, areas of impacts, events (including changes in circumstances), their causes and their potential consequences. Risk identification can involve historical data, theoretical analysis, informed and expert opinions, and stakeholders’ needs. It typically comprises a hazard and threat analysis as well as a vul-nerability analysis.

Risk estimation is the process of comprehending the nature of risk and determining the level of risk. This in-volves developing an understanding of the risk. Risk estimation provides the basis for risk evaluation and de-cisions on whether risks need to be treated, and on the most appropriate risk treatment strategies and methods.

Risk evaluation is the process of comparing the results of risk estimation with risk criteria to determine whether the risk and/or its magnitude is acceptable or tolerable. Risk evaluation assists in the decision about risk treatment.

Currently there are a larger number of security risk assessment methods like ETSI TRVA [i.23], CVSS [i.24], STRIDE/DREAD [i.25], OCTAVE [i.26], FAIR [i.27] and Trike [i.28] that provide dedicated guidance on how to identify the sources of risks, their causes and their potential consequences within different contexts and with different

ETSI

EG 203251 V 0.0.10 (2015-04)15

strategies. Their main purpose is to provide systematic guidance and the definition of a consistent and unambiguous vocabulary for risk identification and handling. In model-based security risk assessment, the security risk assessment is conducted with a language for documenting assessment results and a clearly defined process for conducting the assess-ment. In this regards the CERT provides taxonomy on operational cyber security risks [i.29]. The taxonomy identifies sources of operational cyber security risks and organizes them into four classes. It distinguishes between risks estab-lished by actions of people, by systems and technology failures, by failed internal processes, or by external events. Each class is broken down into further subclasses, which are described by individual elements (e.g. “actions of people” is subdivided into “Inadvertent Actions”, “Deliberate Actions” and “Inaction”). The Factor Analysis of Information Risk (FAIR) [i.27] provides an information security risk taxonomy, which is comprised of two main branches according to the FAIR’s overall risk definition “Risk = Loss Event Occurrence and Probable Loss Magnitude”. The OCTAVE method for example defines the main tasks during risk assessment with threats identification, security measures identi-fications, definition of business impacts, and the definition of security measures costs and their standardized values. A step by step approach eases the estimations on the individual risk factors. It starts with the definition of asset-based threat profiles. In this phase the members of an organization identify important information assets, the threats to those assets and the security requirements of the assets. A second phase targets the identification of infrastructure vulnerabilit-ies. Especially the information technology infrastructure is examined for weaknesses (technology vulnerabilities) that can lead to unauthorized action. The last phase is dedicated to the development of a security strategy. The information generated by the organizational and information infrastructure evaluations are carefully analysed to identify risks to the organization and to the organization’s mission as well as to identify countermeasures.

4.2 Security testingThe term software security testing characterizes activities to check the security properties of software implementations. While a number of approaches have long been around to target specific attacks on systems (e.g. vulnerability scanners) more systematic security testing of systems with respect to specified policies or security properties are a relatively new concern that has started to be addressed in the last few years. In general the software security testing activities can be divided into functional security testing, robustness testing, performance testing and penetration testing. While functional security testing, robustness testing and performance testing is used to check the functionality, availability, and effi-ciency of the specified and carefully planned security functionalities and systems (e.g. firewalls, authentication and au -thorization subsystems, access control), penetration testing or security vulnerability testing directly addresses the identi -fication and discovery of actually undiscovered system vulnerabilities that are introduced by security design flaws. These kind of tests analyse systems for any potential vulnerabilities that may result from poor or improper system con-figuration, known and/or unknown hardware or software flaws, or operational weaknesses in process or technical coun-termeasures. Penetration test objectives are to determine feasibility of an attack and the impact of a successful exploit.

4.3 Risk-based security testing and test-based security risk assessment

The combination of risk assessment and testing first originated from the testing community under the notion risk-based testing (RBT). The main idea of RBT is to use risk assessment in order to improve the testing process. Classical test approaches address risks rather implicitly than systematically. Systems, functions, or modules, which are known to be critical, are tested more intensively than others. The basis of the test planning is often a very simple and unstructured risk assessment, which usually is performed during or in the preparation of the test process. In contrast, there are a num-ber of systematic approaches that refer to the combination of risk assessment and testing in a more formal manner. These approaches can be summarized under the terms risk-oriented or risk-based testing. They characterize a set of methodologies that makes software risks the guiding factor to solve decision problems during testing, e.g. the selection and prioritization of test cases.

Integrating and interweaving the risk assessment and security testing activities allow for a more precise, focused and dynamic security assessment of systems. In principal there are two ways that security testing and security risk assess -ment can be combined. Either the testing process is integrated into risk assessment process or the risk assessment pro-cess is integrated into the testing process.

ETSI

EG 203251 V 0.0.10 (2015-04)16

A risk-based process process to security testing is called risk-based security testing. In such a process risk assessment results are used to guide and focus the testing activities. The notion of risks helps focusing the test-ing resources on the areas that are most likely to cause concern. Results from threat and vulnerability analysis can be used to ease the selection of dedicated test techniques that precisely address already identified risks.

A test-based process to security risk assessment is called test-based risk assessment. Systematic security testing and the respective test results are used to improve the risk assessment results. Security testing may provide feedback on actually existing vulnerabilities that have not been covered during risk assessment. Moreover it allows to adjust risk values on basis of tangible measurements like test results. Security testing could provide a concise feedback whether the properties of the target under assessment have been really met by the risk assessment.

Risk-based security testing and test-based risk assessment can be applied in different phases and to different testing activities in the system life cycle.

[1)] During Design & Implementation risk-based security testing and test-based risk assessment shallshould focuses the integration between security risk-assessment and security functional testing. The main point of reference are security functional requirements and the verification of their implementation by testing. The notion of risk might help to focus the implementation and testing efforts for all development driven testing activitites (e.g. module and unti testing).

[2)] During the verification and validation phase it can (but not necessarily mustwill) be extended to also cover the other security testing activities like performance testing, robustness testing and penetration testing. Risk-based security testing shallshould be used to focus the test design and test implementation efforts, to choose the ap-propriate testing techniques and to communicate the test results in the context of the products security risks.

1)[3)] During the operation and maintenance phase the focus slightly changes. Penetration testing is used to discover new and unknown vulnerabilities. Theses activity can especially help to identify new risks and thus improve the risk assessment as described in test-based risk assessment approaches. Regression testing is usually used to verify whether changed system still meets the original security requirements with respect to functionality, per-formance and robustness. These activities can most probably be optimized by means of risk-based security testing.

ETSI

EG 203251 V 0.0.10 (2015-04)17

Design & Implementation

Verification & Validation

Operation & Maintenance

Secu

rity

risk

asse

ssm

ent

Secu

rity

func

tiona

l tes

ting

Perf

orm

ance

testi

ng

Robu

stne

ss te

sting

Pene

trati

on te

sting

Syst

em L

ifecy

cle P

hase

s

Activities

1

2

3

ETSI

EG 203251 V 0.0.10 (2015-04)18

Figure 1: Risk-based security testing as systematic combination between security risk assessment and security testing.

In the following, this guide does not distinguish between the different kind of security testing nor cares about system life cycle phases. It focuses on a description on process level that is generic and that is applicable to all system life cycle phases as well as to all kinds of security testing.

This guide defines a set of methodologies that combine security risk assessment and security testing. It especially high-lights the activities and integration points where the identification, estimation, and evaluation of risks meets dedicated security-testing activities. These activities can be integrated in any risk assessment process that is compliant with ISO 31000 and with any testing process that is compliant with ISO 29119. The methodologies and activities have been de-veloped and evaluated in the RASEN research project (www.rasenproject.eu).

In Section 5 the combination of security testing and security risk assessment is introduced. Section 6 and 7 precisely specify the aspects of integration and. Sections 5,6, and 7 focuses on a description on process level that is generic and that is applicable to all system life cycle phases as well as to all kinds of security testing. Section 8 shows tha applica-tion of the integration in the different phases of a system life cycle.the following, this guide does not distinguish between the different kind of security testing nor cares about system life cycle phases. It focuses on a description on process level that is generic and that is applicable to all system life cycle phases as well as to all kinds of security test-ing.

ETSI

EG 203251 V 0.0.10 (2015-04)19

5. Combining the security testing and security risk assessment processes workstreams

The overall process of a combined security assessment is derived from ISO-31000 and slightly extended to highlight the identification and evaluation of compliance and quality issues as one of the major tasks that need to be carefully aligned with typical risk assessment activities. It is defined independent from any application domain and independent from the level, target or depth of the security assessment. It could be applied to legal risk and compliance assessments as well as for any kind of technical security assessment and testing processes.

Figure 2 shows the main activities of a combined risk assessment and security testing process. It starts with a preparat-ory phase called “Establishing the ccontext” that includes preparatory activities like “Understanding the Bbusiness and Rregulatory Eenvironment” as well as the “Requirements & Pprocess Iidentification”. During the first phase the high level security objectives are identified and fixed. The latter phase is meant to analyse and document the technical con-text of the target under assessment. Moreover, the figure shows additional support activities like “Communication & cconsult” and “Monitoring and rreview” that are meant to set up the management perspective, thus to continuously control, react, and improve all relevant information and results of the process. From a process point of view these activ-ities are meant to provide the contextual and management related information for the combined security assessment and considered common for security risk assessment workstream as .well as for the test-based risk assessment workstream.

The main part ,part, namely the “SSecurity AssessmentAssessment” part covers the integration between the risk assess-ment workstream and a security testing workstreammain innovation that is described in this guide. It consists of a com-bination of typical security risk assessment activities that are defined in ISO 31000 and typical security testing activities that follow testing standards like ISO 29119.

ETSI

EG 203251 V 0.0.10 (2015-04)20

Security Assessment

Security testingSecurity risk assessment

Establishing the context

Treatment

Requirements & process identification

Understanding the business & regulatory environment

Com

mun

icat

e &

cons

ult

Mon

itorin

g &

revi

ew

1 2

Security Assessment

Security TestingSecurity Risk Assessment

Establishing the Context

Treatment

Requirements & Process Identification

Understanding the Business & Regulatory Environment

Com

mun

icat

e &

Con

sult

Mon

itorin

g &

Rev

iew1 2

Figure 2 The overall combined security assessment process

This guide distinguishes two main perspectives, each represented by a set of activities that are combined to form a workstream carried out during system development or operation.

ETSI

EG 203251 V 0.0.10 (2015-04)21

1) A test-based security risk assessment workstream should start like a typical risk assessment workstream and should use testing results to guide and improve the risk assessment. Security testing is used to provide feed-back on actually existing vulnerabilities that have not been covered during risk assessment or allows to adjust risk values on basis of tangible measurements like test results. Security testing should provide a concise feed-back whether the properties of the target under assessment have been really met by the risk analysis.

2) TheA risk-based process to security testing workstream should start like a typical testing workstreamprocess and uses risk assessment results to guide and focus the testing. Such a process workstream should start with identifying the areas of risk within the target’s business processes and building and prioritizing the testing pro-gram around these risks. In this setting risks help focusing the testing resources on the areas that are most likely to cause concern or supporting the selection of test techniques dedicated to already identified threat scenarios.

[4)] A test-based process to security risk assessment should start like a typical risk assessment process and should use testing results to guide and improve the risk assessment. Security testing is used to provide feedback on actually existing vulnerabilities that have not been covered during risk assessment or allows to adjust risk val-ues on basis of tangible measurements like test results. Security testing should provide a concise feedback whether the properties of the target under assessment have been really met by the risk analysis.

In the following, both workstreamsways are described in more detail by providing a set of activitesactivities that consti-tute the systeatmicsystematic interaction between security risk assessment and security testing. All activitesactivities are documented in a similar manner. That is, each step of the methods are documented using the template shown in Table 1.

Table 1 – Template for documenting process activities

Name The name of the activity

Actors The actors that are referred to in the activity

Tools The tools that are involved in the activity

Precondition The precondition that needs to be enabled when the activity is initiated.

Postcondition The postcondition that describes the result of the activity.

Scenario The scenario that describes the individual actions taken by the actors

Data exchanged/processed

The data that are exchanged during the integration use case

In: The data that go into the activity. Terms from the conceptual model are used to de-scribe the data.

Out: The data that are the outcome of the activity. Terms from the conceptual model are used to describe the data.

The possible actors and tools that can be referred to are described below.

Actors:

Customer (C): The person/organization on whose behalf a security assessment is conducted.

Risk analyst (RA): The person responsible for doing the security risk assessment.

Security test manager (TM): The person responsible for doing the security test management

ETSI

EG 203251 V 0.0.10 (2015-04)22

Security tester (ST): The person responsible for doing the security testing.

Compliance manager (CM): The person responsible for ensuring compliance.

Auditor (A): The person responsible for auditing a system.

Tools:

Security risk assessment tool (SRAT): The tool that supports the security risk assessment.

Security test management tool (STMT): The tool that supports the security test specification.

Security test specification Tool (STST): The tool that supports the security test specification.

Security test derivation tool (STDT): The tool that supports the derivation of test procedures and test cases from the SRAT tool to the STT tool.

Security Test Execution Tool (STET): The tool that supports the derivation of test procedures and test cases from the SRAT tool to the STT tool.

Security test aggregation tool (STAT): The tool that supports the aggregation of test results from the STT tool to the SRAT tool.

ETSI

EG 203251 V 0.0.10 (2015-04)23

6 Test-based activities to security risk assessmentRisk assessment, similar to other development activities that start in the early phases of a development project, are mainly based on a set of assumptions that have been made on the system to be developed. Testing is one of the most relevant means to get quality related information on the actual systems and thus helps to gain empirical arguments on the existence or absence of vulnerabilities, the applicability and consequences of threat scenarios and the quality of countermeasures. Considering this, test-based risk assessment uses test results to gain arguments or evidence for the assumptions that have been made during the initial risk assessment phases. From a testing perspective, this kind of feed-back allows for representing test results in the context of risk analysis artefacts. Beside others, this kind of high level representation of test results can be used to support management issues and control the overall test process during the test management.

ETSI

EG 203251 V 0.0.10 (2015-04)24

Security Assessment

Security Testing

Security Risk Assessment


Risk Identification

Risk Estimation

Risk Evaluation

Treatment

Com

mun

icat

e &

Con

sult

Mon

itorin

g &

Rev

iew

1

2




Figure 2 – Generic workstream for test-based risk assessment

The main purpose of integrating the testing process into the risk assessment process is to use testing to enhance some of the activities of the risk assessment process. This is achieved by ensuring that test results are used as explicit input to the risk assessment. Figure 2 shows how the overall security assessment process (shown in Figure 1) is refined into a process for test-based risk assessment. Here the risk assessment activity has been decomposed into the three activities

ETSI

EG 203251 V 0.0.10 (2015-04)25

Iidentify Rrisk Identificatoins, Restimate risk Estimations and Revaluate risk Evaluations. These three activities, to-gether with the "Eestablishing the Ccontext" and "Ttreatment" activities form the core of the ISO 31000 risk manage-ment process. As indicated in Figure 2, there are in particular two places where testing can in principle enhance the risk assessment process. The first, denoted 1 in the figure, is during risk identification. In a risk assessment process, the risk identification activity is performed with respect to a target of analysis which is described and documented in the "Ees-tablishing the Ccontext" stepphase". In a test-based risk assessment setting however, the risk identification is not only based on the documentation of the target of analysis, but also on relevant test results of target of analysis. Particularly relevant in this setting is testing using automated testing tools such as vulnerability scanners or network discovery tools.

The risk assessment activity that can be enhanced by the testing process (denoted 2 in Figure 2) is risk estimation. The main reason for doing testing here is to gain increased confidence in the correctness of the risk model. In particular, the likelihood estimates of the risk model might have a low confidence if they e.g. depend on vulnerabilities whose pres-ence in the target of analysis is unknown. By doing testing in this setting, we may investigate whether such vulnerabilit-ies really are present in the target of analysis, and then use the test results to update the confidence level of the risk model.

6.1 Test-based risk identificationRisk identification is the process of finding, recognizing and describing risks. This involves identifying sources of risk (e.g. threats and vulnerabilities), areas of impacts (e.g. the assets), events (including changes in circumstances), their causes and their potential consequences. It should confess the analysis of the potential threat or attack surface, the iden-tification of potential threat and vulnerabilities and the derivation of complete threat scenarios, covering the relations between threats, vulnerabilities and unwanted incidents. Risk identification can involve historical data, theoretical ana-lysis, informed and expert opinions, and stakeholder’s needs [i.9]. In this context, testing refers to the process of gener-ating results that identify/indicate actual vulnerabilities or areas of an actual system that are potentially vulnerable. This kind of testing may be performed by e.g. use of network discovering techniques or vulnerabilities scanners.

ETSI

EG 203251 V 0.0.10 (2015-04)26

Threat and Threat Scenario

Identification

Vulnerability Identification

Unwanted Incident (Risk) IdentificationSecurity Testing

Artefacts

Unwanted Incidents (Risks)

a

bTest Incident

Report

Vulnerabilities

Threats and Threat Scenarios

Test Report

Identify Assets

Analyze Attack Surface

Analyze Threats & Potential

Vulnerabilities

Derive Threat ScenariosSecurity Testing

Artefacts

Threat Scenarios

a

bTest Incident

Report

Threat and Vulnerability List

Asset Diagram

Attack Surface DefinitionTest Report

Figure 3 – Test-based risk identification

In Figure 3Error: Reference source not found, it is shown how the risk identification can be structured w.r.t. to the iden-tification of these artefacts. As indicated in the figure, there are in particular two activities that can be integrated with testing:

ETSI

EG 203251 V 0.0.10 (2015-04)27

a) Test-based attack surface analysis

b) Test-based vulnerability identification

[a)]

[b)] In the following, we describe these activities in more detail.

[c)] The process of test-based risk identification uses test results to guide and improve the risk identification process. As illustrated in Figure 10, the process is decomposed in several steps where two of them,

[d)] t est-based attack surface analysis and

[e)] t est-based threat and vulnerability identification,

[f)] show an interaction with results that are generated through testing.

[g)] Test-based attack surface analysis allows for systematic and tool-based discovery of the attack surface of a sys-tem. Particularly relevant in this setting is testing using automated testing tools such as vulnerability scanners or network discovery tools, or results from passing scanning/ monitoring

The purpose of the threat and threat scenario identification this activity is to identify threats and threat scenarios. A threat may be human or non-human, malicious or non-malicious. A hacker is an example of a typical malicious human threat. A threat scenario is a series of events that is initiated by a threat and that may lead to an unwanted incident. A cyber security attack such as SQL injection is a typical example of a threat scenario.

Testing can be used in order to obtain information that can support the identification of threats and threat scenarios. Particularly relevant in this setting are testing techniques that yield information about the interfaces/entry points, the attack-surface, and potential attacks against the target of evaluation. The kinds of testing tools that can be used for this purpose are network discovery tools, web-crawlers, static code analysis tools, and fuzz testing tools.

Table 2 – Test-based risk identification: Test-based threat and threat scenario identificationattack surface analysis (a)

Name Test-based attack surface analysis (a)

Actors Security Risk Analyst (SRA)

Tools Security Risk Assessment Tool (SRAT), Security Testing Tool (STT)

Precondition A test report covering results from automated attack surface analysis. must be available.

A model or other specification documents describing the interfaces of the target of assess-ment.

Postcondition A detailed definition of the attack surface of the target of assessment.

ETSI

EG 203251 V 0.0.10 (2015-04)28

Scenario 1) The SRA shallshould analyse the system model, other specification document, and publicly available information to identify the attack surface of the target of assessment.

2) The SRA shallshould initiate a semi-automatic or automatic scan of the system/network to detect hidden entry points for attacks. The results are documented by means of a scan or test report.

3) Based on this analysis, the SRA shallshould indicate which features or areas of the target of assessment should be prioritized in the risk identification step.


In: Specification documents, test report

Out: Attack surface definition (with prioritization of areas/features)

Note: For a semi-automatic or automatic scan of the target of assessment, the following tool categories shallshould be considered:

Static analysis tools that check the code

V ulnerability scanners or network discovery tools

Crawlers, spiders or other dynamic web site discovery tools

Fuzz-testing tools

Test-based vulnerability identification refers to the use of testing to obtain information that supports the vulnerability identification activity. Testing techniques that yield information about the presence of actual vulnerabilities in the target of evaluation or potential vulnerabilities that may be present in the target of evaluation are relevant in this activity. The kinds of testing tools that can be used for this purpose are penetrating testing tools, model-based security testing tools, static and dynamic code analysis tools, and vulnerability scanners.

Test-based threat and vulnerability identification allows for an update of the already estimated vulnerabilities with vul-nerabilities that have been detected or indicated through testing. Similar to the activity described before, it is recommen-ded to use semi-automatic or automatic vulnerability assessment and testing tools.

Table 3 – Test-based risk identification: Test-based threat and vulnerability identification (b)

Name Test-based threat and vulnerability identification (b)


Tools Security Risk Assessment Tool (SRAT), Security Testing Tool (STT),

Precondition A test incident report covering results from testing or semi-automated vulnerability as-sessmentss must be available.

Postcondition A detailed list of potential threats, potential vulnerabilities and actual vulnerabilities of the target of assessment.

ETSI

EG 203251 V 0.0.10 (2015-04)29

Scenario 1) The SRA shallshould analyse the system model, other specification document, and publicly available information to identify potential threats, potential vulner-abilities and already known vulnerabilities for the target of assessment.

2) The SRA shallshould initiate the active exploration (e.g. penetration testing or semi-automatic or automatic scan) of the actual target of assessment to identify vulnerabilities or indicators for vulnerabilities. The results are documented by means of a test incident report showing the discovered vulnerabilities or indica-tions thereof.

3) Based on this analysis, the SRA shallshould indicate which vulnerabilities (and associated threats) of the target of assessment should be additionally handled in the risk identification step.


In: Specification documents, test report

Out: Threat and vulnerability list

Note: For a semi-automatic or automatic scan of the target of assessment, the following tool categories shallshould be considered:

Static analysis tools that check the code,

Vulnerability scanners or network discovery tools,

Fuzz-testing tools

6.2 Test-based risk evaluationestimationrisks and their causes (i.e. threat scenarios). Accurate risk estimation is essential for a successful outcome of a risk as-sessment. However, risk estimation is one of the hardest activities of a risk assessment since the information basis for the estimation is often imprecise and insufficient, and analysts are often forced to rely on expeort judgment. This might result in a high degree of uncertainty related to the correctness of the estimates.

ETSI

EG 203251 V 0.0.10 (2015-04)30

Figure 4

Consequence Estimation

Estimate ValidationSecurity Testing

Artefacts

Test Report

a

b

Consequences

Likelihood Estimation

Validated Estimates

Likelihoods

Risk evaluation is the process of comparing the results of risk estimation with the risk criteria defined during the context analyisis. The goal is to determine whether the risk and/or its magnitude is acceptable or tolerable.

ETSI

EG 203251 V 0.0.10 (2015-04)31

Risk Grouping & Treatment Identification

Risk Treatment EvaluationSecurity Testing

Artefacts

Test Report

a

bRisk Groups

Risk Prioritization &

Evaluation

Treatment Decision

Risk Evaluation Matrix

Figure 4 – Test-based risk evaluation

In this context, testing shallshould be used to produce an additional technical input that allows for a precise characteriz-ation of some of the properties of a risk model.Testing shall be used to structure the risk evaluation process and to to validate the correctness of the risk model. In particular, the likelihood estimates of the risk model might have a low con-fidence if they, e.g., depend on vulnerabilities whose presence in the target of analysis is unknown. By doing testing in this setting, we may investigate whether such vulnerabilities really are present in the target of analysis, and then use the test results to update the confidence level of the risk model.

As shown in Figure 4, the risk estimation activity can be decomposed into the three sub-activities: Likelihood Estima-tion, Consequence Estimation, and Estimate Validation. The last sub-activity refers to checking and/or gaining confid-ence in the correctness of the risk estimates. As indicated in Figure 4, there are in particular two activities that can be integrated with testing:

a) Test-based likelihood estimation

b)[h)] Test-based estimate validation

Likelihood estimation is the activity of estimating likelihoods for risks and their causes. In a security setting, this in-volves estimating the likelihood that: security attacks will be initiated; attacks will be successful if initiated; successful attacks will lead to identified risks. Likelihoods should be document using the likelihood scales defined in the Estab-lishing the Context step of the overall risk assessment process.

Testing is particularly relevant for obtaining information which can support the estimation of the likelihood that an at-tack will be successful if initiated. This is because security testing is most often used for identifying vulnerabilities, and the presence of these has a direct impact on this likelihood. Thus the testing techniques used for test-based likelihood estimation are similar those used for test-based vulnerability identification (as described in Section 6.1). The main dif-ference between these activities is that in the former, information about the vulnerabilities is only used as a means of supporting likelihood estimation.

ETSI

EG 203251 V 0.0.10 (2015-04)32

As illustrated in Figure 4Figure 11, the test-based risk evaluation process is decomposed into three steps: Risk prioritization & evaluation, risk treatment identification & grouping , and risk treatment evaluation. Two of them can be improved when security testing results are used as an additional

source of information.

During risk prioritization and evaluation, security testing shall be used to validate the correctness and soundness of the overall risk model. Especially when it comes to the evaluation of the likelihood and consequence values for distinct vulnerabilities, testing and the respective results may provide an more precise view on the estimates of this values that have been made during risk estimation.

During risk treatment identification and grouping, test results may be used as an additional input for risk categorisation and grouping (e.g. risks with tested/ not tested vulnerabilities, risks based on vul-

nerabilities that have actually been found during testing etc.) and thus allow a better and more fo-cued treatment decision.

Risk prioritization and evaluation lay ground for the decision whether a risk or a group of risks re-quires treatment as well as for the systematic identification of treatments. The main purpose of this

activity is the comparision of the risk levels identified during risk estimation with the risk criteria that have been identified during the context analysis. Before the actual evaluation of the risks begin, it is recommended to validate the correctness of the risk estimations in the overall risk model. This kind

of validation is typically carried out by experts that review the risk estimations in the risk model. Judgements, whether the risk estimations in the risk model are correct or not, are typically done us-ing plausibility checks and on basis of the expert’s expertise. Systematic testing and the systematic

revision of the risk values on basis of test results, additionally allows to introduce a more fine grained, systematic and concise validation approach. A test-based approach to risk evaluation and

validation is based on technical information that directly comes from a possibly detailed and system-atic analysis. This kind of analysis directly interacts with the target of assessment and thus allows

for a far more concise validation and improvement of the overall risk model than approaches that are based on expert knowledge only.

Table 4 – Test-based risk estimationvaluation: Test-based likelihood estimation (a)Risk validation and adjustement

Name Test-based likelihood estimation Risk evaluation and validation and adjustement(a)

Actors Security Risk Analyst (ST)

Tools Security Risk Assessment Tool (SRAT)

Precondition A risk evaluation matrix, risk model with identified risks and estimations for likelihood and consequences mus.t be available.

Postcondition A revised risk evaluation matrix showing the revised /and/or confirmedestimated likeli-hoodrisk values of the risks identified during risk assessment.

ETSI

EG 203251 V 0.0.10 (2015-04)33

Scenario 1) The SRA shallshould analyse the risk model and shallshould identify elements that can be better understood when tested.

2) The SRA shallshould initiate the testing (if not already initiated by other activ-ities) of these elements and shallshould receive the test report and the test in-cident report.

3) The ST links the items of the test report and test incident report to elements of the risk model and updates/validates the correctness of the estimates the likeli-hood for individual of the risk model elements based on the new information obtained through the testing.


In: Risk evaluation matrix, Risk model, Test Report, Test Incident Report.

Out: Risk model, Risk evaluation matrix.

Validation is the activity of checking or gaining confidence in the correctness of the estimated risk values. In a test-based setting, we recommend that uncertainty related to the correctness of an estimate is explicitly expressed. For in-stance, instead of using single likelihood values such as frequency or probability, we can use intervals of likelihoods to express the belief that the correct likelihood likes somewhere within the interval without knowing precisely where. Un-certainty can then be measured in terms of the breath of the interval - the broader the intervals, the more uncertainty there is.

As for the likelihood estimation activity, testing is particularly useful for obtaining information that support the estima-tion of likelihood of successful attacks. The main difference between test-based likelihood estimation and test-based likelihood validation, is that in the former activity, testing is used to obtain the likelihood in the first place, whereas in the second activity, the purpose is to validate or gain confidence in the correctness of a likelihood value which as already been estimated. If uncertainty is expressed explicitly, the test results may be used lower this uncertainty value. For instance if likelihood intervals are used, the test results may result in a narrowing of the intervals. Recalculating the likelihood values of risks as a result of the updated uncertainty is a good way of showing how the test results have im-pacted the risks.

During Risk Grouping and Treatment Identification, risks are grouped in such a way that treatment identification and treatment decision become easier. Typically, risks are grouped in such a way that they could be treated in a similar manner. A test-based approach to risk categorization and grouping allows to use test results as one of the information sources that dirct the grouping and thus the treat-

ment.

Table 5 – Test-based risk evaluation: Test-based estimate validation Risk-based test procedure iden-tification (ba)

Name Test-based risk categorisation and groupingestimate validation (ba)



Precondition A risk evaluation matrix and risk model with identified risks and estimations for likeli-hood and consequences shouldmust be available.

Postcondition A list of risk categories aor groups that allow for a better evaluation of the risks

ETSI

EG 203251 V 0.0.10 (2015-04)34

Scenario 1) The SRA shallshould analyse the risk model and shallshould identify elements that can be better understood when tested.

2) The SRA shallshould initiate the testing of these elements (if not already initi-ated by other activities) and shallshould receive the test report.

3) The ST links the items of the test report and test incident report to elements of the risk model and updates/revises the estimates for likelihood values of indi-vidual risk model elements based on the information obtained through the test-ing.The SRA shall categorize and group the risks in the risk model unsing the test results as well as other sources of information. Grouping and categorization shall be done in such a way, that the risks that have the chance for a similar treatment are in the same group.


In: Risk evaluation matrix, Risk model, Test Report, Test Incident Report.

Out: Risk model, Risk evaluation matrix.In: Risk evaluation matrix, risk model, test report, test incident

Out: Categorized and grouped risks.

ETSI

EG 203251 V 0.0.10 (2015-04)35

7 6 Risk-based activities to security testingRisk-based activities to security testing help to optimize the overall test process. The result of the risk assessment, i.e. the identified vulnerabilities, threat scenarios and unwanted incidents, are used to guide the test identification and may complement requirements engineering results with systematic information concerning the threats and vulnerabilities of a system. A comprehensive risk assessment additionally introduces the notion of probabilities and consequences related to threat scenarios. These risk values can be additionally used to weight threat scenarios and thus help identifying which threat scenarios are more relevant and thus identifying the ones that need to be treated and tested more carefully. Fur-thermore, risk-based testing approaches can help to optimize the risk assessment itself. Risk assessment, similar to other development activities that start in the early phases of a development project, are mainly based on assumptions on the system to be developed. Testing is one of the most relevant means to do real experiments on real systems and thus be able to gain empirical evidence on the existence or likelihood of vulnerabilities, the applicability and consequences of threat scenarios and the quality of countermeasures. Thus, a test-based risk assessment makes use of risk-based testing results to gain arguments or evidence for the assumptions that have been made during the initial risk assessment phases. Almost all the approaches that combine testing and risk assessment aid the testing by means of one of the following activities:

a) Risk-based resource,effort, test or feature prioritization: This activity supports testing by using risk assess-ment artefacts to prioritize efforts and artefacts during test planning, test design, test implementation, test exe-cution and/or test summary.

b) Risk-based test or test technique identification: This activity supports testing by using risk assessment arte-facts (typically from fault/threat/vulnerability modelling) to identify test purposes, test techniques and test con-dition.

c) Risk based test scenario derivation: This activity supports testing by using risk assessment artefacts (together with a test model) to manually derive or automatically generate test scenarios or test cases.

GFinally, general technical recommendations on security testing techniques [i.4] [i.18] propose the use of risk analysis results to guide security testing. The latter recommendations are very general in nature and describe in this sense no real method for risk-based testing.

7 6.1 Risk-based security testing basic activitiesAlmost all the approaches that combine testing and risk assessment fit best into the category of risk-based testing, i.e. risk assessment is primarily used to aid the testing by means of one of the following activitesactivities:

[i)] Risk-based resource,effort, test oprioritization or feature prioritization: This activity supports testing by using risk assessment artifactsartefacts to prioritize efforts and artifactsartefacts during test planning, test design, test implementation, and/or test execution and/or test summary.

[j)] Risk-based test or test technique identification: This activity supports testing by using risk assessment arti-factsartefacts (typically from through fault/threat/vulnerability modelingmodelling) to identify test purposes, test techniques and test condition.

[k)] Risk based test scenario generation: This activity supports testing by using risk assessment artifactsartefacts (together with a test model) to manually derive or automatically generate test scenarios or test cases.

From a process point of view, the interaction between risk assessment and testing could be best described following the phases of a typical testing process. Figure 5 illustrates the three phases of a testing process that are affected and suppor-ted by risk-based security testing. In the following, this guide describes these phases and the realted activites in more detail.

ETSI

EG 203251 V 0.0.10 (2015-04)36

Figure 5 – Process model for risk-based security testing

In Figure 5, it is illustrated three phases of a testing process that are affected and supported by risk-based security test-ing.In the following, we describe these in more detail.

[1)] Risk-based security test planning deals with the integration of security risk assessment in the test planning process. For that, security risk assessment is used to roughly identify high-risk areas or features of the system under test (SUT) and thus determine and optimize the respective test effort that is needed to verify the related security functionality or to address the related vulnerabilities. Moreover, a first assessment of the identified vulnerabilities and threat scenarios my help to select test strategies and techniques that are dedicated to deal with the most critical security risks.

[2)] Risk-based security test design, implementation deals with the integration of security risk assessment in the test design, implementation and execution process. During the test design and implementation phase, test cases are derived, implemented and assembled to test procedures. Security-risk assessment in general provides two different kinds of information that are useful within this process. On the one hand side it provides detailed in-formation on expected threats and potential vulnerabilities. This information can be used to systematically determine and identify test conditions (testable aspects of a system), test purposes or high-level test scenarios that are dedicated to address the identified threats and vulnerabilities. On the other hand side, the security risk assessment provides quantitative estimations on the risk, i.e. the product of frequencies or probabilities and estimated consequences. This information can be used to select and prioritize either the test conditions or the actual tests when they are assembled to test set.s. Risk-based security test selection criteria can be used to con-trol the selection or the selected generation of test cases. The criteria are designed by taking the risks as well as their probabilities and consequence values to set priorities for the test selection, test case generation as well as for the order of test execution expressed by risk-optimized test procedures.

ETSI

EG 203251 V 0.0.10 (2015-04)37

[3)] Risk-based test execution, analysis and summary deals with a risk-based test execution as well as with the systematicthe systematic analysis and summary of test results. The decision on how extensive testing should be is always a question of the remaining test budget, the remaining time and the probability to discover even more critical errors, vulnerabilities or design flaws. Risk-based test execution allows the prioritization of al-ready existing test cases, test sets or test procedure during regression testing. Risk-based security test analysis and summary aims for improving the evaluation of the test progress by introducing the notion of risk coverage and remaining risks on basis of the intermediate test results as well as on basis of the errors, vulnerabilities or flaws that have been found so far. This process is meant to support the test management process with risk re-lated information that can be used to depict the test results in terms of their relation to the overall security risks.

ETSI

EG 203251 V 0.0.10 (2015-04)38

Security Assessment

Security Testing

Security Risk Assessment


Treatment



Com

mun

icat

e &

Con

sult

Mon

itorin

g &

Rev

iew

2

Test Planning

Test Design & Implementation

Test Execution, Analysis & Summary

1

3

Figure 6 – Process model for risk-based security testing

While security test planning as well as security test execution, analysis and summary are more closely related to the test management process than security test design and implementation, all processes belong to the dynamic test process that is controlled by the test management process.

ETSI

EG 203251 V 0.0.10 (2015-04)39

7 6.2 Risk-based security test planningThe test planning is the activity of developing the test plan. AccorindAccording to ISO 29119 it determines the test ob-jective, the test scope, and the risks associated to the overall testing process. The main outcome of this activities is the test strategy to be used and a plan that depicts the staffing, the required ressoucesresources and a schedule for ththe ein-dividualindividual testing activities.Figure 7activities. Figure shows the integration of security risk assessment results in the overall test planning process. We have identified three integration activities that all serve different purposes:

a) Integrate risk analysis

b) Risk-based test strategy design

c) Risk-based security resource planning and test scheduling

[d)]

[e)]

Organize Test Plan

Development

Risk Analysis & Treatment

Design Test Strategy

Determine Staffing and SchedulesSecurity Risk

Assessment Artefacts

Threat and Vulnerability Assessment a

b

c

Risk Estimation Results

Project Risk Report

Test Strategy

Test Plan

ETSI

EG 203251 V 0.0.10 (2015-04)40


Organize Test Plan

Development

Risk Analysis & Treatment

Design Test Strategy

Determine Staffing and SchedulesSecurity Risk


Threat and Vulnerability Assessment a

b

c

Risk Estimation Results

Project Risk Report

Test Strategy

Test Plan

Figure 7 – Process model for risk-based security test planning

Typically, risk analysis is a substantial part of the test planning process. The risk analysis is done to get an estimate on the specific project risks, considering the availability of test ressourcesresources, considering specific product risks and other project related issues. The security risk assessment typically addresses the security risk of the product (i.e. the test item). As such, this kind of risk assessment can serve the project risk assessment with valuable estimates on the major product risks.

Table 6 – Risk-based security test planning: Integrate risk analysis (a)

Name Integrated risk analysis (a)

Actors Security Test Manager (TM), Security risk analyst (SRA)

Tools Risk Assessment Tool (SRAT), Security Test Management Tool (STMT)

ETSI

EG 203251 V 0.0.10 (2015-04)41

Precondition [a)] Contextual information like legal or regulatory requirements, organizational test and security policies, organizational or higher-level test strategies, and technical limitations as well as ressourceresource limitations are known.

[b)] Security risk assessment results (threat, vulnerability and risk estimations) that capture the technical, business, regulatory and legal requirmentsrequirements are available.

Postcondition A project risk assessment that provides an overall risk picture for the test project, con-sidering project risk that reflect risks that come from the security risk analysis.

Scenario 1) The Test Manager should review the relevant security risks to identify those, which have a special role for security testing.

[2)] The Test Manager should try to identify additional risks like other product risks or project related risks like missing ressourcesresources, technical issues related to the test infrastructure etc.

2)[3)] The Test Manage should develop an overall risk picture for the test project and communicate the risk picture to the Stakeholders.

ArtifactsArtefacts exchanged/processed

In: Vulnerabilities, Tthreat Sscenarios, Uunwanted Iincidents, Llikelihoods, Ccon-sequences, Rrisk Llevel

Out: Project Rrisks

One of the major activities during test planning istis the desingdesign of a test strategy. A test strategy defines the test phases, the types of testing, the test techniques and the test completion criteria. For security testing especially the identification of test techniques is a challenge that should be otimizedoptimized by directly considering the potential threats and vulnerabilities, which have been identified during a security risk identification.

Table 7 – Risk-based security test planning: Risk-based security test strategy design (b)

Name Risk-based security test strategy design (b)

Actors Security Test Manager (TM), Security Risk Analyst (SRA)


Precondition [a)] Contextual information like legal or regulatory requirements, organizational test and security policies, organizational or higher-level test strategies, and technical limitations as well as ressourceresource limitations are known.

[b)] Security risk assessment results (threat, vulnerability and risk estimations) that capture the technical, business, regulatory and legal requirmentsrequirements are available.

d)[c)] Security risks that are relevant for testing have been identified, see integrated risk analysis (a)

Postcondition A test strategy comprising test phases, test types, features to be tested, test techniques and test completion criteria that directly adressaddress the identified threats and vulner-abilities.

ETSI

EG 203251 V 0.0.10 (2015-04)42

Scenario 1) The Test Manager should assigns vulnerabilities and threat scenarios to test items (interfaces, operations, components) and/or test conditions.

2) The Test Manager should try to identify the potential vulnerabilities that have the highest impact on the overall security risks when they are detected.

[3)] The Test Manager should assign test techiquestechniques that are capable to detect the identified vulnerabilities to each test item and/or tor each test con-dition.

[4)] The Test Manager should assign test completeioncompletion criteria to each test item and/or tor each test condition.

[5)] The test Manager should prioritize test item and/or tor each test condition by considering the required test efforts to match the completion criteria and the impact testing may have on the overall security risks (i.ei.e. when vulnerabil-ities are detected or test suites pass without detecting anything)

ArtifactsArtefacts exchanged/processed

In: Vulnerabilities, threat scenarios, unwanted incident, likelihoods, consequences, risk level

Out: List of applicable test techniques, test completion criteria, prioritized list of test items and/or test conditions

When defining a security test strategy, the following sources of information should be considered:

Rules and regulations that apply to the test item or the processes related to the test item

P policies, objectives, and the strategies that are in place at the organisation

P publicly available security best practices (e.g. test pattern libraries and attack pattern libraries)

P publicly available vulnerability scores (e.g. detectability, occurrence and impact scores)

The second major activities during test planning istis the planning of ressourcesresources and the scheduleschedule for the testing activities. Since the main task of security testing is finding vulnerabilities, ressouceresource planning and test schedules should be aligned with the major security risks so that resources and the order of testing allows for a fo-cused testing of the test items or test condition where the detection of vulnerabilities shows the largest impact.

Table 8 – Risk-based security test planning: Risk-based security resource planning and test scheduling (c)

Name Risk-based security resource planning and test scheduling (c)

Actors Security Test Manager (TM)


ETSI

EG 203251 V 0.0.10 (2015-04)43

Precondition [a)] Contextual information like legal or regulatory requirements, organizational test and security policies, organizational or higher-level test strategy, tech-nical and ressourceresource limitation are known,.,

[b)] Security risk assessment results (threat, vulnerability and risk estimations) are available that capture the technical, business, regulatory and legal requirments-requirements.

e)[c)] Test strategy depicting the test items, test conditions, test techniques etc.

Postcondition A test plan that depicts resources, staffing and test schedules respecting certain threats and vulnerabilities and their associated risk scores.

Scenario 1) The Test Manager should check for required security testing competences and should acquire new competences if certain security testing task require these competences. Security risk assessment results may indicate these compet-ences (e.g. when certain potential vulnerabilities or threats need to be ad-dressed).

2) The Test Manager should allocate ressourcesresources considering the re-quired test efforts for that test items or test conditions where testing may have the largest impact in terms of treating or minimizing the identified security risks.

3)[2)] The test Manager should plan the test schedules so that test items or test con-ditions where testing might have the largest impact in terms of treating or minimizing the identified security risks are tested first.

Artefacts ex-changed/processed

In (from SRAT): Vulnerabilities, Tthreat Sscenarios, Uunwanted Iincident, Llikeli-hoods, Cconsequences, Rrisk Llevel

Out (from ST): RessourceResource allocation and test schedules that respect the identified security risks.

In summary, the integration of security testing and security risk assessment is addressed during the test planning phase by three activities, that each contribute with the notion of security risks, threat scenarios and vulnerabilities to the testing activitesactivities.

7 6.3 Risk-based security test design and implementationThe test design and implementation process is mainly dedicated to derive the test cases and test procedures that are

later on appiedapplied to the system under test. To achieve this in a systematic way the overall process should start with a concise definition of the features and test condititioncondition that are the main subjects to test. On basis of that, the relevant test coverage items should be identified, the test cases should be derived and they finally should be assembled to adequate test sets and test procedures. Considering especially security testing, security risks, potential threat scen-arios and potential vulnerabilities provide a good guidance which of the features and test conditions require testing, which coverage items should be covered in which depth and how individual test cases and test procedures should look like.

ETSI

EG 203251 V 0.0.10 (2015-04)44

Identify Feature Sets & Potential Vulnerabilities

Derive Test Conditions

Derive Test Coverage Items

Derive Test Cases

Assemble Test Sets

Derive Test Procedures

Security Risk Assessment Artefacts

Test Procedure Specification

a

b

c

d

Threat and Vulnerability Assessment

Risk Evaluation Results

Test Case Specification

Test Design Specification

Figure 8 – Process model for risk-based security test design

A first step during the test design phase is the identification and categorization of the security features that will be tested. Since security features descripedescribe functional security measures this approach especially allows for testing the correctness of the feature implementation. Security risk assessment can be used to determine the most critical secur-ity features so that these featruesfeatures are tested more intensively and in more detail.

ETSI

EG 203251 V 0.0.10 (2015-04)45

Table 9 – Risk-based security test design: Risk-based identification and prioritization of features sets (a)

Name Risk-based identification and prioritization of features sets (a)

Actors Security Tester (ST), Security Risk Analyst (SRA)

Tools Test Specification Tool (STST), Security Risk AssesmentAssessment Tool (SRAT)

Precondition Security features are documented and the security risk assessment is available

Postcondition Security features to be tested are grouped with respect to potential vulnerabilities and threat scenarios.

Scenario 1) The Security Tester should identify testable security features that need to be covered by security testing. This is done by grouping security features to fea-ture sets that each addresses threat scenarios and/or vulnerabilities that have been identified during security risk assessment.

2) The Security Tester should prioritize the security feature sets using the risk levels that are associated with the threat scenario/vulnerabilities.

[3)] The Security Tester should document the relations between security feature sets and their associated threat scenarios and/or vulnerabilities (maintain tracebilitytraceability).


In: Vulnerabilities, Tthreat Sscenarios, Uunwanted Iincident, Llikelihoods, Ccon-sequences, Rrisk Llevel

Out: Prioritized list of testable security features (security feature sets).

After a set of testable security features have been identified the security tester should derive the test conditions and test coverage items. This could be done on basis of the identified features (see Risk-based identification and prioritiza-tion of features sets (a)) but need to consider that especially security is a non-functional property and that a correct im-plementation of all security features may not ensure a secure system. Thus, additional test conditions and coverage items need to be derived that especially address the detection of currently unknown vulnerabilities (vulnerability and robustness testing). Security risk assessment should be used to provide guidance for the derivation of test conditions and test coverage items for vulnerability and robustness testing.

Table 10 – Risk-based security test design: Risk-based derivation of test conditions and test coverage items (b)

Name Risk-based derivation of test conditions and test coverage items (b)

Actors Security Tester (ST)

Tools Security Tester (ST), Security Risk Analyst (SRA)

Precondition Test Specification Tool (STST), Security Risk AssesmentAssessment Tool (SRAT)

Postcondition Test conditions and test coverage items weighted according to the impact testing may have on the overall associated security risks

ETSI

EG 203251 V 0.0.10 (2015-04)46

Scenario [1)] The security tester should identify test conditions on basis of the security fea-tures, threat scenarios and/or vulnerabilities that have been identified during security risk assessment and/or during a risk-based identification and prior-itization of features sets (a). Please note, testing security features is one ap-proach to security testing that is often not sufficient to cover all major threat scenarios and vulnerabilities. Thus a Security Tester should check whether all relevant threat scenarios already have been covered by risk-based identifica-tion and prioritization of features sets (a) or if there are remaining risks from potential threat scenarios and vulnerabilities exist that need to be coverd-covered by adequate test conditions.

[2)] The Test Designer should identify test coverage items coreespondingcorres-ponding to the test conditions identified in 1). Test coverage items and the respective test depth should be choosenchosen according to the impact testing may have on the overall associated security risks.


In: Security Ffeature Ssets, Vvulnerabilities, Tthreat Sscenarios, Uunwanted Iincident, Llikelihoods, C consequences, Rrisk Llevel, Ttestable Ssets of Ssecurity Ffeatures

Out:TestOut: Test Cconditions and Ttest Ccoverage Iitems weighted according to the impact testing may have on the overall associated security risks.

In the next step, the security tester should derive test cases on basis of test conditions and test coverage items. The se-curity tester determines the pre-conditions for the individual test, he selects adequate input values, the actions to exer-cise the selected test coverage items, and determines the expected results. Since security risk assessment has been used to identify the test conditions and the test coverage items it is already considered through the activitesactivities before. However, threat scenarios and potential vulnerabilities that have been identified during risk assessment might still help by identifying the preconditions, input values, actions and expected results.

Table 11 – Risk-based security test desingdesign: Threat scenario based derivation of test cases (c)

Name Threat scenario based derivation of test cases (c)



Precondition Testable security features, test conditions and test coverage items are known

Postcondition Security test cases that address threat scenarios and potential vulnerabilities

Scenario 1) The Test Designer should identify the preconditions for the tests, the test data, the test actions and the expected results by examining the test conditions, test coverage items, threat scenarios and potential vulnerabilities.

[2)] The Security Tester should document the relations between test cases, security feature sets and threat scenarios and/or vulnerabilities (maintain tracebil-itytraceability).

2)[3)] The Security Tester and a Security Risk Analyst should review the test case specification and their coverage of threat and potential vulnerabilities identi-fied by the security risk assessment.

ETSI

EG 203251 V 0.0.10 (2015-04)47


In: Test Cconditions, Ttest Ccoverage Iitems, Vvulnerabilities, Tthreat Sscenarios, Uunwanted Iincidents, Llikelihoods, Cconsequences, Rrisk Llevel, Ttestable Ssets of Ssecurity Ffeatures

Out: Security Ttest Ccases.

Finally, the test cases should be assembled to test sets and test procedures. While test sets group test cases with common constraints on test environment or test items, test procedures defines the order of test execution and thus have to respect the pre- and postconditionspost conditions. Security risk assessment should be used to prioritize the order test cases and thus the order of testing with respect to the associated risks.

Table 12 – Risk-based security test desingdesign: Risk-based assembly of test procedures (d)

Name Risk-based assembly of test procedures (d)



Precondition Test cases are available and associated with treat scenarios and potential vulnerab-ilitesvulnerabilities

Postcondition Test procedures that are order with respect to their relevance

Scenario 1) The Test Designer should assemble test sets and test procedures in such a way, that the most relevant tests are executed first. The most relevant test cases are the test cases that address the most critical risks.

2) The Test Designer should assemble test sets and test procedures in such a way that so that the post and precondition of the individual test cases match.the most relevant tests are executed first.


In: Test Ccases, Vvulnerabilities, Tthreat Sscenarios, Uunwanted Iincident, Llikeli-hoods, Cconsequences, Rrisk Llevel, Ttestable Ssets of Ssecurity Ffeatures

Out: Security Ttest Pprocedures.

7 6.4 Risk-based test execution, analysis and summaryThe decision how extensive testing should be is always a question of the remaining test budget, the remaining time and the probability to discover even more critical errors, vulnerabilities or design flaws. Risk-based security test analysist-est analysis and summary aims for improving the evaluation of the test progress by introducing the notion of risk cover-age and remaining risks on basis of the intermediatethe intermediate test results as well as on basis of the errors, vul-nerabilities orvulnerabilities or flaws that have been found so far. This process is meant to support the test management process with risk related information that can be used to depict the test results in terms of their relation to the overall security risks. We have identified two integration activities namely:

[a)] Risk-based test configuration and execution priritization

f)[b)] Risk-based test log analysis

g)[c)] Risk-based test summary creation

ETSI

EG 203251 V 0.0.10 (2015-04)48

Execute

Analyze

SummarizeSecurity risk

assessment artifacts

Threat and vulnerability

analysis

a

b

Risk evaluation results

Test incident report

Test summary

Test log

c

Test Execution

Test Analysis

Test SummarySecurity Risk


Threat and Vulnerability

Analysis

a

b

Risk Evaluation Results

Test Incident Report

Test Summary

Test Log

c

Figure 9 – Process model for risk-based test execution, analysis and summary

The execution of test cases can be done several times for the same test cases and test procedures. Normally the execu-tion order for test cases and test procdures is determined at test design by the assembly of test procedures. However, there are a number of regression test scenarios where reprioritization becomes necessary. In this case a risk-based ap-proach for test executions prioritization may help to cover the most relevant remaining security risks.

Table 13 – Risk-based test execution, analysis and summary: Risked-based test execution prioritization (a)

Name Risked-based test execution prioritization (a)

Actors Security Tester (ST)Security Tester (ST)

Tools Test Specification Tool (STST), Security Risk Assessment Tool (SRAT)

Precondition Test cases and/or test procedures are available and associated with threat scenarios and potential vulnerabilities

The test environment is configured and ready to run the testsTest cases and test pro-cedures are defined

ETSI

EG 203251 V 0.0.10 (2015-04)49

Postcondition Test execution that respects the criticality of addressed threats, vulnerabilities and/or features..

Scenario 1) The ST should prioritize test cases and test procedures in such a way that the most relevant tests are executed first. The most relevant test cases are the test cases that address the most critical risks.

2) The ST should run the test cases and/or test procedures.


In: Test cases, test procedures, risk level

Out: Test logIn: Test logs, security risk assessment artifacts (vulnerabilities, threat scenarios, unwanted incident, likelihoods, consequences, risk level)

Out: Incident report

The test analysis process is used for the evaluation of the test results and the reporting of test incidents. This process will be entered after the test execution and it mainly covers the analysis and evaluation of test failures and issues where something unusual or unexpected occurred during test execution. Its main purpose is to categorize the issues that oc-curred during testing and put them into context so that they can be rated by the test manager

Table 14 – Risk-based test executonexecution, analysis and summary: Risked-based test result analysis (b)

Name Risked-based test result analysis (a)



Precondition Test cases have been executed

Postcondition New and/or updated incident are reported and assigned to either already detected vul-nerabilities or to new vulnerabilities. Incidents that probably constitute new actual vul-nerabilities are communicated so that they could be considered in the security risk as-sessment and/or the development.

Scenario [1)] The Security Tester should analyzeanalyse the test results (e.g., the test logs) and identify new incidents.

[2)] The Security Tester should classify newly identified incidents by means of their relation to artifactsartefacts from the security risk assessment (e.g., risks, threat scenarios, vulnerabilities).

[3)] The Security Tester should prioritize the newly identified incidents by means of associated artifactsartefacts from the security risk assessment. Issues re-lated to critical risks should be rated higher than the ones that are associated with minor risks.

3)[4)] New and/or updated incidents are communicated to the relevant stakeholders

ETSI

EG 203251 V 0.0.10 (2015-04)50


In: Test Llogs, Ssecurity Rrisk Aassessment artifactsAartefacts (Vvulnerabilities, Tthreat Sscenarios, Uunwanted Iincident, Llikelihoods, Cconsequences, Rrisk Llevel)

Out: Incident Rreport

Finally, the overall test results, i.e. the test verdicts, the issues and their categorization are summarized in a way, that the stakeholder could understand the outcome of the tests.

Table 15 – Risk-based test execution, analysis and summary: Risked-based test summary creation (b)

Name Risked-based test summary creation (b)



Precondition Test cases have been executed

Test cases already have a traceable relation to security risk assessment artifactsarte-facts.

Postcondition The test results are summarized respecting their relation to the a-priori identified secur-ity risks. The test report contains coverage of security risks

Scenario [1)] The Security Tester should analyzeanalyse the test logs and separate security risks that have been tested successfully (all tests are passed) and those that have not been tested successfully (issues have been found).

4)[2)] The Security Tester should (re-) characterize the security risks by interpret-ing the test results. To do so, the security tester should make use of dedicate test metrics to determine the quality of test procedures and thus the signific-ance and validity of the test results.


In: Test Llogs, Ssecurity Rrisk Aassessment artifactsAartefacts V(vulnerabilities, Tthreat Sscenarios, Uunwanted Iincident, Llikelihoods, Cconsequences, Rrisk Llevel)

Out: Test Ssummary

ETSI

EG 203251 V 0.0.10 (2015-04)51

7 Test-based activites to security risk assessment

ETSI

EG 203251 V 0.0.10 (2015-04)52

In addition to risk-based security testing approaches testing results can additionally be used to affect or optimize the results of the risk assessment itself. Risk assessment, similar to other development activities that start in the early phases of a development project, are mainly based on a set of assumptions that have been made on the system to be developed. Testing is one of the most relevant means to get quality related information on the actual systems and thus helps to gain empirical arguments on the existence or absence of vulnerabilities, the applicability and consequences of threat scenarios and the quality of countermeasures. Considering this, test-based risk assessment uses test results to gain arguments or evidence for the assumptions that have been made during the initial risk assessment phases. From a testing perspective, this kind of feed back allows for representing test results in the context of risk analysis artifacts. Beside others, this kind of high level representation of test results can be used to support management issues and control the overall test process during the test management.

ETSI

EG 203251 V 0.0.10 (2015-04)53

The main purpose of integrating the testing process into the risk assessment process is to use testing to enhance some of the activities of the risk assessment process. This is achieved by ensuring that test results are used as explicit input to the risk assessment. Figure 2 shows how the overall security assessment process (shown in Figure 1) is refined into a process for test-based risk assessment. Here the risk assessment activity has been decomposed into the three activities identify risks, estimate risks and evaluate risks. These three activities, together with the "establishing the context" and "treatment" activities form the core of the ISO 31000 risk management process. As indicated in Figure 2, there are in particular two places where testing can in principle enhance the risk assessment process. The first, denoted 1 in the figure, is during risk identification. In a risk assessment process, the risk identification activity is performed with respect to a target of analysis which is described and documented in the "establish context step". In a test-based risk assessment setting however, the risk identification is not only based on the documentation of the target of analysis, but also on relevant test results of target of analysis. Particularly relevant in this setting is testing using automated testing tools such as vulnerability scanners or network discovery tools.

ETSI

EG 203251 V 0.0.10 (2015-04)54

The risk assessment activity that can be enhanced by the testing process (denoted 2 in Figure 2) is risk evaluation. At this point in the process, risks have already been identified and estimated, and the main reason for doing testing here is to gain increased confidence in the correctness of the risk model. In particular, the likelihood estimates of the risk model might have a low confidence if they e.g. depend on vulnerabilities whose presence in the target of analysis is unknown. By doing testing in this setting, we may investigate whether such vulnerabilities really are present in the target of analysis, and then use the test results to update the confidence level of the risk model.

Risk & compliance assessment

Security testing

Risk assessment


Identify risks

Estimate risks

Evaluate risks

Treatment

Com

mun

icat

e &

con

sult

Mon

itorin

g &

revi

ew

1.

2.

ETSI

EG 203251 V 0.0.10 (2015-04)55

Figure 2 – Generic process for test-based risk assessment

Risk identification is the process of finding, recognizing and describing risks. This involves identifying sources of risk, areas of impacts, events (including changes in circumstances), their causes and their potential consequences. Risk identification can involve historical data, theoretical analysis, informed and expert opinions, and stakeholder’s needs [i.9]. In this context, testing process refers to the process of using testing for identifying/discovering threat test scenarios or areas or vulnerabilities where the risk assessment should be focused. This may be performed by e.g. use of network discovering techniques or vulnerabilities scanners.

7.1 Test-based risk identification

Risk identification is the process of finding, recognizing and describing risks. This involves identifying sources of risk, areas of impacts, events (including changes in circumstances), their causes and their potential consequences. Risk identification may involve historical data, theoretical analysis, informed and expert opinions, and stakeholder’s needs.

ETSI

EG 203251 V 0.0.10 (2015-04)56

Area/feature prioritization

Prioritized risk identificationSecurity risk

assessment artifacts

a

b


System model

Risk model

Figure 10 – Test-based risk identification

The process of test-based risk identification uses test results to guide the risk identification process. As illustrated in Figure 10, the process is decomposed into two steps. In the first step, test results are used to prioritize areas or features of the target of evaluation for the purpose of risk identification. In the second step, the risk identification is performed on the basis of this prioritization. Particularly relevant in this setting is testing using automated testing tools such as vulnerability scanners or network discovery tools, or results from passing scanning/ monitoring.

Table 16 – Test-based risk identification: Area/feature prioritization (a)

ETSI

EG 203251 V 0.0.10 (2015-04)57

Name Area/feature prioritization (a)


Tools Security Risk Assessment Tool (SRAT), Security Testing Tool (STT),

Precondition A test incident report must be available.

A System model may be available.

Postcondition The step must end with a system model that gives a "risk identification"-priority the system features.

Scenario The SRA analyses the system model and the test incident report. Based on this analysis, the SRA indicateswhich features or areas of the system model which should be prioritized in the risk identification step.

ETSI

EG 203251 V 0.0.10 (2015-04)58


In: System model, test incident report

Out: System model (with prioritization of areas/features)

ETSI

EG 203251 V 0.0.10 (2015-04)59

Table 17 – Test-based risk identification: Prioritized risk identification (b)

ETSI

EG 203251 V 0.0.10 (2015-04)60

Name Prioritized risk identification (b)


Tools Security Risk Assessment Tool (SRAT), Test Specification Tool (STST),

Precondition Same as the postcondition for step "area/feature prioritization"

Postcondition The step must end with a risk model documenting the results of the risk identification.

Scenario The SRA identifies risks on the basis of the system model. The identification process is prioritized according the priorities of the features/areas of the system as specified in the system model.

ETSI

EG 203251 V 0.0.10 (2015-04)61


In: System model (with prioritization of areas/features)

Out: Risk model

ETSI

EG 203251 V 0.0.10 (2015-04)62

7.2 Test-based risk evaluation

Risk evaluation is the process of comparing the results of risk estimation with risk criteria to determine whether the risk and/or its magnitude is acceptable or tolerable. In this context, testing process refers to the process of using testing to validate the correctness of the risk model. In particular, the likelihood estimates of the risk model might have a low confidence if they, e.g., depend on vulnerabilities whose presence in the target of analysis is unknown. By doing testing in this setting, we may investigate whether such vulnerabilities really are present in the target of analysis, and then use the test results to update the confidence level of the risk model.

Risk-based test procedure identification

Risk evaluation and

validationSecurity risk assessment artifacts

a

b


Risk evaluation matrix

Figure 11 – Test-based risk evaluation

ETSI

EG 203251 V 0.0.10 (2015-04)63

As illustrated in Figure 11, the test-based risk evaluation process is decomposed into two steps. In the first step, the risk model is used as a basis for identifying high-level test procedures which can be used as a starting point for designing, implementing, and executing tests. In the second step, the test results are used to validate the correctness of the risk model and the resulting risks are evaluated.

Table 18 – Test-based risk evaluation - Risk-based test procedure identification (a)

Name Risk-based test procedure identification (a)



Precondition A risk evaluation matrix and risk model with identified risks and estimations for likelihood and consequences must be available.

Postcondition The step must result in a list of prioritized test procedures

ETSI

EG 203251 V 0.0.10 (2015-04)64

Scenario The ST analyzes the risk model and identifies elements that can be tested given the scope of the risk assessment.

The identified testable elements are prioritized based on the risk values and estimates of the risk model.

Based on the prioritization, a subset of the testable elements are selected and translated into high-level test procedures that have the purpose of checking these elements through testing.


In: Risk evaluation matrix, Risk model

Out:.Test procedures

ETSI

EG 203251 V 0.0.10 (2015-04)65

Table 19 – Test-based risk evaluation - Risk evaluation and validation

ETSI

EG 203251 V 0.0.10 (2015-04)66

Name Risk evaluation and validation (b)



Precondition A risk evaluation matrix, risk model with identified risks and estimations for likelihood and consequences, and test results must be available.

Postcondition The step must result in a risk evaluation matrix showing the risk values of the risks identified in the risk assessment.

Scenario The ST links test results to elements of the risk model and updates/validates the correctness the estimates of the risk model based on the new information obtained through the testing.

ETSI

EG 203251 V 0.0.10 (2015-04)67


In: Risk evaluation matrix, Risk model, Test incident report

Out:Risk model, Risk evaluation matrix

ETSI

EG 203251 V 0.0.10 (2015-04)68

8. Compositional approaches to security risk assessment and testing

8. Compositional approaches to security risk assessment and testingand integration with the system life cycle

By compositional assessment we mean a process for assessing separate parts of a system or several systems independ-ently. ,Such an approach should provide with means for combining separate assessment results into an overall result for the whole system[system [i.2]. The opositeopposite of composition is decomposition, which is a well-known feature from system specification and design[design [i.20]. Decomposition is the process of partitioning a system specification into separate modules that can be developed and analyzedanalysed independently, thus breaking the development prob-lem into more manageable pieces. Each module may moreover be developed at different sites, by independent teams, or within different companies[companies [i.3].

In the literature, composition is mainly addressed at the level of techniques, mostly with respect to languages, and we are not aware of any methodology for risk assessment that explicitly addressed the use of composition. However, such a methodology should address how and why composition should be used. In the following, we identify areas in the pro-cess for test-based risk assessment where composition or decomposition may be of relevance.

A compositional process to security assessment should follow the same steps as the (non-compositional) security as-sessment process. The main difference is, that instead of assessing the system the system as a whole only, is decom-posed it is partially or fullyis decomposed into components or partsparts and that these, components which couldcan be assessed individually. This has several advantages. It allows to consider specific contextual and technical details, that-details that become only visible when a system is broken down into several functional parts. Moreover, it supports pro-cesses with large integration efforts where multiple software or component supplier deliver individual parts of a system. For each of these components therthere can be a separate risk assessment that will be integrated to form the overall sys-tem's view.be assesseds (more or less)idependently and as if each part were a whole system in itselfin more detail.

Figure 9Figure 15 illustrates the application of decomposition and composition in a typical software development life cycle,




Provision of risk assessment results

Refinement

Update


Provision of test results


Component Security Test

System Security Test

System Security Regression Test

Penetration Test

Component Security Regression Test

System Security Risk Analysis

Component Security Risk

Analysis

Figure 12 – Overview of a risk assessment process with composition/decomposition

case wherewhere the target of analysis has beenis assessessed as a whole at the beginning and in parts or components, when the target is decomposed decomposed into three several parts or components which are each assessed using the

ETSI

EG 203251 V 0.0.10 (2015-04)69

same process for security assessment that was described in Section 4. Risk assessment, security testing and the integra-tion thereof follow in principal the same decomposition/composition strategy than the target of assessment itself.

By taking the system’s perspectiverisk assessment workstream that is described in Section 7. After having established the context, the SRA should go through the risk identification, risk estimation and risk evaluation phases. The risk as-sessment targets risks for the whole system, thus system wide assets and incidents are considered. The interaction with security testing should in general follow the rules defined in Section 7.2 and 7.3. If there is no established security test-ing process at that time, there might be a dedicated exploratory testing phase, that is driven by the risk assessment and meant to provide dedicated testing feedback to the risk assessment. However, at an early time in a system development process, there is often neither an established security testing process nor an existing system. In this case, the feedback from the security testing shall be postponed until there is a functional system. and

After having completed the risk assessment for the overall system, the system is typically decomposed into parts. In principal, the decomposition is driven by the development process and respects modularization requirements that come from the system’s architecture or that are determined by supplier relations. Risk assessment shall follow the same de-composition approach that have been choosen for the system and shall take the component’s perspective. Thus, each of the components that have been defined during the systems’s development should be assessed by its own. This kind of analysis allows a much deeper assessment of the technical risks that origin from the individual components. The risk assessment at

By taking the system’s perspective and

However, at certain points in the process, it may make sense to compose or decompose the assessment results before continuing with the rest of the process. In Figure 15, four such points are identified:

After the first four steps of the assessment are completed, the resulting risk models may be composed into a single risk model which will be used as basis for test identification, selection, and prioritization. One of the reasons why this may be desirable is that this allows for a global prioritization of tests. If the test identification and prioritization is done for each risk model separately, then potential tests will not be compared across the risk models. For instance, it may be the

ETSI

EG 203251 V 0.0.10 (2015-04)70

case that one risk model results in the identification high-priority tests, while another results in low-priority tests, then low-priority tests might be selected for testing even though there are tests with higher priority in other risk models.

After step 5.b of the process, test procedures (identified on the bases if the risk model(s)) may be composed or decom-posed. One reason for this is that it might not make sense to decompose the test model in the same way as the target of analysis was decomposed prior to the risk assessment, or if the number of different test teams is different from the num-ber of risk assessment teams.

After step 6 of the process, when tests have been executed, the test results might be composed into a single model/docu-ment. Otherwise it may be difficult to validate the correctness of the risk model on the basis of the test results in the case where all test procedures were identified on the basis of a single risk model in the first place.

After step 7 of the process, when the risk model(s) have been validated and treatments have been suggested, the result-ing risk model may be composed into a global risk model to get an overview of all the risks that have been assessed.

In summary, we have identified four kinds of artifactsartefacts that may be composed or decomposed during the pro-cess: the target of analysis, the risk model, the test procedures, and the test results.

1: Establish context and target of

analysis

2: Risk identification

3: Risk estimation

4: Risk evaluation

5.a: Test identification

5.b: Test selection / proritization

6. Test design, implementation,

and execution

7. Risk validation and treatment


analysis


3: Risk estimation

4: Risk evaluation




and execution



analysis


3: Risk estimation




and execution


Decomposition of target of analysis

Decomposition / composition of risk model

Decomposition / composition of risk model

4: Risk evaluation

Decomposition / composition of test results

Decomposition / composition of test procedures

The overall process should start by taking the system’s perspective. The SRA should start the system security risk as-sessment by following the risk assessment workstream that is described in Section 7. After having established the con-text, the SRA should go through the risk identification, risk estimation and risk evaluation phases. The risk assessment targets risks for the whole system, thus system wide assets and incidents are considered. The interaction with security

ETSI

EG 203251 V 0.0.10 (2015-04)71

testing should in general follow the rules defined in Section 6.2 and 6.3. If there is no established security testing pro-cess at that time, there might be a dedicated exploratory testing phase, thatphase, which is driven by the risk assessment and only meant to provide dedicated testing feedback to the risk assessment. However, at an early time in a system de-velopment process, there is often neither an established security testing process nor an existing system. In this case, the feedback from the security testing should be postponed until there is a functional system.





Refinement

Update




Component Security Test

System Security Test

System Security Regression Test

Penetration Test

Component Security Regression Test

System Security Risk Analysis

Component Security Risk

Analysis

Exploratory Security TestingSystem

perspective

Component perspective

System Lifecycle Phases

Pers

pecti

ves

Continuous Risk Management


After having completed the risk assessment for the overall system, the system is typically decomposed into parts. In principal, the decomposition is driven by the development process and respects modularization requirements that come from the system’s architecturesystem’s architecture or that are determined by integrator/supplier relationships. Com-ponent Security Risk assessment should use the same decomposition approach than development. In fact, each of the major components that have been defined during system development should be assessed on its own. However, cluster-ing of components is allowed and might help to focus efforts on on the major architectural items. In contrast to system risk assessment, the focus of the assessment should move towards an assessmemntassessment of the technical properties of each of the component. Thus, vulnerability assessment and the assessment of the technical impacts should get much more attention than threat and asset identification. Threat and asset identification is usually done on system level and should be deliberately reused when the component perspective is taken.

ETSI

EG 203251 V 0.0.10 (2015-04)72


The two processes, the system security risk assessment and the component security risk assessment, should belong to-gether. The relation between bothbetween both should be seen as an iterative refinement and update process. Security risk assessment procideprovide the overall context. It identifies the high level assets (e.g. often determindeddetermined by the business context otof the system) and defines the overall threats, threat scenarios, vulnerabilities and unwanted incidents. The component security risk analysis allows for a deeper understanding of the technical causes and impacts focussing on vulnerabilities and unwanted incidents. Since component risk analysis is carried out at a later point in time, there is much more system related information available (e.g. interface defintionsdefinitions, details of realization). This information can be used to allow for a better localization and specification of vulnerabilities, unwanted incident and their impact on and propagation to other parts and components of the system.

Similar to system security risk assessment, the interaction betweeninteraction between component security risk assess-ment and security testing should in general follow the rules defined in Section 6.2 and 6.3. In contrast to security risk assessment it can be considered that there is already an established security testing process at that time of the process, so that a dedicated exploratory testing phase, with the only purpose to provide dedicated testing feedback to the risk as-sessment, should not be carried out. Instead, the component security risk assessment should be carried out, having already the component security testing phase in mind. Thus, assessment results and reports should be structured in such a way that they serve as input for the security testing workstream that is defined in the Sections 7.1. to 7.4. Finally, component security risk assessment results should be used to update the system security risk assessment with respect to estimates on probabilities, identified vulnearbilitesvulnerabilities and technical impact.

Figure 15 – Overview of a compositional test based risk assessment process

Security testing should start, when security risk assessment already has gone through its first iteration. Thus, first risk assessemntassessment results are available for the system's perspective as well as for the component perspective. Secur-ity test planning should be done according to Section 7.2 and cover both perspectives, i.e. the security system testing as well as security component testing phase. Security component testing should be used to test for vulnerabilities and the correctness of security features on component level. System security testing should be used to test the integrated system and cover integration & configuration related vulnerabilities and ensure (as far as testing alone can ensure )ensure) the functional correctness of the high level security features. The interaction with security risk assessment should in general

ETSI

EG 203251 V 0.0.10 (2015-04)73

follow the rules defined in Section 7.3 and 7.4. While system security testing should especially interact with system security risk assessment, security component testing should interact with component security risk assessment.

Please note, especially when it comes toon component level testing, static testing activities like source code ananlys-isanalysis should be used in addition to the dynamic testing. Static testing activates have a quite good discovery rate for a larger number of known vulnerabilities.

The overall process that is described above should be considered as ana highly iterative process. System security risk assessment should be used to focus the component security risk assessment activitesactivates as well as the system se-curity testing activities. In return both workstreams, the component security risk assessment workstream as well as the system security testing workstream, provide updates for the system security risk assessment. Similar, component secur-ity risk assessment should be used to directly improve the component security testing activitesactivates. In return, the results from component security testing should be used to update the compenentcomponent security risk assessment and thus, transitively, the system security risk assessment.

Independent of the system life cycle phase the testing is carried out (i.e. the verification & validation phase or in the operation & maintenance phase) test planning, test design and test summary and execution should keep its relation to security risk assessment as described above and in the Sections 7.1 to 7.4. An overall risk management workstream should ensure, that the risk assessment on the different level and the integration of the testing activities are kept up to date and in sync.

ETSI

EG 203251 V 0.0.10 (2015-04)74

Annex <A>:: A conceptual model for risk-based security testing

A.1 TestingStandards like IEEE 829[i.5], ISO/IEC/IEEE 29119[i.6], the ISTQB Glossary of testing terms[terms [i.10], and the UML Testing Profile (UTP) [i.14] define the basic activities and related artifactsartefacts of a testing process. The ma-jor activities can be characterized as follows:

Test planning (results: a test plan containing test conditions, test techniques, test coverage items and test com-pletion criteria)

Test design & implementation (results: test cases and test procedures)

Test execution (results: test logs and test results)

Test evaluation & incident reporting (result: test incidents reports and test incidents)

Since this guide focuses on the relationship between risk assessment and testing, the following model especially reflects the terms and concepts that are relevant to describe the interfaces between security testing and security risk assessment. In this sense the model concentrates on activities like test planning and test specification as well as the management, evaluation and interpretation of the test results. The following model is mainly based on terms and concepts taken from ISO 29119.

ETSI

EG 203251 V 0.0.10 (2015-04)75

Test Item Test Procedure Test Log Test Incident Report

Test Condition Test Case Test Result Test Incident

1 1

1

1

*

* * * 1 *

**

*

*

1 ***

Test Report Test Summary

Figure 16 – Basic testing concepts

Test item – is a work product (e.g. system, software item, requirements document, design specification, user guide) that is an object of testing.

Test condition – is a testable aspect of the test item, such as a function, transaction, feature, quality attribute, or struc-tural element identified as a basis for testing[testing [i.6].

Test case – is a set of preconditions, inputs (including actions, where applicable), and expected results, developed to determine whether or not the covered part of the test item has been implemented correctly[correctly [i.6].

Test procedure – is a sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap up activities post execution[execution [i.6].

Test plan – is a detailed description of test objectives to be achieved and the means and schedule for achieving them, organized to coordinate testing activities for some test item or set of test items [i.6]

Test coverage item – is an attribute or combination of attributes to be exercised by a test case that is derived from one or more test conditions by using a test design technique[technique [i.6].

Test completion criteria – are a set of generic and specific conditions, agreed upon with the stakeholders, for permit-ting a testing process or a testing sub process to be completed.

ETSI

EG 203251 V 0.0.10 (2015-04)76

Test (design) technique – is a compilation of activities, concepts, processes, and patterns used to identify test condi-tions for a test item, derive corresponding test coverage items, and subsequently derive or select test cases[cases [i.6].

Test log – is a recording which tests cases were run, who ran them, in what order, and whether each test passed or failed (IEEE 829 [i.5], ISO/IEC/IEEE 29119[i.6]).

Test result – is an indication of whether or not a specific test case has passed or failed, i.e. if the actual result corres-ponds to the expected result or if deviations were observed[observed [i.6]. Relevant testing standards [i.14] refer to test results with the values none, pass, inconclusive, fail and error.

Test incident – is an event occurring during testing that requires investigation (ISTQB[ISTQB [i.14]).

Test incident report – is a detailed description for any test that failed. It contains the actual versus expected result and other information intended to throw light on why a test has failed. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact of an incident upon testing (IEEE 829[i.5], ISO/IEC/IEEE 29119[i.6]).

Figure 17 – Test pattern

Test pattern – is a collection of best practices/solutions for a known testing problem. It assembles reusable parts of a test plan, e.g. the test design techniques and corresponding test completion criteria, a test coverage item description, applicable test and coverage metrics, estimation on the necessary testing efforts and estimation of test effectiveness with respect to the given problem. Additionally it may contain also test data and specification and assumptions on the test environment as well as testing tool requirements.

A.2 Security TestingSecurity testing is used to experimentally check software implementations with respect to their security properties and their resistance to attacks. For security testing we can distinguish functional security testing and security vulnerability testing. Functional security testing checks if the software security functions are implemented correctly and consistent with the security functional requirements. It is used to check the functionality, efficiency and availability of the spe-

ETSI

EG 203251 V 0.0.10 (2015-04)77

cified security features of a test item. Security vulnerability testing directly addresses the identification and discovery of yet undiscovered system vulnerabilities. This kind of security testing targets the identification of design and implement-ation faults that lead to vulnerabilities that may harm the availability, confidentiality and integrity of the test item.

Test PatternTest Technique

Security Test Pattern

Security Test Technique

Security Test Procedure

Security Test Case

*

Test Procedure Test Case

***

Security Functional Test

Case

Penetration Test Case

Robustness Test Case

Perfromance Test Case

Test Technique

Security Test Technique

Security Test Procedure

Security Test Case

*

Test Procedure Test Case

**

Security Functional Test

Case

Penetration Test Case

Robustness Test Case

Performance Test Case

Figure 18 – Security testing

Security test case – is a set of preconditions, inputs (including actions, where applicable), and expected results de-veloped to determine whether the security features of a test item have been implemented correctly or to determine whether or not the covered part of the test item has vulnerabilities that may harm the availability, confidentiality and integrity of the test item.

Security test procedure – is a sequence of security test cases in execution order together with any associated action that may be required to set up the initial preconditions and any wrap up activities post execution.

Security test (design) technique – is a collection of activities, concepts, processes, and patterns used to identify test conditions for a test item, derive corresponding test coverage items, and subsequently derive or select security test cases.

ETSI

EG 203251 V 0.0.10 (2015-04)78

Security test pattern – is a collection of best practices/solutions for a known security testing problem. It assembles reusable parts of a test plan e.g. the security test design techniques and corresponding test completion criteria, a test coverage item description, applicable test and coverage metrics, estimation on the necessary testing efforts and estima-tion of test effectiveness with respect to the given problem. Additionally it may contain also test data and specification and assumptions on the test environment as well as testing tool requirements.

A.3 Risk assessmentThe conceptual model and notions defined here are based on the ISO 31000 standard[standard [i.9]

Objective Stakeholder

LikelihoodRisk Event

Consequence Risk source

Risk Criterion

Risk Level

11..*

111*

1

1 initi

ates

11

**

Figure 19 – Conceptual model for risk assessment

The terms of the model are defined in the following.

Risk – the combination of the consequences of an event with respect to an objective and the associated likelihood of occurrence (adapted from [i.9]).

Objective – something the stakeholder is aiming towards or a strategic position it is working to attain (adapted from ).

Risk Source – an element which alone or in combination has the intrinsic potential to give rise to risk [i.9].

Stakeholder – a person or organization that can affect, be affected by, or perceive themselves to be affected by a de-cision or activity [i.9].

Event – the occurrence or change of a particular set of circumstances [i.9].

Likelihood – the chance of something happening [i.9].

Consequence – the outcome of an event affecting objectives [i.9].

Risk Criterion – the term of reference against which the significance of a risk is evaluated [i.9].

Risk Level – the magnitude of a risk or combination of risks, expressed in terms of the combination of consequences and their likelihood [i.9].

A.4 Security risk assessmentLund et al. [i.10]classify] classify risk analysis approaches into two main categories:

Offensive approaches: Risk analysis concerned with balancing potential gain against risk of investment loss. This kind of risk analysis is more relevant within finance and political strategy making.

Defensive approaches: Risk analysis concerned with protecting what is already there.

ETSI

EG 203251 V 0.0.10 (2015-04)79

In the context of security, the defensive approach is the one that is relevant.

ObjectiveAsset

Security requirement Threat

Risk source

Vulnerability

*1..*Risk

Security risk

Event

Unwanted incident

Figure 20 – Conceptual model for security risk assessment

The main terms related to security risk assessment and their relationship to previously defined terms in the risk assess-ment domain are illustrated in [i.7].....

Security Risk Assessment – The process of risk asset specialized towards security.

Asset – Anything that has value to the stakeholders (adopted from[from [i.7]).

Security Requirement – A specification of the required security for the system (adopted from[from [i.19]).

Security Risk – A risk caused by a threat exploiting a vulnerability and thereby violating a security require-ment.

Unwanted Incident – An event representing a security risk.

Threat – Potential cause of an unwanted incident[incident [i.7].

Vulnerability – A weakness of an asset or control that can be exploited by a threat[threat [i.7].

ETSI

EG 203251 V 0.0.10 (2015-04)80

Annex BC (informative):: Bibliography

[1] Australian Standard AS 3806-2006 Compliance programs.

[2] Basel Committee on Banking Supervision. Compliance and the compliance function in banks. Bank for International Settlements (2005).

[3] Chatterjee A., and Milam D. Gaining Competitive Advantage from Compliance and Risk Management. In: Pantaleo, Pal D., and Nirmal (eds). From Strategy to Execution. Springer Berlin Heidelberg 2008, ISBN 978-3-540-71879-6.

[4] Ghirana, A., and Bresfelean, V.: Compliance Requirements for Dealing with Risks and Gov-ernance. Procedia Economics and Finance 3, pp. 752 – 756 (2012)

[5] Giblin, C., Liu, A.Y., Müller, S., Pfitzmann, B., and Zhou, X.: Regulations Expressed As Logical Models (REALM). In: Moens, M.-F., and Spyns, P. (Eds.), Legal Knowledge and Information Systems, pp. 37–48. IOS Press (2005)

[6] ETSI: TTCN-3

[7] IEEE: IEEE Standard Glossary of Software Engineering Terminology (IEEE 610.12-1990), ISBN 1-55937-067-7, 1990.

[8] Information Commissioner’s Office.:Privacy by Design. (ICO, November 2008), 2008.

[9] Mahler, T.: Tool-supported Legal Risk Management: A Roadmap. European Journal of Legal Studies 2(3), (2010). The Future of... Law & Technology in the Information Society.

[10] Mahler, T.: Legal Risk Management: Developing and Evaluating Elements of a Method for

[11] OCEG: GRC Capability Model. “Red Book” 2.0. http://www.oceg.org (2009)

[12] Peterson E.A.: Compliance and ethics programs: Competitive advantage through law. Springer 2012, DOI 10.1007/s10997-012-9212-y.

[13] Ponemon Institute.:The Role of Governance, Risk Management & Compliance in Organiza-tions,Study of GRC practitioners. (Ponemon Institute PLC, 2011)

[14] Racz, N., Weippl, E., Seufert, A.: A Frame of Reference for Research of Integrated Governance, Risk and Compliance (GRC). In: De Decker, B., Schaumuller-Bichl, I. (eds.) CMS 2010, LNCS 6109, pp. 106–117. Springer (2010).

[15] Racz N., et al.: Governance, Risk & Compliance (GRC) Status Quo and Software Use: Results from a Survey among Large Enterprises. 21st Australasian Conference on Information Systems, Brisbane Dec 2010.

[16] Vondrák, I.: Business Process Modeling. In: M. Duží et al. (eds.) Information Modeling and Knowledge Bases XVIII, pp. 223-235. IOS Press (2007).

[17] Werf J.M., Verbeek, E., and Aalst, W.M.P.: Context-Aware Compliance Checking. In: Barros, A., Gal, A. and Kindler, E. (eds.): BPM 2012, LNCS 7481, pp. 98–113, 2012. Springer-Verlag Berlin Heidelberg (2012)

[18] Zoellick, B., and Frank, T., Governance, Risk Management, and Compliance: An Operational Ap-proach. A Compliance Consortium Whitepaper, Public draft version 1 (2005).

ETSI

EG 203251 V 0.0.10 (2015-04)81

[19] F. John Reh. www.about.com. http://management.about.com/cs/generalmanagement/g/object-ive.htm, last date accessed 28.08.2013.

[20] Terblanché J. R.: Legal risk and compliance for banks operating in a common law legal system. The Journal of Operational Risk 7(2), 2012.

[21] Vicente P., and Silva, M.M.D.: A Business Viewpoint for Integrated IT Governance, Risk and Compliance. (IEEE World Congress on Services), ISBN: 978-07695-4461-8/11, 2011.

ETSI

EG 203251 V 0.0.10 (2015-04)82

HistoryThis clause shall be the last one in the document and list the main phases (all additional information will be removed at the publication stage).

Document history

<Version> <Date> <Milestone>

V0.0.1 2013/11/15 First version with initial content from RASEN/DIAMONDS

Jürgen Großmann mailto:[email protected]


V0.0.2 2013/12/15 First alignment with V&V (concept & terms) and SecTestTerms


J ürgen Großmann mailto:[email protected]

V0.0.3 2014/02/27 Risk-based security testing

Test-based risk assessment


Jürgen Großmann mail to:[email protected]

V0.0.4 2014/04/01 Initial draft for for sections on:

Risk-based security test planingplanning

Risk-based security test design

Risk-based security test selection



V0.0.5 2014/09/30 Restructured Draft for discussion by MTS Security SIG, Integrated comments from Milan Zoric


Jürge n Großmann mailto:[email protected]

ETSI

EG 203251 V 0.0.10 (2015-04)83

mailto:[email protected]









V.0.0.6 2014/12/31 Restructured Draft for discussion by MTS Security SIG, Integrated comments from Dieter Hogrefe, alignment with



V.0.0.7 2015/01/06 Added life cycle figure in Section 4.3



V.0.0.8 2015/03/105 Restructuring the document. Revised TBRA section. Revised Section 8.


V0.0.9 2015/03/10 Removed the term shall.


V0.0.10 2015/04/14 Revised TBRA section.


ETSI

EG 203251 V 0.0.10 (2015-04)84








SKELETON - ETSI€¦ · Web viewThe interaction with security testing should in general follow...

Documents

Transcript of SKELETON - ETSI€¦ · Web viewThe interaction with security testing should in general follow...